|
1.IntroductionThe major sources of noise corrupted in most digital images are from the process of image acquisition, quantization, and transmission. As one common example, when an image is transmitted through some wireless mobile networks, it may be ruined with noisy signals from atmospheric disturbances, such as thunder and lightning in the environment.1 Thus, an image may be degraded such that it will not have the same original quality. The consequence of degraded images can create potential problems for further image processing and analysis. For example, a clustering algorithm for segmenting an image usually measures the relationships in the pixel space by categorizing the pixels into different classes.2 Therefore, the existence of noise pixels will create different attributes for pixels that originally belonged to the same cluster. This may generate different clustering outcomes. Our goal is to remove noise in color images and restore the image, which, as a result, will be as similar as possible or identical to the original image. In general, algorithms for removing noise in images can be divided into two methods: one for grayscale images and one for color images. Many state-of-the-art noise removal algorithms have been proposed in the past decades for the grayscale image.3–7 As digital imaging technology has advanced, the necessity for color image processing has been in great demand. Although grayscale image processing is still needed in specific fields, such as medical imaging on x-ray computed tomography,8 the wide usage of color and multichannel images is very common in many areas of imaging applications. In this study, we consider color images our primary focus in developing the denoising filter. Prior to building the algorithm for denoising filters, one of the most important prerequisites is to take on the noise models for color images. Image noise occurs in a wide variety of forms.1,5 It is very common for noise to contaminate the pixels of images taken by the sensor. During the process of digitization and transmission, noise can be introduced into each pixel wherein one or more bit-errors will be embedded in the pixels of images. The Gaussian noise model and impulse noise model are two widely used models for characterizing the noise information for denoising filters in digital image processing. The Gaussian model is convenient for the simulation of noise to design and test the efficiency of a denoising algorithm. If the noise causes pixel data loss or saturation, the impulse noise model, also called salt-and-pepper noise, will be suitable for the noise characterization. Gaussian noise is used as a model of dark noise and often as a crude model of shot noise, which is governed by the Poisson distribution. The main sources of Gaussian noise in digital images arise during acquisition, for example, sensor noise caused by factors such as poor illumination, high temperature, and transmission noise. There are also different noises caused by devices, such as dark noise, which is due to the thermal fluctuations of stationary charge carriers. Dark noise frequently occurs in image sensors, such as charge-coupled devices. Noise degrades image quality and causes difficulty in further processing. Hence, an image denoising algorithm requires an image noise model for improving the signal-to-noise ratio in digital images. In this work, we assume that the impulse noise model is the major type of noise to be detected and removed. Impulse noise randomly and sparsely corrupts pixels to two intensity levels, relative high or relative low, compared with its neighboring pixels. There exist several different formulations for the impulse noise model, and each formulation has its own characteristic in generating how the density level of noise will be assigned and how three color channels will be corrupted with a certain level of probability.9–11 Let us assume that the density level of noise probability is and value is the color channel index for red, blue, and green. For convenience, we assume that the red, green, and blue channels are assigned 1, 2, and 3, respectively, with an 8-bit pixel image for the following discussion. If is assigned 0.2, then 20% of the entire pixels of a given image is noise. The first impulse noise model, which is uncorrelated impulsive noise,10 is formulated as follows: where stands for the random vector that represents the impulse noise corruption with or (with equal probability) and represents the original color vectors. The subscript SAP represents the salt-and-pepper noise. Those two vectors result in the final outcomes of the corrupted image with impulse noise restricted to 0 or 255. This model is called a “salt-and-pepper” impulse noise type. This model is limited to only one channel corruption at a time for all three channels. Hence, we use lower case symbols , , and to represent this model.The second impulse noise model is based on the correlation of impulse noise distribution among the color channels. This model can be expressed as follows: where represents a pixel vector and and are similarly defined as in Eq. (1). , , and are the respective probabilities of each channel corruption (these parameters were set up as in Ref. 10). For example, if we assume that the noise level is 0.1 (i.e., 10% noise density), the value is 0.1. Each individual channel will be given 0.25 probabilities under the overall 10% probability. Consequently, the combination of single channel corruptions from , , and and all channel corruption will add up to 0.1 probability in total. The problem with this model is that it limits the noise corruption either on a single channel only or three channels simultaneously but does not permit two channels for the noise corruption simultaneously. This may not be realistic in practical applications.The noise model that we used in our research is not limited to one channel for noise corruption. The noise signal used is also not limited to two relatively high and low values (i.e., values close to 0 or 255). Our model assumes random-valued noise corruption on the multiple channels simultaneously with a randomized noise level from 0 to 255. We assume that all channels are corrupted with the probability . The impulse noise model we used is shown in Eq. (3), where the subscript rv in stands for the random value. This model allows any combination of channels (can be 1, 2, or 3) to be corrupted with impulse noises. This is a more general impulse noise model compared with noise models 1 and 2. As a result, the comparison in terms of visual perception of noise corruption level is more severe than noise model 2. Please note that noise model 1 as shown in Fig. 1(b) looks more noisy than our model in Fig. 1(d) due to the difference in the three-channel noise (i.e., color) and salt-and-pepper. The latter makes dark and white noises more prominent compared with the color. There are many state-of-the-art developments and publications in the literature for impulse noise removal in color images. Similar to grayscale methods, many of them have their own advantages and limitations in terms of performance issues, such as edge-preserving capability and efficiency on noise removal. One of the important functions that has been used in the grayscale methods for the median filter (MF) is called “switching based”-MFs.4–6 This type of MFs selectively suppresses noise pixels, leaving those uncorrupted pixels intact. It can preserve the edge details and avoid blurring the image. This mechanism is also used in some versions of the vector median filter (VMF) for color images.10,12,13 Our proposed idea on Moran’s (MI) (statistic) and four one-dimensional (1-D) Laplacian kernels is designed to improve the performance on the capability of noise detection, which will then be used for noise removal function. Our proposed filter is also a type of filter with a switching mechanism. MI is one of the oldest indicators of spatial autocorrelation frequently used in social sciences, such as geography and sociology. Based on our research in the literature, two groups of researchers have proposed the MI statistic in their work. Chen et al.14 used the MI statistic for measuring the image quality of reconstructed images. Chuang and Huang15 assume that the byte of a pixel in an image can be divided into two parts, signal and noise; the noise occupies the lower bits of a byte. The MI and join-count statistics then examine the noise bits to filter them out. In our study, the MI statistic is used as a measure in a defined window for detecting noise pixels. The motivation for this study is to measure the strength of spatial autocorrelation of pixels in the local neighborhood of the image. MI indicates the different spatial structures of the smooth and rough surfaces, which provide an index for impulse noise determination and preserve original noise-free pixels. In Sec. 3, we will introduce MI and its application on impulse noise detection in detail.The rest of this paper is organized as follows. The related work in Sec. 2 will briefly review research in impulse noise removal of color images and present our initial thought and motivation for improving contemporary filtering methods for color images. A comparison of the classical MF and VMF is given in this section based on our experimental results. In Sec. 3, the proposed algorithm with spatial autocorrelation associated with MI and Laplacian kernels is provided. Section 4 describes the simulation results, and Sec. 5 gives the concluding remarks and future work. 2.Related Work2.1.Classical Median Filter and Vector Median FilterThe classical MF is an efficient filtering algorithm for removing impulse noise in gray-level, color, and multichannel images. It is one of the most common nonlinear type filters. Due to its distinctive property in the impulse response of the MF, this filter has been widely used in suppressing impulsive noise.16 Many authors have analyzed the statistical properties of the filter, and some modifications and generalizations of the MF have been introduced.16 This MF is a simple and efficient algorithm that replaces the center pixel with the median value of the corresponding neighborhood window, such as a size of . A general MF is formulated as where is the filtered image, is the location of a pixel, is the neighborhood window in the image, and the and values are determined by the window size and the floor function (i.e., ) for integer truncation.Even though this filtering algorithm was originally developed for the grayscale image, it can intuitively be used in color images with a component-wise extension. The so-called marginal standard median filter (MSMF)8,17,18 deals with each red, green, and blue channel separately, by calculating the median value of each channel within a neighborhood window. The restored image is then constructed by combining all RGB channels together, as shown in Fig. 2. Consequently, the filtered pixel will have the median value for each channel as a component in the pixel vector. This filter considers the median value in each separated channel without considering the correlation that may exist between color channels. As a result, the restored image may have color distortion (artifacts).8,18 Astola et al.16 proposed a VMF for impulse noise removal for color images in 1990. The VMF is based on ordering the vectors within the neighborhood window by calculating the cumulative pairwise distance and taking the vector corresponding to the lowest-ranked distance as the vector median value for the output. The VMF is formulated as in Eq. (5) where the norm can be implemented as either (i.e., Euclidean distance) or other different norms where () represents a pixel vector in the window and is the pixel vector with the lowest-ranked distance. So far, we reviewed two basic filtering algorithms used for impulse noise removal. Most of the state-of-the-art filters evolved from these two basic filters. Our filter is also motivated by these two basic filters. In Sec. 2.2, we give a brief comparison of these two basic filters from the perspective of both the performance and time complexity of the filters. Please note that we will denote the MSMF as MF since it is just an extension of the traditional MF for color images.2.2.Brief Comparison Between the Median Filter and Vector Median FilterPrior to moving into constructing new algorithms for impulse noise filtering, we briefly compared the MF and VMF for their similarities and differences on the filtering performance. Many publications have pointed out the advantage of the VMF for multichannel data due to the correlation that exists between the multichannel data. However, in some color images, such an advantage may not present. For example, if we take a neighborhood window in a color image with all eight pixel values [255, 255, 0] and one noise pixel [0, 255, 255], both MF and VMF will have identical denoising results. This might be the reason that the MF shows a very competitive result compared with those of VMF in the following experiments based on the peak signal-to-noise ratio (PSNR) and structural similarity index metric (SSIM) measures. Multiple sample images were tested to compare the denoising quality and time complexity. To assess the denoising quality, we used the PSNR19 and SSIM20 as the quantitative measure. The PSNR is calculated below where and and are the width and height of an image, for an individual pixel of the noise-free image, and is the corresponding pixel of the filtered image. The higher value indicates a better denoising result. The SSIM is designed to measure image quality considering human subjectivity in the context of luminance, contrast, and structural similarity.20 The comparison between original image and filtered image can be calculated using the following equation: where and are the average intensities and , are the standard deviations corresponding to images and , respectively, and is the covariance. The values and are constants determined by the dynamic range value.To conduct a comparison study for MF and VMF, Lena, pepper, and baboon images (Fig. 3) were tested with 5%, 10%, and 15% noise levels. The size of the filtering neighborhood window is and each image is . In terms of image complexity, we calculated the entropy for each image using the below equation, which is based on the average () of three color channels where is the sum of entropy for each gray-level probability, , in the histogram of the average of three color channels, , and is the maximum gray level for an image. (Here, is the total number of gray levels.) In general, as an image complexity increases, its entropy value also increases.Our empirical study on the MF and VMF revealed some interesting outcomes. As shown in Tables 1Table 2–3, the overall results from VMF are not significantly different from those of MF. This is shown in Table 3 for the baboon image, in particular, which has the highest entropy among the three images tested. In terms of time complexity, VMF is more than 10 times slower than MF in all test cases. However, VMF is generally considered to be more appropriate for color image processing because it considers inherent correlation between the red, green, and blue color channels of the image and prevents image color distortion.8,10,17,18 Even with some advantages over MF, the essential problem of slow filtering speed is the major weakness of VMF although some high-speed versions have been proposed.21 If we consider the VMF for the real-time application, its time complexity needs be improved. From this perspective, our proposed filter design will not only possess the advantages of the VMF but also improve its filtering process speed. Table 1A comparison of MF and VMF for Lena image with entropy value 15.12.
Note: s denotes seconds. Table 2A comparison of MF and VMF for peppers image with entropy value 15.30.
Note: s denotes seconds. Table 3A comparison of MF and VMF for baboon image with entropy value 15.90.
Note: s denotes seconds. 2.3.Brief Review on Various Vector Median FiltersSince VMF was introduced in the early 1990s, many variations of VMF have been proposed, and some of them use different distance measures.8,9,18,22 The basic vector directional filter (BVDF) utilizes the angle between two pixel vectors within the window.8 The summation of angular distances used in this filter is calculated below where denotes the angle between pixel vectors and using the arc cosine function from the following equation: The problem with the basic vector median approach is that it can alter some pixels even if there are possibilities that those pixels are actually noise free. Consequently, the excessive filtering causes blurring effects on the denoised image. To solve this problem, the center weighted vector median filter (CWVMF)8,9 was proposed to use a weight value for the center pixel in the window. This algorithm, however, lacks an impulse noise detection scheme. Consequently, the adaptive center weighted vector median filter (ACWVMF)18,23 was proposed. The main difference between the ACWVMF and other filters introduced so far is that it has noise detection capability, instead of performing noise removal on every pixel. This ACWVMF filter provides a switching mechanism that differentiates between impulse noise and noise-free pixels. In the ACWVMF, the noise detection step, which serves as a switching mechanism, is based on the concept of aggregated distances assigned to the pixels in the filtering window.18 The difference between the accumulated distances assigned to the central pixel and to the pixel with the lowest rank serves as an indicator of the noise. If the indicator is greater than (or equal) a fixed threshold, the output of the ACWVMF is a weighted mean of the central pixel of the filtering window and the vector median of its samples. Otherwise, the pixel is noise-free. In our proposed algorithm, MI statistic has a better indication than the mechanism used in the ACWVMF for noise or noise-free pixel. This statement will be verified later in the experimental results, as shown in Table 21.Another approach is based on the “peer group” concept.21,24,25 This type of filter excludes corrupted pixels in the window to calculate the median vector value. The peer group, in short, is the group of pixels in the filtering window that minimizes the total sum of distances from the center pixel of the group to adjacent pixels. Therefore, the peer group vector median filter (PGVMF) is based on the trimmed sum of distances. If the number of pixels in the peer group (denoted by ) is equal to 9 in the window size of , it will be identical to the basic VMF algorithm (Fig. 4). The replaced pixel vector is determined by the pixels located within a peer group of pixels, which shows the minimum distance or minimum dispersion. Compared with various filters introduced so far, the robust switching vector median filter (RSVMF)12 has a solid noise detection algorithm that enables a filtering process; if a pixel is determined as a noisy pixel, the RSVMF will use the VMF to remove the noise pixel. Otherwise, the pixel value remains unchanged. The RSVMF works in a similar way as VMF except for the following modification: where is the total number of pixels in a window, is the cumulative Euclidean distances from the center pixel to pixel in the window, and is a constant value. The med function is to select median value from the cumulative distances of using the way we calculate the cumulative distance in the VMF.Another denoising method is to process vector-based filtering through the trimming scheme.26 The basic idea of trimming schemes is to select a group of pixels in the filtering window that may exclude noise pixels through the ranking order. In a sense, this filter is very similar to the PGVMF. However, the RSVMF has some limitation on removing the noise pixels if the noise level is increased to a certain percentage, such as 30% noise that is used in our experiments. All of the algorithms reviewed so far have their own unique characteristics and limitations for filtering impulse noise through vector-based processing. There are several different variations of algorithms reviewed above to improve the result in such a way that the filter can preserve edge details and minimize blurring effects.13 A comprehensive survey on impulse noise removal filters was given in Ref. 10. Our objective is to design an efficient algorithm for detection and removal of the random-valued impulse noise in color images based on the fundamentals of MI statistics and Laplacian kernels. Our preliminary study on multispectral images shows that this filter is effective in removing impulse noise.27 In summary, the vector-type MFs listed in Table 4 were tested and compared in this study. Table 4Different filters being compared in this study.
3.Proposed Algorithm: Moran’s I Vector Median FilterOur proposed algorithm for noise detection and removal in color images consists of three major components: MI for spatial autocorrelation, four 1-D Laplacian kernels, and the VMF. The first two components are used for noise detection in the algorithm, and the VMF is used for denoising. 3.1.Moran’s for Spatial AutocorrelationThe concept of spatial autocorrelation can be traced back to the first law of geography—“Everything is related to everything else, but near things are more related than distant things.” By definition, spatial autocorrelation is clarified as follows: “Spatial autocorrelation refers to the fact that the value of a variable at one point in space is related to the value of that same variable in a nearby location.”28 In fact, the spatial information concept has long been used in pattern recognition and image analysis with different types of formats. To determine the spatial autocorrelation, Moran introduced MI to check the degree of spatial autocorrelation in areal data.29 MI is calculated by the following equation, and its range is between and : Here, we assume that there are regions, and is a measure of the spatial proximity between regions (with value ) and (with value ). The parameter is the mean value of regions. The interpretation of the value is categorized into two specific spatial patterns. If the value closes to , it indicates a strong homogeneous spatial autocorrelation where local similarity or homogenous spatial patterns exist. A value nearing indicates the strong heterogeneous spatial autocorrelation, but it is a negative pattern in which heterogeneous values are grouped together (e.g., a mixture of very high and very low values). This negative spatial autocorrelation is unusual in nature. For example, high-income households are not usually located in urban twilight zones. If the value is zero, there is an absence of any strong spatial autocorrelation or no distinct spatial pattern, such as checkerboard.28 The calculation of MI can be exemplified with the areal data in Fig. 5(a), which is similar to the example in Ref. 27.To find out the weight value of , it is necessary to check the connectivity between regions and (i.e., immediate neighbors). For example, region A has connectivity with B and C. Region D has connectivity with B, C, E, and F, as shown in Fig. 5(a). Hence, a binary connectivity weight matrix [shown in Fig. 5(b)] can be constructed corresponding to the data in Fig. 5(a). From the denominator of Eq. (13), represents the variance of the regional value. As a result, is replaced by 6 [since there are six regions in Fig. 5(a)], MI is calculated as (the detail is omitted here) Hence From the above result, the value gives a small positive spatial autocorrelation for the given data.The implementation of MI for noise detection in color images can be done with the same procedure as shown in the example above. The difference now is that a region used in the previous example will be replaced by a pixel in the filtering window for impulse noise detection. An example is given in Table 5, which shows a filtering window where the intensity range of a given pixel is between 0 and 255 for an 8-bit color image. Based on the 8-connectivity concept,1 we can construct a weight matrix similar to Fig. 5(b). The weight matrix is given in Table 5. Please note that only one channel is shown here. Table 5(a) A hypothetical example of 3×3 filtering window (one channel only), (b) its pixel numbering scheme for the pixels in (a), and (c) the binary connectivity (i.e., adjacency) matrix for (a) that shows which pixel is adjacent (marked by 1) to pixel number from 0 to 8. Otherwise, 0 will be used for nonadjacency. For example, for pixel 0, pixels 1, 3, and 4 are adjacent to it.
Based on the weight matrix formulated, for each color channel of RGB, MI can be calculated to check the degree of spatial autocorrelation. If all MI values from three channels have near value, then all the pixels in the window are relatively similar. In other words, if any MI value from three channels is close to , there is a high probability that impulse noises exist in the filtering window. The threshold value for determining the noise area is set up through our experiments, which will be explained in detail in Sec. 3.3. 3.2.Four One-Dimensional Laplacian KernelsBy means of a single detector based on MI index, it may not be robust to detecting impulse noise precisely. This is because MI measures a region’s spatial autocorrelation. In other words, MI focuses on all pixels of the filtering window, not the central pixel itself in the window. Hence, it is required to have the next level detector for impulse noise. The application of Laplacian kernels is one of the precise and simple detecting algorithms for impulse noise. In fact, several median-based impulse noise detectors often misclassify a noise-free pixel into the noise category.4,17 In our proposed filter, if only MI is used for the detection, there is a chance that pixels located on the lines and edges can be categorized as impulse noise also. Using the Laplacian kernels as the second-level detection will eliminate this problem. In a hypothetical example as shown in Figs. 6(a) and 6(b), we assume this image patch is a line with a direction in 135 deg. We also assume that the patch is identical in three channels. MI value is 0.079 in (a), and the center pixel is noise. Based on our proposed algorithm, if we set the threshold value to 0.0, the center pixel will be detected as a noise pixel (MI value is not less than the threshold) and will go through the denoising process. However, the minimum response from four 1-D Laplacian kernels is 0. Based on the algorithm in Ref. 4, the response is not considered a noise pixel since its value is so small (i.e., 0). The reason is that we usually set the threshold value greater than 0 for the response of Laplacian kernels. Hence, MI can overcome the weakness of the 1-D Laplacian kernels in this example. On the other hand, if a line is given as shown in (b), MI value is . Based on our proposed algorithm, if we set threshold value to 0.0, with the second level of detection (i.e., four 1-D Laplacian kernels), it will be detected as a noise-free pixel based on step 4 in our proposed algorithm (the condition is satisfied, but the condition is not satisfied). The minimum response from four 1-D Laplacian kernels is 0 (with the 135-deg direction of the 1-D Laplacian kernel); based on the algorithm in Ref. 4, the response is not considered a noise pixel since its value is small (i.e., 0). This center pixel will be considered a noise-free pixel. Hence, the weakness of MI can be complemented by the four 1-D Laplacian kernels. In Fig. 6, four 1-D Laplacian kernels are used in our experiments for the second-level noise detection. In fact, both and 1-D Laplacian kernels4 were tested simultaneously to verify which size of the kernel is more appropriate for the second-level detection. In our experiments, the kernels of different sizes do not show any significantly different results. At the first level of noise pixel detection, the corrupted image is tested through MI index; then if necessary, four 1-D Laplacian kernels will be used in the second-level detection. Four 1-D Laplacian kernels are sensitive to edges with four directions. The minimum absolute value from the four kernel operators is chosen as the detection value. The Laplacian kernel response is denoted as , and is a kernel operator as shown below where is a pixel and is a Laplacian kernel. Each kernel is applied to choose the minimum absolute value in the window. This value is compared with the threshold () value to determine whether it is a noise pixel or not [Eq. (16)]. Please note that we use to represent for the pixel in the examination in Eq. (16). If the value is larger than the threshold, it is an impulse noise. If the value is smaller than (or equal to) the threshold, then the current pixel is either a noise-free or edge pixel.4 The determination of this threshold will be discussed in Sec. 4 From the detection based on MI and Laplacian kernel response, a pixel is either classified as a noise pixel, in which the VMF will be called in to remove the noise, or the pixel value remains unchanged.3.3.Proposed Filtering Process with Vector Median FilterThe remaining issue related to the implementation of Moran’s vector median filter (MIVMF) is the determination of the threshold value because the improper threshold setting can produce unwanted filtering results. First, the threshold value for MI should be properly set up to check if pixels in the sliding window are located at the noise area or the relatively homogenous, noise-free area. We chose three images as shown in Fig. 3 to verify the proper threshold value. We tested all three images with different noise levels for 5%, 10%, and 15%, and the threshold value can be properly estimated. If MI is less than , then the Laplacian kernels will be used to determine if it is a noisy pixel. Otherwise, it is a noise-free pixel. As shown in Tables 6 and 7, the best threshold value is somewhere between and 0.1 from the results (bold print) of PSNR and SSIM. In Table 8, the best threshold value is between and 0. From those experimental results, this information can be used as an indicator for choosing a threshold value for MI index for a color image with different complexity in entropy. Consequently, the promising PSNR and SSIM results from Lena, peppers, and baboon images are located around . Hence, we set up our threshold value as for all images in our experiments. Furthermore, the difference in the range of possible threshold values for is so small such that the selection of this index is not critical to the filtered results. Table 6Threshold testing with Lena image: entropy 15.12.
Table 7Threshold testing with peppers image: entropy 15.30.
Table 8Threshold testing with baboon image: entropy 15.90.
The next threshold to be determined is the response of four 1-D Laplacian kernels. We tested many images to find out whether there exists any common threshold value which can be used for an image with different complexity. However, we could not find such a common threshold value with our initial experiments. Therefore, we set up the hypothesis that image complexity might be related with the threshold value of Laplacian kernels. We chose five images, as shown in Fig. 7, with different entropy levels and filtered with various threshold values. In our empirical study on the images tested, the threshold range is between 20 and 120, as shown in Fig. 8. In addition, the noise levels of 5%, 10%, and 15% were tested. In Fig. 8, only Laplacian kernels were used to determine the threshold () for the VMF. For the noise level 10% and 15%, the filtering process was run twice to improve overall outcomes measured by PSNR. For the 5% case, one filtering was good enough for removing impulse noise. The number of iterations is written next to the noise level, as shown in Tables 9Table 10Table 11Table 12–13. Table 9Results on ocean image. The number next to each noise level column shows the number of iterations in the filtering process.
Note: The best threshold is highlighted with bold font. Table 10Results on F-16 image. The number next to each noise level column shows the number of iterations in the filtering process.
Note: The best threshold is highlighted with bold font. Table 11Results on mountain image. The number next to each noise level column shows the number of iterations in the filtering process.
Note: The best threshold is highlighted with bold font. Table 12Results on caster image. The number next to each noise level column shows the number of iterations in the filtering process.
Note: The best threshold is highlighted with bold font. Table 13Results on IC motherboard image. The number next to each noise level column shows the number of iterations in the filtering process.
Note: The best threshold is highlighted with bold font. By observing experimental results from Tables 9–13, we discovered a trend in the pattern the setup of the threshold value for Laplacian kernels response for an image. This trend is to increase the threshold value based on the entropy of an image being denoised, as shown in Fig. 8. Therefore, it is quite straightforward to calculate the entropy of an image to estimate the threshold value. Based on this assumption, we used the interpolation method for assessing a threshold value for an image being denoised in our simulation in Sec. 4. From the discussions above, our proposed denoising Algorithm 1 (MIVMF) is summarized in the steps. Please note that the algorithm can be repeated if needed. In other words, the output from the first pass of the algorithm becomes the input for the second pass and so on. Algorithm 1MIVMF.
Table 14Some additional parameters for filters used in the experiments.
4.Experimental ResultsTo evaluate the performance of the proposed algorithm and compare it with other well-developed denoising filters, we tested all the filters listed in Table 4. Various test parameters, including the filtering window size and threshold values, are listed in Table 14. All test conditions except our proposed filter are based on the suggested parameter values from the literature reviewed. The size of the neighborhood window used for all filters is . To compare the denoising performance, PSNR, SSIM, and time complexity measured in seconds are calculated. Three images from Fig. 3 are used in the experiments. Some of denoised images are shown in Figs. 9Fig. 10Fig. 11–12. Please note that in the noise levels 10% and 15%, both RSVMF and MIVMF were run twice to obtain the best outcome measured by PSNR. Other filters were run only once. (Otherwise, the image will be blurred if they were run more than once.) For the repetition, the output from the first pass of the algorithm becomes the input for the second pass and so on. In our experiments, there is no significant difference for a filter in terms of the time complexity for processing an image with any noise level or image complexity based on the entropy measure. This can be seen from Tables 15Table 16–17. The proposed MIVMF is the best among all the filters tested with the PSNR measure. For the SSIM measure, which is for human eye perception, all the filters are almost identical, except that BVDF is degrading when the noise level is increased. Table 15Tested results of Lena with entropy 15.12 and T0=67 in 5%, 10%, and 15% noise levels.
Note: In 10% noise, the MIVMF was run two iterations, which required 1.48 and 1.05 s (2.53-s total) for first and second iteration, respectively. Similarly, for 15%, it took 1.71 and 1.07 s (2.78-s total). For RSVMF, we also ran two iterations for 10% and 15%. For 10% noise, it took 3.87 and 3.85 s (7.72-s total). For 15%, it took 3.96 and 3.80 s (7.76-s total). s denotes seconds. Table 16Test results of peppers with entropy 15.30 and T0=68 in 5%, 10%, and 15% noise levels.
Note: Similar to the note in Table 15: In 10%, the MIVMF took 1.43 and 0.95 s (2.38-s total) for first and second iteration, respectively. For 15%, it spent 1.66 and 0.99 s (2.65-s total). For RSVMF with 10%, it took 3.73 and 3.63 s (7.36-s total). For 15%, it took 3.75 and 3.64 s (7.39-s total). s denotes seconds. Table 17Test results of baboon with entropy 15.90 and T0=109 in 5%, 10%, and 15% noise levels.
Note: Similar to the note in Table 15: In 10% noise, our proposed MIVMF took 1.42 and 1.05 s (2.47-s total) for first and second iteration, respectively. For 15%, it spent 1.60 and 1.10 s (2.70-s total). For RSVMF with 10%, it took 3.89 and 3.89 s (7.77-s total). For 15%, it took 3.90 and 3.86 s (7.76-s total). s denotes seconds. On the contrary, for an image with medium entropy, such as the peppers image, the PSNR measure for all filters is decreasing when the noise level is increased, as shown in Table 16. This phenomenon is also shown in the SSIM measure for the BVDF. Otherwise, the SSIM is very similar for all the filters tested. The proposed MIVMF has the best PSNR measure for an image with low, medium, and high entropy. For images with higher entropy, a trend exists for all filters that the SSIM is decreasing when the noise level is increasing, except for the VMF in which the 15% noise shows a high SSIM value, as shown in Table 17. In terms of the PSNR measure, all of the filters tested decrease if the image complexity is increased. However, the MIVMF has the overall highest PSNR value at different noise levels and with almost any complexity of images measured in entropy. Based on the results shown in Tables 15–17, there is only slight difference on the performance of RSVMF and MIVMF for the low noise level. Hence, to measure the effectiveness of RSVMF and MIVMF for heavy noises, both filters were tested on 20%, 25%, and 30% noise levels on three images, and the results are shown in Tables 18Table 19–20. (Please note that the results are from three iterations of each filter, and overall time spent is the sum of three iterations.) Test threshold () for RSVMF is 1.5 based on Ref. 12. While the SSIM is similar for both filters, the proposed MIVMF has a better PSNR measure with the higher noise level. The MIVMF has the overall highest PSNR value in those noise levels. In addition, the running time for the MIVMF is less than half of the RSVMF. Table 18A comparison of RSVMF and MIVMF on heavy noise added Lena image.
Note: s denotes seconds. Table 19A comparison of RSVMF and MIVMF on heavy noise added peppers image.
Note: s denotes seconds. Table 20A comparison of RSVMF and MIVMF on heavy noise added baboon image.
Note: s denotes seconds. Table 21The PSNRs for different numbers of iterations and filters are listed: (a) parrot was corrupted with 5% noise, (b) penguin with 5% noise, (c) koala with 5% noise, and (d) parrot with 15% noise. Each filter was run six times in the experiments and is represented with the respective number in the first column, and then each PSNR was calculated for each run. The saturated modes were highlighted with italicized font in the table for the MIVMF filter.
Note: The saturated modes were highlighted with italicized font for the MIVMF filter. To further demonstrate the effectiveness of the proposed algorithm in denoising, we tested on some different images and performed a number of iterations for each filter listed in Table 14. Images tested are shown in Fig. 13 which includes parrot (entropy 15.25), penguin (entropy 15.40), and koala images (entropy 16.18). As mentioned in Table 14, we used 0.0 for MI threshold and threshold is interpolated based on the image entropy. Table 21 shows the PSNR score for each filter tested. To each image 5% noise was added, and each filter was run only once in the experiments. Experimental results show that the proposed algorithm, MIVMF, has the best PSNR scores. In our study, some filters will cause the distortion of images and blur the image details if the filter is run multiple times on the image. To test the efficiency of the proposed algorithm, we ran each filter with a number of iterations to compare the filtered results. The results were obtained with six iterations for each filter and are shown in Table 21. All PSNR scores for VMF, BVDF, CWVMF, ACWVMF, PGVMF, and RSVMF are decreasing when the number of iterations is increased. This phenomenon will continue if the filter is iterated again. This indicates that the filtered image is distorted and image details are not preserved. The PSNR scores with our proposed MIVMF either decrease or increase slightly, as shown in Table 21. In either case, the MIVMF will reach a saturation state, and its PSNR score will not change again. For example, in Tables 21(a) and 21(d) (first column), the PSNR of the MIVMF will stay in 23.57 and 26.43, respectively, no matter how many more iterations are repeated. Similarly, in Tables 21(b) and 21(c) (first column), the PSNR will stay in 29.59 and 30.33, respectively. This important characteristic shows that the MIVMF is an efficient filter with the good detection mechanism for determining if a pixel is noise or not. Hence, the image details will not be damaged or blurred. We also observed that the MIVMF has the highest PSNR value no matter how many iterations were repeated. In addition, the MIVMF runs much faster than other filters, as shown in Tables 15–20. By observing the experimental results in Tables 15–21 and Figs. 9–12, the proposed MIVMF shows a promising result compared with other vector-based filters. We developed an efficient denoising filter for accomplishing better time complexity and efficient denoising results, which was the goal set up at the beginning of this study. Our two-level detection function based on MI and 1-D Laplacian kernels works properly, so it can reduce the processing time and produce better filtered results compared with other similar type of vector-based filters. 5.Conclusion and Future WorkWe propose a noise detection and removal filter that uses MI statistics (i.e., MI value) to calculate the spatial autocorrelation indices and combine with four 1-D Laplacian kernels for random-valued impulse noise detection for color images. As far as we know, this may be the first attempt to use MI statistics for color image denoising. This is an adaptive filter in which the MI value and response of four 1-D Laplacian kernels are calculated for each neighborhood window. This two-level noise detection scheme works efficiently as it has the capability of distinguishing a given pixel as noise or not. The VMF is then used to process those noisy pixels and leave noise-free pixels untouched. Our proposed filter can be considered an efficient extension of the VMF in a sense. As an extension, the proposed filter can achieve better denoising results and efficient time complexity for the color images, which are corrupted with random-valued impulse noises. To determine if a pixel is noise or not, two levels of detection are used. The first level is controlled by the MI value, and the second level is measured by the response of four 1-D Laplacian kernels. An empirical study was able to estimate the threshold value for MI statistics. An interpolation method, which can roughly estimate the threshold value that is used in the comparison with the response of four 1-D Laplacian kernels for an image being denoised based on the entropy measure, was also established. The threshold value calculated with this interpolation method was tested in our experiments and illustrated with good denoising results. MI statistics shows an interesting result in our exploration of the application of color images for detecting random-valued impulse noises. We compared the proposed MIVMF with other vector-type MFs, including VMF, BVDF, CWVMF, ACWVMF, PGVMF, and RSVMF, using the criteria of PSNR and SSIM. The proposed MIVMF shows promising denoising results based on these criteria. The MIVMF is faster than the other filters tested in the experiments regardless of the noise level and image complexity. For the PSNR, the MIVMF has a higher value than the compared filters. By the visualization, the proposed MIVMF can avoid the image blurring results, preserve edge details in the image, and achieve superior noise reduction. We found that this kind of spatial statistic is very useful in the color image denoising algorithm. Although we were able to estimate the threshold values that were successfully used in our proposed filter, the issue of determining an optimal and adaptive threshold value for detecting a noise pixel remains a challenging problem in our exploration of MI statistics. If this threshold value can be estimated with an adaptive approach, it would be more useful to generate better denoising outcomes. We only applied the concept of spatial autocorrelation based on MI statistics and 1-D Laplacian kernels for the impulse noise detection in color images. We concentrated on the nonfuzzy vector-type MFs in this work. Several fuzzy vector-type filters that work well for noise detection and removal also exist. For future work, we plan to investigate the following four tasks: (1) to develop an adaptive method that can estimate the threshold values more accurately, (2) to determine if the MIVMF is suitable and efficient for hyperspectral remote sensing images, (3) to extend our detection scheme to other noise models, such as Gaussian noise, and (4) to explore the fuzzy vector-type MFs and determine if the fuzzy mathematics can be used to improve the proposed MIVMF performance for denoising. AcknowledgmentsThe authors would like to thank all the reviewers for their constructive comments, which helped us to improve the quality of the paper. ReferencesM. Lebrun et al.,
“Secrets of image denoising cuisine,”
Acta Numer., 21 475
–576
(2012). http://dx.doi.org/10.1017/S0962492912000062 0962-4929 Google Scholar
X. L. Xie and G. Beni,
“A validity measure for fuzzy clustering,”
IEEE Trans. Pattern Anal. Mach. Intell., 13
(8), 841
–847
(1991). http://dx.doi.org/10.1109/34.85677 ITPIDJ 0162-8828 Google Scholar
Z. Zhou,
“Cognition and removal of impulse noise with uncertainty,”
IEEE Trans. Image Process., 21
(7), 3157
–3167
(2012). http://dx.doi.org/10.1109/TIP.2012.2189577 Google Scholar
S. Zhang and M. A. Karim,
“A new impulse detector for switching median filters,”
IEEE Signal Process. Lett., 9
(11), 360
–363
(2002). http://dx.doi.org/10.1109/LSP.2002.805310 Google Scholar
H.-L. Eng and K.-K. Ma,
“Noise adaptive soft-switching median filter,”
IEEE Trans. Image Process., 10
(2), 242
–251
(2001). http://dx.doi.org/10.1109/83.902289 Google Scholar
E. Abreu et al.,
“A new efficient approach for the removal of impulse noise from highly corrupted images,”
IEEE Trans. Image Process., 5
(6), 1012
–1025
(1996). http://dx.doi.org/10.1109/83.503916 Google Scholar
F. Ahmed and S. Das,
“Removal of high-density salt-and-pepper noise in images with an iterative adaptive fuzzy filter using alpha-trimmed mean,”
IEEE Trans. Fuzzy Syst., 22
(5), 1352
–1358
(2014). http://dx.doi.org/10.1109/TFUZZ.2013.2286634 Google Scholar
R. Lukac and K. N. Plataniotis, Color Image Processing, Method and Applications, CRC Press, Boca Raton, Florida
(2007). Google Scholar
V. Novoselac and B. Zovko-Cihlar,
“Image noise deduction by vector median filter,”
in 54th Int. Symp. ELMAR-2012,
(2012). Google Scholar
M. E. Celebi, H. A. Kingravi and Y. A. Aslandogan,
“Nonlinear vector filtering for impulse noise removal from color images,”
J. Electron. Imaging, 16
(3), 033008
(2007). http://dx.doi.org/10.1117/1.2772639 Google Scholar
B. Smolka, K. Plataniotis, A. N. Venetsanopoulos,
“Nonlinear techniques for color image processing,”
Nonlinear Signal and Image Processing: Theory, Methods, and Applications, 445
–505 CRC Press, Boca Raton, Florida
(2004). Google Scholar
M. E. Celebi and Y. A. Aslandogan,
“Robust switching vector median filter for impulsive noise removal,”
J. Electron. Imaging, 17
(4), 043006
(2008). http://dx.doi.org/10.1117/1.2991415 Google Scholar
R. Lukac et al.,
“Vector sigma filters for noise detection and removal in color images,”
J. Visual Commun. Image Represent., 17 1
–26
(2006). http://dx.doi.org/10.1016/j.jvcir.2005.08.007 JVCRE7 1047-3203 Google Scholar
T.-J. Chen et al.,
“A novel image quality index using Moran I statistics,”
Phys. Med. Biol., 48
(8), N131
(2003). http://dx.doi.org/10.1088/0031-9155/48/8/402 PHMBA7 0031-9155 Google Scholar
K.-S. Chuang and H.-K. Huang,
“Assessment of noise in a digital image using the join-count statistic and the Moran test,”
Phys. Med. Biol., 37
(2), 357
–369
(1992). http://dx.doi.org/10.1088/0031-9155/37/2/004 PHMBA7 0031-9155 Google Scholar
J. Astola, P. Haavisto and Y. Neuvo,
“Vector median filters,”
Proc. IEEE, 78
(4), 678
–689
(1990). http://dx.doi.org/10.1109/5.54807 IEEPAD 0018-9219 Google Scholar
S. Liu,
“Adaptive scalar and vector median filtering of noisy color images based on noise estimation,”
IET Image Process., 5
(6), 541
–553
(2010). http://dx.doi.org/10.1049/iet-ipr.2009.0408 Google Scholar
R. Lukac and B. Smolka,
“Application of the adaptive center-weighted vector median framework for the enhancement of cDNA microarray images,”
Int. J. Appl. Math. Comput. Sci., 13
(3), 369
–383
(2003). Google Scholar
A. Nagabhushna,
“Quality assessment of resultant images after processing,”
Comput. Eng. Intell. Syst., 3
(7), 105
–112
(2012). Google Scholar
W. Malpica and A. Bovik,
“SSIM based range image quality assessment,”
in 4th Int. Workshop on Video Processing and Quality Metrics for Consumer Electronics,
(2009). Google Scholar
B. Smolka and A. Chydzinski,
“Fast detection and impulsive noise removal in color images,”
Real-Time Imaging, 11 389
–402
(2005). http://dx.doi.org/10.1016/j.rti.2005.07.003 Google Scholar
R. Lukac et al.,
“Vector filtering for color imaging, open a world of possibility,”
IEEE Signal Process. Mag., 22
(1), 74
–86
(2005). http://dx.doi.org/10.1109/MSP.2005.1407717 Google Scholar
T. Chen and H. Wu,
“Adaptive impulse detection using center-weighted median filter,”
IEEE Signal Process. Lett., 8
(1), 1
–3
(2001). http://dx.doi.org/10.1109/97.889633 Google Scholar
C. Kenny et al.,
“Peer group image enhancement,”
IEEE Trans. Image Process., 10
(2), 326
–334
(2001). http://dx.doi.org/10.1109/83.902298 Google Scholar
B. Smolka and M. Perczak,
“Generalized vector median filter,”
in Proc. of the 5th Int. Symp. on Image and Signal Processing and Analysis,
(2007). http://dx.doi.org/10.1109/ISPA.2007.4383700 Google Scholar
B. Smolka,
“Soft switching technique for impulsive noise removal in color images,”
in 5th Int. Conf. on Computational Intelligence, Communication Systems and Networks,
(2013). http://dx.doi.org/10.1109/CICSYN.2013.22 Google Scholar
E. Chang et al.,
“A denoising algorithm for remote sensing images with impulse noise,”
in IEEE Int. Conf. on Geoscience and Remote Sensing (IGARSS16),
(2016). http://dx.doi.org/10.1109/IGARSS.2016.7729750 Google Scholar
P. Rogerson, Statistical Methods for Geography, 2nd ed.Sage Publications, London
(2006). Google Scholar
P. Moran,
“The interpretation of statistical maps,”
J. R. Stat. Soc. Ser. B, 10 243
–251
(1948). Google Scholar
BiographyChih-Cheng Hung is a professor of computer science at Kennesaw State University (KSU), Georgia, USA. He was with the Intergraph Corporation from 1990 to 1993 for research and development of remote sensing image processing software. He was also with the Information Processing Lab in Anyang Normal University, China, and the Laboratory for Machine Vision and Security Research in KSU. His research interests include image processing, pattern recognition, and artificial intelligence. Eun Suk Chang received his BS degree in geoeducation from Korea University in Seoul, South Korea, in 2003 and his MS degree in computer science from KSU, Georgia, in 2015. Currently, he works as a Windows developer at RenewedVision.com. He was with the Laboratory for Machine Vision and Security Research in KSU. His research interests include geography information science, remote sensing, image processing, spatial statistics, and data mining. |