2.1

Image model

The SIR model is a probabilistic framework to determine algorithms for detection, recognition, location, parameter estimation or segmentation of an object in an image. We assume that the observed scene is composed of two zones: the target and the background. Furthermore, the target’s and the background’s gray level are supposed to be unknown and we model their values as independent random variables.

In the following mathematical developments, one-dimensional notations are used for simplicity, and bold font symbols will denote N-dimensional vectors. For example s = {s_i ∣i ∈ [1, N]} denotes the input image composed of N pixels. For each considered case, the purpose of the image processing algorithm is to estimate an unknown parameter which will be denoted symbolically θ. For example, in detection applications, θ is a binary value, for recognition (or more precisely discrimination), it is a value belonging to a discrete set, and for location, it is the position of some characteristic points of the object (for example the center of gravity). For orientation estimation, θ is a set of possible angles. For segmentation purpose, θ is the shape of the object. In the latter case, if the shape is approximated by a polygonal contour, θ is the set of coordinates of the nodes of the polygon (see table 1).

Table 1:

Examples of application and nature of the parameter θ.

Let denote a binary window function that defines a certain location, orientation or shape for the target, so that is equal to one within the target and to zero elsewhere. Note that in the following, we will use the same notation w^θ for the previously defined binary function and for the set of pixels for which this function is 1. Let us consider the different hypotheses H_θ that consist in assessing a binary window w^θ to the target in the input image s, so that we can write:

where the target’s gray levels a and the background noise b are random variables. These random variables are characterized by their respective pdfs and where μ_a and μ_b, are the parameters of the pdfs which will be considered as a priori unknown. These parameters can be scalars or vectors if more than one scalar parameter is needed to determine the pdf.

Equation 1 with the pdfs and is the image model. The parameter of interest is θ while the parameters μ_a and μ_b are nuisance parameters. We will consider the maximum a posteriori (MAP) estimation of the parameter θ. The optimal estimate is thus obtained by maximizing the conditional probability P[H_θ∣s]. This conditional probability can be obtained by using Bayes law [8]:

Considering that all hypotheses H_θ are equiprobable, the MAP estimation of θ is equivalent to the Maximum likelihood estimation obtained by maximizing P[s∣H_θ]. In the following, we will analyze the ML estimation since generalization to the MAP estimate is obvious using Eq. 2. With the image model of Eq. 1, the likelihood is:

where we have explicitly denoted the dependence on the unknown parameters μ_a and μ_b. Please remember that we use the same notation w^θ (or w̄^θ) for binary support functions and for the set of pixels for which the value of these functions is 1.

The question is now how to deal with the nuisance parameters μ_a and μ_b in order to express the likelihood as a function of the only parameter of interest θ and of the input image s. There exists several methods to deal with nuisance parameters and the three most frequently used are the Maximum Likelihood (ML) estimation, the Maximum A Posteriori (MAP) estimation and the marginal Bayesian approach. With the marginal Bayesian and the MAP approaches, the nuisance parameters are considered as random variables and prior density probability functions have to be chosen [8, 17].

Let π_a(μ_a) and π_b(μ_b) denote these priors, the marginal Bayesian approach is simple from a theoretical point of view and is based on the Bayes relation:

where a symbolic notation has been used for the multidimensional integrals:

and

if μ_a and μ_b are n dimensional parameters. With Eq. 4 the likelihood P[s∣H_θ] is obtained and the problem is solved from a theoretical point of view.

With the MAP approach, instead of eliminating the nuisance parameters as with the marginal Bayesian approach, one considers estimates of their values. If we are not interested in the nuisance parameter’s values, this approach is suboptimal (see [18] for a discussion in an analogous situation). Nevertheless, this method can be of interest from a practical point of view, since it can be easier to determine the MAP estimates of the nuisance parameters than integrating as in Eq.4. The MAP estimates of the nuisance parameters are the values which maximize P[s, μ_α, μ_b∣ H_θ]. They are obtained by the following equation:

where argmax (Z) is the value of the parameters y which maximizes Z. The estimate of can be obtained by maximizing the pseudo-likelihood:

It is worth noting that since µ_a^MAP[s] and µ_b^MAP[s] are functions of s, the considered criterion P [s, μ_α^MAP[s], μ_b^MAP[s]∣H_θ] with the MAP approach is not a likelihood, as it is the case with the marginal Bayesian approach (see Eq.4).

The ML method is analogous to the MAP approach from a technical point of view but is not based on the modelization of the nuisance parameters as random variables. The important consequence is that no prior has to be introduced as for the marginal Bayesian and the MAP approaches. The ML estimates are given by:

The estimation of θ is obtained by maximizing the pseudo-likelihood:

One can note that the ML approach is analogous to the MAP approach if a uniform (also denominated non informative) prior for the nuisance parameters is considered. The ML approach is simpler since no prior pdfs are needed (although the obtained solution can be unstable [19]) and in the following we will mainly discuss results for the ML solutions.

2.2

The exponential family

Members of the exponential family include the Bernoulli, Gamma, Gaussian, Poisson, Rayleigh and many other familiar statistical distributions [20]. These distributions can be used to describe realistic situations. The case of binary images is simple to handle since the probability law for the gray levels can be described with a Bernoulli pdf. It is also well known that at low photon levels, the noise present in images is described by Poisson pdf (due to the discrete nature of the events, which are the arrivals of photons on the sensor). This situation occurs for example, in astronomical imagery when the exposure time of the sensor is short. Synthetic Aperture Radar (SAR) intensity images are corrupted by a multiplicative noise, also known as speckle [21], which can be described with Gamma pdf. This issue has been widely studied over the past years and it is now well known that in order to obtain efficient algorithms, the statistical properties of the speckle have to be taken into account in the design of image processing algorithms (see [22], [23] and references therein). Ultrasonic medical images correspond to amplitude detection of the incident acoustic field and the speckle noise can be described with a Rayleigh pdf [24]. Finally, we will also discuss the case of optronic images and the relevance of normal laws when a whitening preprocessing is used [25].

Probability density functions (pdf) which belong to the exponential family are defined by [20]:

where μ = [μ₁,μ₂,…,μ_n]^Τ is the vector of parameters of the pdf, κ(x) is a scalar function of x while α(μ) and f (x) are p-component vector functions of respectively μ and x. We summarize in table 2 the pdfs of the exponential family which will be discussed in the following as well as the parameters which will be considered unknown.

Table 2:

pdf of the considered laws of the exponential family and their corresponding parameters. δ(x) is the Dirac distribution, N is the set of integers and n! = n(n − 1)..2.1.

These pdf possess simple sufficient statistics [20]. Let us consider a sample χ_u of n_u random variables distributed with a pdf . A sufficient statistic Τ[χ_u] for μ_u is a function of the sample χ_u that contains all the information relevant to estimating the parameter μ_u in the ML sense. If the pdf belongs to the exponential family, the likelihood is:

The ML estimate of μ_u is thus:

which can also be written:

with:

which clearly defines the sufficient statistics of the exponential family. In table 3 we provide the sufficient statistics for the pdfs of the exponential family which will be discussed in the following. For that purpose, let us define S_u the set of n_u random variables from which the parameters are inferred (i.e. the set of pixels from which the unknown parameters are estimated). In particular n_a is the number of pixels in w^θ and n_b is the pixel number in the background region w̄^θ.

Table 3:

Mathematical expressions of the sufficient statistics for the parameters defined in Table 2.

For the image processing problems we consider, the likelihood is a function of w^θ. Let us denote L(s,w^θ) the likelihood obtained with the marginal Bayesian approach or the pseudo likelihood obtained with the MAP or the ML approach, and ℓ(s, w^θ) its logarithm. It is easy to show the following property.

Property:

Whatever the adopted approach to deal with the nuisance parameters, the loglikelihood of an hypothesis H_θ is:

with:

where . Functions F_a and F_b depend on the considered pdf and on the prior on the nuisance parameters for the marginal Bayesian and MAP approaches. They are equal in the case of a ML estimation of the nuisance parameters.

It is clear that the last term G(s) is independent of the hypotheses H_θ if M̄^θ is the image to be analyzed or a subwindow in this image which is chosen independently of H_θ. We will see in the following that the estimation of the likelihood of H_θ is a function of the input image s through the determination of the sufficient statistics. In table 4, the expressions of the varying part of the loglikelihood defined in Eq.14 are provided when the nuisance parameters are estimated with the ML method. We propose in the next sections to illustrate these concepts with different kinds of applications.

Table 4:

Mathematical expressions which define the varying part of the loglikelihod (see Eq. 14) in terms of the sufficient statistics defined in Table 3. ln(z) is the neperian logarithm.

3.1

Introduction and limitations of the ML approach

In order to perform the important task of detecting and locating a target appearing on a random background, a pattern recognition system must discriminate between the background and the target. The background can thus be considered as noise. This noise is not additive, since it does not affect the target: it is said to be non-overlapping. Classical linear filters have been shown to often fail in presence of such noise [4], and an explanation of this phenomenon has been presented in [6]. Different techniques [26] [9] have been proposed in the past for detection and location of a target with a known internal structure and an unknown uniform illumination in presence of non-overlapping background noise. When the target’s gray levels are unknown it is necessary to introduce different approaches [10, 27, 25, 28, 11]. Such a situation can happen when the target is subject to sun reflections in optical images, when temperature changes in infrared images or when only a shape model is available for the location of the target in the input image. In this case, with the proposed solutions [10, 27], the pixel values of both the target and the background have been modeled as random variables with Gaussian pdfs but with different parameters. The only a priori knowledge is thus the silhouette of the target, which defines the frontier between the target and the background. These models have been recently generalized to Gamma pdfs [27] and to binary images [29]. We discuss in the following the general solution for the exponential family which includes the previous cases as particular cases.

With the SIR approach the input image model is:

a and b represent the gray levels of, respectively, the target and the background zone. The unknown parameter θ is now simply the position of the object in the scene. The ML solution for the estimation of the location θ can be written (see Eq. 14):

Here again, the functions F_a and F_b depend on the considered pdf and on the prior on the nuisance parameters for the marginal Bayesian and MAP approaches but are equal in the case of the ML estimation of these parameters. The mathematical equations for Bernoulli, Gaussian, Gamma, Poisson and Rayleigh pdfs can be easily obtained from tables 2, 3 and 4.

The main practical problem with the SIR models is that the input image is assumed to be composed of two homogeneous random fields a_i and b_i. Furthermore, the random variables are assumed to be independently distributed. These conditions may not be fulfilled in real-world images. We discuss in the following two techniques in order to overcome these limitations.

3.2

The maximum likelihood ratio test (MLRT) approach

In the SIR image model, the background region w̄^θ, that is, the whole image but the target, is considered to have homogeneous statistics. This is often a non-realistic assumption since real-world backgrounds are in general better modeled with several zones having different average values. In order to overcome this problem, we will estimate the statistics in a small subwindow M^θ centered on the assumed target location θ (see Fig. 1). Indeed, if we consider a sufficiently small subwindow, the hypothesis that the background is homogeneous becomes a better approximation.

Figure 1:

Sketch of the two hypotheses considered in the MLRT scheme.

In order to better understand the method we propose, let us temporarily set aside the object location problem and let us consider the simpler problem of object detection. It consists in determining if there is an object of shape w in the center of the sub-image M^θ or not. More precisely, we want to discriminate between the two following hypotheses:

The value of the threshold τ₀ sets a compromise between the probability of detection and the probability of false alarm. Using this method, we can determine if the target is present or not at each location θ. If there may be several targets in the image, it is thus possible to determine their locations.

Let us now return to the problem of object location, which is slightly different : We assume that we know that there is only one target in the image (which can be the case in tracking applications for example), and we want to determine its location. In order to do so, we can extend the previously described detection algorithm to a location algorithm by choosing as the estimate of the target location the position which maximizes τ(θ). In other words, the estimated location will be:

In the following, we will call this estimation approach the ”maximum likelihood ratio test” (MLRT). This procedure is a heuristic extension of the optimal detection algorithm. Note that similar procedures have been used for locating edges in optical images (with Gaussian grey level statistics) [31] and in Synthetic Aperture Radar images (with Gamma grey level statistics) [22].

We shall now specify the expression of the likelihood ratio τ(θ) for a SIR image belonging to the exponential family:

where n_c is the number of pixels of the scanning subwindow M^θ and thus n_c = n_a + n_b. Here again, the functions F_a and F_b are dependent of the considered pdf and the prior on the nuisance parameters for the marginal Bayesian and MAP approaches but are equal in the case of a ML estimation of these parameters.

We now specialize Eq. 24 to particular pdf’s belonging to the exponential family when the ML estimation of the nuisance parameters is considered. For simplicity reasons, let us introduce the following notations:

where ℓ = 1 or 2.

In the Bernoulli case, Eq. 24 becomes

in the Gaussian case,

in the Gamma case,

in the Poisson case

and in the Rayleigh case

3.3

The whitening process

In some real-world images, the statistics of both the target and the background cannot be approximated with good precision with uncorrelated random fields. In these situations, the SIR filter is then suboptimal and can fail. In figure 2, two scenes and the maximum of each line of their respective generalized correlation planes obtained with the SIR technique (i.e. ℓ(s∣H_θ)) are shown. The maximum of the correlation plane represents the estimated location of the target. In scene (a), the pdf’s of both the target and the background are white and Gaussian whereas those of scene (b) are also Gaussian but correlated. One can note that the SIR algorithm adapted to white Gaussian statistics fails on scene (b) whereas it is able to locate the target on scene (a).

Figure 2:

(a) : Reference shape, (b) : Example of a scene with white Gaussian textures for the target and the background. The shape of the target is the reference shape in (a), (c) : Example of a scene with correlated Gaussian textures for the target and the background. The shape of the target is the reference shape in (a), (d) : Result of processing (b) with the SIR algorithm adapted to white Gaussian statistics, (e) : Result of processing (c) with the SIR algorithm adapted to white Gaussian statistics. Note : (d) and (e) are plots of the maximum of each line of the output plane of the considered method.

We want to design an optimal algorithm for the location of a random correlated target appearing on a random correlated background. The main problem consists in finding texture models that characterize real situations and for which the optimal solution is mathematically simple. Such a method has been recently designed using the same random Markov field model for both the target and the background [32]. This case represents a difficult, but particular situation.

In this subsection, we propose to apply a preprocessing to the input image in order to obtain an image with white Gaussian textures and then to apply the SIR method which is optimal in that case. However, as soon as the textures of both the target and the background are strongly correlated, the preprocessing introduces a third region in the preprocessed image, which characterizes the frontier between the target and the background. Following reference [25] we will thus discuss how to model this region with a white Gaussian random field and we design a SIR filter that takes into account the three regions (i.e. the background, the target and the frontier).

The Fourier Transform of s is denoted ŝ (or ŝ(v) at frequency v), z* is the complex conjugate value of z and |z| its modulus. We define the whitening filter in the Fourier domain by:

where ϵ is a small positive constant introduced as a regularization parameter which avoids divergence when |ŝ(v)| is close or equal to zero. The Fourier Transform v̂ of the preprocessed image v is thus:

One can note that since s and ĥ are real, v is also real. It is easy to show that the square modulus of v̂ is approximately constant. We can conjecture that the pixels of the preprocessed image z are approximately Gaussian uncorrelated variables. In figure 3, we show a target with a correlated texture which appears on a random correlated background and the obtained preprocessed image. One can show that describing the pixel values of the preprocessed image as Gaussian random variables is a good approximation. If we model the preprocessed image with two independent regions and if the nuisance parameters are estimated in the ML sense, the SIR method leads to (see Eq. 14):

where

where ℓ = 1 or 2.

Figure 3:

(a) : Example of a scene with strongly correlated Gaussian textures for both the target and the background, (b) : Whitened version of (a).

However, as one can remark in figure 3, the preprocessing can introduce three regions in the preprocessed image. Indeed, as soon as the textures are strongly correlated, a frontier appears between the target and the background.

Let f denote that frontier (and respectively a the target and b the background) and let w^f (resp. w^a and w^b) define the new disjoint window functions composed of n_f (resp. n_a and n_b) pixels so that (resp. and ) is equal to one within the frontier (resp. the target and the background) and to zero elsewhere when the target is located at the center of the image. We thus propose to describe the preprocessed image z in the following way [33]:

when the target is supposed to be centered on the θ^th pixel of the image.

Using an analogous approach as previously, we can design a SIR filter that takes into account three regions. This leads to:

where , and are defined as is Eq.34. All these quantities can be determined by correlating binary masks with images z and z² [25]. They can be obtained with a simple optoelectronical architecture or using FFT algorithm applied to the images z_i and z_i².

3.4

The implementation issue

An interesting point is that ℓ[s|H_θ] in the standard SIR approach and ln[τ(θ)] in the MLRT approach can easily be rewritten using correlation operations. Let us consider the MLRT approach, and let [f*g]i denote the correlation between f and g:

and let w = w°. Eq. 25 becomes:

where we remember that M = w + w̄. Since the most intensive computations are involved in with u = a, b, c or f, this new formulation is very attractive because it is closely connected to the detection architecture described in [10]. Indeed, the detection and location steps require the same correlation functions. A simple optoelectronical architecture could thus perform the detection and/or location with this kind of hardware.

4.1

Introduction

An important goal of computational vision and image processing is to automatically recover the shape of objects from various types of images. Over the years, many approaches have been developed to reach this goal and in this section, we focus on the segmentation of a unique object in the scene. The unknown parameter θ is now the shape of the object in the scene.

A classical approach consists in detecting edges and linking them in order to determine the shape of the object presents in the image. However this approach does not use the knowledge that the object is simply connected. On the other hand, deformable models (also called ”snakes”) incorporate knowledge about the shape of the object from the start. Broadly speaking, a snake is a curve which has the ability to evolve (under the influence of a mathematical criterion) in order to match the contour of an object in the image. The first “snakes” [14] were driven by the minimization of a function in order to move them towards desired features, usually edges. This approach and its generalization [34] [35] [36] are edge-based in the sense the information used is strictly along the boundary. They are well adapted to a certain class of problems, but they can fail in presence of strong noise.

The SIR-based snake we will describe in the following belongs to the deformable template methods, which are parametric shape models with relatively few degrees of freedom. They constitute another interesting approach to recover the shape of an object [37] [38] [16]. The template is matched to an image, in a manner similar to the snake, by searching the value of a parameter vector θ (i.e. the node positions) that minimizes an appropriate mathematical criterion. One can cite for example strategies based on the consideration of the inner and the outer regions defined by the snake, which have been recently investigated [15] [16] [39] [40]. It is interesting to note that a statistical processing method can take full advantage of many suitable descriptions of the measured signals (see for example [41] [12] [24] [42] [43]).

The SIR approach allows one to determine this appropriate criterion. First, we generalize the approaches proposed in [12] [42] [44] to different statistical laws which belong to the exponential family and which are well adapted to describe physical situations. This technique is actually an extension of the optimal detection approach introduced in [10] and generalized in the previous sections.

4.2

The SIR snake model

The purpose of segmentation is therefore to estimate the most likely shape for the target in the scene. Note the difference with the optimal location problem of section 3 where the silhouette of the target was known whereas its position had to be found. To achieve the shape estimation issue, we use a k-node polygonal active contour that defines the boundary of the shape. w^θ is now a polygon-bounded support function, one-valued on and within the snake and zero-valued elsewhere and θ is the set of the positions of each node of the contour. Let us consider the different hypotheses H_θ that consist in assessing a shape w^θ to the target by assessing a position to each node of the contour, so that we can write:

The optimal choice for w is the one which maximizes the conditional probability P[H_θ|s]. The ML estimation of the shape (i.e. θ) is obtained by maximizing the a priori probability P[s|H_θ]; which corresponds to the likelihood of the hypothesis.

Under the previous assumptions we can now specify the expression of the likelihood ℓ[s|H_θ] for a SIR image which belongs to the exponential family.

Here again, the functions F_a and F_b are dependent of the considered pdf and the prior on the nuisance parameters for the marginal Bayesian and MAP approaches but are equal in the case of a ML estimation of these parameters.

We now illustrate this result for some particular cases of the pdf family and we use the same notations as in Eq. 25.

In the Bernoulli case, Eq. 40 becomes:

in the Gaussian case,

in the Gamma case,

in the Poisson case,

and in the Rayleigh case,

One can note that the whitening process introduced in the previous section can be also used in order to obtain white random fields well described with Gaussian pdf [13].

4.3

The implementation issue

The window function w^θ that optimizes the criterion ℓ[s|H_θ] realizes the ML optimal segmentation of the target in the scene. The technical problem is thus to find the value of θ which maximizes ℓ[s|H_θ](also denoted l(θ) in the following):

We use a stochastic iterative algorithm to perform the optimization of l(θ) and thereby the segmentation. At each iteration m of the process, carry out the following steps:

This process is continued until l(θ^m) does not increase anymore.

An interesting point is that l(θ^m) can easily be rewritten using correlation operations. Let [f * g]₀ denote the central value of the correlation between f and g:

One thus has:

Note the similarity between Eqs. 48 and 38. The detection, location and segmentation steps require the same correlation functions. A simple optoelectronical architecture could thus perform these tasks with the same hardware. As will be shown in Section 5.5, joint utilization of location and segmentation algorithms enables us to perform efficient target tracking.

4.4

Generalization of the SIR segmentation approach

Constrained deformation

In the previous section, the shape of the snake was not constrained, and could converge to any arbitrary polygon. In some applications, one may have some a priori knowledge about the object’s shape, and thus constrain the evolution of the snake to a smaller class of possible shapes. This enables faster and more robust segmentation.

To formalize this approach, let θ be the node locations and let us consider a set of transformations Κ_α(θ) with α∈ where is the set of possible values of α. In the case of the location task, the transformation is a translation of parameter α and thus:

More interesting can be the case of in-plane rotation R_α since the estimation of the target orientation can be performed with a more efficient technique than the general snake algorithm of the previous subsections. Indeed, instead of randomly moving the nodes one can select among all the rotated versions of w the angle α which maximizes the likelihood:

This concept can be generalized to other transformations such as isotropic or anisotropic scaling.

Recognition

Let us assume that the purpose is to recognize the object or, in other words, to discriminate between different classes. For example, one can imagine that the purpose is to determine whether the target is a car, a truck or a bus. One can now consider that θ belongs to the discrete set Ɗ of the possible classes of objects. For the above considered example one has Ɗ = {car, truck, bus}. The recognition is thus obtained with:

where ℓ[s|H_α] is a nonlinear function of the intercorrelation of s_i and of (s_i)² with the shapes w^α of the reference objects.

5.1

Binary images

The image in figure 4.a represents an object (a bird) appearing against a complex background. Suppose that this image is to be processed with an optical correlator in which the input image is displayed on a binary spatial light modulator. We need to binarize the image before processing it. In many instances, it has been noticed that it was more efficient to edge-enhance an image before binarizing it; this operation increases its contrast, making it easier to find a good threshold. The result of edge-enhancing and binarizing figure 4.a is represented in figure 4.b. Note that in binarized real-world images, the background noise is often non-homogeneous. For this type of images, the MLRT is thus more efficient than the ML algorithm [29],

Figure 4:

(a) : Synthetic scene with an object (eagle) appearing on an urban background. The whole scene is corrupted with white Gaussian additive noise, (b) : Scene edge enhanced with Sobel operator and binarized, (c) : Reference object w. (d) : Result of processing (b) with the MLRT algorithm adapted to Bernoulli statistics. The reference object was (c), and the window M was constructed by dilating (c) two times with a 3 × 3 structuring element.

The result of processing figure 4.b with the MLET algorithm adapted to Bernoulli statistics is shown in Figure 4.d.

By looking more accurately at the binarized image in Figure 4.b, we can see that it is very noisy. This is because a low threshold has been chosen. It is better to have a low threshold since almost all the information-carrying edges are included in the images. Very little information is thus lost, but the drawback is that a lot of spurious edges remain after the binarization step. These edges are in general non-homogeneously distributed over the image, and this makes it important to use location algorithms robust to non-homogeneous background noise, such as the MLET.

Figure 5 represents segmentation results on two binarized real-world images corrupted with additive Gaussian noise. The images in the left column display the initial shape of the snake. The images in the right column represent the snake after convergence. We can see that the searched shape has been correctly segmented. Note that the initial shape does not need to be very close to the true one for the snake to converge properly. This robustness to snake shape initialization is an important feature of the proposed algorithm in real-world applications.

Figure 5:

(a) and (c) : Binarized versions of gray level real-world scenes. The white rectangle represents the initial shape of the snake, (b) and (d) : Snake after convergence.

5.2

Speckled images

Figure 6.a displays two tank-shaped small targets (78 pixels) appearing on a non-homogeneous background with exponential statistics. This background has been generated with the method described in Ref. [29], where we have replaced the Bernoulli variates with exponential variates. The tank in the upper right quadrant has been rotated by 10° with respect to the reference object. The result of processing the scene with the MLET algorithm adapted to speckle statistics appears in Figure 6.b. We can see that the MLET, as most correlation-based algorithms, is robust to small deformations of the target with respect to the reference object.

Figure 6:

(a) : Scene containing the searched object in the middle, and a rotated version (10°) of the searched object in the upper right quadrant. The background is nonhomogeneous with exponential statistics, (b) : Above : Reference object w. Below : Result of processing (a) with the MLRT algorithm adapted to Gamma statistics. The window M was constructed by dilating the reference object three times with a 3 × 3 structuring element.

5.3

Low flux images

Figure 7.a displays a real-world image containing a boat appearing against a mountain background with atmospheric blurring. Figure 7.b represents the same image synthetically perturbed with Poisson noise, simulating for example photon-limited imaging. Figure 7.d shows the result of processing this image with the MLRT algorithm adapted to Poisson noise.

Figure 7:

(a) : Real-world gray level scene, (b); Scene (a) perturbed with Poisson noise (c) : Reference object w. (d) : Result of processing (b) with the MLRT algorithm adapted to Poisson statistics. The reference object was (c), and the window M was constructed by dilating (c) three times with a 3 × 3 structuring element.

Figure 8 also represents a real image synthetically perturbed with some amount of Poisson noise. The car is segmented using the snake energy adapted to Poisson noise.

Figure 8:

(a) : Real-world scene synthetically corrupted with Poisson noise. The white rectangle represents the initial shape of the snake, (b) : Snake after convergence.

5.4

Optronic images

Figure 9.a is a synthetic image representing an airplane on a contrasted urban background. The whole scene is severely blurred. Note that the target gray levels are nonuniform. They are not known a priori, since the only information used by the algorithm is the binary shape displayed in figure 9.c. Fig 9.b represents a whitened version of the scene. It can be shown that the statistics of the gray levels in the whitened scene are approximately uncorrelated and Gaussian. The ML algorithm adapted to white Gaussian statistics is applied to the whitened image. The obtained result is displayed in figure 9.d. We can see that we are able to correclty locate the target despite its low contrast and the severely cluttered background.

Figure 9:

(a) : Synthetic scene with an airplane appearing on an urban background. The whole scene is blurred and corrupted with white Gaussian additive noise, (b); Whitened version of scene (a), (c); Reference object w. (d): Result of processing (b) with the ML algorithm adapted to Gaussian statistics. The reference object was (c).

Figure 10.a also represents an airplane on a contrasted urban background. The whole image is severely blurred. This means that edge-based snake techniques [14, 34, 35, 36] would not be efficient, since the edges between the target and the background are not sharper than the edges internal to the background. The proposed region-based snake method, which relies on all target and background pixels is able to segment the image, as can be seen in figure 10.b. Note that the snake has been applied to the whitened version of figure 10.a.

Figure 10:

(a) and (c) : Synthetic (a) and real-world (c) scenes. The white rectangle represents the initial shape of the snake, (b) and (d) : Snake after convergence (on the whitened version of the corresponding scene)

Figure 10.c represents a real-world image of a car on a road. Here again, the snake is applied to the whitened version of the scene. Note that the the snake has correctly converged although its initial shape (see figure 10.c) was very different from the true one. This is a further proof of the robustness of the proposed algorithm to the initial shape of the snake.

5.5

Applications to tracking

We now illustrate the feasability of using cooperatively location and segmentation approaches to achieve efficient target tracking on image sequences, even if the shape of the target changes during the sequence. Assume that on the image acquired at time t, the target has been segmented using the proposed snake method. This segmentation produces a binary reference shape that enables us to locate the target in the image acquired at time t +1, e.g. with the MLRT algorithm. This is possible since the MLRT is robust to limited deformation of the target with respect to the reference object. It can thus locate the target even if its shape has slightly changed compared to the previous frame. We then use the obtained position estimate for centering the binary reference. This centered reference is used as the initial shape of a snake which has to converge to the new shape of the object. This process is repeated until the end of the sequence. In summary, using jointly MLRT and snake algorithms consists in first determining the object location (which corresponds to very constrained variation of the shape), and then in segmenting the shape whose position is approximately known. In many instances, the shape variations from one image to the next are small, and only few snake iterations are needed to converge to the new shape.

Let is first consider the problem of tracking walking persons. Typical images can be seen in figure 11. We can note that due to the walk, the apparent shape of the person changes during the sequence. In order to get rid of the influence of the structured background, we make an acquisition of the scene without people. We then substract each frame to this reference frame after having registered them. The MLRT algorithm for location and the snake are applied to this difference image. Such a procedure can be useful in surveillance applications for example. We can see in figure 11 that the object is correctly located and segmented in the image sequence.

Figure 11:

Example of tracking on an image sequence. The sequence is composed of 9 frames, and frames n⁰ 1, 4 and 9 are shown. Left column : image with initial snake. This initial snake is the shape segmented in the previous image translated to the estimated object location obtained with the MLRT algorithm. For image n°1, the initial shape is a square. Right column : image with snake after convergence.

The second example consists in tracking a car driving on a highway, and moving away from the camera. Due to this movement, the shape of the target varies during the sequence, and the binary reference used by the location algorithm must be periodically refreshed using the snake segmentation algorithm. We can see in figure 12 the result of applying the proposed tracking method to this sequence. Here again, each frame is substracted to an image of the highway without cars, and the segmentation algorithm is applied to a partially whitened version of this difference image. In order to show the robustness of the proposed algorithm, the snake has been initialized in each image to a square approximately centered on the object. We can see that the car is correctly tracked.

Figure 12:

Example of tracking on an image sequence. The sequence is composed of 9 frames, and frames n° 1, 4 and 9 are shown. Left column : image with initial snake. This initial snake is a square approximately centered on the object. Right column : image with snake after convergence.