An objective measurement framework for signal-level image fusion performance, based on a direct comparison of visual information in the fused and input images, is proposed. The aim is to model and predict subjective fusion performance results otherwise obtained through extremely time- and resource-consuming perceptual evaluation procedures. The measure associates visual information with edge, or gradient, information that is initially parametrized at all locations of the inputs and the fused image. A perceptual-information preservation model is then used to quantify the success of information fusion as the accuracy with which local gradient information is transferred from the inputs to the fused image. By considering the perceptual importance of different image regions, such local fusion success estimates are integrated into a single, numerical fusion performance score between 0 (total information loss) and 1 (ideal fusion). The proposed metric is optimized and validated using extensive subjective test results and validation procedures. The results clearly indicate that the proposed metric is perceptually meaningful in that it corresponds well with the results of perceptual fusion evaluation. Finally, an application of the proposed evaluation approach to fusion algorithm selection and fusion parameter optimization demonstrates its general usefulness.
Signal-level image fusion has in recent years established itself as a useful tool for dealing with vast amounts of image data obtained by disparate sensors. In many modern multisensor systems, fusion algorithms significantly reduce the amount of raw data that needs to be presented or processed without loss of information content as well as provide an effective way of information integrations. One of the most useful and widespread applications of signal-level image fusion is for display purposes. Fused images provide the observer with a more reliable and more complete representation of the scene than would be obtained through single sensor display configurations. In recent years, a plethora of algorithms that deal with problem of fusion for display has been proposed. However, almost all are based on relatively basic processing techniques and do not consider information from higher levels of abstraction. As some recent studies have shown this does not always satisfy the complex demands of a human observer and a more subjectively meaningful approach is required. This paper presents a fusion framework based on the idea that subjectively relevant fusion could be achieved in information at higher levels of abstraction such as image edges and image segment boundaries are used to guide the basic signal-level fusion process. Fusion of processed, higher level information to form a blueprint for fusion at signal level and fusion of information from multiple levels of extraction into a single fusion engine are both considered. When tested on two conventional signal-level fusion methodologies, such multi-level fusion structures eliminated undesirable effects such as a fusion artifacts and loss of visually vital information that compromise their usefulness. Images produced by inclusion of multi-level information in the fusion process are clearer and of generally better quality than those obtained through conventional low-level fusion. This is verified through subjective evaluation and established objective fusion performance metrics.
A number of pixel level image fusion schemes have been proposed in the past which combine registered input sensor images into a single fused output image. The two general objectives that underpin the operations of these schemes are a) the transfer of all visually important information form input images into a fused image and b) the minimization of undesirable distortions and artifacts which may be generated in the fused image. Fusion is usually achieved by i) the decomposition of input images into representations of their spectral bands and ii) a selection process which transfers information from input bands to yield the required representation of a single fused output image. Furthermore, decomposition is often based on multi-resolution pyramidal representations and the selection process operates on corresponding input image pyramidal levels using selection templates which focus on local spectral characteristics. The performance of such a multi-resolution pixel level image fusion system depends primarily on the actual decomposition and selection algorithms used. Thus for a given decomposition selection arrangement, fusion performance is dependent on the pyramid size (i.e. number of level) and template size. Pyramid and template sizes on the other hand greatly influence the system's computational complexity. This paper is concerned with the performance optimization/characterization of several multi- resolution image fusion schemes, in general and with performance/ complexity trade-offs in particular. Performance is measured using a subjectively meaningful, objective fusion metric which has been proposed recently by authors and which is based on the preservation of image edge information. Thus fusion systems based on derivatives of Gaussian low-pass pyramid and the Discrete Wavelet transform are examined and their performances versus decomposition/selection parameters are defined and compared. The performance/algorithmic complexity results presented for these multi-resolution fusion systems highlight clearly their strengths and weaknesses.
This paper addresses the issue of objectively measuring the performance of pixel level image fusion systems. The proposed fusion performance metric models the accuracy with which visual information is transferred from the input images to the fused image. Experimental results clearly indicate that the metric is perceptually meaningful.
The work described in this paper focuses on cross band pixel selection as applied to pixel level multi-resolution image fusion. In addition, multi-resolution analysis and synthesis is realized via QMF sub-band decomposition techniques. Thus cross-band pixel selection is considered with the aim of reducing the contrast and structural distortion image artifacts produced by existing wavelet based, pixel level, image fusion schemes. Preliminary subjective image fusion results demonstrate clearly the advantage which the proposed cross-band selection technique offers, when compared to conventional area based pixel selection.
Conference Committee Involvement (1)
Multisensor, Multisource Information Fusion: Architectures, Algorithms, and Applications
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.