The next generation of automotive Night Vision Enhancement systems offers automatic pedestrian recognition with a
performance beyond current Night Vision systems at a lower cost. This will allow high market penetration, covering the
luxury as well as compact car segments.
Improved performance can be achieved by fusing a Far Infrared (FIR) sensor with a Near Infrared (NIR) sensor.
However, fusing with today's FIR systems will be too costly to get a high market penetration. The main cost drivers of
the FIR system are its resolution and its sensitivity. Sensor cost is largely determined by sensor die size. Fewer and
smaller pixels will reduce die size but also resolution and sensitivity. Sensitivity limits are mainly determined by
inclement weather performance. Sensitivity requirements should be matched to the possibilities of low cost FIR optics,
especially implications of molding of highly complex optical surfaces. As a FIR sensor specified for fusion can have
lower resolution as well as lower sensitivity, fusing FIR and NIR can solve performance and cost problems.
To allow compensation of FIR-sensor degradation on the pedestrian detection capabilities, a fusion approach called
MultiSensorBoosting is presented that produces a classifier holding highly discriminative sub-pixel features from both
sensors at once. The algorithm is applied on data with different resolution and on data obtained from cameras with
varying optics to incorporate various sensor sensitivities. As it is not feasible to record representative data with all
different sensor configurations, transformation routines on existing high resolution data recorded with high sensitivity
cameras are investigated in order to determine the effects of lower resolution and lower sensitivity to the overall
detection performance. This paper also gives an overview of the first results showing that a reduction of FIR sensor
resolution can be compensated using fusion techniques and a reduction of sensitivity can be compensated.