1.

INTRODUCTION

One of the “Key” technology and know-how in sensor systems is the developing of “Smart Sensors” which integrate the sensing function with the signal extraction, processing and “understanding”. This outstanding goal has been pursued with particular effort in application fields such as surveillance and remote sensing, where minimum size and high level multifunction performances were considered as the main achievements to be reached. So the term “Smart Sensors” has been originated to indicate sensing structures capable of gathering in an “intelligent” way and of pre-processing the acquired signal to give aimed and selected information. In a broad sense, they include any sensor system covering the whole electromagnetic spectrum: this paper deals specifically with a new class of Smart Sensors in the infrared spectral bands whose developments started many years ago when the integrated processing capabilities based on advanced read-out, integrated with signal processing, was still far from the complexity needed in advanced IR surveillance and warning systems because of the enormous amount of unwanted signals emitted by operating scenario especially in military applications [1, 2, 3].

Later on, thanks to the CCD read-out technology, it was recognized that the rapid advances of “very large scale integration” (VLSI) processor technology and mosaic infrared detector array technology could be combined to develop new generations of Smart Sensor systems with much improved performances. Therefore sophisticated signal processing operations had been developed in these new systems by integrating microcomputers and other VLSI signal processors within the sensor arrays, on the same focal plane, avoiding complex computing located far away from the sensors.

New objectives and requirements for new focal plane processing have been developed in these new smart sensor systems by introducing inside the sensor itself some of the basic function of living eyes, such as dynamic stare, non-uniformity compensation, spatial and temporal filtering. Historically we can consider two main classes of IR Smart Sensors: the first one oriented to specific, smart applications, in which most of the intelligence of pre-processing is inside the design and structure of the Sensor itself [1-3, 7-9] and the other one, supported by the impressive growth of integrated micro-circuitry which, thanks to the CCD/CMOS integrated readout, using sophisticated pre-processing based on Smart Sensing techniques (these devices known as “vision chips” in the visible range have been later on strongly investigated and successfully developed) [4, 6]. According to the discovery of the Smart Sensor technology, Smart Sensors should in fact integrate technical design and development from optics, detector materials, electronics and algorithms into the sensor’s structure itself, rather than trying to get the required performances by relying on massive improvements in the aspect of the number of pixels and related electronics read-out and processing technology. As a result, the general complexity of the overall device should be much less than the one achievable by just reducing, for instance, the number of sensor pixels.

Therefore the performance of the Smart Sensor can be achieved with lower technological risk and with integrated structure which allows smaller size and higher reliability and often higher performances for specific applications like e.g. surveillance and alarm systems.

One of the most complex and challenging application area for “Smart Sensors” is the Infrared field where the information to be extracted is based on the detection of very small signals buried in electronic noise loaded by a highly diffused background noise and by a flood of intensive “unwanted signals”. The nature of infrared scene with low contrast and high background flux implies that unlike their visible counterparts, infrared imaging devices require some processing of detector output signal to correct non uniformity and remove the background pedestal [3, 5, 9].

Without this on-focal-plane processing, most of the data from the focal plane are useless clutter or unwanted data, because only a few pixels of the whole acquired pattern contain targets information. In conventional approaches this great amount of data were acquired through the read out electronics starting from low-noise small signal amplifiers, then via the analogue to digital converters transferred to digital signal pre-processing, before the final task of separating and rejecting the clutter by complex signal processing. In contrast, the Smart Sensor rejects this clutter before it is read out off the focal plane so that most of the useless data due to IR clutter noise/“unwanted signals” can be rejected before processing. Therefore, focal plane processing in infrared Smart Sensors is end-to-end image and patterns processing completely integrated in the Sensor itself, in the sense that decision for detection is made inside the sensor structure and eventually finalized at a later stage, anyway after the tracking, recognition and other operations have been performed in the early smart sensing stage. Fairly sophisticated signal processing techniques have to be developed to accomplish these objectives: the major signal processing steps consist of the suppression of background clutter and the enhancement of the target-to-noise ratio to a level adequate for threshold. In the classical threshold stage approach, a decision is made for target acquisition by setting a threshold level which accepts a small number of false alarms and target tracking and recognition are performed in the post-threshold stage. In Smart Sensor the target recognition is performed already during the acquiring stage by using feature and shape pattern recognition as threshold activation. In fact, in the pre-threshold stage, target signals are generally deeply buried in background clutter noise and unwanted signals which can be orders of magnitude higher than the target signal intensity. Therefore, imaginative pattern recognition processing techniques using all spatial, temporal and spectral information of both, targets and background clutter, should be developed for suppressing background clutter and unwanted signal, but maintaining or even enhancing the target signal.

1.1

Historical Introduction

“Smart Sensors” is nowadays a quite diffused and even abused term, “Just about everything today in the technology area is a candidate for having the prefix smart added to it” [6]. Moreover, since Smart Sensors have been more and more based of the microelectronics devices integration, the definition generally accepted and reported in most of the specialized literature is: “Generally the term smart sensor has been referred as coined in the mid-1980s and the term “Smart” is attributed to intelligence required by such devices thanks to the integration of microcontroller unit (MCU), digital signal processor (DSP), and application-specific integrated circuit (ASIC) technologies developed by actively working on smarter silicon devices for the input and output sides of the control system. Later the meaning of this successful term was extended to the term micro-electromechanical system (MEMS), used to describe a structure created with semiconductor manufacturing processes for sensors and actuators” [6].

To understand what actually happened not 30 years as reported, but almost 40 years ago, and what is occurring today and what can be forecasted in future for “Smart Sensors”, a brief history of what has really occurred is reported.

First of all, the term “Smart Sensors” was probably created in the middle of 70’s, surely at that times this term was used in Italy as a rough, short translation of “Sensore Intelligente” from Italian language, and was coined for indicating a very advanced IR FPAs as result of a great effort for realizing an optimized, smart technological solution needed for developing reliable, efficient Infrared Surveillance and Alarm Systems for the detection of missiles threats [1, 2, 7, 8]. At those times, around the beginning of 70’, the scenario of IR sensors, mainly for military and high-tech astronomy, was based on the development of IR Imaging Systems with the highest number of sensor pixels, obtained at those times, by opto-mechanical scanning of linear detector arrays. The rapid proliferation of applications of FLIR in military field was evidencing financial problem for development and acquisition process for various system’s development, therefore US was putting a lot of efforts in developing highly sensitive, standard IR linear detectors arrays based on MCT technology and, for achieving a reliable mass producing, the Centre for Night Vision and Electro-Optics developed the design concept for IR Common Modules, based on the so called “first generation” linear arrays of intrinsic MCT photoconductive detectors (60 up to 180 elements). This technological-industrial effort was allowing high performance LWIR forward looking IR imaging (FLIR) systems, operating at 80K with a single stage cryogenic engine and making them much more compact, lighter, and significantly lower in power consumption. In parallel, even if little later, in the middle 70’s, first efforts were done for transferring the CCD technology to IR the FPA (Focal Plane Array) technology especially in the 3 to 5 µm wavelength and later on in 8-12 µm, always with the main task of the highest number of Pixels for high resolution IR imaging [10-12].

In fact, the invention of Charge Coupled Devices (CCDs) in 1969 [10] made it possible to start the developing of the “second generation” IR FPA detector arrays coupled with on-focal-plane electronic analogue signal readouts which could multiplex the signal from a very large array of detectors. In the middle of 70’s US Scientists, first in the world, were developing IR FPA –CCD detectors arrays starting from the 3 to 5 µm bands and extending up to the 8 to 12 µm IR band at the end of 70’s [11, 12]. Most of their attention was anyway dedicated to the developing of high resolution IR Imagers, with the highest number of sensing pixels and the best sensitivity. In 1975 the first CCD TV camera was realized and this was allowing to forecast the “2^nd generation FPAs” capable of a staring vision, although the necessity of very high spatial resolution and high reliability even in complex structures, with extremely high number of pixels (up to one million pixels), were pushing towards alternative solutions, with materials less difficult than CMT, in the manufacturing process (e.g. extrinsic silicon detectors based on Silicon Schottky Diodes) [5].

In the middle of 70’s, in Italy, where the technology in Infrared sensors was also quite advanced, although lower than in USA and UK especially for the lacking of CCD technology [13], a very ambitious Program of developing a low false alarm, real time, Omni-directional IR Missiles Alarm and Warning System was started [14].

As previously indicated, in the Infrared Surveillance and Alarm Systems the information to be extracted is generally based on very small signals buried in highly intensive and diffused background noise and often high intensity “unwanted signals”, this implies that infrared Sensor Systems require some processing of detector signals to correct non-uniformity and remove the background effect and to avoid that. Without this on-focal-plane processing, most of the data would be useless clutter or unwanted data, because of the whole acquired pattern only a few pixels contain targets information of selected targets.[1-3, 14]. At those times the performances of integrated computers were still limited and needing high electrical power and great size and weight, especially if the alarm signals extraction had to treat a very high number of scanned pixel points, like in the case of opto-mechanical scanning system of linear detectors arrays (the apparently one possible solution at those times).

So the problem of signal extraction and successive processing of an impressive mass of data, moreover immersed in high intensity clutter/unwanted signals, was an impressive problem to deal with and successfully resolve.

The solution to this complex problem luckily came thanks to a really “Smart Idea” that was originated by observing the behavior of a Fly in presence of an approaching threat [14]. In fact we realized that approaching a Fly with hands quite slowly, the Fly doesn’t flies away, instead if your hand is approaching fast, the fly escapes away. The explanation was found in scientific papers of the early 70’s, where there was explained that Fly Eye can detect and track small, fast moving targets thanks higher-order neurons within the fly brain ( quite limited in size and performances), known as ‘small target motion detectors’ (STMD), that respond efficiently to moving features, even when the velocity of the target is matched to the background (i.e. with no relative motion cues), often against cluttered moving scenario [16-18]. This detection capability was really illuminating the technological solution for developing a “Smart Sensor”, that based on the fly eye structure could be capable of detecting fast moving, point targets, just as missiles launched and/or approaching from operational distance.

We made therefore the decision of developing a Bidimensional IR FPAs structured like a “Fly Eye” (at those times we used the term “Insect Eye”), moreover this was allowing a great enhancement of the signal-to-noise ratio (thanks to the staring properties of Bidimensional Arrays the signal to noise was improved for the integration-dwell time) and the staring and detection was allowing a temporal filtering analysis in real time rejecting a lot of “unwanted signals [14]. Trusting that Bidimensional Arrays was the winning solutions, we took the decision of developing a new, advanced technology for fabricating l IR FPAs with high responsivity PbTe/PbSnTe thin films [13]. Moreover, due the heavy limit of not having the CCD technology in–house (and even in the whole Italy), we developed an original X-Y addressing capable of reading a bidimensional FPA, up to hundred thousand pixels in less than 1 ms (this technique was patented after experimental field tests) [15]. This addressing system to read a matrix of optical sensors was obtained by an active compensation circuit which supplied a signal equal to the signal originated by all the other sensors addressed in parallel to the sensor pixel to be read (Figure 1).

Figure 1.

X-Y Electronic Addressing of IR FPAs [2, 15].

The term “Smart Sensors” was therefore developed since middle 70’s, almost 40 years ago, for overcoming the great limitations in IR Alarm Warning Systems due to the nature of infrared threats signatures that immersed in scenes with low contrast and high background flux were implying that, unlike their visible counterparts, the requirement of some filtering pre-processing of detectors output signal avoiding conventional approaches process where these useless data were acquired through the read out electronics, the analogue to digital converters, and the digital signal processor before finally separating and rejecting the clutter [1, 2, 7, 14].

The “Smart Sensor” technology was patented at the end of 70’s, after the first successful tests in field trials [7, 8, 14].

So, overcoming all those limitations and complexity, the “Fly Eye Smart Sensors” rejected this clutter before it was read off the focal plane, so that most of the useless data were not processed. These “Smart Sensors” were also allowing to reduce the read out bandwidth, lowering the refrigeration demands of the cooled sensors; limiting the use of high speed electronics and digital signal processors that at the times were quite expensive and more difficult to harden than the lower bandwidth components used in the Smart Sensor. A description of the Smart Sensor rejecting the IR background clutter and the unwanted signals before they were read, so that most of the useless data were not processed is described in the technological development. Since then, several devices have been called “Smart Sensors” as one of the “Key” technologies in FPA, allowing integration of the sensing function with the signal extraction, processing and even “aimed” understanding [7-9, 19-21].

2.

SMART PROCESSING

As mentioned before, one of the most complex and challenging application area for “Smart Sensors” is Infrared Surveillance where the information to be extracted is based on the detection of very small signals buried in electronic noise loaded by a highly diffused background noise and by a flood of intensive “unwanted signals”.

The nature of infrared scene with low contrast and high background flux implies that, unlike their visible counterparts, infrared imaging devices require some processing of detector output signal to correct non uniformity and remove the background pedestal. Successful clutter/unwanted signals suppression is based on the use of adequate discriminating characteristics to identify uniquely the distinguishing aspects of the targets from clutter and unwanted signals. Naturally the use of more discriminating parameters simultaneously operated is strongly effective on the possibility of detecting small target signals even buried within background emission.

So multi-discriminating processing utilizing parallel multi-domain spatial, temporal and spectral filtering is strategic for extracting very weak target signals in presence of strong unwanted disturbing signals emitted from background as in the case of IR Surveillance and Warning systems. Especially properly designed high pass filters in the spatial frequency domain have been developed for suppressing the background/unwanted signals.

Temporal filtering is particularly useful to extract the “moving targets” from the steady or slow-moving background allowing an increase of orders of magnitude in the signal to noise ratio; several temporal filters have been developed for detecting moving targets by extending this basic “frame differencing” approach.

In real world, successive frames are not perfectly registered, consequently background will not be completely cancelled, and therefore more sophisticated temporal filtering must be developed to deal with these real world complications.

In summary, an optimum choice of focal plane processing depends critically on the signatures of “both” the targets and background/unwanted signals and must be tailored to the situation.

Finally enhancement of signal-to-noise ratio is possible by using adaptive filtering, which can be further separated into two groups: deterministic and statistical. In the deterministic group, various filter designs can be been developed based on the knowledge of deterministic background signature. This class of filtering can be further implemented by using a heuristic “background normalization” based on the knowledge of deterministic background signature updated and corrected by the post processor on the “acquired and expectable background signature distribution”.

Detection of very dim targets buried in background clutter hundreds and thousands of times stronger than the target signals is a complex task to be realized in simple classical infrared systems.

Yet conventional approaches process this great amount of data through the read out electronics, the analogue to digital converters, and the digital signal processor before finally separating and rejecting the clutter.

Fairly sophisticated signal processing techniques have to be developed to accomplish these objectives. The major signal processing steps consist of the suppression of background clutter and the enhancement of the target-to-noise ratio to a level adequate for threshold. In the classical threshold stage approach, a decision is made for target acquisition by setting a threshold level which accepts a small number of false alarms. Target tracking and recognition are performed in the post-threshold stage; instead Smart Sensors are performing the target recognition during the acquiring stage or, better, are using feature and shape pattern recognition as threshold activation (Figure 2) [9, 19-21].

Figure 2.

Sensor structure evolution (a) and smart sensor functions (b) [9][19].

2.1

Technology Development

This complex filter and elaboration of the detected signal was a great stimulus for a new technology capable of supplying selective detection of fast moving point targets (such as missiles threats at operational distances) based on the “Smart Idea” of developing a “Smart Sensor” functioning like the “Fly Eye” (at those times we used the term “Insect Eye”). So, as it was explained years later, the target discrimination based on intracellular recordings from neurons in the fly visual system, determining how Flies detect and track small and fast moving features, often against cluttered moving backgrounds, was an intriguing challenge especially considering that the detection is done by higher-order neurons within the fly brain, known as ‘small target motion detectors’, that respond to moving features, even when the velocity of the target is matched to the background (i.e. with no relative motion cues) [22, 23].

Concentrating the description on the time space correlation for feature/movement response in particular for the discrimination of point sources from extended background emissions and/or of fast events (moving targets or changeable emissions) within static or slow moving scenario.

As later reported, a reticulated structured detector, which is electronically modulated to obtain a spatial-temporal correlation of the focused spot target, buried within the diffused background emission, can allow the detection of point source or a well defined shaped target improving the signal to clutter ratio. This dynamic spatial filtering was implemented with special featured structures which were capable of preferable detection for selectable forms (e.g. point sources and/or linearly structured targets). Spectral signature data of targets could be moreover obtained from a dispersive optical system in conjunction with a columnar sub-structure designed to capture this data [7, 9, 19].

Read out processing for clutter rejection was performed in analogue on the focal plane. Since push broom operation allowed the utilization of a more sparsely populated array, there was a space between columns to implement this analogue processing on the focal plane. Example of a reticulum structured multispectral sensor is shown in Figure 3 and Figure 4.

Figure 3.

Feature reticule structured sensor.

Figure 4.

Multispectral reticule structured sensor.

In the 70’s, when most of the advanced investigations on Fly Eye smart detection were far from the deep and advanced investigation exploded after the 90’s, we realized the “Smart Point Target Moving Sensor” by implementing a differential detectors (quite sensitive as a “sensor of zero”), obtained with a three finger electrodes structure (for taking care of the unlucky case that the target could have been positioned just over a finger electrode), and by alternating the bias voltage of each area of IR photoconductive detectors internal to adjacent electrodes. This three electrodes differential structure was repeated within a larger area sensor 8 times, for digital reasons, so that we were able of fabricating a 1024 (32x32) IR FPA’s with an effective spatial resolution equivalent to 1024x8 sub-sensor x3 differential sub-pixels, that is more than 24.000 FPA sensor pixels. Last, but not least the frame rate of 1KHz was matched to the integration time of the sensors (< 1 ms) getting a high Signal to Noise ratio thanks to the long integration time achievable with the staring system, but having enough time resolution (1 ms) for making differential analysis of the recorded signal at each frame time (matched to the temporal response analysis in case of missile launches alarm and warning systems).

Looking inside the realization technology of Fly Eye IR Smart Sensors the main task was to limit or even to substitute the highly complex elaboration of signal output, deriving from the enormous mass of data, coming from the high number of sensing pixels flooded by infrared clutter scenario and unwanted signals emission, by implementing an elaboration filter capability in the sensor structure itself.

This filtering capability associated to an integrated electronic processing could allow implementing correlations in the spectral, temporal and spatial domains, so that it was possible to contain the flux of acquired data extracting only those with higher information content. Examples of such correlations can be exemplified by some well defined signal extractions (Point Source Detection, Edge Enhancement and Morphological Structure Recognition). These correlations associated to appropriate temporal signatures, can allow discriminating and identifying the targets, like it is performed by an insect eye thanks to a spatial-temporal correlation. This electronic integrated modulation is based on the “Insect Eye” structure which allows extracting automatically the signal due to a point source from more intense unwanted signal/clutter due to extended sources (Figure 5).

Figure 5.

Smart Sensor emulating Fly-Eye recognition structure with an integrated 3 electrodes modulation [19, 20, 21].

So, we substituted an external opto-mechanical chopper used for achieving high resolution sensor pixels scanning linear IR arrays by modulating the incoming collimated spot size signal with a sufficiently high resolution (better than 1mrad elementary field of view) bidimensional IR FPAs.

This signal processing was realized by emulating the fly-eye structure with a fingered three electrodes structure which, thanks to an electronic modulation in a differential way between the two adjacent sub-pixels, could detect spot size almost cancelling the signal due to a diffused irradiation source (e.g. clouds and diffused sun irradiation) which are widespread in more than one single subpixel (a third electrode structure was inserted for avoiding the missing of the detection of point target in case that the focused spot was falling just in the middle of two adjacent electrodes). Figure 6 shows the structure of a large area Smart Sensor with 3 phases integrated higher resolution elementary sensors capable of extracting point source and rejecting slow moving extended clutter sources.

Figure 6.

Detection of Point Source Target in presence of high clutter unwanted signals with increased resolution obtained by a 3 differential electrodes: This 3 electrodes structure was repeated 8 times within large area 1024 FPAs sensor elements.

This Smart Sensors technology was kept restricted within a close military cultural environment for more than 15 years and exploded in applications and performances in the 90’s years, mainly thanks to the impressive improvements in the integrated signal read-out and processing achieved by CCD-CMOS integrated microcircuit technologies in FPAs.

In fact at the end of 80’s, thanks to the CCD read-out technology, it was recognized that the rapid advances of “very large scale integration” (VLSI) processor technology and mosaic detector array technology could be combined to develop new generations of Smart Sensor systems with much improved performances. Therefore, sophisticated signal processing operations have been developed in these new systems by integrating microcomputers and other VLSI signal processors, within or next to the sensor arrays on the same focal plane avoiding complex computing, located far away from the sensors. This great performances growth was particularly evident in the visible range with development of “Artificial Human Eye”, where new objectives and requirements for advanced focal plane processing have been developed by introducing inside the sensor itself some of the basic function of living eyes, such as dynamic stare, non-homogeneity compensation, spatial and temporal filtering[4, 23].

Fig. 7 describes the Block diagram of the Smart Sensor electronically modulated. The Block A has the two fundamental functions:

• 1) Control of Block C (by the Bias-Set) and the Output (V_Out)

• 2) Command Signals (V_Control) to Block B for generating the voltage biasing

Figure 7.

Block Scheme for improving single sub-pixel element resolution in large area signal extraction.

The Block B has the function to generate the biasing voltages (V_Bias (t)) with the desired form and frequency. These Voltages are supplied by Block C which is connecting the Signal Output (V_Out) with the electrodes (e₁, e₂, …, e_n).

We can observe that the form of electrodes can be any and is defined only by the desired spatial frequency and that the electrodes configuration of the bias V₁(t) e V₂(t.) and of the output V_out, should be alternate.

Moreover the electrodes configuration and the Bias V₁ (t), V₂ (t) should be related by:

and

where

and m is the number of elementary cells defined by single electrodes, n₁ the number of elementary cells is the sensor area defined between the electrode V₁ (t) and the output, n₂ the number of elementary cells is the sensor area defined between the electrode V₂ (t) and the output, d is the distance between the first and the last one, k is an integer number.

This dynamic spatial filtering can be implemented with a special feature structure which is capable of preferable detection for selectable forms (e.g. point sources, linearly structured objects, etc.)

2.2

Experimental Results and Structures of Smart Sensor IR FPAs and IR Warning Systems

Figure 8.

IR FPAs (1024 sensors, each one with 3x8 sub-pixels) Elettronica Spa Rome –Italy 1976.

Figure 9.

(a) IRWR TARSIO (Tactical Infrared System Omnidirectional) MLAW Airborne Warning and Alarm System (b) CERBERO Infrared Surveillance Alarm and Warning System 2π Navy System, Elettronica S p a Rome –Italy (1980).

Figure 10.

(a) Point Source Target Signal Immersed in High Intensity Clutter [7] and (b) Point Source Target Signal Enhanced by Smart Sensor Integrated Extraction [7].

2.2.1

Microbolometer Smart Sensors

Successful developments achieved in most of IR FPA technologies were showing the importance of monolithic approach, unfortunately strongly limited by spectral range and cooling requirements in the case of Silicon Schottky diodes, but the recent successful results obtained in silicon microbolometers is overcoming both limitations. Especially thanks the complete integrability with electronics readout microbolometers array appear to be a good choice due to its high spatial uniformity and, being a silicon based technology, can allow large, high yield focal plane arrays, economically fabricated.

Particularly interesting is the case of Large Area Microbolometers for applications where high sensitivity coupled to simplicity and low cost of the IR arrays are required (e.g. automotive, large scale distributed collision avoidance systems). In this case the integrated modulation above reported is even more cost/effective, but important physical phenomena conditioning the signal detection have to be taken in account. The most important phenomenon is the signal crosstalk due to thermal diffusion of the absorbed IR radiation which is interacting with the signal modulation according to the following considerations and equations. In fact, when considering Infrared Smart Sensors based on microbolometers, technology some further physical phenomena coupled to complex thermodynamic problems are to be considered and evaluated.

The main one is the signal crosstalk between closely adjacent sensor pixels which moreover is strongly dependent on the size and shape of incoming radiation. Various incident forms of IR radiation have been evaluated by theoretical modelling and experimental test measurements with interesting results.

Three cases of IR incident radiation have been studied:

• - radiation extended all over the sensing area, almost constant in all the large area sensor pixels

• - radiation with an irradiated area just close to the area of a single sensing pixel

• - radiation confined to a spot incident in the centre of a large area sensing pixel

It is clear that for using the smart concept previously described it is necessary to operate the signal extraction by introducing adequate corrections formulas. Simulation and experimental studies have shown that the smart elaboration based on differential amplifying between any sensor pixel and its adjacent ones allows discriminating point source target from diffused irradiation even in presence of a sensible thermal crosstalk.

Average temperature gradients distribution for the array in air is shown in Figure 11.

Figure 11.

Simulation data for the array in air.

It is evident that the cross talk between each elementary sensing pixel and its adjacent one, that is about 30% in the case of an irradiated signal diffused all over the pixel and order of magnitude less in the case of a focused point target irradiation thanks to the differential signal extraction obtained by the smart sensing structure is possible to obtain a consistent lowering the false alarm rate enhancing the signal-to-noise ratio.

3.

APPLICATIONS

An important application of the IR Smart Sensors above described is a car system for “Collision avoidance in low visibility”. In fact thanks to the better visibility trough fog in IR field in respect to visible many systems proposals have been done by car producers for the use of thermal viewers to be installed on board. These IR systems, up to now, have shown heavy limits, even if using the IR room temperature microbolometers, for their cost, for maintenance and reliability and over all for the requirements of “human interface” (it is evident that few car drivers can use an helmet type display or can have enough skill to look at a display while driving in very low visibility). For these reasons a new generation of simple, reliable, Smart Sensors operating at room temperature with no costly thermo-stabilization, which supply a sound and light alarm in case of presence of an obstacle on the road could be a winning solution. A schematic view of the system for collision avoidance is shown in Figure 12.

Figure 12.

Block Scheme for improving single sub-pixel element resolution in large area signal extraction.

In Figure 13(a) is shown a high resolution thermal image of two possible obstacles in a winter environment: a car parked for more than 10 minutes and a running car. In Figure13 (b) is shown the signal detected by the IR Smart Sensor structure supplying automatic alarm in both cases.

Figure 13.

Thermal Imaging (a) and Smart Sensor Signal (b).

4.

FUTURE SMART SENSORS

Forecasting the future it is often a gamble game unless we stay learning lessons from the past and looking at the present: in both cases it is sure that Smart Sensing will maintain an impressive growth and not only because of the abusing the “Smart” terminology, but thanks either to the impressive improvements in consolidate Solid State Very Large Integration Microcircuits VLSI including MEMs Technology, either to the new emerging Sciences and Technologies (Nanotechnology) especially in Security and Biomedical applications. The current value of the Smart Sensor market has been estimated about 65 million devices, but it is expected to reach 2.8 trillion devices by 2019. Smart Sensors market may rise up to 10 percent a year and is forecast to reach 6.0 billion US $ by the year 2015 although in presence of a sensitive reduction of cost thanks to micro-technology and nanotechnology [24].

Moreover the successful results in performances/cost and the great returns in environment, energy, transportation, telecommunication, security and biomedicine have pushed a lot of investments in important system applications (e.g. smart cities, smart buildings, smart water resources, etc..) where “intelligent sensors” will select the most relevant data and, rather than just passing collected information, such sensors will monitor and evaluate the information’s quality performing advanced analysis and elaborations allowing urban and territorial planning.

Close to classical “Smart Sensors” there will be a great development of “Smart Dust”, that is a “Cloud of Sensors”, with micro size dimensions distributed on large area and connected by wireless networks that allow data transmission to central computer systems. The forecasting of an explosion of Smart Dust thanks to the science of building molecule-size electronic devices via nanotechnology, in realty has not yet maintained the performances promised after more than a decade of work: in fact smart dust networks haven’t yet reached their promise as a technology that will revolutionize medicine, security, space exploration and more, even if recently “smart dust sensors” appear to be closer to the reality stage especially on the ITC Networks side for monitoring systems. Anyway, such wireless tracking sensors have achieved cost performances being quite cheap (few tens of dollars each).

These new sensor technologies, components systems and signal processing will be integrated into so-called Smart Grids/Intelligent networks, in most applications (energy, bio-medicine, transport, environment, etc) and sensing technologies (photonic, electronic, magnetic, biological, and optoelectronic, etc) measuring, diagnosing or imaging various parameters, physical, chemical, biological, medical and so on.

“Smart Dust NanoSensors” are promising to be revolutionary because the sensors are small enough to be put anywhere and work wirelessly, sharing data. Generally Smart Dust is based on microelectromechanical systems, or MEMs, which connected through tiny microcomputer chips can measure temperatures, vibrations or surface pressures triggering an automatic action, such as e.g. controlling a building’s temperature or controlling the flow of oil flux in a pipeline [24, 25]

Besides this advanced integrated forms of Smart Sensing, mainly based on the integration of new, advanced technologies (microelectronics, MEMs, Nanotechnology) into so-called Smart Grids and intelligent networks, it is reasonable to forecast a great growth of “Smart Sensors” originated by their aimed specific intelligence, that is something similar to the first original Smart Sensors ( i.e development of “Task Aimed Intelligent Sensors”, which are Smart Sensors designed and developed for specific applications), even if improved in their performances by using advanced technologies, up to now Microelectronics/MEMs and in the future mainly Nanotechnology. Nanotechnology in fact can enable functional materials, devices, and systems by controlling matter at the atomic and molecular scales, and might allow sensing and interactive actions exploiting novel properties and phenomena by considering that most physical, chemical and biological sensors depend on interactions occurring at these levels Operating on the scale of atoms and molecules, emerging nanotechnologies promise dramatic changes in sensor designs and capabilities [26].

Therefore, Nanotechnology has to be evaluated not simply as the trend toward the smallest that started with the miniaturization realized by microtechnology and that allowed better specificity in smaller, smarter, and less costly sensors, thanks to the use of ICs, fibre optics, micro-optics and MEMS technologies. Nanomaterials and nanostructures are really promising application areas that can offer two functions, especially for chemicals and biological substances, which are recognition of the molecule or other object of interest and transduction of that recognition event into a useful signal for activating a reaction (e.g. therapy in biomedicine). In fact Nanosensors can be really smart if designed and realized for specific applications such as detection of chemicals and biologics molecules, if placed in blood cells, even as nanoshells that interacting with a specific selected molecules can detect and destroy tumours [27, 28]. Most important examples will be probably in Bio-Medicine Diagnostics and Therapy (e.g. Reconfigurable DNA sensors can be programmed to perform complex operations, including medical diagnostics and controlled drug delivery: in fact these sensors could penetrate living cells and detect targets with high sensitivity and selectivity. In theory, the sensors could also store, deliver and release drugs in a controlled manner [29]. In realty, although the excitement over nanotechnology and its prospective uses is generally well founded, the development and integration of nanosensors must take into account the realities imposed by physics, chemistry, biology, engineering and commerce. For example, great limitations arise when nanotechnologies are integrated into macro-sized systems with the need of providing and controlling the flow of matter, energy and information between the nano and macro scales. So, many of the design considerations for nanosensors are similar to those for microsensors, notably interface requirements, heat dissipation, and the need to deal with interference and noise, both electrical and mechanical. Each interface in a microsystem is subject to unwanted transmission of electrical, mechanical, thermal, and, possibly, chemical, acoustical, and optical fluxes. Flow control is especially critical in chemical and biological sensors into which gaseous or liquid analytes are brought and from which they are expelled. Furthermore, the very sensitive, tailored surfaces of these sensors are generally degraded from the effects of foreign substances, heat, and cold. Generally, these limits are overcome by installing higher number of nano-sensors, thanks to their really infinitely small size and contained cost, allowing malfunctioning devices to be ignored in favour of good ones, thus a getting successful system’s work with prolonged useful lifetime [30].

5.

CONCLUSIONS

In conclusion we can distinguish two main classes of IR Smart Sensors: the first one supported by the impressive growth of integrated micro-circuitry which, thanks to the CCD/CMOS integrated readout, can allow sophisticated pre-processing using the Smart Sensing techniques (these devices known as “vision chips” in the visible range have been recently strongly investigated and successfully developed ) and the other one, more oriented to specific low cost smart applications, in which most of the intelligence of pre-processing is already integrated in the design for the specific application of the sensor itself.

The new approach of the Smart Sensors with filtering and processing capabilities integrated in the sensor structure has revolutionised development trends of FPAs IR sensors. In particular, two important technological tasks were achieved:

The new silicon microbolometers technology actually is the solution with major market possibility, thanks to the high number of pixels and the high uniformity in sensitivity, with real capability of achieving the level of performances already achieved in the visible (e.g. Smart Sensors with a completely integrability with signal read-out and processing). Moreover, thanks to their capability of being structured with complex form within their large area, completely integrable with silicon microcircuits with the convenience of the room temperature operability, in the short medium term a great civil market will be generated by IR Smart Sensors technology. The number of pixels of IR FPA, which should be as large as possible, shall be limited by the processing technology and by the size of each sensitive pixel (which can be different for various usable sensitive materials, but it is anyway limited by optical diffraction-Airy diffraction limit). The total number of pixels therefore, considering the resolving power of the collimating optics and the overall diffraction limit, should be limited to ten millions in the F.I.R. [5].

Future front-end processes shall be more dedicated to the development of integrated electronics and in medium-long term to advanced processing (Smart Sensors). Therefore a most important aspect for future IR sensors shall be the real compatibility and integration with silicon microelectronics. Front end processes don’t foresee specific limitations, but surely great attention will be given to Microsystems technology, especially for three dimensional structures. This is particularly true for microbolometers which will be more and more three dimensional microsystems devices (fil factor should be close to1).Process integration, devices and structures, will enhance the importance of chip integration and assembly especially due to the heavy weight in thermal dissipation of electrical output feedthroughs and connecting wires. Cooling requirements for the photon noise reduction or band gap freeze-out cleaning will be greatly eliminated, but thermal problems arising from the great number of working pixels and, even more of integrated electronic processing, should be not underestimated. Design, modelling and simulation should be diffused and widespread in every IR laboratory, possibly with theoretical models and designing rules accessible on WEB network.

Future growth of IR FPAs will be more and more dependent on silicon microelectronics know-how and manufacturing, with the most evident result of a wide use of silicon microbolometers especially in civil applications, while an increasing strategic value will have the integration of signal enhancement and processing within the sensors (Smart Sensors).

Lastly, the explosion of nanotechnology although limited in its applications from the development of efficient nano-micro-macro interfaces ( probably solved in the near future by excluding physical interfaces thanks to electromagnetic waves ( microwaves, THz, electro-optical ) connections), will allow to develop unimaginable developments especially in Security and Biomedicine ( e.g. Smart Sensors, even in Dust Powder form, for gas, chemical, pathogen and pollutant detection, graphene and carbon nanotubes based sensors; Biological selective detection and identification, which allow diagnostics and nano-therapy actions, by the application of biological sensing; principles ).