1.

INTRODUCTION

Autonomous navigation has become a central technology for space missions with applications varying from rover operations to the descent and landing of spacecraft. In the past decades, traditional 2D passive cameras have been broadly used for this task¹, but the computer vision algorithms necessary for this task take a great deal of CPU and the overall performance depends a lot on the lighting conditions. A 3D camera is a device that enables the perception of depth in images. It has been described as a key component for future space missions involving automatic Guidance, Navigation and Control (GNC) of spacecraft. However, up to now, systems possibilities were limited, because of the non-availability of adequate detectors: due to the low number of available pixels, most existing cameras are based on scanning mechanisms, which introduces reliability issues and limits the frame rate and in consequence the operability in fast motions conditions.

The development of 3D camera technology, capable of directly providing range measurements, is leading to a promising alternative to meet the challenging requirements of space exploration, mainly in terms of reliability, speed and CPU and power consumption. A lot of effort has been dedicated on the evaluation of the most suitable strategy to record the third coordinate and compose the 3D frame. In this context, a Flash LiDAR system is able to record full 3D images with a single laser pulse, thus eliminating the need for high speed scanners, which is a clear advantage for space applications. At present, the only available infra-red Flash LiDAR system is from the United States (Advanced Scientific Concepts) and it is based on the InGaAs technology². Focal Plane Arrays (FPA) based on HgCdTe avalanche photodiodes (APDs) have been shown to exhibit linear gains up to 1000 or more with extremely low excess noise factor and low dark current³. The two main disadvantages of this type of detector is the need to be cooled to near 100K and the cost⁴. Despite these draw backs, HgCdTe (also called MCT) based LiDAR systems present themselves as a competitive alternative for applications with great requirements in terms of gain, noise and reliability.

In this communication, we propose and validate a calibration scheme for a Flash LiDAR system prototype developed by CEA-Leti. The test bench is described as well as the strategy for assessing time-of-flight (3D information of a scene) along with the need of calibration to improve the overall quality of the measurements. The calibration method and detection model are presented as well as the tests for validation under laboratory conditions.

2.

DETECTOR AND READ-OUT CIRCUIT DESIGN

In 2009, CEA-LETI has developed a 320x256 hybrid Focal Plane Array (FPA) for Flash Lidar Imaging. The detector array consists of 30μm pixel pitch HgCdTe APDs operating at 80K and a Read-Out Circuit (ROIC) fabricated on a standard 0.18μm CMOS process. HgCdTe APDs operates in linear mode (LMAPDs) with a high and uniform gain at a low reverse bias and presents an excess noise factor close to one (F≈1). This set of features makes HgCdTe APDs particularly well-suited for applications with expected low number of photons, as it is the case for landing on extraterrestrial bodies, and for integration in 3D FPAs. Complete information about the ROIC architecture and its main specifications can be found in a previous study⁵.

A simplified schematics of the pixel’s architecture is depicted in Figure 1. The operation is based on a capacitive integration amplifier (CTIA). Two feedback capacitors are connected to the CTIA: one having a small capacitance value (C_3D) used for fast pulse detection; and a bigger one used for the purpose of flux integration (C_2D). When the reflected pulse arrives, the current generated by the APD is integrated at C_3D until reaching a threshold level V_th. At this moment, the analog voltage ramp that serves as a time base is sampled and this value is hold as a measurement of the time-of-flight (3D). The bigger capacitance is then connected and the current integration continues, the final voltage level reached is a measurement of the intensity of the reflected pulse (2D)⁶. In this work, “intensity” refers to the measurement of the number of photons.

Figure 1

Simplified schematics of the pixel⁶.

Figure 2 is a representation of the time base used to measure the time-of-flight. At time 0, the laser is fired and the voltage ramp is triggered to start increasing its output voltage level. At time t₀ the reflected pulse reaches the detector, but the system takes until t_th to reach the threshold level V_th. In this prototype, V_th can be set by the operator of the system and can be adjusted in order to reject background and dark noise. The consequence is a delay on the correct instant of sampling that will depend mainly on the threshold level V_th, the gain provided by the APD and the reflected pulse energy. Since in space applications we expect situations with very low number of incoming photons, a calibration scheme that accounts for this distortion is needed.

Figure 2

Representation of the analog voltage ramp (time base). The time need for the pulse to arrive at the detector t_o and it is represented by the gray area and the additional time t_extra need to reach the threshold level at time t_th is represented by the orange area. The saturation level corresponds to the maximum time limit for detection.

3.

CALIBRATION METHOD

As described in Figure 2, the 3D measurement value provided is a linear function of the time needed to reach the established threshold voltage level (t_th), where the linear coefficients “a” and “b” are determined by the time base. This indeed refers to the sum of the actual time-of-flight of the pulse and the extra time needed to integrate the pulse until V_th is reached, as shown in equation (1).

The actual time-of-flight (t_o) is simply a function of the distance to the target (D) according to equation (2), where c is the speed of light.

On the other hand, t_extra can be seen as the time needed to accumulate a threshold number of electrons at the C_3D capacitor, which, in its turn, can be changed in our system through three variables: the established threshold level itself (V_th); the gain provided by the avalanche photodiode; and the reflected pulse energy (P_reflected). Considering the first two items as constant during the landing process for simplicity, t_extra is expected to increase with lower reflected P_reflected. Thus, as a first order approximation, t_extra can be considered proportional to the square of the distance to the target, as presented in equation (3), where “e” is a constant and “R” is the reflectivity of the target.

Finally, the combination of equation (1), (2) and (3) establishes the following quadratic relationship between the sampled value V_3D and the distance to the target (D), where p_1,2,3 are coefficients to be determined via calibration.

Equation (4) has the inconvenience of expressing V_3D as a function of the Reflectivity “R”, which is generally not known with great precision in a real scenario. In order to overcome this ambiguity, an empirical model for the 2D measurement (intensity) was derived as presented in equation (5).

Where q_1,2,3,4 are also coefficients to be determined via calibration. The accuracy of this model is demonstrated in Figure 3 - Intensity measurement with respect to the distance for three targets with different reflectivities. Full lines represent the empirical model used on this work. Figure 3, where the full lines represent the model described by equation 5.

Figure 3

Intensity measurement with respect to the distance for three targets with different reflectivities. Full lines represent the empirical model used on this work.

By applying equations (4) and (5) it is possible to simultaneously resolve the reflectivity of the target “R” and its distance to the detector “D” using the detection measurements V_2D and V_3D. However, this set of nonlinear equations does not have a simple closed form solution. In this work, we use a gradient descent algorithm for finding a minimum of equations (4) and (5) that resolves “D” and “R”.

4.

EXPERIMENTAL SETUP

The laboratory test bench used to evaluate the impact of t_extra on the time-of-flight measurement V_3D is illustrated in Figure 4. A calibrated target with known reflectivity is illuminated by a 1.57μm pulsed laser. The pulse width is 8ns and the energy is 8mJ, resulting in an instant power of 1MW. Thermal and parasitic radiations on the FPA were limited using a cold low-pass filter and a hot band-pass filter. The band-pass filter was centered on the laser wavelength.

Figure 4

Simplified schematic of the experimental setup used for evaluation of the nonlinear relationship of the 3D measurement with respect to the distance as well as for calibration⁶

Three calibrated targets with reflectivity of 5%, 10% and 50% at the laser wavelength were used on the tests. The distance between the LiDAR system and the targets is varied between 4m and 24m, using a 50cm step between each acquisition, while the delay between the emission of the laser pulse and the beginning of the voltage ramp t_p was fixed. The voltage ramp duration was set to 160ns, allowing the observation of objects within a distance of 24m. All this set of measurements was performed with a mean APD gain of 35. A commercial range finder is attached to the imaging system and used to measure the ground truth.

The detection system outputs two images for every laser pulse, one corresponding to the time-of-flight measurement (3D) and another related to intensity measurement (2D), proportional to the accumulated number of photons reflected by the scene. At every position, 50 images of each kind are averaged and a nonuniformity correction (NUC) is applied, along with a bad pixel replacement (BPR) procedure based on the nearest neighbor algorithm. Finally, a single mean value for V_3D and V_2D is calculated by averaging the pixels corresponding to the target in each image.

The result of this procedure is presented in Figure 5, in which the impact of the reflectivity on the 3D measurement becomes clear. In a situation with a high number of photons, as demonstrate with the 50% reflectivity target, the 3D measure could be considered linear with respect to the distance. Ideally, all targets should behave exactly in this way, but as the reflectivity of the target decreases, so the number of photons received, inducing a more nonlinear response of the system.

Figure 5

Uncalibrated 3D measurements of three targets with known reflectivity with respect to the distance evaluated by the range finder.

5.

RESULTS

Figure 6 shows the results after applying the calibration method to the data that have been used for the graph in Figure 5. The method was able to correct the nonlinear distortion for all targets. The root mean square error (RMSE) between 6m and 20m was 21cm. It is noticeable that at short distance (less than 6 meters), the calibrated distance is erroneous for the 50% reflectivity target. This effect is attributed to a noisier detection at a high flux of photons situation, in which multipath reflections may be prominent in the laboratory conditions. Above 20m and 22m, the results for the 5% and 10% reflectivity targets are saturated as expected due to the impossibility of retrieving useful information on those conditions.

Figure 6

Calibrated measured distance of three targets with respect to distance evaluated by the range finder

During a space mission, knowledge of the reflectivity of the target is unlikely. In order to illustrate this situation a simple checkboard of black and white squares is used as a target. The 3D image of the target measured by the LiDAR should be completely uniform, but due to the difference in the number of photons reflected, the white squares appear closer than the black squares (see Figure 7 (a)). After applying the calibration procedure, the result is much more uniform, as it is illustrated in Figure 7 (b). The spatial dispersion of 40cm in the raw image is improved to 18cm when calibration is performed. The accuracy to the ground truth, evaluated using the range finder, was also improved from 120cm to 21cm after calibration.

Figure 7

Application of the calibration model on a checkboard target. (a) Uncalibrated raw data (b) Calibrated data.

6.

CONCLUSION

CEA-Leti has developed a FPA based on HgCdTe APDs that are potentially competitive alternative for Flash LiDAR systems for space applications. The time-of-flight measurement is done by sampling a voltage ramp that serves as a time base at the moment of detection of the reflected laser pulse. This work demonstrated that a detection delay implies on a quadratic relation between the time-of-flight measured and the distance to the target, rather than a linear one. A calibration method that fuses the 2D (intensity) and 3D (time-of-flight) information of a LiDAR system has been developed and tested. The pair of nonlinear equations is solved using the Gauss-Newton algorithm to determine the distance and the reflectivity of the target. The method was capable of accurately compensate the nonlinear impact of the detection delay on a laboratorial trial using calibrated targets with known reflectivity. The RMSE between 6m and 20m for every target on this test was 21cm. The method was also applied to a checkboard target, constituted of black and white squares, leading to an enhancement on precision and accuracy. The prospects for future works include the expansion of the model in order to incorporate the influence of the threshold level and the gain, as well as a test campaign at a real-like scenario on ESA/ESTEC’s premises.