1.

Introduction

In this paper, we introduced a suitable metric to estimate the reconstruction accuracy of an optical three-dimensional (3-D) measurement system used under the influence of a refractive index gradient. Such conditions may be caused by the heat transfer from a hot measurement object into the ambient air, which leads to a deflection of the measurement light from its assumed linear path. This light deflection effect is an additional source of uncertainty when measuring the 3-D geometry of a hot-state object. The uncertainty estimation cannot be based on the comparison of the measured data with the calibrated geometry of a measurement standard, as is regular practice, since the calibrated features correspond to precisely the calibration at defined conditions, usually ambient pressure $p=101325\mathrm{Pa}$ and temperature $T=293\mathrm{K}$. Also, the geometry of any component at hot-state cannot be measured by a tactile coordinate measurement machine since it cannot be excluded that the elevated temperatures have damaging effects on the tip or may cause conditions for which the system is no longer calibrated.

Members of this research group have tried to estimate the added uncertainty by analyzing the acquired data by comparing it with a mathematical model¹ and have conducted a multiphysics simulation² to estimate the effect of the light deflection on a simulated 3-D geometry measurement. In addition, we have measured hot objects in different ambient pressure situations³ and from different angles⁴ to try to estimate metrics to evaluate the reconstruction quality. An indirect metric was found in the estimation of the inhomogeneous refractive index field induced by the heat transfer of the hot object into the ambient air using the background-oriented schlieren method.^5,6 Nevertheless, a direct proof and estimation of the influence of the light deflection effect was not conducted.

Other groups have also developed 3-D measurement techniques for hot-state components, but they omitted a reconstruction quality estimation or based it on the predicted geometry of a heated component. The geometry prediction was based on a combination of the measured object temperature, the thermal expansion coefficient of the components material, and the measured geometry of the component at room temperature. Standard quality estimation methods for optical 3-D geometry measurement systems, e.g., the backprojection error from Hartley and Zisserman,⁷ were not applied. The results from the literature are recapitulated in the following to give an impression of the objectives of other researchers and the achieved uncertainties and methods to estimate those uncertainties.

Liu et al.⁸ used a line projection device combined with a two-camera stereo vision setup to calculate the diameter of a hot cylindrical forged part. The radiation from self-emission of the hot object was blocked by a lowpass filter. There was no hot reference object available for the experiment, wherefore the accuracy was not estimated. Their proposed solution was based on the calculation of the fundamental matrix from the corresponding pixel points⁷ and the matching of points to the epipolar constraint. Their second validation approach was based on a heating simulation and the expected expansion of the measurand. Here, their results match with an error of $\sim 0.7\%$. Zhang et al.⁹ used a laser light section method for the geometry reconstruction of cylindrical and rectangular hot heavy forged parts. The main research topic was the investigation of movement of the measurement system and the width of the laser line projection. An accuracy estimation was not conducted. Zatočilová et al.¹⁰ developed a stereo vision system for rotationally symmetric forgings under high temperature. However, since they were not able to identify homologous points in both cameras, they opted to reconstruct the hull of the object by fitting two perperdicular planes in the images. They estimated that their possible accuracy was in the millimeter range.

Schöch et al.^11,12 developed a holistic geometry measurement system for arbitrary component shapes based on multiple light section scanners and a linear motion platform. They estimated the accuracy of their system at room temperature to be around 0.15 mm for a diameter measurement of 300 mm. For validation under hot conditions, they measured low-expansion workpieces (lengths $l=\{70\mathrm{mm},130\mathrm{mm}\}$ with a thermal expansion coefficient of $0.63\times {10}^{-6}{\mathrm{K}}^{-1}$) at about 850°C. While the reconstructed object lengths were within the expected range, there was no explicit statement of the achieved accuracy or the influence of the light deflection effect by the inhomogeneous refractive index field. They additionally estimated the theoretical maximum influence of the light deflection effect on light-section measurements to be about $30\mu \mathrm{m}$ for objects at a temperature of 100°C. Their calculation was based on the light deflection caused by a homogeneous refractive index field of a slightly different refractive index outside the sensor housing. They concluded that the light deflection effect can be neglected due to its diminishing importance compared to the size of the measurand of about 300 mm in diameter.

Du et al.¹³ used a two-dimensional (2-D) laser radar, comprising a single laser beam and two servo-motors, to measure the geometry of cylindrical hot heavy forged parts. The measurement error was estimated to about 2 mm for a cylindrical reference object of $450\mathrm{mm}\times 275\mathrm{mm}$ ($\text{height}\times \text{diameter}$) at room temperature. For the same object at hot state, they estimated the error to be 5 mm for the diameter and 8 mm for the height. The single point error was not investigated, and the reference method was not stated. Zhang et al.¹⁴ combined a light section system with reflection models to reconstruct the edges of a hot component. This simplifies the evaluation of the acquired 3-D data to calculate the length of an object. The accuracy of the method was estimated by comparing the reconstructed the geometry of the cold object with a reference measurement; therefore, neither a single point accuracy nor a hot reference method was given. Hawryluk and Dworzak et al.^15,16 used the combination of a light-section laser-scanner with a measuring arm, which is a six-axis serial arm reconstructing the 3-D position of its tip by measuring its joint angles. The system was used to reconstruct the geometry of the tools used in a hot-forming process. A wear analysis of the tools was conducted based on the acquired geometry. In our opinion, the wear thus described is not sufficiently examined in terms of its plausibility due to uncertainties caused by inhomogeneous ambient conditions or inherent uncertainties. Bračun et al.¹⁷ used a laser-scanner to reconstruct the geometry of a hot specimen. They focused on the compensation of different lighting conditions due to inhomogeneous emission coefficients on the object’s surface as well as the measurement accuracy. They identified scale chips on the object’s surface and the surface mircotopography as the most significant influences. Zhou et al.¹⁸ used a stereo vision system in combination with a line projector. The main research was focused on the line detection along measurement object edges in the stereoscopic image pairs. The detected lines were used to reconstruct the object’s geometry edges through epipolar geometry.⁷ Zhang et al.¹⁹ introduced a laser scanner-based shape reconstruction method for large ring forgings. The main aspects of their approach were given by data fusion of consecutive measurements from different sensors to the holistic geometry of the component. For the validation of their data, they ompared the measurement results to the components radius, which was measured by a caliper. The estimated errors were within 1.2 mm for a radius of about 450 mm.

2.

Basics of Optical 3-D Geometry Measurement

Optical 3-D geometry measurement techniques are state of the art when it comes to contactless quality control for the shape of technical components, especially for free-form objects.²⁰ The mapping of an arbitrary 3-D point $\mathbf{x}={(x,y,z)}^T$ onto a pixel location ${\mathbf{u}}_b={(u_b,v_b)}^T$ on the sensor of a camera is described by the perspective projection of the pinhole camera model, according to

The optical measurement of a 3-D geometry therefore needs to compensate for the loss of dimensions, e.g., through the use of two or more cameras observing the same scene. The reconstruction is then based on the triangulation of homologous 2-D points in multiple cameras. A successful reconstruction needs the satisfaction of Eq. (1) for each corresponding point; therefore, the identification of the system-inherit parameters [${\mathbf{K}}_i$, ${(\mathbf{Rt})}_i$ for each camera $i$] and the detection of the homologous points are necessary.

A calibration routine using a known target is used to compute those camera parameters, e.g., the one developed by Zhang.²¹ The reconstruction itself is then based on the intersection of two lines from two cameras $i,j$, resulting in a reconstructed 3-D point ${\mathbf{x}}^{\prime }({\mathbf{u}}_{c,i},{\mathbf{u}}_{c,j})$. Since these lines in 3-D space are usually skewed due to a leftover calibration error or numerical uncertainties, different algorithms were developed to reconstruct the 3-D point. A popular method is an optimization based on the epipolar geometry.⁷

However, homologous points may not be detectable on measurement objects without a structured surface. An active triangulation system, which incorporates an additional illumination unit, is used to counter this. One approach uses different projection images to add an artificial structure to the surface, e.g., by projecting a pseudorandom pattern to create homologous points. Such structured surface can then be reconstructed using the described triangulation technique with multiple cameras. A different approach uses a coded projection sequence to retrieve the corresponding projector pixel ${\mathbf{u}}_p$ in a given camera pixel location ${\mathbf{u}}_p({\mathbf{u}}_c)$, e.g., the multifrequency phase-shift method developed by Peng et al.²² The projector is modeled here as an inverse camera to then reconstruct 3-D-points from the given pixel values of camera $c$ and projector $p$, i.e., ${\mathbf{x}}^{\prime }({\mathbf{u}}_p,{\mathbf{u}}_c)$, using the described triangulation techniques.

For the researched metric to estimate the reconstruction accuracy of an optical 3-D geometry measurement technique, in our case a fringe projection system, it is necessary to identify corresponding points in a setup comprising one projector and multiple cameras. To this means, Bräuer-Burchard et al.²³ and Reich et al.²⁴ investigated the additional knowledge gained from a system with more than one camera $n_c>1$ and a single projector $n_p=1$. They developed an algorithm to change the basis of the calculation from the camera sensor to the virtual projector sensor ${\mathbf{u}}_p({\mathbf{u}}_c)\to {\mathbf{u}}_c({\mathbf{u}}_p)$. Using this method unlocks additional stereo pairs for triangulation by finding the corresponding camera pixels, i.e., not only reconstructing the 3-D points ${\mathbf{x}}_i^{\prime }=f({\mathbf{u}}_p,{\mathbf{u}}_{c,i})$ but also using ${\mathbf{x}}_{i,j}^{\prime }=f({\mathbf{u}}_{c,i},{\mathbf{u}}_{c,j})$. There are $m=(n-1)!$ stereo pairs in such a system with $n=n_p+n_c$ optical devices. Therefore, there are $m$ reconstructed 3-D points ${\mathbf{x}}_{i,j}^{\prime }({\mathbf{u}}_p)$ for each projector pixel. To benefit from the fractional projector pixel locations retrieved by the phase-shift algorithm, they developed a method to arbitrarily scale the projector resolution. The main advantages of this method are the optimization of the point cloud density by changing the projector resolution in a virtual raster (VR).

3.

Proposed Method

First, a simplified model of the influence of refractive index fields on the reconstruction of 3-D points is described. The model is based on the compression of an inhomogeneous refractive index field to a medium containing two refractive indices with a discreet and sharp boundary layer. The assumptions are used to explain the background of the proposed method. Based on this, the presented method is brought into relation with the known backprojection error and the deviation of corresponding object points.

3.1.

Theoretical Background

Suppose a ray of light, traveling through a medium with refractive index $n_1$ and a medium with $n_2$, is subject to refraction at the boundary interface between $n_1$ and $n_2$. The angle of refraction $\beta$ is calculated from the incident angle $\alpha$ and the quotient for refractive indices $\frac{n_1}{n_2}$, according to

Eq. (2)

$$\sin \beta =\frac{n_1}{n_2}\sin \alpha .$$

Suppose the refraction is caused by a discreet and plane parallel medium of thickness $d_w$; the length $l_r$ of the refracted ray in $n_2$ is

Eq. (3)

$$l_r=d_w/\cos \beta .$$

The refracted ray of light is shifted parallel by $p$ compared with the virtual nonrefracted ray, according to

Eq. (4)

$$p=l_r\sin (\alpha -\beta ).$$

Let us suppose the ray of light is connecting one arbitrary 3-D point ${\mathbf{x}}_k$ and the projection center of a camera ${\mathbf{c}}_i$ (see Fig. 1). The view ray ${\mathbf{a}}_{k,i}$ for the camera with center ${\mathbf{c}}_i$ then connects the camera center and the first intersection of the ray of light with the interface layer ${\mathbf{d}}_{k,i}$, according to

Eq. (5)

$${\mathbf{a}}_{k,i}=\frac{{\mathbf{d}}_k-{\mathbf{c}}_i}{{\Vert {\mathbf{d}}_k-{\mathbf{c}}_i\Vert }_2}.$$

This view ray is subject to a different amount of light deflection $p_{k,i}$ for each different camera position ${\mathbf{c}}_i$ and for each different point ${\mathbf{x}}_k$.

Fig. 1

Refraction of a light ray connecting ${\mathbf{x}}_k$ and ${\mathbf{c}}_i$ based on an interface layer with thickness $d_w$ and refractive indices $n_1$ and $n_2$. The solid dashed line represents the virtual view ray for the camera ${\mathbf{c}}_i$, as its path is assumed to being linear. The pale line represents the actual path the light takes, including two refractions at the boundary layers between $n_1$ and $n_2$.

For a 3-D geometry measurement system based on stereo triangulation, the reconstruction of an arbitrary point ${\mathbf{x}}_k$ in a system with more than two cameras $n>2$ results in $m=(n-1)!$ reconstructed points ${\mathbf{x}}_{k,i,j}^{\prime }$, each one from a combination of cameras to a stereo pair $i,j\in m$. For assumed homogeneous refractive index conditions, all reconstructed points are, theoretically, equal to the corresponding and observed point in space, according to

Eq. (6)

$${\mathbf{x}}_{k,i,j}^{\prime }={\mathbf{x}}_k,$$

for each camera pair $i,j\in m$ and $n_1=n_2$. In inhomogeneous conditions $n_1\ne n_2$, each reconstructed point ${\mathbf{x}}_{k,i,j}^{\prime }$ is subjected to the combined light deflections $p_{k,i}$ and $p_{k,j}$ of both light paths, which, again, is different for each camera, i.e., $p_{k,i}\ne p_{k,j}$. Therefore, the distance $b_{k,i,j}$ between the point in space ${\mathbf{x}}_k$ and the reconstructed points ${\mathbf{x}}_{k,i,j}^{\prime }$ is expected to increase with larger thicknesses $d_w$ and larger refractive index differences $\mathrm{\Delta }{n}_{\mathrm{1,2}}=|n_1-n_2|$. While being different for different used camera pairs, this distance $b_{k,i,j}$ cannot not be measured in a real setup because it is not feasible to accurately reconstruct the point ${\mathbf{x}}_k$. Therefore, it is proposed to base the estimation of the reconstruction quality on the spatial extent of the set of reconstructed points ${\mathbf{x}}_{k,i,j}^{\prime }$. A natural choice for an easy-to-calculate representation for said extent is the statistical deviation of all of the reconstructed points ${\mathbf{x}}_{k,i,j}^{\prime }$ for $i,j\in m$.

3.2.

Estimation of the Reconstruction Quality for Multistereo-Pair Systems

The main requirement for all of the proposed methods is the unambiguity of the corresponding pixel locations. This is easily achievable for random-pattern or single-point triangulation systems due to the unambiguous nature of these systems. For fringe projection systems, this requirement can be achieved using Bräuer-Burchardt’s VR method (see Sec. 2), in which all camera pixels are calculated as a function of the projector pixel ${\mathbf{u}}_{c,i}=f({\mathbf{u}}_p)$.

Different calculation methods for a quality estimation are shown by Hartley and Zisserman.⁷ The commonly used method is the backprojection, i.e., the recalculation of the pixel location ${\mathbf{u}}_b$ from the reconstructed 3-D point $\mathbf{x}$ [see Eq. (1)]. The Euclidean distance $e_b$ to the activated pixel used for the reconstruction $\mathbf{u}$ is then called backprojection error, according to

Eq. (7)

$$e_b={\Vert \mathbf{u}-{\mathbf{u}}_b\Vert }_2.$$

When applying this method to a setup with more than two cameras $n>2$, all stereo pairs $m$ are taken into account. The backprojection error is then calculated separately for each pair $i,j\in m$, according to

Eq. (8)

$$e_b({\mathbf{x}}_{i,j}^{\prime })={\left|\right|{\mathbf{u}}_{i,h}-\frac{1}{{\lambda }_i}{\mathbf{K}}_i·{(\mathbf{Rt})}_i·{\mathbf{x}}_{i,j,h}^{\prime }\left|\right|}_2+{\left|\right|{\mathbf{u}}_{j,h}-\frac{1}{{\lambda }_j}{\mathbf{K}}_j·{(\mathbf{Rt})}_j·{\mathbf{x}}_{i,j,h}^{\prime }\left|\right|}_2.$$

To generalize the results, the average norm of the reprojection error for each reconstructed point within all observing cameras is computed, according to

Eq. (9)

$$e_{b,s}=\frac{1}{m}\sum e_b({\mathbf{x}}_{i,j}^{\prime })\text{for}i,j\in m.$$

To take the assumptions from Sec. 3.1 into account, the backprojection can also be based on the mean of correspondingly reconstructed points, i.e., on

Eq. (10)

$${\mathbf{x}}_m^{\prime }=\frac{1}{m}\sum {\mathbf{x}}_{i,j}^{\prime }\text{for}i,j\in m.$$

Now, we calculate the backprojection error $e_{b,i,m}$ for each camera or projector $i\in n$ from that mean point ${\mathbf{x}}_m^{\prime }$, according to

Eq. (11)

$$e_{b,i,m}={\left|\right|{\mathbf{u}}_{i,h}-\frac{1}{{\lambda }_i}{\mathbf{K}}_i·{(\mathbf{Rt})}_i·{\mathbf{x}}_{m,h}^{\prime }\left|\right|}_2.$$

To make this error comparable throughout systems with different numbers of cameras and projectors $n$, the average of this value is calculated, according to

Eq. (12)

$$e_{b,m}=\frac{1}{n}\sum e_{b,i,m}\text{for}i\in n.$$

However, this value might be a function of the used camera resolutions. Therefore, we also propose basing the error in the metric system, i.e., by calculating the reconstruction quality as the statistical variance of the reconstructed points, according to

Eq. (13)

$$E_m=\frac{1}{m-1}\sum {\left|\right|{\mathbf{x}}_{i,j}^{\prime }-{\mathbf{x}}_m^{\prime }\left|\right|}_2\text{for}i,j\in m.$$

A summary of the proposed reconstruction quality metrics as a flowgraph is shown in Fig. 2.

Fig. 2

Flowchart of the developed reconstruction quality metrics.

4.

Experimental Setup

In this section, the setup for the experiments is shown. The proposed methods are tested as a 2-D simulation model under homogeneous refractive index conditions, i.e., $n_1=\text{const}.$, $n_2=\text{const}.$, $n_1\ne n_2$. In addition, an optical 3-D fringe projection setup is used in a laboratory environment under homogeneous conditions, i.e., with a glass window, and under inhomogeneous conditions, i.e., measuring a hot object. In the simulation model, the light propagation is assumed to be in one plane as it simplifies the refraction and line–line intersection calculation.

4.1.

Model Setup

To test the proposed approach, a simplified 2-D simulation is set up (see Fig. 3). For the model, a homogeneous and plane parallel refractive index field of thickness $d_w$ and refractive index $n_2$ is inserted into an otherwise also homogeneous propagation medium with a refractive index of $n_1$. The path of light from a point ${\mathbf{x}}_k={(x,y)}^T$ in 2-D space to the center ${\mathbf{c}}_i={(c_{x,i},c_{y,i})}^T$ of a virtual camera $i$ is calculated using Snellius’ law (dashed lines) and a brute-force method. The mean of the leftover point-line distance from camera center $c_i$ to the refracted path of light is about ${10}^{-4}$. The vector from the camera center ${\mathbf{c}}_i$ to the first intersection ${\mathbf{d}}_{k,i}$ between $n_1\to n_2$ is then used as directional vector ${\mathbf{a}}_{k,i}^{\prime }$ (dashed lines) to calculate the virtual intersections

Eq. (14)

$${\mathbf{x}}_{k,i,j}^{\prime }:{\mathbf{c}}_i+{\lambda }_i{\mathbf{a}}_{k,i}^{\prime }={\mathbf{c}}_j+{\lambda }_j{\mathbf{a}}_{k,j}^{\prime },$$

for any combinations $i,j$ of the present camera pairs. The reconstructed points are then evaluated according to Eq. (13).

Fig. 3

Setup of the 2-D model experiment. All lengths are without units. The refraction of view rays ${\mathbf{a}}_{k,i}$ in an ambient refractive index field $n_1=1.2$ from the camera centers ${\mathbf{c}}_i$ is calculated through an inserted, homogeneous refractive index field $n_2=1.4$ of thickness $d_w=5$. The refracted view rays are marked with dashed/dotted lines and all intersect in ${\mathbf{x}}_k$. The reconstructed points ${\mathbf{x}}_{k,i,j}^{\prime }$ are on the intersection of the virtual nonrefracted view rays. For $n_2<n_1$, the points ${\mathbf{x}}_{k,i,j}^{\prime }$ are reconstructed with a greater $y$ value, i.e., are above the backprojected point ${\mathbf{x}}_k$.

4.2.

3-D Measurement Setup

The used optical 3-D fringe projection setup combines one green-LED projector (Wintech PRO 4500 based on TI DLP LightCrafter 4500) and four cameras (Allied Vision Prosilica GT 2050 and Prosilica GT 2300 with MeViS-C lens) to a multicamera fringe projection system [see Figs. 4(a) and 5(a)]. There are bandpass filters on the camera lenses used to block incoming radiation from the self-emission of the hot objects. The triangulation bases and triangulation angles are listed in Table 1.

Fig. 4

The used measurement setup. (a) Image of four-camera one-projector 3-D geometry measurement system. The short code for the identification of the cameras is also shown; (b) Image of the used measurement object at $T_c\approx 1300\mathrm{K}$. One pattern of the sinusoidal sequence is projected onto the object. The object lays in a prism holder with an anchor point on the tail end.

Table 1

Triangulation base and triangulation angle for all available stereo pairs in the fringe projection setup. The code for the stereo combination is taken from Fig. 4(a).

Stereo pair combination	Triangulation base in mm	Triangulation angle in deg
$c_{lL}:p$	271	39
$c_{lR}:p$	285	40
$c_{uL}:p$	240	33
$c_{uR}:p$	209	29
$c_{lL}:c_{lR}$	141	21
$c_{lL}:c_{uL}$	464	65
$c_{lL}:c_{uR}$	452	65
$c_{lR}:c_{uL}$	501	71
$c_{lR}:c_{uR}$	153	21
$c_{uL}:c_{uR}$	448	62

For an areal measurement, Peng et al.’s²² multifrequency phase-shift method is used to calculate the projector pixel in a given camera’s pixel ${\mathbf{u}}_p=f({\mathbf{u}}_{c,i})$ by projecting a coded sinusoidal sequence onto a specimen. The measurement object with a pattern of the projected sequence is shown in Fig. 4(b). Bräuer’s²³ VR method is then used to calculate the phasemaps ${\mathbf{u}}_{c,i}({\mathbf{u}}_p)$, therefore having calculated $n_c=4$ corresponding pixels for each projector pixel ${\mathbf{u}}_p$.

The 3-D-points are reconstructed from each available view-ray pair using epipolar geometry.⁷ Camera–camera pairs on the same side of the projector ($c_{lR}:c_{uR}$ and $c_{lL}:c_{uL}$) are not used here since the expected uncertainty is higher compared with the other pairs due to a smaller triangulation angle and triangulation base (see Table 1). The number of measured 3-D points for each projector pixel is shown in Fig. 5(b).

Fig. 5

Measurement setup from the projector perspective. (a) Image of the used measurement object at $T_c\approx 300\mathrm{K}$ in front of the measurement system. From the measurement system, only one camera and the projector are displayed; (b) Number of measured points for each projector pixel ${\mathbf{u}}_p$. The theoretical maximum number is $m=(n-1)!=10$ for $n=5$ cameras/projectors. In the presented case, this number is reduced to $m=8$ for excluding the camera-camera pairs on the same side of the setup. The red rectangle represents the examined area for the experiments.

A series of experiments is conducted to compare and evaluate the proposed methods. In these experiments, the geometry of a cylinder of diameter $d_c=50\mathrm{mm}$ is measured, which is placed in a prism holder with an anchor point (see Fig. 4(b)). The reconstruction quality is estimated for this cylinder under regular conditions $T_c\approx 300\mathrm{K}$ and with the cylinder at the forging temperature of $T_c\approx 1300\mathrm{K}$. To test the methods under the influence of a known refractive index field, a perspex disc with a thickness of $d_w=5\mathrm{mm}$ and a refractive index of $n_2\approx 1.49$ is placed between the measurement unit and the cold specimen (not shown). Each experiment consists of five consecutive measurements per measurement scenario, each taking about a span of 2 s. The experimental setup is kept constant between each set of measurements except for the measurement object, which is placed in an oven with $T_{\text{oven}}\approx 1300\mathrm{K}$ for about 1 h. Each measurement contains the reconstruction of ${\mathbf{x}}_{i,j}^{\prime }({\mathbf{u}}_p)$ for each stereo pair $i,j\in m$. The proposed reconstruction quality metrics are then calculated from the identical measurement results, the per-pixel mean of those metrics are shown in the result maps (Figs. 7Fig. 10–9), and all measured points are evaluated for the histograms in Fig. 10.

5.

Results

The results for the 2-D simulation model and the 3-D fringe projection system are shown in this section. The expectations for the measurement results are based on the experience of this research group in regards to the geometry measurement of hot objects. In general, the reconstruction quality is expected to decrease when measuring a hot object through the heat-induced refractive index field compared with the measurement of the same cylinder at room temperature, as long as the cylinder is placed in a similar pose for both measurements. The exact same placement is not reached nor is the same diameter of the cylinder measured since both are influenced by the thermal expansion of the cylinder from $T_c\approx 300\mathrm{K}$ to $T_c\approx 1300\mathrm{K}$. The measurement with the inserted glass window should yield a lower quality reconstruction value than both other measurements. This is mainly due to Eq. (4), by which the amount of light deflection is described. Overall, it is not expected to reach a perfect reconstruction quality since there is an intrinsic leftover error from the (always) imperfect calibration routine.

5.1.

Results for the 2-D Simulation Model

The points of investigation are tested in the 2-D simulation model (see Sec. 4.1 and Fig. 3). To this end, different points ${\mathbf{x}}_k$ with $k\in \{\mathrm{0,1},\dots 79\}$ and window thicknesses $d_w=\{\mathrm{1,3},\mathrm{5,7}\}$ are compared over variations of the refractive index difference $\mathrm{\Delta }{n}_{\mathrm{1,2}}=n_2-n_1$.

The results for a constant surrounding refractive index $n_1=1.2$ are displayed in Fig. 6(a). Here, the variances $e_m$ are nearly zero for $\mathrm{\Delta }{n}_{\mathrm{1,2}}=0$ and increase with increasing refractive index differences $e_m\propto |\mathrm{\Delta }{n}_{\mathrm{1,2}}|$. The deviations increase equally for both $\mathrm{\Delta }{n}_{\mathrm{1,2}}<0$ and $\mathrm{\Delta }{n}_{\mathrm{1,2}}>0$ until a threshold of $-0.1<\mathrm{\Delta }{n}_{\mathrm{1,2}}<0.1$. Afterward, the deviation gradient stays constant for $\mathrm{\Delta }{n}_{\mathrm{1,2}}<0$ and decreases for $\mathrm{\Delta }{n}_{\mathrm{1,2}}>0$ until a saddle point at about $\mathrm{\Delta }{n}_{\mathrm{1,2}}=0.25$.

Fig. 6

Results for the 2-D simulation model as the Euclidean distance of the variances $e_m$ over refractive index differences $\mathrm{\Delta }{n}_{\mathrm{1,2}}$ and thickness of the window $d_w\in \{\mathrm{1,3},\mathrm{5,7}\}$. The solid lines are the result of the arithmetic mean of 81 reprojected points ${\mathbf{x}}_k$. The results of the single-point reconstruction are in the background. (a) Result for $n_1=const.=1.2$ and $n_2$ from 1.0 to 2.4; (b) result for $n_1=1.00028$ and $n_2$ from 1.0 to $2n_1=1.00056$. The ambient refractive index $n_1$ in the presented case is the refractive index of air at standard conditions (ambient pressure 1 bar and 273 K). All length and thickness information is given in arbitrary units.

For varying thicknesses $d_w$, the deviations $e_m$ increase with increasing thickness $d_w$. There are also larger interpoint variances for an increasing thickness $d_w$ based on the location of the backprojected point ${\mathbf{x}}_k$. These are detected by observing the thickness of the more transparent points in the background.

The deviations $e_m$ for a scenario with an ambient refractive index of $n_1=1.00028$ (air at standard conditions) are shown in Fig. 6(b). Here, similar observations can be made compared to Fig. 6(a). However, the decrease of deviations $e_m$ when overcoming the threshold $\mathrm{\Delta }{n}_{\mathrm{1,2}}>0.25$ is not a part of the discussion of results. The relative interpoint differences are larger compared with those in Fig. 6(a), while being smaller on an absolute scale. The minimum deviation is at the expected place of $\mathrm{\Delta }{n}_{\mathrm{1,2}}=0$.

The results from the laboratory experiment are split into the three proposed quality estimation methods $\{e_{b,s},e_{b,m},E_m\}$ since they vary in magnitude. The maps of the single camera backprojection error $e_{b,s}({\mathbf{u}}_p)$ [from Eq. (9)] are shown in Fig. 7. The region of interest, from which the data are extracted, is shown in Fig. 5(b). The regular backprojection error is not homogeneously distributed and has a mean value of about 0.443 pixel for $T_c\approx 300\mathrm{K}$. There seem to be few differences compared with the quality estimation of the cylinder at $T_c\approx 1300\mathrm{K}$ with a mean deviation of $\sim 0.445\text{pixel}$ [see Fig. 7(b)]. There are larger differences in the measurement with the inserted window in the light path of about 2.05 pixel [see Fig. 7(c)].

Fig. 7

Mean of backprojection error for each single camera. (a) Cold object at $T_c\approx 300\mathrm{K}$; (b) warm object at $T_c\approx 1300\mathrm{K}$; and (c) cold object with window in the path of the measurement light.

The maps of the backprojection error $e_{b,m}({\mathbf{u}}_p)$ for each averaged corresponding object point ${\mathbf{x}}_m^{\prime }$ and Eq. (12) are shown in Fig. 8, while the maps from the metric deviations of the corresponding 3-D points $E_m$ and Eq. (13) are shown in Fig. 9. In both cases, the differences from the measurements of the cold cylinder $T_c\approx 300\mathrm{K}$ to the measurements of the warm cylinder $T_c\approx 1300\mathrm{K}$ [compare Figs. 8(a) to 8(b) and Figs. 9(a) to 9(b)] seem to be small but detectable, while the differences between deviations from the cold measurement to the measurement with the inserted window are larger.

Fig. 8

Mean of backprojection error from mean of 3-D points for each single camera. (a) Cold object at $T_c\approx 300\mathrm{K}$; (b) warm object at $T_c\approx 1300\mathrm{K}$; and (c) cold object with window in the path of the measurement light.

Fig. 9

Metric deviations from the mean of the 3-D points $E_m$. (a) Cold object at $T_c\approx 300\mathrm{K}$; (b) warm object at $T_c\approx 1300\mathrm{K}$; and (c) cold object with window in the path of the measurement light.

The histograms of these results (see Fig. 8) are sorted by the used evaluation metric. Therefore, the comparable results from $e_{b,s}$ are shown in Fig. 10(a), and the results from $e_{b,m}$ and $E_m$ are shown in Figs. 10(b) and 10(c).

Fig. 10

Histograms for the different quality estimation methods. (a) For $e_{b,s}$. Mean value for cold object is 0.443 pixel, for warm object is 0.445 pixel, and for cold object with window is 2.047 pixel; (b) for $e_{b,m}$. Mean value for cold object is 1.499 pixel, for warm object is 1.664 pixel, and for cold object with window is 6.197 pixel; (c) for $E_m$. Mean value for cold object is 0.262 mm, for warm object is 0.306 mm, and for cold object with window is 1.152 mm.

6.

Discussion of Results

Considering the results of the 2-D simulation model in Sec. 5.1, the proposed quality metric as the deviations $e_m$ of the redundantly reconstructed points ${\mathbf{x}}_{k,i,j}^{\prime }$ for $i,j\in m$ meets the expectations. The single point deviation is highly dependent on the incident angle ${\alpha }_{k,i}$ of each of the view rays for a camera $i$, as is the parallel shift $p_{k,i}$. However, the average of the reconstruction error $e_m$ for all points ${\mathbf{x}}_k$ is strictly monotonously increasing with increasing nominal optical wavelength differences $d_w\mathrm{\Delta }{n}_{\mathrm{1,2}}$ for $|\mathrm{\Delta }{n}_{\mathrm{1,2}}|<0.1$. The relation between single point deviation and $e_m$ is recognizable in the thickness of the lines in Fig. 6, where each single-point reconstruction error is displayed in the background. The simulation model has therefore proven the validity of the proposed method for the detection of small refractive index differences in the reconstruction of points by triangulation.

In the case of the real experiment in Fig. 4, which includes a 3-D reconstruction of a hot specimen, it is demonstrated that the metrics introduced for quality estimation have a significant sensitivity for the detection of significant homogeneous refractive index influences [compare Figs. 7(a), 8(a), 9(a) to 7(c), 8(c), 9(c), respectively]. The differences are easily detectable in both the error maps $e_{b,s}({\mathbf{u}}_p)$, $e_{b,m}({\mathbf{u}}_p)$, $E_m({\mathbf{u}}_p)$ as well as in the histograms in Fig. 10.

For the overall objective of detecting the influence of the heat induced refractive index gradient, the pixelwise differences between $T_c\approx 300\mathrm{K}$ and $T_c\approx 1300\mathrm{K}$ on the respective estimation method seem to be insignificantly small. Even a statistical analysis of the acquired results yielded no comprehensible conclusion for $e_{b,s}$ [see Fig. 10(a)], while being subject to interpretation for $e_{b,m}$ and $E_m$ [see Figs. 10(b) and 10(c)]. In the histograms, the distribution of both error metrics seems to be (approximately) Gaussian, and there is a difference of $\approx 11\%$ for $e_{b,m}$ and $\approx 17\%$ for $E_m$. The difference is both significant and robust considering the evaluated number of points of about $k\approx {10}^6$. The relative small reconstruction quality difference is mainly due to the relative large error in the reference results, which are a consequence of an imperfect calibration process. Extrapolating from the 2-D results, the measurement with $d_w=5$ and $\mathrm{\Delta }{n}_{\mathrm{1,2}}=0.5$ yields $e_m\approx 1.3$ and the measurement with $d_w=5$ and $\mathrm{\Delta }{n}_{\mathrm{1,2}}=-0.00028$ yields $e_m\approx 7\times {10}^{-3}$, i.e., a relative difference of $e_{m,\mathrm{\Delta }{n}_{\mathrm{1,2}}=0.5}/e_{m,\mathrm{\Delta }{n}_{\mathrm{1,2}}=-0.00028}\approx 185$. Comparing this with the relative differences in $e_{b,m}$ of $\approx 3.72$ and in $E_m$ of $\approx 3.76$ yields a much larger sensitivity of the model experiment compared with the measurements under laboratory conditions. This is mainly due to the additional influence of the calibration error in the workshop experiments.

The intent was to directly compare the displayed results as a difference map, e.g., as $E_m{(u_p,v_p)}_{T_c=300\mathrm{K}}-E_m{(u_p,v_p)}_{T_c=1300\mathrm{K}}$. This task proved to be impractical. The reasons for this effect are manifold.

The occurring and to be quantified light deflection moves the reconstructed points on the VR, so ${\mathbf{u}}_p({\mathbf{x}}_{T_c=300\mathrm{K}})\ne {\mathbf{u}}_p({\mathbf{x}}_{T_c=1300\mathrm{K}})$. In addition, the distribution of the reconstruction errors is inhomogeneous, even for the experiment in standard conditions [see, e.g., Fig. 7(a)]. Therefore, a small change in the object’s geometry or placement results in a change of the reconstruction error at that pixel position. These changes occur easily, mainly due to the different measurement conditions of the experiment at different states. Another reason is the surface of the object, which is constantly changing throughout heating and cooling periods as is its diameter. In addition, the position of the object cannot be assumed to be fixed, even though the cylindrical measurement object is placed in a prism holder with an anchor point.

7.

Conclusion and Future Work

In this paper, a method for the estimation of the reconstruction quality of an optical 3-D geometry measurement system under inhomogeneous refractive index conditions was presented. The method is based on a multicamera single-projector fringe projection system without using additional geometric standards, e.g., a sphere or a cylinder with a calibrated diameter. The reconstruction metrics are based on redundantly measured 3-D points and the analysis of the geometric mean of these points.

The theoretical background for the proposed methods, as well as the results of a 2-D simulation model, was laid out. In the experiment, the influence of the inhomogeneous refractive index gradient was joined in a homogeneous refractive index field of varying thicknesses and refractive index gradients. The deviation of the reconstructed points $e_m$ was introduced as a reconstruction quality metric for the 2-D case, and its feasibility was verified by the results of the simulation model. Here, the mean deviation is proportional to the optical wavelength difference $e_m\propto d_w\mathrm{\Delta }{n}_{\mathrm{1,2}}$ for small refractive index differences $\mathrm{\Delta }{n}_{\mathrm{1,2}}<0.1$.

For the 3-D laboratory experiment, the feasibility of the proposed quality metrics was proven by the comparison of an uninfluenced measurement with a measurement in which a glass plate was inserted into the optical path. However, the comparison of a measurement of a cylinder at $T_c\approx 300\mathrm{K}$ with the measurement at $T_c\approx 1300\mathrm{K}$ revealed no differences for the state-of-the-art quality metric of the single-camera backprojection error $e_{b,s}$. The evaluation of the results for the quality metric based on redundantly reconstructed points showed a small but significant difference between the hot and cold measurements. Here, the metric deviation $E_m$ yields a slightly higher sensitivity to the influence of the refractive index gradient than the backprojection $e_{b,m}$ of the mean points ${\mathbf{x}}_m^{\prime }$.

Overall, the results from the proposed reconstruction quality metric yield detectable differences when measuring hot objects compared with cold objects. This enables the estimation of a relative reconstruction quality for measurements in which geometric measurement standards cannot be used. For these, it is necessary to establish a ground truth for each measurement scenario by measuring a similar object in cold state.

In the future, the proposed and presented reconstruction quality metrics will be used to assess different compensation methods for the light deflection effect induced by an inhomogeneous refractive index field around wrought-hot objects. The combination of the presented metric and a similar geometric measurement standard can be used to calculate an absolute quality metric in reference to said standard. Also, it is intended to reduce or restructure the intrinsic reconstruction error to enable an analysis of hot objects without the need for a comparison with a cold object of similar shape.