1.

Introduction

Polarization cameras are division of focal plane imaging polarimeters¹ that use an array of micropolarizer filters aligned to the detector array pixels, typically with the micropolarizers oriented at angles 0 deg, 45 deg, 90 deg, and 135 deg. We provide a polarization camera calibration approach that is simpler than existing methods, and which does not require a motorized rotation stage or the use of highly uniform flat field illumination, such as that produced by an integrating sphere.^2–7 Taking advantage of this calibration approach, we show how to separate the effects of temporal noise from manufacturing variation when measuring the camera—a separation that is essential if we wish to fairly compare one camera’s performance to another. As more polarization cameras become commercially available, it becomes increasingly important to have a practical and unbiased method for evaluating and comparing these cameras.

Polarization cameras have conventionally been assembled by manufacturing the detector array and the micropolarizer array separately, then aligning them to one another and fixing them in place. As a result, there is a small vertical displacement between the polarizer layer and the detection layer that allows for cross talk between neighboring pixels when the light is incident from nonzero angles of incidence (see Fig. 1).⁴ More recently, attempts have been made to manufacture the sensor and polarization filters together as part of an integrated process to minimize cross talk and improve alignment.⁸ From a user’s standpoint, reduced cross talk here will appear as an increase in the diattenuation value of the individual pixels.

Fig. 1

The detection layer of a polarization camera, with micropolarizers (a) attached above the sensor layer and (b) integrated into the sensor layer. Optical rays shown in blue are cross talk rays from one pixel to its neighbor.

Micropolarizers use finely spaced wire grid patterns that can be difficult to manufacture, so that micropolarizer filters have historically had trouble achieving the polarization purity that monolithic polarization devices readily achieve. Although monolithic polarizers can readily achieve extinction ratios of better than ${10}^3$, or even ${10}^6$, micropolarizers have generally been limited to extinction ratios of $<30$.^2,4 Moreover, previous reports in the literature have indicated that micropolarizer orientation accuracy can vary by as much as 0.5 deg from pixel to pixel, whereas mounts for monolithic polarizers can readily achieve angular orientation accuracy of better than 0.01 deg.^4,9

After introducing our approach for calibrating microgrid polarization cameras, we show measurements obtained from three commercial polarization cameras. Using parameter variance formulas derived from a measurement model, we show how one can separate the effects of measurement noise from manufacturing variations, allowing for quantitative performance assessment and fair comparison among cameras. This procedure is demonstrated on three commercially available polarization cameras, showing the behavior of not just a select number of pixels in the center region of the image, but the entire set of pixels across the array.

2.

Calibration Procedure

A nonideal linear polarizer is a diattenuator defined by its maximum and minimum transmission values $q$ and $r$ as the diattenuator is rotated with respect to incident linearly polarized light. Thus, while an ideal polarizer achieves 100% transmission and 0% transmission at 0-deg and 90-deg orientation angles, the nonideal diattenuator will achieve $q$ and $r$. The Mueller matrix for a diattenuator element with the $q$ axis oriented at angle $\theta$ can be written as $M(q,r,\theta )$, as given in Eq. (1), where the diattenuation is $D=(q-r)/(q+r)$, and $A=\frac{1}{2}(q+r)$ is the mean transmission. For a given diattenuator, it is possible to achieve an ideal diattenuation of $D=1$ even when the peak transmission of the polarizer is very low $A\ll 1$. The parameter $A$ can thus be considered the “efficiency” of the polarizing element. The extinction ratio $X$ is derived from the diattenuation as $X=q/r=(1+D)/(1-D)$.

To measure the diattenuation properties of a polarization camera, we generate linearly polarized light sequentially oriented at four different $\theta$ angles and measure the detected intensity at each of the four positions (see Fig. 2). Thus, the source light Stokes vector is ${\mathbf{s}}_{\mathrm{src}}={(I_{\mathrm{p}},\mathrm{0,0},0)}^{\mathrm{T}}$, and the Mueller matrix of the generating polarizer ${\mathbf{M}}_{\mathrm{lp}}(\theta )$, which together generate a fully polarized state, $s_{\mathrm{in}}=I_{\mathrm{p}}·{(1,\cos \theta ,\sin \theta ,0)}^T$ for use in calibrating the camera pixels. The quantity $I_{\mathrm{p}}$ is the light flux in photons/sec incident on the given pixel of interest.

Fig. 2

Experimental layout for the calibration polarization state generator and the microgrid polarization camera. A diffuser is used to ensure that the light source is unpolarized. The inset at right shows a $2\times 2$ section of the detector array.

The behavior of a single pixel in the polarization camera can be modeled as a linear diattenuator ${\mathbf{M}}_{\mathrm{ld}}(q,r,\alpha )$ followed by detection vector $\mathbf{d}=(\eta ,\mathrm{0,0},0)$ with quantum efficiency $\eta$. Thus, the above generated state will be detected as

Setting the generating polarizer to orientations $\theta =0\mathrm{deg}$, 45 deg, 90 deg, and 135 deg, we obtain four measurements:

Whereas previous calibration methods fitted pixel diattenuation parameters using images taken at a large number of different input polarization angles,^4,5,10 the choice of the four angles (0 deg, 45 deg, 90 deg, and 135 deg) produces simple formulas for estimating the incident intensity $I_{\mathrm{e}}$ and the polarization properties at each pixel $\alpha$, $D$, and $X$:

From these equations, we can see that the estimates for the pixel polarization properties do not depend explicitly on the incident light intensity $I_{\mathrm{e}}$. As a result, as long as the light intensity does not vary significantly from one pixel to its neighboring pixels, calibrating the camera does not require spatially uniform illumination. While differences in light level at the camera will produce differences in noise at each pixel, this can be made small in comparison to the differences produced by manufacturing variations.

3.

Estimating Parameter Variances

From Eqs. (8)–(11), we can estimate the polarization properties of a pixel from the calibration measurements $g_{\theta }$. It is also useful to assess how much individual pixels vary from the overall mean—the parameter nonuniformity. Differences in manufacturing process between, say, the center of the detector array and its edges, may result in the pixel diattenuation and orientation at the edge being different than at the center. If the manufacturing differences are small in comparison to the error induced by measurement noise, then we should be able to ignore them. Otherwise, we need to make sure to calibrate for the individual variations in pixel properties.

Deriving formulas for the parameter variances involves the application of the well-known expression for parameter variance: $\mathrm{var}(\widehat{x})=⟨{\widehat{x}}^2⟩-{⟨\widehat{x}⟩}^2$, where $⟨·⟩$ indicates taking the mean value. To proceed, we insert the equation for the estimator $\widehat{x}$ and solve. Starting with ${\widehat{I}}_{\mathrm{e}}$, we obtain

Since $⟨{\widehat{I}}_{\mathrm{e}}⟩=I_{\mathrm{e}}$, we can write that $\mathrm{var}({\widehat{I}}_{\mathrm{e}})=⟨{\widehat{I}}_{\mathrm{e}}^2⟩-I_{\mathrm{e}}^2$. For uniformly distributed independent Gaussian (IG) noise, this gives

Next, we can calculate the variance of $\widehat{\alpha }$ by inserting Eq. (9) into the variance formula. While the result is a nonlinear equation, we can obtain a second-order power series representation by taking the Maclaurin series expansion in the noise variables $n_{\theta }$ and extract the second-order terms.^11,12 This produces a lengthy polynomial expression in the four noise variables, but if we assume that the noise terms are independent of one another, then the ensemble average of all mixed-noise terms (i.e., terms having $n_0{n}_{45}$ as factors) becomes zero. This greatly simplifies the expression. Finally, we substitute the IG noise model into the result, giving

A similar series-expansion procedure allows for calculating the variances of $\widehat{D}$ and $\widehat{X}$ as well:

The parameter variance equations are summarized in Table 1.

Table 1

Formulas for the mean and variance of detector array polarization properties.

4.

Experimental Results

To verify that we can characterize polarization cameras and separate manufacturing variability from noise in the results, we collected calibration data from three polarization cameras (“$A$,” “$B$,” and “$C$”) from three different manufacturers (Table 2). The experimental setup uses a linearly polarized incoherent light source, generated using a white light LED, diffuser, rotatable linear polarizer (Glan–Thompson type), and a narrowband spectral filter: $532\pm 10\mathrm{nm}$ for cameras $A$ and $B$, $520\pm 20\mathrm{nm}$ for camera C (Fig. 2). The spectral filters are needed in order to make the camera comparison as fair as possible, since camera C uses a nonremovable filter in front of its detector array.

Table 2

The four polarization cameras measured.

The initial datasets were collected by setting the generating polarizer to 0 deg and summing over many frames in order to reduce the measurement noise. The same procedure was then used for the 45 deg, 90 deg, and 135 deg orientations of the generating polarizer, and for the estimation of the background. After subtracting the background, the summed images were scaled from digital counts to photoelectron units using previous calibration of the cameras’ gain values. The radiometric gain values used for the four cameras were 2.2, 8.0, and 1.3 photoelectrons per digital count for cameras $A$, $B$, and $C$, respectively. Taking the sum of the images, rather than mean as is more common, is done in order to maintain Poisson statistics in the data.

Unlike previous calibration methods, we do not use an integrating sphere in order to make the illumination field uniform, as exact uniformity is unnecessary for the parameter estimates. The main purpose of the diffuser in the experimental setup is to remove any residual polarization from the light source. With the diffuser in place, the estimated degree of linear polarization of the light source is measured to be $<0.1\%$.

First, we sum together 2000 calibration images for the case of camera A, 100 images for cameras $B$ and $C$, into a single “sum image.” With the sum image, we implement the parameter estimation Eqs. (8)–(11) at each pixel. Taking the histograms of the resulting parameter images, we calculate the mean and standard deviations of each parameter, giving the results shown in Figs. 3 and 4 and summarized in Table 3. Thus, these parameter histograms have a distribution determined by a mix of (temporal) noise at each pixel as well as pixel-to-pixel variation in polarization properties. Note that since there is an easy-to-adjust degree of freedom in the orientation angle, we have defined the mean of 0-deg micropolarizers’ orientation angle to be exactly zero, so that the remaining orientation angles are defined with respect to it.

Fig. 3

Spatial histogram results for the three cameras, calculated after summing 2000 frames for camera A, 100 frames for cameras B and C. Note that all of the orientation angles are adjusted so that the mean of ${\alpha }_0$ is exactly zero. The calculated mean at the top left of each subfigure is given for each of the four pixel types, followed by the measured standard deviation and the estimated noise-only standard deviation values [obtained from Eq. (16)] in parentheses, i.e., mean (meas_std) (est_std). Gaussian curves for the fitted histogram mean and standard deviations are shown as solid curves overlying the histograms.

Fig. 4

Spatial histogram results for the three cameras, calculated after summing 2000 frames for camera A, 100 frames for cameras B and C. Gaussian curves for the histogram mean and standard deviations are shown as solid curves overlying the histograms. In camera A’s data, the histograms for $\widehat{D}$ have been truncated to 1, and those $X$ have been truncated at 2000 to prevent unphysical values. The four colors are coded to the four pixel orientation types: 0 deg, 45 deg, 90 deg, and 135 deg.

Table 3

Measured pixel properties for the four polarization cameras obtained from the 2000-frame-sum images (camera A) or 100-frame-sum images (cameras B and C). Each entry shows the mean value of each parameter taken across the entire array, together with the temporal-noise-removed estimate of the spatial standard deviation (the variation due to differences in manufacture) in square brackets.

The measurement results show a substantial range of performance for parameter mean and spatial variation among the three measured cameras, especially in the estimated micropolarizer extinction ratios. However, it is important to keep in mind that each camera uses pixels of different sizes, with different quantum efficiencies, different readout electronics, and different integration times. Thus, the measurement variation alone is not in itself sufficient; for fair comparison, it is necessary to separate the variation due to measurement noise from variation due to manufacturing differences of the pixels.

Looking at the distributions for ${\widehat{I}}_e$ in Fig. 4, we find that the measured standard deviations for ${\widehat{I}}_{\mathrm{e}}$ are far larger than the theoretical noise-only standard deviations, i.e., $\mathrm{var}({\widehat{I}}_{\mathrm{e}})\gg \frac{1}{4}⟨{\widehat{I}}_{\mathrm{e}}⟩$. This is an indication that the illumination is significantly nonuniform, or that there is significant pixel-to-pixel variation in the external quantum efficiency $\eta A$ [defined in Eq. (7)].

Looking at the histograms for $\widehat{D}$ and $\widehat{X}$ in Fig. 4, we can see that the two distributions are generally similar, but with the value and the width of the probability distribution $\mathrm{pr}(\widehat{X})$ increasing rapidly as $⟨\widehat{D}⟩$ approaches 1. This close association of the two distributions is also evident in Eq. (20) for $\mathrm{var}(\widehat{X})$, where a factor of ${(1-D)}^4$ appears in the denominator. Whereas the variance for $\widehat{D}$ is determined primarily by the factor of $I_{\mathrm{e}}$ in the denominator Eq. (18), we see that the variance for $\widehat{X}$ will be amplified by $(1-D{)}^{-4}$ while the mean value as given in Eq. (11) will be amplified by only ${(1-D)}^{-1}$. If we write the signal-to-noise ratio for the estimate of the extinction coefficient, we obtain

Thus, for a fixed number of measurement photoelectrons, the SNR declines as $D$ approaches 1, and this behavior is the property we see exhibited most clearly in the histogram for camera A—the camera where the diattenuation is the highest.

The tail on the right hand side of the extinction ratio distribution $\mathrm{pr}(\widehat{X})$ moves rapidly to higher values as $D$ comes closer to 1. This is a well-known property for “Gaussian ratio distributions”—in this case the ratio of $(D+1)$ to $(D-1)$, for which the mean becomes undefined and the variance becomes infinite.^13,14 While this behavior poses no serious problems for cameras B and C, camera A’s diattenuation is sufficiently close to 1 that after background subtraction a small fraction of pixels are left with zero or negative values when in the crossed-polarization condition. As a result, the tail of the distribution for $\widehat{D}$ extends past 1, so that for these pixels the extinction coefficient becomes infinite (for $D=1$), then wraps around to negative infinity and approaches zero from the negative number side (for $D>1$). These values are unphysical, of course, so that in Fig. 4 we have truncated the distribution for $D$ at 1, and the distribution for $X$ at 1250. Only a small portion of the distribution tail lies above these extreme values, so that without truncation the data distribution mean will be dominated by these rare values and will produce mean and variance values that are highly unstable, just as theory predicts.

As a result of the non-Gaussian shape for $\mathrm{pr}(\widehat{X})$, the median and mode of the distribution are more useful summary metrics of the distribution than the mean, and distribution quantiles can be used in place of the variance in order to describe the width. In Table 4, we compare the extinction ratio mean, median, and mode for camera A. Whereas for cameras B and C these three metrics are almost the same, we can see that for the case of camera A the mean is biased toward high values by the long one-sided tail of the distribution. In this situation, the median is probably the most useful single metric for camera users to use in evaluating camera pixel properties. For example, if we take the mean of the diattenuation distribution prior to applying the equation for the extinction coefficient, $X=(1+D)/(1-D)$, we obtain a result (column 4 in Table 4) that closely approximates the median value.

Table 4

Extinction coefficient summary statistics for camera A, using the data shown in Fig. 4.

4.1.

Using Temporal Noise Estimates to Validate the Variance Estimates

To validate the variance formulas, we capture a long sequence of frames and analyze the behavior of the polarization parameters for individual pixels over time. This removes the effect of spatial variability so that only temporal noise is present. For cameras A and B, we collected a sequence of 2000 frames, while for camera C, we were only able to capture 750 frames during a single calibration period. At each individual frame, we calculate the pixel properties $I_{\mathrm{e}}$, $\alpha$, $D$, and $X$, and from the resulting set of 2000 (or 750) estimates, we calculate the parameter variance. This is shown as the first point at the upper left of the plots in Fig. 5, corresponding to $N=1$, where $N$ is the number of frames summed before calculating the parameters.

Fig. 5

Temporal standard deviation of each parameter for a single pixel in each camera. The horizontal axis indicates the number $N$ of frames summed prior to calculating the parameter standard deviation. The dots indicate measurements, whereas the curve indicates the theoretical standard deviation value calculated from the measured light level. Note that the camera A subfigure for $\widehat{X}$ uses a semilogarithmic plot, while all of the other subfigures use a linear plot.

Next, we take the same dataset and sum every pair of frames (i.e., $N=2$) before calculating the parameters. For a shot-noise-limited measurement, this is equivalent to doubling the integration time and thus collecting twice the number of photoelectrons. Calculating the variance for this new set of 1000 (or 375) parameter estimates, we obtain the second point shown in the plots of Fig. 5.

Following this procedure for increasing values of $N$, we simulate the effect of measuring with steadily improving SNR. Using $⟨{\widehat{I}}_{\mathrm{e}}⟩$ in place of $I_{\mathrm{e}}$ in each of the variance Eqs. (14)–(20), we plot the corresponding noise-only variances predicted from theory as a solid curve.

4.2.

Separating Spatial Variability from Temporal Noise

Although the temporal noise measurements of Fig. 5 show a close fit to predictions, we can also see that the measured variance of all the pixels in the image (Figs. 3 and 4) are much larger than the values predicted from the variance formulas. This is an indication that pixel-to-pixel (deterministic) variability is dominating the measured variation, not temporal (stochastic) noise. To confirm this conclusion, we show a single row of pixels from each of the calibration images (Fig. 6), selecting every other pixel in order to avoid issues of differences between micropolarizer orientations. Here, we see that the $\widehat{\alpha }$, $\widehat{D}$, and $\widehat{X}$ variances are dominated by a broad systematic variation across the row rather than by uncorrelated noise.

Fig. 6

Parameter estimates for a row through the camera images, calculated after summing 2000 frames for camera A, 100 frames for cameras B and C. The inset numbers give the measured mean and standard deviation, and the theoretical noise-only standard deviation, of the row data.

With the effects of stochastic and deterministic variation now clear, we can use our variance formulas to remove the stochastic portion from a measured variance value, leaving only the spatial pixel-to-pixel differences. If we assume that the two sources of variation are Gaussian-distributed and uncorrelated, then their combined effect is given by a convolution of the two distributions: for stochastic variables $x$ and $y$, the variance of the sum $z=x+y$ is given as

$$\mathrm{var}(z)=\mathrm{var}(x)+\mathrm{var}(y).$$

Therefore, if we know $\mathrm{var}(z)$ and $\mathrm{var}(y)$ and want to solve for the standard deviation of $x$, then

$$\mathrm{std}(x)=\sqrt{\mathrm{var}(z)-\mathrm{var}(y)}.$$

If we take the data for the estimated extinction coefficient ${\widehat{\alpha }}_0$ from camera C in Fig. 3, then $\mathrm{var}(z)={[0.365]}^2$ and $\mathrm{var}(y)={[0.065]}^2$, so that $\mathrm{std}(x)=0.359$. In this case, we see that the observed spatial variation is almost entirely due to pixel-to-pixel manufacturing differences rather than to random noise. This and the corresponding results for $\widehat{\alpha }$ and $\widehat{D}$ for each of the four cameras is given in square brackets as the “manufacturing variation” in Table 3. By removing the stochastic component, we now have a direct means of comparing the spatial variability in the pixel polarization parameters.

5.

Conclusions

To assess the performance of a microgrid polarization camera, it is natural to look first for the mean angle of orientation and the mean extinction coefficient of each of the four orientations of detector array micropolarizers. The spatial variation in these parameters, however, is also an important concern: any calibration that does not estimate each pixel individually is subject to increasing error as the spatial variability increases. To eliminate this source of error, we can calibrate each pixel individually, but the inability to average over an ensemble of many samples means that the calibration measurements will have a much lower SNR. Using the variance formulas of Eqs. (14)–(20), we have a simple method of estimating parameter calibration accuracy, so that users can determine how many calibration frames of data are needed.

We have also shown that the most popular metric for quantifying the performance of polarizers, the mean of the extinction coefficient, becomes problematic when the polarizer approaches perfection. As the diattenuation value approaches 1, the tail of the distribution for $\widehat{X}$ lengthens, so that the mean becomes biased and highly unstable. Unless the mean of the diattenuation distribution is many standard deviations below 1, i.e.,

It is also important to keep in mind that the micropolarizer characteristics will, in general, depend on the wavelength and the incident angle of incidence (and therefore on the lens numerical aperture), so that a user may need to recalibrate the sensor for each spectrum at which it is to be used, and to a lesser extent, with each lens.¹⁵

The four cameras examined in Sec. 4 above show a wide spread in estimated performance, and it is natural to ask just how important it is to work with a camera that has pixels of extinction ratio 300 versus one with, say, an extinction ratio of 10. Tyo and Wei¹⁶ and Roussel et al.¹⁷ have shown that even for an extinction ratio as low as $X=5$, the SNR in the Stokes vector elements increases by a factor of only $\sim 2.7$ relative to an ideal diattenuator ($X=\infty$). This would seem to argue that there is little benefit to be had once the extinction ratio exceeds 10 or 20, but most researchers and engineers persist in pushing hard to get the highest $X$ values. It seems likely that the reason for this gap between the theoretical value of high $X$ and the practical value placed on it by users lies with the ease of calibration. For a polarimeter employing high extinction ratio elements, one can approximate the polarizers as ideal diattenuators without a heavy cost in error. A polarimeter with low extinction ratio elements, on the other hand, requires a high accuracy calibration, and the polarimetric estimation equations will need to be more complex in order to accommodate the nonideal diattenuation values.

Another place where the higher extinction ratio provides tangible benefits is for imaging polarimetry of natural outdoor scenes. In this situation, many pixels will be seeing light with a degree of polarization close to zero, so that the polarimetric SNR will be poor for measuring the spatially resolved angle of polarization.¹⁷ Using a higher extinction ratio will allow for seeing smaller polarization features above the noise level.