The Canadian Space Agency (CSA) is developing a pre-operational spaceborne Hyperspectral Environment and Resource Observer (HERO). HERO will be a Canadian optical Earth observation mission that will address the stewardship of natural resources for sustainable development within Canada and globally. To deal with the challenge of extremely high data rate and the huge data volume generated onboard, CSA has developed two near lossless data compression techniques for use onboard a satellite. CSA is planning to place a data compressor onboard HERO using these techniques to reduce the requirement for onboard storage and to better match the available downlink capacity. Anomalies in the raw hyperspectral data can be caused by detector and instrument defects. This work focuses on anomalies that are caused by dead detector elements, frozen detector elements, overresponsive detector elements and saturation. This paper addresses the effect of these anomalies in raw hyperspectral imagery on data compression. The outcome of this work will help to decide whether or not an onboard data preprocessing to remove these anomalies is required before compression. Hyperspectral datacubes acquired using two hyperspectral sensors were tested. Statistical measures were used to evaluate the data compression performance with or without removing the anomalies. The effect of anomalies on compressed data was also evaluated using a remote sensing application.

This paper presents a study on the compression of hyperspectral satellite data using JPEG 2000 and residual encoding (RE). The first step in the process is to apply a decorrelating transform in the spectral-direction or z-direction. In most cases in this study, the Karhunen-Loeve Transform (KLT) is used. For comparison, some examples are also included where the discrete wavelet transform (DWT) is used for this purpose as well as examples with a purely 2-D approach that uses no z-direction transform. Bit-rate allocation techniques are used in order to take advantage of the energy compaction obtained when applying a transform in the z-direction. The transformed slices and their corresponding bit rates are input into JPEG 2000 in order to obtain the compressed bit stream. In this study, the compressed bit stream is decompressed at the encoder side in order to compute the recovered data. These data are then subtracted from the original data in order to calculate the residuals, which are then quantized and losslessly encoded separately using JPEG2000 itself in order to control the maximum absolute error (MAE). An analysis between using and omitting residual encoding with respect to MAE is included. It is observed that a decrease in the MAE by a factor of 3 is achieved for this data with very small overhead when the residual encoding is utilized. The two data sets used in this study are the well known Cuprite radiance imagery from AVIRIS and a set from the Hyperion satellite system, both of which are available in 16 bits per value form.

This paper describes an efficient algorithm and its Java implementation for a recently developed mean-squared error (MSE) rate-distortion optimal (RDO) inter-slice bit-rate allocation (BRA) scheme applicable to the JPEG2000 Part 2 (J2KP2) framework. Its performance is illustrated on hyperspectral imagery data using the J2KP2 with the Karhunen- Loeve transform (KLT) for decorrelation. The results are contrasted with those obtained using the traditional logvariance based BRA method and with the original RDO algorithm. The implementation has been developed as a Java plug-in to be incorporated into our evolving multi-dimensional data compression software tool denoted CompressMD. The RDO approach to BRA uses discrete rate distortion curves (RDCs) for each slice of transform coefficients. The generation of each point on a RDC requires a full decompression of that slice, therefore, the efficient version minimizes the number of RDC points needed from each slice by using a localized coarse-to-fine approach denoted RDOEfficient. The scheme is illustrated in detail using a subset of 10 bands of hyperspectral imagery data and is contrasted to the original RDO implementation and the traditional (log-variance) method of BRA showing that better results are obtained with the RDO methods. The three schemes are also tested on two hyperspectral imagery data sets with all bands present: the Cuprite radiance data from AVIRIS and a set derived from the Hyperion satellite. The results from the RDO and RDOEfficient are very close to each other in the MSE sense indicating that the adaptive approach can find almost the same BRA solution. Surprisingly, the traditional method also performs very close to the RDO methods, indicating that it is very close to being optimal for these types of data sets.

This paper presents current status of lossless compression of ultraspectral sounder data. The lossless compression results from the transform-based (e.g. JPEG2000, 3D SPIHT, and Lossless PCA), prediction-based (e.g. JPEG-LS, CALIC, and linear prediction using OOMP), and clustering-based (e.g. PVQ, DPVQ, PPVQ and FPVQ) methods are presented. The ultraspectral sounder data features strong correlations in disjoint spectral regions affected by the same type of absorbing gases. Some robust data preprocessing scheme (e.g. BAR) is also demonstrated to improve compression gains of existing state-of-the-art compression methods such as JPEG2000, 3D SPIHT, JPEG-LS, and CALIC.

In this paper, which is part of an ongoing sequence of papers devoted to the subject of efficient noise-tolerant lossless compression of satellite data for transmission, we describe an algorithm for this purpose which effectively addresses the above criteria. Our algorithm exhibit the potential to achieve noise-tolerant compression ratios averaging 3.2 : 1. An earlier approach, which we presented at Third GOES-R User Conference in May of 2004, was the first such method to break the 3 to 1 compression barrier for this class of data.

Two data compression algorithms intended for the compression of hyper and ultraspectral data are reviewed. These methods have been successfully applied to the compression of NASA JPL AVIRIS hyperspectral images. The two algorithms are based on slightly different requirements and assumptions. The first one is a low complexity, real-time, inter-band, least squares optimized predictor (SLSQ) whose raster-scan nature makes it amenable for on-board implementation. The second is a partitioned vector quantization algorithm (LPVQ) with tunable quality ranging from lossless to lossy. LPVQ is more complex, but it allows fast browsing and pure-pixel classification in the compressed domain, so it is more suitable to archival and distribution of compressed data. Both approaches compare well to the state-of-the-art in the compression of AVIRIS data. Preliminary results on the compression of AIRS ultraspectral sounder data are presented.

Ultraspectral sounder data is known for its huge size and sensitivity to noise in retrieving geophysical parameters. Lossless compression of it is thus desired for archival and transfer purposes. In this paper, we propose a lossless compression scheme which consists of a linear prediction, a lattice encoder, and an entropy encoder. The linear prediction serves as a whitening tool to produce a normally distributed residual. The lattice encoder can use low bit rates to approximate a Gaussian source. The entropy encoder is then used for encoding the final residual whose variance is reduced by the lattice encoder. The compression on the standard ultraspectral test dataset set of 10 AIRS granules shows that the proposed scheme outperforms DPVQ, JPEG2000 and the common compression utility BZIP2.

The radio frequency spectrum (9 kHz-275 GHz) is a limited finite resource for which many radio services compete. Most of the 525 radio frequency bands are shared by two or more of the 30 recognized services. Worldwide, except for some individual country differences, in the 8025-8400 MHz range, often referred to as the X-band, the Earth exploration-satellite service (EESS) shares its space-to-Earth allocation with the fixed-satellite service (Earth-to-space) and the terrestrial fixed and mobile services. Additionally, the adjacent band, 8400-8450 MHz, is allocated to the space research service (deep space) in the space-to-Earth direction and radio frequency emissions from spaceborne or airborne stations can be particularly serious sources of interference to these space research operations. Managing the use of this spectrum requires consideration of not only the sharing among the allocated services but also within a given service. In particular, the continued growth in use of the EESS allocation by Earth-exploration satellites is nearing the capacity of the 375 MHz. In order to permit the maximum utilization, it is incumbent upon satellite network operators to consider innovative design techniques that result in spectral efficiency without causing harmful interference to other systems using the allocation. International and U.S. spectrum regulators, as well as the entities that manage Earth resource satellites using this band, have established guidelines that support such spectrum efficiency. This paper provides the three Federal agencies' thoughts on the current policy and the steps being taken to ensure the continued availability of this spectrum into the future.

GOES-R, planned for launch around 2012, is currently under development by the National Oceanic and Atmospheric Administration (NOAA) of the United States. It will be the first in a new series of geostationary (GEO) environmental satellites to provide greater capabilities for weather, atmosphere, climate, and ocean monitoring. All the onboard sensors together may generate a combined raw sensor data rate of as much as 200 Mbps on the downlink, while the global rebroadcast data rate to the users after ground compression may be as much as 32 Mbps. To transmit such a high data rate through a channel of limited bandwidth, the adoption of a high-order modulation, such as QPSK, 8PSK, or 16QAM, is necessary. As a result, much higher transmit power than that for the binary modulation is needed in order to achieve the required bit error rate, which is particularly stringent for GOES-R due to the needed protection to the compressed data. Thus, the forward error correction (FEC) coding, which is a technique that can provide significant improvement of power efficiency, becomes crucial for GOES-R. This paper presents various methods of combining high-order modulations and FEC codes. We have proposed a baseline code waveform for GOES-R, which can satisfy both bandwidth and power efficiency requirements. In this paper, we also assess other commercially available code waveforms and compare their performances with that of our baseline waveform.

This paper presents results obtained from an end-to-end, proof-of-concept system for a GOES-R series satellite communication system, that integrates a multilevel modulator, turbo coding, and a nonlinear traveling wave tube amplifier (TWTA). Multilevel modulation schemes allow high-speed data communications in a limited amount of spectrum, enabling higher data rates for GOES-R user downlink, as compared to the GOES user downlinks within the existing L-band allocation. Bandwidth-efficient modulations, such as 8-PSK and 16-QAM allow transmission of 3 or 4 times more data in the same amount of bandwidth than a standard BPSK modulation. This improvement, however, comes at the price of increased linearity requirements for the end-to-end link. This constraint is especially important for the power amplifier, which is typically a nonlinear device. TWTAs are frequently used on satellites for transmitter power amplification. These high-power devices operate at highest efficiency when in saturation mode. However, their transfer function is highly nonlinear in this mode, causing significant degradation in the link bit error rate (BER). Applying forward error correction based on turbo codes improves the BER by providing an additional noise margin of up to 5 dB. This paper presents measured BER curves for different Turbo codes, taken at different power levels relative to saturation. The results demonstrate that very low BER (below 10^-10)can be achieved when using 8-PSK even when operating within 1 dB of saturation. This research and study was done by the Aerospace corporation in support of NOAA, and its future GOES-R series satellites.

Turbo codes and Low Density Parity Check (LDPC) codes are well known to provide Bit Error Rate (BER) performance close to the Shannon capacity limit. Bandwidth constrained satellite channels could potentially benefit by employing higher order PSK modulations. However, employing higher order PSK modulations may not be practical for satellite amplifiers due to the increased power requirements. The excellent performance of serial concatenated turbo codes could be used to maintain satellite amplifier power levels to those relatively close to the Shannon limit. The performance of the system, however, is dependent on the satellite channel, which typically includes phase noise and some degree of nonlinearity in the satellite amplifier. The performance of various waveforms and PSK modulations employing Serial Concatenated Turbo Codes are investigated using a model of a non-ideal satellite channel. The hardware complexity of the serial concatenated turbo decoder at the ground receiver is also considered.

Research has been undertaken to examine the robustness of JPEG2000 when corrupted by transmission bit errors in a satellite data stream. Contemporary and future ultraspectral sounders such as Atmospheric Infrared Sounder (AIRS), Cross-track Infrared Sounder (CrIS), Infrared Atmospheric Sounding Interferometer (IASI), Geosynchronous Imaging Fourier Transform Spectrometer (GIFTS), and Hyperspectral Environmental Suite (HES) generate a large volume of three-dimensional data. Hence, compression of ultraspectral sounder data will facilitate data transmission and archiving. There is a need for lossless or near-lossless compression of ultraspectral sounder data to avoid potential retrieval degradation of geophysical parameters due to lossy compression. This paper investigates the simulated error propagation in AIRS ultraspectral sounder data with advanced source and channel coding in a satellite data stream. The source coding is done via JPEG2000, the latest International Organization for Standardization (ISO)/International Telecommunication Union (ITU) standard for image compression. After JPEG2000 compression the AIRS ultraspectral sounder data is then error correction encoded using a rate 0.954 turbo product code (TPC) for channel error control. Experimental results of error patterns on both channel and source decoding are presented. The error propagation effects are curbed via the block-based protection mechanism in the JPEG2000 codec as well as memory characteristics of the forward error correction (FEC) scheme to contain decoding errors within received blocks. A single nonheader bit error in a source code block tends to contaminate the bits until the end of the source code block before the inverse discrete wavelet transform (IDWT), and those erroneous bits propagate even further after the IDWT. Furthermore, a single header bit error may result in the corruption of almost the entire decompressed granule. JPEG2000 appears vulnerable to bit errors in a noisy channel of satellite transmission, and thus has difficulty to preserve the quality of ultraspectral sounder data. A channel decoded bit error rate (BER) of 10-11 or better may be necessary for a granule error rate of 0.00116 in a compressed ultraspectral sounder data stream that is transmitted in a satellite channel. This work at The Aerospace Corporation and the University of Wisconsin, CIMSS, was under separate contracting from and performed for the National Oceanic and Atmospheric Administration (NOAA) National Environmental Satellite, Data, and Information Service (NESDIS), a component of the U.S. Department of Commerce.

At NASA's Goddard Space Flight Center (GSFC), space qualified integrated circuits for several key elements in space communication systems have been in development to increase data return in bandwidth constrained channels for future missions. Particularly in the area of digital communication, the development includes data compression, channel coding and modulation. In on-board data compression area, development focuses on a high-speed compression scheme that serves both push-broom and frame sensors. The compression ratio can be easily adjusted for different applications from lossless to visually lossless. The algorithm conforms to the Consultative Committee on Space Data Systems (CCSDS) new compression recommendation to be released 2005. The radiation-tolerant (RT) hardware will afford 20 Msamples/sec processing on sensor data. For bandwidth efficient channel coding, newly developed low density paritycheck codes (LDPCC) will double channel utilization as compared to previously used concatenated convolutional/Reed- Solomon (CC/RS) coding scheme. An RT implementation of the encoder is expected to work up to 1 Gbps serving both low-rate and high-rate missions. In modulation, a versatile multi-function base-band modulator allows missions the flexibility to choose from 2 bits/symbol/Hertz quadrature phase shift keying (QPSK)-type schemes, to 2.0, 2.25, 2.5, and 2.75 bits/symbol/Hertz 8 phase shift keying trellis-coded modulation (8-PSK TCM) schemes--all CCSDS recommendations. Along with 8PSK, 16-quadrature amplitude modulation (16-QAM), 16-ampliture phase shift keying (16-APSK), all modulations are implemented in a single RT chip with expected throughput of over 300 Mbps. This paper describes the development of these three technology areas and gives an update on their availability for space missions.

When a photon is detected by a Geiger-mode avalanche photo-diode (GMAPD), the detector is rendered inactive, or blocked, for a certain period of time. In this paper we derive the blocking probability for a GMAPD whose input is either an unmodulated, Bernoulli modulated or pulse-position-modulated (PPM) Poisson process. We demonstrate how the PPM and Bernoulli cases differ, illustrating that the PPM blocking probability is larger than the Bernoulli. The blocking rates may be decreased by focusing the incident light on an array of detectors. We show that the binomial output statistics of an array of GMAPDs may be modeled as Poisson and measure the error in this approximation via the relative entropies of the two distributions.

The purpose of this paper is to examine the possibility of increasing the downlink E.I.R.P. density limits for operation of Fixed Satellite Service (F.S.S.) links in the Ku band. The Ku band is extensively used for voice, video, data, remote sensing applications and for transmission to VSAT (Very Small Aperture Terminals) networks. However, the limit on downlink E.I.R.P. density is considered to be conservative and it's assumed that much higher E.I.R.P. densities should be possible without causing significant interference to adjacent users. If this is proved, it would be possible to achieve lower Bit Error Rates and thereby increase reliability of satellite links. The increased power transmission capability would also allow transmission of more signals with the available bandwidth or lead to reduction in size of receiving antennas. With the help of existing F.C.C. limits on antenna gain patterns and the I.T.U. interference criterion, an increase in downlink E.I.R.P. density per carrier by more than 66% is shown to be possible. This increase in power would dramatically affect Fixed Satellite Services.

In anticipation of the large jump in data volumes projected from future operational satellite and weather radar systems, and as lead agency for climate science, NOAA is developing ambitions programs for data and information stewardship. These programs are intended to provide broad access to a suite of products and services and to ensure long-term preservation of not simply the data, but of the information content of the observations. This paper outlines the vision and scope of these programs.

Nonreversible variable-length codes (e.g. Huffman coding, Golomb-Rice coding, and arithmetic coding) have been used in source coding to achieve efficient compression. However, a single bit error during noisy transmission can cause many codewords to be misinterpreted by the decoder. In recent years, increasing attention has been given to the design of reversible variable-length codes (RVLCs) for better data transmission in error-prone environments. RVLCs allow instantaneous decoding in both directions, which affords better detection of bit errors due to synchronization losses over a noisy channel. RVLCs have been adopted in emerging video coding standards--H.263+ and MPEG-4--to enhance their error-resilience capabilities. Given the large volume of three-dimensional data that will be generated by future space-borne ultraspectral sounders (e.g. IASI, CrIS, and HES), the use of error-robust data compression techniques will be beneficial to satellite data transmission. In this paper, we investigate a reversible variable-length code for ultraspectral sounder data compression, and present its numerical experiments on error propagation for the ultraspectral sounder data. The results show that the RVLC performs significantly better error containment than JPEG2000 Part 2.

In this paper we provide a study concerning the suitability of well-known image coding techniques originally devised for lossy compression of still natural images when applied to lossless compression of ultraspectral sounder data. We present here the experimental results of six wavelet-based widespread coding techniques, namely EZW, IC, SPIHT, JPEG2000, SPECK and CCSDS-IDC. Since the considered techniques are 2-dimensional (2D) in nature but the ultraspectral data are 3D, a pre-processing stage is applied to convert the two spatial dimensions into a single spatial dimension. All the wavelet-based techniques are competitive when compared either to the benchmark prediction-based methods for lossless compression, CALIC and JPEG-LS, or to two common compression utilities, GZIP and BZIP2. EZW, SPIHT, SPECK and CCSDS-IDC provide a very similar performance, while IC and JPEG2000 improve the compression factor when compared to the other wavelet-based methods. Nevertheless, they are not competitive when compared to a fast precomputed vector quantizer. The benefits of applying a pre-processing stage, the Bias Adjusted Reordering, prior to the coding process in order to further exploit the spectral and/or spatial correlation when 2D techniques are employed, are also presented.

A method combining the empirical mode decomposition (EMD) and the principal component analysis (PCA) was recently proposed for lossless compression of ultraspectral sounder data. In that method, data residual is obtained via the linear regression of the data against m intrinsic mode functions (IMFs) which are obtained from the EMD of the data mean, followed by the linear regression of the IMF regression error against a truncated number, n, of their corresponding principal components (PCs). In this paper we show that this two-stage (m IMFs + n PCs) linear transform approach is not as good as its counterpart two-stage (m PCs + n PCs) linear transform approach in terms of data residual and compression ratio of ultraspectral data, given the same number of IMFs and PCs used respectively at the first stage, followed by the same number of PCs used at the second stage. Mathematically, the two-stage (m PCs + n PCs) linear transform approach is equivalent to a single linear transform with (m + n) PCs. In other words, the simple PCA compression method outperforms this combined EMD and PCA compression method. This is expected because the PCA (also called the Karhunen-Loève transform or the Hotelling transform) is known to be the optimal linear transform in the sense of minimizing the mean squared error.

In lossless compression of the ultraspectral sounder data, the arithmetic coder is often used as the last stage of the compression scheme. While the arithmetic coder is good at compressing arbitrarily close to the entropy of the source, it is also known for its severe weakness to transmission noise. Even a single bit error in the transmission process can cause havoc and make the subsequent decoded stream completely useless. To cope with it, this paper adopts an arithmetic coder with a forbidden symbol and uses the maximum a posteriori (MAP) technique for decoding. Based on the ultraspectral sounder data, evaluation on the error correction capability of this error-resilient arithmetic coder will be reported.

We present a lossless compressor for multispectral Landsat images that exploits interband and intraband correlations. The compressor operates on blocks of 256 x 256 pixels, and performs two kinds of predictions. For bands 1, 2, 3, 4, 5, 6.2 and 7, the compressor performs an integer-to-integer wavelet transform, which is applied to each block separately. The wavelet coefficients that have not yet been encoded are predicted by means of a linear combination of already coded coefficients that belong to the same orientation and spatial location in the same band, and coefficients of the same location from other spectral bands. A fast block classification is performed in order to use the best weights for each landscape. The prediction errors or differences are finally coded with an entropy - based coder. For band 6.1, we do not use wavelet transforms, instead, a median edge detector is applied to predict a pixel, with the information of the neighbouring pixels and the equalized pixel from band 6.2. This technique exploits better the great similarity between histograms of bands 6.1 and 6.2. The prediction differences are finally coded with a context-based entropy coder. The two kinds of predictions used reduce both spatial and spectral correlations, increasing the compression rates. Our compressor has shown to be superior to the lossless compressors Winzip, LOCO-I, PNG and JPEG2000.

The Space Dynamics Laboratory (SDL) has developed a JPEG 2000 image compression demonstration board that implements both Tier1 and Tier2 JPEG 2000 encoding in two Xilinx Virtex II FPGAs. This board was built as a first step toward developing JPEG 2000 image compression hardware that could be used for remote sensing on the ground, in the air, or in Earth orbit. It has been used to demonstrate the power and flexibility of the JPEG 2000 standard in hardware, compressing both 8-bit and 12-bit grayscale images based on decoded image quality as well as output bit rate control. Images have also been compressed in both lossless and lossy modes. The board produces a JPEG 2000 file that includes all header and packet information needed to decode the output file. The output file can be decompressed directly or manipulated in software to enhance certain features of the compressed image. Throughput for the demonstration board is a function of wavelet type and bit depth. The demonstration board can be used in several different configurations as presented in this paper.