Overview

Transform-based technique is the most popular two-dimensional (2-D) image compression technique that has been extended to three-dimensional (3-D) or HSI compression. It is known as transform-based technique as it transforms the pixels values into the frequency domain by applying transformation function to all the three dimensions of image. There are some great transformation techniques such as discrete cosine transform (DCT), discrete Fourier transform, discrete wavelet transform (DWT), and Karhunen–Loeve transform (KLT) that are used in image compression. It can remove both spectral and spatial correlation depending on the domain on which it is applied. The technique can be applied in combination with nearly all other methods such as prediction-based, VQ, tucker-based, compressive sensing, and in learning-based algorithms. Some state-of-the-art algorithms in this field are 3D-DWT,⁸ 2D-KLT,⁹ 3D-set partitioning embedded block (SPECK),¹⁰ and 3D-low memory block tree coding (LMBTC).¹¹

Technique

Compression using the transform-based method follows some steps that may vary for different algorithms but can be generalized as in Fig. 2. Forward transform applies a transformation function (cosine, wavelet, or Fourier) to one of the spatial- or spectral-domain or both, then performs decorrelation, and generates coefficients. Following which, quantization is completed; this removes factors that are close to zero. In the last step, encoding techniques are applied to the quantized coefficients to generate bit-streams. It could be transmitted or stored with a reduced number of bits per pixel to save space (in storage) and bandwidth (in transmission).

Fig. 2

Steps of transform-based algorithm.

Some transform-based his compression algorithms in the scope of this article are discussed below. Karami et al.¹² proposed a transform-based technique in which 3D-DCT is applied hisHSI. It converts the raw pixels into frequency-domain coefficients using a cosine and inverse cosine transformation function on all the three dimensions of the image. High and low energy coefficients are separated out of which low energy coefficients are dropped using quantization. Sparse Tucker decomposition (TD) is then applied to modified coefficients to generate a compressed image. The reverse process is followed in decompression that generates original image with some loss due to irreversible quantization process. Karami et al.⁸ proposed another transformation technique named 3D-DWT-TD on the HSI, which uses DWT to transform spatial-domain pixels into frequency domain using wavelet function on all the three dimensions of HSI. Four submatrices are generated containing edge-based information, horizontal information, and vertical and approximation information in each submatrix. TD is then applied on all these four matrices separately followed by generation of mode matrices. Entropy coder is then used to code the core tensor and original image is reconstructed using reverse process.

Transformation-based technique has also been applied with machine learning techniques such as support vector machine (SVM)¹³ that has the following steps. Cubical HSI is first divided into small frames to reduce the complexity, and 3D-DCT is applied at the compression end on subimages. It is followed by a 3-D zig-zag quantizer that removes the unnecessary coefficients. SVM regression is used on the leftover coefficients to generate support vectors and weights, which are then encoded by entropy coder. Töreyn et al. proposed a hybrid algorithm named as joint photographic experts group-lossless (JPEG-LS)¹⁴ in which one-dimensional (1-D) integer wavelet transform is applied on spectral bands. It gives a residual image that is encoded by Golomb-rice encoding. Decompression is used on the bit-streams that reconstruct the original image without any loss. The performance of the proposed method is comparatively better than JPEG. Kozhemiakin et al.¹⁵ proposed a compression method based on 3-D AGU coder that calculates cross-correlation factor for images in different channels. Frequency coefficients are obtained from 3-D DCT where quantization step is set proportional to noise standard deviation. AGU coder is applied at the last level of compression. Giordano and Guccione¹⁶ proposed a combination of clustering and transformation for compression of HSI implemented on graphical processing unit (GPU). It is a region of interest (ROI)-based compression method that clusters the input image into five application-specific classes with an assumption that reflectance value of pixels is preloaded into memory. Labeling of a block is done according to a rule as “ROI” and others as “not-ROI.” Then, principal component analysis (PCA) is applied to reduce the spectral redundancy, and PCs with variance 99.9% are retained. The labeled image is also processed by 2-D DWT for spatial redundancy. Another use of machine learning technique in combination with DCT was proposed in PCA-DCT,¹⁷ where PCA is applied to find the feature vector, similarities, and dissimilarity in the form of residual image. Subsequently, DCT is applied to compress image. To further improve the compression performance, Mei et al.¹⁸ proposed a hybrid algorithm named folded-PCA in combination with JPEG2000. In this technique, the covariance matrix is calculated by folding the spectral vector into a matrix, and eigenvectors are used to obtain principal components that can represent features of the entire image. JPEG2000 is then applied to the reduced image to compress it further. The algorithm was again extended in weighted principal component analysis (WPCA)¹⁹ where an adaptive cosine estimator algorithm was applied for target detection. Then, PCA is applied to HSI after converting it into 2-D matrix but with some modification in the weight matrix. Mean pixel matrix and covariance matrix are calculated by giving more weight to pixels around the detected target.

Wang et al.¹⁰ proposed a joint decoder method where the 3-D wavelet transform to find high- and low-frequency regions. Turbo channel coding is applied for encoding high-frequency region (represents antierror) and 3D-set partitioning in hierarchical trees (SPIHT) for low-frequency region (image energy). The decoder uses low-frequency information to predict high-frequency coefficients. It also creates side information that is jointly decoded. Guerra et al.²⁰ proposed a lossy compression algorithm named HyperLCA using transformation function to achieve better CR at the cost of reasonable computational complexity. It has three steps namely spectral transformation function to remove spectral redundancy followed by preprocessing stage and lossless encoding as its last step of compression. Most different pixels are selected in preprocessing stage, which can be coded independent of any spatial alignment. Golomb rice coding is used to preserve this information until the image has been finally decompressed. Integer HyperLCA²¹ was later proposed as an extension to the original algorithm that improves the performance further by dividing parameter’s floating point values into an integer part and a decimal part. Luminance²² transform proposed a transformation function considering an assumption that intensity of light falling on various bands of the same HSI is almost equal. The authors used luminance transform to reduce the difference of brightness and contrast between spectral bands at the same spatial location. DCT was then applied on the resultant image to minimize the spatial correlation in HSI. The results obtained were better than applying only DCT to raw image. An extension²³ of wavelet-based transformation was proposed by Khan et al. The method used 1-D convolution to decompose the image temporally and fractional wavelet filter (FrWF) transform to remove spectral and spatial correlations, and the coefficients are then quantized. The factors are grouped as significant and nonsignificant using dyadic wavelet transform. Then, it employs 2-D SPIHT to encode these coefficients using a tree-based orientation that can represent insignificant coefficients using a single value. A lossless compression algorithm, regression wavelet analysis-clustered (RWA-C),²⁴ was proposed by Ahanonu et al. that used cluster analysis to divide the image into $N$ clusters. Wavelet transformation is applied on these clusters to decorrelate spectral information and obtain wavelet coefficients. Linear regression is used on spectral coefficients within a cluster, and significant factors are found through least square regression. Memory requirement of techniques is an important issue that is taken care by 3D-LMBTC¹¹ to encode a higher bit plane to lower bit plane using wavelet coefficients. A block is matched to every other block to find the significance that is encoded in further steps.

An integer-based hybrid transformation method²⁵ ILKT-IDWT has the following steps. The input HSI is first converted into multiple 1-D vectors that are clustered and tiled using eigen matrix decomposition. Invertible integer KLT map is then applied to the spectral matrix, followed by an integer DWT to spatially decorrelate the image data. Three different wavelet-based coding are proposed to be implemented on the decorrelated image, they are spatial-oriented tree wavelet (STW), wavelet difference reduction (WDR), and adaptively scanned wavelet difference reduction. In the first method, coefficients are ordered from high magnitude to lower magnitude in pyramid structured tree, making it a complicated process. Second method discovers three categories of arrays using an iterative approach that divides threshold value by 2 each time. Third method applies an adaptive way of scanning among different arrays. Computational complexity of HyperLCA method was reduced in the method given by Díaz et al.²⁶ The proposed approach utilizes the parallelism in HyperLCA algorithm and performs execution on the Nvidia Jetson TK1 and TX2 GPU. Three models of parallel implementation are proposed to accelerate compression. First model executes transformation on GPU while performing all the steps of HyperLCA sequentially. In the second model, coding and transform are executed by different central processing unit (CPU) processes. Third model implemented the transform using three threads of GPU and each code block on different CPU process. Support vector regression (SVR)-based compression was implemented in SVR-DWT,²⁷ where 3-D DWT is applied on the input HSI followed by SVR on normalized coefficients to identify the support vectors and weights for spectral information. Weights are quantified using floating point quantizer and encoded by entropy coding technique. Spatial data are separately encoded by lossless differential pulse code modulation (DPCM) to preserve the low-frequency image details. Decompression stage is exact reverse of compression stages. Another transformation function named graph Fourier transform (GFT)²⁸ is used to decorrelate the HSI in spectral domain. Laplacian matrix is used to obtain the transformation vectors uniquely for each signal. Impact of GFT on correlation is calculated to assess the quantization value for Gaussian Laplacian vectors that are selected depending on the amount of loss permitted by the application. Fuzzy logic²⁹ has also been used in compression by Monica and Widipaminto. The proposed method modifies the fuzzy transform that finds a correspondence between a set of $n$-dimensional vectors and continuous functions with the help of membership functions. Existing Perfilieva’s fuzzy transform uses sinusoidal membership function that is modified to pseudoexponential function. The input image is first broken down into frames of size ($8\times 8$) and pixels are normalized into [0, 1]. Each frame is then transformed into $n$-dimensional matrix using fuzzy transform. The advantages, limitations, and future directions of each algorithm are listed in Table 1.

Table 1

Transform-based HSI compression techniques.

Research challenges and future directions

This technique can be applied to both data-center and onboard compression due to fast calculations. It has several advantages such as error-tolerance mechanism, high compression performance, flexible in using rate-control mechanism, and global optimum solution. Disadvantages³⁰ of transform-based compression include high computation time as it performs a large number of computations such as multiplication, transpose, and the inverse of matrix. Optimal performance can be obtained at low bit-rates (BR) only. It³¹ destroys the inherent structure of HSI and gives rise to high-order dependency since it considers the image as a matrix.

The computation time of transform-based technique can be reduced by exploiting parallelism in the algorithms and applying it on high-performance computing (HPC) architecture, which can improve its performance. In multitemporal HSI, time domain is also available along with spectral and spatial domains that give rise to temporal correlation. Four-dimensional (4-D) transform-based techniques need to be developed to address such issues. Existing algorithms can be implemented using the latest transformation function with the objective of performance improvement.

Overview

It is an alternative to transform-based algorithms with technical and implementation benefits. In this technique, the value of a pixel is predicted after applying some mathematical functions to the previous pixels. It is developed especially for 3-D images, exploits correlation in both spatial and spectral directions, and removes them. Prediction³² in HSIs is mainly applied on spectral-domain with the help of a filter after spatial decorrelation gets completed. Mostly used filter functions to calculate weight matrices are recursive least square (RLS) and least mean squares (LMS) filter. Prediction-based algorithm is also used in conjunction with other algorithms to improve performance. Some state-of-the-art algorithms are 3D-DPCM,³³ superpixel-based segmentation-CRLS,³⁴ RLS-adaptice length prediction,³⁵ and LMS-APL.³⁶

Technique

Prediction-based technique is easy to implement on HSIs and can be easily explained by Fig. 3. The first step removes correlation in the spatial domain for all bands, followed by a prediction of pixels of $p$’th band by performing some mathematical operations on pixels of $p\text{-}1$ bands. A weight matrix is used for this purpose, generated by a filter function depending on the algorithm. Residuals are calculated by subtracting the original image from the predicted image, which is encoded by entropy or Golomb coder.³⁷

Fig. 3

Steps of prediction-based algorithm.

Methodology used by some prediction-based algorithms is described below. Table 2 lists the algorithms with their advantages, limitations, and some future directions.

Table 2

Prediction-based HSI compression techniques.

The standard developed by working group of multi/hyperspectral data compression, consultative committee for space data systems (CCSDS), for HSI compression techniques to be used in space missions, named as, CCSDS-123.0-B is based on the predictive compression technique. Issue 1 of the method was introduced in 2012 that focused on the lossless compression of images captured by multiple satellites. Significant limitation of the standard was considerable compression time taken during the process that led to the development of various modified techniques, some of them are discussed here. An enhancement in CCSDS standard algorithm was proposed by Conoscenti et al.³⁸ by introducing three application-specific extensions such as constant signal-to-noise ratio (SNR), rate control, and hybrid coding. Low energy areas with tremendous noise have been removed from the prediction process by keeping an upper bound on the relative error. Rate control algorithm proposed in the article is a low complexity method that gives the user a control over accepted loss. Finally, a hybrid encoder has been proposed to improve the coding performance. Zhaoa et al.³⁹ proposed an approach to predict the pixels of HSI. Input image is partitioned into a group of bands (GOB) using segmentation techniques, and intraband prediction is applied to the first band in each GOB to remove the spatial correlation. Rest of the bands are passed with fractal encoding that performs interband prediction using the local search algorithm. Fractal parameters and residual error thus generated are transformed and quantized further using DCT to further remove redundancy in spatial axes. The coefficients are further processed with entropy coder to generate bitstreams. Bogdan et al.⁴⁰ evaluated the performance of CCSDS 121 predictor by implementation on various hardware accelerators field programmable gate array (FPGA). Its results can be used to design a requirement-based FPGA for different satellites. Skip block-based distributed source coding (SB-DSC)⁴¹ technique has been used that uses multiple encoders to code different blocks of an image after calculation of absolute error of the image. DWT is applied on the raw input 3-D image to separate the low- and high-frequency pixels, which can then be separately coded based on 3-D SPIHT and transmitted through different channels using Turbo channel coding. Pixels blocks with mean absolute error less than three and maximum absolute error less than four are skipped from coding.

3-D multiband linear predictor (MBLP)⁴² algorithm is proposed to identify redundancy in third dimension from the information of already predicted bands using the two algorithms namely 2-D linearized median predictor and MBLP. Residuals are then coded using PAQ8 algorithm in conjunction with arithmetic coding. A compression method based on the binary tree data structure is proposed by Shahriyar et al.⁴³ that decomposed HSI into similar sized blocks stored in binary tree. Entire block is coded by arithmetic coding to reduce the size of compressed data. Shen et al.⁴⁴ proposed a method that removed redundancy in the spatial domain by chaos small-world algorithm. Spectral decorrelation is done with the help of RLS filter in which boundary pixels and normal pixels are processed with a different technique. Multivariate Gaussian distributions encoding is used to generate bit-stream. RLS filter is also used in RLS-optimal prediction band (OPB)-P³⁵ to predict the pixel values with varying number of bands. The selection of optimal number of bands, to be used for all the bands, is calculated from the spectral signature of first band. The method has been implemented on GPU to reduce the compression time and optimize the intermediary operations. Fjeldtvedt et al.⁴⁵ proposed a hardware implementation of CCSDS-123 standard, which performs local sum, local difference, and directional difference beforehand. The dot product of weight matrix and already calculated central difference is explicitly performed on hardware. Residual mapping, encoding, and packing are done in the last stage of compression to reduce the overall complexity of hardware. Barrios et al.⁴⁶ proposed a different implementation of lossless CCSDS algorithm on various high-level synthesis tools. The results of each have been compared with existing algorithms and suggestions are provided. Super RLS³⁴ method includes some steps such as intraband encoding, superpixel segmentation, vectorization, RLS prediction, and entropy encoding. In the first step, spatial correlation is removed from the input image by subtracting the arithmetic mean of neighborhood pixels from each pixel in each band. Then, a segmentation algorithm is proposed to partition the image into small regions based on some similarity, and leading pixels in each area are called superpixels. These superpixels represent the entire block to which they belong, and vectorization technique is used to generate supervoxel for each area. RLS prediction is used to create residuals for each voxel that is entropy coded at the compression end.

Prediction based on data dependency is proposed in long short-term memory-recurrent neural network (LSTM-RNN)⁴⁷ algorithm. Since weight of the filters used in prediction is dependent on the previous weights, this method trains a network to learn the time series data obtained in the form of weights. Then, prediction is applied on the input image and context-based conditional average prediction is used to reduce first-order entropy. Following which, adaptive filtering is employed that uses gradient descent algorithm to minimize the residuals. C-DPCM-RNN³³ is another technique that uses neural network to predict the pixels of spectral bands after sufficient training. In the algorithm, different predictors are used to predicting each spectral line. First band is directly encoded and transmitted, from second band to $N$’th band. C-DPCM is used to generate the residuals. ($N+1$)’th band onward, a trained deep neural network is used, where $N$ is the prediction order selected after obtaining the training accuracy. Li et al.⁴⁸ proposed a faster implementation of prediction-based compression C-DPCM on GPU. The algorithm proceeds by clustering of spectral lines into $M$ classes using $k$-means algorithm. Calculation of prediction coefficients for each class using traditional DPCM on GPU, followed by encoding of the residual image. RWA-C²⁴ already discussed in transformation-based technique can be classified in this category. Afjal et al.⁴⁹ evaluated the effect of band reordering on context-based adaptive lossless image coder and lossless CCSDS prediction-based compression algorithm. Band reordering is the technique that rearranges the band sequence in the image to be compressed to code good predictor bands first that will affect the prediction of later bands. Three different approaches used to find optimal band reordering are proposed that are based on various heuristics. Band reordering based on consecutive continuity breakdown heuristics (BRCCBH) obtains the highest correlated bands first and rest bands are arranged in decreasing order of correlation. Band reordering based on weighted-correlation heuristic assigns some weight values to current bands in the reordered list and uses weighted correlation factor initialized either by first band or maximum correlated pair. Based on segmented of bands (BRSB) divides the bands in set of multiple segments using average correlation value, which is followed by BRCCBH for band reordering. Rodriguez et al.⁵⁰ proposed another technique for hardware acceleration of lossless CCSDS 123 algorithm. It uses dynamic and partial reconfiguration-based architecture that manages HyLoC, a low complexity compressor core for fast and real-time compression. The number of cores can be modified according to the requirements of application that make it a ready to use hardware platform. Prediction-based reconstruction is proposed by Cang and Wang⁵¹ in the article utilizing compressed sensing and interspectral reconstruction. Similar bands are grouped by correlation factor, and a standard band is selected in each group (second band is generally chosen due to band correlation). Gaussian matrix is used to sparsely represent the standard reference band. General bands are predicted from the reference band itself iteratively by reducing the error until considerable lossy image is obtained. A low complexity predictive lossy compression algorithm is implemented on space qualified hardware accelerator Virtex-5 by Bascones et al.⁵² The algorithm is highly pipelined with minimum use of FPGA and multiple steps working together.

Research challenges and future directions

Prediction-based compression has several benefits over transform-compression such as low-complexity, better performance for average, and large BR.³⁰ It supports optimum global results, as an entire image is used at once by the algorithm for prediction. Near-lossless compression can be achieved by prediction compression with the help of a quantizer.⁵³ This technique has many drawbacks such as low performance, poor fault tolerance, and error propagation, and images are processed only after conversion of 3-D matrix into 2-D matrix that too on the small neighborhood. Performance of existing algorithms can be improved by the development of hybrid algorithms, i.e., combining two or more techniques, including new filters and considering an optimal number of prediction bands. Error mapping with residuals and selection of learning parameters with compressed sensing technique can significantly increase the performance. Modifications in prediction-based HSI compression can lead to an optimal solution for all applications.

Overview

VQ is a data compression technique that takes 3-D HSI data cube as input and returns a compressed image. Two significant steps of VQ are training (codebook generation) and coding (code vector matching).^54,55 Quantization is mostly used along with transform-based or learning-based techniques as it uses a training algorithm to generate an optimal codebook. Algorithms based on VQ have high complexity, so the principal objective of the method is to develop an efficient algorithm that has fast execution.⁵⁶ State-of-the-art algorithms in the field are vector quantization principal component analysis (VQPCA)⁵⁷ and online learning dictionary.⁵⁸

Technique

Compression using VQ can be divided into three phases:⁵⁹ the first phase that generates codebook is called the design phase. The second phase is the encoding phase in which HSI is taken as input and converted to blocks and then to $n\text{-}D$ vectors. Followed by a search algorithm that is used to find an optimal vector in codebook with minimum distortion and its index is sent to the receiver. Encoding phase can be better understood by Fig. 4. After this third and last phase starts called the decoding phase, the index at the receiver is searched in codebook already present at the decoder end, and code-vectors are regenerated to reconstruct entire image.

Fig. 4

Encoding phase of vector-quantization technique.

A state-of-the-art VQ method was proposed by Li et al.⁶⁰ in which input pixels are clustered using correlation vector (CV). Least square residual is then used to predict the spectral bands of each cluster. The residuals are encoded using the concept of VQ with minimal side information. The algorithm has been implemented and evaluated for most of the applications. Báscones et al.⁵⁷ proposed a algorithm (VQPCA) based on the concepts of wavelet transform, VQ, dimensionality reduction techniques such as PCA, and standard spatial compression technique JPEG2000. The raw pixels are decorrelated using VQ, which are then passed by PCA to obtain the important components in a few bands. JPEG2000 is applied to the bands with maximum information and the resultant is transmitted or stored after entropy coding. It results in a lossy image compression technique, which can be extended to near-lossless one after slight modification.

Table 3 presents their advantages, limitations, and future directions.

Table 3

VQ-based HSI compression techniques.

Research challenges and future directions

Some advantages of the technique are near-lossless compression algorithm and better compression performance. It also has some challenges associated with it like the requirement of substantial resources for codebook generation, more processing time⁶¹ to convert a large number of pixels into vectors. Due to seasonal change and atmospheric effects, a single codebook cannot meet the demand of onboard compression and generation of various small codebooks is costly.

Overview

The technique is famous for on-board compression algorithms as it shifts the computation complexity of encoder to the decoder. It is used in real-time compression as it senses a small chunk of data, compresses it, transmits the compressed data to the receiver, and then accepts another piece. State-of-the-art algorithms for compressed sensing are sparsification of HSI⁶² and reconstruction (SHSIR), reweighted Laplace prior-based HCS (RLPHCS), OMP, and structured sparsity (SSHBCS) that show better performance for small BR. The main objective of compressive sensing is to reduce memory usage during computation.⁶³ It can also be used as the hardware-based or traditional⁶⁴ software-based.⁶⁵

Technique

Some algorithms based on compressive sensing are listed in Table 4 for better comparison. Compressive sensing algorithms have the concept of different encoding and decoding algorithms, which can be described using three steps. HSI signals are sensed at the encoder, and very few samples (sensing matrix) of this 3-D image are converted to a 2-D matrix of dimension $N\times B$, where $N=$ number of pixels and $B=$ number of bands. This matrix is converted to a 2-D matrix of much smaller dimension by applying different algorithms. This small matrix is encoded and transmitted through a channel. Then, next part of the same image is sensed, and the process is repeated until the entire image is sent to the decoder. The decoder reconstructs all the samples together and thus has high complexity.

Table 4

Compressive sensing-based HSI compression techniques.

An existing technique proposed by Xu et al.⁶⁶ divided the input HSI into blocks, with each block having its reasonable bit rate. Multiple linear regression is applied to obtain side information for each one. Optimal quantization step size is assigned, which can help in efficient decompression separately. CSDL-JP2⁶⁷ is another state-of-the-art compressive sensing technique in which matrix of measurement code is used to generate a database for coded snapshots. Real-time compression is done by deciding on encoder from snapshot database. A deep neural network is used by the sparse recovery algorithm to regenerate the original image. Gunasheela and Prasantha+⁶² proposed SHSIR method with the following steps. Image is first represented in 2-D matrix of size ($P\times B$) dimension, where $P=$ number of pixels per band and $B=$ number of spectral bands. Then, compressive sensing is applied in spectral axis. Linear mixing model is used to approximate the resultant matrix. Parameters of which are optimized by Bragman iteration. This method was extended by generating spectral vectors for each spatial pixel. The compressed image using SHSIR⁶² is modeled by the linear operator, and convex optimization is used with compressive sensing technique to improve the performance. his-CSR⁶⁸ is a method that can reconstruct the original values of pixel by sensing a small part of it. The algorithm consists of two stages namely sensing and reconstruction. A random matrix is used to obtain the measurements, which is combined with the parameters and multipliers to get the initial image. Blocking technique is then used to group the tensor cube, its output is $k$-NN classified followed by stacking. It is followed by reconstruction step where nonlocal similarity and low rank approximation is utilized to regain the original image.

Research challenges and future directions

This technique has many benefits as low encoder complexity, smaller memory requirements, low bandwidth for transmission, and better performance. Contrary to this, there are many challenges in the technique such as expensive decompression, identification of the sensing matrix as it should follow isometric, and full rank property.⁶⁹ Reconstruction hisHSI⁷⁰ is a very complex process involving spectral unmixing and convex optimization. Its use in real-time compression can be a great achievement if bulky computation devices can reduce decoder complexity. Another future dimension opens up due to the fact that most of the applications require synchronized rate of encoder and decoder that has not yet been considered in any article.

Overview

Tensor decomposition is one of the latest techniques for image compression that gives high performance compared to traditional methods. Tensor could be considered as an $n$-dimensional matrix, which can be decomposed very easily. In this technique, HSI is stored into 3-D tensor ($Y$) and one of the TD techniques⁷⁰ is applied to decompose the 3-D tensor ($Y$) into lower dimension tensors ($X$). Decomposed tensor is then encoded and transmitted through the channel. Some state-of-the-art algorithms of the techniques non-negative tucker decomposition (NTD)⁸-DWT, convolution neural network (CNN)³¹-NTD, and NTD-DCT¹² have shown excellent results.

Technique

The technique is mostly applied along with some techniques such as transform-based, learning-based, or prediction-based. The steps of the algorithm are described in Fig. 5.

Fig. 5

Tensor decomposition compression technique.

Genetic algorithm-based compression technique has been proposed by Karami et al.⁷¹ named particle swarm optimization (PSO) NTD. The algorithm follows the procedure in which NTD is applied to the original image, which is then combined with the linear mixing model. It generates a smaller tensor with reduced dimensions and three factor matrices that is a temporary decomposition. The optimization problem is solved by applying genetic algorithm (GA) with various parameters and multiple mutations are performed to obtain a final optimized solution. The objective is to minimize the root mean square difference between multiple matrices of input and decompressed image. Rajan and Murugesan proposed a hybrid algorithm DWT-TD-ALS-RLE⁷² based on adaptive least square (ALS) and run length encoder (RLE). In the method, 2D-DWT is initially applied on each band across spectral domain to remove redundancy. Coefficients of each band are combined to generate a 3-D tensor, which undergoes TD along with ALS to obtain reduced tensor with minimum error. It is followed by RLE to generate bitstream that can be transmitted or stored in comparatively less bandwidth or memory. The original image can be reconstructed during decompression. Dictionary learning technique is used in tensor decomposition in multidimensional block-sparse representation and dictionary learning (MBSRDL).⁷³ his is initially represented as a 3-D tensor and three dictionaries are trained using the sophisticated dictionary learning algorithm. Both spatial and spectral domains are compressed separately using TD. Tensor decomposition has also been used along with deep learning technique in CNN-NTD,³¹ where CNN-based transform is proposed to transform large-scale spectral tensor into small-scale. Then, NTD is applied to further reduce the dimensionality of small scale tensor obtained in first step. Resultant tensor has been transformed into frequency domain using 3-D DCT to remove spatial and spectral correlation. Entropy encoding is used to generate bit streams from the high energy coefficients. Aidini et al.⁷⁴ proposed another tensor-based compression in which compressed image is quantized and transmitted to Earth, where it is decompressed and analyzed for processing. Tensor recovery algorithm proposed in this article is an extension of quantized matrix recovery that obtains the original dimensions of the image. It is then passed to super-resolution algorithm that uses coupled dictionary learning to regain the original pixel values. The problem of identification of matrix pertaining to a particular signal is solved using alternative direction multiplier method (ADMM). A CNN network is proposed to learn spatial features from high-resolution images that obtain remarkable results. Critical analysis and future scope of the tensor-based technique are listed in Table 5.

Table 5

Tensor decomposition-based HSI compression techniques.

Research challenges and future directions

These algorithms have high compression performance with reduced run-time but it suffers from many limitations. High computation complexity, manual parameter updating procedure, data dependency, etc. are some of the challenges in the method. The future scope in this technique is to utilize the parallelism existing in the algorithm for parallel implementation. It can also be extended to automate the process of selection of dimensions to compress image at a particular rate. Also, more hybrid algorithms combining tensor-based compression and other such techniques can be developed.

Overview

The technique compresshisHSI using very few values from a range of pixel values using some quantization method and values near to zero are dropped. It helps to reduce the use of storage and bandwidth by coding only a small set of values. It is mostly used in the classificatihisof HSI as features could be separated by a distinct boundary when sparse representation is used. The technique helps in the ROI-based compression when combined with learning-based compression. Some state-of-the-art algorithms in the technique are compressive-projection principal component analysis,⁷⁵ GIST, SpaRSA,⁷⁶ spectral–spatial adaptive sparse representation (SSASR),⁷⁷ and TwIST.⁷⁶

Technique

Sparse coding is used by multiple algorithms in different style,and a generalized and in-depth description of the technique is given in Fig. 6. First step of the algorithm is vectorization in which pixels with different features are mapped and converted to vectors. Next step is sparse coding where these vectors are converted to sparse vectors, which are then encoded into bit-streams. Algorithms based on sparse representation within the scope of this article are described below. Table 6 provides their advantages, limitations, and future research directions. SSASR method was proposed in 2017 for transformation of spectral signatures of pixels to sparse coefficients, most of which are zero. Superpixels are obtained from the image to divide a large size image into multiple blocks of small size, which are converted into vectors of equal size. Adaptive sparse coding is then applied to generate sparse coefficients. These pixels are quantified by discrete quantization, which are then encoded by Huffman coding to generate the bitstream. Jifara et al.⁷⁷ proposed a method based on the spectral curve, which is unique for different materials. Spectral curve is described by sparse dictionary that gets updated using the concept of online learning. It is a lossy compression technique that uses the proximity-based optimization technique. Online dictionary learning⁵⁸ reduces the time and cost associated with large size of dictionary coding and transmission, and it learns iteratively by selecting one item from training set. An advantage offered by this method was sparse representation of the spectral curve of blocks of pixels. SpaRSA and GIST proximity-based optimization algorithms have shown the optimum results for the purpose of anomaly detection.

Fig. 6

Sparse-based technique.

Table 6

Sparse representation-based HSI compression techniques.

Fu et al.⁷⁸ clustered original pixels into general-pixels represented by simultaneous sparse coding, which gives only nonzero coefficients. Coefficients are quantized using some threshold value that is a bit rate deciding factor. Quantizer gives the control to user to decide the quality of the reconstructed image by modifying the bit rate. Quantized coefficients are further compressed by DPCM filter and converted to binary bitstream by Huffman coding. Another state-of-the-art algorithm for sparse representation algorithm is SpaRSA⁷⁶ that has the following steps. One element from the training set is taken at a time to update the dictionary of pixels. Sparse representation is used to store the coefficients with a loss function of optimization algorithm. Dictionary update and dictionary learning are the two algorithms used to minimize the loss function for application-specific compression. CSDL-JP2⁶⁷ categorized under compressive sensing technique is also an example of sparse representation algorithm that has very high computational complexity.

Overview

Multitemporal HSI is a set of HSIs⁷⁹ collected from the same location at a different time. A new temporal (or time) domain gets added to the original 3-D image matrix, forming a 4-D matrix. It can be thought of as video compression, but the concept of video and multitemporal is entirely different. Compression of the 4-D image⁸⁰ is called multitemporal compression or 4-D compression, which is very important for military operations, disaster management, prevention from calamities, space observation, etc.

Technique

4-D HS image with details is given in Fig. 7, where $(x,y)$ and $z$ represent spatial and spectral domains, respectively, and $T$ represents time domain. All four parameters are variable in a 4-D image.

Fig. 7

Multitemporal HSI.

Multitemporal compression⁸¹ is obtained by enhancing the 3-D prediction-based technique to 4-D prediction based for temporal decorrelation. Lossless compression is expected in 4-D images as these are processed and used by automated programs running on the computer. Methodology of some state-of-the-art algorithms is presented below. Table 7 provides their advantages, limitations, and future research directions. Zhu et al.⁷⁹ proposed a compression algorithm applicable on temporal HSIs using the concept of change detection. A reference image is selected among the multitemporal images that should be present at the decoder end. Matrix operations are used to detect change in temporal domain of images with respect to the reference image. It suggested three techniques for efficient compression of 4-D images starting from likelihood ratio of the detected to spectral concatenation and independent approach. Two temporal HSIs are concatenated on the spectral axis forming a single HSI with twice the number of bands as original in the spectral concatenation method. Independent approach ignores the reference image at all during spatial decorrelation. In all the three techniques, PCA, SubPCA, and DWT spectral transforms are applied on the images to improve the compression performance by reducing the number of bands and spatial data to be coded.

Table 7

Multitemporal-based HSI compression techniques.

Shen et al.³⁶ proposed an adaptive learning-based compression of multitemporal HSIs. It used correntropy least mean square (CLMS) algorithm for prediction of pixels on the basis of already predicted spectral and temporal information. The performance of the method is comparatively high due to the presence of temporal correlation. It ensures lossless compression by coding the prediction error using Golomb rice coding and arithmetic coding. The method was further improved with the help of fast-lossless-4D predictor⁸¹ in which pixels are predicted by using a linear combination of neighboring pixels in all four dimensions. By subtracting data with their local mean in each band and then for each pixel in a band in raster scan order. Taking its residual and finally updating the weight parameter. Residuals are encoded by entropy coder at the compression end and the decompression follows an exact reverse of the compression steps.

Research challenges and future directions

Implementation of 4-D compression algorithms is quite easy as traditional 3-D algorithms are extended to the fourth dimension. Complexity that includes running time and computation resources is very high for these algorithms. Future scope of the technique is to develop hybrid algorithms combining prediction-and learning-based techniques. Performance of transform-based technique can be evaluated in the temporal domain.

Overview

It is one of the most popular techniques as it involves machine learning and deep learning in compression. The method has always been studied along with prediction-based technique as it also predicts pixel values. However, it has widely known features that learn and update parameters automatically. It⁸² is used with all other techniques after little modifications and achieves better performance. Some acclaimed machine learning algorithms applicable in the technique are SVM,¹³ artificial neural network (ANN),⁸³ backpropagation network,⁸⁴ CNN,³¹ independent component analysis (ICA)/PCA,⁸⁵ and clustering algorithms.¹⁶

Technique

Table 8 provides the advantages, limitations, and future research directions of these algorithms. Figure 8 shows general steps of learning-based algorithm using CNN and some transformation function to compress an his. In the first step, CNN is applied to compress 3-D data cube representihisHSI. Then, transform-based algorithm does domain transformation of a smaller 3-D cube and then coefficients are encoded. In backward CNN, original image is reconstructed by applying CNN again with minimal error.

Table 8

Learning-based HSI compression techniques.

Fig. 8

Steps of CNN-based compression technique.

Research challenges and future directions

The method sustains high complexity of machine learning and deep learning algorithms. It also requires more resources but can be easily implemented in hardware and other HPC architecture. The method can be improved by implementing the fundamentals of deep learning and advanced machine learning in compression and developing more hybrid algorithms with automated processes.

Lossy

Lossy compression can be defined as the process of compression in which the original image cannot be restored in the reconstructed image. Some information is discarded at the compression end, which cannot be recovered during decompression. It is mainly used when the application is error-tolerant, i.e., specific loss in data has no effect on the output. It results in high compression performance by reducing the size of the compressed image and thus maintains a trade-off between space and precision. Lossy compression is dominated by hardware implementation.⁸⁹

Lossless

It consists of the techniques that can precisely reconstruct the original image without any loss of information. Used in applications where even small loss in error is not acceptable such as military operations, global positioning system (GPS) tracking, and target identification. Lossless compression⁹⁰ results in reduced compression performance, i.e., CR. In HSI compression techniques, lossless compression is preferred as these images store important information that is used in the analysis, classification, target identification, etc.

Near-lossless

The term has been used interchangeably as “controlled-lossy” compression, which means a loss in information can be controlled according to desired compression performance. It can be understood as a fuzzy set between lossy and lossless compression that changes its form according to the application. It can be used in medical imaging, remote sensing, etc. Moreover, very few works have been done in this field, and the area needs more exploration.⁹¹

On-board compression

Compression performed on raw signal/images at the source of acquisition is termed as onboard compression. Satellites or air-borne devices can carry minimal resources that too can be affected by radiation. So these algorithms are devised to perform in resource constraint environment.

Data center compression

HSIs are compressed at receiver, where the resources are available in bulk. Algorithms executed in this environment need not suffer from scarcity of resources and thus have better performance.

Region of interest-based compression

ROI-based compression can be stated as the technique in which part of the image is compressed with high BR and remaining portion with a small BR. A portion of the image containing vital information is identified in the first step of such algorithms. There can be several such parts that can be compressed with the same algorithm but different BR depending on the significance of information stored in it.

Full image compression

It is a technique in which the entire image is compressed with the same BR, and no target identification is needed in advance. Performance of these algorithms is not as good as the performance of ROI-based algorithms.

For transmission

Compression performed to transmit signal at some other location requires a stream of bits along with header and side information. The algorithm developed to generate a stream of bits that should not be used to construct compressed image to preserve time and resources.

For storage

The compressed data are stored for future use, which can be reconstructed to the original image when needed. Increased steps in these algorithms qualify it to be classified into a different category.⁹²

3-D data cube

Algorithms falling in this category consider HSI in 3-D cube or cuboid and directly apply steps of compression. Spatial and spectral decorrelation need not be performed separately in this case.

2-D compression

Some algorithms can only be applied only on 2-D array, so they first convert 3-D-HSI to a 2-D array. There are two approaches for the conversion, in the first approach, each 2-D band (of size $p$, $q$) is converted to 1-D vector (of size $p\times q$) in raster scan order and each band is appended columnwise. The second approach is to remove correlation in spectral dimension and apply 2-D compression algorithm in each band, considering each band as a separate image.

Sequential implementation

These algorithms do not require any specialized hardware or machine for implementation. They get executed with high run-time on a regular machine.

Implementation on HPC architecture

Algorithms having independent blocks (that can be executed concurrently with other blocks) is implemented on HPC architecture with reduced run-time. There are three types of architectures on which algorithms are executed with little modifications.

An HSI compression algorithm may come under more than one category according to its features. Table 9 classifies algorithms according to these categories, where some abbreviations are used. Compression performed at the location of acquisition and at the data-center is represented by onboard and DC, respectively. ROI represents the ability of the algorithm to support ROI-based compression. Algorithms considering input image as 3-D data cube or transforming 3-D tensor to 2-D matrix for compression are represented under the strategy. Implementation environment where algorithm is implemented is sequential implementation (seq), shared memory/distributed memory (SM/DM), GPU, and hardware accelerators such as FPGA. This categorization helps us to find out the scope for future research according to exploration level.

Table 9

Categorization of compression algorithms based on various parameters.

Table 10

Datasets and evaluation metrics used by different algorithms.