# Learning-Based Hyperspectral Imagery Compression through Generative Neural Networks

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Methodology

_{r}admits a density, and it is supported by a low-dimensional manifold. Rather than estimating distribution p

_{r}, we define a random variable X with a fixed distribution p(x) and pass it through a parametric function g

_{θ}: X → Y (the GNN) that directly generates the HSI following some distribution p

_{θ}. By changing parameter θ, we can change the distribution and make it as close as possible to the real distribution p

_{r}.

_{i}(x) calculated as the number of pixels with a specific integer value divided by the total number of pixels in the HSI. Note that the discrete formula in Equation (3) does not converge to Equation (2) when N → ∞.

#### 2.1. Architecture of the GNN

#### 2.1.1. Bilinear Interpolation for Upsampling

_{1}and R

_{2}, are computed by linear interpolation, which is

_{1}and R

_{2}as follows:

#### 2.1.2. Latent Code and Entropy

#### 2.1.3. Model Weight Pruning

#### 2.2. Multiple Images and Huge Image Training

#### 2.2.1. Embedding and Multi-Image Training

#### 2.2.2. Huge Images and Batch Training

## 3. Experimental Result

#### 3.1. Data and Implementation Description

^{2}. The spectral range of the HSI of Xiongan New Area (Matiwan Village) is from 400 to 1000 nm, with 256 bands and a spatial resolution of 0.5 m. The image size is 3750 × 1580 pixels. The whole image is displayed in Figure 8.

^{−4}. There was no dropout layer, because we did not treat overfitting as a problem.

#### 3.2. Generated Image Sequence Demonstration

^{−4}and was measured by mean square error (MSE) using our modified loss function. Initially, the MSE was used during training as a loss function. However, we observed that it was not the optimal metric after several experiments. We employed a loss function that was a power function of the difference between the original and generated HSIs, which was given by

^{−4}and, finally, converged at around 2.7 × 10

^{−4}. In contrast, other values of p did not result in such a fast convergence rate, and sometimes, the loss became extremely large and unstable. For a small value of p, we observed a singularity point in the generated image, even if the total loss was small.

#### 3.3. Huge-Image Training Experiments

#### 3.4. Comparison of Compression Capacity

#### 3.5. Comparison of Different HSIs

## 4. Conclusions and Future Works

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Babu, K.S.; Ramachandran, V.; Thyagharajan, K.K.; Santhosh, G. Hyperspectral image compression algorithms—A review. Adv. Intell. Syst. Comput.
**2015**, 325, 127–138. [Google Scholar] - Babu, K.S.; Ramachandran, V.; Thyagharajan, K.K.; Cheng, K.; Dill, J. Lossless to Lossy Dual-Tree BEZW Compression for Hyperspectral Images. IEEE Trans. Geosci. Remote Sens.
**2014**, 52, 5765–5770. [Google Scholar] - Wang, L.; Wu, J.; Jiao, L.; Shi, G. Lossy-to-Lossless Hyperspectral Image Compression Based on Multiplierless Reversible Integer TDLT/KLT. IEEE Geosci. Remote Sens. Lett.
**2009**, 6, 587–591. [Google Scholar] [CrossRef] - Huang, H.; Shi, G.; He, H.; Duan, Y.; Luo, F. Dimensionality reduction of hyperspectral imagery based on spatial-spectral manifold learning. IEEE Trans. Cybernetics
**2019**, 50, 2604–2616. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Jimenez, L.O.; Landgrebe, D.A. Hyperspectral data analysis and supervised feature reduction via projection pursuit. IEEE Trans. Geosci. Remote Sens.
**1999**, 37, 2653–2667. [Google Scholar] [CrossRef] - Bruce, L.M.; Koger, C.H.; Jiang, L. Dimensionality reduction of hyperspectral data using discrete wavelet transform feature extraction. IEEE Trans. Geosci. Remote Sens.
**2002**, 40, 2331–2338. [Google Scholar] [CrossRef] - Goyal, V.K.; Fletcher, A.K.; Rangan, S. Compressive sampling and lossy compression. IEEE Signal Process. Mag.
**2008**, 25, 48–56. [Google Scholar] [CrossRef] - García-Vílchez, F.; Muñoz-Marí, J.; Zortea, M.; Blanes, I.; González-Ruiz, V.; Camps-Valls, G.; Plaza, A.; Serra-Sagristà, J. On the Impact of Lossy Compression on Hyperspectral Image Classification and Unmixing. IEEE Geosci. Remote Sens. Lett.
**2011**, 8, 253–257. [Google Scholar] [CrossRef] - Sriraam, N.; Shyamsunder, R. 3-D medical image compression using 3-D wavelet coders. Digit. Signal Process.
**2011**, 21, 100–109. [Google Scholar] [CrossRef] - Pickering, M.R.; Ryan, M.J. An architecture for the compression of hyperspectral imagery. In Hyperspectral Image Compression; Chapter 1; Motta, G., Rizzo, F., Storer, J.A., Eds.; Springer: New York, NY, USA, 2006; pp. 1–34. [Google Scholar]
- Taubman, D. High performance scalable image compression with EBCOT. IEEE Trans. Image Process.
**2000**, 9, 1158–1170. [Google Scholar] [CrossRef] [PubMed] - Fowler, J.E.; Rucker, J.T. 3D wavelet-based compression of hyperspectral imagery. In Hyperspectral Data Exploitation: Theory and Applications; Chang, C.-I., Ed.; John Wiley & Sons, Inc.: Hoboken, NJ, USA, 2007. [Google Scholar]
- Rucker, J.T.; Fowler, J.E.; Younan, N.H. JPEG2000 coding strategies for hyperspectral data. In Proceedings of the International Geoscience and Remote Sensing Symposium, Seoul, South Korea, 25–29 July 2005; Volume 1, pp. 128–131. [Google Scholar]
- Penna, B.; Tillo, T.; Magli, E.; Olmo, G. Progressive 3-D coding of hyperspectral images based on JPEG 2000. IEEE Geosci. Remote Sens. Lett.
**2006**, 3, 125–129. [Google Scholar] [CrossRef] - Kulkarni, P.; Bilgin, A.; Marcellin, M.W.; Dagher, J.C.; Kasner, J.H.; Flohr, T.J.; Rountree, J.C. Compression of earth science data with JPEG2000. In Hyperspectral Image Compression; Chapter 2; Motta, G., Rizzo, F., Storer, J.A., Eds.; Springer: New York, NY, USA, 2006; pp. 347–378. [Google Scholar]
- Du, Q.; Fowler, J.E. Hyperspectral Image Compression Using JPEG2000 and Principal Component Analysis. IEEE Geosci Remote Sens. Lett.
**2007**, 4, 201–205. [Google Scholar] [CrossRef] - Ampe, E.M.; Hestir, E.L.; Bresciani, M.; Salvadore, E.; Brando, V.E.; Dekker, A.; Malthus, T.J.; Jansen, M.; Triest, L.; Batelaan, O. A Wavelet Approach for Estimating Chlorophyll-A From Inland Waters With Reflectance Spectroscopy. IEEE Geosci. Remote Sens. Lett.
**2014**, 11, 89–93. [Google Scholar] [CrossRef] [Green Version] - Golub, G.H.; Loan, C.F.V.; Jolliffe, I. Principal Component Analysis; Springer: New York, NY, USA, 2002. [Google Scholar]
- Masters, D.; Luschi, C. Revisiting small batch training for deep neural networks. arXiv
**2018**, arXiv:1804.07612. [Google Scholar] - Brauns, E.B.; Dyer, R.B. Fourier Transform Hyperspectral Visible Imaging and the Nondestructive Analysis of Potentially Fraudulent Documents. Appl. Spectrosc.
**2006**, 60, 833–840. [Google Scholar] [CrossRef] - Radiuk, P.M. Impact of training set batch size on the performance of convolutional neural networks for diverse datasets. Inf. Technol. Manag. Sci.
**2017**, 20, 20–24. [Google Scholar] [CrossRef] - Zhang, L.; Wei, W.; Zhang, Y.; Tian, C.; Li, F. Reweighted laplace prior based hyperspectral compressive sensing for unknown sparsity. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, 7–12 June 2015; pp. 2274–2281. [Google Scholar]
- Radford, A.; Metz, L.; Chintala, S. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv
**2015**, arXiv:1511.06434. [Google Scholar] - Goodfellow, I.J.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative adversarial networks. In Proceedings of the Annual Conference on Neural Information Processing Systems, Montreal, QC, Canada, 8–13 December 2014; pp. 2672–2680. [Google Scholar]
- Danfeng, H.; Lianru, G.; Jing, Y.; Bing, Z.; Antonio, P.; Jocelyn, C. Graph Convolutional Networks for Hyperspectral Image Classification. IEEE Trans. Geosci. Remote Sens.
**2020**. [Google Scholar] [CrossRef] - Gray, R.M. Entropy and Information Theory; Springer Science & Business Media: New York, NY, USA, 2011. [Google Scholar]
- Simonyan, K.; Zisserman, A. Very deep convolutional networks for large-scale image recognition. Available online: https://arxiv.org/abs/1409.1556 (accessed on 4 September 2014).
- Blei, D.M.; Kucukelbir, A.; Mcauliffe, J.D. Variational Inference: A Review for Statisticians. J. Am. Stat. Assoc.
**2017**, 112. [Google Scholar] [CrossRef] [Green Version] - Frajka, T.; Zeger, K. Downsampling dependent upsampling of images. Signal Process. Image Commun.
**2004**, 19, 257–265. [Google Scholar] [CrossRef] - Conrad, K. Probability distributions and maximum entropy. Entropy
**2004**, 6, 10. [Google Scholar] - Faddeev, D.K. The notion of entropy of finite probabilistic schemes (Russian). Uspekhi Mat. Nauk.
**1956**, 11, 15–19. [Google Scholar] - Giles, C.L.; Omlin, C.W. Pruning recurrent neural networks for improved generalization performance. IEEE Trans. Neural Netw.
**1994**, 5, 848–851. [Google Scholar] [CrossRef] - Levy, O.; Goldberg, Y. Neural Word Embedding as Implicit Matrix Factorization. Adv. Neural Inf. Process. Syst.
**2014**, 3, 2177–2185. [Google Scholar] - Mirza, M.; Osindero, S. Conditional generative adversarial nets. arXiv
**2014**, arXiv:1411.1784. [Google Scholar] - Kingma, D.; Ba, J. Adam: A Method for Stochastic Optimization. arXiv
**2014**, arXiv:1412.6980. [Google Scholar] - Chang, C.-I.; Chiang, S.-S. Anomaly detection and classification for hyperspectral imagery. IEEE Trans. Geosci. Remote Sens.
**2002**, 40, 1314–1325. [Google Scholar] [CrossRef] [Green Version]

**Figure 9.**True color-generated hyperspectral images (HSI) 512 × 512 pixels in size at different epochs.

**Figure 11.**Comparison of the power loss function measured using the mean square error (MSE) criterion.

**Table 1.**Comparison of the mean square error (MSE) for different values of p at the 15-thousandth epoch.

Criteria | p = 1 | p = 1.5 | p = 2 | p = 3 |
---|---|---|---|---|

MSE | 3.25 × 10^{−4} | 2.69 × 10^{−4} | 5.83 × 10^{−4} | 5.75 × 10^{−4} |

${l}^{p}$(norm loss) | 1.212 × 10^{−2} | 1.87 × 10^{−3} | 5.83 × 10^{−4} | 3.21 × 10^{−5} |

**Table 2.**Compression quality comparison for different generative neural network (GNN) structures before and after pruning. PSNR: peak signal-to-noise ratio and CR: compression ratio.

Criteria | 2 Blocks | 3 Blocks | 4 Blocks | 5 Blocks | 6 Blocks | 7 Blocks |
---|---|---|---|---|---|---|

MSE | 1.46 × 10^{−4} | 3.05 × 10^{−4} | 3.56 × 10^{−4} | 4.50 × 10^{−4} | 8.71 × 10^{−4} | 6.44 × 10^{−4} |

PSNR | 44.38 | 41.17 | 40.50 | 39.48 | 36.62 | 37.92 |

Size of GNN | 207 MB | 54.7 MB | 16.4 MB | 7.04 MB | 4.79 MB | 4.34 MB |

CR | 0.66 | 2.50 | 8.35 | 19.46 | 28.60 | 31.57 |

MSE(After pruning) | 1.48 × 10^{−4} | 3.09 × 10^{−4} | 3.67 × 10^{−4} | 4.95 × 10^{−4} | 1.01 × 10^{−3} | 9.32 × 10^{−4} |

Size of GNN(After pruning) | 72.5 MB | 20.7 MB | 7.38 MB | 4.9 MB | 4.31 MB | 4.15 MB |

CR (After pruning) | 1.89 | 6.62 | 18.56 | 27.96 | 31.78 | 33.01 |

Compression Ratio | GNN | 3D SPECK | 3D SPIHT |
---|---|---|---|

31.78/32 | 36.62 | 28.91 | 30.16 |

18.56/16 | 40.50 | 36.34 | 37.49 |

6.62/6 | 41.17 | 48.83 | 49.55 |

**Table 4.**Performance comparisons on different hyperspectral images (HSIs) with three and four blocks without pruning.

Name of HSI | MSE | Compression Ratio | PSNR | |||
---|---|---|---|---|---|---|

Matiwan village | 3.05 × 10^{−4} | 3.56 × 10^{−4} | 2.50 | 8.35 | 41.17 | 40.51 |

Pavia center | 3.47 × 10^{−4} | 3.59 × 10^{−4} | 1.45 | 4.22 | 40.62 | 40.47 |

Dc mall | 1.97 × 10^{−4} | 3.29 × 10^{−4} | 1.532 | 4.29 | 43.08 | 40.85 |

Botswana | 2.34 × 10^{−4} | 2.29 × 10^{−4} | 1.92 | 3.21 | 42.33 | 42.42 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Deng, C.; Cen, Y.; Zhang, L.
Learning-Based Hyperspectral Imagery Compression through Generative Neural Networks. *Remote Sens.* **2020**, *12*, 3657.
https://doi.org/10.3390/rs12213657

**AMA Style**

Deng C, Cen Y, Zhang L.
Learning-Based Hyperspectral Imagery Compression through Generative Neural Networks. *Remote Sensing*. 2020; 12(21):3657.
https://doi.org/10.3390/rs12213657

**Chicago/Turabian Style**

Deng, Chubo, Yi Cen, and Lifu Zhang.
2020. "Learning-Based Hyperspectral Imagery Compression through Generative Neural Networks" *Remote Sensing* 12, no. 21: 3657.
https://doi.org/10.3390/rs12213657