# Pushbroom Hyperspectral Data Orientation by Combining Feature-Based and Area-Based Co-Registration Techniques

## Abstract

**:**

## 1. Introduction

#### 1.1. Scientific Context

#### 1.1.1. The Leman-Baikal Project

- Geometric correction: the scan lines collected by the pushbroom scanner shall be properly georeferenced.
- Radiometric correction: the output spectra shall be filtered for inherent noise, and corrected for atmospheric and water-surface reflection effects.

#### 1.1.2. Problem Formulation

- Compute a georeferenced orthomosaic composed with the scan lines collected by the hyperspectral pushbroom sensor.
- Retrieve the corrected orientation parameters. They include Interior Orientation Parameters (IOP: principal distance, principal point of the pushbroom camera, and potentially the distortion parameters of the lens) and Exterior Orientation Parameters (EOP: roll, pitch, yaw, and three position parameters) for all the scan lines.
- Estimate the boresight between the IMU and the pushbroom sensor.

#### 1.2. State of the Art

- GCPs
- A model for the vehicle’s trajectory
- A priori knowledge on the misalignment between the IMU and the pushbroom scanner, which is important since, for low to average quality IMUs, the initial attitude alignment takes an arbitrary wrong value at the beginning of each flight.

## 2. Proposed Methodology

#### 2.1. Pre-Processing Step: Radiometric Matching

#### 2.2. Bundle Adjustment of Frame Images

#### 2.3. Initial Ortho-Projection of the Scan Lines

#### 2.4. Systematic Error Correction

- Bad intrinsic camera calibration (inaccurate focal length, principal point coordinates, or lens distortion parameters).
- Non-negligible boresight between the IMU and the pushbroom camera.

#### 2.4.1. Matching Points with SURF

#### 2.4.2. Interior Orientation and Boresight Estimation

#### 2.5. Geocorrection Using Particle Image Velocimetry

#### 2.5.1. PIV Theory

- Split the two images into a grid of cells, which sizes are given by the user.
- Find the new location of each cell of the first image into the second image, by finding the maximum of the cross-correlation of the cell (normalised in mean and standard deviation) and its neighbourhood in the second image (also normalised).

#### 2.5.2. Application: Elastic Deformation

#### 2.6. Estimation of the Orientation Parameters

^{−2}for the first $2\times {N}_{L}\times {N}_{P}$ constraints (Equation (12)).

## 3. Results

## 4. Conclusions

## Acknowledgments

## Conflicts of Interest

## References

- Muller, R.; Lehner, M.; Muller, R.; Reinartz, P.; Schroeder, M.; Vollmer, B. A program for direct georeferencing of airborne and spaceborne line scanner images. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2002**, 34, 148–153. [Google Scholar] - Bettemir, Ö.H. Prediction of georeferencing precision of pushbroom scanner images. IEEE Trans. Geosci. Remote Sens.
**2012**, 50, 831–838. [Google Scholar] [CrossRef] - Cramer, M.; Stallmann, D. System calibration for direct georeferencing. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2002**, 34, 79–84. [Google Scholar] - Hruska, R.; Mitchell, J.; Anderson, M.; Glenn, N.F. Radiometric and geometric analysis of hyperspectral imagery acquired from an unmanned aerial vehicle. Remote Sens.
**2012**, 4, 2736–2752. [Google Scholar] [CrossRef] - Tuo, H.; Liu, Y. A new coarse-to-fine rectification algorithm for airborne push-broom hyperspectral images. Pattern Recognit. Lett.
**2005**, 26, 1782–1791. [Google Scholar] [CrossRef] - Wang, J.; Ge, Y.; Heuvelink, G.B.; Zhou, C.; Brus, D. Effect of the sampling design of ground control points on the geometric correction of remotely sensed imagery. Int. J. Appl. Earth Obs. Geoinform.
**2012**, 18, 91–100. [Google Scholar] [CrossRef] - Wu, A.; Lee, Y. Geometric Correction of High Resolution Image Using Ground Control Points. In Proceedings of the 22nd Asian Conference on Remote Sensing, Singapore, 5–9 November 2001; Volume 5, p. 9. [Google Scholar]
- Lee, C.; Theiss, H.J.; Bethel, J.S.; Mikhail, E.M. Rigorous mathematical modeling of airborne pushbroom imaging systems. Photogramm. Eng. Remote Sens.
**2000**, 66, 385–392. [Google Scholar] - Lowe, D.G. Object Recognition from Local Scale-Invariant Features. In Proceedings of the 7th IEEE International Conference on Computer Vision, Kerkyra, Greece, 20–27 September 1999; Volume 2, pp. 1150–1157. [Google Scholar]
- Haala, N.; Fritsch, D.; Stallmann, D.; Cramer, M. On the performance of digital airborne pushbroom cameras for photogrammetric data processing—a case study. Int. Arch. Photogramm. Remote Sens.
**2000**, 33, 324–331. [Google Scholar] - Lee, C.; Bethel, J. Georegistration of airborne hyperspectral image data. IEEE Trans. Geosci. Remote Sens.
**2001**, 39, 1347–1351. [Google Scholar] [CrossRef] - Rzhanov, Y.; Pe’eri, S. Pushbroom-Frame Imagery Co-Registration. Mar. Geod.
**2012**, 35, 141–157. [Google Scholar] [CrossRef] - Cariou, C.; Chehdi, K. Automatic georeferencing of airborne pushbroom scanner images with missing ancillary data using mutual information. IEEE Trans. Geosci. Remote Sens.
**2008**, 46, 1290–1300. [Google Scholar] [CrossRef] - Akhtman, Y.; Constantin, D.; Rehak, M.; Nouchi, V.M.; Bouffard, D.; Pasche, N.; Shinkareva, G.; Chalov, S.; Lemmin, U.; Merminod, B. Leman-Baikal: Remote Sensing of Lakes Using an Ultralight Plane. In Proceedings of the 6th Workshop on Hyperspectral Image and Signal Processing, Lausanne, Switzerland, 24–27 June 2014. [Google Scholar]
- Poli, D.; Zhang, L.; Gruen, A. Orientation of satellite and airborne imagery from multi-line pushbroom sensors with a rigorous sensor model. Int. Arch. Photogramm. Remote Sens.
**2004**, 35, 130–135. [Google Scholar] - Yeh, C.K.; Tsai, V.J. Direct georeferencing of airborne pushbroom images. J. Chin. Inst. Eng.
**2015**, 38, 653–664. [Google Scholar] [CrossRef] - Baiocchi, V.; Crespi, M.; De Vendictis, L.; Giannone, F. A New Rigorous Model for the Orthorectification of Synchronous and Asynchronous High Resolution Imagery. In Proceedings of the 24th EARSeL Symposium, Basel, Switzerland, 25–27 May 2004; Volume 24, pp. 461–468. [Google Scholar]
- Habib, A.; Xiong, W.; He, F.; Yang, H.L.; Crawford, M. Improving Orthorectification of UAV-Based Push-Broom Scanner Imagery Using Derived Orthophotos From Frame Cameras. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.
**2017**, 10, 262–276. [Google Scholar] [CrossRef] - Jensen, R.R.; Hardin, A.J.; Hardin, P.J.; Jensen, J.R. A new method to correct pushbroom hyperspectral data using linear features and ground control points. GISci. Remote Sens.
**2011**, 48, 416–431. [Google Scholar] [CrossRef] - Ramirez-Paredes, J.P.; Lary, D.J.; Gans, N.R. Low-altitude Terrestrial Spectroscopy from a Pushbroom Sensor. J. Field Robot.
**2016**, 33, 837–852. [Google Scholar] [CrossRef] - Suomalainen, J.; Anders, N.; Iqbal, S.; Roerink, G.; Franke, J.; Wenting, P.; Hünniger, D.; Bartholomeus, H.; Becker, R.; Kooistra, L. A lightweight hyperspectral mapping system and photogrammetric processing chain for unmanned aerial vehicles. Remote Sens.
**2014**, 6, 11013–11030. [Google Scholar] [CrossRef] - Barbieux, K.; Constantin, D.; Merminod, B. Correction of airborne pushbroom images orientation using bundle adjustment of frame images. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.
**2016**, 41, 813. [Google Scholar] [CrossRef] - Cucci, D.A.; Rehak, M.; Skaloud, J. Bundle adjustment with raw inertial observations in UAV applications. ISPRS J. Photogramm. Remote Sens.
**2017**, 130, 1–12. [Google Scholar] [CrossRef] - Mukherjee, S.; Joshi, P.; Mukherjee, S.; Ghosh, A.; Garg, R.; Mukhopadhyay, A. Evaluation of vertical accuracy of open source Digital Elevation Model (DEM). Int. J. Appl. Earth Obs. Geoinform.
**2013**, 21, 205–217. [Google Scholar] [CrossRef] - Duane, C.B. Close-range camera calibration. Photogramm. Eng.
**1971**, 37, 855–866. [Google Scholar] - Börner, A.; Wiest, L.; Keller, P.; Reulke, R.; Richter, R.; Schaepman, M.; Schläpfer, D. SENSOR: a tool for the simulation of hyperspectral remote sensing systems. ISPRS J. Photogramm. Remote Sens.
**2001**, 55, 299–312. [Google Scholar] [CrossRef] - Bay, H.; Ess, A.; Tuytelaars, T.; Van Gool, L. Speeded-up robust features (SURF). Comput. Vision Image Underst.
**2008**, 110, 346–359. [Google Scholar] [CrossRef] - Adrian, R.J.; Westerweel, J. Particle Image Velocimetry; Number 30; Cambridge University Press: Cambridge, UK, 2011. [Google Scholar]
- Soria, J. An investigation of the near wake of a circular cylinder using a video-based digital cross-correlation particle image velocimetry technique. Exp. Therm. Fluid Sci.
**1996**, 12, 221–233. [Google Scholar] [CrossRef] - Thielicke, W.; Stamhuis, E. PIVlab–towards user-friendly, affordable and accurate digital particle image velocimetry in MATLAB. J. Open Res. Softw.
**2014**, 2. [Google Scholar] [CrossRef]

**Figure 1.**(

**a**) Illustrates the setting for our airborne acquisitions. (

**b**) Represents the different frame coordinates for the three sensors: the frame camera, the pushbroom (PB) camera, and the IMU.

**Figure 3.**Processing of the frame images using Agisoft Photoscan: (

**a**) alignment of frames and computation of the orthophoto; and (

**b**) computation of the DEM of the area (Selenga Delta Village).

**Figure 4.**Orthoprojection of the scan lines (cyan) on top of the reference orthomosaic (red) above a village near the Selenga Delta of Lake Baikal. (

**a**) is the orthoprojection of the pushbroom pixels (without interpolation) and (

**b**) is the image as seen after bilinear interpolation of the projected values.

**Figure 5.**Illustrations of the various steps of our algorithm for a test flight over a village of the Selenga Delta (Russia, 17 August 2014). All images show the superimposed scan lines (in cyan) on top of the reference orthophoto (in red).

**Figure 6.**Scatter of the difference vectors for the pairs of points matched by the SURF , for a flight over a village of the Selenga Delta. All the matches are represented by red crosses. Tilted blue crosses correspond to the pairs kept after removing outliers.

**Figure 7.**Averages of the cross-correlations of corresponding patterns between a reference orthophoto and a co-registered mosaic, for the patterns and their gradients.

**Figure 8.**Examples of co-registered patterns using PIV on original images (

**a**–

**c**) and on gradient images (

**d**–

**f**).

**Figure 9.**Superposition of the reference orthophotos and the mosaic produced with the scan lines: (

**a**), (

**c**) before geocorrection and (

**b**), (

**d**) after geocorrection. (

**a**,

**b**): Selenga Village 2; (

**c**,

**d**): Selenga Rivers.

**Figure 10.**Superposition of the reference orthophotos and the mosaic produced with the scan lines: (

**a**,

**c**) before geocorrection and (

**b**,

**d**) after geocorrection. (

**a**,

**b**): Gremyachinsk; (

**c**,

**d**): shore of Lake Geneva.

**Table 1.**Maximum correlations of compensated parameters observed in different scenarios at first iteration of the least squares optimisation.

Parameters | Maximum Correlation | Correlated Parameters |
---|---|---|

Boresight, ${u}_{pp}$, ${v}_{pp}$, | 1; | Roll and ${v}_{pp}$; |

f, ${K}_{1}$, ${K}_{2}$, ${P}_{1}$, ${P}_{2}$ | −1 | Pitch and ${u}_{pp}$ |

Boresight, f, ${K}_{1}$, ${K}_{2}$, | 0.71; | Roll and ${P}_{2}$; |

${P}_{1}$, ${P}_{2}$ | 0.69 | Pitch and ${P}_{1}$ |

Boresight, f, ${K}_{1}$, ${K}_{2}$ | 0.68 | ${K}_{1}$ and ${K}_{2}$ |

**Table 2.**Values and standard deviations of boresight parameters and IOP estimated by the least squares adjustment.

Parameters | Value | $\mathit{\sigma}$ |
---|---|---|

Roll | 1.1° | $9.3\times {10}^{-3}$° |

Pitch | $-0.54$° | $8.8\times {10}^{-3}$° |

Yaw | $-0.17$° | $5.2\times {10}^{-2}$° |

Focal length | 11.4 mm | $1.5\times {10}^{-2}$ mm |

${K}_{1}$ | $294.2\phantom{\rule{4.pt}{0ex}}{\mathrm{m}}^{-2}$ | $44.8\phantom{\rule{4.pt}{0ex}}{\mathrm{m}}^{-2}$ |

${K}_{2}$ | $-1.6\times {10}^{8}\phantom{\rule{4.pt}{0ex}}{\mathrm{m}}^{-4}$ | $350.7\phantom{\rule{4.pt}{0ex}}{\mathrm{m}}^{-4}$ |

${P}_{1}$ | $0.54\phantom{\rule{4.pt}{0ex}}{\mathrm{m}}^{-1}$ | $0.57\phantom{\rule{4.pt}{0ex}}{\mathrm{m}}^{-1}$ |

${P}_{2}$ | $0.74\phantom{\rule{4.pt}{0ex}}{\mathrm{m}}^{-1}$ | $0.22\phantom{\rule{4.pt}{0ex}}{\mathrm{m}}^{-1}$ |

Selenga Village 1 | Selenga Village 2 | Selenga Rivers | Gremyachinsk | Lake Geneva Shore | |
---|---|---|---|---|---|

Flight Altitude (m) | 1000 | 1000 | 1000 | 1000 | 500 |

Flight Attitude | Stable | U-turn | Stable | Stable | Wavy |

Surface Homogeneity | Heterogeneous | Heterogeneous | Heterogeneous | Homogeneous | Heterogeneous |

Content | Fields, built structures | Fields, built structures | Rivers, swamps | Water, sand | Built Structures |

Frame Camera GSD (m) | 0.28 | 0.28 | 0.28 | 0.31 | 0.14 |

Pushbroom Across-Track GSD (m) | 0.6 | 0.6 | 0.6 | 0.6 | 0.3 |

Pushbroom Along-Track GSD (m) | 0.6 | 0.6 | 0.6 | 0.6 | 0.5 |

**Table 4.**Planar RMSE for each test area, and percentage evolution from previous step, at each step of the geocorrection.

Planar RMSE | Selenga Village 1 | Selenga Village 2 | Selenga Rivers | Gremyachinsk | Lake Geneva Shore |
---|---|---|---|---|---|

Initial Projection | 24.5 m/40.8 px | 24.6 m/41 px | 24.5 m/40.8 px | 45.5 m/75.8 px | 10.8 m/27 px |

After IOP and Boresight Correction | 2.4 m/4 px (−90%) | 13.6 m/22.7 px (−45%) | 1.3 m/2.2 px (−95%) | 8.2 m/13.7 px (−82%) | 3.7 m/9.2 px (−66%) |

After PIV Deformation | 0.9 m/1.5 px (−67%) | 1.6 m/2.7 px (−88%) | 0.6 m/1 px (−54%) | 6.1 m/10.2 px (−26%) | 1.8 m/4.5 px (−51%) |

With Corrected Orientation Parameters | 1.1 m/1.8 px (+22%) | 1.8 m/3 px (+13%) | 0.8 m/1.3 px (+33%) | 6.5 m/10.8 px (+6%) | 1.6 m/4 px (−11%) |

© 2018 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Barbieux, K.
Pushbroom Hyperspectral Data Orientation by Combining Feature-Based and Area-Based Co-Registration Techniques. *Remote Sens.* **2018**, *10*, 645.
https://doi.org/10.3390/rs10040645

**AMA Style**

Barbieux K.
Pushbroom Hyperspectral Data Orientation by Combining Feature-Based and Area-Based Co-Registration Techniques. *Remote Sensing*. 2018; 10(4):645.
https://doi.org/10.3390/rs10040645

**Chicago/Turabian Style**

Barbieux, Kévin.
2018. "Pushbroom Hyperspectral Data Orientation by Combining Feature-Based and Area-Based Co-Registration Techniques" *Remote Sensing* 10, no. 4: 645.
https://doi.org/10.3390/rs10040645