# Synergy of Sentinel-1 and Sentinel-2 Time Series for Cloud-Free Vegetation Water Content Mapping with Multi-Output Gaussian Processes

^{1}

^{2}

^{3}

^{4}

^{5}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Methodology

#### 2.1. Theoretical Background

#### 2.1.1. Single-Output Gaussian Processes Modeling

#### 2.1.2. Multi-Output Gaussian Processes Modeling

#### 2.2. Study Area

#### 2.3. Sentinel-2 Time Series Preprocessing

#### 2.4. Sentinel-1 Time Series Preprocessing

#### 2.5. MOGP Models Parametrization

#### 2.6. Experimental Setup

#### 2.7. Delineation of Retrieval Workflow

- Building of VWC time series applying a GP model trained with in situ data of the BVCR 2020 crop campaign to S2 imagery, and pre-processing of RVI time series for S1 orbit 68 and orbit 141 imagery, respectively;
- Assembling the S1 & S2 dataset containing multitemporal VWC retrieved values and S1 post-processed RVI data for a specific ROI of the BVCR study site;
- Setting up the MOGP kernels with Q = 4 and initializing the parameters using SM;
- Training the MOGP models with the S1 & S2 dataset using the Adam optimizer and assessing the regression statistics error metrics (MAE, MAPE, RMSE, and NRMSE) for best model selection;
- Multi-seasonal mapping of VWC retrieved given the best evaluated MOGP model and S1 & S2 stacked datasets at pixel level over two distinct bounded fields and corresponding process performance;
- Reconstructing of artificially removed S2 GP VWC data gaps over winter wheat cropland considering the BVCR 2020 and 2021 crop campaigns.

## 3. Results

#### 3.1. S1 SAR RVI & S2 GP VWC Temporal Profiles

#### 3.2. Training MOGP Kernels for VWC Time Series Modelling

#### 3.2.1. Cross-Correlation Matrixes for the MOGP Trained Kernels

#### 3.2.2. Optimized MOGP Kernel for Mapping the VWC of the Winter Wheat 2020 and 2021

#### 3.3. Spatiotemporal Mapping of Reconstructed VWC Based on S1 & S2 Synergy

## 4. Discussion

#### 4.1. Time and Frequency Domain Similarities in the S1 & S2 Dataset

#### 4.2. MOGP Modelling and Assessment

#### 4.3. S1 & S2-Based Spatiotemporal Mapping of Vegetation Water Content

#### 4.4. Advantages and Opportunities for Improvement of the Fusing Approach

## 5. Conclusions

## Author Contributions

## Funding

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## Appendix A. Sentinel–1 & Sentinel–2 Acquisition Dates

**Table A1.**Sentinel–2 and Sentinel–1 acquisition dates corresponding to the winter wheat 2020 crop campaign at the BVCR. The (–) symbol means no acquisition. The (

*****) indicates the S2 removed images from the S1 & S2 dataset for model validation.

Winter Wheat 2020 Crop Campaign | ||
---|---|---|

S2 Acquisition Date | S1_{(Orbit 68)} Acquisition Date | S1_{(Orbit 141)} Acquisition Date |

- | 2020-08-27 | - |

2020-08-29 | - | - |

- | - | 2020-09-01 |

- | 2020-09-02 | - |

2020-09-13 * | - | 2020-09-13 |

2020-09-18 * | - | - |

- | 2020-09-20 | - |

2020-09-23 * | - | - |

- | - | 2020-09-25 |

- | 2020-09-26 | - |

2020-09-28 * | - | - |

- | 2020-10-02 | - |

- | - | 2020-10-07 |

- | 2020-10-08 | - |

2020-10-13 * | - | - |

- | 2020-10-14 | - |

- | - | 2020-10-19 |

- | 2020-10-20 | - |

- | 2020-10-26 | - |

- | - | 2020-10-31 |

- | 2020-11-01 | - |

2020-11-02 * | - | - |

- | 2020-11-07 | - |

- | - | 2020-11-12 |

- | 2020-11-13 | - |

2020-11-17 * | - | - |

- | 2020-11-19 | - |

- | - | 2020-11-24 |

- | 2020-11-25 | - |

2020-11-27 * | - | - |

- | 2020-12-01 | - |

- | - | 2020-12-06 |

2020-12-07 * | 2020-12-07 | - |

- | 2020-12-13 | - |

- | - | 2020-12-18 |

- | 2020-12-19 | - |

2020-12-22 | - | - |

- | 2020-12-25 | - |

- | - | 2020-12-30 |

- | 2020-12-31 | - |

- | 2021-01-06 | - |

**Table A2.**Sentinel–2 and Sentinel–1 acquisition dates corresponding to the winter wheat 2021 crop campaign at the BVCR. The (–) symbol means no acquisition. The (

*****) indicates the S2 removed images from the S1 & S2 dataset for model validation.

Winter Wheat 2021 Crop Campaign | ||
---|---|---|

S2 Acquisition Date | S1_{(Orbit 68)} Acquisition Date | S1_{(Orbit 141)} Acquisition Date |

- | 2021-08-16 | - |

- | 2021-08-22 | - |

2021-08-24 * | - | - |

- | - | 2021-08-27 |

- | 2021-08-28 | - |

- | 2021-09-03 | - |

- | - | 2021-09-08 |

- | 2021-09-09 | - |

- | - | 2021-09-20 |

- | 2021-09-21 | - |

- | 2021-09-27 | - |

- | - | 2021-10-02 |

2021-10-03 * | 2021-10-03 | - |

2021-10-08 * | - | - |

- | 2021-10-09 | - |

- | - | 2021-10-14 |

- | 2021-10-15 | - |

2021-10-18 * | - | - |

- | 2021-10-21 | - |

- | - | 2021-10-26 |

- | - | 2021-10-27 |

2021-11-02 * | 2021-11-02 | - |

- | 2021-11-08 | - |

- | 2021-11-14 | - |

2021-11-17 * | - | - |

- | - | 2021-11-19 |

- | 2021-11-20 | - |

- | 2021-11-26 | - |

- | - | 2021-12-01 |

- | 2021-12-02 | - |

2021-12-07 | - | - |

- | 2021-12-08 | - |

- | - | 2021-12-13 |

- | 2021-12-14 | - |

- | 2021-12-20 | - |

2021-12-22 | - | - |

2022-01-01 | 2022-01-01 | - |

## Appendix B. Hyperparameters of the CONV Models Trained over the Winter Wheat Test Sites

**Table A3.**Hyperparameters of the MOGP CONV model trained with the S2 GP VWC and S1 RVI orbits 68 and 141 time series over the winter wheat 2020 ROI-1.

Name | Range | Value |
---|---|---|

M[0].CONV.weight | ($1\times {10}^{-8}$, ∞) | [0.16140878 0.12014237 0.25099972] |

M[0].CONV.variance | (0.0, ∞) | [[$4.57955225\times {10}^{-6}$] [$1.48168009\times {10}^{-5}$] [$4.89282973\times {10}^{-5}$]] |

M[0].CONV.base_variance | ($1\times {10}^{-8}$, ∞) | [29.65490441] |

M[1].CONV.weight | ($1\times {10}^{-8}$, ∞) | [0.18422568 0.12714211 0.09511325] |

M[1].CONV.variance | (0.0, ∞) | [[0.00208712] [0.00021101] [0.00029573]] |

M[1].CONV.base_variance | ($1\times {10}^{-8}$, ∞) | [$3.84536139\times {10}^{-6}$] |

M[2].CONV.weight | ($1\times {10}^{-8}$, ∞) | [0.161608 0.36756376 0.38364755] |

M[2].CONV.variance | (0.0, ∞) | [[$5.97699880\times {10}^{-6}$] [$8.28188220\times {10}^{-4}$] [$2.73658687\times {10}^{-3}$]] |

M[2].CONV.base_variance | ($1\times {10}^{-8}$, ∞) | [55.15439607] |

M[3].CONV.weight | ($1\times {10}^{-8}$, ∞) | [0.45055456 0.09223841 0.01531059] |

M[3].CONV.variance | (0.0, ∞) | [[$3.78006500\times {10}^{-6}$] [$2.45277978\times {10}^{-7}$] [$3.81411018\times {10}^{-7}$]] |

M[3].CONV.base_variance | ($1\times {10}^{-8}$, ∞) | [54.76679359] |

Gaussian.scale | ($1\times {10}^{-8}$, ∞) | [0.07039943 0.05906305 0.03154559] |

**Table A4.**Hyperparameters of the MOGP model trained with the S2 GP VWC and S1 RVI orbits 68 and 141 time series over the winter wheat 2021 ROI-2.

Name | Range | Value |
---|---|---|

M[0].CONV.weight | ($1\times {10}^{-8}$, ∞) | [0.05051712 0.27439207 0.38695247] |

M[0].CONV.variance | (0.0, ∞) | [[$4.86858144\times {10}^{-6}$] [$7.13908231\times {10}^{-6}$] [$1.31486331\times {10}^{-3}$]] |

M[0].CONV.base_variance | ($1\times {10}^{-8}$, ∞) | [34.01715996] |

M[1].CONV.weight | ($1\times {10}^{-8}$, ∞) | [0.07826687 0.21647057 0.08729357] |

M[1].CONV.variance | (0.0, ∞) | [[$2.67026775\times {10}^{-5}$] [$2.09623224\times {10}^{-6}$] [$3.66018092\times {10}^{-5}$]] |

M[1].CONV.base_variance | ($1\times {10}^{-8}$, ∞) | [19.31982864] |

M[2].CONV.weight | ($1\times {10}^{-8}$, ∞) | [0.5937755 0.30263363 0.22857684] |

M[2].CONV.variance | (0.0, ∞) | [[$1.10220013\times {10}^{-4}$] [$1.21975560\times {10}^{-2}$] [$6.16156520\times {10}^{-6}$]] |

M[2].CONV.base_variance | ($1\times {10}^{-8}$, ∞) | [49.46172915] |

M[3].CONV.weight | ($1\times {10}^{-8}$, ∞) | [0.0563912 0.01698611 0.03144775] |

M[3].CONV.variance | (0.0, ∞) | [[$2.51113006\times {10}^{-5}$] [$1.23150713\times {10}^{-5}$] [$1.27814011\times {10}^{-2}$]] |

M[3].CONV.base_variance | ($1\times {10}^{-8}$, $\infty )$ | [0.08717407] |

Gaussian.scale | ($1\times {10}^{-8}$, ∞) | [0.04004209 0.06703326 0.0397214 ] |

## References

- Verrelst, J.; Camps-Valls, G.; Muñoz-Marí, J.; Rivera, J.P.; Veroustraete, F.; Clevers, J.G.P.W.; Moreno, J. Optical remote sensing and the retrieval of terrestrial vegetation bio-geophysical properties—A review. ISPRS J. Photogramm. Remote Sens.
**2015**, 108, 273–290. [Google Scholar] [CrossRef] - Verrelst, J.; Malenovský, Z.; Van der Tol, C.; Camps-Valls, G.; Gastellu-Etchegorry, J.P.; Lewis, P.; North, P.; Moreno, J. Quantifying Vegetation Biophysical Variables from Imaging Spectroscopy Data: A Review on Retrieval Methods. Surv. Geophys.
**2019**, 40, 589–629. [Google Scholar] [CrossRef][Green Version] - Quemada, C.; Pérez-Escudero, J.M.; Gonzalo, R.; Ederra, I.; Santesteban, L.G.; Torres, N.; Iriarte, J.C. Remote Sensing for Plant Water Content Monitoring: A Review. Remote Sens.
**2021**, 13, 2088. [Google Scholar] [CrossRef] - D’Urso, G.; Richter, K.; Calera, A.; Osann, M.A.; Escadafal, R.; Garatuza-Pajan, J.; Hanich, L.; Perdigão, A.; Tapia, J.B.; Vuolo, F. Earth Observation products for operational irrigation management in the context of the PLEIADeS project. Agric. Water Manag.
**2010**, 98, 271–282. [Google Scholar] [CrossRef] - Clevers, J.G.P.W.; Kooistra, L.; Schaepman, M.E. Estimating canopy water content using hyperspectral remote sensing data. Int. J. Appl. Earth Obs. Geoinf.
**2010**, 12, 119–125. [Google Scholar] [CrossRef] - Wocher, M.; Berger, K.; Danner, M.; Mauser, W.; Hank, T. Physically-based retrieval of canopy equivalent water thickness using hyperspectral data. Remote Sens.
**2018**, 10, 1924. [Google Scholar] [CrossRef][Green Version] - Gerhards, M.; Schlerf, M.; Mallick, K.; Udelhoven, T. Challenges and Future Perspectives of Multi-/Hyperspectral Thermal Infrared Remote Sensing for Crop Water-Stress Detection: A Review. Remote Sens.
**2019**, 11, 1240. [Google Scholar] [CrossRef][Green Version] - Bowman, W.D. The relationship between leaf water status, gas exchange, and spectral reflectance in cotton leaves. Remote Sens. Environ.
**1989**, 30, 249–255. [Google Scholar] [CrossRef] - Ustin, S.L.; Riaño, D.; Hunt, E.R. Estimating canopy water content from spectroscopy. Israel J. Plant Sci.
**2012**, 60, 9–23. [Google Scholar] [CrossRef][Green Version] - Berger, M.; Moreno, J.; Johannessen, J.A.; Levelt, P.F.; Hanssen, R.F. ESA’s sentinel missions in support of Earth system science. Remote Sens. Environ.
**2012**, 120, 84–90. [Google Scholar] [CrossRef] - Drusch, M.; Del Bello, U.; Carlier, S.; Colin, O.; Fernandez, V.; Gascon, F.; Hoersch, B.; Isola, C.; Laberinti, P.; Martimort, P.; et al. Sentinel-2: ESA’s Optical High-Resolution Mission for GMES Operational Services. Remote. Sens. Environ.
**2012**, 120, 25–36. [Google Scholar] [CrossRef] - Amin, E.; Verrelst, J.; Rivera-Caicedo, J.P.; Pipia, L.; Ruiz-Verdú, A.; Moreno, J. Prototyping Sentinel-2 green LAI and brown LAI products for cropland monitoring. Remote Sens. Environ.
**2021**, 255, 112168. [Google Scholar] [CrossRef] - Delloye, C.; Weiss, M.; Defourny, P. Retrieval of the canopy chlorophyll content from Sentinel-2 spectral bands to estimate nitrogen uptake in intensive winter wheat cropping systems. Remote Sens. Environ.
**2018**, 216, 245–261. [Google Scholar] [CrossRef] - Brede, B.; Verrelst, J.; Gastellu-Etchegorry, J.P.; Clevers, J.G.; Goudzwaard, L.; den Ouden, J.; Verbesselt, J.; Herold, M. Assessment of workflow feature selection on forest LAI prediction with sentinel-2A MSI, landsat 7 ETM+ and Landsat 8 OLI. Remote. Sens.
**2020**, 12, 915. [Google Scholar] [CrossRef][Green Version] - Verrelst, J.; Rivera, J.; Veroustraete, F.; Muñoz Marí, J.; Clevers, J.; Camps-Valls, G.; Moreno, J. Experimental Sentinel-2 LAI estimation using parametric, non-parametric and physical retrieval methods—A comparison. ISPRS J. Photogramm. Remote Sens.
**2015**, 108, 260–272. [Google Scholar] [CrossRef] - Estévez, J.; Salinero-Delgado, M.; Berger, K.; Pipia, L.; Rivera-Caicedo, J.P.; Wocher, M.; Reyes-Muñoz, P.; Tagliabue, G.; Boschetti, M.; Verrelst, J. Gaussian processes retrieval of crop traits in Google Earth Engine based on Sentinel-2 top-of-atmosphere data. Remote. Sens. Environ.
**2022**, 273, 112958. [Google Scholar] [CrossRef] - Torres, R.; Snoeij, P.; Geudtner, D.; Bibby, D.; Davidson, M.; Attema, E.; Potin, P.; Rommen, B.; Floury, N.; Brown, M.; et al. GMES Sentinel-1 mission. Remote Sens. Environ.
**2012**, 120, 9–24. [Google Scholar] [CrossRef] - Ulaby, F.T.; Aslam, A.; Dobson, M.C. Effects of Vegetation Cover on the Radar Sensitivity to Soil Moisture. IEEE Trans. Geosci. Remote Sens.
**1982**, GE-20, 476–481. [Google Scholar] [CrossRef] - Karam, M.A.; Fung, A.K.; Lang, R.H.; Chauhan, N.S. A microwave scattering model for layered vegetation. IEEE Trans. Geosci. Remote Sens.
**1992**, 30, 767–784. [Google Scholar] [CrossRef][Green Version] - Bousbih, S.; Zribi, M.; Lili-Chabaane, Z.; Baghdadi, N.; El Hajj, M.; Gao, Q.; Mougenot, B. Potential of Sentinel-1 Radar Data for the Assessment of Soil and Cereal Cover Parameters. Sensors
**2017**, 17, 2617. [Google Scholar] [CrossRef][Green Version] - Rozenstein, O.; Siegal, Z.; Blumberg, D.G.; Adamowski, J. Investigating the backscatter contrast anomaly in synthetic aperture radar (SAR) imagery of the dunes along the Israel–Egypt border. Int. J. Appl. Earth Obs. Geoinf.
**2016**, 46, 13–21. [Google Scholar] [CrossRef] - Gao, S.; Niu, Z.; Huang, N.; Hou, X. Estimating the Leaf Area Index, height and biomass of maize using HJ-1 and RADARSAT-2. Int. J. Appl. Earth Obs. Geoinf.
**2013**, 24, 1–8. [Google Scholar] [CrossRef] - McNairn, H.; Kross, A.; Lapen, D.; Caves, R.; Shang, J. Early season monitoring of corn and soybeans with TerraSAR-X and RADARSAT-2. Int. J. Appl. Earth Obs. Geoinf.
**2014**, 28, 252–259. [Google Scholar] [CrossRef] - Zhang, Y.; Venkatachalam, A.S.; Huston, D.; Xia, T. Advanced signal processing method for ground penetrating radar feature detection and enhancement. In Nondestructive Characterization for Composite Materials, Aerospace Engineering, Civil Infrastructure, and Homeland Security 2014; SPIE: Bellingham, WA, USA, 2014; Volume 9063, pp. 276–289. [Google Scholar] [CrossRef]
- Caballero, G.; Pezzola, A.; Winschel, C.; Casella, A.; Sanchez Angonova, P.; Orden, L.; Berger, K.; Verrelst, J.; Delegido, J. Quantifying Irrigated Winter Wheat LAI in Argentina Using Multiple Sentinel-1 Incidence Angles. Remote Sens.
**2022**, 14, 5867. [Google Scholar] [CrossRef] [PubMed] - Mattia, F.; Balenzano, A.; Satalino, G.; Lovergine, F.; Peng, J.; Wegmuller, U.; Cartus, O.; Davidson, M.W.J.; Kim, S.; Johnson, J.; et al. Sentinel-1 & Sentinel-2 for SOIL Moisture Retrieval at Field Scale. In Proceedings of the IGARSS 2018–2018 IEEE International Geoscience and Remote Sensing Symposium, IEEE, Valencia, Spain, 22–27 July 2018; pp. 6143–6146. [Google Scholar] [CrossRef]
- Satalino, G.; Mattia, F.; Balenzano, A.; Lovergine, F.P.; Rinaldi, M.; De Santis, A.P.; Ruggieri, S.; García, D.A.N.; Gómez, V.P.; Ceschia, E.; et al. Sentinel-1 & Sentinel-2 Data for Soil Tillage Change Detection. In Proceedings of the IGARSS 2018–2018 IEEE International Geoscience and Remote Sensing Symposium, IEEE, Valencia, Spain, 22–27 July 2018; pp. 6627–6630. [Google Scholar] [CrossRef]
- Veloso, A.; Mermoz, S.; Bouvet, A.; Le Toan, T.; Planells, M.; Dejoux, J.F.; Ceschia, E. Understanding the temporal behavior of crops using Sentinel-1 and Sentinel-2-like data for agricultural applications. Remote Sens. Environ.
**2017**, 199, 415–426. [Google Scholar] [CrossRef] - Pipia, L.; Muñoz-Marí, J.; Amin, E.; Belda, S.; Camps-Valls, G.; Verrelst, J. Fusing optical and SAR time series for LAI gap filling with multioutput Gaussian processes. Remote Sens. Environ.
**2019**, 235, 111452. [Google Scholar] [CrossRef] [PubMed] - Druce, D.; Tong, X.; Lei, X.; Guo, T.; Kittel, C.M.M.; Grogan, K.; Tottrup, C. An Optical and SAR Based Fusion Approach for Mapping Surface Water Dynamics over Mainland China. Remote Sens.
**2021**, 13, 1663. [Google Scholar] [CrossRef] - Caballero, G.; Delegido, J.; Verrelst, J. Estimación del LAI de la vegetación a partir de la sinergia Sentinel 1 -Sentinel 2. ResearchGate
**2018**. [Google Scholar] [CrossRef] - Tona, C.; Bua, R. Open Source Data Hub System: Free and open framework to enable cooperation to disseminate Earth Observation data and geo-spatial information. EGU Gen. Assem. Conf. Abstr.
**2018**, 20, 13038. [Google Scholar] - Pipia, L.; Amin, E.; Belda, S.; Salinero-Delgado, M.; Verrelst, J. Green LAI Mapping and Cloud Gap-Filling Using Gaussian Process Regression in Google Earth Engine. Remote. Sens.
**2021**, 13, 403. [Google Scholar] [CrossRef] - Gorelick, N.; Hancher, M.; Dixon, M.; Ilyushchenko, S.; Thau, D.; Moore, R. Google Earth Engine: Planetary-scale geospatial analysis for everyone. Remote Sens. Environ.
**2017**, 202, 18–27. [Google Scholar] [CrossRef] - Kumar, L.; Mutanga, O. Google Earth Engine Applications Since Inception: Usage, Trends, and Potential. Remote Sens.
**2018**, 10, 1509. [Google Scholar] [CrossRef][Green Version] - Rasmussen, C.E.; Williams, C.K.I. Gaussian Processes for Machine Learning; The MIT Press: New York, NY, USA, 2006. [Google Scholar]
- Belda, S.; Pipia, L.; Morcillo-Pallarés, P.; Verrelst, J. Optimizing Gaussian Process Regression for Image Time Series Gap-Filling and Crop Monitoring. Agronomy
**2020**, 10, 618. [Google Scholar] [CrossRef] - Bonilla, E.V.; Chai, K.; Williams, C. Multi-task Gaussian Process Prediction. Adv. Neural Inf. Process. Syst.
**2007**, 20. Available online: https://proceedings.neurips.cc/paper/2007/hash/66368270ffd51418ec58bd793f2d9b1b-Abstract.html (accessed on 21 February 2023). - Álvarez, M.A.; Rosasco, L.; Lawrence, N.D. Kernels for Vector-Valued Functions: A Review. MAL
**2012**, 4, 195–266. [Google Scholar] [CrossRef][Green Version] - Goovaerts, P. Geostatistics for Natural Resources Evaluation; Oxford University Press: Oxford, UK, 1997. [Google Scholar]
- Lin, Q.; Hu, J.; Zhou, Q.; Cheng, Y.; Hu, Z.; Couckuyt, I.; Dhaene, T. Multi-output Gaussian process prediction for computationally expensive problems with multiple levels of fidelity. Knowl.-Based Syst.
**2021**, 227, 107151. [Google Scholar] [CrossRef] - Alvarez, M.A.; Ward, W.; Guarnizo, C. Non-linear process convolutions for multi-output Gaussian processes. In Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics, Naha, Okinawa, Japan, 16–18 April 2019; pp. 1969–1977. Available online: https://proceedings.mlr.press/v89/alvarez19a.html (accessed on 21 February 2023).
- de Wolff, T.; Cuevas, A.; Tobar, F. MOGPTK: The Multi-Output Gaussian Process Toolkit. Neurocomputing
**2020**, 424, 49–53. [Google Scholar] [CrossRef] - Kim, Y.; Jackson, T.; Bindlish, R.; Lee, H.; Hong, S. Radar Vegetation Index for Estimating the Vegetation Water Content of Rice and Soybean. IEEE Geosci. Remote Sens. Lett.
**2012**, 9, 564–568. [Google Scholar] - Rasmussen, C.E. Gaussian Processes in Machine Learning. In Advanced Lectures on Machine Learning; Springer: Berlin, Germany, 2004; pp. 63–71. [Google Scholar] [CrossRef][Green Version]
- Snee, R.D. Validation of Regression Models: Methods and Examples. Technometrics
**1977**, 19, 415–428. [Google Scholar] [CrossRef] - Love, B.C.; Jones, M. Bayesian Learning. In Encyclopedia of the Sciences of Learning; Springer: Boston, MA, USA, 2012; pp. 415–417. [Google Scholar] [CrossRef]
- Wackernagel, H. Multivariate Geostatistics: An Introduction with Applications; Springer: Berlin, Germany, 2013. [Google Scholar]
- Barry, R.P.; Hoef, J.M.V. Blackbox Kriging: Spatial Prediction without Specifying Variogram Models on JSTOR. J. Agric. Biol. Environ. Stat.
**1996**, 1, 297–322. [Google Scholar] [CrossRef] - Ver Hoef, J.M.; Barry, R.P. Constructing and fitting models for cokriging and multivariable spatial prediction. J. Stat. Plan. Inference
**1998**, 69, 275–294. [Google Scholar] [CrossRef] - Higdon, D. Space and Space-Time Modeling using Process Convolutions. In Quantitative Methods for Current Environmental Issues; Springer: London, UK, 2002; pp. 37–56. [Google Scholar] [CrossRef]
- Casella, A.; Orden, L.; Pezzola, N.A.; Bellaccomo, C.; Winschel, C.I.; Caballero, G.R.; Delegido, J.; Gracia, L.M.N.; Verrelst, J. Analysis of Biophysical Variables in an Onion Crop (Allium cepa L.) with Nitrogen Fertilization by Sentinel-2 Observations. Agronomy
**2022**, 12, 1884. [Google Scholar] [CrossRef] - Caballero, G.R.; Platzeck, G.; Pezzola, A.; Casella, A.; Winschel, C.; Silva, S.S.; Ludueña, E.; Pasqualotto, N.; Delegido, J. Assessment of Multi-Date Sentinel-1 Polarizations and GLCM Texture Features Capacity for Onion and Sunflower Classification in an Irrigated Valley: An Object Level Approach. Agronomy
**2020**, 10, 845. [Google Scholar] [CrossRef] - Caballero, G.; Pezzola, A.; Winschel, C.; Casella, A.; Sanchez Angonova, P.; Rivera-Caicedo, J.P.; Berger, K.; Verrelst, J.; Delegido, J. Seasonal Mapping of Irrigated Winter Wheat Traits in Argentina with a Hybrid Retrieval Workflow Using Sentinel-2 Imagery. Remote Sens.
**2022**, 14, 4531. [Google Scholar] [CrossRef] - Berger, K.; Rivera Caicedo, J.P.; Martino, L.; Wocher, M.; Hank, T.; Verrelst, J. A Survey of Active Learning for Quantifying Vegetation Traits from Terrestrial Earth Observation Data. Remote. Sens.
**2021**, 13, 287. [Google Scholar] [CrossRef] - Settles, B. Active Learning Literature Survey. University of Wisconsin–Madison, Department of Computer Sciences. 2009. Available online: https://minds.wisconsin.edu/handle/1793/60660 (accessed on 21 February 2023).
- Salinero-Delgado, M.; Estévez, J.; Pipia, L.; Belda, S.; Berger, K.; Paredes Gómez, V.; Verrelst, J. Monitoring Cropland Phenology on Google Earth Engine Using Gaussian Process Regression. Remote Sens.
**2021**, 14, 146. [Google Scholar] [CrossRef] - Reyes-Muñoz, P.; Pipia, L.; Salinero-Delgado, M.; Belda, S.; Berger, K.; Estévez, J.; Morata, M.; Rivera-Caicedo, J.P.; Verrelst, J. Quantifying Fundamental Vegetation Traits over Europe Using the Sentinel-3 OLCI Catalogue in Google Earth Engine. Remote Sens.
**2022**, 14, 1347. [Google Scholar] [CrossRef] [PubMed] - Verrelst, J.; Rivera-Caicedo, J.P.; Reyes-Muñoz, P.; Morata, M.; Amin, E.; Tagliabue, G.; Panigada, C.; Hank, T.; Berger, K. Mapping landscape canopy nitrogen content from space using PRISMA data. ISPRS J. Photogramm. Remote. Sens.
**2021**, 178, 382–395. [Google Scholar] [CrossRef] - Gutman, G.; Ignatov, A. The derivation of the green vegetation fraction from NOAA/AVHRR data for use in numerical weather prediction models. Int. J. Remote Sens.
**1998**, 19, 1533–1543. [Google Scholar] [CrossRef] - Gitelson, A.; Zur, Y.; Chivkunova, O.; Merzlyak, M. Assessing carotenoid content in plant leaves with reflectance spectroscopy. Photochem. Photobiol.
**2002**, 75, 272–281. [Google Scholar] [CrossRef] - Jia, K.; Liang, S.; Gu, X.; Baret, F.; Wei, X.; Wang, X.; Yao, Y.; Yang, L.; Li, Y. Fractional vegetation cover estimation algorithm for Chinese GF-1 wide field view data. Remote Sens. Environ.
**2016**, 177, 184–191. [Google Scholar] [CrossRef] - Song, W.; Mu, X.; Ruan, G.; Gao, Z.; Li, L.; Yan, G. Estimating fractional vegetation cover and the vegetation index of bare soil and highly dense vegetation with a physically based method. Int. J. Appl. Earth Obs. Geoinf.
**2017**, 58, 168–176. [Google Scholar] [CrossRef] - García-Haro, F.J.; Campos-Taberner, M.; Munoz-Mari, J.; Laparra, V.; Camacho, F.; Sanchez-Zapero, J.; Camps-Valls, G. Derivation of global vegetation biophysical parameters from EUMETSAT Polar System. ISPRS J. Photogramm. Remote. Sens.
**2018**, 139, 57–74. [Google Scholar] [CrossRef] - Lee, J.S.; Jurkevich, L.; Dewaele, P.; Wambacq, P.; Oosterlinck, A. Speckle filtering of synthetic aperture radar images: A review. Remote. Sens. Rev.
**1994**, 8, 313–340. [Google Scholar] [CrossRef] - Lee, J.S. Refined filtering of image noise using local statistics. Comput. Graph. Image Process.
**1981**, 15, 380–389. [Google Scholar] [CrossRef] - Pan, Z.; Hu, Y.; Cao, B. Construction of smooth daily remote sensing time series data: A higher spatiotemporal resolution perspective. Open Geospat. Data Softw. Stand.
**2017**, 2, 1–11. [Google Scholar] [CrossRef][Green Version] - Savitzky, A.; Golay, M.J.E. Smoothing and Differentiation of Data by Simplified Least Squares Procedures. Anal. Chem.
**1964**, 36, 1627–1639. [Google Scholar] [CrossRef] - Neeff, T.; Dutra, L.V.; Dos Santos, J.R.; Freitas, C.C.; Araujo, L.S. Power spectrum analysis of SAR data for spatial forest characterization in Amazonia. Int. J. Remote Sens.
**2005**, 26, 2851–2864. [Google Scholar] [CrossRef] - Parra, G.; Tobar, F. Spectral Mixture Kernels for Multi-Output Gaussian Processes. Adv. Neural Inf. Process. Syst.
**2017**, 30. [Google Scholar] [CrossRef] - Ulrich, K.R.; Carlson, D.E.; Dzirasa, K.; Carin, L. GP Kernels for Cross-Spectrum Analysis. Adv. Neural Inf. Process. Syst.
**2015**, 28. Available online: https://proceedings.neurips.cc/paper/2015/hash/285ab9448d2751ee57ece7f762c39095-Abstract.html (accessed on 21 February 2023). - Alvarez, M.; Lawrence, N. Sparse Convolved Gaussian Processes for Multi-output Regression. Adv. Neural Inf. Process. Syst.
**2008**, 21. Available online: https://proceedings.neurips.cc/paper/2008/hash/149e9677a5989fd342ae44213df68868-Abstract.html (accessed on 21 February 2023). - van der Wilk, M.; Rasmussen, C.E.; Hensman, J. Convolutional Gaussian Processes. 2017. Available online: https://doi.org/10.48550/ARXIV.1709.01894 (accessed on 21 February 2023).
- Kingma, D.P.; Ba, J. Adam: A Method for Stochastic Optimization. arXiv
**2014**. [Google Scholar] [CrossRef] - Tobar, F. Bayesian Nonparametric Spectral Estimation. Adv. Neural Inf. Process. Syst.
**2018**, 31. Available online: https://proceedings.neurips.cc/paper/2018/hash/abd1c782880cc59759f4112fda0b8f98-Abstract.html (accessed on 21 February 2023). - Verrelst, J.; Rivera, J.; Moreno, J.; Camps-Valls, G. Gaussian processes uncertainty estimates in experimental Sentinel-2 LAI and leaf chlorophyll content retrieval. ISPRS J. Photogramm. Remote Sens.
**2013**, 86, 157–167. [Google Scholar] [CrossRef] - Paek, S.W.; Balasubramanian, S.; Kim, S.; de Weck, O. Small-Satellite Synthetic Aperture Radar for Continuous Global Biospheric Monitoring: A Review. Remote Sens.
**2020**, 12, 2546. [Google Scholar] [CrossRef] - Titsias, M.K. Variational Model Selection for Sparse Gaussian Process Regression; University of Manchester: Manchester, UK, 2008. [Google Scholar]
- Kiefer, J.; Wolfowitz, J. Stochastic Estimation of the Maximum of a Regression Function. Ann. Math. Stat.
**1952**, 23, 462–466. [Google Scholar] [CrossRef]

**Figure 1.**Overview of BVCR test sites (ROI) for the winter wheat campaigns of the years 2020 and 2021. The red and orange dotted rectangles delimit the MOGP subsets analyzed in Section 3.3. Reference system: WGS84 (EPSG 4326). (For interpretation of the references to color in this figure legend, the reader is referred to the Web version of this article).

**Figure 2.**Illustration of the processing workflow to obtain maps of vegetation water content over irrigated winter wheat, as described in Section 2.7. The ordinal numbers in the graph refer to the workflow processing steps. The maps show the output obtained by our MOGP VWC models over the BVCR study site in Argentina.

**Figure 3.**Temporal profiles (mean value and standard deviation) of averaged selected ROI-1 over a winter wheat parcel belonging to the BVCR 2020 crop campaign. S2 GP VWC time series from December 2018 to the end of January 2021 (

**a**). S1 RVI orbit 141 smoothed (dashed blue line) and original (dashed black line) time series (

**b**). S1 RVI orbit 68 smoothed (dashed red line) and original (dashed black line) time series (

**c**). S1 RVI Orbit 68 (red line) and S1 orbit 141 (blue line) RVI time series (

**d**).

**Figure 4.**Temporal profiles (mean value and standard deviation) of averaged selected ROI-2 over a winter wheat parcel belonging to the BVCR 2021 crop campaign. S2 GP VWC time series from October 2019 to the end of January 2022 (

**a**). S1 RVI orbit 141 smoothed (dashed blue line) and original (dashed black line) time series (

**b**). S1 RVI orbit 68 smoothed (dashed red line) and original (dashed black line) time series (

**c**). S1 RVI Orbit 68 (red line) and S1 orbit 141 (blue line) RVI time series (

**d**).

**Figure 5.**Performance of the SOGP and MOGP models predictions for VWC data reconstruction based on the S1 & S2 synergy. The green dots represent the latent S2 GP VWC data used to compute the error metrics whereas the red dots correspond to the S2 GP VWC cloud-free observations utilized to train the regression models. The red-shaded area represents the artificially created S2 data gap. SOGP and MOGP predictions performance for the selected winter wheat ROI-1 of the year 2020 (

**a**) and 2021 (

**b**).

**Figure 6.**Cross-correlation matrix of the S1 & dataset among the channels of the trained MOGP models. The elements of the matrix’s diagonal show the auto-correlations of each channel in the dataset. MOSM model for wheat 2020 (

**a**). CSM model for wheat 2020 (

**b**). SM-LMC model for wheat 2020 (

**c**). CONV model for wheat 2020 (

**d**). MOSM model for wheat 2021 (

**e**). CSM model for wheat 2021 (

**f**). SM-LMC model for wheat 2021 (

**g**). CONV model for wheat 2021 (

**h**).

**Figure 7.**CONV posterior predicted mean for a three-channel model. Red dots indicate the original observations for S2 GP VWC (CH-1), S1 RVI orbit 68 (CH-2), and S1 RVI orbit 141 (CH-3) while the green ones are the latent samples. Winter wheat 2020 ROI-1 (

**a**). Winter wheat 2020 ROI-2 (

**b**).

**Figure 8.**Comparison of S2 GP VWC (original) and S1 & S2 MOGP VWC (reconstructed) maps over a selected subset of the study site (red-dashed rectangle in Figure 1) corresponding to the artificially removed dates: 2020/9/23, 2020/10/13, 2020/11/2, 2020/11/17, and 2020/11/27 of the winter wheat 2020 phenological cycle. For each assessment date, the position of the extracted samples on the phenological curve (yellow dot), the S2 GP VWC map, the reconstructed S1 & S2 MOGP VWC map and the scatterplot between the original and reconstructed VWC maps are shown.

**Figure 9.**Comparison of S2 GP VWC (original) and S1 & S2 MOGP VWC (reconstructed) maps over a selected subset of the study site (orange-dashed rectangle in Figure 1) corresponding to the artificially removed dates: 2021/10/3, 2021/10/8, 2021/10/18, 2021/11/2, and 2021/11/17 of the winter wheat 2021 phenological cycle. For each assessment date, the position of the extracted samples on the phenological curve (yellow dot), the S2 GP VWC map, the reconstructed S1 & S2 MOGP VWC map and the scatterplot between the original and reconstructed VWC maps are shown.

**Table 1.**ROI boundaries in geographic coordinates (WGS84), x-pixels quantity (Qty-x), y-pixels quantity (Qty-y), and ROI area selected for training the MOGP models with the S1 & S2 dataset. ROI-1 belongs to the winter wheat 2020 site whereas the ROI-2 to 2021 site.

North | West | South | East | Qty-x | Qty-y | Area [ha] | |
---|---|---|---|---|---|---|---|

ROI-1 | −39.398 | −62.645 | −39.404 | −62.636 | 10 | 12 | 1.2 |

ROI-2 | −39.391 | −62.618 | −39.392 | −62.616 | 12 | 13 | 1.56 |

**Table 2.**Error metrics and training time of the MOGP and SOGP evaluated kernels for the 2020 winter wheat averaged ROI-1.

S2 GP VWC and S1 RVI Orbit 68 | |||||
---|---|---|---|---|---|

MOGP Kernel | MAE [g m${}^{-2}$] | MAPE [%] | RMSE [g m${}^{-2}$] | NRMSE [%] | Time [s] |

MOSM | 828.85 | 56.42 | 927.56 | 44.34 | 10.58 |

CSM | 242.7 | 15.43 | 360.55 | 17.24 | 17.85 |

SM-LMC | 346.16 | 22.56 | 495.49 | 23.69 | 12.68 |

CONV | 250.17 | 19.48 | 313.11 | 14.97 | 21.42 |

SM | 881.4 | 58.91 | 1005.71 | 48.07 | 6.03 |

S2 GP VWC and S1 RVI orbit 141 | |||||

MOSM | 1025.79 | 69.92 | 1116.62 | 53.38 | 9.37 |

CSM | 283.95 | 19.76 | 378.01 | 18.07 | 16.06 |

SM-LMC | 482.25 | 31.99 | 580.76 | 27.76 | 11.49 |

CONV | 255.42 | 25.25 | 419.36 | 20.05 | 19.25 |

SM | 883.69 | 59.05 | 1009.05 | 48.23 | 4.98 |

S2 GP VWC, S1 RVI orbit 68 and S1 RVI orbit 141 | |||||

MOSM | 907.21 | 62.61 | 992.18 | 47.43 | 18.56 |

CSM | 472.31 | 32.75 | 512.23 | 24.49 | 35.18 |

SM-LMC | 463.04 | 30.75 | 546.85 | 26.14 | 22.67 |

CONV | 249.3 | 21.83 | 336.74 | 16.1 | 40.27 |

SM | 881.77 | 58.93 | 1006.25 | 48.1 | 10.29 |

**Table 3.**Error metrics and training time of the MOGP and SOGP evaluated kernels for the 2021 winter wheat averaged ROI-2.

S2 GP VWC and S1 RVI Orbit 68 | |||||
---|---|---|---|---|---|

MOGP Kernel | MAE [g m${}^{-2}$] | MAPE [%] | RMSE [g m${}^{-2}$] | NRMSE [%] | Time [s] |

MOSM | 1606.97 | 91.26 | 1746.35 | 77.76 | 11.59 |

CSM | 1420.06 | 79.84 | 1549.94 | 69.02 | 19.85 |

SM-LMC | 1229.57 | 64.98 | 1362.06 | 60.65 | 13.9 |

CONV | 238.07 | 41 | 328.01 | 14.61 | 22.31 |

SM | 1408.52 | 75.06 | 1550.67 | 69.05 | 7.23 |

S2 GP VWC and S1 RVI orbit 141 | |||||

MOSM | 1606.95 | 91.26 | 1746.33 | 77.76 | 9.96 |

CSM | 864.12 | 54.02 | 928.28 | 41.33 | 18.25 |

SM-LMC | 1262.46 | 69.72 | 1378.87 | 61.4 | 12.24 |

CONV | 274.33 | 43.77 | 352.11 | 15.68 | 21.78 |

SM | 1408.52 | 75.06 | 1550.67 | 69.05 | 6.98 |

S2 GP VWC, S1 RVI orbit 68 and S1 RVI orbit 141 | |||||

MOSM | 1640.51 | 94.6 | 1778.6 | 79.2 | 21 |

CSM | 1446.8 | 82.65 | 1576.08 | 70.18 | 36.08 |

SM-LMC | 1395.58 | 74.98 | 1535.22 | 68.36 | 24.21 |

CONV | 190.44 | 25.69 | 227.12 | 10.11 | 45.02 |

SM | 1408.52 | 75.06 | 1550.67 | 69.05 | 10.2 |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2023 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Caballero, G.; Pezzola, A.; Winschel, C.; Sanchez Angonova, P.; Casella, A.; Orden, L.; Salinero-Delgado, M.; Reyes-Muñoz, P.; Berger, K.; Delegido, J.; Verrelst, J. Synergy of Sentinel-1 and Sentinel-2 Time Series for Cloud-Free Vegetation Water Content Mapping with Multi-Output Gaussian Processes. *Remote Sens.* **2023**, *15*, 1822.
https://doi.org/10.3390/rs15071822

**AMA Style**

Caballero G, Pezzola A, Winschel C, Sanchez Angonova P, Casella A, Orden L, Salinero-Delgado M, Reyes-Muñoz P, Berger K, Delegido J, Verrelst J. Synergy of Sentinel-1 and Sentinel-2 Time Series for Cloud-Free Vegetation Water Content Mapping with Multi-Output Gaussian Processes. *Remote Sensing*. 2023; 15(7):1822.
https://doi.org/10.3390/rs15071822

**Chicago/Turabian Style**

Caballero, Gabriel, Alejandro Pezzola, Cristina Winschel, Paolo Sanchez Angonova, Alejandra Casella, Luciano Orden, Matías Salinero-Delgado, Pablo Reyes-Muñoz, Katja Berger, Jesús Delegido, and Jochem Verrelst. 2023. "Synergy of Sentinel-1 and Sentinel-2 Time Series for Cloud-Free Vegetation Water Content Mapping with Multi-Output Gaussian Processes" *Remote Sensing* 15, no. 7: 1822.
https://doi.org/10.3390/rs15071822