Submit to Remote Sensing Review for Remote Sensing Propose a Special Issue

Journal Menu

Journal Browser

Accuracy Assessment and Validation of Remotely Sensed Data and Products

Print Special Issue Flyer
Special Issue Editors
Special Issue Information
Keywords
Published Papers

A special issue of Remote Sensing (ISSN 2072-4292). This special issue belongs to the section "Remote Sensing Image Processing".

Deadline for manuscript submissions: closed (31 December 2020) | Viewed by 44701

Share This Special Issue

Special Issue Editors

Dr. Giorgos Mallinis

E-Mail Website
Guest Editor

Laboratory of Photogrammetry and Remote Sensing (PERS Lab), School of Rural and Surveying Engineering, The Aristotle University of Thessaloniki, GR-54124 Thessaloniki, Greece
Interests: land use/land cover (LULC) mapping; forests; classification development and comparison; geographic object-based image analysis; natural disasters; UAS; ecosystem services
Special Issues, Collections and Topics in MDPI journals

Dr. Charalampos Georgiadis

E-Mail Website
Guest Editor

School of Civil Engineering, Laboratory of Photogrammetry and Remote Sensing Unit (PERS Lab), School of Rural and Surveying Engineering, The Aristotle University of Thessaloniki, Univ. Box 465, GR-54124 Thessaloniki, Greece
Interests: remote sensing; land use/land cover (LULC) mapping; photogrammetry; unmanned aerial systems (UAS); LiDAR; GIS; 3D modelling; mobile mapping systems; image analysis
Special Issues, Collections and Topics in MDPI journals

Special Issue Information

Dear Colleagues,

With the rapid advancement of sensor technology, automation of processing chains, and the increased availability of free or low-cost, nearly continuous measurements of the Earth’s surface, there is a growing interest in remote sensing applications in various scientific domains and themes. This interest has been also fueled by the need for accurate spatially-explicit information on Earth’s physical cover to address demands for policies and actions for sustainable development in terms of efficiency in resource use, disaster risk reduction, and ecosystem monitoring and protection. However, such use of remotely sensed data, requires reliable and quantitative accuracy reports to support confidence in the information generated. Accuracy assessment and validation is essential in remote sensing-based projects, since decision making or scientific analysis with data of unknown or little accuracy will result in information with low reliability, error propagation effects, and, subsequently, be of limited value.

The aim of this Special Issue is to explore new challenges and new insights related to the assessment of the thematic and positional accuracy of remotely sensed data and derived products.

Research contributions, as well as surveys, are welcome. In particular, novel contributions covering, but not limited to, the following subtopics are welcome:

Accuracy assessment of approaches focusing on time-series analysis of remotely-sensed data
Benchmarking and evaluation of different classification approaches
Accuracy metrics for object extraction from remote sensing images
Error and uncertainty analysis within the accuracy assessment process
Design and protocols for the validation of large‐area remote sensing products
Accuracy assessment of novel platforms and sensors, i.e., unmanned aerial systems (UAS), LiDAR data, etc.

Dr. Giorgos Mallinis
Dr. Charalampos Georgiadis
Guest Editors

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Remote Sensing is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2700 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

Positional accuracy
Horizontal accuracy
Thematic accuracy
Sampling design
Accuracy metrics

Published Papers (11 papers)

Download All Papers

Order results

Result details

Show export options Show export options

Select all

Export citation of selected articles as:

Research

Jump to: Other

29 pages, 7317 KiB

Open AccessArticle

Accuracy, Bias, and Improvements in Mapping Crops and Cropland across the United States Using the USDA Cropland Data Layer

by Tyler J. Lark, Ian H. Schelly and Holly K. Gibbs

Remote Sens. 2021, 13(5), 968; https://doi.org/10.3390/rs13050968 - 04 Mar 2021

Cited by 57 | Viewed by 6106

Abstract

The U.S. Department of Agriculture’s (USDA) Cropland Data Layer (CDL) is a 30 m resolution crop-specific land cover map produced annually to assess crops and cropland area across the conterminous United States. Despite its prominent use and value for monitoring agricultural land use/land cover (LULC), there remains substantial uncertainty surrounding the CDLs’ performance, particularly in applications measuring LULC at national scales, within aggregated classes, or changes across years. To fill this gap, we used state- and land cover class-specific accuracy statistics from the USDA from 2008 to 2016 to comprehensively characterize the performance of the CDL across space and time. We estimated nationwide area-weighted accuracies for the CDL for specific crops as well as for the aggregated classes of cropland and non-cropland. We also derived and reported new metrics of superclass accuracy and within-domain error rates, which help to quantify and differentiate the efficacy of mapping aggregated land use classes (e.g., cropland) among constituent subclasses (i.e., specific crops). We show that aggregate classes embody drastically higher accuracies, such that the CDL correctly identifies cropland from the user’s perspective 97% of the time or greater for all years since nationwide coverage began in 2008. We also quantified the mapping biases of specific crops throughout time and used these data to generate independent bias-adjusted crop area estimates, which may complement other USDA survey- and census-based crop statistics. Our overall findings demonstrate that the CDLs provide highly accurate annual measures of crops and cropland areas, and when used appropriately, are an indispensable tool for monitoring changes to agricultural landscapes. Full article

(This article belongs to the Special Issue Accuracy Assessment and Validation of Remotely Sensed Data and Products)

► Show Figures

Figure 1

21 pages, 5160 KiB

Open AccessArticle

Ground Validation and Error Sources Identification for GPM IMERG Product over the Southeast Coastal Regions of China

by Xinxin Sui, Zhi Li, Ziqiang Ma, Jintao Xu, Siyu Zhu and Hui Liu

Remote Sens. 2020, 12(24), 4154; https://doi.org/10.3390/rs12244154 - 18 Dec 2020

Cited by 36 | Viewed by 2446

Abstract

The Integrated Multi-satellitE Retrievals for the Global Precipitation Measurement mission (IMERG) has been widely evaluated. However, most of these studies focus on the ultimate merged satellite-gauge precipitation estimate and neglect the valuable intermediate estimates which directly guide the improvement of the IMERG product. This research aims to identify the error sources of the latest IMERG version 6 by evaluating the intermediate and ultimate precipitation estimates, and further examine the influences of regional topography and surface type on these errors. Results show that among six passive microwave (PMW) sensors, the Microwave Humidity Sounder (MHS) has outstanding comprehensive behavior, and Special Sensor Microwave Imager/Sounder (SSMIS) operates advanced at precipitation detection, while the Sounder for Atmospheric Profiling of Humidity in the Intertropics by Radiometry (SAPHIR) has the worst performance. More precipitation events are detected with larger quantitative uncertainty in low-lying places than in highlands, in urban and water body areas than in other places, and more in coastal areas than in inland regions. Infrared (IR) estimate has worse performance than PMW, and the precipitation detectability of IR is more sensitive to the factors of elevation and the distance to the coast, as larger critical successful index (CSI) over lowlands and coastal areas. PMW morphing and the mixing of PMW and IR algorithms partly reverse the conservative feature of the precipitation detection of PMW and IR estimates, resulting in higher probability of detection (POD) and false alert ratio (FAR). Finally, monthly gauge calibration improves most of the statistical indicators and reduces the influence of elevation and surface type factor on these errors. Full article

(This article belongs to the Special Issue Accuracy Assessment and Validation of Remotely Sensed Data and Products)

► Show Figures

Graphical abstract

23 pages, 797 KiB

Open AccessArticle

About the Pitfall of Erroneous Validation Data in the Estimation of Confusion Matrices

by Julien Radoux and Patrick Bogaert

Remote Sens. 2020, 12(24), 4128; https://doi.org/10.3390/rs12244128 - 17 Dec 2020

Cited by 10 | Viewed by 2152

Abstract

Accuracy assessment of maps relies on the collection of validation data, i.e., a set of trusted points or spatial objects collected independently from the classified map. However, collecting spatially and thematically accurate dataset is often tedious and expensive. Despite good practices, those datasets are rarely error-prone. Errors in the reference dataset propagate to the probabilities estimated in the confusion matrices. Consequently, the estimates of the quality are biased: accuracy indices are overestimated if the errors are correlated and underestimated if the errors are conditionally independent. The first findings of our study highlight the fact that this bias could invalidate statistical tests of map accuracy assessment. Furthermore, correlated errors in the reference dataset induce unfair comparison of classifiers. A maximum entropy method is thus proposed to mitigate the propagation of errors from imperfect reference datasets. The proposed method is based on a theoretical framework which considers a trivariate probability table that links the observed confusion matrix, the confusion matrix of the reference dataset and the “real” confusion matrix. The method was tested with simulated thematic and geo-reference errors. It proved to reduce the bias to the level of the sampling uncertainty. The method was very efficient with geolocation errors because conditional independence of errors can reasonably be assumed. Thematic errors are more difficult to mitigate because they require the estimation of an additional parameter related to the amount of spatial correlation. In any case, while collecting additional trusted labels is usually expensive, our result show that the benefits for accuracy assessment are much larger than collecting a larger number of questionable reference data. Full article

(This article belongs to the Special Issue Accuracy Assessment and Validation of Remotely Sensed Data and Products)

► Show Figures

Graphical abstract

21 pages, 4443 KiB

Open AccessArticle

Consistency Analysis and Accuracy Assessment of Three Global 30-m Land-Cover Products over the European Union using the LUCAS Dataset

by Yuan Gao, Liangyun Liu, Xiao Zhang, Xidong Chen, Jun Mi and Shuai Xie

Remote Sens. 2020, 12(21), 3479; https://doi.org/10.3390/rs12213479 - 22 Oct 2020

Cited by 64 | Viewed by 4877

Abstract

Land-cover plays an important role in the Earth’s energy balance, the hydrological cycle, and the carbon cycle. Therefore, it is important to evaluate the current global land-cover (GLC) products and to understand the differences between these products so that they can be used effectively in different applications. In this study, three 30-m GLC products, namely GlobeLand30-2010, GLC_FCS30-2015, and FROM_GLC30-2015, were evaluated in terms of areal consistency and spatial consistency using the Land Use/Cover Area frame statistical Survey (LUCAS) reference dataset over the European Union (EU). Given the limitations of the traditional confusion matrix used in accuracy assessment, we adjusted the confusion matrices from sample counts by accounting for the class proportions of the map and reported the standard errors of the descriptive accuracy measures in the accuracy assessment. The results revealed the following. (1) The overall accuracy of the GlobeLand30-2010 product was the highest at 88.90 ± 0.68%; this was followed by GLC_FCS30-2015 (84.33 ± 0.80%) and FROM_GLC2015 (65.31 ± 1.0%). (2) The consistency between the GLC_FCS30-2015 and GlobeLand30-2010 is higher than the consistency between other products, with an area correlation coefficient of 0.930 and a proportion of consistent pixels of 52.41%, respectively. (3) Across the area of the EU, the dominant land-cover types such as forest and cropland are the most consistent across the three products, whereas the spatial consistency for bare land, grassland, shrubland, and wetland is relatively low. (4) The proportion of pixels for which the consistency is low accounts for less than 16.17% of pixels, whereas the proportion of pixels for which the consistency is high accounts for about 39.12%. The disagreement between these products primarily occurs in transitional zones with mixed land cover types or in mountain areas. Overall, the GlobeLand30 and GLC-FCS30 products were found to be the most consistent and to have good classification accuracy in the EU, with the disagreement between the three 30-m GLC products mainly occurring in heterogeneous regions. Full article

(This article belongs to the Special Issue Accuracy Assessment and Validation of Remotely Sensed Data and Products)

► Show Figures

Graphical abstract

32 pages, 5597 KiB

Open AccessArticle

Triple Collocation Analysis for Two Error-Correlated Datasets: Application to L-Band Brightness Temperatures over Land

by Verónica González-Gambau, Antonio Turiel, Cristina González-Haro, Justino Martínez, Estrella Olmedo, Roger Oliva and Manuel Martín-Neira

Remote Sens. 2020, 12(20), 3381; https://doi.org/10.3390/rs12203381 - 16 Oct 2020

Cited by 11 | Viewed by 3377

Abstract

The error characterization of satellite observations is crucial for blending observations from multiple platforms into a unique dataset and for assimilating them into numerical weather prediction models. In the last years, the triple collocation (TC) technique has been widely used to assess the quality of many geophysical variables acquired with different instruments and at different scales. This paper presents a new formulation of the triple collocation (Correlated Triple Collocation (CTC)) for the case of three datasets that resolve similar spatial scales, with two of them being error-correlated datasets. Besides, the formulation is designed to ensure fast convergence of the error estimators. This approach is of special interest in cases such that finding more than three datasets with uncorrelated errors is not possible and the amount of data is limited. First, a synthetic experiment has been carried out to assess the performance of CTC formulation. As an example of application, the error characterization of three collocated L-band brightness temperature (TB) measurements over land has been performed. Two of the datasets come from ESA (European Space Agency) SMOS (Soil Moisture and Ocean Salinity) mission: one is the reconstructed TB from the operational L1B v620 product, and the other is the reconstructed TB from the operational L1B v620 product resulting from application of an RFI (Radio Frequency Interference) mitigation technique, the nodal sampling (NS). The third is an independent dataset, the TB acquired by a NASA (National Aeronautics and Space Administration) SMAP (Soil Moisture Active Passive) radiometer. Our analysis shows that the application of NS leads to TB error reduction with respect to the current version of SMOS TB in 80% of the points in the global map, with an average reduction of approximately 1 K over RFI-free regions and approximately 1.45 K over strongly RFI-contaminated areas. Full article

(This article belongs to the Special Issue Accuracy Assessment and Validation of Remotely Sensed Data and Products)

► Show Figures

Graphical abstract

24 pages, 8215 KiB

Open AccessArticle

National Scale Land Cover Classification for Ecosystem Services Mapping and Assessment, Using Multitemporal Copernicus EO Data and Google Earth Engine

by Natalia Verde, Ioannis P. Kokkoris, Charalampos Georgiadis, Dimitris Kaimaris, Panayotis Dimopoulos, Ioannis Mitsopoulos and Giorgos Mallinis

Remote Sens. 2020, 12(20), 3303; https://doi.org/10.3390/rs12203303 - 11 Oct 2020

Cited by 29 | Viewed by 6464

Abstract

Land-Use/Land-Cover (LULC) products are a common source of information and a key input for spatially explicit models of ecosystem service (ES) supply and demand. Global, continental, and regional, readily available, and free land-cover products generated through Earth Observation (EO) data, can be potentially used as relevant to ES mapping and assessment processes from regional to national scales. However, several limitations exist in these products, highlighting the need for timely land-cover extraction on demand, that could replace or complement existing products. This study focuses on the development of a classification workflow for fine-scale, object-based land cover mapping, employed on terrestrial ES mapping, within the Greek terrestrial territory. The processing was implemented in the Google Earth Engine cloud computing environment using 10 m spatial resolution Sentinel-1 and Sentinel-2 data. Furthermore, the relevance of different training data extraction strategies and temporal EO information for increasing the classification accuracy was also evaluated. The different classification schemes demonstrated differences in overall accuracy ranging from 0.88% to 4.94% with the most accurate classification scheme being the manual sampling/monthly feature classification achieving a 79.55% overall accuracy. The classification results suggest that existing LULC data must be cautiously considered for automated extraction of training samples, in the case of new supervised land cover classifications aiming also to discern complex vegetation classes. The code used in this study is available on GitHub and runs on the Google Earth Engine web platform. Full article

(This article belongs to the Special Issue Accuracy Assessment and Validation of Remotely Sensed Data and Products)

► Show Figures

Graphical abstract

20 pages, 1583 KiB

Open AccessArticle

Inter-Comparison of AIRS Temperature and Relative Humidity Profiles with AMMA and DACCIWA Radiosonde Observations over West Africa

by Marian Amoakowaah Osei, Leonard Kofitse Amekudzi, Craig R. Ferguson and Sylvester Kojo Danuor

Remote Sens. 2020, 12(16), 2631; https://doi.org/10.3390/rs12162631 - 14 Aug 2020

Cited by 9 | Viewed by 3432

Abstract

The vertical profiles of temperature and water vapour from the Atmospheric InfraRed Sounder (AIRS) have been validated across various regions of the globe as an effort to provide a substitute for radiosonde observations. However, there is a paucity of inter-comparisons over West Africa where local convective processes dominate and radiosonde observations (RAOBs) are limited. This study validates AIRS temperature and relative humidity profiles for selected radiosonde stations in West Africa. Radiosonde data were obtained from the AMMA and DACCIWA campaigns which spanned 2006–2008 and June–July 2016 respectively and offered a period of prolonged radiosonde observations in West Africa. AIRS performance was evaluated with the bias and root mean square difference (RMSD) at seven RAOB stations which were grouped into coastal and inland. Evaluation was performed on diurnal and seasonal timescales, cloud screening conditions and derived thunderstorm instability indices. At all timescales, the temperature RMSD was higher than the AIRS accuracy mission goal of ±1 K. Relative humidity RMSD was satisfactory with deviations <20% and <50% for both lower and upper troposphere respectively. AIRS retrieval of water vapour under cloudy and cloud-free conditions had no significant difference whereas cloud-free temperature was found to be more accurate. The seasonal evolution of some thunderstorm convective indices were also found to be comparable for AIRS and RAOB. The ability of AIRS to capture the evolution of these indices imply it will be a useful dataset for the African Science for Weather Information and Forecasting Techniques (SWIFT) high impact weather studies. Full article

(This article belongs to the Special Issue Accuracy Assessment and Validation of Remotely Sensed Data and Products)

► Show Figures

Graphical abstract

26 pages, 4297 KiB

Open AccessArticle

Determination of the Key Comparison Reference Value from Multiple Field Calibration of Sentinel-2B/MSI over the Baotou Site

by Caixia Gao, Yaokai Liu, Jinru Liu, Lingling Ma, Zhifeng Wu, Shi Qiu, Chuanrong Li, Yongguang Zhao, Qijin Han, Enyu Zhao, Yonggang Qian and Ning Wang

Remote Sens. 2020, 12(15), 2404; https://doi.org/10.3390/rs12152404 - 27 Jul 2020

Cited by 6 | Viewed by 2191

Abstract

Field calibration is a feasible way to evaluate space-borne optical sensor observations via natural or artificial sites on Earth’s surface with the aid of synchronous surface and atmospheric characteristic data. Since field calibration is affected by the coupled effects of surface and atmospheric characteristics, the single calibration results acquired under different surface and atmospheric conditions have different biases and different uncertainties, making it difficult to determine the consistency of these multiple calibration results. In view of this, by assuming that the radiometric performance is invariant during field calibration and the calibration samples are independent of each other, the surface–atmosphere invariant Key Comparison Reference Value (KCRV) is essentially derived from various calibration results. As the number of calibration samples increases, the uncertainty in the KCRV should decrease, and the KCRV should approach the “true” value. This paper addresses a novel method for estimating a weighted average value from multiple calibration results that can be used to compare each calibration result, and this value is accepted as the KCRV. Furthermore, this method is preliminarily applied to the field calibration of the Multispectral Instrument (MSI) onboard the Sentinel-2B satellite via the desert target at the Baotou site, China. After employing a chi-squared test to verify that 12 calibration samples are independent from each other, the KCRV of the 12 calibration samples at the Baotou site is derived, which exhibits much lower uncertainty than a single sample. The results show that the KCRVs of the relative differences between the simulated and observed at-sensor reflectance are 3.75%, 5.11%, 6.09%, and 5.03% for the four bands of Sentinel-2B/MSI, respectively, and the corresponding uncertainties are 1.84%, 1.87%, 1.90%, and 1.93%. It is noted that the KCRV uncertainty obtained with only 12 calibration samples is reduced significantly, and in the future, more samples in other instrumented sites will be used to validate this method thoroughly. Full article

(This article belongs to the Special Issue Accuracy Assessment and Validation of Remotely Sensed Data and Products)

► Show Figures

Graphical abstract

22 pages, 14688 KiB

Open AccessArticle

A Robust Method for Generating High-Spatiotemporal-Resolution Surface Reflectance by Fusing MODIS and Landsat Data

by Junming Yang, Yunjun Yao, Yongxia Wei, Yuhu Zhang, Kun Jia, Xiaotong Zhang, Ke Shang, Xiangyi Bei and Xiaozheng Guo

Remote Sens. 2020, 12(14), 2312; https://doi.org/10.3390/rs12142312 - 18 Jul 2020

Cited by 9 | Viewed by 2583

Abstract

The methods for accurately fusing medium- and high-spatial-resolution satellite reflectance are vital for monitoring vegetation biomass, agricultural irrigation, ecological processes and climate change. However, the currently existing fusion methods cannot accurately capture the temporal variation in reflectance for heterogeneous landscapes. In this study, we proposed a new method, the spatial and temporal reflectance fusion method based on the unmixing theory and a fuzzy C-clustering model (FCMSTRFM), to generate Landsat-like time-series surface reflectance. Unlike other data fusion models, the FCMSTRFM improved the similarity of pixels grouped together by combining land cover maps and time-series data cluster algorithms to define endmembers. The proposed method was tested over a 2000 km² study area in Heilongjiang Provence, China, in 2017 and 2018 using ten images. The results show that the accuracy of the FCMSTRFM is better than that of the popular enhanced spatial and temporal adaptive reflectance fusion model (ESTARFM) (correlation coefficient (R): 0.8413 vs. 0.7589; root mean square error (RMSE): 0.0267 vs. 0.0401) and the spatial-temporal data fusion approach (STDFA) (R: 0.8413 vs. 0.7666; RMSE: 0.0267 vs. 0.0307). Importantly, the FCMSTRFM was able to maintain the details of temporal variations in complicated landscapes. The proposed method provides an alternative method to monitor the dynamics of land surface variables over complicated heterogeneous regions. Full article

(This article belongs to the Special Issue Accuracy Assessment and Validation of Remotely Sensed Data and Products)

► Show Figures

Graphical abstract

Other

Jump to: Research

11 pages, 4380 KiB

Open AccessLetter

Inter-Comparison of Normalized Difference Vegetation Index Measured from Different Footprint Sizes in Cropland

by Jae-Hyun Ryu, Sang-Il Na and Jaeil Cho

Remote Sens. 2020, 12(18), 2980; https://doi.org/10.3390/rs12182980 - 14 Sep 2020

Cited by 11 | Viewed by 3237

Abstract

Remote sensing techniques using visible and near-infrared wavelengths are useful for monitoring terrestrial vegetation. The normalized difference vegetation index (NDVI) is a widely used proxy of vegetation conditions, and it has been measured at various footprint sizes using satellite, unmanned aerial vehicle (UAV), and ground-installed sensors. The goal of this study was to analyze the spatial characteristics of NDVI data by comparing the values obtained at different footprint sizes. In particular, the NDVI was evaluated in garlic and onion fields that featured ridges and furrows. The evaluation was performed using data from a leaf spectrometer, field spectrometers, ground-installed spectral reflectance sensors, a multispectral camera onboard a UAV, and Sentinel-2 satellites. The correlation coefficients between NDVIs evaluated from the various sensors (excluding the satellite-mounted sensors) ranged from 0.628 to 0.944. The UAV-based NDVI (NDVI_UAV) exhibited the lowest root mean square error (RMSE = 0.088) when compared with field spectrometer data. On the other hand, the satellite-based NDVI data (NDVI_Sentinel-2) were poorly correlated with those obtained from the other sensors as a result of the footprint mismatch. However, by upscaling the NDVI_UAV data to the pixel size of Sentinel-2, the comparison was improved, and the following statistics were obtained: correlation coefficient: 0.504–0.785; absolute bias: 0.048–0.078; RMSE: 0.063–0.094. According to the aforementioned results, ground-based NDVI data can be used to validate NDVI_UAV data without further processing and NDVI_UAV data can be used to validate NDVI_Sentinel-2 data after upscaling to the Sentinel-2 pixel size. Overall, the results presented in this study may be helpful to understand and integrate NDVI data at different spatial scales. Full article

(This article belongs to the Special Issue Accuracy Assessment and Validation of Remotely Sensed Data and Products)

► Show Figures

Graphical abstract

11 pages, 778 KiB

Open AccessLetter

The T Index: Measuring the Reliability of Accuracy Estimates Obtained from Non-Probability Samples

by François Waldner

Remote Sens. 2020, 12(15), 2483; https://doi.org/10.3390/rs12152483 - 03 Aug 2020

Cited by 4 | Viewed by 4044

Abstract

In remote sensing, the term accuracy typically expresses the degree of correctness of a map. Best practices in accuracy assessment have been widely researched and include guidelines on how to select validation data using probability sampling designs. In practice, however, probability samples may be lacking and, instead, cross-validation using non-probability samples is common. This practice is risky because the resulting accuracy estimates can easily be mistaken for map accuracy. The following question arises: to what extent are accuracy estimates obtained from non-probability samples representative of map accuracy? This letter introduces the T index to answer this question. Certain cross-validation designs (such as the common single-split or hold-out validation) provide representative accuracy estimates when hold-out sets are simple random samples of the map population. The T index essentially measures the probability of a hold-out set of unknown sampling design to be a simple random sample. To that aim, we compare its spread in the feature space against the spread of random unlabelled samples of the same size. Data spread is measured by a variant of Moran’s I autocorrelation index. Consistent interpretation of the T index is proposed through the prism of significance testing, with T values < 0.05 indicating unreliable accuracy estimates. Its relevance and interpretation guidelines are also illustrated in a case study on crop-type mapping. Uptake of the T index by the remote-sensing community will help inform about—and sometimes caution against—the representativeness of accuracy estimates obtained by cross-validation, so that users can better decide whether a map is fit for their purpose or how its accuracy impacts their application. Subsequently, the T index will build trust and improve the transparency of accuracy assessment in conditions which deviate from best practices. Full article

(This article belongs to the Special Issue Accuracy Assessment and Validation of Remotely Sensed Data and Products)

► Show Figures

Journal Menu

Journal Browser

Accuracy Assessment and Validation of Remotely Sensed Data and Products

Share This Special Issue

Special Issue Editors

Special Issue Information

Keywords

Published Papers (11 papers)

Research

Other

Further Information

Guidelines

MDPI Initiatives

Follow MDPI