Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
7.9
5.0
Journals
Remote Sensing
Special Issues
Reproducibility and Replicability in Remote Sensing Workflows
remotesensing-logo
Submit to Remote Sensing Review for Remote Sensing Propose a Special Issue

Journal Menu

► Journal Menu

Journal Browser

► Journal Browser
share announcement

Reproducibility and Replicability in Remote Sensing Workflows

A special issue of Remote Sensing (ISSN 2072-4292).

Deadline for manuscript submissions: closed (30 September 2022) | Viewed by 11342

Special Issue Editor

Dr. Jason A. Tullis

E-Mail Website
Guest Editor
Department of Geosciences and Center for Advanced Spatial Technologies, 321 JBHT, J. William Fulbright College of Arts & Sciences, University of Arkansas, Fayetteville, AR 72701, USA
Interests: remote sensing and GIS-assisted decision support for ecosystem services; geospatial unmanned aircraft systems (UAS); provenance, reproducibility and replicability (R&R) in GIScience; international geospatial capacity building

Special Issue Information

Dear Colleagues,

In 2019, the National Academies of Sciences, Engineering, and Medicine published a consensus study report on reproducibility and replicability (R&R), with important implications for computationally intensive sciences. For half a century, remote sensing workflows have leveraged cutting-edge computational innovation in the face of complex sensor, hardware, and software versioning combined with general challenges to replicability across spatiotemporal dimensions. As we navigate a global pandemic, unprecedented drivers of personal remote sensing now amplify prior geospatial R&R-related questions such as provenance, interoperability, multiuser access, privacy, ownership, scale of analysis, fitness for use, intellectual property, and regulatory control. These and other factors both complicate the role of remote sensing and invite unprecedented opportunity for innovation within a convergent GIScience paradigm.

Remote Sensing announces a Special Issue dedicated to R&R in remote sensing workflows. The Special Issue seeks to balance the need for practical R&R demonstrations with a careful analysis of the core drivers and constraints behind R&R in GIScience. While methodologies presented may narrowly address specific geospatial applications, authors should contextualize findings and interpretations in the interests of convergent stakeholders. Papers are invited that address the following (or related) topics:

  • practical demonstration and comparative analysis of R&R in remote sensing workflows;
  • R&R innovations in geospatial unmanned aircraft systems (UAS), artificial intelligence and deep learning, big data, or other remote sensing trends;
  • innovations and interoperability in provenance-enabled remote sensing services, methodologies, or workflows and corresponding implications for convergent GIScience stakeholder interests;
  • historical analysis of R&R in remote sensing workflows over the last 50 years, particularly as they relate to changing geopolitical, economic, industrial, computational, academic, and other drivers; or
  • best practices to enable R&R in international geospatial capacity building.

Prof. Dr. Jason A. Tullis
Guest Editor

Manuscript Submission Information

Manuscripts should be submitted online at www.mdpi.com by registering and logging in to this website. Once you are registered, click here to go to the submission form. Manuscripts can be submitted until the deadline. All submissions that pass pre-check are peer-reviewed. Accepted papers will be published continuously in the journal (as soon as accepted) and will be listed together on the special issue website. Research articles, review articles as well as short communications are invited. For planned papers, a title and short abstract (about 100 words) can be sent to the Editorial Office for announcement on this website.

Submitted manuscripts should not have been published previously, nor be under consideration for publication elsewhere (except conference proceedings papers). All manuscripts are thoroughly refereed through a single-blind peer-review process. A guide for authors and other relevant information for submission of manuscripts is available on the Instructions for Authors page. Remote Sensing is an international peer-reviewed open access semimonthly journal published by MDPI.

Please visit the Instructions for Authors page before submitting a manuscript. The Article Processing Charge (APC) for publication in this open access journal is 2700 CHF (Swiss Francs). Submitted papers should be well formatted and use good English. Authors may use MDPI's English editing service prior to publication or during author revisions.

Keywords

  • reproducibility
  • replicability
  • remote sensing
  • workflows
  • provenance
  • interoperability
  • convergence
  • capacity building
  • unmanned aircraft systems (UAS)
  • artificial intelligence
  • deep learning
  • geoprocessing
  • image processing
  • versioned source control
  • ethics in GIScience

Published Papers (4 papers)

Download All Papers
Order results
Result details
Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:

Research

Jump to: Review

12 pages, 1062 KiB  
Communication
Enhancing Reproducibility and Replicability in Remote Sensing Deep Learning Research and Practice
by Aaron E. Maxwell, Michelle S. Bester and Christopher A. Ramezan
Remote Sens. 2022, 14(22), 5760; https://doi.org/10.3390/rs14225760 - 15 Nov 2022
Cited by 2 | Viewed by 1800
Abstract
Many issues can reduce the reproducibility and replicability of deep learning (DL) research and application in remote sensing, including the complexity and customizability of architectures, variable model training and assessment processes and practice, inability to fully control random components of the modeling workflow, [...] Read more.
Many issues can reduce the reproducibility and replicability of deep learning (DL) research and application in remote sensing, including the complexity and customizability of architectures, variable model training and assessment processes and practice, inability to fully control random components of the modeling workflow, data leakage, computational demands, and the inherent nature of the process, which is complex, difficult to perform systematically, and challenging to fully document. This communication discusses key issues associated with convolutional neural network (CNN)-based DL in remote sensing for undertaking semantic segmentation, object detection, and instance segmentation tasks and offers suggestions for best practices for enhancing reproducibility and replicability and the subsequent utility of research results, proposed workflows, and generated data. We also highlight lingering issues and challenges facing researchers as they attempt to improve the reproducibility and replicability of their experiments. Full article
(This article belongs to the Special Issue Reproducibility and Replicability in Remote Sensing Workflows)
►▼ Show Figures

Graphical abstract

16 pages, 958 KiB  
Article
How to Improve the Reproducibility, Replicability, and Extensibility of Remote Sensing Research
by Peter Kedron and Amy E. Frazier
Remote Sens. 2022, 14(21), 5471; https://doi.org/10.3390/rs14215471 - 31 Oct 2022
Cited by 2 | Viewed by 2563
Abstract
The field of remote sensing has undergone a remarkable shift where vast amounts of imagery are now readily available to researchers. New technologies, such as uncrewed aircraft systems, make it possible for anyone with a moderate budget to gather their own remotely sensed [...] Read more.
The field of remote sensing has undergone a remarkable shift where vast amounts of imagery are now readily available to researchers. New technologies, such as uncrewed aircraft systems, make it possible for anyone with a moderate budget to gather their own remotely sensed data, and methodological innovations have added flexibility for processing and analyzing data. These changes create both the opportunity and need to reproduce, replicate, and compare remote sensing methods and results across spatial contexts, measurement systems, and computational infrastructures. Reproducing and replicating research is key to understanding the credibility of studies and extending recent advances into new discoveries. However, reproducibility and replicability (R&R) remain issues in remote sensing because many studies cannot be independently recreated and validated. Enhancing the R&R of remote sensing research will require significant time and effort by the research community. However, making remote sensing research reproducible and replicable does not need to be a burden. In this paper, we discuss R&R in the context of remote sensing and link the recent changes in the field to key barriers hindering R&R while discussing how researchers can overcome those barriers. We argue for the development of two research streams in the field: (1) the coordinated execution of organized sequences of forward-looking replications, and (2) the introduction of benchmark datasets that can be used to test the replicability of results and methods. Full article
(This article belongs to the Special Issue Reproducibility and Replicability in Remote Sensing Workflows)
►▼ Show Figures

Figure 1

27 pages, 11785 KiB  
Article
Revisiting the Past: Replicability of a Historic Long-Term Vegetation Dynamics Assessment in the Era of Big Data Analytics
by David Frantz, Patrick Hostert, Philippe Rufin, Stefan Ernst, Achim Röder and Sebastian van der Linden
Remote Sens. 2022, 14(3), 597; https://doi.org/10.3390/rs14030597 - 26 Jan 2022
Cited by 9 | Viewed by 3818
Abstract
Open and analysis-ready data, as well as methodological and technical advancements have resulted in an unprecedented capability for observing the Earth’s land surfaces. Over 10 years ago, Landsat time series analyses were inevitably limited to a few expensive images from carefully selected acquisition [...] Read more.
Open and analysis-ready data, as well as methodological and technical advancements have resulted in an unprecedented capability for observing the Earth’s land surfaces. Over 10 years ago, Landsat time series analyses were inevitably limited to a few expensive images from carefully selected acquisition dates. Yet, such a static selection may have introduced uncertainties when spatial or inter-annual variability in seasonal vegetation growth were large. As seminal pre-open-data-era papers are still heavily cited, variations of their workflows are still widely used, too. Thus, here we quantitatively assessed the level of agreement between an approach using carefully selected images and a state-of-the-art analysis that uses all available images. We reproduced a representative case study from the year 2003 that for the first time used annual Landsat time series to assess long-term vegetation dynamics in a semi-arid Mediterranean ecosystem in Crete, Greece. We replicated this assessment using all available data paired with a time series method based on land surface phenology metrics. Results differed fundamentally because the volatile timing of statically selected images relative to the phenological cycle introduced systematic uncertainty. We further applied lessons learned to arrive at a more nuanced and information-enriched vegetation dynamics description by decomposing vegetation cover into woody and herbaceous components, followed by a syndrome-based classification of change and trend parameters. This allowed for a more reliable interpretation of vegetation changes and even permitted us to disentangle certain land-use change processes with opposite trajectories in the vegetation components that were not observable when solely analyzing total vegetation cover. The long-term budget of net cover change revealed that vegetation cover of both components has increased at large and that this process was mainly driven by gradual processes. We conclude that study designs based on static image selection strategies should be critically evaluated in the light of current data availability, analytical capabilities, and with regards to the ecosystem under investigation. We recommend using all available data and taking advantage of phenology-based approaches that remove the selection bias and hence reduce uncertainties in results. Full article
(This article belongs to the Special Issue Reproducibility and Replicability in Remote Sensing Workflows)
►▼ Show Figures

Figure 1

Review

Jump to: Research

24 pages, 1431 KiB  
Review
Context for Reproducibility and Replicability in Geospatial Unmanned Aircraft Systems
by Cassandra Howe and Jason A. Tullis
Remote Sens. 2022, 14(17), 4304; https://doi.org/10.3390/rs14174304 - 01 Sep 2022
Cited by 1 | Viewed by 1947
Abstract
Multiple scientific disciplines face a so-called crisis of reproducibility and replicability (R&R) in which the validity of methodologies is questioned due to an inability to confirm experimental results. Trust in information technology (IT)-intensive workflows within geographic information science (GIScience), remote sensing, and photogrammetry [...] Read more.
Multiple scientific disciplines face a so-called crisis of reproducibility and replicability (R&R) in which the validity of methodologies is questioned due to an inability to confirm experimental results. Trust in information technology (IT)-intensive workflows within geographic information science (GIScience), remote sensing, and photogrammetry depends on solutions to R&R challenges affecting multiple computationally driven disciplines. To date, there have only been very limited efforts to overcome R&R-related issues in remote sensing workflows in general, let alone those tied to unmanned aircraft systems (UAS) as a disruptive technology. This review identifies key barriers to, and suggests best practices for, R&R in geospatial UAS workflows as well as broader remote sensing applications. We examine both the relevance of R&R as well as existing support for R&R in remote sensing and photogrammetry assisted UAS workflows. Key barriers include: (1) awareness of time and resource requirements, (2) accessibility of provenance, metadata, and version control, (3) conceptualization of geographic problems, and (4) geographic variability between study areas. R&R in geospatial UAS applications can be facilitated through augmented access to provenance information for authorized stakeholders, and the establishment of R&R as an important aspect of UAS and related research design. Where ethically possible, future work should exemplify best practices for R&R research by publishing access to open data sets and workflows. Future work should also explore new avenues for access to source data, metadata, provenance, and methods to adapt principles of R&R according to geographic variability and stakeholder requirements. Full article
(This article belongs to the Special Issue Reproducibility and Replicability in Remote Sensing Workflows)
►▼ Show Figures

Graphical abstract

Show export options expand_more Show export options expand_less
Select all
Export citation of selected articles as:
 
Displaying articles 1-4
Back to TopTop