Remote Sensing in Field Crop Monitoring: A Comprehensive Review of Sensor Systems, Data Analyses and Recent Advances

Abstract

The key elements that underpin food security require the adaptation of agricultural systems to support productivity increases while minimizing inputs and the adverse effects of climate change. The advances in precision agriculture over the past few years have substantially enhanced the efficiency of applying spatially variable agronomic inputs for irrigation, such as fertilizers, pesticides, seeds, and water, and we can attribute them to the increasing number of innovations that utilize new technologies that are capable of monitoring field crops for varying spatial and temporal changes. Remote sensing technology is the primary driver of success in precision agriculture, along with other technologies, such as the Internet of Things (IoT), robotic systems, weather forecasting technology, and global positioning systems (GPSs). More specifically, multispectral imaging (MSI) and hyperspectral imaging (HSI) have made the monitoring of the field crop health to aid decision making and the application of spatially and temporally variable agronomic inputs possible. Furthermore, the fusion of remotely sensed multisource data—for instance, HSI and LiDAR (light detection and ranging) data fusion—has even made it possible to monitor the changes in different parts of an individual plant. To the best of our knowledge, in most reviews on this topic, the authors focus on specific methods and/or technologies, with few or no comprehensive reviews that expose researchers, and especially students, to the vast possible range of remote sensing technologies used in agriculture. In this article, we describe/evaluate the remote sensing (RS) technologies for field crop monitoring using spectral imaging, and we provide a thorough and discipline-specific starting point for researchers of different levels by supplying sufficient details and references. We also high light strengths and drawbacks of each technology, which will help readers select the most appropriate method for their intended uses.

1. Introduction

The urgent need to adopt precision agriculture practices that are related to the application of spatially variable inputs to improve the efficiency of agricultural production requires the deployment of accurate and reliable crop monitoring techniques to provide information on the spatial variation of key agronomic parameters. The world’s population is growing at its fastest speed, and researchers have estimated that it will reach over 9 billion people by 2050 [1]; however, all humankind still relies on agriculture to provide its most basic needs of food and fiber [2]. As has been reported by the Natural Environment Research Council [3], the adverse effects of climate change have also begun to substantially deteriorate agricultural productivity. Therefore, following an earlier remark by the technology executive committee of the United Nations Framework Convention on Climate Change (UNFCCC) (2014) [4], there is an urgent need to quickly embrace emerging agricultural technologies that are capable of increasing productivity to meet the growing food demand with minimum inputs while minimizing the adverse effects of climate change. In response, there is an ongoing global campaign to implement precision agriculture, and remote sensing technology is a vital element of its success [5]. According to Morisse [6], the key elements that underpin food security require the adaptation of agricultural systems to support increased productivity while minimizing the adverse effects of climate change.

The advent of remote sensing technology in agriculture offers a potentially effective means of extracting “state of the field crop” information to monitor crop growth. One of the most promising recent advancements in the field is hyperspectral imaging (HSI), which is a technology that combines spectroscopy and imaging [7]. Whereas imaging provides the intensity at every pixel of the image, spectroscopy provides a single spectrum, such that a spectral image provides a three-dimensional (3D) dataset, which is typically called a data cube. The other earlier technologies include multispectral imaging (MSI) and the conventional RGB imaging technique. The advancement in spectral imaging technology and optical sensing has enabled the development of more sophisticated MSI and HSI devices, with vast applications in agriculture including field crop monitoring [8] and food quality inspection [9]. The key advantage of spectral imaging in agriculture is that we can use it to nondestructively extract accurate phenotypic information over a large spatial range and within a given time frame [10,11]. We can then process this information and use it for holistic data-driven analyses and for making technical decisions for improving agricultural productivity [12].

In addition to field crop monitoring, spectral imaging has been extensively applied to both harvest and postharvest management systems for the purposes of grading and quality assurance, as well as for evaluating their overall acceptance [13]. Furthermore, researchers have also shown that HSI has the potential to discriminatively identify grain contaminants, and especially when they are physically (and sometimes visually) similar, which is a process that is burdensome and expensive when using traditional methods [14,15]. As the physical, chemical, and biological characteristics of food grains typically indicate their quality and safety [14,16], spectral imaging offers accurate, rapid, real-time, nondestructive, and nonchemical detection technologies to enhance safety and assure food quality. In this regard, researchers have conducted numerous grain analysis studies using spectral imaging, including the color classification of grain [17], virtuousness assessment of wheat kernels, identification of sound or stained grains [18,19], classification of vitreous and nonvitreous wheat kernels, and discrimination of wheat classes [18,20].

In this survey, we present a comprehensive review of the various remote sensing technology applications in agriculture. We place a major emphasis on field crop monitoring using spectral imaging to provide researchers with one-stop consolidated information on the topic. We divide the remainder of this article as follows: in Section 2, we provide an overview of the general theoretical background of spectral imaging for remote sensing ap-plications in agriculture. In Section 3, we present the data processing approaches and analyses methods, including the data fusion approaches that are applied in field crop monitoring. Finally, we highlight the conclusions that we drew from this survey in Section 4.

2. Theoretical Background

In this section, we present a detailed theoretical background on the topics related to the application of remote sensing technology in agriculture, with a major focus on the working principles of the sensors and radiation sources used and the commonly adopted imaging technology. Additionally, we discuss the key spectral properties of crops that facilitate the use of remote sensing technology in agriculture.

Remote sensing (RS) technology is the science of acquiring and measuring the information of certain properties of phenomena, objects, or materials without coming into direct contact with the subject under surveillance [11]. The remote sensing information is carried by electromagnetic radiation, which travels in space at the speed of light in the form of harmonic wave patterns at different wavelengths [21]. Fundamentally, the properties of these objects (or areas), in terms of their associated levels of electromagnetic energy, allow for their detection, as well as for delineating and distinguishing between them [22]. The most informative wavelengths in remote sensing cover the visible light (VIS), near-infrared (NIR), shortwave-infrared (SWIR), far-infrared (FIR), and microwave bands [21]. Although it is informative for scene analysis, not all this information is visible to the naked eye; therefore, researchers have developed dedicated sensors (which we describe in Section 2.1) and improved them over time to aid in the retrieval of the reflectance radiation from different scenes of interest.

2.1. Sensors Systems

Remote sensing sensors record data in either analog format, such as aerial photo-graphs taken from an aircraft mounted with a film camera, or digital format, such as a two-dimensional matrix (or image) composed of pixels that store electromagnetic radiation (EMR) values recorded by digital cameras/sensors that are mounted on a satellite or aircraft, which is more common at present [22]. These sensors come in two forms: passive and active. Passive remote sensing sensors record radiation that is reflected by or emitted from objects; consequently, they record naturally occurring EMR. Meanwhile, active sensors emit their own radiation, which interacts with the target object of study and returns to the measuring instrument [11]. For example, a radio detection and ranging (or RADAR) system emits artificial EMR towards the target and then records how much of that EMR is returned to the system by reflection [22].

In general, regardless of the type of sensor deployed, most agricultural remote sensing sensors are designed to record a specific portion of the electromagnetic radiation (as described in Section 2.2). Naturally, the sun emits radiation at all wavelengths (termed solar radiation); however, only a less-harmful portion of this radiation reaches the Earth’s surface. Radiation at shorter wavelengths is more energetic and thus has greater potential for harm. The Earth’s surface receives radiation that ranges from a small portion of ultra-violet (UV) radiation known as near-UV radiation, with wavelengths between 290 and 400 nm, to visible light, with wavelengths between 400 and 700 nm, and near-infrared (NIR) radiation, with wavelengths between 700 and 1100 nm. In contrast, radiation such as gamma rays (<0.1 nm), X-rays (0.01–10 nm), middle–extreme-ultraviolet rays (10–300 nm), shortwave-infrared (SWIR) rays (1100–2500 nm), mid-infrared (MIR) rays (2.5–50 µm), far-infrared (FIR) rays (FIR) (from 50 µm to 1 mm), microwave rays (from 1 mm to 1 m), and radio waves (1–30,000 m) are typically filtered before they reach the Earth’s surface.

2.2. Electromagnetic Spectrum

Remote sensing technology depends on the measurement and interpretation of EMR arrays. This radiation carries specific electrical and magnetic properties. We refer to the wavelength range that corresponds to the electromagnetic radiation as the electromagnetic spectrum. We can use the manner of the interaction between the electromagnetic spectrum with any material for the qualitative and quantitative analyses of various materials. Researchers often use the electromagnetic spectrum to analyze the various chemical and physical properties of food and agriculture materials [23]. The electromagnetic spectrum ranges from shorter wavelengths (including gamma and X-rays) to longer wavelengths (including microwaves and broadcast radio waves), as detailed in

Table 1. Broad categorization of the electromagnetic spectrum.

Broad Category	Wavelength (m) (Low to High)	Frequency (Hz) (High to Low)
Gamma radiation	$<{10}^{-11}$	$>3\times {10}^{19}$
X-ray radiation	${10}^{-9}-{10}^{-11}$	$3\times {10}^{17}-3\times {10}^{19}$
Ultraviolet radiation	$4\times {10}^{-7}-{10}^{-9}$	$7.5\times {10}^{14}-3\times {10}^{17}$
Visible radiation	$7\times {10}^{-7}-4\times {10}^{-7}$	$4.3\times {10}^{14}-7.5\times {10}^{14}$
Infrared radiation	$1\times {10}^{-5}-7\times {10}^{-7}$	$3\times {10}^{12}-4.3\times {10}^{14}$
Microwave radiation	$0.01-{10}^{-5}$	$3\times {10}^9-3\times {10}^{12}$
Radio waves	$>0.01$	$<3\times {10}^9$

. The three basic factors that define the electromagnetic spectrum are the frequency (f), wavelength (λ), and photon energy (E). The frequency refers to the number of cycles per unit time, which are measured in hertz (Hz) (number of cycles per second). The wavelength refers to the distance between two successive cycles, which is measured in meters (m), and which is inversely proportional to the frequency. The following equations describe the relationships between the frequency, wavelength, and energy:

$$f=\frac{c}{\lambda },$$

(1)

$$f=\frac{E}{h} ,$$

(2)

$$E=\frac{hc}{\lambda } ,$$

(3)

where h is the Planck’s constant (6.6206957 × 10⁻³⁴), and c is the speed of light in a vacuum (299,792,458 m/s).

Depending on the type of interaction with the object (Figure 1), we can use the electromagnetic spectrum in different types of spectroscopic techniques to study material properties. The most applied interaction types for agricultural applications include reflectance, where the radiation is bounced back in either regular or irregular directions; ab-sorption, where the electromagnetic radiation is absorbed by the object (e.g., in photosynthesis); transmission, where the object allows the passage of electromagnetic radiation; emission, where the object emits the electromagnetic radiation that is the result of an energy state transition (e.g., fluorescence).

The different categories of electromagnetic spectra (Table 1) have several applications in agricultural production chains. Although we focus on field crop monitoring in this paper, we present a brief overview and summary of the general applications of electromagnetic spectra with respect to the agricultural production value chain. In this article, the value chain includes in-field operations, product processing, and consumption.

2.2.1. Gamma-Ray Imaging and Spectroscopy

Gamma rays are quanta or photons of extremely high frequency, low wavelengths, and high-energy electromagnetic radiation that are emitted from naturally occurring iso-topes. This radiation is ionizing, and it has high energy and is capable of penetration for effective irradiation. The radioactive isotopes of the elements that are capable of emitting gamma radiation are called radionuclides. Most radionuclides naturally occur, except for potassium (K) and the decay series of uranium (U) and thorium (Th), which produce gamma rays with sufficient energy and intensity that we can measure them with gamma-ray spectroscopy [24]. Researchers have found these radionuclides in soils and rocks in varying amounts. For this reason, gamma-ray spectroscopy is applied in agricultural production processes to characterize the soil properties for arable farming. Mahmood [25] adopted proximal gamma-ray spectroscopy to predict several soil properties using windowed and full-spectrum analysis methods under both managed soil conditions and a conventional field. Their methods could be used to predict the clay, pH, and total nitrogen with good precision (R² ≥ 0.56) in managed soil, whereas they could only be used to predict the total nitrogen with good accuracy in an organic field. Thus, they concluded that gamma-ray spectroscopy can be used for soil characterization for which the seedbed condition is important, although it cannot be used to determine small differences in the soil structure. Strati [24] also proved the feasibility and reliability of proximal gamma-ray spectroscopy in the modeling of the soil water content in agricultural fields, considering one case study on a tomato field. Serafini [25] describes another application of proximal gamma-ray spectroscopy in precision agriculture for the discernment of the rain water and irrigation water in soil without any supporting meteorological information.

2.2.2. X-ray Imaging

X-ray radiation, which is similar to gamma radiation, has a high frequency and energy and low wavelengths, and it is often used for irradiation and plant breeding applications, as well as for soil property characterization [23]. An example application of X-ray imaging is in the topographic study of agricultural soils using X-ray fluorescence and gamma-ray spectroscopy. De Castilhos [26] used energy-dispersive X-ray fluorescence (EDXRF) and gamma-ray spectrometry data, combined with principal component analysis (PCA), to characterize the soil chemical properties and analyze the concentration variation within the topographic sequence and depth in an agricultural field. Researchers have applied portable X-ray fluorescence imaging to the evaluation of heavy metals in agricultural soils, including the reported assessment of the heavy metals and soil organic carbon, in agricultural landfills [27]. According to their results, the method can be used to predict the soil organic carbon reasonably well (validation R² = 0.7), without the depth as an auxiliary predictor. The results of this study are similar to those of [28], except that, in the latter, the authors integrated arc emission spectroscopy, but with the same aim of carrying out a rapid risk assessment of the heavy metals in agricultural soils.

With regard to Gamma-ray and X-ray imaging, it is notable that both these remote sensing applications are limited to only a short distance between the object of interest and sensing device and for a short time. To avoid confusion with wide area remote sensing, these methods of sensing are specifically referred to as proximal remote sensing. Although these techniques are commonly adopted laboratory-based studies, some recent articles project the future potential their application in the field for monitoring heavy metals in leaves. For instance, Antenozio et al. [29] demonstrated the suitability of the micro-X-ray fluorescence (μ-XRF) technique to monitor arsenic (a metalloid that is toxic to living organisms) accumulation in plants. Moreover, the study performed a detailed analysis at different stages of the development of the plant and at different plant organs. In addition, other studies show that proximal sensing techniques are useful for monitoring macro and micronutrients [30].

2.2.3. Ultraviolet Imaging and Spectroscopy

The ultraviolet (UV) region of the electromagnetic spectrum is characterized by wavelengths that are lower than those of visible light (Table 1). Ultraviolet (UV) spectroscopy has recently found increasing applications in different fields, such as food and agriculture, forensic sciences, astronomy, and microscopy [31]. In UV imaging, the viewing of the surface topology of a material without light penetration is possible, as UV light is absorbed onto the surface of the material. One of the applications of UV light is for spectroscopic imaging, for which it is used as an excitation source for the study of a given material under controlled conditions. Zhao and Nakano [32] divide UV imaging into reflected UV imaging, which is commonly applied in astronomy and forensic science, and fluorescence UV imaging, which is commonly used in molecular biology. Although the application of UV imaging and its mass adoption in agriculture is still a prominent research issue, in several studies, researchers have demonstrated its vast potential in the areas of agriculture and food products. Petal et al. [33] demonstrated the potential of a reflected UV imaging technique for the detection of hidden defects/ruptured tissues on the surfaces of mangoes.

2.2.4. Mid-Infrared Imaging and Spectroscopy

The mid-infrared region has a unique property: it provides us with the ability to detect biochemical compounds, such as the sugars and acids in leaves, as well as in other materials, such as corn, jellies, and food supplements [34]. Furthermore, the development and advancement of Fourier-transform infrared (FTIR) technology has substantially promoted the use of the mid-infrared region in the agricultural and food industries. FTIR technology, coupled with other advanced technologies, such as Raman and NIR spectroscopy, has vast applications in quality, authenticity, and adulteration detection in the food industry [35]. Additionally, mid-infrared imaging has many agricultural applications, including in the early detection of plant water stress [36], nondestructive qualitative testing of the lambda-cyhalothrin residues on vegetables [37], and prediction of the daily methane emissions of dairy cows [38], among other applications.

2.2.5. Thermal Infrared Imaging

Thermal imaging (or infrared thermography) is based on infrared thermal energy, and it is used to carry out the noncontact detection of surface temperatures by acquiring thermal variations and converting the thermal spectral reflectance into a visible image [39]. In agriculture, thermal imaging can be used for intelligent irrigation monitoring [40], aerial monitoring and the quantification of crop abiotic stresses [41], early pest infestation detection in crop lands [42], and the aerial monitoring and estimation of the evapotranspiration in agricultural fields [43], among other applications.

2.2.6. Microwave Imaging

Similar to other electromagnetic-spectrum-based imaging systems, microwave im-aging also represents a series of noninvasive and nondestructive techniques that were de-signed to sense materials or scenes to retrieve certain physical properties and/or infer in-formation about the condition of the target under study. Researchers have proposed the adoption of electromagnetic fields at microwave frequencies for inspecting unknown tar-gets or scenes [44]. In agriculture, thermal heating based on microwaves can be effectively used to disinfect food, as well as nonfood materials and soils, and (most importantly) to kill pests and bacteria. Ghavami [45] used microwave imaging to nondestructively discriminate between seeded and seedless samples of lemons and grapefruits. Some of the most recent advancements in microwave imaging techniques in agriculture include the nondestructive inspection of large-trunked trees (e.g., palms) for disease infection [46], and nondestructive crop root phenotyping to inspect for possible infections [47]. This research field has shown great potential in the estimation of the sizes of root tuber crops, such as carrots, potatoes, and onions, among others; thus, it provides a means of facilitating root tuber yield predictions.

2.2.7. Radio Wave Imaging

Radio waves, and particularly in the range from 3 kHz to 300 GHz, are commonly used in a variety of devices, including air conditioners, computer-related peripherals (e.g., wireless routers, keyboards, mice), and so on. The design and implementation of radio-based systems for food and agriculture applications has been advancing in recent years. Researchers have developed devices that work at low frequencies (e.g., less than 200 KHz) for the inspection of food products as air-coupled ultrasonic systems [48]. They have also developed technologies that utilize high frequencies (e.g., up to 210 GHz) for the imaging/detection of the materials in food products. For example, in one case study, the authors considered the imaging of crickets buried in flour [49]. In recent years, researchers have taken radio wave imaging in agriculture in various directions, such as underground soil sensing using subsurface radio wave propagation The present-day state-of-the-art technology the IoT was made possible through the ease of the networking and communication using radio waves.

2.2.8. Color and Near-Infrared Imaging and Spectroscopy

To this point, in the various abovementioned applications, the focus has been on ap-plications that use active-sensor-based systems. However, these technologies are not frequently used due to the relatively high costs of the artificial generation of the radiation and safely concerns, and especially in outdoor applications, as they are known to have harmful effects. Nonetheless, almost all commonly used remote sensing systems in agriculture are passive in nature, as they rely on naturally occurring EMR (i.e., radiation that survives atmospheric scattering and reaches the Earth’s surface), which thus lowers the associated costs. We present the electromagnetic spectrum regions that are commonly used in agricultural remote sensing for field crop monitoring in Figure 2. These regions range from the visible to near-infrared bands (400–2500 nm wavelength range).

The information associated with each region in the electromagnetic spectrum can be helpful for a particular purpose for understanding field crops. The visible spectrum (0.4–0.7 µm) is used for vision-based sensing (i.e., measures to quantify all valuable data ob-servable by the naked eye), such as chlorophyll studies, green indices, and morphological analyses of leaves and fruits [50]. Similarly, the infrared region (0.3–100 µm) is used to extract data that are hidden from the naked eye, such as the plant water content and stress-related indices [51]. However, the infrared (IR) spectrum that is used in remote sensing comprises two subcategories, based on the radiation properties: (1) the reflected IR spectrum (0.7–3.0 µm), which is used for remote sensing purposes in a similar manner to the visible spectrum; (2) the emitted or thermal IR spectrum (3.0–100 µm), which is radiation that is emitted from the Earth’s surface in the form of heat energy.

2.3. Imaging Techniques

Spectral imaging combines the conventional imaging and spectroscopic techniques into one system to produce both spatial and spectral information of a given scene. According to Xu and Mishra [52], this simplifies the simultaneous measurement of the multiple physical (e.g., size, shape, and color) and chemical (e.g., water, fat, sugar content) characteristics of the target scene(s). Spectral information reveals the spectral signature of the analyte of interest, which can be spatially visualized in the form of distribution maps. Spectral imaging data (x,y,λ) (Figure 3), which are also referred to as hypercubes, are three-dimensional (3D) data, including two spatial dimensions (x rows and y columns) and one spectral dimension (λ wavelengths). Although the field of spectral imaging has many inconsistent terminologies, such as spectral imaging and imaging spectroscopy (or imaging spectrometry), in this article, we use all three of these terms to refer to the same concept. The two major imaging techniques are multispectral imaging (MSI) and hyperspectral imaging (HSI).

2.3.1. Multi-Spectral Imaging

Multispectral imagery is composed of a few image layers of a given scene, with each layer acquired at a particular section (also called band) of the electromagnetic spectrum [53]. The most common multispectral sensors have 3–10 spectral band measurements at each pixel of the produced image. For example, Landsat 8 Operational Land Imager (OLI) and thermal infrared sensor (TIRS) images consist of nine spectral bands, with a spatial resolution of 30 m for Bands 1–7 and 9. Sentinel, Landsat, Quickbird, and Spot are well-known satellites that use multispectral sensors to produce images that are composed of bands in the wavelength range of 443–2190 nm (that is, from the ultra-blue band (coastal and aerosol studies) to the shortwave-infrared (SWIR) band) acquired at different spatial resolutions [54].

In most of the studies in the literature, the authors demonstrate the vast range of MSI applications in field crop monitoring, including the phenotyping of crop biomasses [55,56,57], influence of the reproductive organ (RO) on the crop bidirectional reflectance distribution function (BRDF) and NDVI [58], estimation of the photosynthetic pigment [58], and assessment of the nitrogen variability [59], among other applications.

Additionally, multispectral imaging has also been widely adopted in agriculture for other applications, including seed phenotyping for the purposes of seed variety/species segregation and/or quality grading [60], soil nutrient estimations in agricultural soils [61], and the monitoring of the chlorophyll contents in crops [62]. Furthermore, the use of multispectral imaging has improved the agricultural sector and the utilization of various cutting-edge technologies, such as machine-learning and big data analyses.

Despite the many MSI applications, there are some shortcomings to MSI technology, which the authors of [63] examine. One such shortcoming is the low number of discrete spectral bands that are retrieved to aid studies that extend beyond the plant indices. Researchers have also had color restoration problems (i.e., RGB representation) when using MSI due to the overlap of the visible and near-infrared spectral bands [64], among others. However, MSI remains the most applied technique in remote sensing for field crop monitoring, compared with HSI, which is because the HSI processing techniques are still largely under development.

2.3.2. Hyperspectral Imaging

Unlike MSI, which collects a few image layers of the same scene, when spectral images are captured for narrow spectral bands rather than discrete bands, the system is referred to as hyperspectral imaging (HSI). These systems include sensors that collect dozens or hundreds of spectral bands with a wide range of spectral coverages [65]. Moreover, in several studies, researchers have proven that HSI technology can outperform MSI in different agricultural-based applications due to its ability to detect and discriminate between the specific features of objects with several narrow contiguous spectral channels. For example, in digital soil mapping (DSM) [66,67,68], HSI yielded more promising results due to its ability to retrieve extensive spectral signatures. Moriya et al. [69] also report a better performance for the detection of citrus gummosis with hyperspectral images than with three-band multispectral images.

Additionally, because HSI acquires an entire spectrum at each point, there is no need to have prior knowledge of the scene because all the available information in the dataset can be extracted during the postprocessing and analysis [70]. Furthermore, the latter authors point out that HIS analyses also take advantage of the structural relationships among the different spectra in a neighborhood, allowing for more elaborate spectral structural models for the more accurate analysis and classification of images.

One other problem that is associated with HSI is referred to as “smile” or “frown”. Spectral smile curves are the spectral distortion to an across-stack wavelength shift from the central wavelength owing to the change in the dispersion angle with the field position. This problem is primarily associated with push broom sensors [71], and it is visualized as a wave shape. Nevertheless, HSI has found comprehensive applications not only in remote sensing field crop monitoring and other agricultural applications, but also in other fields of research, such as medical diagnosis [72] and facial recognition [73].

2.3.3. Approaches to Spectral Imaging

Scanning Systems

Researchers use three main spectral scanning technologies to obtain spatial and spectral information: point scanning, line scanning, and band sequential scanning [74]. Spectral scanning combines the dispersive spectrometer with raster scanning. As depicted in Figure 4, point scanning captures one spectral data point at a time, while line scanning captures a slit of spatial information [75]. However, these two scanning approaches require a longer period of time to collect the data cubes [74]. Similar to the point and line scan approaches, band sequential scanning can acquire a high-resolution 2D image one wavelength at a time. While it allows for the measurement of continuous spectra, the major limitation of scanning is the high losses at the entrance slit of the spectrometer, which leads to long acquisition times.

Snapshot Systems

The snapshot approaches use a matrix of bandpass filters on the detector surface and spectral systems, which employ a tunable spectral filter in front of a monochrome imaging camera. The approaches are straightforward and can obtain both the spatial and spectral information of a scene with one exposure, which makes them superior to the former methods, as no scanning is required. However, there are three major disadvantages to snapshot technology: (1) the induced image blurring due to motion artifacts; (2) low spectral resolution; and (3) a limited number of wavelengths. For this reason, scanning devices are the most widely used instruments for collecting spectral images [76].

Fourier Transform Spectroscopy

Fourier-transform (FT) spectroscopy is an alternative approach to measuring continuous spectra. It combines a monochrome imaging sensor with an interferometer [77]. The principle behind FT spectroscopy is that light is split into two collinear delayed replicas, and a detector is used to measure their interference pattern as a function of their relative delay. The FT of the resulting interferogram produces the continuous-intensity light spectrum [78].

FT spectroscopy has many advantages over the dispersive techniques of measuring continuous spectra, which include the simultaneous measurement of all the wavelengths, and the increase in the number of photons that reach the sensor, which produces a higher signal-to-noise ratio. Another powerful advantage is the variable spectral resolution, which is adjustable, depending on the interests of the individual end user, by varying the maximum scan delay of the interferometer via software, without affecting the throughput of the device. FT spectroscopy also has a higher throughput because there are no slits, as well as a high wavelength accuracy due to the laser beam calibration of the device. Finally, it allows the user to adjust the spatial resolution independent of the spectral resolution [77]. There are many FT-based spectroscopy devices commonly in use, including the Raman spectrometer [79] and Hadamard spectrometer [80], and most of them are best suited to indoor applications (i.e., laboratory and industrial process lines) due to their designs.

Recently, there have been advancements in FT spectroscopy for outdoor applications. One of the well-known devices is the “Hera Iperspettrale” developed by NIREOS [81]. The Hera Iperspettrale utilizes a patented common-path birefringent interferometer (CPI) [82] in combination with a bidimensional CMOS sensor. The CPI is known for overcoming the limitations of the Wollaston and Savart prism-based imagers, providing negligible chromatic dispersion and the small geometrical separation between the interfering replicas, which lead to a high degree of coherence at each pixel and strong interference modulation. The data cube is measured in the time domain by step-scanning a compact ultra-stable interferometer in front of the CMOS sensor. The FT at each and every pixel of the image is then automatically computed with software, which provides the final hyperspectral data cube. The spectrum at each pixel is a continuous curve; thus, the number of bands is virtually unlimited and is not defined by the hardware but rather by the software [81]. Its capability of generating a virtually unlimited number of bands makes it superior and more powerful than most well-known hyperspectral devices, and it is thus more suitable for increasing the depth of field crop monitoring.

3. Data Processing and Analysis Techniques

In this section, our intention is to expose the reader to the vastly different processing and analysis algorithms. In this context, data processing refers to the transformation of raw datasets into valuable and usable information, whereas data analysis refers to the derivation of the intelligence from the preprocessed data itself (i.e., creating insight and new information).

3.1. Preprocessing Methods

Every raw remotely sensed data point contains a number of artifacts and errors due to the working condition of the devices, the effect of the research environment, and other factors. Correcting such anomalies and/or noises (e.g., abnormal pixels, uneven bright-ness, unwanted regions, redundant data, etc.) is called preprocessing. These noises, if un-corrected, introduce incorrect and/or unrelated signals and affect the subsequent processing. Preprocessing refers to all the actions that are taken prior to the actual data analysis process. According to Famili et al. [83], a generic way of understanding data pre-processing is as the transformation $T$of the raw data vectors $X_{ik}$ to a set of new data vectors $Y_{ij}$; (i.e., $Y_{ij}=T\left(X_{ik}\right)$). In particular, $Y_{ij}$ preserves the valuable information in $X_{ik}$, eliminates at least one of the anomalies in $X_{ik}$, and is more useful than$X_{ik}$. The valuable information comprises the components of knowledge that exist in the data, such as meaningful patterns, and the goal of the data analysis is to explore and present this information in a meaningful manner.

The preprocessing of remotely sensed images prior to image analysis is essential due to its direct and important influence on the quality of the further analysis, improving the sub-sequent data-processing efficiency. We introduce the most common preprocessing methods for solving the abnormalities associated with the spectral images in agriculture in the following subsections. We also highlight the strengths and weaknesses of each method.

3.1.1. White and Dark Calibration

White and dark calibration, which is also known as pixel-level image calibration, is performed to convert the acquired light intensity into standard reflectance/transmittance values. In laboratory setups, it is also performed to eliminate random noise signals, which are sometimes referred to as the “dark current” [84]. A standard diffuse reflectance surface with a specific reflectance and completely blocked camera lens are the most common techniques used to take white and dark reference images, respectively [85]. To convert the measured absolute reference values of each pixel to relative reference values after acquiring the white reference and dark images, we apply the following equation:

$$R=\frac{I_m-I_d}{I_w-I_d}\times R_{w },$$

(4)

where $I_m$, $I_d$, and $I_r$ are the measured raw, dark, and white reflectance values, respectively. $R$ and $R_r$ are the calibrated relative reflectance value and reflectance factor of the white panel, respectively.

3.1.2. Compression of Data Size

As a way of improving the spectral resolution and removing redundant signals, re-searchers primarily utilize data size compression (i.e., increasing or reducing the dimensionality of the data) based on some predefined conditions. Spectral imaging often results in large-data-dimensional spaces, such as hyperspectral images, with up to several hundred (or even several thousand or millions in some cases) of pixels. These massive data cubes call for computationally powerful hardware machines, such as GPU-powered computers. According to the authors of [86], leaving this large dimensionality unchecked results in a phenomenon known as the curse of dimensionality (also called the Hughes effect). As a criterion, when the number of spectral channels is greater than one-third of the difference in the sample number and classes, then overfitting issues are likely to occur [87]. As a result, spectral data size compression, and particularly spectral dimensionality reduction, is essential to reducing the overfitting risk, saving storage space, and accelerating the computation. Many methods are used to achieve the size reduction goal, including cropping out rows and columns that do not contain useful information, pixel binning, and feature selection and extraction.

Pixel Binning

Binning is the process used to combine multiple pixel charges collected by several adjacent CCD pixels in the horizontal, vertical, or both directions into a single larger charge, which is known as the binned pixel (or superpixel) [88]. For example, take n and m to be the numbers of pixels that form superpixels in the vertical and horizontal directions, respectively; the binning process returns a single binned pixel, which increases the signal-to-noise ratio (SNR) and reduces the computational load [89]. However, pixel binning causes a loss of the image resolution that is equal to the binning level; thus, there is a tradeoff between increasing the SNR and losing resolution, for which a decision must be made [90].

Feature Selection and Extraction

To remove the redundant or irrelevant information from the original data, the isolation of several representative variables or features is essential. In the case of spectral images, feature selection (FS) is the process of identifying a subset of informative wavebands in an attempt to reduce the dimensionality and improve the accuracy in the subsequent image analysis, such as classification or regression, which requires the band selection, which is a challenging task, and which requires prior knowledge of the data at hand. To avoid a subjective selection, researchers have developed several mathematically proven methods to aid in the decision making, which include forward feature selection, Fisher scores, and ranking. Ranking is the commonly applied technique in spectral FS, as it al-lows for the sorting of all the wavebands according to certain criteria and the selection of those with large weight coefficients. According to the authors of [90], the obtained weight coefficients can be treated as the loadings for the principal component analysis (PCA) regression coefficients of calibration models [91], and correlation coefficients between new variables and targets [92], among others.

In contrast to FS that is aimed at isolating only the representative variables, feature extraction (FE) methods can be used to transform the original data into a new low-dimensional coordinate system using the original features/variables. Despite the fact that FE methods require all the original data in the analysis process, unlike FS methods, they still achieve higher classification accuracy [93]. The transformation of the original data into a new feature space is performed through projections, such as orthogonal centroid algorithms (OCAs), projection pursuit (PP), PCAs, minimum noise fraction (MNF), and independent component analyses (ICAs) [94]. Among these methods, the PCA algorithm is the most common method for both FS and FE in the preprocessing of spectral images (especially HSI). In PCA, a multivariate data matrix (X) is transformed into a new coordinate system to produce new uncorrelated orthogonal variables, which are referred to as principal components (PCs) or loadings (W). The PCs are automatically arranged in a descending order of dominance based on the associated eigenvalues such that the first PC contains the greatest variance, followed by the second, third, and so on [95]. To calculate the new PC score matrix (T), we use a mathematical expression: $\mathrm{T}=\mathrm{WX}$.

Because most of the useful information is contained in the first few PCs, the removal of PCs with small variances can eliminate redundant and unnecessary information. However, useful information and noise cannot be completely separated using PCs; thus, decreasing the number of PCs can cause the loss of useful information, whereas retaining too many PCs will retain the noise as well [93]. In the 1960s, researchers proposed solutions, such as using the eigenvalues according to the Kaiser criterion [96], or the scree test [97] to determine the number of PCs to retain for further analysis. Kaiser proposed keeping the eigenvalues equal to or greater than one in order to retain as much useful information as possible, whereas Cattell proposed the scree test to identify the point at which the smooth decrease in the eigenvalues appears to the right.

Another commonly used method to minimize noise before size compression is the spectral derivative categorized under the spectrum pretreatment approaches. The most commonly applied spectral derivative method is the Savitzky–Golay (SG) derivation algorithm [98], which is a method that uses a polynomial $p\left(x\right)$ to approximate the original spectral curve while minimizing the noise:

$$p\left(x\right)={{\displaystyle \sum }}_{i=0}^d{a}_i{x}^i,$$

(5)

The method uses the concept of a rolling window; thus, the increase in the order of the polynomial must be accompanied by an appropriate increase in the window width or else the produced polynomial will simply follow the noise oscillation. One other spectral derivative algorithm in the literature that minimizes the noise in spectral data is the Norris–Williams (NW) derivation [99]. Other spectrum pretreatment methods include multiplicative scatter (MSC) and standard normal variate (SNV), which are generally categorized as scatter correction approaches [100].

In addition to the abovementioned image calibration, size compression, and spectrum pretreatment approaches, another preprocessing technique is spectral unmixing (SU). SU is the procedure by which the measured spectrum of a mixed pixel is decom-posed into a cluster of constituent spectra (or endmembers) and a set of corresponding fractions or abundances that indicate the proportion of each endmember present in the pixel [101]. Unlike the abovementioned preprocessing methods, studies in which researchers apply SU to the analysis of spectral data in agriculture are few.

One other powerful preprocessing method for field crop monitoring is tasseled cap transformation [102], which was designed to analyze and map the vegetation and urban development changes detected by various satellite sensor systems. The transformation has several advantages, including a reduction in the amount of data from several multispectral bands to three primary components: brightness, greenness, and wetness (or yellow stuff for Landsat MSS), as well as a reduction in the atmospheric influences and noise components in the imagery, which enables a more accurate analysis. A unique characteristic of PCA is that it can be used to transform the image data into a new coordinate system with a new set of orthogonal axes. The brightness axis (primary axis), which is statistically derived, is calculated as the weighted sum of the reflectance of all the spectral bands, and it accounts for most of the variability in the image. The brightness is influenced by bare or partially covered soil and other manmade and natural features. The second axis orthogonal to the first axis is the greenness axis, which is associated with green vegetation, while the third axis, wetness, is orthogonal to the first two axes and is associated with the soil moisture, water, and other moist features.

Finally, in addition to the abovementioned classical approaches, image cleaning is a conventional approach to spectral data preprocessing. Generally, measured images of all the objects within the rectangular camera’s field of view are retrieved and not just the tar-get object(s), which means that the image contains other redundant and irrelevant regions. These irrelevant regions include, among others, the background and labels. Other artifacts are induced by specular reflection and the environmental and device conditions. There-fore, dealing with these unwanted pixels and regions is essential, as they can cause incorrect or unrelated signals and consequently affect the subsequent analysis. The methods that are used to deal with these artifacts include manual selection, thresholding, image filtering, and interactive selection [103].

3.2. Analysis Methods

For decades, there have been developments and advancements in the remote collection of information on and images of the Earth’s surface and the atmosphere using aircraft and satellites. The collection of this information enables the characterization of the natural features or physical objects on the ground, the observation of surface areas and objects on a systematic basis and the monitoring of their changes over time, and the ability to integrate these data with other information as a decision-making aid. Analyses of remote sensing data can enhance our understanding of the terrestrial surface or particular areas of interest in terms of the composition, form, and function. Over the years, the need for computer-based analyses of remotely sensed image data has been increasing, along with the increasing volumes and types of digital image data that are available from various platforms and sensors. Therefore, according to Tüshaus et al. [104], the effective utilization of remote sensing image data necessitates the construction of an accurate means of extracting the information that they contain in forms that are relevant to particular applications. In this section, we survey the most common analysis methods that are relevant to field-crop-monitoring applications, which include vegetation indices and machine-learning and deep-learning approaches.

3.2.1. Vegetation Indices

The remote sensing of vegetation using passive sensors provides information on the electromagnetic wave reflectance from canopies in the form of spectral reflectance signatures [105]. These signatures carry information about the state, biogeochemical composition, and structure of a leaf area and/or canopy [106]. Consequently, we normally calculate vegetation indices (VIs) as the ratio of two wavebands to differentiate an absorbing feature from a nonadsorbing reference feature. For example, VIs may provide a measure of the canopy greenness, which is influenced by several biophysical quantities related to the chlorophyll content and leaf area of the canopy [106].

Multispectral Vegetation Indices

The first vegetation index proposed in the literature was the ratio vegetation index (RVI) [107]. Vegetation indices are based on the principle that leaves absorb relatively more red than infrared light. The major RVI application is for biomass estimations and monitoring, and usually for those with high-density coverages, as this index is sensitive to vegetation and can be easily correlated with the plant biomass. According to this study, the RVI is sensitive to the atmospheric effect when the vegetation cover is scarce (less than 50% cover), which weakens its representation of the biomass.

Later, the authors of [108] proposed the difference vegetation index (DVI) to determine the changes in the soil background. Researchers developed several vegetation indices in the 1970s for different applications, including the normalized difference vegetation index (NDVI), which is a typical parameter for analyzing ground differences and crop development processes. The NDVI quantifies the crop reflectance characteristics by measuring the difference between the near-infrared and red bands. However, factors such as the soil brightness, canopy coverage, and shadow affect the NDVI results, so sensor calibration is required [105]. The NDVI is the most common index, and we mathematically calculate it as follows:

$$NDVI=\frac{R+NIR}{R-NIR} .$$

(6)

Up to this point, vegetation indices could not account for the atmospheric effects on the output of the scene analysis. Vegetation indices in which the atmospheric effects are considered started to emerge in the early 1990s. The first vegetation index in which the atmospheric effects were considered was the atmospherically resistant vegetation index (ARVI), proposed by the authors of [109]. This index is effective at reducing the dependence of the vegetation indices on the atmospheric effects, as quoted in the study. However, with the advent of UAVs, the effect of the atmospheric influence on the vegetation indices was substantially minimized, which then led to the emergence of several modified vegetation indices [105]. Researchers widely apply vegetation indices that are based on UAV remote sensing in agriculture due to their ability to intensify the visible light and NIR spectral reflectance differences for improved crop monitoring (

Table 2. Spectral vegetation indices for field crop monitoring (three indices considered under each category and/or sub-category).

Category	Type	Vegetation Index Name	Formula	Property Measured	References
Basic vegetation index	Ratio Vegetation index	Ratio vegetation index (RVI)	$\mathrm{RVI}=\frac{NIR}{R}$	Chlorophyll content	[110]
		Green Ratio Vegetation Index (GRVI)	$\mathrm{GRVI}=\frac{NIR}{G}$	Nitrogen content	[105]
		Chlorophyll index with red edge (CIrededge)	$\mathrm{CIrededge}=\frac{NIR}{REG}+1$	Chlorophyll content & Leaf area index	[111]
	Difference vegetation index	Difference vegetation index (DVI)	$\mathrm{DVI}=\mathrm{NIR}-\mathrm{R}$	Chlorophyll content	[110]
		Green difference vegetation index (DVIGRE)	$\mathrm{DVIGRE}=\mathrm{NIR}-\mathrm{G}$	Chlorophyll content	[110]
		Red edge difference vegetation index (DVIRED)	$\mathrm{DVIGRE}=\mathrm{NIR}-\mathrm{REG}$	Chlorophyll content	[110]
Functional vegetation index	Atmospherically adjusted vegetation index	Atmospherically resistant vegetation index (ARVI)	$\mathrm{AVRI}=\frac{\mathrm{NIR}-R_{RB}}{\mathrm{NIR}+R_{RB}}$ $R_{RB}=R-\gamma \left(R-B\right)$	-	[112]
		Green Atmospherically Resistant Index (GARI)	$\mathrm{GARI}=\frac{\mathrm{NIR}-\mathrm{G}+1.75\left(\mathrm{B}-\mathrm{R}\right)}{\mathrm{NIR}+\mathrm{G}-1.75\left(\mathrm{B}-\mathrm{R}\right)}$	Chlorophyll	[112]
		Visible Atmospherically Resistant Index (VARI)	$\mathrm{VARI}=\frac{\mathrm{G}-\mathrm{R}}{\mathrm{G}+\mathrm{R}-\mathrm{B}}$	Biomass	[113]
	Soil-adjusted vegetation index	Soil-adjusted vegetation index (SAVI)	$\mathrm{SAVI}=\frac{1.5\left(\mathrm{NIR}-\mathrm{R}\right)}{\mathrm{NIR}+\mathrm{R}+0.5}$	Nitrogen	[114]
		Optimized soil-adjusted vegetation index (OSAVI)	$\mathrm{OSAVI}=\frac{1.16\left(\mathrm{NIR}-\mathrm{R}\right)}{\mathrm{NIR}+\mathrm{R}+0.16}$	Chlorophyll	[112]
		Modified Soil-Adjusted Vegetation Index (MSAVI)	$\mathrm{MSAVI}=\frac{1.5\left(\mathrm{NIR}-\mathrm{R}\right)}{\mathrm{NIR}+\mathrm{R}}+0.5$	Chlorophyll	[112]
Modified vegetation index		Normalized difference vegetation index (NDVI)	$\mathrm{NDVI} =\frac{\mathrm{NIR}-\mathrm{R}}{\mathrm{NIR}+\mathrm{R}}$	Chlorophyll content	[115]
		Modified simple ratio (MSR)	$\mathrm{MSR} =\frac{\frac{\mathrm{NIR}}{R-1}}{{\left(\frac{\mathrm{NIR}}{R+1}\right)}^{0.5}}$	Chlorophyll content	[115]
		Normalized difference red edge (NDRE)	$\mathrm{NDRE} =\frac{NIR-REG}{NIR+REG}$	Nitrogen	[116]

Note: NIR, Near-infrared; R, Red; G, Green; REG, red-edge; and B, blue are the reflectance bands with wavelengths of 800–2500 nm, 620–50 nm, 495–570 nm, 700–800 nm, and 450–495 nm, respectively.

).

Hyperspectral Vegetation Indices

The abovementioned VIs are mainly suited to MSI, which provides low-resolution spectral information for fine-scale field crop monitoring. HSI is now available and is be-coming more popular because it provides hundreds of fine contiguous spectral bands that range from visible to infrared light. The first proposed HSI-based VIs were the red-edge-position (REP) VI [117], normalized pigment chlorophyll ratio index (NPCI) [118], and modified chlorophyll absorption in reflectance index (MCARI) [119], among others. These VIs have the same working concepts as multispectral-based VIs. However, in recent years, researchers have made advancements in the newly developed hyperspectral indices that are based on spectra curve areas and transformed spectra, such as the first-derivative, reciprocal, and logarithmic transformations.

For example, in order to effectively monitor the LAI of rice, the authors of [120] conducted multiple spectral transformations and vegetation index calculations using hyperspectral data, filtered the characteristic bands associated with the LAI using different processing methods, established three different models for comparison, and came up with the optimal monitoring model. According to their results, the correlation between the canopy spectrum and LAI was substantially improved after the first-derivative transformation.

The authors of [121] estimated the chlorophyll contents of crops from hyperspectral data using a normalized area over reflectance curve (NAOC). The index is based on the calculation of the area over the reflectance curve obtained by high-spectral-resolution reflectance measurements derived from the integral of the red–near-infrared interval divided by the maximum reflectance in that spectral region. According to their results, there was a linear correlation between the NAOC and leaf chlorophyll content. The method offers a simple way of estimating the leaf chlorophyll from a remote sensing hyperspectral image in a heterogeneous scene that is characterized by different crop and soil types, without the need for additional ground measurements. Thus, a single hyperspectral image is enough to establish a map of the leaf chlorophyll. Other VIs that are suited to hyperspectral images include those found in [122,123,124,125].

3.2.2. Machine Learning

Machine learning is a subset of artificial intelligence that involves the use and/or development of algorithms and statistical models that can learn from data and improve their performance over time on a specific task without being explicitly programmed. There are several different classes of machine learning algorithms, which can be broadly categorized based on the type of learning they perform: (1) Supervised learning algorithms, (2) Unsupervised learning algorithms, (3) Semi-supervised learning algorithms, and (4) Reinforcement learning algorithms. Furthermore, within these broad categories, machine learning can be subcategorized (based on the transformation approach used) into linear and nonlinear transformation regression algorithms (see Table 3).

Linear transformation regression algorithms are a type of machine learning algorithm that models the relationship between a dependent variable (also known as the response variable) and one or more independent variables (also known as the predictor variables) using a linear function. Thus, the model assumes that the relationship between the variables is linear, which means that the change in the dependent variable is directly proportional to the change in the independent variable(s).

There are several different types of linear regression algorithms that have been developed and used in the literature, including:

• Simple linear regression: This is the most basic type of linear regression, in which there is only one independent variable. It involves finding the values of the parameters that minimize the sum of the squared residuals (the difference between the observed value and the predicted value) for all observations in the data. It is used to model the relationship between two continuous variables, such as the prediction of crop yield based on the different amounts of fertilizer application.
• Multiple linear regression: This is a more general form of linear regression that allows for multiple independent variables. It is used to model the relationship between a dependent variable and several independent variables, for instance, assessing various factors that impact crop yield, such as soil nutrient levels, temperature, and precipitation.
• Ridge regression: This is a regularized form of linear regression that introduces a penalty term for large values of the regression coefficients. It is used to prevent overfitting and improve the generalizability of the model. Ridge regression is often used in situations where there are a large number of independent variables, and the risk of overfitting is high.
• Lasso regression: This is another regularized form of linear regression that introduces a penalty term for large values of the regression coefficients. Unlike ridge regression, which penalizes all large coefficients equally, lasso regression performs “feature selection” by setting some coefficients to zero, effectively eliminating them from the model. Lasso regression is often used in situations where there are many independent variables and some of them are not important for predicting the dependent variable.

On the other hand, nonlinear transformation regression algorithms comprise a machine learning algorithm that models the relationship between the dependent and independent variables using a nonlinear function. This means that the model does not assume a linear relationship between the variables, and the change in the dependent variable may not be directly proportional to the change in the independent variable(s). Nonlinear transformation regression algorithms can be more flexible and accurate than linear transformation algorithms when the relationship between the variables is more complex.

There are also several different types of nonlinear regression algorithms that have been developed and used in the literature, including:

• Polynomial regression: This is a type of nonlinear regression in which the relationship between the dependent variable and the independent variables is modeled using a polynomial function. Examples include quadratic regression (a second-order polynomial), cubic regression (a third-order polynomial), and higher-order polynomial regression.
• Exponential regression: This is a type of nonlinear regression in which the relationship between the dependent variable and the independent variables is modeled using an exponential function. Examples include linear exponential regression, logarithmic exponential regression, and power exponential regression.
• Logistic regression: This is a type of nonlinear regression that is used for binary classification, where the dependent variable can take on only two values (e.g., 0 or 1). It is used to model the probability regarding whether a given phenomenon is true (e.g., the probability that an image pixel belongs to a certain class).
• Neural networks: These are a type of nonlinear regression algorithm that are based on the structure and function of the human brain. They consist of multiple interconnected “neurons” that can learn complex relationships between the input variables and the output variable.
• Kernel regression: This is a type of nonlinear regression that uses a kernel function to model the relationship between the dependent variable and the independent variables. It is often used in situations where the data are not linearly separable.
• Gaussian process regression: This type of nonlinear regression uses a Gaussian process to model the relationship between the dependent variable and the independent variables. It is a Bayesian approach that can be used to model complex, nonlinear relationships and make probabilistic predictions about the output variable.

There are many applications of machine learning in various agricultural disciplines. For example, in the field of crop monitoring, machine learning algorithms can be used to analyze data from remote sensing technologies, such as satellite imagery and spectral imaging, to predict and monitor various crop characteristics, including yield, quality, and pests and diseases. Machine learning algorithms can also be used to optimize irrigation and fertilization strategies, or to improve the efficiency of precision farming techniques.

Remotely sensed data processing and analysis have been greatly enhanced with the development of machine-learning (ML) approaches due to their ability to handle both linear and nonlinear problems [126]. In general, their main areas of application are classification, clustering, regression, and dimension reduction [127]. Specifically, agriculture has substantially benefited from the intrinsic capabilities of ML to enhance on-farm activities, such as field crop monitoring [128,129]. Machine learning comprises algorithms that are capable of learning from data without being explicitly programmed. In this section, we present a detailed discussion on the three broad categories of machine learning: unsupervised, supervised, and reinforcement learning.

Table 3. Most common machine learning approaches to hyperspectral image analysis for field crop monitoring.

Algorithm Name	Learning	Transformation Approach	Sample Studies	Performance	References
Support vector machine	Supervised	Nonlinear	Support Vector Machines for crop/weed identification in maize fields	93.1%	[130]
			Quantification of Nitrogen Status in Rice by Least Squares Support Vector Machines and Reflectance Spectroscopy	94.2%	[131]
			Detection of scab in wheat ears using in situ hyperspectral data and support vector machine optimized by genetic algorithm	75.0%	[132]
Decision tree	Supervised	Nonlinear	Mapping Cynodon Dactylon Infesting Cover Crops with an Automatic Decision Tree-OBIA Procedure and UAV Imagery for Precision Viticulture	89.8%	[133]
			Automation and integration of growth monitoring in plants (with disease prediction) and crop prediction	>95.0%	[134]
			Greenness identification based on HSV decision tree	−	[135]
Random forest	Supervised	Nonlinear	Predicting Biomass and Yield in a Tomato Phenotyping Experiment Using UAV Imagery and Random Forest	−	[136]
			An Automatic Random Forest-OBIA Algorithm for Early Weed Mapping between and within Crop Rows Using UAV Imagery	87.9%	[133]
			Predicting Canopy Nitrogen Content in Citrus-Trees Using Random Forest Algorithm Associated to Spectral Vegetation Indices from UAV-Imagery	R2 = 0.9	[137]
K-nearest neighbors	Supervised	Nonlinear	Estimation of nitrogen nutrition index in rice from UAV RGB images coupled with machine learning algorithms	R2 > 0.5	[138]
			Performance Analysis of k-Nearest Neighbor Method for the Weed Detection	>93.0%	[139]
			Early Weed Detection Using Image Processing and Machine Learning Techniques in an Australian Chili Farm	63.0%	[140]
Naïve Bayes	Supervised	Linear	AI Crop Predictor and Weed Detector Using Wireless Technologies: A Smart Application for Farmers	89.3%	[141]
			Identification of Soybean Foliar Diseases Using Unmanned Aerial Vehicle Images	95.0%	[142]
			Naïve Bayes Classification of High-Resolution Aerial Imagery	94.0%	[143]
Logistic regression	Supervised	Nonlinear	Biomass Estimation Using 3D Data from Unmanned Aerial Vehicle Imagery in a Tropical Woodland	R2 = 0.65	[144]
			The Predictive Power of Regression Models to Determine Grass Weed Infestations in Cereals Based on Drone Imagery—Statistical and Practical Aspects	83.0%	[145]
			Automation solutions for the evaluation of plant health in corn fields	79.2%	[146]
Linear discriminant analysis	Supervised	Linear	Weed detection with Unmanned Aerial Vehicles in agricultural systems	87.0%	[147]
			Using continuous wavelet analysis for monitoring wheat yellow rust in different infestation stages based on unmanned aerial vehicle hyperspectral images	−	[148]
K-means clustering	Unsupervised	Linear	Rice yield estimation based on k-means clustering with graph-cut segmentation using low-altitude UAV images	67.0%	[149]
			Wheat ear counting using k-means clustering segmentation and convolutional neural network	>98.0%	[150]
			Detection of tomatoes using spectral-spatial methods in remotely sensed RGB images captured by UAV	88.2%	[151]
Principal component analysis	Unsupervised	Linear	Use of principal components of UAV-acquired narrow-band multispectral imagery to map the diverse low stature vegetation fraction of absorbed photosynthetically active radiation (fAPAR)	77.0%	[152]
			The Extraction of Wheat Lodging Area in UAV’s Image Used Spectral and Texture Features	87.0%	[153]
			Monitoring Agronomic Parameters of Winter Wheat Crops with Low-Cost UAV Imagery	0.7 < R2 < 0.97	[154]
Independent component analysis	Unsupervised	Linear	Field heterogeneity detection based on the modified Fast ICA RGB-image processing	78.0–89.0%	[155]
Gaussian Process Regression	Supervised	Nonlinear	Biomass estimation in batch biotechnological processes by Bayesian Gaussian process regression	−	[156]
			Spectral band selection for vegetation properties retrieval using Gaussian processes regression	79.0–95.0	[157]

Table 3. Most common machine learning approaches to hyperspectral image analysis for field crop monitoring.

Algorithm Name	Learning	Transformation Approach	Sample Studies	Performance	References
Support vector machine	Supervised	Nonlinear	Support Vector Machines for crop/weed identification in maize fields	93.1%	[130]
			Quantification of Nitrogen Status in Rice by Least Squares Support Vector Machines and Reflectance Spectroscopy	94.2%	[131]
			Detection of scab in wheat ears using in situ hyperspectral data and support vector machine optimized by genetic algorithm	75.0%	[132]
Decision tree	Supervised	Nonlinear	Mapping Cynodon Dactylon Infesting Cover Crops with an Automatic Decision Tree-OBIA Procedure and UAV Imagery for Precision Viticulture	89.8%	[133]
			Automation and integration of growth monitoring in plants (with disease prediction) and crop prediction	>95.0%	[134]
			Greenness identification based on HSV decision tree	−	[135]
Random forest	Supervised	Nonlinear	Predicting Biomass and Yield in a Tomato Phenotyping Experiment Using UAV Imagery and Random Forest	−	[136]
			An Automatic Random Forest-OBIA Algorithm for Early Weed Mapping between and within Crop Rows Using UAV Imagery	87.9%	[133]
			Predicting Canopy Nitrogen Content in Citrus-Trees Using Random Forest Algorithm Associated to Spectral Vegetation Indices from UAV-Imagery	R2 = 0.9	[137]
K-nearest neighbors	Supervised	Nonlinear	Estimation of nitrogen nutrition index in rice from UAV RGB images coupled with machine learning algorithms	R2 > 0.5	[138]
			Performance Analysis of k-Nearest Neighbor Method for the Weed Detection	>93.0%	[139]
			Early Weed Detection Using Image Processing and Machine Learning Techniques in an Australian Chili Farm	63.0%	[140]
Naïve Bayes	Supervised	Linear	AI Crop Predictor and Weed Detector Using Wireless Technologies: A Smart Application for Farmers	89.3%	[141]
			Identification of Soybean Foliar Diseases Using Unmanned Aerial Vehicle Images	95.0%	[142]
			Naïve Bayes Classification of High-Resolution Aerial Imagery	94.0%	[143]
Logistic regression	Supervised	Nonlinear	Biomass Estimation Using 3D Data from Unmanned Aerial Vehicle Imagery in a Tropical Woodland	R2 = 0.65	[144]
			The Predictive Power of Regression Models to Determine Grass Weed Infestations in Cereals Based on Drone Imagery—Statistical and Practical Aspects	83.0%	[145]
			Automation solutions for the evaluation of plant health in corn fields	79.2%	[146]
Linear discriminant analysis	Supervised	Linear	Weed detection with Unmanned Aerial Vehicles in agricultural systems	87.0%	[147]
			Using continuous wavelet analysis for monitoring wheat yellow rust in different infestation stages based on unmanned aerial vehicle hyperspectral images	−	[148]
K-means clustering	Unsupervised	Linear	Rice yield estimation based on k-means clustering with graph-cut segmentation using low-altitude UAV images	67.0%	[149]
			Wheat ear counting using k-means clustering segmentation and convolutional neural network	>98.0%	[150]
			Detection of tomatoes using spectral-spatial methods in remotely sensed RGB images captured by UAV	88.2%	[151]
Principal component analysis	Unsupervised	Linear	Use of principal components of UAV-acquired narrow-band multispectral imagery to map the diverse low stature vegetation fraction of absorbed photosynthetically active radiation (fAPAR)	77.0%	[152]
			The Extraction of Wheat Lodging Area in UAV’s Image Used Spectral and Texture Features	87.0%	[153]
			Monitoring Agronomic Parameters of Winter Wheat Crops with Low-Cost UAV Imagery	0.7 < R2 < 0.97	[154]
Independent component analysis	Unsupervised	Linear	Field heterogeneity detection based on the modified Fast ICA RGB-image processing	78.0–89.0%	[155]
Gaussian Process Regression	Supervised	Nonlinear	Biomass estimation in batch biotechnological processes by Bayesian Gaussian process regression	−	[156]
			Spectral band selection for vegetation properties retrieval using Gaussian processes regression	79.0–95.0	[157]

Supervised Machine Learning

Supervised machine learning involves training a model using labeled data, which then helps in the classification and/or prediction of future data. According to the authors of [158], the labeled data guide the machine to search for a specific pattern in the data; thus, supervised learning implies the use of a sample set in which the output is known. Classification and regression are the major subdomains of supervised learning. The tools that are commonly applied to achieve the tasks in this subdomain include support vector machines, decision trees, random forests, logistic regression, naïve Bayes, linear discriminant analyses, and artificial neural networks.

Support Vector Machine

The support vector machine is one of the most common tools in machine learning. SVMs can be used to ascertain the optimal hyperplanes that separate the points of one class from those of others due to their ability to select the planes that pass through the largest possible gaps between the points of distinct classes. The classification of the new points is then performed depending on which side of the plane they reside on. This capability of creating optimal hyperplanes also has the advantage of reducing generalization errors, and thus the chance of overfitting. However, in most studies, researchers have demonstrated that support vector machines are effective for high-dimensional spaces that require learning from several features, and the authors of [159] also found that SVMs perform equally well on relatively small datasets with few data points. Furthermore, they require less memory storage, as only a subset of points is required to represent the boundary surface. However, it should be noted that SVMs require intensive computation during the model training, and they do not return the confidence level of a prediction, which we can calculate using k-fold cross-validation [160].

In recent years, studies on field crop monitoring using support vector machines have yielded promising results, indicating their potential to improve agricultural productivity. Among these studies, the authors of [161] focused on developing a systematic model based on SVM for the monitoring of sugarcane crops. The model considers the temperature, humidity, and moisture as the monitoring parameters responsible for the healthy growth of crops, and it can also be used to detect any trace of disease infection (if any) through the aerial images acquired at regular intervals. A validation accuracy of 96% was achieved; thus, this method is reliable for the monitoring of growing crops. In another study on field crop monitoring based on SVM, the authors intended to automate the detection of crop leaf diseases from remotely sensed images [162]. The constructed algorithm achieved a good validation accuracy (87.6%), outperforming the other tested auxiliary models. Other relevant recent studies include autumn crop yield prediction using SVM [163], the detection of scab in wheat ears using HSI and SVM [132], and the identification of irrigated crop types using datasets derived from the Sentinel-2 satellite [164], among others.

Decision Trees

Decision trees are methods used for data discrimination, and they have tree-like structures with internal nodes that represent the test feature targets, of which each branch represents the result of a test, and each leaf node represents the class label. Therefore, a decision tree comprises three types of nodes: root nodes, internal nodes, and leaf nodes, and it follows a classification rule that represents the pathway from root to leaf. The decision taken at each node is based on Boolean tests. The authors of [158] describe it as a greedy algorithm, as it has a top-down iterative divide-and-rule structure. Initially, the root nodes take the entire training dataset, which is iteratively split, depending on the selected features. This process is repeated at each node, at which the test features are split based on certain decision tree functions, such as entropy and the Gini index. Entropy provides the amount of disorder in each set, and it returns zero if all the points in the target classes are the same, whereas the Gini index measures the criteria required to reduce the probability of misclassification. The algorithm result can be enhanced through a technique known as ensemble learning. Researchers widely use decision trees, as they do not require the creation of auxiliary dummy variables. However, according to the authors of [165], one major issue with decision trees is the large growth of the trees, which results in one observation per leaf. Researchers have used different spectral preprocessing techniques that employ decision tree algorithms, such as Savitzky–Golay derivatives [98], multiplicative signal correction (MSC) [166], and normalization [167], to improve the spectral features.

In the field of agricultural crop monitoring, researchers have employed decision trees in different studies, including one in which the authors focused on the greenness identification from images captured over field crops for growth monitoring [135]. A recent application of the decision tree algorithm in crop monitoring was the development of a monitoring system for the grain loss of paddy rice by the authors of [168]. Their system performed well, with an average test accuracy of 99.3% at a moisture content of 30% and a grain:impurity ratio of 1:2.5. However, as the moisture content and grain:impurity ratio increased, the accuracy of the system decreased, which implies that the optimal conditions need to be ascertained for the best possible results.

Random Forest

A random forest is a modified form of the decision tree algorithm in which a decision tree is created with a subset of training samples that are selected on a random basis with replacements. Similarly, a random number of features is also taken from each set of features. Random forests follow this process of decision tree construction many times, which results in a set of classifiers. As with decision trees, each grown tree in the random forest algorithm predicts its target class at the time of the prediction for every instance. Therefore, the class that is the most predicted by the trees becomes the suggested class of the final classifier. However, the choice of the number of parameters requires more attention. Che [169] argued that, generally, the higher the number of trees, the better the performance of the obtained classifier; however, this comes with a high computational cost. The performances of random forests are more prone to overfitting because they involve the multiple averaging of the decision trees [170]. Randal and Olson [171] suggested that a reduction in the size of the bootstrap sample, which may also increase the randomness of the algorithm, reduces the overfitting effect; however, they also mentioned that reducing the size of the bootstraps somewhat affects the overall performance of the classifier. In this regard, in most practical applications, the bootstrap size is equal to the size of the initial training set to provide a fair tradeoff between the variance and bias [133].

Researchers have employed random forests for HSI analysis in several ways, including in the detection of plant diseases, fungal infection, and bruises in fruits and vegetables, the classification of different agricultural products, and quality analyses of processed fish products [172]. In a recent study in which the authors deployed the random forest algorithm for field crop monitoring, they focused on predicting the biomasses and yields of individual tomato plants from UAV imagery [136], as the biomass and yield are fundamental variables for assessing the production and performance of agricultural crops and/or systems [136].

k-Nearest Neighbors

k-nearest neighbors (k-NN) is a machine-learning algorithm in which the classification of new cases is based on the classification of their nearest neighbors. For example, the authors of [173] state that “If it walks like a duck, quacks like a duck, and looks like a duck, then it’s probably a duck”, which means that the classification of a given pixel is based on how similar it is to its neighboring pixels. The principle that underlies k-NN classification is the k-NN rule: a test data point is assigned to the class that is most repeatedly represented among the k-nearest training data points. For cases in which two or more such classes exist, then the test data point is assigned to the class with the minimum average distance to it. The most common distance calculated in k-NN is the Euclidian distance (Frobenius); however, we can also use other distance metrics to calculate the distance points, such as the Manhattan distance (city block) and P-norm distance (Minkowsky) [174]. According to the latter study, it is important to ascertain the optimal k value to find a good balance between under- and overfitting. A small k value will render the model prone to noisy points, and if the k value is too large, then the neighborhood may encompass points from other classes. Nevertheless, the main advantages of k-NN are that the cost of the learning process is low, with no required optimization, and it is easy to program with high accuracy.

The simplicity, ease of programming, and high accuracy recorded in the literature regarding the k-NN method have greatly encouraged its adoption in different spectral analyses for field crop monitoring, such as crop canopy classification, crop disease monitoring, and crop nutrient content analyses [139]. Additionally, the k-NN method is a fast and reliable way of finding the best wavelet disintegration layer and for the effective wavelength selection of HSI data via the fusion of the wavelet basis functions and k-nearest neighbors.

Naïve Bayes

The naïve Bayes (NB) model is a probabilistic model that assumes conditional independence (i.e., Bayes’ theorem) among the predictor variables to describe the outcome probabilities of related events. At first, for each data point associated with a class, we calculate a posterior probability with the aid of the prior class probabilities and class conditional probabilities of the variables once we know the class level. We follow this by the assignment of the observation to the class with the highest posterior probability [175]. The main weakness of this algorithm is its assumption of conditional independence, which means that it is unable to learn the interactions between two predictor variables/features [158].

There have been successful applications of the NB model in HSI field crop monitoring, including in crop prediction [176] and remote sensing imagery classification [177], among others. Although in most of these studies, the authors achieved high classification accuracies (over 85%) with naïve Bayes, the authors of [158] argue that the reported classification accuracy of the NB classifier has also been lower than 75%, which indicates a weak classification model.

Logistic Regression

Researchers commonly use logistic regression (LR) to evaluate the relationship be-tween independent variables (features) and a dependent variable (label, to be predicted) by calculating the probabilities using the logistic function [178]. LR belongs to a class of discriminative models classified as supervised machine learning. Researchers primarily use it for land-cover classification in remote sensing, and particularly for pixel-wise classification; however, it also serves as a building component for more complex algorithms that use ensemble and deep learning [179]. The working principle of LR is that it models the probability distribution of a given class as the logistic function of the weighted sum of the input features. In other words, researchers employ LR for the binary classification of materials, as it returns discrete binary outcomes between 0 and 1. Logistic regression is widely used due to its accurate performance, simplicity, and ease of interpretation. However, the main disadvantage of logistic regression is that it cannot be used to model nonlinear problems for which the data are not linearly separable, and it is also prone to overfitting [145].

Linear Discriminant Analysis

Linear discriminant analysis (LDA) involves ascertaining the projection hyperplane that minimizes the interclass variance and maximizes the distance between the projected means of the classes. LDA is based on the principle of defining a lower-dimensional feature subspace in which the data point separability is optimized [180]. One advantage of using linear discriminant analysis is that a generalized eigenvalue system can be solved to obtain a solution to a given problem, which allows for the fast and massive processing of data samples. Researchers commonly use LDA in feature extraction, as it improves the computational efficiency and minimizes the level of overfitting that arises from the Hughes phenomenon in nonregularized models [181]. Although researchers commonly use LDA for binary class problems, they have also proposed its use for multiclass generalization [180].

Due to its robustness, LDA is widely used in field crop monitoring based on hyper-spectral data [180,182,183,184]. In a recent study, Alajas et al. [185] used a hybrid LDA and decision tree approach to extract useful features for the classification of healthy and diseased grape leaves and to predict the percentage of the damaged area, for which they achieved a high accuracy (over 97%).

Partial Least-Squares Discriminant Analysis

Partial least squares discriminant analysis (PLS-DA) is one of the most common incorporated supervised ML methods in the present day chemometric analysis and software packages. Barker et al. [186] describe the well-established theoretical background of PLS-DA, which is derived from the two concepts of partial least squares (PLS) and LDA. Researchers developed the PLS algorithm to linearize models with nonlinear parameters used for overdetermined regression problems [187]. The algorithm was initially implemented through the nonlinear iterative partial least squares (NIPALS) algorithm [188]. However, currently, advancements in classical multivariate statistical theories have enabled the implementation of PLS in solving well-posed eigenstructure problems that clearly exhibit the PLS strike relationships between the variance summary and score correlation. Combining the advantages of PLS and LDA (i.e., the maximization of the intergroup variability relative to the measure of the shared intragroup variability), PLS-DA is one of the most powerful tools in chemometrics.

In the field of agricultural remote sensing, Stellacci et al. [189] compared the PLS-DA model performance in the selection of the optimal hyperspectral bands to discriminate the nitrogen status in durum wheat with other statistical methods. They performed the quantification of the leaf-nitrogen concentrations on samples collected at the same locations and on the same dates, and they used them as the response variables in regressive methods. The authors demonstrated that PLS-DA is best suited for response variables. They also applied PLS-DA to map the damage from rice diseases. According to their results, the PLS-DA could classify the rice diseases at the subfield scale, with an overall accuracy of 75.62%. The authors also report that the approach was later successfully applied during a typical ecoepidemic outbreak of rice dwarf, rice blast, and glume blight in China. Other applications of PLS-DA in crop monitoring are the monitoring and characterization of crops and plants [190], the qualitative and quantitative diagnoses of the nitrogen nutrition of tea plants under field conditions [191], pest and disease detection in horticulture crops by a field robot [192], and commercial tree species discrimination from airborne hyperspectral imagery [193] among other applications.

Gaussian Process Regression (GPR)

Gaussian processes (GPs) are a type of probabilistic model that can be used for both regression and classification tasks in machine learning. They are based on the idea of using a Gaussian distribution to represent the joint distribution of the function and its inputs. GPR have a number of attractive properties, including the ability to handle input uncertainty and the ability to provide uncertainty estimates for predictions.

One of the key advantages of GPR is that they can be used to make predictions even when there is limited or noisy data available. This is because GPR takes into account the correlations between the input variables and the output variable, rather than just relying on the mean of the output variable [194]. This allows GPR to make more informed predictions, even when there is a small amount of data available, whereas the main challenges with using GPR is that they can be computationally intensive, especially when working with large datasets. However, there are a number of techniques that can be used to reduce the computational complexity of GPR, such as using approximations [194,195] or restricting the model to a subset of the data [196].

The authors of [197] discuss the various advances in the field of GPR that have been made in recent years, including new algorithms that take into account the characteristics of the signal and noise in the data, as well as techniques that use automatic relevance kernels to extract knowledge and rank features automatically.

For purposes of field crop monitoring, GPR has been applied to field crop spectral analysis in many ways; for instance, GPR was applied in the retrieval of chlorophyl content from hyperspectral reflectance data collected using an airborne system [198] and wheat leaf rust disease detection [199]. Several research surveys confirm that GPR outperforms most machine learning algorithms [199,200].

Ridge Regression

Ridge regression is one of the methods for estimating the parameters of a linear regression model, commonly used to deal with collinearity, a problem that often occurs in multiple linear regression. Ridge regression is similar to least squares regression, but it adds a penalty term to the cost function that helps to reduce the magnitude of the model coefficients. This helps to prevent overfitting and improve the generalization of the model to new data.

The key advantage of ridge regression is that it can be used to perform feature selection, since the penalty term encourages the model to give smaller coefficients to less important variables. This can be useful in situations where there are a large number of predictor variables, as it can help to reduce the complexity of the model and improve its interpretability. One of its main drawbacks is that it can be sensitive to the scale of the predictor variables, which can make it difficult to compare the importance of different variables. Additionally, ridge regression may not perform well when there are a large number of irrelevant predictor variables, as the penalty term may not be sufficient to reduce the impact of these variables on the model. At the same time, better versions of ridge regression continue to be developed within the machine learning community. For instance, the authors of [201] refined the ridge regression method by proposing new ridge parameters to improve the model performance.

In the field of crop monitoring, ridge regression has been applied in various studies including some recent studies, such as the study authored by Ahmed et al. [202], to estimate wheat yield by integrating the kernel ridge regression (KRR) method coupled with complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) and the grey wolf optimization (GWO), and that of Ji et al. [120], to quantitatively monitor leaf area index (LAI) in rice based on hyperspectral feature bands and ridge regression algorithm. Also with an airborne multispectral scanner, the authors of [203] applied Ridge Regression method to estimate the leaf chlorophyll concentration using high-resolution (2 cm) images for crop health monitoring.

Lasso Regression

Least absolute shrinkage and selection operator (LASSO) is one other type of linear regression that is used to perform variable selection (a process of selecting the most important predictor variables to include in the model). It does this by adding a penalty term to the cost function that constrains the model to give smaller coefficients to less important variables. Unlike ridge regression, which penalizes all large coefficients equally, LASSO regression performs “feature selection” by setting some coefficients to zero, effectively eliminating them from the model. The penalty term is typically a function of the L1 norm, which is the sum of the absolute values of the model coefficients [204]. The main aim of LASSO regression is to find the model coefficients that minimize the sum of the squared errors between the predicted values and the observed values, subject to the constraint that the sum of the absolute values of the coefficients is less than a certain threshold. This constraint helps to reduce the complexity of the model and improve its interpretability, particularly when there are a large number of predictor variables.

Several studies relating to field crop monitoring have investigated the applicability of LASSO in this field. For example, the authors of [205] applied LASSO regression to model crop yield prediction, and also Haumont et al. [206] adopted LASSO regression to model Leek dry-biomass and nitrogen uptake across multiple sites and growing seasons based on multispectral UAV imagery.

There are some limitations to lasso regression as well. Similar to ridge regression, LASSO regression can also be sensitive to the scale of the predictor variables [204]. Furthermore, the authors of [205] highlighted that LASSO regression may also not be effective if there are a lot of predictor variables that are not relevant to the model, as the penalty term may not be strong enough to minimize the influence of these variables on the model.

Unsupervised Machine Learning

Unsupervised machine learning encompasses all the ML techniques that do not re-quire the user to supervise the model but allow the model to work by itself to discover the patterns and information in a given dataset. Therefore, researchers primarily use them to deal with unlabeled data. Unsupervised learning models enable the performance of more complex processing tasks compared with supervised learning [207]; however, they can be more unpredictable than supervised ML methods [208]. In this section, we discuss the three most common unsupervised learning algorithms for agricultural purposes.

k-Means Clustering

k-means clustering is a partitioning-based technique for grouping based on the iterative rearrangement of the data points between clusters. Its principle of operation is that it divides the dataset variables into nonoverlapping clusters or groups based on the uncovered features to produce clusters of variables with a high level of intracluster similarity and a low level of intercluster similarity [209]. The algorithm begins by randomly selecting any centroids (i.e., average locations or arithmetic means of all the points), denoted by the symbol k. The nearest points to each centroid point are assigned to that specific cluster. This is followed by re-calculation of the new centroid by averaging the position coordinates of all the points present in that cluster. The process is iterated until convergence of the clusters occurs [210].

Its ease of implementation, efficiency of computation, and low memory requirements have made k-means clustering one of the most widely applied unsupervised machine-learning methods in the exploratory data analyses and data mining in any field of research, including hyperspectral image analyses. Additionally, k-means clustering has also been adopted as an initial step for more computationally demanding algorithms, such as Gaussian mixtures, providing an approximate separation of the data as a starting point, and minimizing the noise that is present in the dataset [209]. Some recent applications of k-means clustering in field crop monitoring include the detection and classification of the pests in crops based on proximal images. In particular, Faithpraise et al. [211] used k-means clustering for the classification of 10 pest species from images captured in the field. Wang et al. [212] applied k-means clustering to segment pests (white flies) from crop leaves as a prerequisite step for intelligent pest diagnosis. Concerning the latter study, the authors specifically applied k-means clustering to the classification of each pixel after the preliminary step of image gridding, and they used the RGB color space to preselect the potential cluster centers. Researchers have also used k-means clustering to enhance the results produced by deep-learning models [213].

Principal Component Analysis

Principle component analysis is a dimensionality reduction algorithm through which multidimensional data are reduced to a lower-dimensional space while retaining most of the useful information. The method inputs a set of possibly correlated variables to create a new set of linearly uncorrelated variables. The 1st, 2nd, …, nth coordinates of the projected data represent the highest variance, followed by the second-highest variance, and so on. Every newly retrieved coordinate is termed a principal component (PC). Of the total number of PCs obtained (which is equal to the number of original dimensions in the data), those with higher variances are chosen [214].

The key advantages of PCA are its low sensitivity to noise, low computational capacity and memory requirement, and increased efficiency, as the process takes place in a smaller dimension. However, in earlier studies, researchers found that its major disadvantage is that it makes it difficult to accurately evaluate the covariance matrix [215]. According to the latter study, Li et al. [216] reported this disadvantage in a study in which even the simplest invariance could not be captured by the PCA unless availed with such information. Nonetheless, PCAs have been widely used for field crop monitoring, both in earlier and recent years. Villez et al. [217] assessed the application of unfold PCA for online plant stress monitoring and sensor failure detection in two truss tomato plants and young apple plants grown in a greenhouse. Among the recent studies, Skotadis et al. [218] used PCA to analyze the unique response patterns generated after the exposure of the sensing array to two gas analytes to automate the real-time detection of pesticides within the scope of smart farming. Additionally, Danner et al. [219] also applied PCA to reduce the amount of input data from >200 bands to 15 components.

Independent Component Analysis

Independent component analysis (ICA) is a probability-based method that is also categorized under the dimensionality reduction machine-learning algorithms. Its main goal is to retrieve the essential maximally independent and non-Gaussian component signals from the observed mixed data signals [220]. In other words, according to Pati et al. [221], ICA is an extended PCA designed to optimize the non-Gaussian nature (or minimize the Gaussian nature) of the dataset(s). While PCA is based on the principle of the optimization of the covariance matrix of the second-order statistical data, ICA is based on the optimization of higher-order statistics, such as kurtosis; thus, it yields independent components (unlike PCA, which yields uncorrelated components). However, according to Saha [221], there are only a limited number of studies in which the researchers have applied ICA in spectral data exploration and analysis. However, from a general perspective, we can use ICA transformation to transform hyperspectral data into independent components to better complete the detection and separation of the hidden noise in the image and perform dimensionality reduction, noise minimization, anomaly detection, classification and spectral endmember extraction, and data fusion [222]. In the latter study, the authors compared the performances of different feature transforms, including the PCA, minimum noise fraction (MNF), and ICA transform, and according to their results, PCA outperformed ICA and MNF.

Reinforcement Machine Learning

Reinforcement learning is among the three elementary machine-learning categories, together with supervised and unsupervised learning. It imitates how humans and animals learn without a mentor, and it is concerned with how the software agents in a given environment are rewarded when they complete an assigned task by themselves [223]. According to Monakhova [220], reinforcement-learning tasks are commonly modeled as Markov decision processes (MDPs), which involve a 5-tuple (S,A,P,R,Ɣ), where S represents a finite set of states, A represents a finite action set, P represents the transition probability, R represents the re-ward immediately provided, and Ɣ represents the discount factor. Due to its robustness, reinforcement learning has the potential to transform the essence of automation in agriculture, as it can be used to teach robots to modify their behaviors according to the relationship between them and the surrounding environment [224]. Zhang [225] conducted a case study of such an application on fully automating the UAV aerial scouting of whole crop fields based on reinforcement learning and convolutional neural networks (CNNs) to enhance precision agriculture. The researchers designed a fully autonomous aerial surveying approach that preserves the battery life by sampling field sections to sense and predict the crop health of the whole field. When compared with the conventional nonautomated UAS scouting approach, their autonomous scouting approach sampled 40% of the field and predicted the crop health with 89.8% accuracy, substantially reducing the labor costs and increasing the agricultural profits. Other applications of reinforcement learning in the literature include autonomous greenhouse control for crop growth monitoring [225], among others.

3.2.3. Deep Learning

Deep learning is an extended (“deeper”) form of the machine-learning algorithm that researchers use to explicitly extract features from raw data for classification, clustering, regression, and detection. While there are several deep-learning networks in use, the most commonly applied network for image analysis is the convolutional neural network (CNN) [226]. Typically, the structure of a deep-learning algorithm for image analysis consists of three basic layers: convolutional layers, pooling layers, and fully connected layers. The convolutional layers are made up of kernels (filters) that are responsible for the extraction of distinct features, such as edges, from the input image. The pooling layer reduces the spatial size of the input image data and thereby reduces the computational requirements in the subsequent processes. The most common technique employed in the pooling process is max-pooling [227]. Finally, in the fully connected layer, all the nodes in the first layer of the network system are interconnected with all the nodes in the second layer. Depending on the employed network architecture (e.g., unsupervised pretrained network, CNN, recurrent neural net-work, or recursive neural network), other layers, such as gates, memory cells, activation functions, encoding/decoding schemes, and so on, can be appended to the three basic layers detailed above [228].

The use of deep-learning algorithms has substantially benefited hyperspectral imaging (

Table 4. Deep learning approaches for hyperspectral image analysis in field crop monitoring.

Category	Network Architecture	Backbone	Performance	References
Pixel-based	SegNet	VGG-16	89.8%	[229]
	FCN	AlexNet	86.7%	[231]
	DeepLabv3+	ResNet-18	97.0%	[229]
	LeNet	LeNet	92.2%	[232]
	Mask R-CNN		91.8%	[233]
	FDN-92			[234]
Object-based	Faster R-CNN	Inception-v2, VGG, ZF, ResNet-50	87.2%	[235,236]
	RetinaNet	ResNet	93.0%	[237,238]
	YOLOv3	DarkNet-53	94.0%	[237,236]
	YOLOv2	DarkNet-19	96.6%	[236,239]
	SSD	VGG-16	83.2%	[239]

). Some of the major applications in hyperspectral crop growth monitoring include the assessment of the crop water stress, crop canopy classification, crop pest detection, and the detection of disease symptoms in leaves. Deep-learning models may be either pixel-based (usually for clustering purposes) or object-based (usually for object detection (e.g., pest detection)). In terms of the pixel-based models, Der Yang [229] employed UAV images with the aid of deep-learning image processing to estimate the rice lodging over a large area of paddies. They established an image semantic segmentation model that uses two neural network architectures—FCN-AlexNet and SegNet—due to their good performances in the interpretation of objects of various sizes and their computation efficiencies. For the identification of the rice lodging from the images, they achieved F1 scores of 0.80 for the FCN-AlexNet and 0.79 for the SegNet. Similarly, researchers have applied object-based deep-learning models in field crop monitoring. In a case study, Wang [230] developed a custom triple-branch HSI–LiDAR convolutional-neural-network backbone to simultaneously extract the spatial, spectral, and elevation features of land-cover objects, including those on agricultural lands. They particularly used the single-shot-detector (SSD) architecture based on the VGG-16 backbone, and they attained a performance accu-racy of 83.2%.

3.3. Data Fusion

Remote sensing primarily provides data in four dimensions: the spatial (2D), spectral (1D), temporal (1D), and radiometric (referring to numerical precision) dimensions [240]. Recent advancements in sensors have made it possible to retrieve a wide variety of information from different materials within a given area of interest. This information ranges from spectral information obtained from passive sensors (i.e., MSI and HSI data) to height and shape information obtained from LiDAR sensors, as well as to texture, amplitude, and phase information via a synthetic aperture radar (SAR). The availability of these different forms of data from different sensors requires the integration of the diverse information that they contain to enhance the scene classification and object detection capabilities. However, according to the authors of [241], we require more efforts to automate the interpretation of remote sensing data, and especially considering the recent attempts in remote sensing data fusion.

Nonetheless, all these datasets provide different representations of the same physical scene. To handle these different representations, data fusion offers potential solutions and has been widely explored over the last decade, as researchers have attempted to determine the best possible combination of these remotely sensed datasets. Generally, all tasks that require any kind of parameter estimation derived from multisource data can benefit from the use of data fusion methods [242]. For instance, researchers commonly apply the spectral information from HSI to discriminate various materials based on their reflectance values in agricultural monitoring [243]. Likewise, we can use LiDAR image point clouds (e.g., sorghum plant phenotyping (Figure 5)) to analyze the depth, height, and volumetric information, which is useful for distinguishing objects within the same scene or different parts of the same object. Furthermore, LiDAR systems have been adopted even in marking agricultural land boundaries for the purpose of control in the system of direct payments for agriculture (Integrated Administration Control System—IACS) [244]. Therefore, the integration of all these multisource information into a single analysis pipeline to describe a particular scene can greatly improve the output of such analysis. For this reason, data fusion has received enormous attention from researchers across the globe for a wide variety of applications.

Most researchers in the literature have defined data fusion as a technique that com-bines data from multiple sensors and the related information from associated databases to achieve improved accuracy and more specific inferences than those achieved when using a single sensor. Similarly, in a concise manner, we refer to data fusion as a means of combining multisource data to achieve the improved accuracy of the data analysis, as op-posed to single-source data processing, with the major aim of harnessing remote sensing for field crop monitoring.

In this subsection, we bring together the advances in multisource and multitemporal data fusion approaches with respect to field crop monitoring by providing an overview and discipline-specific starting point, supplying the necessary details and references, for new researchers at different levels (i.e., students, and researchers) who are eager to con-duct novel investigations on this challenging topic. More specifically, we dedicate this section to the topics of point cloud data fusion, spatiotemporal fusion, and spatiospectral fusion.

3.3.1. Point Cloud Data Fusion

As we discuss in the previous sections of this article, researchers have proposed and/or deployed many methodologies to perform several operations on spectral images for field crop monitoring. In a similar manner, point cloud data have been adopted for many agricultural applications, such as crop phenotyping analyses [245,246,247,248] and the autonomous navigation of farm tractors and robots in agricultural environments [249,250].

Although we will draw a broader picture, the discipline-specific starting point is the point cloud data model, which is the initial data model common to all multisource fusion methods that include point clouds. The authors of [251] define the point cloud data model as a set of georeferenced points in 3D Cartesian space that is related to a geospatial reference system (e.g., the Universal Transverse Mercator). We present the visualization of a point cloud and its features in Figure 5. The point features can originate from an active measurement process, such as the LiDAR intensity [252] or they can be derived from postprocessing techniques, such as spectral reconstruction [253].

The end goals of point cloud data fusion are to utilize the 3D geometric, spatial structural, and lidar backscatter information inherent in point clouds and fuse it with spectral data sources, or in some cases, geoinformation layers, such as geographic information system (GIS) data. We can generally categorize the methodological approaches to point cloud data fusion in the literature into three main concepts: (1) the point cloud level, at which the primary point cloud is enriched with new point features; (2) the image/voxel level, at which new image layers are derived that represent the 3D point cloud information; (3) the feature level, at which the point cloud information is fused on the segment/object level [240]. The choice of method is largely dependent on the target model after the data fusion. For example, the classification of a dataset calls for point cloud or image level processing.

The concept of enriching the point cloud with features (also known as point cloud coregistration or alignment) involves texturing the point clouds with the image data, usually from either a multisensor system, such as lidar systems mounted with spectral cam-eras, or from photogrammetric point clouds (i.e., the structure from motion and dense image matching). Under this, precise coregistration is used to transfer the labels of the classified spectral data to the corresponding 3D points. With this approach, the authors of [254] related the spectra from 2D multispectral aerial imagery and thermal imagery to 3D point cloud crop models to classify vines in different vigor classes. According to their results, the discriminant power of single indices can be increased and enhanced by their combination, with the best performance obtained using the whole set.

Unlike the point cloud level methodological concept, the image/voxel level is con-cerned with the transformation of point clouds into 2D images or voxels that can be ana-lyzed by the image-processing approaches discussed in Section 3.2 of this article. We can directly derive a number of 2D images from the rich point clouds that come from the point cloud geometry, lidar backscatter, and full-waveform lidar data. Finally, the feature/object level concept is based on the first two concepts in terms of the data model, which is used to derive objects, followed by a classification step. We can use approaches such as image or point cloud segmentation to derive the attributes for the classification.

Hyperspectral and LiDAR Data Fusion

HSI technology has certain limitations in terms of differentiating between objects that are composed of the same spectral characteristics (e.g., the same crop species under the same health conditions). In the same way, LiDAR point cloud data alone cannot differentiate between objects in absolute proximity and of the same size, shape, and elevation (e.g., the Eleusineindica species in a finger millet field) due to the lack of spectral information pro-vided by these sensors.

Advancements in multisensor data fusion to take advantage of the information pro-vided by the different sensors that simultaneously capture a particular scene provide potential solutions to the abovementioned issues [255]. Researchers have developed many computer vision algorithms to implement such a solution, including machine-learning and deep-learning algorithms. For HSI and LiDAR data fusion, we can broadly categorize the available techniques in the literature into three major categories: data association, state estimation, and decision fusion [242].

The aim of data association (also known as pixel-level fusion) is to improve the quality via the fusion of the oriented observation data. State estimation (also called feature-level fusion) is carried out by extracting features from different data sources to create new features or feature vectors, which are usually adopted to facilitate the subsequent application. Decision-level fusion inputs different sensor data to separately interpret the land-cover features and obtain the land-cover feature categories, and it then applies certain decision rules to fuse them. As there is typically a large gap in the imaging patterns between different sensors, the most commonly implemented fusion strategy for HSI and LiDAR fusion is feature-level fusion [256]; however, more recently, researchers have combined feature-level fusion with decision fusion to improve the fusion accuracy. Furthermore, they have widely applied CNN architectures in HSI and LiDAR fusion due to their robust feature representation abilities and ability to naturally adapt to images [256]. As there is a considerably large number of publications in the literature regarding data fusion, including the basic steps involved, we do not intend to review all the studies in this section. Instead, we aim to highlight studies in which the authors used the most recent techniques for hyperspectral and LiDAR data fusion. In particular, we explore the methods that use convolutional neural network (CNN) architectures in detail, considering that they are more powerful than conventional techniques for supervised inference tasks [256].

Li [257] proposed a CNN and composite kernel-based novel framework for the fusion of hyper-spectral images and LiDAR-derived elevation data (Figure 6). They took advantage of a combination of extinction profiles (first proposed for use in remote sensing data processing by Ghamisi et al. [258]; this is a technique-based extinction filter using extrema-oriented connected independent filters) and CNN features to enable the joint use of low- and high-level features to improve the classification performance. They initially applied extinction profiles to both data sources to extract the spatial and elevation features from hyperspectral and Li-DAR-derived data, respectively. Next, they designed a three-stream CNN to independently extract the useful spectral, spatial, and elevation features from both data sources. Instead of a simple stacking strategy, as first proposed in [241,259], they again designed a multisensor composite kernel (MCK) scheme to fuse the heterogeneous spectral, spatial, and elevation features extracted by the CNN. This scheme resulted in the higher spectral, spatial, and elevation separability of the extracted features, which enabled them to effectively perform multisensor data fusion in the kernel space.

The simple stacking strategy proposed in [241,259] requires a fully connected layer to fuse the feature extracts; however, fully connected layers often contain a substantially large number of parameters, which increases the training difficulty when considering a small number of training samples. Support vector machines and extreme-learning machines, with their composite kernel versions, were the two machine-learning classifiers employed to produce the final classification result. The concept of the kernel method here refers to using a nonlinear mapping function (e.g., Hilbert kernel) to transfer the input data from the original feature space into a higher feature space, such that the nonlinear problem in the feature space can be transformed into a linear problem. Furthermore, the theoretical elegance of the kernel trick makes it an effective tool for HSI analysis due to its insensitivity to the Hughes phenomenon (also referred to as the curse of dimensionality).

The researchers applied the proposed framework on two commonly used public datasets with different characteristics: an urban data set captured over Houston, the United States, and a rural dataset captured over Trento, Italy. The framework yielded high overall accuracies of 92.57% and 97.91% for the Houston and Trento datasets, respectively. Moreover, according to the authors, the proposed fusion framework can be regarded as a general data fusion framework that can be applied to any other dataset containing both hyperspectral and LiDAR data. They particularly highlight its use in agricultural data to better classify certain classes (e.g., healthy or stressed crops).

Ge et al. [260] extended the work of Ghamisi [257] by proposing a residual fusion strategy that combines the extinction profile (EP) extraction method with the local binary pattern (LBP) method to improve the accuracy and reliability of the feature extraction. They fed the combined output of the two feature extraction strategies into a kernel collaborative representation-based classifier with Tikhonov regularization (KCRT), which Li et al. [261] demonstrated as superior to collaborative representation-based classification (CRC), first proposed by Wang et al. [262], which is because the regularization term in the original CRC could not be adaptively adjusted according to the similarity between the training and testing samples.

Later on, Xia et al. [263] proposed another approach: semisupervised graph fusion (SSGF). Particularly, they introduced the unsupervised fusion graph into the semisupervised local discriminant analysis (SELD) framework proposed by the authors of [264,265] to learn the projection matrix. They first applied morphological filters to LiDAR data and the first few components of the hyperspectral data to model their respective heights and spatial information. Then, they used the proposed SSGF algorithm to generate the spectral, elevation, and spatial features in a lower subspace to obtain the new features. The objective of SSGF is to maximize the class separation ability and preserve the local neighborhood structure by using both labeled and unlabeled samples.

Although the CNN-based approaches have produced excellent performance measures up to this point, compared with conventional techniques, the main disadvantage of this approach is that the feature extraction of different data sources is individually performed, instead of jointly, exploiting the features from both modalities. According to Mohla [266], this results in the loss of some important shared high-level features from both modalities. Furthermore, Chen [267] earlier argued that such an approach may create a substantial imbalance among the different features, which can lead to inequality in the represented information. For this reason, a few researchers have attempted to address this issue.

Wang et al. [268] proposed a novel multiattentive hierarchical fusion net (MAHiDFNet) to achieve the feature-level fusion and classification of HSI and LiDAR data. Particularly, they first developed a three-branch HSI–LiDAR CNN backbone, similar to that in [257], to simultaneously extract the spatial, spectral, and elevation features of the land-cover objects. Mohla [266] first proposed this simultaneous feature extraction during HSI–LiDAR data fusion for use in the context of land-cover classification. The researchers fused the oriented feature embedding by applying a hierarchical fusion strategy based on the output of the triple-branch CNN. Then, they adopted an attention-module-based shallow feature fusion strategy to highlight the modality-specific and modality-integrated features. To construct the hierarchical feature fusion framework in the deep feature fusion stage, they fused the acquired multimodality features, and they finally fed the fused features into the classification module to obtain the classification result at the pixel level.

In summary, there have been substantial efforts made towards finding effective ways to deal with the large data that are currently being captured in agricultural environments. Despite these efforts, data and analysis gaps are still prevalent due to the fact that agricultural environments are mostly uncontrolled and there are a vast number of factors that need to be considered and properly measured for the full characterization of a given area. According to Silveira [269], one of these factors concerns the complexity that is associated with agricultural environments. For example, the information that is captured is not sufficient to cover all the variations found in real practice. The information captured from a single sensor is often incapable of providing explicit solutions, even if the problem at hand is well defined, and effective algorithms are applied. Fusing the information retrieved from different sensors that provide different data types is one possible solution that researchers have explored over the past few years. Data fusion enables the exploration of the complementarities and integration of different data types to obtain more reliable and useful information on the areas of analysis. Whereas some success has been achieved, as discussed in this section, there are still a number of challenges that hinder the more extensive adoption of this type of approach, and especially for agricultural areas with highly complex environments [270].

3.3.2. Spatiotemporal Fusion

As the size of the region-of-interest (ROI) continues to increase, such as in remote sensing at the regional and global scales, there is a tradeoff between the spatial resolution and temporal resolution (i.e., the temporal revisit frequency). As an example, NASA’s Moderate-Resolution Imaging Spectroradiometer (MODIS) sensor and Aqua satellites collect data at different spatial ranges, from 250 m to 1 km, which are often too coarse to ex-tract the explicit land-cover information that may exist at a finer spatial scale than the sensor resolution. The multispectral onboard Sentinel-2B instrument captures the Earth’s reflected radiance with a high revisit time (i.e., 5 days) and high spatial resolution (four bands at 10 m, six bands at 20 m, and three bands at 60 m) [271]. Thus, due to its low temporal resolution, it is not suitable for monitoring short-term disastrous agricultural events, such as floods.

However, due to recent technical advances, the remote sensing community now has access to both dense timeseries data and high spatial- and spectral-resolution images. For example, Sentinel-3 Ocean and the Land Color Imager (OLCI) sensor currently provide images at a spatial resolution of 300 m and a temporal resolution of 2.8 days, which are expected to increase upon the launch of Sentinel-3B. Moreover, Planet’s microsatellites acquire daily images at a spatial resolution of 3.125 m. Likewise, for relatively manageable small-scale applications, the recent advancements in remotely piloted aircraft systems, or drones, provide a considerable amount of multisource data with high spatial and temporal resolutions. However, these studies require access to historical data, which have fine temporal and spatial resolutions.

The authors of [271,272], along with those of other articles in the literature, provide in-depth well-established discussions and/or implementations of several spatiotemporal data fusion methods and techniques based on remote satellites. In the context of satellite remote sensing, spatiotemporal fusion can be performed either by combining low- and high-spatial resolution imagery from the same satellite system (e.g., 30 m multispectral imagery with 15 m pan-chromatic images from the Landsat satellite), or from different satellite systems [273]. Researchers have developed several methods to achieve spatiotemporal fusion across different application fields, such as crop and urban growth monitoring. According to [274], we can broadly classify these methods into three major groups: (1) reconstruction-based methods; (2) learning-based methods; and (3) unmixing-based methods. However, the authors of [272] argue that these three broad categories are not sufficient to cover all the spatiotemporal fusion methods. Therefore, they propose five categories based on the specific techniques used to link fine and coarse images: (1) unmixing-based methods; (2) weight-function-based methods; (3) Bayesian-based methods; (4) learning-based methods; and (5) hybrid methods.

Spectral unmixing-based methods approximate the fine pixel values by unmixing the coarse pixels based on the theory of linear spectral mixtures to extract the endmembers and abundances (i.e., the proportions at the subpixel level) [275]. High-resolution datasets provide a number of endmembers and abundances obtained from a, and the spectral signature of the endmembers is unmixed from the coarse-resolution dataset. The assumption behind linear spectral mixing theory is that the reflectance that corresponds to each coarse spatial-resolution pixel is a linear combination of the responses of each land-cover class that contributes to the mixture [276]. The four steps in spectral unmixing required to predict a fine-resolution image are as follows: (1) the classification of the image with high spatial resolution using unsupervised methods, such as k-means; (2) the computation of the endmember fractions of each coarse pixel; (3) the unmixing of the coarse pixels at the prediction date within a moving window; and (4) the assignment of the unmixed reflectance to fine pixels.

Learning-based methods take advantage of the intrinsic capabilities of machine learning to predict the unobserved finer-temporal-resolution images from coarse-spatial-resolution images. The most commonly employed machine-learning methods for spatiotemporal data fusion include regression trees [277], random forest [278], dictionary-pair learning [279], extreme learning [280], artificial neural networks [281] and convolutional neural networks [280].

Bayesian-based methods, as the name suggests, use the Bayesian probabilistic theory of estimation to combine images. Under this framework, the goal of data fusion is to achieve the desired fine image at the prediction date by maximizing its conditional prob-ability relative to the fine and coarse input images [282]. In other words, Bayesian-based data fusion aims to discover how to model two relationships (i.e., between the observed coarse image and fine image observed at the same date and the one observed at different dates) between the input images and predicted image.

Researchers use weighted-function-based methods, which are also called reconstruction-based methods [271], to estimate the synthetic spectral reflectance at the pixel level by means of the weighted sum of the similar neighboring pixels of the input image source. Researchers have developed many methods under the weighted-function-based approach, which, among others, include the following: the spatial and temporal adaptive reflectance fusion model (STARFM) [283]; enhanced STARFM semiphysical fusion approach [284]; spatiotemporal adaptive data fusion algorithm for temperature mapping (SADFAT) [255]; spatial temporal adaptive algorithm for mapping reflectance change (STAARCH) [285]; spatiotemporal integrated temperature fusion model (STITFM) [274]; spatiotemporal enhancement method for medium-resolution LAI (STEM-LAI) [286]; spatiotemporal vegetation index image fusion model (STVIFM) [287]; and database unmixing (DBUX) [288]. Among these methods, the most applied reconstruction-based image fusion method is the STARFM [271] due to its ability to accurately predict the surface reflectance on a daily basis at an effective resolution that is close to that of the Landsat enhanced thematic mapper Plus (ETM+) (i.e., 30 m). However, its performance largely depends on the characteristic patch size of the landscape and reduces to some extent when used on landscapes with extremely heterogeneous fine grains.

The integration of two or more of the four abovementioned categories constitutes the fifth category: hybrid methods, the purpose of which is to improve the spatiotemporal da-ta fusion performance through the utilization of the advantages associated with the individual methods. One of the recently developed hybrid methods is flexible spatiotemporal data fusion (FSDAF), which combines weighted-function-based methods and unmixing-based methods [289], which makes it effective even in highly heterogenous landscapes. Other hybrid methods in the literature include the spatial and temporal reflectance unmixing model (STRUM) [275], and the spatial–temporal remotely sensed images and land-cover maps fusion model (STIMFM) [290].

3.3.3. Spatiospectral Fusion

Similarly, as is the case for spatiotemporal fusion, different satellite sensors can be used to observe the surface of the Earth at different spatial resolutions over a given wave-length range. For instance, Planet microsatellites acquire multispectral images at a spatial resolution of 3.125 m, and a single single-band panchromatic (PAN) image has a spatial resolution of 0.73 m (SkySat-3 and SkySat-14), whereas the multispectral instrument onboard Sentinel-2B captures the Earth’s reflected radiance with a lower spatial resolution (four bands at 10 m, six bands at 20 m, and three bands at 60 m). Therefore, the aim of spatiospectral fusion is to combine fine-resolution and coarse-resolution images (e.g., 3.125 m SkySat-3 multispectral images and 0.73 m SkySat-3 PAN images) to derive fi-ne-spatial-resolution images for all the bands. The use of only a single PAN image as the fine-resolution image is referred to as “pan-sharpening” (the example given above for SkySat-3), whereas the use of multiple fine images is referred to as multiband image fusion (e.g., fusing the different resolution bands of Sentinel-2B).

Over time, researchers have developed many techniques to perform spatiospectral fusion, which they have broadly divided into five categories: (1) multiresolution analysis (MRA); (2) component substitution (CS); (3) geostatistical analysis; (4) subspace representation; (5) sparse representation [240]

CS [291] was developed in 1987 to merge multiresolution SPOT HRV and Landsat TM data, and it was designed to spectrally transform the multispectral data into a different feature space to separate the spatial and spectral information into different components. PCA [292] is one of the most applied methods for transformation. Other methods include Gram–Schmidt [293] and hue saturation [294], among others. In these approaches, the component that contains the spatial information is replaced with the panchromatic image for the transformation into the sharpened image.

After the CS approach, researchers continued to propose different techniques with intrinsic advantages, such as multiresolution analysis [295], the main advantage of which is the spectral consistency, such that if the combined image is degraded in the spatial domain, then the degraded image is spectrally consistent with the input coarse-spatial-resolution and fine-spectral-resolution image [240]. Researchers have also developed geostatistical analysis to preserve the spectral properties of the original coarse images so that, when the downscaled image is upscaled to the original coarse spatial resolution, the obtained result is identical to the original one [296]. Subspace representation is a technique that approaches the spatiospectral fusion problem through the spectral characteristic analysis of the scene of interest using a subspace that is spanned by a set of basis vectors, such as the principal component bases and spectral signatures of endmembers [297]. Recently, the authors of [298] proposed another approach, sparse representation, which is based on patch-wise sparse representation to exploit the various external fine-spatial-resolution multispectral images as the database.

Pedram et al. [240] present a detailed comparative review of these approaches, with a statistical analysis of the history and annual average number of citations for each category of the abovementioned spatiospectral techniques. According to their research, the most applied techniques in the last two decades have been component substitution and multiresolution analysis. In recent years, the remaining three approaches have achieved considerable applications due to their simplicity, outperforming the previous approaches.

Moreover, the use of hyperspectral imaging and data fusion techniques is becoming more and more important in almost all environmental and agricultural related aspects indifferent from field crop monitoring. For instance, there has been a lot more ongoing studies on monitoring the recovery of abandoned industrial areas through the recovery of land with woods or agricultural areas in some cases. A recent study by Giuseppe et al. [299] shows the potential to identify and characterize areas contaminated by asbestos in a mining site using remote sensing techniques coupled with data fusion based on the PRISMA satellite hyperspectral images.

4. Conclusions

According to our review, the advent of remote sensing has substantially fueled the success of precision agriculture in response to the rapidly growing global food demand amidst the effects of climate change. Spectral imaging has played a vital role in facilitating crop monitoring to aid the decision making in the implementation of spatially variable agronomic practices and/or inputs. Driven by cutting-edge data processing techniques, spectral imaging for field crop monitoring has become a prominent research area in the remote sensing community. Furthermore, the recent data fusion approaches have eliminated the need to compromise between the spatial and spectral resolutions. Due to recent technical advances, the remote sensing community now has access to both dense timeseries data and high-spatiospectral-resolution images, without the need to approximate the compromised components using fusion methods. Moreover, machine-learning and deep-learning approaches have substantially enhanced the processing and analysis of spectral information. However, in these approaches, it is assumed that there are sufficient computational resources, and that no data transmission cost is incurred for their optimal application; however, this is not always the case.