Robust Coffee Rust Detection Using UAV-Based Aerial RGB Imagery

Abstract

Timely detection of pests and diseases in crops is essential to mitigate severe damage and economic losses, especially in the context of climate change. This paper describes a method for detecting the presence of coffee leaf rust (CLR) using two databases: RoCoLe and a database obtained from an unmanned aerial vehicle (UAV) equipped with an RGB camera. The developed method follows a two-stage approach. In the first stage, images are processed using ImageJ software, while, in the second phase, Python is used to implement morphological filters and the Hough transform for rust identification. The algorithm’s performance is evaluated using the chi-square test, and its discriminatory capacity is assessed through the generation of a Receiver Operating Characteristic (ROC) curve. Additionally, Cohen’s kappa method is used to assess the agreement among observers, while Kendall’s rank correlation coefficient (KRCC) measures the correlation between the criteria of the observers and the classifications generated by the method. The results demonstrate that the developed method achieved an efficiency of 97% in detecting coffee rust in the RoCoLe dataset and over 93.5% in UAV images. These findings suggest that the developed method has the potential to be implemented in the future on a UAV for rust detection.

1. Introduction

Agriculture plays a vital role as the primary source of the global food supply. It faces significant challenges driven by the growing demand for food products, the increasing global population, conflicts, preservation of water resources, environmental protection, climate change, scarcity of cultivable land, declining agricultural workforce, and the pursuit of sustainability [1,2,3,4].

Monitoring coffee plantations is a key practice for successful crop management and is essential for early detection of diseases, enabling necessary measures to prevent loss or decrease in yield. Coffee leaf rust (CLR) (Hemileia vastatrix) [5] has spread worldwide and, together with anthracnose and coffee berry borer, it causes significant annual losses in Central America and around the world [6,7]. Hence, there is a need to find methods that can identify pests in their early stages.

In El Salvador, as well as in many other regions worldwide, coffee production is predominantly concentrated in hard-to-reach mountainous areas. Farmers invest significant resources each year in hiring personnel to cover extensive areas and detect potential pest infestations. However, this practice leads to substantial increases in production costs. Sometimes, pests go unnoticed until they have already spread across multiple hectares, necessitating the hiring of more personnel and the acquisition of larger quantities of chemical products to combat them. Consequently, coffee production in these areas is adversely affected, resulting in lower yield levels. Despite these challenges, the demand for coffee continues to increase each year [8,9].

Over the past decade, precision agriculture (PA) has become increasingly significant and is widely regarded as the future of agriculture [10]. PA utilizes various technologies, such as remote sensing systems, sensors, the Internet of Things (IoT), and unmanned aerial vehicles (UAVs). The integration of artificial intelligence (AI) with these technologies has led to a revolution in the field of PA [11,12,13,14].

Remote sensing systems have been used to detect CLR. For example, Cortez et al. [15] identified CLR using multispectral orbital sensors and vegetation indices (VIs). The authors concluded that VIs based on band ratios in the near-infrared and red-edge regions are more sensitive to spectral changes in vegetation due to variations in CLR occurrence. Additionally, Pires et al. [16] used Landsat-8/OLI-TIRS and Landsat-7/ETM+ images to evaluate CLR in different irrigation management systems. They found that, during periods of higher CLR presence, there was a reduction in the average near-infrared (NIR) and green reflectance and an increase in the reflectance in the red, SWIR-1, and SWIR-2 regions.

Limited research has been conducted on the utilization of UAVs for CLR detection. Soares et al. [17] presented a method for early detection in a controlled environment, where 160 seedlings were inoculated with H. vastatrix and compared against 160 uninfected control seedlings. A UAV equipped with a multispectral camera was employed for data collection. A support vector machines (SVM) algorithm was developed. The study achieved a Kappa agreement index of 0.6, indicating moderate agreement, and demonstrated an 80% accuracy in disease detection. The RGB sensor provided promising results when the disease became visually evident.

On the other hand, Velásquez et al. [18] implemented a method that combined remote sensing (using UAVs equipped with multispectral cameras), wireless sensor networks (employing a multisensor approach), and deep learning (DL) techniques. The developed algorithm was evaluated using a constructed testbed prototype, which included a scaled coffee crop. It was observed that there were no significant differences between the inspection conducted by experts and the results achieved by the developed method.

Marin et al. [19] developed a framework based on decision tree models for detecting the severity of coffee leaf rust solely using vegetation indices extracted from multispectral images captured by a remotely piloted aircraft (RPA). The authors demonstrated that the Logistic Model Tree (LMT) method was the most effective in detecting CLR disease.

Other research has been conducted using databases obtained, in most cases, with smartphones or RGB cameras. Chavarro et al. [20] presented a study on the impact of five types of hyperparameters on the performance of coffee leaf rust classification models. They used images from the RoCoLe, Bracol, D&P, Digipathos, and Licole databases. In the study, it was found that the models obtained using ResNet50 had an accuracy of less than 70%, compared to those achieved with DenseNet201, which had an accuracy exceeding 90% (94.60%).

Faisal et al. [21] presented several hybrid models to extract features from input images, using a combination of Swin Transformer, MobileNetV3, and variational autoencoder (VAE). The fusion of hybrid features from Swin Transformer and MobileNetV3 resulted in the detection of CLR with an accuracy of 84.29%. The developed algorithm was evaluated using the RoCoLe database.

Numerous studies conducted for CLR detection have been limited to controlled environments or databases obtained from smartphones. Additionally, many research efforts using remote sensing systems and UAVs may not be directly applicable in the context of El Salvador and other countries due to the diverse coffee cultivation practices employed. In coffee cultivation, various techniques are utilized, such as shaded monoculture (the most common in El Salvador), rustic polyculture, traditional polyculture, and commercial polyculture [22]. In all these cases, the coffee plants are situated under shade trees, making it challenging to observe them from the air unless the UAV flies at a low altitude (below the trees). These methods can be effectively applied in coffee monoculture scenarios [22]. Hence, there is a need to implement methods in real environments, using UAVs, that allow for early identification of CLR in the presence of shade trees in the crops, which constitutes the objective of the current work.

The document is organized as follows. Section 2 provides an overview of coffee leaf rust, the RoCoLe dataset, the UAV employed in this study, and a description of the plantation where the images were captured. Next, the developed method is described. Section 3 presents the obtained results. Section 4 contains the discussion of the results, emphasizing the effectiveness and importance of the developed method. Finally, Section 5 presents the conclusion, limitations, and suggestions for future research.

2. Materials and Methods

2.1. Characterization of Coffee Leaf Rust and the RoCoLe Database

C. arabica (Arabica coffee) and C. canephora (Robusta coffee) are the coffee varieties most susceptible to CLR and they are also the most commercially popular coffee species in the world [23].

CLR affects the leaves of coffee plants. The infection process can occur rapidly, taking less than 6 h. It consists of two main subprocesses: the germination of urediniospores and the penetration through the stomata on the underside (abaxial surface) of the leaf. Hemileia vastatrix requires temperatures ranging from 10 to 35 °C and moisture for its development [6].

In Figure 1a, healthy coffee leaves are observed. However, once infected, small lesions of light-yellow color appear on the undersides of the leaves, as shown in Figure 1b. These lesions grow and start producing orange spores that cover almost the entire surface of the lesion (Figure 1c). As the lesions continue to expand, they merge. The center of the spot eventually dries up and turns brown, while the edges of the lesions continue to spread and produce new spores (Figure 1d,e). Eventually, the affected leaves fall off, leaving the branches without foliage [22,24].

The RoCoLe dataset is open and publicly accessible, consisting of 1560 images of Robusta coffee leaves depicting various conditions (healthy and unhealthy), including the presence of diseases such as rust and red spider mite (Figure 1). The images were captured using a 5 MP smartphone camera, at a working distance of 200 to 300 mm without zoom [25].

RoCoLe provides six categories: healthy, rust level I, rust level II, rust level III, rust level IV, and presence of red spider mite. For each class, there are 791, 344, 166, 62, 30, and 167 images, respectively. This database was selected to be used in this work because it is widely employed in the development of algorithms for CLR detection. This will enable a comparison of the performance of the developed method with others created for the same purpose [21].

2.2. Study Area in El Salvador

The study area is situated in the department of Ahuachapán, El Salvador, with co-ordinates 13°53′47.2344″ N, −89°52′14.217″ W (Figure 2). Ahuachapán exhibits a tropical climate, characterized by significantly higher rainfall in summers than in winters. It falls under the Aw category according to the Köppen–Geiger classification. The average temperature is 21.9 °C and the annual average precipitation reaches 1544 mm. Throughout the year, there is a temperature variation of 2.1 °C. The average elevation in the study area is approximately 900 m above sea level.

The coffee crop under study is Arabica (Coffea arabica L.), specifically the Bourbon variety. It occupies an area of 75 hectares. Cultural practices are carried out annually, which include removing damaged branches. The trees’ canopy, responsible for providing shade to the crop, undergoes pruning every 2 years, with the objective of maintaining a shade (40%)–sunlight (60%) ratio. These trees are endemic to the region and belong to different species, with an average height of 18 m (Figure 3).

2.3. UAV and Camera System

Image collection was conducted using a professional DJI FPV drone (DJI, Shenzhen, China) (Figure 4). This UAV is equipped with a digital RGB (Red-R, Green-G, Blue-B) camera with a maximum image resolution of 3840 × 2160 pixels, a 1/2.3 CMOS sensor with 12 million effective pixels, a field of view (FOV) of 150 degrees, and an internal GPS receiver. The UAV control system consists of a ground control station connected to DJI FPV Goggles for visualizing the perspective from the UAV. This control system was used for all missions.

All missions involved a two-person team, comprising a pilot responsible for operating the ground control station and managing UAV takeoff and landing and an observer tasked with identifying and notifying the pilot of any potential obstacles or hazards during the flight.

The UAV flew at a height of 2.8 m above the ground (Figure 3) with a speed of 3 m/s, capturing photos with a 10% overlap in both the frontal and lateral directions. Images were captured between 8:00 AM and 10:00 AM. The images were taken at this time upon the suggestion of the farmers, as, earlier in the day, in a mountainous area, sunlight is scarce. Moreover, at this time, there is typically no fog. After midday, during the rainy season, clouds begin to form and rain becomes frequent. The flight path was parallel to the slopes to maintain the altitude and aligned with the direction of the crop planting (Figure 5). A total of 96 photos were taken.

2.4. Experimental Design

A method (Figure 6) was developed, comprising two main steps: applying a filter to the images using ImageJ software, followed by implementing a rust detection algorithm in Python.

2.4.1. Image Processing Using ImageJ Software

ImageJ software (version 1.54b, open-source, Bethesda, MD, USA) was used for processing the images. ImageJ, based on Java, is a highly versatile program that runs on various operating systems, including Microsoft Windows, macOS, and Linux. With its plugin architecture and integrated development environment, ImageJ has established itself as a renowned platform in the field of image processing, widely used in disciplines such as medical, biological, and agricultural sciences [26,27,28].

ImageJ incorporates a variance filter, which is employed to process coffee images. This filter is specifically designed to reduce noise and enhance image smoothness while preserving essential details. The implementation of the variance filter in ImageJ follows the adaptive approach introduced by Kuan et al. [29], where the smoothing parameters are automatically adjusted based on the local variance of the image. This approach allows the filter to adapt to the specific characteristics of each image region. By using a variable neighborhood window, the filter automatically adjusts the smoothing parameters based on the local variance of the image. The calculation of the local variance is performed [29]:

$${\sigma }^2\left(x,y\right)=\frac{1}{N}{\sum }_{i=1}^N{[f\left(x+i,y+j\right)-\mu (x,y\left)\right]}^2$$

(1)

where ${\sigma }^2\left(x,y\right)$ is the variance at point (x, y) of the image, N is the size of the neighborhood window (was selected N = 5), $f\left(x+i,y+j\right)$ represents the intensity values of the neighboring pixels, and $\mu (x,y)$ is the mean of the intensity values in the neighborhood.

2.4.2. Image Processing Using Python

After applying the variance filter in the ImageJ software, the images are processed using Python 3.10.11. In Python, the images are first checked to determine whether they belong to the RoCoLe dataset or were captured by the UAV. This is accomplished by examining the image dimensions. RoCoLe images have dimensions of 1152 × 2048 pixels, while images captured by the UAV have dimensions of 3840 × 2160 pixels.

If the images were captured by a UAV, a segmentation process is applied (Figure 7). The top 40% of the image is cropped, along with 20% from each side. This segmentation of the original image is performed to achieve a more precise detection of coffee leaf rust. The images are captured by the drone, flying over the planted furrows, with a field of view (FOV) of 150 degrees and at a height of 2.8 m. This causes the upper part of the image to contain information about plants that are farther away, leading to a tendency to lose crop details and, thus, the method may not detect CLR accurately. At the same time, on the sides (lower right and left) of the image, there may appear details of the UAV and sometimes the furrows. Considering the average size of the plants in the furrows, the sides were cropped. It is important to note that this does not imply that parts of the crop remain unanalyzed since, in consecutive images, the previously cropped upper part is analyzed. This process is repeated along all the furrows.

Next, the images undergo processing in the Python-based rust detection algorithm (Figure 6). This algorithm’s objective is to create a mask that identifies the presence of red and orange colors in the image obtained from ImageJ. This software automatically marks the detected variations on the processed image. It was observed that, in the case of coffee leaf rust, it consistently followed a color pattern (red and orange) that formed a circular structure. The “cv2.findContours” function from the Python OpenCV library was employed, which relies on the topological information and pixel connectivity within the image. The algorithm searches for initial contour points and proceeds to track neighboring pixels, forming a point sequence that represents the contour. Finally, a binary image is generated to visualize the identified regions.

A morphological analysis is performed on the binary image [30]. For this purpose, an elliptical kernel with a size of (5, 5) is applied. A combination of morphological opening and closing (in that sequence) is used to smooth the binary image, remove noise and small objects, and close gaps in larger objects. Subsequently, to detect circles in the binary image, the circular Hough transform [31,32] is employed, successfully detecting coffee rust in the crop.

A general-purpose computer with an Intel Core i7 processor running at 2.90 GHz and 8 GB of RAM was used for the method implementation. The average time required to obtain the result was 2.4 s per image.

2.5. Statistic Evaluation

The Chi-square test was used to assess the performance of the algorithm in detecting rust. On the other hand, the images obtained by the UAVs, as well as the RoCoLe dataset, were evaluated by two experts in the field. They evaluated them completely blinded to the image data. To determine whether there is agreement or concordance between the observers’ classifications and the software results, the Kendall’s rank-order correlation coefficient (KRCC) was used.

The Cohen’s kappa coefficient was employed to assess observer agreement. To interpret kappa values, predefined ranges are utilized. A kappa value below 0 indicates agreement worse than chance, while a range of 0.01 to 0.20 is considered slight agreement. A range of 0.21 to 0.40 represents fair agreement, while a range of 0.41 to 0.60 is classified as moderate agreement. A range of 0.61 to 0.80 indicates substantial agreement and a range of 0.81 to 1 is considered almost perfect agreement.

Furthermore, to evaluate the model’s discriminatory ability based on the observer criteria, a Receiver Operating Characteristic (ROC) curve is generated.

The statistical analyses were conducted using the SPSS statistical software (version 22.0; IBM, Chicago, IL, USA). For all statistical analyses, p-values less than 0.05 were considered statistically significant and a confidence level of 95% was selected.

3. Results

This section describes the process and results obtained from the experiment. Firstly, the results obtained using RoCoLe dataset are presented, followed by those achieved using a DJI UAV with an RGB camera.

3.1. Results Obtained Using RoCoLe Dataset

In Figure 8, the results achieved using the developed method are presented. It can be observed that, after processing the image with ImageJ, the variance filter successfully detects the yellow rust spots, enclosing them within red and/or orange circles or semicircles. These identified regions are further processed in Python. Additionally, it is notable that the leaves generally exhibit dark or light colors.

When the evaluation of the developed method began, it was observed that it detected rust in images that had been classified as healthy in the RoCoLe dataset. This dataset focuses on presenting either healthy or diseased leaves based on the classification created by its authors. However, there are instances where diseased leaves are present around the targeted leaf. For this reason, experts were consulted, as the objective of this study is to detect rust in the entire image, rather than in a specific area. The experts confirmed that the developed method is sensitive to the presence of rust across the entire image (Figure 9) and, thus, the rust-detected areas validated by the experts were included in the category of leaves with rust level I, resulting in the distribution shown in

Table 1. Distribution performed by experts for rust classification and efficiency of the developed method.

Variable	Total	Efficiency
	n = 1560	(%)
Healthy leaf, n (%)	695 (44.51)	97.98%
Level I leaf rust, n (%)	487 (31.22)	98.15%
Level II leaf rust, n (%)	166 (10.64)	100%
Level III leaf rust, n (%)	62 (3.97)	100%
Level IV leaf rust, n (%)	30 (1.93)	100%
Red spider mite, n (%)	120 (7.69)	--

.

On the other hand, Table 1 shows the efficiency of the proposed method in detecting rust. The objective was to detect the presence of rust and not to estimate the severity of the infestation in the classification provided by the authors of the dataset. The results indicate that the method, under these conditions, is efficient in detecting the CLR. The designed method does not detect the red spider mite. It is important to clarify that the method does not aim to detect the red spider mite.

Table 2. Statistical analysis of the method’s performance in rust detection using the RoCoLe dataset.

Variables	Chi-Square Test		KRCC
	Value	p	Value	p
Rust Presence	1457.73	<0.001	0.967	<0.001

presents the statistical analysis conducted. It shows that there were statistically significant differences (p < 0.001), indicating that rust was efficiently detected. When applying Cohen’s kappa, a κ value of 0.869 with p < 0.001 was obtained throughout the study.

3.2. Results Obtained with the UAV’s RGB Camera

Using the DJI FPV UAV, images of Bourbon coffee plantations were obtained. In this work, the designed method focuses on the detection of CLR, not on its classification. Figure 10 shows the results achieved by applying the developed method to the acquired images. The segmentation of the original image into a smaller one enhanced the algorithm’s efficiency in detecting the presence or absence of the disease. The developed method achieved an efficiency of 93.75%. By applying the Cohen’s kappa method, a κ = 0.954 with p < 0.001 was obtained in the entire study.

In

Table 3. Statistical analysis of the method’s performance in rust detection using UAV-acquired images.

Variables	Total	Chi-Square Test		KRCC
	n = 96	Value	p	Value	p
Rust presence (%)	41 (42.71)	73.308	<0.001	0.874	<0.001

, the statistical results obtained using the Chi-square test and KRCC are shown. In both cases, significant differences were found with a p-value of <0.001.

4. Discussion

In recent years, several databases have been made available to the scientific community for evaluating methods developed for CLR detection. Among these databases are RoCoLe, BRACOL [33], D&P [34], Digipathos [35], and Licole [36]. Of all these databases, RoCoLe has been the most widely used. However, it is worth noting that many works developed for CLR detection have been limited to evaluating their results using databases obtained primarily through smartphones or in controlled environments. It is important to highlight that this trend should change in the future.

Table 4. Comparison of the performance of the developed method with previously published algorithms using the RoCoLe dataset.

Models	Accuracy (%)
ResNet50 [37]	67.40
VGG16 [37]	79.83
MobileNetV3 + Swin-Transformer [21]	84.29
DenseNet201 [20]	94.60
Developed method	97.00

presents a comparison of the performance between the developed method and algorithms designed with AI using the RoCoLe dataset. The results indicate that the method presented in this work exhibits the best performance among all evaluated methods.

The ROC curves presented in Figure 11 illustrate the performance evaluation of the developed method. For the RoCoLe dataset (Figure 11a), the method showed excellent predictive performance, with an area under the curve (AUC) of 0.983, indicating a high discrimination capacity between positive and negative classes. The results were statistically significant (p < 0.001), with a 95% asymptotic confidence interval ranging from 0.976 to 0.991.

In the case of images obtained by the UAV (Figure 11b), the method demonstrated a good predictive performance, with an AUC of 0.935. The result was also found to be statistically significant (p < 0.001) and the 95% asymptotic confidence interval for the AUC was between 0.876 and 0.993.

In Figure 12, an example image is presented where the developed method exhibits a failure in detecting rust in the RoCoLe dataset. The algorithm identifies a section of the leaf as rust, although it is barely discernible to the human eye. Expert analysis could not definitively confirm the presence of the disease in this particular case. However, this result holds promise as it suggests the potential for detecting rust at an early stage, which is of great significance in disease management strategies.

The developed method exhibited similar failures in detecting rust in the images acquired using UAV, comparable to those observed in the RoCoLe dataset. However, it still demonstrated the ability to distinguish coffee leaf rust in the field, as evident in Figure 8, Figure 9, Figure 10 and Figure 12. In ImageJ, the variance filter mainly identifies weeds, furrows, and tree trunks as white, while only structures with different colors are detected in the crop foliage. Yellow leaves that are drying out or damaged, such as those affected by the red spider mite, are not identified by the developed method as possible CLR (Figure 8). Moreover, it was confirmed that the shadows of the trees do not significantly influence the results of the images taken by the UAV (Figure 10).

UAVs have a wide range of applications in the agricultural field, with their main use being in crop monitoring through aerial imaging. They have become a valuable tool in modern agriculture, assisting farmers in optimizing their operations and achieving better outcomes [11,38]. In recent decades, companies such as DJI [39], PrecisionHawk [40], AEROBOTICS [41], and AGRIVI [42] have designed UAVs with cameras for use in precision agriculture. These companies primarily focus on offering services for field monitoring, comprehensive analysis of weather conditions, detailed tracking of crop progress, evaluation of plant stress, and notifications of risks related to adverse weather events or pest outbreaks. The goal is to provide farmers with effective agronomic advice based on precise and up-to-date information.

Despite all these advancements, there is still a lack of available tools in the market that can accurately detect and identify the presence of pests in various types of crops. Moreover, studies indicate that the development of methods is still in its nascent stages [43,44]. Furthermore, the incidence of pests in these crops increases each year, making it vital to develop algorithms utilizing drones for pest detection in order to effectively combat them and prevent crop losses [44].

A limitation of this study is the use of a small number of images acquired by the UAV to evaluate the developed method. Future experiments are planned to demonstrate its effectiveness on a larger scale. These tests will be carried out under the same conditions as presented in this study, since coffee is cultivated in El Salvador and many other places worldwide using shade trees.

It is important to acknowledge that conducting these experiments and their subsequent practical implementation requires a UAV pilot with experience, as the flights need to be performed below the trees. Additionally, the trees must be well pruned to prevent accidents with the UAV. Ensuring the safety of the flight operations is crucial in such agricultural applications.

Furthermore, there are plans for additional studies to enhance the developed method. An algorithm will be developed that incorporates statistical techniques to mitigate potential false positives and false negatives. This enhancement aims to achieve higher precision in disease detection and ensure more reliable results when evaluating UAV-acquired images. To broaden the scope of the research, a larger dataset of coffee crops affected by rust disease will be collected, encompassing various coffee varieties. This expanded dataset will enable more comprehensive comparisons and improve the robustness of the method. Moreover, the exploration of multispectral imagery and the application of VIs will be pursued. These technologies have the potential to significantly enhance the disease identification capability, offering new insights and opportunities for accurate and early detection of CLR.

Additionally, the feasibility of utilizing advanced AI methods, particularly those that are computationally efficient, will be explored to further enhance the obtained results. The goal is to investigate the potential of AI techniques to improve the precision and accuracy of the developed method even further. Notably, the developed method has already demonstrated better performance compared to more recent methods that incorporate AI. The enhanced capabilities offered by advanced AI would provide farmers with critical insights, including the exact geographic co-ordinates of the disease’s location and a detailed statistical analysis of its presence in the fields. Equipped with this comprehensive information, farmers can take timely and accurate measures to effectively control the disease and minimize its impact on their crops. This would lead to better disease management and improved crop yield.

5. Conclusions

In this study, we evaluated the effectiveness of a developed method using ImageJ and Python software for coffee leaf rust detection. The results obtained were promising, showing a detection efficacy of 97% for coffee rust in the RoCoLe dataset and over 93.5% in UAV images. Additionally, we compared the efficiency obtained using the RoCoLe dataset with four AI algorithms, leading to the conclusion that the method presented in this study outperforms the others.

The developed method holds great potential for future implementation on UAVs for the detection of rust in coffee plantations. By integrating this method into UAV technology, it offers a promising solution for efficient and widespread monitoring of rust infestations. This advancement can enable proactive measures and timely interventions to combat the spread of rust, ultimately aiding in the protection and preservation of coffee crops.