Identification of Pest Attack on Corn Crops Using Machine Learning Techniques ^†

Abstract

The agriculture sector plays a very important role in the increasing population, year by year, to fulfill requirements and contributes significantly to the economy of the country. One of the main challenges in agriculture is the prevention and early detection of pest attack on crops. Farmers spend a significant amount of time and money in detecting pests and diseases, often by looking at plant leaves and analyzing the presence of diseases and pests. Late detection of pest attacks and improper use of pesticide application can cause damage to plants and compromise food quality. This problem can be solved through artificial intelligence, machine learning, and accurate image classification systems. In recent years, machine learning has made improvements in image recognition and classification. Hence, in this research article, we used convolutional neural network (CNN)-based models, such as the Cov2D library and VGG-16, to identify pest attacks. Our experiments involved a personal dataset consisting of 7000 images of pest-attacked leaf samples of different positions on maize plants, categorized into two classes. The Google Colab environment was used for experimentation and implementation, specially designed for cloud computing and machine learning.

1. Introduction

In the last few years, artificial intelligence and machine learning growth has been fastest in the agriculture sector, playing a crucial role in increasing productivity and contributing to the economies of countries worldwide. Currently, machine learning is not only used by IT professionals but is also being utilized in various industries by different specialists [1]. For example, it is used for tasks such as face detection [2,3], in online marketing such as for the prediction of stock [4], for breast cancer detection in the medical field [5], as well as for lung disease prediction [6]. Also, in the field of the agriculture sector, it is used for example in image processing and detection, classification of disease, pest, wheat spike, crop yield estimation [7], crop identification [8], counting numbers of plants, and identifying insect attacks, and for solving crop related problems. The application makes it easier to detect, recognize and classify pests [9], insects, and diseases [10,11].

Maize, a staple food after wheat and rice, is infected every year by the fall armyworm (Spodoptera frugiperda), a highly polyphagous pest belonging to the Noctuidae family. Fall armyworms reportedly attack over 350 host plants across 76 families [12]. At the global scale, crop damage by fall armyworms across hosts includes maize losses (19.5–41.1%), (Zea mays L.) [13], sorghum, (Sorghum bicolor (L.) Moench) [14], rice losses (24.6–40.9%) [14], (Oryza sativa), soybean losses (11.0–32.4%), (Glycine max (L.) Merr) [15], cotton, (Gossypium hirsutum L.), barley, (Hordeum vulgare L.) [16] and wheat losses (10.1–28.1%), (Triticum aestivum L.) [17], potato losses (8.1–21.0%), (Solanum tuberosum L.) [18] with graminaceous plants being preferred [12,19].

The dataset of maize fall armyworm pests is not available for training, testing, and validation of the models. The results from this research will be used later for the variable rate spraying system and real-time detection for the management of maize fields.

Our previous research [20] performed detection, based on deep learning for wheat crop weeds, which exactly performs the detection and classification of one type of weed in wheat crops. The comparison with other advanced techniques in our models shows the following benefits. (1) It examines the possibility of a plant being impacted by different weeds at the same time in the same sample. (2) It utilized images with different device cameras with several resolutions. (3) It can easily deal with different lighting conditions, sizes of weeds, and backgrounds of weeds. (4) It can provide easy use of a variable rate spraying system or real-time spot detection in the field without any use of expensive and high technology.

The three main objectives of the study were as follows.

• A real pest dataset was required to be developed containing images of maize crop leaves collected from the Shandong University technology research form, located in Zibo city of Shandong Province, China.
• This dataset needed to contain images of one type of pest collected in different weather conditions and with different time intervals.
• The Cov2D and VGG16 state-of-the-art models for maize leaf pest detection, classification and identification were to be utilized.

Review Literature

Table 1. Deep learning and machine learning used in various studies.

S.No	Year	Journal Name	Model	Purpose	Pest/Disease	Classes	Accuracy %	Crop Name	References
1	2017	IEEE	VGG16 + SVM	Classification	Pest/Disease	10		Tomato	[21]
2	2022	Computers & Electronics in Agriculture	VGG16, VGG19, ResNet50, InceptionV3	Classification	Pest/Disease/NPK deficiency	7		Rice	[22]
3	2020	IEEE	VGG16, VGG19, ResNet50, ResNet50v2, ResNet101v2,	Classification	Pest/Disease	4		Rice	[23]
4	2022	Agriculture	ANN, CNN, VGG-16, MobileNetV2	Classification	Pest/Disease/Nutrient deficiency	2		Zingiberaceae	[24]
5	2020	Biosystems engineering	VGG16, InceptionV3, MobileNetv2, NasNet Mobile, SqueenzeNet v1.1, Simple CNN	Identification/Recognition	Pest/Disease	9		Rice	[25]
6	2023	Computers in Biology and Medicine	AlexNet, VGG-16, GoogleNet, ResNet-50	Detection	Pest	3		Citrus fruit	[26]
7	2019	Turkish Journal of Electrical Engineering and Computer Sciences	AlexNet, VGG-16, VGG-19, SqueezeNet, GoogleNet, Inceptionv3, InceptionResNetv2, ResNet50, ResNet101	Detection	Pest/Disease	8		Not given	[27]
8	2020	Computers & Electronics in Agriculture	Inception-v3, Resnet-50, VGG-16, VGG-19, Xception	Detection and classification	Pests	13	93.82, 91.87, 91.80, 91.33, 90.52	soybean	[28]
10	2020	Computers & Electronics in Agriculture	VGG-16, VGG-19, ResNet50, ResNet152, GoogLeNet,	Recgnition	Pest	10	98.91	Not given	[29]
11	2023	Procedia Computer Science	DenseNet201, Hyperparameter Search 2D layer, Mobilenet, VGG16, InceptionV3	Detection	Weeds, Pest, and Disease	9	87.85, 91.85, 78.71, 99.62, 71.07	Not given	[30]
12	2021	bioRxiv	CNN, VGG16, INCEPTION V3, XCEPTION	Identification	Disease/Insect	10	24.28, 71.74, 77.19, 77.90, 82.83, 82.11, 80.33, 82.89	Tomato	[31]

presents a literature review discussing the various studies on different model of detection using deep learning and machine learning in recent years.

2. Materials and Methods

The method for detection consists of two parts: an agricultural pest dataset from the field and the construction of a deep convolutional neural network (DCNN) for detection. Pest dataset collection involves capturing pest images and labeling and separating them accordingly. The pest detection model is composed of four key components: multicategory classification, pest identification, a pest feature extraction network, and a region proposal network for pest objects.

2.1. Pest Image Acquisition

A total of 5500 image datasets were used for detection. A total of 2750 images were consistently used from different points collected in Shandong, China. Images of maize crops and pest-attack leaves at different growth stages were taken under different weather conditions, such as daylight, evening, and with cloud. In this process, we ensured that the pest dataset was of sufficient quantity and accuracy to facilitate data processing and analysis at a later stage. Pest dataset images with pixel resolution (416 × 416) were taken through a Logitech C920 Pro HD webcam with a resolution of 1080 × 2400 (FHD+) (Full HD 1080 p/30 fps HD 720 p/30 fps) pixels. Some pictures were also taken with a hand and other human parts in the background. The original pest dataset pictures of different locations from one research area are available in

Table 2. The unhealthy images and healthy images of the maize crop dataset.

Network Models	Pest-Attacked Pictures	Healthy Pictures
VGG-16	2750	2000
Cov2D	2750	2000

and Figure 1. The pictures of healthy leaves were captured on a pest-affected farm.

2.2. Data Labeling

Data labeling was carried out by professional labeling techniques using IrfanView 4.66 software. Pest location coordinates and both classes of pest datasets were saved as XML files in YOLO format. The number of labeled samples corresponded to the number of bounding boxes annotated in each image. Additionally, each image contained multiple labels, depending on the number of pests present in the crop [20]

2.3. Image Preprocessing

The dataset was resized to 416 × 416 pixels for the shortened duration of training in the training stage, and IrfanView was used to resize the images. Furthermore, the training efficiency of the deep neural network model was improved. The labeling annotation tools were used manually to annotate rectangular boxes to apply to all pest-attacked leaves. A total of 5500 images were labeled for the training of all deep learning models and after annotating the images were divided for the training dataset. The created dataset of pest-attack maize leaves was divided into training and testing sets, and the pest-attack image samples in the training and test were 5500 and 1000, respectively.

2.4. Image Detection

Detecting the pest type and species is important for improving crop yield and protecting the crop from pest attacks, so we used different models such as Cov2D and VGG16 under the TensorFlow framework to obtain better accuracy of results.

2.5. Data Splitting

To evaluate the model’s ability to be successful and produce the intended result of our model, we followed a standard process of dividing the images into a training set, a validation set, and a testing set. These datasets were divided into the proportions of 70%, 20%, and 10%. This was conducted to optimize the detection model, to leverage more data during training, for the validation set in order to build and train the val dataset, and to improve overall performance of the detection model. Thus, the dataset division played a vital role in achieving better and more accurate overall results of both models in detecting fall armyworm pests.

2.6. Hyperparameters of the Models

Table 3. Hyper parameters of VGG-16 and Cov2D models.

Hyper Parameters	Values
	Cov2D	VGG-16
Activation	Sigmoid	Sigmoid
Dropout	0.4	0.8
Optimizer	Adam	Adam
Learning rate	0.001	0.001
Loss	Binary_crossentropy	Binary_crossentropy
Metrics	Accuracy	Accuracy
Epochs	10	10

shows hyper parameters of our models.

2.7. Network Architecture Model

In this study, we used a pre-trained model based on accuracy for maize crop pest-attack identification. The details of the model architecture are listed in

Table 4. VGG-16 and Cov2D model pretrained network architecture.

Network Model	VGG-16	Cov2D
Total layers	16	64
2. Max pool layers	5	5
3. Dense layers	3	1
4. Drop-out layers	0.8	0.4
5. Flatten layers	1	1
6. Filter size	3 × 3	3 × 3
7. Stride	2 × 2	2 × 2
Trainable-parameters	25,089	790,337

. Each model has the same filter size, but the other hyper parameters are different and play a very important role in model accuracy. The filter size plays a vital role in extracting specific features from the feature maps, and the feature maps depend on the specific values of the filters. In this research study we used an actual pretrained model network with the actual combination of convolution layers and actual filter size.

2.8. Vgg-16 and Cov2D Tuning Details

In this study, we used two CNN models to identify pest attacks on the maize leaves. Each input size of the image for networks is 416 × 416, and in the first two layers, it has 64 channels and a filter size of 3 × 3 and 2 strides. 256 channels with a 3 × 3 filter in the next two layers of VGG-16, followed by max pooling with two strides. There were two convolution layers with 256 channels three 3 × 3 layers after the pooling layer. Two convolution layers and two sets of three convolution layers with a pooling layer, along with 3 × 3 filters. Details are shown in Figure 2 for both the models’ architecture.

2.9. Evaluation Metrics

To evaluate the model performance, we utilized the average precision for each division and the mean average precision. The precision, recall and F1 score were calculated by the following equations:

$$Precision=\frac{TP}{TP+FP} n$$

$$Recall=\frac{TP}{TP+FN}$$

$$F1=2Precision\times \left(\frac{Recall}{Precision+Recall}\right)$$

TP represents the number of true positives, FP denotes the number of false positives, and the FN value corresponds to the number of false negatives.

3. Results and Discussion

3.1. Experimental/Hardware Setup

For this study, we utilized an Intel(R) Core 3(TM) i5-10210U CPU @ 1.60 GHz 2.11 GHz CPU with 8 GB RAM for training and validating our model. Furthermore, for training the VGG-16 and Cov2D models, the AMD Riyzen Threadripper 3970X 32-Core Processor 3.69 GHz memory was used. The development environment chosen was PyCharm, and the programming language was python 3.8. The main framework for our model was TensorFlow, along with the advanced version of OpenCV. Simultaneously, we used a deep learning framework.

3.2. Evaluation of Results

3.2.1. VGG16 Model Performance

This part of the study employed a state-of-the-art deep learning model using VGG16 and Cov2D models for the detection of plant pests specifically targeting the fall armyworm. The fall armyworm dataset is not available publicly, and we used it to further train the two hybrid models and transfer learning systems. The primary dataset used in this experiment was the fall armyworm dataset, and this dataset has not been used in any other models.

To ensure proper evaluation, we divided the fall armyworm dataset into three parts: training, testing, and validation samples. Specifically, 80% of the dataset was used for training, 0.1% for validation, and 20% for testing the pretrained VGG-16 and Cov2D models. Both models were run for 10 epochs, and it was found that in our pretrained model after the 1st epoch, the model accuracy was increased, and the loss was decreased. The graph in Figure 3 and the corresponding accuracy results are shown in

Table 5. Comparative analysis performance of the VGG-16 model.

Network Model	Training Accuracy (%)	Training Loss (%)	Test Accuracy (%)	Test Loss (%)
VGG-16	99.9	0.0240	99.9	0.0526

. Furthermore, the results for test accuracy, test loss, train accuracy, validation accuracy, and training loss accuracy are also provided in Table 5. It is evident from these results that the pretrained models performed effectively in detecting plant pests.

3.2.2. Cov2D Model Performance

Using the same dataset for evaluating the Cov2D model, we followed the same division of the fall armyworm dataset as we used for the VGG16 model above, for better model performance. This division was carried out to ensure fair evaluation and comparison between the two models. The confusion matrix in

Table 6. Comparative analysis performance of the Cov2D-based CNN model.

Network Model	Training Accuracy (%)	Training Loss (%)	Test Accuracy (%)	Test Loss (%)
Cov2D	99.9	0.0627	98.5	0.3832

shows the performance of the Cov2D model on the test set, with each row and column index representing a type of pest, specifically the fall armyworm. The obtained results suggest that the Cov2D model correctly detected the fall armyworm pest with impressive average training and testing accuracies of 99.9% and 98.5%, respectively. Additionally, the training loss was 0.062%, while the testing loss of 0.383% was slightly higher, as shown in Table 6.

3.3. Comparison with Other Work

In this section, various models are compared with the state-of-the-art convolutional neural network CNN model introduced in [32,33]. The authors used an online dataset comprising 70% of the images for training and the remaining 30% for testing. [34,35] for the different models listed in

Table 7. Different CNN model metrics.

Dataset Used	Pre-Trained Models	Multi-Classes	Accuracy %	References
Plant-Village	AlexNet	7	98.8	[32]
=do=	ResNet50	6	97.1	[33]
=do=	Inspection-V4	38	97.59	[34]
Captured from field	YoloV3-Tiny	3	33.2	[35]
	YoloV4-Tiny	3	45.0
Captured from field	YoloV5s	1	59	[20]
	YoloV5l	1	67
	YoloV5m	1	84
Collected from field	VGG-16	2	99.9	Our Work
	Cov2D	2	99.9

; the momentum parameter, basic learning, and dropout layers were set according to [20]. As a result, this study conducted a comparative analysis between different fine-tuned models with our proposed model, presented in Table 7, which were carefully selected to ensure remarkable accuracy for both testing and training, as presented in Table 4 and Table 5 and Figure 4.

4. Conclusions

In this research article, we conducted successful analysis of two different deep learning models under tensor flow framework to identify suitable models for detecting the fall armyworm pest in summer maize leaves. A dataset of 7000 images was collected from Shandong university research farm at Zibo city, Shandong province, China.

The evaluation of state-of-the art CNNs (convolutional neural networks) using deep learning was based on various metrics including training accuracy, testing accuracy, training loss, testing loss, recall, and F1-score. Our proposed models VGG-16, and Cov2D were compared against other well-known models AlexNet, ResNet, Inspection-V4, YoloV3-Tiny, YoloV4-Tiny, and YoloV5s, m, l, shown in Table 7. It was found that VGG-16 and Cov2D model had outstanding accuracy results.

One of the key advantages of VGG-16 and Cov2D models was their ease of training and testing of the models; the different trainable parameters are shown in Table 3. Additionally, Table 4 shows that the hyperparameters were the same for both models except the dropout parameter. Hence the VGG-16 model is more suitable for a leaf detection system when there is a new pest to be included in the model. The selected models achieved detection accuracy and loss accuracy, respectively presented in Table 5 and Table 6.

The future research work will involve evaluating the performance metrics of the proposed models on larger and different types of dataset to assess their generalization and robustness. Additionally, we plan to train and test models on larger datasets for real-time detection in a spot spraying system to improve the efficiency and accuracy of pest management practices in the agricultural field in order to increase production and save costs on pest control.