Design and Experiment of a Broken Corn Kernel Detection Device Based on the Yolov4-Tiny Algorithm

Abstract

At present, the wide application of the CNN (convolutional neural network) algorithm has greatly improved the intelligence level of agricultural machinery. Accurate and real-time detection for outdoor conditions is necessary for realizing intelligence and automation of corn harvesting. In view of the problems with existing detection methods for judging the integrity of corn kernels, such as low accuracy, poor reliability, and difficulty in adapting to the complicated and changeable harvesting environment, this paper investigates a broken corn kernel detection device for combine harvesters by using the yolov4-tiny model. Hardware construction is first designed to acquire continuous images and processing of corn kernels without overlap. Based on the images collected, the yolov4-tiny model is then utilized for training recognition of the intact and broken corn kernels samples. Next, a broken corn kernel detection algorithm is developed. Finally, the experiments are carried out to verify the effectiveness of the broken corn kernel detection device. The laboratory results show that the accuracy of the yolov4-tiny model is 93.5% for intact kernels and 93.0% for broken kernels, and the value of precision, recall, and F1 score are 92.8%, 93.5%, and 93.11%, respectively. The field experiment results show that the broken kernel rate obtained by the designed detection device are in good agreement with that obtained by the manually calculated statistic, with differentials at only 0.8%. This study provides a technical reference of a real-time method for detecting a broken corn kernel rate.

1. Introduction

Corn is an indispensable food crop for people; corn kernels are damaged in the process of harvesting, threshing, transport, and storage. In particular, the corn kernel breakage rate is one of the important indexes to evaluate the harvest quality, and reflects the performance of harvesting machinery. For a long time, corn harvesters have lacked detection of the kernel breakage rate, and usually rely on the experience of drivers to control working parameters to avoid damaged corn kernels as much as possible. However, this method is extremely inefficient and unsatisfactory, and severely restricts the development of intelligent corn combine harvesters.

At present, scholars worldwide have done much work in agricultural product quality inspection. Nevertheless, due to the extremely harsh working environment of corn harvesting, the large feed amount of corn ears, and the scattered distribution of dust, bracts, and damaged mandrels, it is difficult to detect the small corn kernels that accumulate, overlap, and shield in the vehicle-mounted environment, which is unfavorable to harvest efficiency. Therefore, a real-time broken corn kernel detection device is necessary for improving combine harvester working performance, and the rationality, response speed, and accuracy of the recognition algorithm, together with the reliability, convenience, and stability of the hardware structure should be fully considered in the design scheme.

The technology of deep learning represented by convolutional neural networks has developed rapidly. Perugachi-Diaz et al. [1] collected the growth images of more than 10,000 Chinese cabbage seedlings in 14 days, and the growth prediction model was constructed by AlexNet to predict the success rate of Chinese cabbage growth. Chen et al. [2] utilized VGGNet to identify the leaf disease images of rice and corn plants, which can accurately identify rice diseases in complex environments, with an accuracy rate of 92%. Altuntaş et al. [3] used VGG19 to distinguish haploid and diploid seeds; the accuracy rate reached 94.22%, which indicated that the VGG19 algorithm had higher accuracy in identifying small differences of corn seeds. Mohanty et al. [4] compared the effects of AlexNet and GoogLeNet in virus detection of crop leaves; the results showed that GoogLeNet was more effective. Flores et al. [5] adopted several CNN algorithms to learn and distinguish RGB and CIR images of corn and soybean, and it was found that GoogLeNet is an effective discrimination method with an accuracy of 99.9% and calculation speed of 20 ms. Rasmussen et al. [6] employed Faster R-CNN to detect broken corn silage particles after harvesting; the results showed that the detection accuracy was 45.2%, which could identify large particles from miscellaneous ones. Monhollen et al. [7] developed a corn kernel loss rate detection program based on Faster R-CNN, which achieved an average accuracy of 0.90, the additional field tests obtained the accuracy of 0.91. Suo et al. [8] used yolov4 to study the transfer of kiwifruit detection, and obtained the highest mAP of 91.9% with an image processing speed of 25.5 ms. Zhang et al. [9] proposed a water-meter pointer-reading recognition method based on improved yolov4; the detection accuracy of this method reached 98.68%, which indicated that the lightweight algorithm could quickly and accurately identify targets. Li et al. [10] proposed a rapid detection model for green pepper based on yolov4-tiny, the average precision is 95.11%, the model size is 30.9 MB, and the frame rate is 89 FPS. The CNN algorithms used in this research have reached a high level of accuracy in identifying crop characteristics, which can provide a good research basis for the detection of broken corn kernels. Compared with the above algorithms, yolov4-tiny has fewer input parameters, a simpler network structure, and a faster processing speed, which make the method feasible for mobile and embedded devices.

In order to meet the needs of a fast and portable corn kernel detection method, this paper designed a broken corn kernel detection device based on the yolov4-tiny algorithm, which provides a new approach for quick and accurate identification and classification of broken corn kernels. First, the hardware structure was designed, considering the antiblocking design of corn kernels, including a closed detection environment with a sufficient light source; the images of corn kernels are captured by the CCD camera, and transported by the monolayer mechanism and the convey belt. The monolayer mechanism is designed to eliminate the influence of a large feed quantity during the detection process. This scheme ensures the continuity of the detection process, and it is suitable for a vehicle-mounted environment and large harvesting quantities of corn kernels. Then, in view of the obvious differences in the contour features of intact corn kernels and broken corn kernels, and lack of an effective algorithm to detect the kernels with a large feeding amount, the yolov4-tiny model is utilized to ensure portability and real-time detection. The experiment results show that the broken corn kernel detection device has high accuracy, reliability, and stability, thus providing a solution for real-time detection of corn harvest effects in a field operation environment.

2. Materials and Methods

2.1. Design of Broken Corn Kernel Detection Device

Owing to the harsh working environment, large vibration of the machine body, and easy accumulation of corn kernels during transport, it is difficult to realize real-time detection of corn kernels used existing recognition algorithms. In order to solve the problem, the hardware structure of a broken corn kernel detection device was designed, as shown in Figure 1. The detection device is mainly composed of the frame, feeding system, monolayer mechanism, power system, image acquisition system, and image processing system. Among them, the feeding system includes the collection hopper, blanking trough, round sieve holes of corn kernels, and slide-out plate for corn kernels; the power system includes the DC motor, conveyor belt, and belt wheel; the belt wheel is driven by the DC motor to impel the conveyor belt rotating at linear speed of 0.3 m/s. The feeding system consists of a collecting hopper, blanking trough, flow-regulating plate and sieve plate; the image acquisition system includes a CCD camera, LED light, and Gigabit network cable.

The monolayer mechanism is designed to ensure the CCD camera can capture the images of individual corn kernels, prevent the corn kernel accumulation and overlap from adversely affecting the acquisition process and image processing effect, as shown in Figure 2. The single and discrete distribution of corn kernels falling on the conveyor belt mainly depends on a sieve with round holes; the sieve is obliquely installed in the blanking trough. The round holes have the same size as corn kernels, so the falling speed of corn kernels slows when passing through the round holes; the corn kernels fall on the conveyor belt and distribute in a single layer, while the remaining corn kernels repeatedly collide with sieve plates and bounce back to the granary. By adjusting the position of the sieve on the frame, and then adjusting the installation angle, the amount of corn kernels can be controlled. The periphery of the blanking trough is a closed structure to prevent the corn kernels from falling outside or entering the synchronous belt.

2.2. Working Conditions

The working conditions of each key component mechanism follow.

(1)

The LED light source is composed of 4 LED tubes, fixed in the inner side at the top of the frame, and located on both sides of the CCD camera; it provides a stable illumination intensity in the closed detection box to ensure the stability of the color characteristics of the collected corn kernel images, as shown in Figure 1a.

(2)

The DC motor (Baikong, 57BL115S21-230TF9, Shanghai, China) operates the drive shaft to rotate the synchronous belt, which transports the corn kernels to the image acquisition area.

(3)

The image acquisition system adopts a CCD camera (Medway Vision Company, MV-GE200C-T). It is perpendicular to the timing belt and fixed above the device by a bracket. The images of corn kernels collected by CCD camera are transmitted to the image processor system through the Gigabit network cable.

(4)

The image processing system is the Jeston TX2 Development Board of NVIDIA Corporation, which adopts NVIDIA PascalTM GPU architecture, has 56 NVIDIA CUDA cores, and 8 GB of running memory [11]. The board has the advantages of small size and fast processing speed, and meets the requirements of onboard application for real-time, convenient installation, and processor portability.

2.3. Working Principle

The broken corn kernel detection device installs in the granary of a corn combine harvester, as shown in Figure 1b. When the broken corn kernel rate device is operating, the corn kernels are harvested in the granary by the auger, then enter the collection hopper of the device and fall to the convey belt through the monolayer mechanism. At the same time, some corncobs and bracts can be screened out of the detection area by the sieve plate, which reduces the interference of impurities in the image and improves the detection accuracy.

2.4. Experiment Method

In order to test the reliability and accuracy of the broken corn kernel detection device under different working conditions, based on the corn kernel direct harvester, the verification experiment was carried out on 29 September 2021. The verification experiment followed “GB/T 21962-2020 Corn combine harvesters” and “GB/T 21961-2008 Test Methods for Maize Combine Harvester” [12,13] protocol. The working width of the corn harvester was 4 m; we randomly selected an area with a length of 20 m without lodging corn plants.

The working parameters of the corn kernel direct harvester are shown in

Table 1. The working parameters of the corn kernel direct harvester.

Number	Threshing Cylinder Speed/(r·min−1)	Concave Clearance/mm	Operating Speed/(km·h−1)
1	300	35	3.0
2	300	40	3.5
3	300	35	4.0
4	300	30	3.5
5	350	35	3.5
6	350	40	3.0
7	350	30	4.0
8	350	35	3.5
9	350	40	4.0
10	350	30	3.0
11	400	35	4.0
12	400	35	3.0
13	400	40	3.5
14	400	30	3.5

; the threshing cylinder speed was selected with 3 levels: 300, 350, and 400 r/min; the concave clearance was selected with 3 levels: 35, 40, and 45 mm, and the traveling speed was also selected with 3 levels: 3.0, 3.5, and 4.0 km/h. The test scheme was designed in Design Expert software using the BBD (Box–Behnken design) method [14,15]. Design Expert is commonly used test design software embedded with the BBD method. The 3 test factors and 3 levels were added into Design Expert, which automatically generated 17 groups of experiments, of which 4 groups were repetitive. However, this study only utilized the software to provide an experiment, so the repeated groups were eliminated, and a total of 14 groups of experiments were conducted, and each group was repeated 3 times. Additionally, the Box–Behnken design method can be used for experiments with 3–7 factors, in which the nonlinear influence of factors can be evaluated. It is applicable to tests in which all factors are measured values. When used, many continuous tests are not needed; there was no axial point in the design, which was suitable for this verification experiment.

After the experiment, we randomly weighed more than 500 g of corn kernels from the granary 3 times. The average value of the broken corn kernel rate statistic, calculated manually, is recorded as R₁; the calculation method is shown in Equation (1), while the broken corn kernel rate obtained by the detection algorithm is recorded as R₂:

$$R_1=\frac{1}{3}{\displaystyle \sum _i^{i=3}\frac{m_i}{m_{gi}}}\times 100\%$$

(1)

where m_i is weight of the broken corn kernels sample, i = 1, 2, 3, g; m_gi is weight of all corn kernels sample, i = 1, 2, 3, g.

2.5. Image Acquisition

For ensuring the rationality and scientific of the data set, a total of 3000 images showing samples of intact and damaged kernels were collected for training the detection algorithm. The image resolution was 1200 × 1600 and the image format jpg. Figure 3 shows a part of collected images of corn kernels for training.

Under the action of the vibrating screen and threshing cylinder, severe vibration would still occur. In addition, some fragments of straw and bract enter the detection device, which led to some salt-and-pepper noise points in the collected image, as shown in Figure 4a. If the image was detected directly without pretreatment, the detection accuracy would be reduced. By comparing the effects of a Gaussian filter [16], mean filter [17], and median filter [18], it was found that there were still some salt-and-pepper noise points in the images after the mean filter, and the images after the Gaussian filter were not sufficiently clear to reflect the characteristic information of the corn kernels accurately. Obviously, the median filter can effectively remove salt-and-pepper noise in the corn images; the images were relatively clear and the details of corn were completely preserved. Therefore, median filtering was adopted as the noise processing method in this paper. The basic principle of median filtering is to randomly select an m × m matrix as the pixels of each channel of the collected RGB image of kernel, then arrange these m² pixel values in the order from small to large and take the median of the series to replace the pixel values in the center of the matrix [19]. Images of corn kernels after filtering are shown in Figure 4b. After median filtering, the images were obviously improved and clearer; the salt-and-pepper noise points were filtered out, which provides a good basis for subsequent broken kernel identification.

2.6. Training of the Yolov4-Tiny Model

In addition to the classic CNN such as LetNet, AlexNet, VGG19, ResNet50, and GoogleNet, the yolov4-tiny algorithm proposed by Bochkovskiy et al. [20] in June 2020 is a lightweight and fast target detection algorithm that is a compressed and improved version based on yolov4. The process of using the transfer learning training model is shown in Figure 5a. The layers of the yolov4-tiny model are reduced from 162 to 38 layers, and the training parameters are only one-tenth that of yolov4. The yolov4-tiny network extracts corn kernel image features through the backbone feature extraction network, and uses a feature pyramid to fuse different ranges of kernel feature information [21,22,23].

With the increase in the depth of the network, the detection accuracy of the model becomes increasingly higher. However, when the network model is too complex, network degradation occurs, and the training accuracy of the model drops rapidly, becoming even worse than that of the shallow network and resulting in a decline or disappearance of the gradient. As shown in Figure 5b, CSPDarknet53 adds a residual module based on Darknet53 [24,25], and the residual network is connected by skipping layers, which makes it easier to optimize the broken kernel rate detection model during training. Some features of the corn kernel images pass through residual blocks, while the other part passes through convolution layers, which transfers the fused corn kernel features to the next stage and makes full use of the spatial structure information of the corn kernel feature image. The feature extraction network is used to extract the feature information from the input corn kernel image, which increases the depth of the network, reduces the calculation amount, solves the problem of gradient disappearance in the deep network, improves the classification accuracy of the detection model, and reduces the memory requirement, thus improving the learning ability of the network.

The higher the level, the wider the network receptive field is, and the more information contained in the corn kernel images. However, the resolution of the images is relatively low. After many convolution operations, the image features of many corn kernels become blurred, which makes it impossible to accurately represent the corn kernel feature information. If the model classifies corn kernels according to the features of the last layer, many details of the corn kernels are lost. Although this operation has little effect on large-sized objects, for small-sized objects such as corn kernels, the area of one corn kernel in the image is relatively small, and there is less pixel information, which easily leads to the loss of feature information in the down sampling process, resulting in a decrease in detection accuracy. In order to solve the problem of missing features in small-sized target detection, the detection accuracy is improved by fusing the features of corn kernels in different receptive fields of the upper and lower layers. The output feature matrix can better display the feature information of high-level corn kernels and the location coordinates of low-level corn kernels. Yolov4-tiny model up samples the output 13 × 13 feature images through the convolution layer, and then fuses it with the 26 × 26 feature images, and then superimposes convolution, finally outputting 13 × 13 and 26 × 26 scale feature images. These feature images are used to classify and predict corn kernels at different scales, thereby improving the detection accuracy of corn kernels as small targets.

The processing steps of corn kernel images follow. Firstly, as shown in Figure 6, the images are labeled with LabelImg, and the corn kernels are divided into a two-class test sample set. Then, a rectangular region containing the seeds is framed in the image. The label of intact kernels is 1, while the label of broken kernels is 2. The file information is saved in xml format. The xml file contains the coordinate position and tag information of all kernels. The corn kernels contained in the image set are divided into a training set and a test set according to the proportion 7:3. The system training processing steps follow. (1) Preprocessing and labeling the collected images provides the position information and labeling information of the kernels. (2) The sample set is randomly divided into training, verification, and detection sets, according to the ratio 0.7:0.15:0.15. The network parameters are initialized and the initial learning rate is set to 0.001, the number of iterations is set to 40, and the learning rate attenuation value is set to 0.0005. (3) Through the trunk feature extraction network, the training set images are convoluted to obtain the texture feature map, and the texture feature data information of each layer is normalized in batches. (4) Maximum pooling is performed on the feature map obtained in step (3) to produce a generalized image of corn kernels. (5) The gap between the recognition result and the tag is calculated, and the network training parameters are updated. (6) When the loss rate falls within the set range or reaches the maximum number of iterations, the training is finished.

2.7. Evaluation of the Yolov4-Tiny Model

The 3000 images of corn kernels were trained in the yolov4-tiny model after preprocessing. In order to simplify the calculation process, Leaky ReLU was used as the activation function of CSPDarknet [26,27]. The expression of Leaky ReLU is shown in Equation (2):

$$y_i=\left\{\begin{array}{l}\begin{array}{ll}{x}_i& x_i\ge 0\\ \frac{x_i}{a_i}& x_i\le 0\end{array}\end{array}\right.$$

(2)

where, a_i ∈ (1, +∞) are a set of constant parameters.

A cross-stage partial connections structure is used; the CSPBlock divides the input feature map into two parts and concatenates the two parts in the cross-stage residual edge. In the multifeatured fusion stage, the yolov4-tiny model constructs a feature pyramid network to extract feature maps. Through the feature pyramid network, we obtain two effective feature maps of different sizes. The detections are then estimated; the yolov4-tiny model utilizes the fused feature maps by the classification and location of the targets. The detections with a confidence score lower than the preset threshold is removed. The confidence score for each detection is defined by Equation (3):

$$Conf=Pr\left(object\right)\times Io{U}_{pred}^{truth}$$

(3)

where Pr(object) is the possibility that the detection box contains an object, and $Io{U}_{pred}^{truth}$ is the IoU between the predicted bounding box R^pred and the ground-truth box R^truth, which is defined by Equation (4):

$$Io{U}_{pred}^{truth}=\frac{\left|R^{pred}\cap R^{truth}\right|}{\left|R^{pred}\cup R^{truth}\right|}$$

(4)

Next, the yolov4-tiny model adopts the classification loss function to measure the category error between the predicted box and the ground-truth box. The classification loss function is shown as Equation (5).

$$L_{cls}=-{\displaystyle \sum _{i=0}^{S\times S}{I}_{ij}^{obj}{\displaystyle \sum _{c\in classes}\left[{\widehat{p}}_i(c)\log \left(p_i\left(c\right)\right)+\left(1-{\widehat{p}}_i(c)\right)\log \left(1-p_i(c)\right)\right]}}$$

(5)

Later, the yolov4-tiny model uses the CIoU loss function for bounding box regression. The CIoU loss function is defined as Equation (6).

$$L_{CIoU}=1-Io{U}_{pred}^{truth}+\frac{{\rho }^2\left(b^{pred},b^{truth}\right)}{c^2}+\frac{16}{{\mathsf{\pi }}^4}\frac{{\left(\arctan \frac{w^{truth}}{h^{truth}}-\arctan \frac{w^{pred}}{h^{pred}}\right)}^4}{1-Io{U}_{pred}^{truth}+\frac{4}{{\mathsf{\pi }}^2}{\left(\arctan \frac{w^{truth}}{h^{truth}}-\arctan \frac{w^{pred}}{h^{pred}}\right)}^{2’}}$$

(6)

where ρ²(∙) is the Euclidean distance; b^pred and b^truth are the central points of R^pred and R^truth, respectively; c is the diagonal length of the smallest enclosing box covering R^pred and R^truth; and w and h are the width and height of the bounding box, respectively.

In order to accurately evaluate the performance of the model, the AP, precision, recall, and F1 core were utilized to examine the yolov4-tiny model. The precision and recall have four states after the test sample is predicted: true positive (TP), false positive (FP), true negative (TN), and false negative (FN). The definition of these indexes is shown in Equation (7). The sample dividing threshold is 50%.

$$\left\{\begin{array}{l}precision=\frac{TP}{TP+FP}\\ recall=\frac{TP}{TP+FN}\\ F1=\frac{2\times precision\times recall}{precision+recall}\end{array}\right.$$

(7)

3. Results and Discussion

3.1. Laboratory Experiment Results

This section evaluates the proposed broken corn kernel detection algorithm, based on the yolov4-tiny model, and its running environment is shown in

Table 2. Experimental environment.

Configuration	Parameter
CPU	HMP Dual Denver; ARM Cortex-A57 Quadcore
GPU	256 core NVIDIA Pascal
Accelerated environment	CUDA10.1
Development environment	Ubuntu 20.04.3 LTS
Operating system	Linux

. The loss curve of the training set and Val-loss curve of testing set are shown in Figure 7. The batch size of training and verification is set to 32, and the established CNN is trained on the GPU platform. After 40 iterations, the classification accuracy is stable at 97% on the test set. It can be seen from the loss curve that the loss rate drops rapidly by epoch 5, followed by a gentle decline stage during which there is no large fluctuation, and the final loss value is 7.8, showing that the model fits well in the learning process.

In addition, we randomly selected 800 corn kernels to verify the accuracy of the yolov4-tiny model in the laboratory environment, including 400 intact kernels and 400 broken kernels. After starting the equipment, we manually feed corn kernels into the device, counting the correctly identified intact corn kernels and broken corn kernels. The recognition results are recorded and compared with the judgment detection results to obtain the model recognition accuracy. The results are shown in

Table 3. Detection result of 800 corn kernels.

Category	Number of Identified as Intact Corn Kernels	Number of Identified as Broken Corn Kernels	Accuracy of Detection Model/%
Intact corn kernels	374 (TP)	26 (FN)	93.5%
Broken corn kernels	29 (FP)	372 (TN)	93.0%

; the recognition accuracy of the model was calculated as 93% for broken corn kernels and 93.5% for intact corn kernels, which basically met the requirements of real-time detection; it has high accuracy and reliability.

The correctly identified intact corn kernels are marked as TP, the incorrectly identified are marked as FN; meanwhile, the correctly identified broken kernels are marked as TN and the incorrectly identified marked as FP. According to Equation (7), the precision, recall, and F1 score are calculated as 92.8%, 93.5%, and 93.11%, respectively. The F1 score combines the results of precision and recall; when F1 is higher, it shows that the yolov4-tiny model is more effective.

3.2. Field Experiment Results

In order to verify the working effect of the broken corn kernel detection device, a field experiment was carried out based on a corn grain direct harvester; the results of the broken corn kernel detection algorithm for groups 2, 3, 7, and 8 are captured and shown in Table 1 and Figure 8a–d. The correctly detected corn kernels are marked by a blue box, while the erroneously detected corn kernels are marked by a red box. The first number on the top of the box represents the type of corn kernels; the second number represents the confidence.

In Figure 8a, the corn kernels all were intact, while there were none erroneously detected as broken. In Figure 8b, a corn kernel is marked by a red box, with confidence of 0.40, indicating that this corn kernel was intact but incorrectly identified as broken. Other kernels were detected correctly. In Figure 8c, four corn kernels were incorrectly identified as broken, with confidence smaller than 0.5; this may be due to the vibration of the harvester in the field operation, or the impact of a large number of corn kernels thrown by the screw on the device, which led to a short-term dimming of the LED light source, thus worsening the detection environment; this problem can be actively overcome by adding damping devices, rather than passively eliminating only the tires of the harvester. In Figure 8d, a broken corn kernel was correctly predicted as an intact corn kernel, and its confidence was only 0.33, which means that the characteristics of this broken corn kernel were very similar to those of a intact corn kernel. The detection algorithm had classification errors; thus, the recognition accuracy of this subtle difference needs to be improved.

Figure 8e shows the P–R curve of five images during the detection process, which are randomly captured to calculate the value of AP; the results are 97.18%, 86.11%, 96.59%, 95.42%, and 94.51%, and the fps is capped at about five frames per second. In the report of Li et al. [10], the green pepper detection algorithm, based on an improved yolov4-tiny model, is 95.11%, and the frame rate is 89 FPS; the difference is mainly reflected in the size of the model. The detection algorithm proposed in this paper has a smaller volume, so it has a fast calculation speed and suitable for the real-time requirements of a vehicle-mounted environment. We captured the calculation results of the broken corn kernel detection algorithm during 14 groups of experiments; in each experiment, we selected five images to count the values of TP, FP, and TN, and calculated the value of precision and recall; the results are shown in

Table 4. The evaluation indexes of the field experiment result.

Number	TP	FP	FN	Precision	Recall
1	82	4	6	95.35	93.18
2	103	11	4	90.35	96.26
3	89	4	4	95.70	95.70
4	77	7	4	91.67	95.06
5	102	12	10	89.47	91.07
6	105	6	8	94.59	92.92
7	94	8	7	92.16	93.07
8	92	4	6	95.83	93.88
9	81	6	6	93.10	93.10
10	112	10	16	91.80	87.50
11	107	9	14	92.24	88.43
12	96	7	12	93.20	88.89
13	91	11	8	89.22	91.92
14	88	9	7	90.72	92.63

. The average precision and recall are 92.53% and 92.4%, respectively, showing that the broken corn kernel detection device has satisfactory detection accuracy in the laboratory and vehicle field environment, and it is proved that this detection device can operate stably and reliably in a complex and changeable environment, with severe dust and other extreme conditions.

In addition, Table 4 shows the results of the broken kernel rate statistic calculated manually and that read by the broken corn kernel detection device. Under different working conditions, the broken kernel rate has obvious differences. The higher the threshing cylinder speed, the smaller the concave clearance; the faster the operating speed, then the more broken kernels. According to the requirements of the standard, the broken kernel rate should be less than 2%; groups 6 and 8 reached this standard, and the results of the broken kernel rate detection device were 1.5% and 1.8%, respectively. At the same time, these results are in good agreement with the broken kernel rate obtained by manually calculating the statistic, which prove the reliability and accuracy of the designed detection device and the algorithm.

As shown in Table 4, according to previous experimental experience, the broken kernel rate is significantly affected by the harvester traveling speed, threshing cylinder speed, and concave clearance. The faster the traveling speed, the higher the threshing cylinder speed and concave clearance, the larger the broken kernel rate. The model utilized in this paper can be applied to the detection of the broken corn kernel rate of a corn combine harvester in the field environment, and solved the problems that the defects of low detection accuracy, harsh working conditions, and low automation degree of most detection algorithms, provided support for the research of intelligent control of corn combine harvesters. The broken corn rate detection device could identify corn kernels well; however, the detection accuracy of some corn kernels with small broken and overlapping parts was reduced to a certain extent, especially some corn kernels with irregular shapes that easily caused a false detection, which needs further investigation.

Figure 9 shows the experiment results of the manually calculated statistic and the broken corn kernel detection device. The average broken kernel rate of the detection device R2 is 5.22% and the average broken kernel rate by manual R1 is 4.42%. The broken kernel rate detected by the detection device is slightly higher than the manually calculated statistic result R1, and the average differential is 0.80%, which avoids the deficiency of subjective feature selection in machine vision detection methods. The difference is mainly caused by the broken corn kernels that are small relative to other crops, such as vegetable leaves and fruits, resulting in inaccurate identification. This matter can be improved by the detection accuracy of the yolov4-tiny model through enlarging the number of kernel training sets and different types of kernels. Besides these reasons, the backbone network has insufficient feature extraction capabilities, and this may also be due to a slight mistake in the manually calculated statistic, which can be within the acceptable range.

During the harvesting process of the corn combine harvester, it is necessary to ensure the accuracy of the detection results and the real-time performance of the detection. The detection system is usually located in an embedded system or mobile device, which requires high memory storage of the model, so the calculation amount should not be too large. Many detection models have large network depth and high detection accuracy, but their detection speed is slow, which cannot meet real-time detection requirements in practical applications. The yolov4-tiny detection model not only ensures that the detection accuracy of the model meets the detection requirements, but also has good detection speed and meets the requirements of real-time detection of the broken corn kernel rate. Considering the working environment of real-time detection of broken corn kernel rate and the operation ability of the embedded development board, the yolov4-tiny algorithm met the requirements of the corn kernel detection process.

4. Conclusions

In this paper, we proposed a target classification algorithm based on the yolov4-tiny model to detect broken corn kernels. The yolov4-tiny model strengthens the detection of small-particle size crops; the accuracy of the detection algorithm was verified by laboratory experiment. In addition, we also designed a broken grain detection device that meets the requirements of mobile and embeddable vehicle-mounted equipment, and verified its working performance through field experiment. The accuracy rate of intact corn kernels was 93.5% and that of broken corn kernels was 93.0%. The evaluation indexes of precision, recall, F1 score, and AP reflected that the broken corn kernels can be well distinguished, but detecting tiny damage was somewhat difficult. The broken kernel detection model can automatically and effectively learn corn kernel features from images; the average differential between the manually calculated statistic result and the broken corn kernel detection device test result was only 0.80%. Experiments showed that the yolov4-tiny model can not only ensure detection speed of the lightweight model, but also improves the detection performance and realizes the performance standard of large-scale target detection model.

In future corn combine harvester application, we will focus on research that increases the variety and quantity of samples, improves the detection accuracy, and develops the working reliability of the detection system. Through intelligent detection, it is helpful to realize the intelligent and automatic control of the corn harvest process.