Design and Test of the Seedling Cavitation and Lodging Monitoring System for the Rape Blanket Seedling Transplanter

Abstract

To realize the real-time monitoring of the lodging and cavitation state of the rape blanket seedling transplanter, a real-time online monitoring system set for cavitation and lodging based on machine vision was designed. The system was mainly composed of an image acquisition module, a height sensor, and an IPC (Industrial Personal Computer). When the system starts to work, the height of the lens from the ground and the collected images are saved in real time, and the operating status of the equipment is judged according to the height of the lens from the ground, and the actual corresponding size of the image pixels is corrected. We proposed a calculation method for judging the cavitation and lodging state of the seedlings based on the comparison between the actual planting position and the theoretical planting position of the seedlings. The processing and analysis software for the cavitation and lodging monitoring system of the rape blanket transplanter was compiled, and the monitoring system was developed. The dynamic test showed that the maximum absolute error of the cavitation recognition rate was 1.23%, and the maximum absolute error of the lodging recognition rate was 3.80%. The total time consumed of a 1 m image collected in real time from the collection, image processing and stitching, to seedling status recognition of the rape blanket seedling field was about 1 s. The accuracy and system processing time of the seedling cavitation and lodging state judgment can meet the accuracy and real-time requirements of cavitation and lodging state recognition for field transfer of rape blanket seedlings.

1. Introduction

Rape seedling transplanting is an effective planting method to solve the stubble contradiction. At present, the transplanting of rape seedling in China mainly includes bare root seedlings, potted seedlings, and blanket seedlings [1]. Rape blanket seedling transplanting has attracted extensive attention in recent years because of its high efficiency and soil adaptability [2,3,4,5]. However, there are also some problems including the seedling pieces being squeezed or torn during the process of cutting and taking seedlings; the mechanism was reversed to send the seedlings, and the mechanism transmission error caused the seedlings to deform, resulting in uneven cutting. When the transplanting process was blocked, and seedlings dropped and buried, the machine operation failures could be found in time through real-time monitoring of seedling planting status, such as cavitation and lodging during the transplanting process. In the field of seedling distribution and seedling identification research after transplantation, scholars at home and abroad have conducted many related studies. The relationship between the seedling cavity area on the plug tray and the total pixel was used to predict the lack of cabbage seedlings after planting with a three-layer neural network [6,7]. Field self-propelled robots were used to collect images of peanut seedlings to identify the number of seedlings and the distance between them [8,9]. Drones were used to collect tobacco images after planting and projection methods were used to calculate the average plant spacing to determine the position of missing seedlings [10,11,12]. A binocular stereo vision system for corn leaf positioning and recognition was designed. The images of corn seedlings were obtained by CCD cameras and were recognized by multi-scale hierarchical convolutional neural network models. Low-altitude drones were used to obtain images of field cotton, and established support vector machines based on color features and grayscale models of plant counting [13,14,15]. The above research can realize the identification and statistics of the field distribution of the seedlings after planting [16,17], but it cannot provide the real-time detection and acquisition of the seedling status during the planting process, and cannot provide real-time feedback on the seedling planting status during the transplanting operation.

The purpose of this research study is to build a real-time monitoring system for the cavitation and lodging of the rape blanket seedling transplanter based on image acquisition and processing. The transplanted rape blanket seedlings are collected in real time through an industrial camera, and the height sensor is used to record the height of the lens from the ground in real time. Judging the height of the camera from the ground to realize the real-time perception of the working status of the machine, combined with the calculation method for determining the lodging and cavitation of the rape blanket seedlings, allows the realization of the real-time monitoring of the transplanting cavitation and lodging status of the rape blanket seedlings, which can evaluate the performance of the transplanter. Real-time monitoring of the quality of transplanting operations and out-of-tolerance alarms can realize online monitoring of cavitation and lodging in the transplanting of rape blanket seedlings and evaluation of operational performance.

2. Materials and Methods

2.1. Test Platform

The seedling cavitation and lodging monitoring system were developed based on the 2ZGZK-6 rape blanket seedling transplanter. The main parameters of the rape blanket seedling transplanter are shown in

Table 1. Main parameters of rape blanket transplanter.

Items	Parameters	Items	Parameters
Rated power [kw]	88–100	Line spacing [cm]	30
Machine quality [kg]	1600	Hill spacing [cm]	12/14/16/18/20/22
Work width [cm]	2300	Depth of planting [cm]	10–50
Line of works	6	Working speed [m/s]	0.6–1.2

.

2.2. Rape Blanket Seedling

The rape variety was Ningza 1818, the seedling age was 30 days when the rape blanket seedlings were transplanted, and the water content of the seedlings was between 50 and 52%. We used the 2ZYG-6 type rape blanket seedling transplanter to cut into pieces and take the seedlings with an area of 23 mm × 17 mm, moving forward at a planting speed of 1 m/s. We randomly selected 40 seedling pieces and tested the basic parameters of each piece such as number, seedling height, seedling size, maximum leaf width, and true leaf area. The characteristic morphological parameters of the tested rape blanket seedlings in the suitable planting period are shown in

Table 2. Morphological characteristic of rape blanket seedlings.

Name	Average Value ± Standard Deviation	Variance	Standard Error	Mean 95% CI (LL)	Mean 95% CI (UL)
Seedling height	78.625 ± 13.258	175.779	2.096	74.516	82.734
Number of leaves	5.875 ± 1.870	3.497	0.296	5.296	6.454
Maximum leaf width	41.175 ± 15.248	232.507	2.411	36.450	45.900
Maximum area	514.625 ± 162.260	26,328.394	25.656	464.341	564.909
Minimum area	214.725 ± 92.153	8492.153	14.571	186.167	243.283
Number of seedlings per hole	1.800 ± 0.728	0.531	0.103	1.598	2.002

.

2.3. Structure and Working Principle

Seedling cavitation and lodging monitoring system are as shown in Figure 1. The hardware of the cavitation and lodging monitoring system for the rape blanket seedling transplanter mainly includes bracket, Planar array CCD industrial camera (Minsvision-MS-UB500C), camera lens (Hua Teng Weishi HTF0618-5MP, focal length 6 mm, aperture 1:1.4, image surface size 1/2”), ranging sensor (Beixing TF-luna lidar module, accuracy ±0.5 cm, distance resolution 1 cm, frame rate 100 Hz), IPC (Industrial Personal Computer) (CPU core frequency 1.8 GHz, memory 16 G, 512 G hard disk, windows10 operating system), and other parts.

The bracket was fixed on the rear beam of the transplanter, the industrial camera and the ranging sensor were fixed on the bracket, and the industrial camera lens was vertically downward. Under the unified control of the IPC, the industrial camera and the ranging sensor start to work synchronously. When the reading of the ultrasonic ranging sensor was lower than the set threshold of 800 mm (the distance between the ranging sensor and the ground height value when the machine was actually working), the industrial camera started to work. In order to accurately determine the starting point of planting, start to save the image when the green plant appears in the middle of the image. When the height of the ultrasonic ranging sensor exceeds the threshold of 900 mm, it indicates that the transplanter was changing directions or stops transplanting. At this time, stop collecting images and the errors alarm and return to the initial state after finishing saving the data. During the planting process, the collected images were spliced and processed online to calculate planting status monitoring indicators such as lodging and cavitation, which were posted on the screen. When the lodging or cavitation status exceeds the set threshold, an alarm display would be displayed through the interface prompt light. The diagram of seedling cavitation and lodging monitoring system process are as shown in Figure 2.

3. Image Acquisition and Processing

3.1. Preprocessing

The images acquired in the field transplanting process were greatly affected by the external environment light, among other factors. The image contrast was worse, and the noise greatly affects the accuracy of seedling recognition. In the seedling image after transplanting, the stalk image occupies fewer pixels and has a significant impact on the planting position. The judgment was of great significance and it was necessary to filter the images of the seedlings after transplanting so as to achieve a smooth image and a better capture of the edges in the image of the seedlings. The CLAHE [18] algorithm was used for the histogram equalization processing of the collected original image. After processing by the CLAHE algorithm (Figure 3b), the brightness and contrast of the image were improved. The seedlings were separated from the background image obviously, but there was still noise in the image, and the image needed to be smoothed to reduce the noise. The bilateral filtering [19] algorithm was used to filter the smoothed image. The bilateral filtering effect (Figure 3c) was used to threshold the filtered image to achieve rape capture of the “green” characteristics of seedlings.

3.2. Image Stitching

The purpose of image stitching is to remove the overlapping areas of information in adjacent frames of images and to combine multiple images in the sequence of images into a continuous view of the transplanting job, preparing for subsequent processing such as image segmentation and calculation of seedling planting status. The rape blanket seedling transplanter does not always walk in a straight line during the transplanting operation, and the forward speed would change. When a lower frame rate is used for monitoring, the image is blurred and the seedlings cannot be accurately identified, and the high frame rate monitoring would save it. More repetitive information affects the subsequent processing efficiency. In order to fully express the seedling data in the transplanting process and reduce the amount of calculation processing, only the key frames of the image were spliced. When the theoretical maximum forward speed of the tool and the camera’s viewing angle range are known, the calculation is performed at the maximum value of the tool speed, and the minimum frame rate is selected as shown in (1):

$$f_{min}=\frac{v}{H}$$

(1)

In the formula: $v$—maximum operating speed of the implement, m/s, $f_{min}$—minimum frame rate of image stitching, $H$—the distance from the plant to the direction of the camera’s field of view, m.

Image matching algorithm based on feature points was used for image splicing. The maximum theoretical operating speed of the rape blanket seedling transplanter is 1.2 m/s, and the distance between the plant spacing direction within the camera’s viewing angle is 0.5 m, and $f_{min}$ is obtained by Formula (1). The value is 2.4 because the same number of corner features of every two adjacent images needs to be greater than 4, and to ensure that the same number of corner features is greater than 4, the value of $f_{min}$ is 3. In this paper, a resolution of 640 × 480 SUM4 is used for image acquisition, the exposure time is set to 5 ms, and the maximum acquisition frame rate is 76 frames/s. Therefore, three key frame images are selected at uniform intervals in 1 s for image stitching.

3.3. Background Segmentation

In order to reduce the amount of calculation processing and improve the calculation accuracy and efficiency of the lodging and cavitation, the image background segmentation method was used to extract the seedlings from the background. First, we identified the canopy area of rape seedlings through the ultra-green grayscale method and binary extraction method, and then used morphological processing methods, such as corrosion expansion, to segment the rape seedlings and weeds in the treated area, complete edge detection, and mark the leaves. The full pixel edges of the seedlings are marked.

After adopting ultra-green grayscale, the stems of the seedlings cannot be displayed, and there are holes in the image of the seedlings. In order to eliminate the interference, the image is corroded based on the image morphology processing method. The circular structural elements are selected, and the image is corroded with a template with a radius of 2. Next, we eliminated the noise and then performed the closing operation to eliminate the holes in the seedling area and obtain a binarized image that can be edge-fitted. The processed image is shown in Figure 4a. We performed edge extraction on the morphologically processed image and superimposed the edge image with the original image. The effect is shown in Figure 4b. It can be seen from the figure that the position coverage of the seedling image reaches 100%, and the edge is fitted, and the processed image can satisfy the recognition calculation of the seedling state.

The image scanning method is adopted to traverse all the points of the image pixels line by line, and the parameters such as the coordinate position of the pixel centroid, the pixel area, the long axis, and the short axis are calculated. According to the basic geometric parameter characteristics of the rape blanket seedlings in Table 2, and according to the area filtering method, the minimum area less than the confidence interval is regarded as noise. In the binarized image, it is solved according to a single connected domain. For the case of multiple seedlings in one hole, the arithmetic average of the centroids of all leaves is calculated, as shown in Figure 4c.

3.4. Missing and Lodging Calculations

In the process of transplanting rape blanket seedlings, the judgment and calculation of the lodging and cavitation state of the seedlings is one of the difficult points. The planting position of the seedlings has the characteristics of row-wise arrangement, and the plant spacing and row spacing are known during the transplanting process, so the planting position is unique in theory. As shown in Figure 5, the plant spacing is S and the height of the lens from the ground is H.

The calculation point of the centroid of the seedling is called the actual planting position, and the actual planting point is marked as $\left[X_{ui}, Y_{ui}\right]$. The seedlings are placed on the planting crop line, and the points arranged according to the plant spacing on the top, which is called the theoretical planting position. The theoretical planting position is marked as $\left[X_{usj}, Y_{usj}\right]$ at point i; the ratio of the actual distance to the pixel distance is K.

(1)

Cavitation calculation

From Figure 4, the appearance of cavitation in the image means that no seedlings are detected when the theoretical planting position takes the seedling height as the radius, and this position is recorded as the position of lack of seedlings. According to the statistics of the seedling geometric size, the maximum height of the seedling is 105.141 mm, which is recorded as $L_R$, and the corresponding pixel point distance is recorded as l. The calculation formula is shown in (2):

$$l=\frac{L_R}{K}$$

(2)

In Formula (2), K is the calibration coefficient corresponding to the actual length of the pixel. The distance between the theoretical planting point and the actual planting point is recorded as ${\mathsf{\Delta }}_c$, and the calculation formula is shown in (3):

$${\Delta }_c=\sqrt{{(X_{ui}-X_{usi})}^2+{(Y_{ui}-Y_{usi})}^2}$$

(3)

Each theoretical planting position is traversed and calculated for all actual planting positions in turn, which can avoid the problem of missing counts due to the lack of seedlings of consecutive plants. When there is a point with ${\mathsf{\Delta }}_c$ ≤ $l$, it indicates that there are seedlings at the theoretical planting position, which is marked as qualified; when there is no ${\mathsf{\Delta }}_c$ ≤ $l$, it indicates that there is no seedling at the theoretical planting position, which is recorded as a cavitation.

• (2) Lodging calculation

Lodging in the field shows that the maximum width of the seedling leaves is greater than the maximum width of the rape blanket seedling when it is upright. In the image, it shows that the maximum distance between the actual planting position and the theoretical planting position on the leaf exceeds the maximum width of the leaf. On the basis of recording qualified seedlings, we calculated the long axis length b of the connected domain image, the distance between the centroid ($x$,$y$) and the planting point ($x_b$,$y_b$), and the half-pixel distance $L$ of the long axis length, as shown in (4):

$$L=\sqrt{{(x-x_b)}^2+{(y-y_b)}^2}+\frac{b}{2}$$

(4)

There is a correlation between the number of leaves and the number of single-hole seedlings. Within the corresponding pixel range of 45 mm, the number of single-hole seedling leaves more than 6 is recorded as two-hole seedlings falling towards each other. The lodging judgment formula is shown in (5):

$${\Delta }_l=L-\frac{45}{K}$$

(5)

When ${\mathsf{\Delta }}_l$ ≤ 0, it indicates that the planting position has no lodging, and when ${\mathsf{\Delta }}_l$ > 0, it indicates that the seedling at the planting position has lodging. As shown in Figure 5, we drew a circle with the seedling height and maximum width as the radius. When the distance from the theoretical planting position to the actual planting position is outside the range of the seedling height, it indicates that the position is cavitation. When the distance from the theoretical planting position to the actual planting position is within between the height of the seedling and the maximum width, it indicates that the seedling at this place is lodging. When the distance from the theoretical planting position to the actual planting position is less than the maximum width, it indicates that the seedling status at this position is qualified.

4. Test Results and Discussion

4.1. Field Test

A field test was conducted in Dafeng District, Yancheng City, Jiangsu Province at 4 pm in November 2020. The weather was cloudy and the light intensity was weak. The interface of the test prototype and the detection platform is shown in Figure 6. The test platform is a 2ZGZK-6 rape blanket seedling transplanter. The soil was rotated before transplanting and the background color was dark. The soil type was yellow-brown, light loam with a soil moisture content of 25.04%, the previous crop was rice, the straw was crushed and returned to the field in full, and the stubble height was 15 cm.

The rape variety was Ningza 1818 and the seedling age was 30 days. The forward speed of the machine was set between 0.8~1.0 m/s, the plant spacing of the transplanter was set to 16 cm, and each group of experiments was transplanted 8 times in the same direction, and the length of each walk was 12 m. The total number of cavitation and lodging were recorded.

4.2. Test Results and Analysis

For 8 trials, the designed cavitation and lodging online monitoring system was used to automatically identify and mark the transplanting cavitation and lodging of rape blanket seedlings, and the positions marked as lodging and cavitation were artificially referenced “Operation Quality for Rape Mat Seedling Transplanter” (NY/T 3887-2021) of the Ministry of Agriculture and Rural Affairs of the People’s Republic of China to judge the correctness of the state of the seedlings marked as cavitation and lodging. The recognition status data of the cavitation and lodging monitoring system is shown in

Table 3. Test data.

Number	Total Number of Holes	Number of Cavitation Recognized by the System	Number of Cavitation Identified Correctly	Absolute Error of Cavitation Recognition Accuracy Rate/%	Number of Lodging Recognized by the System	Number of Lodging Identified Correctly	Absolute Error of Lodging Recognition Accuracy Rate/(%)	Total Time/ (s)	Time-Consuming for Single-Frame Picture/(ms)
1	87	7	6	1.15	8	7	1.15	12.7	33.2
2	84	3	3	0	8	6	2.38	12.8	33.5
3	84	5	5	0	7	6	1.19	12.6	33.2
4	72	3	3	0	6	5	1.39	12.8	32.8
5	81	7	6	1.23	8	6	2.47	13.1	33.7
6	79	7	7	0	9	6	3.80	13.1	33.4
7	66	8	8	0	8	6	3.03	12.9	33.9
8	74	4	4	0	7	6	1.35	12.8	33.9

.

Since the actual transplanting operation process has a small number of cavitation and lodging holes, and the judgment of the number of qualified holes is also involved in judging the status of lodging and cavitation, the calculation of the recognition accuracy is based on the total number of transplanting holes. The absolute error of cavitation and lodging rate is used as the basis for judging the accuracy of system identification. The time consumed for a single-frame picture is obtained using the software system timing.

It can be seen from the experimental analysis results in Table 3 that the maximum absolute error of the accuracy of the cavitation recognition rate is 1.23%, and the maximum absolute error of the lodging recognition accuracy is 3.80%, while the average is 2.09%. In each test line, the system takes about 12 s on average from image acquisition, image processing, and stitching to recognize the cavitation and lodging state; processing a single frame image takes about 33 ms. Relative to the machine’s forward speed, the processing time-consuming of per meter images is about 1 s. The time-consuming match with the maximum forward speed of the machine matches 1.0 m/s, so it can meet the real-time processing requirements.

5. Conclusions

Aiming at the problem that the rape blanket seedling transplanter lacks cavitation and lodging real-time monitoring that affects the quality and efficiency of the transplanting operation, a new state judgment calculation method for seedling lodging and cavitation, based on image processing, and the comparison between the actual planting positions of the seedlings with the theoretical planting positions was proposed. According to the statistical data of seedling morphological characteristics in each hole of rape blanket seedlings, the seedling height and half of the seedling height are determined as the calculation criteria for cavitation and lodging, respectively. Aiming at the problem of image noise caused by large changes in the field environment, the effective area in the image is enhanced by CLAHE and bilateral filtering on the image, and the noise in the image is effectively removed. In order to reduce the processing time-consuming of the system, the key frame of the image is selected and the image stitching algorithm, based on corner detection, is obtained, and the parameter combination of image stitching that meets the requirements of seedling recognition accuracy and has less processing and calculation time-consuming is obtained.

The distance measuring sensor and the industrial camera are synchronized to work. Through the mutual real-time calibration of the pixels corresponding to the actual size, the problem of viewing angle changes caused by the ups and downs of the industrial camera during the transplanting process is solved, and the real-time judgment of the working condition of the machine is also realized. It provides a basis for realizing the statistics of seedling planting quality. The dynamic test showed that the maximum absolute error of the cavitation recognition rate was 1.23%, and the maximum absolute error of the lodging recognition rate was 3.80%. The total time consumed for 1 m image collected in real time from the collection, image processing and stitching, to seedling status recognition of rape blanket seedling field was about 1 s. The accuracy and system processing time of seedling cavitation and lodging state judgment can meet the accuracy and real-time requirements of cavitation and lodging state recognition for field transfer of rape blanket seedlings.