Design and Experiment of a Garlic Orientation and Orderly Conveying Device Based on Machine Vision

Abstract

Aiming at the problem that the existing garlic-root-cutting equipment requires the orientation and orderly conveying of garlic, a garlic orientation and orderly conveying method based on machine vision and mechanical orientation mechanisms was proposed and a garlic orientation and orderly conveying device was designed. To realize garlic posture recognition, garlic and garlic roots were used as detection objects, the YOLOv5s objection detection model was used to establish a garlic feature detection model, and the model was trained. The garlic orientation and orderly conveying device was developed and a garlic posture recognition test and garlic orientation and orderly conveying test were completed. The test results show that the success rate of posture recognition is 98.67% and the average accuracy rate of the garlic deviation angle is 99.11%. When the conveying speed is 95 $\mathrm{mm}\cdot {\mathrm{s}}^{-1}$ and the rotating speed is 55 rpm, the orientation success rate is 95.6% and the conveying efficiency reaches 75 garlic per minute, which meets the design requirements. The accuracy of the garlic posture recognition method and the rationality of the garlic orientation and orderly conveying device design are verified, and the automatic orientation and orderly conveying of garlic is realized.

1. Introduction

China is a major garlic producer, consumer, and exporter in the world. In recent years, to improve the overall economic benefits in the garlic industry, the garlic deep processing industry has been vigorously developed [1,2]. Multistep processing, including root cutting, breaking, peeling, cleaning, drying, and packaging, are commonly required when deep processing garlic products [3]. The garlic-root-cutting process is the most time-consuming process. It takes 3–4 s to complete the feeding and root-cutting process of a single garlic. Currently, the correct orientation and orderly feeding of garlic is realized through manual feeding, which is highly labor intensive and inefficient. To reduce labor costs and improve the automation level of garlic-root cutting, there is an urgent need to solve the problems of automatically orienting and feeding garlic in an orderly manner.

To date, many universities and research institutions have carried out research on the garlic-root-cutting mechanization, focusing on solving the technical problems of garlic-root-cutting machinery [4,5,6]. Aiming at the automatic orientation and conveying of garlic-root cutting, the existing equipment adopts a vibration method for feeding [7], which has a poor orientation effect on garlic without stems and requires further manual adjustment. In recent years, with the development of computer technology, the agricultural product orientation method, in which machine vision technology and mechanical mechanisms are combined, has provided new ideas for the orientation of agricultural products with complex shapes and unclear physical characteristics [8,9,10,11]. Shang et al. proposed an automatic orientation system for fructus aurantii based on machine vision, which recognizes the posture of fructus aurantii based on its image and realizes its orientation operation through rotating clamping shafts on both sides [12]. Li et al. proposed an automatic orientation method for corn seeds [13]. Based on machine vision technology, the orientation of the corn seeds was adjusted to an ideal position by an adjustment device according to the recognition results. Hou et al. proposed a garlic bulbil adjustment device based on bilateral image recognition [14]. The garlic seeds were picked by the seed spoon and then arranged horizontally in an orderly manner. Posture recognition was carried out according to the image characteristics of the head and tail of the garlic seeds. Finally, the orientation and conveying of the garlic seeds were realized by the orientation mechanism combined with the posture recognition results.

In this paper, aiming at the problems of the automatic orientation and orderly conveying of garlic, as required by existing garlic-root-cutting equipment, based on the analysis of garlic shape characteristics, a garlic orientation and orderly conveying method based on machine vision and mechanical structures is proposed.

2. Materials and Methods

2.1. Structure and Working Principle

According to the operation requirements of automatic and orderly picking, orienting and feeding garlic and aiming at the problems of missing garlic stems or lodging during storage and transportation, a garlic orientation and orderly conveying device for garlic root cutting was proposed. The required garlic posture for the garlic-root-cutting mechanism is shown in Figure 1. The working process of the machine is shown in Figure 2. Garlic is single granulated and initially transported through the feeding mechanism and then the posture recognition of garlic is carried out. According to the posture recognition results, garlic posture preadjustment is carried out under the action of the preadjustment mechanism, and finally, the garlic orientation operation is completed through the righting mechanism.

According to the garlic orientation and orderly conveying device shown in Figure 3, it is mainly composed of (1) a feeding mechanism, (2) a conveying mechanism, (3) a preadjustment mechanism, (4) a righting mechanism, and (5) an image acquisition module. The feeding mechanism is driven by the cylinder to achieve reciprocating motion and push the garlic in the hopper to the outlet of the feeding mechanism, and garlic separation is completed in the push process. Finally, under the action of gravity and the intervention rod, the garlic leaves the feeding mechanism and falls into the conveyor trough in the conveying mechanism to realize separation and preliminary orientation. After being processed by the feeding mechanism, the garlic is in an upright and lateral posture, as shown in Figure 4. This paper selects garlic from Shandong Province of China as the experimental object. The garlic is about 30 mm to 40 mm in height and 45 mm to 60 mm in diameter. Due to the growth law of garlic, there is a difference in the diameter of the same garlic. The maximum diameter of garlic shown in Figure 4 is 55 mm and the minimum diameter is 50 mm.

The garlic image was obtained by the image acquisition module and its posture was identified by the garlic posture recognition system [15,16,17]. The image acquisition module consists of industrial camera and industrial lens. HW500 industrial camera and GY0814-3MP lens are selected. The camera is installed directly above the preadjustment position, the installation height is 400 mm, and the focal length of the lens is 8 mm. The image collected by the image acquisition module is a garlic image with three channels and the size of the image is 300 × 300 dpi. The posture information includes the preadjusted posture direction and deviation angle $\theta$. According to the recognition results, if the garlic is in an upright posture, there is no need to adjust it; when the garlic is in a lateral posture, it is necessary to adjust its orientation through the preadjustment mechanism. The preadjustment mechanism is composed of a turntable, a flange coupling, and a preadjustment motor, as shown in Figure 5. The turntable is directly fixed on the output shaft of the motor through the coupling and the motor below the preadjustment mechanism is driven to rotate according to the deviation angle. At the same time, the preliminary adjustment of the garlic on the turntable is completed. After the preadjustment mechanism is completed, the garlic is in a lateral posture with a unified orientation, as shown in Figure 5a. The centralizer needs to adjust the garlic, which is in a lateral posture, to an upright posture to meet the requirements of the garlic-root-cutting operation.

The righting mechanism is composed of the righting motor, turnover plate, belt drive assembly, crank, and connection rod, as shown in Figure 6. The righting motor drives the crank to rotate through the belt drive assembly and drives the turnover plate to rotate through the connection rod. The working principle of the righting mechanism is based on the characteristics that the gravity center of garlic is close to the side of the garlic root. When the garlic enters the righting mechanism, its orientation will be adjusted under the action of gravity; that is, the stem will be reversed upward and the root will be reversed downward. After garlic turnover is completed, the garlic root will be close to the upper surface of the turnover plate and then the turnover plate will be moved from an open state to a closed state (the turnover plate and the conveyor line support plate are in the same plane) so that the garlic is in an upright posture and the orientation operation is realized.

2.2. Design of the Garlic Posture Recognition System

2.2.1. Principle of Garlic Posture Recognition

The garlic feeding mechanism can separate garlic (with or without stems) and preliminarily orientate it into two possible postures: an upright posture or a lateral posture. As shown in Figure 7, whether garlic has a stem, the images of garlic in an upright posture and in a lateral posture are obviously different. The garlic roots are not visible in the image of garlic in an upright posture. By judging whether there are garlic root features in garlic images, garlic can be distinguished as being in the upright or lateral posture. Furthermore, according to the growth characteristics of garlic, garlic roots grow on the bottom center of garlic and garlic orientation can be defined as the garlic center pointing to the garlic root center, as shown in Figure 7b. In the lateral image, garlic orientation can be determined by two key points: the garlic root center and garlic center. In Figure 7, Point A is the key point of the garlic center, and Point B is the key point of the garlic root center.

To realize garlic posture adjustment, the deviation angle $\theta$ should be obtained by the garlic posture recognition system. The coordinate system is established with the upper left corner of the garlic image as the origin, as shown in Figure 8. The deviation angle $\theta$ can be calculated by ${\theta }_1$, where ${\theta }_1$ is the angle between the vector AB and the negative direction of the y-axis and the value range of $\theta$ is (−180, 180). The calculation formula is as follows:

$$\theta =\left\{\begin{array}{ll}{\theta }_1\hfill & x_1>x_2\hfill \\ -{\theta }_1\hfill & x_1<x_2\hfill \\ 0\hfill & x_1=x_2 \mathrm{and} y_1\ge y_2\hfill \\ 180\hfill & x_1=x_2 \mathrm{and} y_1<y_2\hfill \end{array}\right.$$

(1)

where

$${\theta }_1=\arccos (\frac{y_1-y_2}{\sqrt{{(x_1-x_2)}^2+{(y_1-y_2)}^2}})$$

(2)

where $\left(x_1,y_1\right)$ are the coordinates of garlic center A and $\left(x_2,y_2\right)$ are the coordinates of garlic root center B.

According to the analysis above, the garlic posture recognition method based on garlic feature detection is proposed. The principle is to extract garlic and garlic root features in the image and, according to these two features, the orientation of garlic is judged based on whether the garlic is in an upright or a lateral posture. Figure 9 shows the flow chart of garlic posture recognition based on garlic feature detection.

2.2.2. Garlic Feature Detection Model Based on YOLOv5s

Based on the principle of garlic posture recognition, it is necessary to solve the detection problem of garlic and garlic root features. In the process of feature detection, garlic damage and occlusion of broken skin directly lead to deviation from the garlic feature center position. At the same time, to ensure the device has a faster recognition speed, the YOLOv5s model [18,19,20,21] is used in this paper.

(1)

YOLOv5s Detection Principle

The YOLOv5s network structure first extracts the features of garlic through the backbone network and divides the input garlic image into $N\times N$ nonoverlapping grid cells. The grid cell located in the target center is responsible for the detection of garlic features. Each grid cell needs to predict category information and the probability of garlic features in the current grid cell is $P_r$. Each grid cell generates B prediction boxes with different objectives and each prediction box contains five predicted values ($t_x,t_y,t_w,t_h$ and confidence). The center coordinates, width, and height of the anchor frame are as follows.

$$\left\{\begin{array}{l}{b}_x=\sigma (t_x)+c_x\\ b_y=\sigma (t_y)+c_y\\ b_w=p_w{e}^{t_w}\\ b_h=p_h{e}^{t_h}\end{array}\right.$$

(3)

where $c_x$ and $c_y$ are the horizontal and vertical coordinate values of the upper-left corner of the grid cell, respectively; $p_w$ and $p_h$ are the width and height values of the anchor box, respectively; $t_x,t_y,t_w$ and $t_h$ are the center coordinate values, width, and height values of the prediction frame; and $\sigma (t_x)$ is the Sigmoid function.

The confidence calculation formula of the prediction box is:

$$C_j^i=P_{r}IO{U}_p^t$$

(4)

where

$$IO{U}_p^t=\frac{PB\cap GT}{PB\cup GT}$$

(5)

where $C_j^i$ represents the confidence of the j prediction box in the i grid cell; $IO{U}_p^t$ represents the intersection ratio of the predicted frame and the ground-truth frame; $PB$ represents the prediction box; and $GT$ represents the manually annotated ground-truth box.

The loss function (total_loss) in the YOLOv5s model consists of three parts: the classification loss function (cls_loss), the bounding box loss function (box_loss), and the confidence loss function (obj_loss). BCE_Loss is used to calculate the class probability and the loss of the target confidence score and CIOU_Loss is used to calculate the bounding box loss. Finally, the candidate frame with the highest confidence is extracted from the candidate prediction frames by the non-maximum suppression algorithm to determine the information in the final target detection frame.

(2)

Model Training

The screened garlic from Shandong is taken as the research object. In practical applications, garlic damage and the occlusion of broken skin are the key factors that affect the robustness of the garlic feature detection model. Therefore, the images collected in this paper include images of garlic under normal conditions, images of garlic under different damage conditions, and images of garlic occluded by broken skin. In total, 505 images were collected and 2020 images were generated using rotation, brightness, saturation, and exposure changes for the garlic feature recognition dataset.

The garlic feature detection training environment in this paper is based on the Windows 10 operating system and the computer processor used for model training is Intel(R) Xeon(R) Sliver 4110 CPU@2.10 GHz (2 CPU). The running memory is 64 GB, the storage memory is 4 TB, the GPU is NVIDIA Quadro P2000, and the graphics card driver version is NVIDIA 471.41 and CUDA version 10.0.130. The construction and training of the garlic feature detection model based on YOLOv5s is completed under Python 3.6.8 and PyTorch. The maximum number of iterations (epochs) during model training is set to 300 and the batch size is set to 16. Figure 10 shows the training loss curve of the YOLOv5s model. As the number of iterations increases, the total loss of the model basically converges to a stable value.

2.3. Evaluation Indicators of Garlic Posture Recognition and Garlic Feeding Tests

To verify the effectiveness of the garlic posture recognition algorithm based on YOLOv5s garlic feature detection, the posture recognition success rate ${\eta }_1$ and the correctly determined deviation angle rate ${\eta }_2$ were selected as the evaluation indicators of garlic posture recognition algorithm and the calculation formulas are:

$${\eta }_1=\frac{C_{\mathrm{U}}+C_{\mathrm{D}}}{T_{\mathrm{U}}+T_{\mathrm{D}}}\times 100\%$$

(6)

$${\eta }_2=\frac{C_{\mathsf{\theta }}}{C_{\mathrm{D}}}\times 100\%$$

(7)

where $T_{\mathrm{U}}$ is the number of actual upright garlic, $T_{\mathrm{D}}$ is the number of actual lateral garlic, $R_{\mathrm{U}}$ is the number of recognized upright postures, $R_{\mathrm{D}}$ is the number of recognized lateral postures, $C_{\mathrm{U}}$ is the number of correctly recognized upright postures, $C_{\mathrm{D}}$ is the number of correctly recognized lateral postures, and $C_{\mathsf{\theta }}$ is the deviation angle of the lateral posture.

To verify the performance of garlic orientation and orderly conveying device, the success rate of orientation $\psi$ is taken as the evaluation indicator. The success rate of orientation $\psi$ is the ratio of garlic upright posture number N to garlic total number M.

$$\psi =\frac{N}{M}\times 100\%$$

(8)

3. Results and Discussion

3.1. Garlic Posture Recognition Test

To test the trained garlic feature detection model based on YOLOv5s, the average garlic feature detection time for each garlic image (including three channels) is approximately 0.018 s and the detection effect is shown in Figure 11.

The garlic posture recognition method is proposed based on YOLOv5s garlic feature detection. After determining the garlic features, the garlic orientation is determined and the garlic deviation angle can be calculated. The garlic posture recognition effect is shown in Figure 12. Since the garlic righting mechanism has a high fault tolerance rate, the acceptable error range for the deviation angle is determined to be $\pm 5°$.

To verify the effectiveness of the garlic posture recognition algorithm based on YOLOv5s garlic feature detection, a garlic posture recognition test was carried out and a comparison was made with the traditional garlic feature detection algorithm based on the HSV model. Taking garlic from Shandong as the object, in total, 100 images were collected, and 300 single-channel garlic images were segmented. The garlic posture recognition tests based on YOLOv5s and HSV models were carried out, respectively, and the posture recognition success rate ${\eta }_1$ and the correctly determined deviation angle rate ${\eta }_2$ were used as evaluation indicators. The statistical results of the garlic posture recognition test are shown in

Table 1. Statistical results of the garlic posture recognition test.

Method	Total Garlic Images	Actual		Recognized		Correctly Recognized			Success Rate (%)	Accuracy Rate (%)
		A	B	A	B	A	B	Angle of Deviation
YOLOv5s	300	184	116	188	112	184	112	111	98.67	99.11
Traditional method				132	168	110	94	63	68.0	67.02

. In Table 1, A is the upright posture and B is the lateral posture.

In the test, a condition exists where the garlic root is completely obstructed by the broken garlic skin, which leads to the recognition system misidentifying garlic with a lateral posture as that with an upright posture. The feature information of garlic broken skin is similar to that of garlic, which causes the inaccurate identification of garlic and leads to errors in the calculated garlic deviation angle. Garlic skin damage is the main cause of the above two conditions.

It can be seen from Table 1 that the success rate of the garlic posture recognition algorithm is 98.67% and the average accuracy rate of the garlic deviation angle is 99.11%, which is superior to the traditional algorithm in terms of both the success rate and accuracy rate of posture recognition. In summary, the proposed posture recognition algorithm based on YOLOv5s garlic feature detection has the advantages of a good recognition effect and high feasibility, which can meet the design requirements of the garlic posture adjustment mechanism.

3.2. Garlic Feeding Test

A prototype is developed based on the scheme of the garlic orientation and orderly conveying device and the garlic posture recognition system, as shown in Figure 13. The equipment is powered by a 220 V AC power supply and the external air pump provides a gas source for the feeding mechanism. The garlic posture recognition software developed in this paper is run on a Lenovo xiaoxin Pro 16 computer.

Taking garlic without fiber as the test object, an operation performance verification test for the garlic orientation and orderly conveying device was carried out. Taking the orientation success rate as the evaluation index and the feeding speed of the conveying mechanism and the rotational speed of the turntable as the influencing factors, the test of garlic orientation and orderly feeding was designed, as shown in

Table 2. Test results of the orientation and orderly conveying of garlic.

No.	Conveying Speed (mm·s−1)	Rotation Speed of Turntable (rpm)	Orientation Success Rate (%)
1	85	50	86.7
2	85	55	91.3
3	85	60	81.3
4	95	50	86
5	95	55	95.6
6	95	60	80
7	105	50	90.7
8	105	55	87.3
9	105	60	76.7

. The purpose is to optimize a set of machine operation parameters under the automatic operation mode, aiming at the high-quality conveying of garlic.

According to the test design, cold-stored dry garlic with a transverse diameter of 50–60 mm, treated by cutting fiber, was selected as the test material, and the garlic orientation and orderly conveying device were selected as the experimental object. The experiment was carried out in the Key Laboratory of Transplanting Equipment and Technology of Zhejiang. The test results are shown in Table 2. According to the test results, the success rate of orientation is 95.6% when the conveying speed is 95 $\mathrm{mm}\cdot {\mathrm{s}}^{-1}$ and the rotational speed of the turntable is 55 rpm. The orientation effect is shown in Figure 14.

To further verify the orientation and conveying efficiency of the garlic orientation and orderly conveying device in this paper, it is compared with the garlic automatic conveying device using vibration feeding, as shown in

Table 3. Comparison of garlic conveying efficiency.

Type	Conveying Efficiency (/min)	Single Channel Production Beat (s)
This paper	75	2.4
Double-channel automatic garlic shredder [22]	40	3.0
8-channel intelligent garlic automatic root cutting machine [23]	170	2.8

. Under the same number of channels, the device proposed in this paper has faster production rhythm and higher overall conveying efficiency. In addition, the existing garlic automatic feeding equipment only relies on the feature of garlic stems for orientation and the non-stem garlic needs manual adjustment or elimination, which further reflects the advantages of the garlic orientation and orderly conveying device.

Based on the above test results and comparison, the garlic orientation and orderly conveying device proposed in this paper can realize the automatic orientation and orderly conveying of garlic and can adapt to the feeding operation of garlic, with and without stems. The orientation success rate of this device is 95.6% and the conveying efficiency reaches 75 per minute, which meets the design requirements.

4. Conclusions

(1)

Taking the cold-stored dry garlic after cutting fiber as the object, an orientation and orderly conveying device for garlic root cutting was proposed. The automatic orientation and orderly conveying of garlic was realized by using the feature that the gravitational center of garlic is close to the garlic root and combined with machine vision technology. The problem of the poor automatic orientation effect of stemless garlic was solved. Compared with the existing vibration feeding methods, the conveying efficiency was significantly improved. At the same time, it provides a new idea for the orientation and conveying of bulb crops, such as garlic, and provides technical support for the development of garlic-related equipment.

(2)

A garlic posture recognition method based on garlic feature detection was proposed by using machine vision technology. The garlic feature was detected by the YOLOv5s model and garlic posture recognition was realized. The training results show that the YOLOv5s model can effectively learn garlic image features. The problem of garlic posture recognition in the garlic orientation process was solved. The success rate of posture recognition of the garlic posture recognition algorithm based on YOLOv5s is 98.67% and the average accuracy rate of the garlic deviation angle is 99.11%. Compared with the traditional HSV model, the success rate of posture recognition and the average accuracy rate of the garlic deviation angle are increased by 30.67% and 32.09%, respectively. It can be concluded that the garlic posture recognition algorithm based on YOLOv5s can effectively recognize the posture of garlic, which can meet the design requirements of the garlic preadjustment mechanism for garlic posture recognition.

(3)

A garlic orientation and orderly conveying device for garlic root cutting was developed, which realized the automatic orientation and orderly conveying of garlic and can adapt to the feeding operation of garlic, with and without stems. Through the garlic orientation and orderly conveying test, the optimal operating parameters of the whole machine were determined. The conveying speed is 95 $\mathrm{mm}\cdot {\mathrm{s}}^{-1}$ and the rotational speed of the turntable is 55 rpm. Under this set of parameters, the orientation success rate is 95.6%, the conveying efficiency is 75 per minute, and the single-channel production beat is 2.4 s. Compared with the double-channel automatic garlic shredder and the eight-channel intelligent garlic automatic root-cutting machine, mentioned in references [22,23], the single-channel production beat of this paper is improved by 0.6 s and 0.4 s, respectively, which proves that the conveying efficiency of the garlic orientation and orderly conveying device is high.