Research on Flexible End-Effectors with Humanoid Grasp Function for Small Spherical Fruit Picking

Abstract

The rapid, stable, and undamaged picking of small-sized spherical fruits are one of the key technologies to improve the level of intelligent picking robots and reduce grading operations. Cherry tomatoes were selected as the research object in this work. Picking strategies of two-stage “Holding-Rotating” and finger-end grasping were determined. The end-effector was designed to separate the fruit from the stalk based on the linear motion of the constraint part and the rotating gripper. This work first studied the human hand-grasping of cherry tomatoes and designed the fingers with sinusoidal characteristics. The mathematical model of a single finger of the gripper was established. The structural parameters of the gripper were determined to meet the requirements of the grabbing range from 0 to 61.6 mm. Based on the simulation model, the constraint part was set to 6 speeds, and the fruit sizes were set to 20 mm, 30 mm, and 40 mm, respectively. When the speed was 0.08m/s, the results showed that the grabbing time was 0.5381 s, 0.387 s, and 0.2761 s, respectively, and the maximum grabbing force was 0.9717 N, 3.5077 N, and 4.0003 N now of clamping, respectively. It met the picking requirements of high speed and low loss. The criterions of two-index stability and undamaged were proposed, including the grasping index of the fixed value and the slip detection of variance to mean ratio. Therefore, the control strategy and algorithm based on two-stage and two-index for rapid, stable, and non-destructive harvesting of small fruit were proposed. The results of the picking experiment for seventy-two cherry tomatoes showed that the picking success rate was 95.82%, the average picking time was 4.86 s, the picking damage rate was 2.90%, the browning rate was 2.90% in 72 h, and the wrinkling rate was 1.49% in 72 h, which can meet the actual small spherical fruit picking requirements. The research will provide an idea for the flexible end-effectors with humanoid grasp function and provides a theoretical reference for small spherical fruit picking.

1. Introduction

Due to the rich nutrients and unique flavor, the demand for cherry tomatoes is increasing [1,2,3]. However, harvesting is the most time-consuming and laborious part of its production. Therefore, realizing mechanization and automation is a significant development direction of fruit harvesting robots. However, the growing environment of cherry tomatoes is complex, and the skin is fragile. The quality of harvest directly affects the economic benefits. Its delicate and fragile biological characteristics make mechanized undamaged harvesting face great challenges [4,5,6,7].

In recent years, scholars had developed different fruit-picking end-effectors [8,9]. Xiong et al. [10] designed a strawberry end-effector with cutting function, and the success rate of picking reached 97.1%. Tested and compared three traditional methods of collecting mushrooms by hand, Huang et al. [11] developed a vacuum end-effector, and the results showed that the picking rate of the target mushroom reached 100% by the bending picking method. To reduce the scrapes of picking, Miao et al. [12] developed an apple end-effector with a compliant mechanism whose success rate is about 95.3%. It ensures the constant output force required for low destructive fruit clamping. Roshanianfard et al. [13] created a five-fingered pumpkin manipulator with a grabbing radius of 76.2-265 mm, which was electrically driven and had an internal shock-grabbing mode. In addition, the end-effectors of litchi [14], kiwi [15], chrysanthemum [16], blueberry [17], green pepper [18], cucumber [19], and other fruits and vegetables were designed and studied.

However, fruits and vegetables have different biological characteristics, sizes, and growth conditions. The design of specific end-effectors has gradually become one of the essential research directions based on their factors. According to the principle of bionics, Wang et al. [20] made the citrus picking end actuator with occlusion mode by imitating the snake head. At 46° as the optimum harvest posture, the harvest rate reached 74%. By studying the grasping behavior of human hands during tomato picking, Wang et al. [21] developed a new 16-channel data acquisition electronic glove, which showed the maximum pressure contribution from the distal regions of the thumb, middle finger, and index finger by picking each fruit. Gao et al. [22] developed a pneumatic-controlled end-effector. It could pick cherry tomatoes continuously and steadily. The average time of picking single cherry tomato was 6.4 s, and the highest success rate was 84%. In summary, the design of an end-effector according to the characteristics of fruits and vegetables can improve the success rate of picking and reduce damage [23,24,25,26].

In this work, fingers with sinusoidal contours were designed, and the picking strategy of the two-stage “Holding-Rotating” picking strategy of finger-end grasping was determined. The end-effector was designed to separate the fruit from the stalk based on the linear motion of the constraint part and the rotating gripper. The clamping regulating system based on clamping pressure feedback was built to realize undamaged picking. The double threshold picking index was proposed, which included the grasping index of the fixed value and the slip detection of variance to mean ratio. The performance of the end effector was verified by cherry tomatoes picking test.

2. Design of End-Effector System

2.1. Design of the Gripper Inspired by Human Hand Grasping

This work compared the grasping experiments of different size spherical fruits by hand. When picking larger fruit, such as apples and oranges, each finger joint rotated to make the palm close to the fruit, realizing the enveloping grasping. In contrast, when picking small fruit, such as cherry tomatoes and plums, only the ends of thumb, index finger, and middle finger contact with the fruit. The rotation angle of the metacarpophalangeal joint was the largest. The distal and proximal phalangeal joints turned slightly. Fingers formed a more fixed arc shape in the grasping process. The comparison of fruit grasping in different volumes shows in Figure 1.

Therefore, inspired by the dynamic harvesting of small fruits by human hands, the gripper was designed, which has three features, including sinusoidal contour, single-joint rotation, and finger-end grasping. The equation of the sinusoidal curve refers to the outer shape, and its shape is shown in Figure 2a. The equation of the sinusoidal curve of the single finger is Equation (1):

$$y=12\mathit{sin}\left[\frac{1}{30}\left(x - 20\pi \right)\right]+6\sqrt{3}, x\in \left[0,45\pi \right]$$

(1)

The design of gripper is shown in Figure 2. Three fingers were in each of the three holes of the constraint part. The fingers were fixed to the chassis by the “L-shaped” steel parts. The steel parts can produce elastic deformation. The fingers close when the constraint part advances. When the constraint part retreats, the fingers gradually recover to their initial positions. Due to the sine curve is smooth, continuous, and differentiable, the constraint part moves smoothly in contact with the fingers. On the one hand, it is intended to transform the finger opening and closing into the constraint behavior, which realizes the stable movement and precise control of the fingers. On the other hand, the problems of grasping caused by rotational speed, torque, and precision of joint motor are avoided, to a certain extent.

2.2. Design and Working Principle of End-Effector

The end-effector designed in this paper was used to harvest small spherical fruits. The picking strategy of “Holding-Rotation” was used to realize picking. It was mounted on a picking robot based on binocular stereo vision. The picking robot guided the end-effector to aim at the fruit employing binocular vision. The gripper closed, grabbed the fruit, then rotated with the fruit to separate the fruit from the stalk (Figure 3).

During the holding stage, the gripper opens and closes to achieve fruit gripping. Its essence is to restrain the movement of fingers by linear movement of constraint part. Therefore, the end-effector rotates the screw through a stepper motor, driving the pole and the slider in reciprocating linear motion along the Slide track. Rotating stage. The DC motor drives the gear to turn after grasping the fruit. Then, the gripper and the fruit are rotated synchronously, separating the fruit from the stalk. The pressure sensors are FSR-402 sensors, which can obtain the gripping force in real time, and the measuring range is 0.1~100 N. The stepper motor is two-phase four-wire 42-stepper motor (42BYGH24). The torque is 0.13 N, and the step angle is 1.8°. The stepper motor drive model is DM542C. The DC motor is the planetary DC decelerating motor with the working voltage of 24 V and a rotating speed of 300 r/min.

The picking time was taken as a technical index to realize the quick picking in this work. It was set to 1.5 s, which means from the beginning of the clamping to the separation of the stalks, including 1 s for the clamping phase and 0.5 s for the rotation phase.

3. Simulation and Analysis of End-Effector Motion

3.1. Establishment of the Single-Finger Kinematics Model and Setting of Key Component Parameters

According to the size statistics of cherry tomatoes planted in the standardized greenhouse, their diameters are about 20~40 mm. The kinematics model of the gripper was established first to meet the size requirements, then the parameters of relative parts were determined. Due to the gripper consists of three symmetrical fingers, the single finger was modeled and analyzed. Figure 4 shows the schematic representation of the structural parameters of gripper’s the single finger. The rectangular coordinate systems xoy and XOY were created. The center of the chassis was set to the origin of XOY. The connection between the fingers and the chassis were set to the origin of XOY. The axes were established in the direction of pole motion and perpendicular to it, respectively.

According to the grasping characteristics, the opening and closing of the fingers could be regarded as the rotation around O. BB’ was the circular arc with O as the center and R as the radius. Q and Q’ were set to random points on the OB and OB’, respectively. The homogeneous coordinates of Q in the xoy coordinate system is:

$$Q_{\mathit{xoy}}=\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]{=T\times R\times Q}_{\mathit{XOY}}$$

(2)

where${ Q}_{\mathit{XOY}}=\left[\begin{array}{c}X\\ Y\\ 1\end{array}\right]$ is the homogeneous coordinate of Q in XOY; $R=\left[\begin{array}{ccc}\cos \theta & \sin \theta & 0\\ -\sin \theta & \cos \theta & 0\\ 0& 0& 1\end{array}\right]$ is the rotation matrix;$T=\left[\begin{array}{ccc}1& 0& 0\\ 0& 1& h_0\\ 0& 0& 1\end{array}\right]$ is the translation matrix of the position of the XOY relative to XOY. After rotating $\theta$ clockwise, the equation for OB’ under XOY is:

$$X\sin \theta +Y\cos \theta =12\sin \left[\frac{1}{30}\left(X\cos \theta -Y\sin \theta -20\pi \right)\right]+6\sqrt{3}, \theta \in [0,45\pi \cos \theta ]$$

(3)

The coordinate of the Q’ on the OB’ in XOY is:

$$Q_{\mathit{xoy}}^{\prime }=\left[\begin{array}{c}{x}_{Q^{\prime }}\\ y_{Q^{\prime }}\end{array}\right]=\left[\begin{array}{c}X\cos \theta +Y\sin \theta \\ h-X\sin \theta +Y\cos \theta \end{array}\right]$$

(4)

When the movement of the constraint part caused the finger to rotate θ, X_A was always on the x-axis and y = 0. The movement distance of constraint part X_A was set to L. From Equation (3), the relationship between θ and L can be expressed as a complex nonlinear equation, which in MATLAB can be described by the solve function as:

$$f(\theta ,m)=L\sin \theta -12\sin \left[\frac{1}{30}\left(L\cos \theta -20\pi \right)\right]-6\sqrt{3}, \theta \in \left[0, 45\pi \cos \theta \right]$$

(5)

The n, the open range of the finger end, can be expressed as:

$$\left\{\begin{array}{c}{n=2y}_{B^{\prime }}\\ y_{B^{\prime }}{=h-X}_B{\sin \theta +Y}_B\cos \theta \end{array}\right.$$

(6)

Figure 5 was inferred from Equations (5) and (6). With the increase in L, θ and n changed synchronously, and showed a negative correlation. As the constrain part advanced, the finger’s movement could be divided into three stages. Stage one was the constraint part did not touch the finger, and θ and n did not change. Stage two was a gripper closing stage. As the constraint part advanced to the L_C, it touched the finger and caused it to start closing. Then, L increased, θ increased, and n decreased. When the constraint reaches L_D, θ and n reached the maximum and minimum, respectively. Stage three was gripper opening stage. As the constraint part pass through L_D, θ decreased and n increased.

Therefore, stage one can be used as the stopping region of constraint the part in the natural state. The constraint part does not encounter fingers, so that the parts can be well protected. In stage two, the gripper end had the maximum and minimum gripping range. This stage can be used as the movement region of the constraint part in the grasping process. Stage three cannot be used as the region of movement of the constraint part.

Therefore, stage two was discussed to adapt the cherry tomato sizes. From Equation (1), when y = 0, $L_C=31.4 \mathrm{mm}$, ${\theta }_{\mathit{min}}=0°$, and $n_{\mathit{max}}=2h+20.78 \mathrm{mm}$. Equation (5) was calculated by the chain rule as:

$$z=\frac{\mathrm{d}\theta }{\mathrm{d}L}=-\frac{{{\mathrm{F}}^{\prime }}_L}{{{\mathrm{F}}^{\prime }}_{\theta }}$$

(7)

Let z = 0, and L can be calculated as:

$$L=\frac{30}{\mathit{cos}\theta }\left(\frac{2\pi }{3}+\mathit{arccos}\frac{5\mathit{sin}\theta }{2\mathit{cos}\theta }\right)$$

(8)

When the above formula was associated with$f\left(\theta ,m\right)=0$, $L_D$= 94.64 mm, ${\theta }_{\mathit{max}}=12.45°$, and $n_{\mathit{min}}=2h-40.64$mm. Thus, the range of n was $2h-40.64$mm to $2h+20.78$ mm. The open range of the finger end was only related to h. Therefore, when h was determined to be 20.4 mm, the open range of the gripper end was about 0~61.6 mm. It can meet the sizes requirement of cherry tomato picking.

According to the above analysis, the length of a single finger was 140 mm. As the fingers were printed in 3D, the thicknesses were set to 5 mm, and the widths were set to 10mm for grasping reliability. The length of the pole should be greater than 94.64 mm because the active regions of the constraint part were stage one and two. Therefore, the length of the pole was set to 120 mm because it was to be fixed to the slider. The length of screw was set to 150 mm for security during system operation. The pitch was 10 mm. The interior radius of the constraint part was set to 21 mm to ensure that the finger reaches the above grabbing range, which is slightly larger than h. Outer radius was set to 25 mm. The Radius of the Chassis was set to 50 mm to mesh with the gears of the DC motor. The width of steel shrapnel is the same as the width of the finger. The parameters of key parts of the end-effector were determined according to h. (

Table 1. The table of the key parts parameters.

Parts	Parameters	Value/mm
Pole	Length	120
Fingers	Length	140
	Thickness	5
	Width	10
Chassis	Radius	50
Constraint part	Interior radius	21
	Outer radius	25
Screw	Length	150
	Pitch	10
Steel shrapnel	Width	10

).

3.2. Design and Analysis of Finger End

The stability of fruit grasping is one of the key techniques for fast and undamaged fruit picking. Additionally, the finger end is the only part in contact with the fruit. It is a crucial factor in achieving a stable grasp.

3.2.1. Design of the Finger End

The human finger pulp has the biological characteristics of a thicker cuticle and subcutaneous fat pad. In grasping small fruits, the last finger pulps gradually form the concave geometric surface that are fitted to the fruit surface. Hand-picking realizes the grasping characteristics of large contact areas, strong envelopment, and flexible contact. Therefore, the finger ends were designed as the geometric surface close to the spherical surface of the cherry tomato. The parameters of the geometric surface are shown in Equation (7) (unit: mm). The model is shown in Figure 6. Its surface was covered with the 3 mm soft material called Dragon Skin 10 for flexible contact.

$$\left\{\begin{array}{c}\begin{array}{c}{x}^2{+y}^2{+z}^2=40\\ -5 \le x \le 5\end{array}\\ \begin{array}{c}y > 0\\ -10 \le z \le 10\end{array}\end{array}\right.$$

(9)

3.2.2. Feasibility Analysis of Grasping

The gripper is rotated synchronously with the fruit to separate the fruits from the stalk. During this period, the fingers contact the target fruit, and the two objects do not allow to slide against each other to achieve stable grasping. Thus, the whole system [27,28] is in equilibrium. The following relationships of mechanical and dynamic need to be satisfied:

$$\left\{\begin{array}{c}\mathit{Gf}=\omega \\ J^{T}f=\tau \end{array}\right.$$

(10)

$$\left\{\begin{array}{c}{G}^{T}u=\dot{x}\\ J \dot{q}=\dot{x}\end{array}\right.$$

(11)

where $\omega$ is the external force vector; $\tau$ is the joint torque vector; f is the total contact vector acting on the fruit; $u$ is the velocity vector; $\dot{q}$ is the joint velocity vector; $\dot{x}$ is the contact velocity vector; $J$ is the gripper Jacobian matrix and determinant; and $G$ is the gripping matrix of the gripper.

Therefore, if the object is required to be constrained entirely, then the relative acceleration vector in the contact force space must be zero. The finger ends and the target fruit are relatively static. For a given $\tau$, the above-mentioned feasible velocity must be zero in all directions of binding force action. For this reason, for a given system state and external force, the amplitude $\lambda$ of normal contact force should be greater than 0 (${\lambda }_C>0$) to satisfy the linear complementary condition of contact constraint. ${\lambda }_C$ includes normal contact force $F_N$ and friction force $F_f$. It must satisfy the constraint condition $F_f \le { \mu F}_N$. (Figure 7).

So, at the contact point $C_i$, the contact force $F_{C_i}={\left[\begin{array}{cc}{F}_{C_i,n}& F_{C_i,t}\end{array}\right]}^T$ satisfies the constraint condition:

$$F_{C_i,t}=\sqrt{{\lambda }_{{i,t}_1}^2{+\lambda }_{{i,t}_2}^2 }\le { \mu F}_{C_i,n}$$

(12)

where ${\lambda }_{{i,t}_1}$ and ${\lambda }_{{i,t}_2}$ are special solution vectors of ${\lambda }_C$. They correspond to the magnitude and curl of the friction at $C_i$. ${\lambda }_i$ is the amplitude component of the normal contact force at $C_i$. Let $f_i{ \times \lambda }_i{=F}_{C_i,n}$, and the constant factor $f_i \ge \sqrt{{\lambda }_{{i,t}_1}^2{+\lambda }_{{i,t}_2}^2}{/\mu \lambda }_i$. For ${\lambda =\lambda }_C{+a\lambda }_0$, all normal contact force amplitudes ${\lambda }_i > 0$ were guaranteed. The dynamic model of grasping constraint can be satisfied by the feasible solution of contact force. a is a scalar quantity.

3.3. Simulation and Analysis of the Gripper

ADMAS 2018 software was used to carry out an analysis of motion simulation, exploring the movement characteristics of the gripper. It would provide the theoretical basis for realizing fast and undamaged control algorithms. The end-effector clamping principle is that constrain part controls the opening and closing of three fingers. In the simulation, only the relevant parts of the gripper were studied. Cherry tomatoes are non-standard spheres. So, the ball was used instead of actual fruit in the simulation model. The physical properties of cherry tomatoes were taken as the parameters of the ball. The fruit-holding process was idealized in simulation (Figure 8).

First, the ball was deactivated to verify the rationality of the gripper design. The constraint part moved from the chassis to the end of the mechanical finger. Figure 9 shows how the open range of the finger ends varies with the distance of the constrain part. The open range of the finger ends were from 72.8 mm to 9.8 mm, respectively, which met the actual grasping demand of cherry tomato sizes.

The ball diameters were set to three sizes, 20 mm, 30 mm, and 40 mm, respectively, to explore the motion parameters of the constraint part. The balls were activated in the simulation. The distance was 24 mm from the constraint part to the chassis, and it was set to the beginning position. The constraint part and fingers did not touch each other under this parameter. The simulation results are shown in

Table 2. Parameters of constraint part movement on holding cherry tomatoes.

r/mm	L1/mm	L2/mm	ΔL/mm
20	24	66.9	42.9
30		54.9	30.9
40		46.2	22.2

Note: r is the fruit size; L₁ is the beginning distance of the constraint part from the chassis; L₂ is the distance from the constraint part to the chassis when the finger ends touch the fruit; ΔL is the moving distance.

. When the finger ends connected the ball, the distances between the constraint part and the chassis were 66.9 mm, 54.9 mm, and 46.2 mm, respectively. The moving distances of constraint part were 22.2 mm, 30.9 mm, and 42.9 mm, respectively. It is shown that the movement distance of the constraint part was related to the diameter of the ball when gripping. The larger the fruit sizes, the smaller the movement distance. There was a negative correlation between fruit sizes and the movement distances of the constraint part.

The target required a holding time of 1s. In this paper, the maximum moving distance ΔL was set to 50 mm, and 50 mm/s was the minimum velocity of the constraint part. Based on this, for the convenience of calculation and electrical control, this section discussed the holding force of the finger ends on the three fruit sizes at six speeds. The speeds were 60 mm/s, 70 mm/s, 80 mm/s, 90 mm/s, and 100 mm/s, respectively.

From the mechanical structure of the gripper, it is easy to know that the gripping force of the finger ends on the fruit increases continuously from the touching to the gradual clamping process. Therefore, this work only needs to explore the maximum gripping force amplitude of contact moment between the fingers and the fruit. The simulation step of contact moment was adjusted to 10,000. The definition of the maximum gripping force amplitude of contact moment is:

$$F_s=\mathit{max}\left[F_{\Delta t}\right]$$

(13)

where Δt is the time of contact moment, Δt = 0.001 s; $F_{\Delta t}$ is the gripping force of contact moment, N; $\left[F_{\Delta t}\right]$ is the gripping force set of contact moment;$max\left[F_{\Delta t}\right]$ is the maximum gripping force amplitude of contact moment. The simulation results are shown in Figure 10 and

Table 3. Results of motivation simulation.

Speed /mm·s−1	Time/s			Fs/n
	20 mm	30 mm	40 mm	20 mm	30 mm	40 mm
50	0.8826	0.6314	0.4490	0.2074	0.9792	1.4509
60	0.7320	0.5190	0.3724	0.5230	1.3712	1.9064
70	0.6256	0.4435	0.3169	0.7816	2.4866	2.9877
80	0.5381	0.3872	0.2761	0.9717	3.5077	4.0003
90	0.4755	0.3432	0.2449	1.9838	4.6872	6.3619
100	0.4271	0.3079	0.2200	3.5447	7.5650	10.1627

.

At the same speed, the holding time increased with the decrease in fruit size. Due to gripping small-size fruit, the moving distance of constraint part was long. Under the same fruit size, the larger the speed of constraint part was, the shorter the holding time was. The holding time of three kinds of cherry tomatoes was less than 1 s, which reached the expected target. At the same speed, $F_s$ increased with fruit sizes. The minimum values of $F_s$ were 0.2074 N, 0.9792 N, and 1.4509 N at the speed of 50 mm/s, respectively. At the same fruit size, $F_s$ increased with the speed of the constraint part. When the constraint part speed was 0.1 m/s, $F_s$ reached 10.1627 N [22,29] with 40 mm, which produced a risk of damaging the cherry tomatoes.

The indicators of time and gripping force should be considered to ensure a fast and undamaged grab. When the speed of the restrained part is 0.08 m/s, the comprehensive simulation results show that the clamping time were 0.5381 s, 0.3872 s, and 0.2761 s, respectively, and the maximum gripping force were 0.9717 N, 3.5077 N, and 4.0003 N, respectively, for three fruit sizes. This speed meets the requirement of fast and undamaged when picking. Therefore, 0.08 m/s can be used to the actual speed of the constrain part, then the program and algorithm can be designed.

4. Control Method of Undamaged Picking

Due to the complex growing environment, size, and different postures, the values of piking force cannot be determined in the actual picking process. The movement distance and speed of the constrain part cannot be set as the only condition for picking control. If the clamping force is too small, the finger and the fruit will slide [30,31,32,33,34], and the picking will fail. Too much clamping force will damage the fruit. Thus, the feedback signals should be added. It is the best choice to make the manipulator and fruit synchronous rotation with the smallest clamping force. It is a critical state where no slip just happens. This strategy can be used as the control target of undamaged picking.

Therefore, the experiment designed the two-stage “Holding-Rotating” picking strategy. The clamping stage is the key to realizing undamaged picking, and the clamping force directly affects the degree of fruit damage. The rotation stage is another key to achieving the separation of the fruit handle, and it is the stage where the fruit slides relative to the manipulator. It is an effective way to design the undamaged picking algorithm to explore the minimum clamping force and slip criterion through the clamping experiment.

4.1. Experiment and Analysis of Clamping Fruit

The pressure on the sensor’s surfaces changes as the gripper grasps and rotates the fruits. Its output signals will also vary according to working principle. Therefore, this experiment mainly collected the corresponding time signal changes of sensor outputs to explore the undamaged grab strategy in the picking process. The precise mathematical relationship between the grasping force and the output of the sensor’s electrical signal had not been established.

Figure 11 is the experiment of grasping and slipping. Fixing cherry tomato, constrain part at 0.08m/s advanced to close fingers and clamped fruit. Then, the gripper was rotated through the DC motor, producing a slide between it and the fruit. Three pressure sensors were calibrated first, and their output values were collected in real-time. The sampling frequency of sensors was set to 0.2 kHz. The aim was to explore the law of grasping and find the basis for relative slip. The Dragon Skin 10 with 3 mm was covered between the finger ends and the sensors.

Figure 12 shows the output values of sensors during the experiment. The changes in sensor outputs had the following five stages according to the results. (Ⅰ) The fingers closed with the constrain part motivation. However, the finger ends did not touch the fruit. The output values were zero. (Ⅱ) The constraint part moved, and the ends grasped the fruit. The output values increased with the increase in the clamping force. (Ⅲ) When the constraint part stopped moving, the output values were maintained at a relatively stable value. (Ⅳ) The slip happened when the gripper was rotated. The output values fluctuated greatly. (Ⅴ) When the gripper stopped, the slip disappeared. The output values were maintained at a relatively stable value.

The output values of the three sensors were inconsistent according to the above results. The reason is the asymmetry on the cherry tomato surface. In addition, fingers clamped the fruit in different positions. However, the trends of output values were the same during the picking process. Obviously, in combination with the above picking strategy, it was a key that minimum gripping force was set during III, and it was another key that slip was judged during IV. The other three stages need not be judged. Therefore, the above two keys were studied to be critical for achieving undamaged and stable picking based on the sensor output values.

The greater the pressure on the sensor surface, the smaller its resistance, and the greater its output values. Therefore, the output values were set during III. In addition, the output values of the three sensors were different under stable clamping. Therefore, when the output of any sensor is greater than the minimum clamping force, it is judged to have reached the minimum clamping force condition. During IV, the values of the three sensors did not have the specific change rule, which cannot judge slip directly. Still, their fluctuation was obvious, and the numerical discreteness degree became higher. Therefore, this work introduced five statistical statistics to find the judgment basis of slip. They were adjacent difference (AD), average (A), average deviation (AD), variance (DX), and standard deviation (SD), respectively. They are shown in Equations (14)–(18).

$$\mathit{AD}=x\left(i+1\right) - x\left(i\right)$$

(14)

$$A=\frac{{{\displaystyle \sum }}_{i=1}^{n}x\left(i\right)}{i}$$

(15)

$$\mathit{AD}=\frac{{{\displaystyle \sum }}_{i=1}^n\left|x\left(i\right)\right.-\left.A\right|}{i}$$

(16)

$$\mathit{DX}=\frac{{{\displaystyle \sum }}_{i=1}^n{\left(x\right(i)-A)}^2}{i}$$

(17)

$$\mathit{SD}=\sqrt{\mathit{DX}}$$

(18)

where i represents the sampling frequency; x(i) represents the output value; n represents the total amount of data collected since III. The sensor output values were analyzed according to the above formulas in stages III and IV. The results are shown in Figure 13.

It can be seen from Figure 13a that the difference in adjacent data fluctuates greatly. When a little external interference is encountered in the picking, the adjacent data difference will increase, resulting in wrong judgments. Compared with the smooth trend of the average in Figure 13b, the average deviation, the variance, and the standard deviation had upward inflection points, as shown in Figure 13c–e. However, these three statistics occurred in small fluctuations before the 30th sampling. The reason is that the original output values of the sensor fluctuated instantly during the III to IV. The trend of these three statistics was first up, then down in the 30th–210th sampling of sensor three. The reason is that the pressure sensor three output values appeared to float in the early stage. These three statistics happened small range rise at the 290th–310th sampling of pressure sensor one, corresponding to the frequency of the original value that appeared a small hill. The reasons might be due to the irregular shape of cherry tomatoes or the small gripping force. The inevitable slight fluctuation affected the mean deviation, variance, and standard deviation, and it also affected the correct judgment of slip.

To reduce the influence of unavoidable slight fluctuations of data and realize more reliable slip detection, this study proposed the “Variance-to-Mean Ratio (VMR)” as the slip criterion, and it is shown in Formula (19). Figure 13f is the result of the variance-to-mean ratio.

$$\mathit{VMR}=\frac{\mathit{SD}}{A}$$

(19)

The variance-to-mean ratio remained near 0 and did not fluctuate significantly during phase III. It eliminates the judgment that values fluctuate due to fluctuations in the raw data. In addition, it also had significant inflection points to judge slide during Ⅴ. The reason is that averages simply flattened out small data fluctuations but did not change the trend of the standard deviation. Therefore, the variance-to-mean ratio can be used as the sliding criterion. However, the corresponding samplings of the three sensors were different when the rising inflection points occurred. For example, sensor one changed first, followed by the other two. It shows that the three fingers slide for the cherry tomato at different times during rotation. The first appeared inflection point should be used as the judgment of the slip criterion.

4.2. Design of Algorithm for Undamaged Harvesting

The two-stage “Holding-Rotating” picking strategy was used to achieve fast and undamaged picking. During the holding stage, when the gripper clamps the fruit with the minimum gripping force, the constrain part stops. $\alpha$ was set to the value of the sensor output corresponding to the minimum holding force. During the rotation stage, the gripper is rotated to separate the fruit from the stalk. In case of a slip, the constraint part advances, the fingers are closed, and the clamping force is increased to prevent sliding. The rotation of the gripper is realized by the DC motor. The running time of the DC motor was set to $t_M$. The rising increment was set to $\gamma$, which was used as the direct judgment indicator of slip. It is shown in Equation (20). $\rho$ was set to the number of incremental successive occurrences.

$$\gamma =\mathit{VMR}\left(i+1\right)- \mathit{VMR}\left(i\right)$$

(20)

When γ occurs $\rho$ times, it is judged to have produced slip. Let the forward distance of the constraint part be l when slip occurs. Based on multi-group clamping test, when $\alpha$= 30, γ = 0.1, $\rho$= 3, $t_M$= 0.5 s, and l = 0.5 mm, it had a better judging and gripping effect. Therefore, according to the two-index judgment, the grabbing index$\alpha$ and the slide index $\gamma$, the control algorithm of the two-stage and two-index was designed for fast, stable, and undamaged harvesting of small fruits. The flow chart of control algorithm is shown in Figure 14.

5. Experiment and Analysis of Picking

5.1. Material and Method

The picking experiment was conducted on 6 July 2022. The end-effector was mounted at the end of the arm based on the cartesian coordinate system. The maximum axial velocity at the manipulator’s end is set to 0.8 m/s. Guided by ZED 2 (STEREOLABS, England) binocular stereo camera, the end-effector was sent to the position where it could pick the target fruit. Then, the end-effector worked and picked the fruit. In the harvesting experiment, the Arduino Due controller was used to realize the two-stage and two-index harvesting strategy.

A total of 72 cherry tomatoes were tested to verify the harvesting performance of the end-effector. There are two damage types to cherry tomatoes. (1) The skin of the cherry tomato is damaged, and the internal tissues are exposed. Then, the fruit develops mold or decay. (2) The cherry tomatoes skin is not damaged, but its cells are damaged. In this case, an enzymatic reaction occurs. The color of the pressed area darkens. When the tomato cells breathe faster, the tomatoes’ water will be lost, leading to atrophy. Thus, picking success rate, the average picking time, the picking damage rate, the browning rate, and the wrinkling rate were used as measures of rapid and undamaged picking. The definitions are as follows:

$$T=\frac{{{\displaystyle \sum }}_1^h{T}_0{ - T}_1}{h}$$

(21)

$$S=\frac{r}{h} \times 100\%$$

(22)

$$D=\frac{f}{h}\times 100\%$$

(23)

$$B_t=\frac{p}{r-f}\times 100\%$$

(24)

$$C_t=\frac{q}{r-f}\times 100\%$$

(25)

where T is the average time of picking, s; $T_0$ is the beginning time of picking, s; T is the time of unloading, s; S is the success rate,%; r is the number of fruits picked successfully; h is the total number of fruits; D is the damage rate, %, the damaged fruit was clipped by the end-effector; f is the damaged number; $B_d$ is the browning rate, %; p is the browning number; $C_d$ was the wrinkling rate, %; q was the wrinkling number; and t is the hours after picking, t = 24 h, 48 h, 72 h, respectively. The picked tomatoes were stored at an average temperature of 28 °C. Browning and wrinkling tomatoes were recorded. The picking experiment is shown in Figure 15.

5.2. Analysis of Picking Results

The statistical results of the picking test are shown in

Table 4. Results of picking experiment.

Item		Number	Result/%
Success		69	95.83
Damaged		2	2.90
Browning	24 h	0	0
	48 h	1	1.49
	72 h	2	2.90
Wrinkling	24 h	0	0
	48 h	0	0
	72 h	1	1.49

. The results of the picking experiment for 72 cherry tomatoes showed that the average picking time was 4.86 s, the picking success rate was 95.82%, and the picking damage rate was 2.90%. The damage was mainly caused by two factors. (1) The edges of the fringe ends were in contact with the tomato surface. The pressure feedback signals were inaccurate or missing. Judgment was an error. (2) Under the influence of the stalk, the branches, and the leaves, the ends clipped them. The finger did not fully attach to the fruit surface. The gripping force is too large during the rotation.

The browning rates at 24 h, 48 h, and 72 h were 0%, 1.49% and 2.90%, respectively. The wrinkling rates at 24 h, 48 h, and 72 h were 0%, 0%, and 1.49%, respectively. The risk of browning and wrinkling increased with time after picking. Of course, browning and wrinkling were not only related to the gripping force, but also the ripeness of tomato fruit. In this work, the mature tomatoes were tested, and the effects of different maturity were not studied. Different samples of damaged cherry tomatoes are shown in Figure 16.

Therefore, based on the success rate, picking time, damage rate, browning rate, and wrinkle rate, the end-effector developed in this paper had a good fast and undamaged clamping effect, and excellent picking performance.

6. Conclusions

(1)

The two-stage “Holding-Rotating” picking strategy of the finger ends grasping was determined. The end-effector was designed to separate the fruit from the stalk based on the linear motion of the constraint part and the rotating gripper.

(2)

The mathematical model of the gripper single finger was established, and the gripper structural parameters were determined to meet the requirements of the grabbing range from 0~61.6 mm. 80 mm/s was set by the constraint part through the simulation test for achieving undamaged and fast requirements.

(3)

The statistical principle was used to analyze the sensor’s output to study the rule of stable grasping and slip through the two-stage picking test. The criterion of two-index stability and non-loss is proposed, which includes the grasping index of a fixed value and the slip detection of dynamic variance-to-mean ratio. Therefore, the control strategy and algorithm based on two-stage and two-index for rapid, stable, and non-destructive harvesting of small fruit were proposed. Seventy-two cherry tomatoes were picked. The results of the picking experiment for 72 cherry tomatoes showed that the picking success rate was 95.82%, the average picking time was 4.86 s, the picking damage rate was 2.90%, the browning rate was 2.90% in 72 h, and the wrinkling rate was 1.49% in 72 h.