Mobile Robot System for Selective Asparagus Harvesting

Abstract

Asparagus harvesting presents unique challenges, due to the variability in spear growth, which makes large-scale automated harvesting difficult. This paper describes the development of an asparagus harvesting robot system. The system consists of a delta robot mounted on a mobile track-based platform. It employs a real-time asparagus detection algorithm and a sensory system to determine optimal harvesting points. Low-level control and high-level control are separated in the robot control. The performance of the system was evaluated in a laboratory field mock-up and in the open field, using asparagus spears of various shapes. The results demonstrate that the system detected and harvested 88% of the ready-to-harvest spears, with an average harvesting cycle cost of $3.44\mathrm{s}\pm 0.14\mathrm{s}$. In addition, outdoor testing in an open field demonstrated a 77% success rate in identifying and harvesting asparagus spears.

1. Introduction

Agriculture, one of the oldest and most essential industries, has undergone a significant transformation as a result of the introduction of innovative technologies. Robotics is a technology that has the potential to revolutionize the agronomy industry [1,2,3]. Both Lytridis et al. [4] and Bergerman et al. [5] suggest that the incorporation of robotics into agriculture could address a number of issues, including labor scarcity, rising production costs, and environmental sustainability; in addition, it could improve precision, efficiency, and productivity, resulting in increased yields and profitability [3,4,6]. The current state of the art in agricultural robotics includes crop monitoring [7], weed control [8,9], harvesting [10,11], and post-harvesting [12]. Robots are utilized for a variety of agricultural tasks, including soil monitoring [13], seeding [14], seedling manipulation [15], fertilization [16], irrigation [17], and crop protection [18,19]. These technologies enable efficient and accurate crop monitoring, allowing for timely interventions to promote optimal growth.

Despite its economic significance, asparagus presents unique harvesting challenges. Asparagus spears are delicate, and their growth is variable, making large-scale automated harvesting challenging [20]. In recent years, significant progress has been made in the development of robot systems for asparagus harvesting, with the goal of increasing efficiency and decreasing labor costs. The University of Waikato and Robotics Plus Limited have developed a robotic asparagus harvester that is pulled by a tractor [21]. A highly effective convolutional neural network (CNN) has been developed, to detect asparagus spears [22]. Irie et al. [23] presented an asparagus harvesting robot that used an electrical cart equipped with a 3D camera and robotic arm to harvest spears along the furrow: the time needed to harvest a single asparagus spear was reduced to 13.7 s. Irie and Taguchi [24], in a later study, upgraded the sensory system, to include a laser scanner, reducing the harvesting time to approximately 7.5 s per spear. Funami et al. [25] created an electro-pneumatic, four-degrees-of-freedom robotic system capable of harvesting asparagus spears from ridges, even in dense vegetation. The asparagus harvesting trolley, designed by Zhang et al. [26], utilizes a pneumatic punching and cutting mechanism, and two pinch rollers, to grab and move cut spears onto a conveyor belt.

Inaho Inc., Kamakura, Japan (https://en.inaho.co/, accessed on 8 June 2023), introduced a tracked robotic system with a robotic arm and an infrared detection device for spear detection for greenhouse asparagus harvesting [27]. Several funded initiatives have sought to develop comprehensive harvesting solutions for asparagus: the AmLight project aimed to create an automatic harvesting system for green asparagus with stalk detection in ambient light; the MANTIS project sought to develop an automated asparagus harvester; GARotics (http://echord.eu/garotics/, accessed on 8 June 2023) presented a mobile platform on wheels, equipped with an RGBD camera for spear detection, and two Cartesian manipulators capable of harvesting five spears per meter [28].

To ensure accurate and selective harvesting, numerous asparagus detection methods have been investigated. The University of Waikato compared the performance of faster, region-based convolutional neural networks (FRCNN) to that of a single shot multibox detector (SSD), and concluded that FRCNN-based detectors are better suited to asparagus detection [29]. Kennedy et al. [30] presented a single-view representation of information from a multi-camera system, combined with temporal filtering, to reliably locate asparagus spears in laboratory and outdoor settings in real time. Sakai et al. [31] used two laser scanners for vertical scanning, to detect asparagus spears, with an accuracy of approximately 75% and a harvesting time of more than 10 s per spear. Leu et al. [28] utilized RGB and depth images to generate a point cloud for the classification, recognition, and tracking of asparagus. Hong et al. [32] utilized an improved YOLOv5 algorithm for asparagus recognition and detection in complex environments, achieving an average mean precision of nearly 99% percent. Lastly, Liu et al. [33] introduced depth-aided mask RCNN for asparagus detection, demonstrating a better balance between precision and speed than existing algorithms.

The development of a novel harvesting robot was motivated by the need to address manual labor shortages. Existing options are limited to large machines, and some are only appropriate for confined environments, such as greenhouses. Moreover, current software solutions are computationally intensive, require extensive training databases, and are unsuitable for deployment on low-cost hardware. The development of a lightweight robot intended to replace a single large machine with multiple smaller ones, thereby increasing overall efficiency, was our primary objective, the design of which should be adaptable to a variety of tasks, while the robotic system should include a robust and computationally efficient spear detection algorithm. By providing a cost-effective solution, we hope to make asparagus production accessible to a greater number of growers.

This paper presents a novel design for a lightweight agricultural robot intended for the efficient harvesting of stem vegetables, with a focus on green asparagus. In Section 2, we describe the mechanical design and software control system of the robot, and the implementation of a point-cloud-segmentation-based asparagus detection algorithm. Section 3 provides a comprehensive evaluation of the performance of the robotic system in controlled laboratory settings and in actual asparagus fields. Section 4 analyzes the test results, and provides recommendations for future work.

2. Materials and Methods

To optimize asparagus harvesting, a mobile platform with precise and quick mechanisms was required. To aid in this endeavor, a real-time asparagus detection algorithm was used, to determine the optimal harvesting points. This algorithm was designed to be lightweight, to promote efficient harvesting.

2.1. Hardware Design

The harvesting robot system comprised a delta robot mounted on a mobile platform with tracks. The gripper on the delta robot was customized for grasping and cutting asparagus spears. The platform was also equipped with a sensory system for detecting asparagus spears.

2.1.1. Delta Robot

We developed a fast and dependable robotic mechanism for harvesting asparagus spears in an efficient manner. Our solution employed a delta parallel robot with three identical arms that were triple symmetric, which provided superior dynamics and velocity compared to serial mechanisms. Each arm consisted of an actuated rotary joint and two universal joints, and was powered by a Beckhoff AM8113 servo motor and a WPLE040 gear reducer with a 1:10 gear ratio. The arms were connected to a common platform with three degrees of freedom, enabling precise control over the position of the robot’s tool center point (TCP). Figure 1 demonstrates the robot’s CAD model.

The three arms were evenly spaced at a radius of $0.15\mathrm{m}$ around a central point. The first segment was $0.25\mathrm{m}$ in length, and the second segment was $0.50\mathrm{m}$ in length. The diameter of the TCP platform was $0.13\mathrm{m}$. The robot’s workspace had the shape of a bowl, with $[-0.58,0.56]\mathrm{m}$ values in the x direction, $[-0.50,0.50]\mathrm{m}$ values in the y direction, and $[-0.75,0]\mathrm{m}$ values in the z direction. In the center, the maximum height value was $-0.24\mathrm{m}$. The robot’s workspace encompassed a cuboid with overall dimensions of $0.7\mathrm{m}\times 0.7\mathrm{m}\times 0.36\mathrm{m}$.

A fourth degree of rotational freedom was added to the mechanism, in order to ensure that the gripper approached the asparagus in the proper manner. This additional rotation enabled the proper orientation of the tool to be set. A DC motor, in combination with timing belt transmission, was used for actuation, to ensure the tool’s correct orientation. As asparagus spears grow randomly, this upgrade was essential for approaching each spear successfully.

For a reliable grasp on asparagus spears of all sizes and shapes, we developed a specialized gripper. The gripper consisted of three parts: a rigid finger made of hard plastic; a rubber finger in the shape of a fin; and a narrow blade for cutting the spear. We utilized the blade from a utility knife, because it was sturdy, long-lasting, readily accessible, and simple to replace. Figure 2 displays a detailed CAD model of the gripper.

Using an axis-in-axis design, we added a second DC motor to the robot’s upper plate, so that the gripper could rotate and be actuated. This design guaranteed that the gripper could effectively grasp and cut the asparagus spears, providing a comprehensive harvesting solution.

2.1.2. Mobile Platform

The delta robot was mounted on a mobile platform with two rubber tracks that provided traction for field driving. The distance between the tracks was 600 mm, providing sufficient space for traversing asparagus fields. The outer dimensions of the mobile platform, excluding the sensor, were $1\mathrm{m}\times 0.66\mathrm{m}\times 1.06\mathrm{m}$ (length × width × height). The sensor bar added $0.35\mathrm{m}$ to the overall length when it was included. The tracks were propelled by a Beckhoff AM8131 (Beckhoff Automation GmbH & Co. KG, Verl, Germany) synchronous motor with a combination of angular and chain reduction resulting in a total ratio of 70:1. This configuration permitted the platform to attain a nominal speed of $0.2$ m/s.

The mobile platform was made of extruded aluminum, which allowed the delta robot to be affixed at a height of 0.9 m. The platform also contained a battery, a low-level programmable logic controller Beckhoff CX5140 (Beckhoff Automation GmbH & Co. KG, Verl, Germany), and a high-level controller based on an ultra-compact PC Gigabyte GB-BRi7-10710 (Giga-Byte Technologies CO., LTD, New Taipei City, Taiwan) that was affixed above the delta robot. A Sick LMS111 (SICK AG, Waldkirch, Germany) laser scanner was installed at the front of the platform, oriented in the direction opposite the motion, in order to detect the asparagus. Figure 3 displays the CAD model of the platform, with its components labeled.

2.2. Control

The control of the asparagus harvesting robot was divided into two parts (see Figure 4): (a) low-level control, for controlling the individual motors, and for calculating the forward/inverse kinematics of both the delta robot and the mobile platform; (b) high-level control, for asparagus spear detection and localization, as well as path planning for the delta robot.

2.2.1. Low-Level Control

We implemented low-level control on the Beckhoff CX5140 (Beckhoff Automation GmbH & Co. KG, Verl, Germany) programmable logic controller, using three Beckhoff EL7211 servo motor controllers to control the delta robot’s motors, two Beckhoff EL7201 (Beckhoff Automation GmbH & Co. KG, Verl, Germany) servo motor controllers to control the track motors, and the Beckhoff EL7342 (Beckhoff Automation GmbH & Co. KG, Verl, Germany) 2-channel motion interface for the DC motors. The motor controllers were configured to accept velocity as the reference input, in order to facilitate motor control. The frequency of the control loop was 1 kHz.

• Communication

We used the automation device specification (ADS) protocol, a transport layer developed for data exchange between software modules within the TwinCAT system, to communicate with the high-level control. To facilitate ADS communication, two bidirectional data blocks were created: one for the mobile platform, and one for the delta robot. The frequency of the data exchange between the Beckhoff controller and the high-level controller was 1 kHz.

• Mobile platform kinematics

A simple mathematical model was used to describe the kinematics of the mobile platform’s low-level control [34]. Under idealized conditions, ignoring factors such as wheel slip and friction, the kinematics of the mobile platform (linear velocity $v_P$ and angular velocity ${\omega }_P$) could be described by the following equations:

$$\begin{array}{cc}\hfill v_P& =\frac{NR}{2}({\omega }_{M,R}+{\omega }_{M,L});\hfill \end{array}$$

(1)

$$\begin{array}{cc}\hfill {\omega }_P& =\frac{NR}{L}({\omega }_{M,R}-{\omega }_{M,L}).\hfill \end{array}$$

(2)

In the above equations, ${\omega }_{M,R}$ and ${\omega }_{M,L}$ denoted the angular velocities of the right and left track motors, respectively, N represented the gear ratio for the track motors, R represented the pitch radius of the track propulsion gear, and L was the distance between the two tracks. Integrating the obtained linear velocity $v_P$ and angular velocity ${\omega }_P$ yielded odometry data, which were then transmitted to high-level control, using the ADS protocol.

The reference angular velocities for the right-track and left-track motors (${\omega }_{M,D,R}$ and ${\omega }_{M,D,L}$), as shown in Equations (3) and (4), were calculated based on the reference velocities of the mobile platform, $v_{P,D}$ and ${\omega }_{P,D}$:

$$\begin{array}{cc}\hfill {\omega }_{M,D,R}& =\frac{2v_{P,D}+{\omega }_{P,D}L}{2R};\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill {\omega }_{M,D,L}& =\frac{2v_{P,D}-{\omega }_{P,D}L}{2R}.\hfill \end{array}$$

(4)

These reference velocities served as inputs for motor controllers with internal proportional–integral–derivative (PID) control loops.

• Delta robot joint control

The desired reference velocities for each joint of the delta robot were received via ADS communication from the high-level control system. Similar to track control, these reference velocities were then used as inputs for the internal PID control loop of the motor controllers.

2.2.2. High-Level Control

Robot Operating System (ROS) Noetic Ninjemys, running on a Gigabyte compact PC with Ubuntu 20.04 LTS, provided high-level control. The ROS system architecture was divided into multiple software packages, allowing for a structured approach to various problems, and facilitating error diagnosis. These packages contained:

• Beckhoff communication: the C++ package acted as a central node, facilitating ADS communication between the Beckhoff controller and the ROS computer;
• mobile platform: the package enabled the movement of the mobile platform, using tracks, and included a program for manual control;
• delta robot: the package controlled the parallel robot, providing functions for proportional–derivative control, manual operation, and trajectory planning;
• acquire asparagus: the package handled the detection of asparagus locations, using sensors; it included programs for reading and converting data from a laser scanner, generating point clouds, and extracting asparagus from them.

A schematic representation of high-level control is depicted in Figure 5.

By organizing the system into these packages, the ROS architecture encouraged modularity, encapsulation, and efficient coordination among the asparagus-picking robot system’s components.

• Delta robot kinematics

The delta robot was made up of three identical serial mechanisms (arms) that were parallel in construction. For each arm, there were three separate controls, which could be used to operate the robot: this required determining the desired position of each motor, relative to the desired position of the robot’s TCP.

Inverse kinematics [35] was required for this purpose. The following was the solution for a single arm. In Figure 6, all relevant geometry variables are marked. We defined the vector $\mathbf{D}={[D_x,D_y,D_z]}^{\mathrm{T}}$ that extended from the position of the motor ${\mathbf{p}}_1$ to the position of the joint on the TCP platform of the same arm ${\mathbf{p}}_3$. The projection $L_2^{\prime }$ of the second arm segment, with length $L_2$, onto the plane perpendicular to the axis of rotation of the first segment, could be calculated as follows:

$$L_2^{\prime }=\sqrt{L_2^2-p_{1,y}^2}.$$

(5)

Next, we determined the projection $D^{\prime }$ of the vector $\mathbf{D}$ onto the same plane:

$$D^{\prime }=\sqrt{D_x^2+D_z^2}.$$

(6)

To determine the angle $\alpha$ between the first segment with a length of $L_1$ and the vector $\mathbf{D}$, we used the cosine theorem:

$$\begin{array}{cc}\hfill L_2^{\prime 2}& =L_1^2+D^{\prime 2}-2L_1{D}^{\prime }cos\alpha ;\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill \alpha & =arccos\left(\frac{L_2^{\prime 2}-L_1^2-D^{\prime 2}}{-2L_1{D}^{\prime }}\right).\hfill \end{array}$$

(8)

The angle $\gamma$ described the angle between the vector $\mathbf{D}$ and the position of the first segment at an angle of $0^{\circ }$:

$$\gamma =arctan\left(\frac{-D_z}{D_x}\right).$$

(9)

From the angles $\alpha$ and $\gamma$, the joint value q could be determined:

$$q=\gamma -\alpha .$$

(10)

To facilitate the aforementioned method, it was necessary to define vectors $\mathbf{D}$ in the coordinate system of each arm. The $\mathbf{D}$ vectors were defined as follows:

$$\begin{array}{cc}\hfill {\mathbf{D}}_1& =-\mathbf{a}+{\mathbf{R}}_1^{\mathrm{T}}{\mathbf{x}}_{TCP}+\mathbf{b};\hfill \end{array}$$

(11)

$$\begin{array}{cc}\hfill {\mathbf{D}}_2& =-\mathbf{a}+{\mathbf{R}}_2^{\mathrm{T}}{\mathbf{x}}_{TCP}+\mathbf{b};\hfill \end{array}$$

(12)

$$\begin{array}{cc}\hfill {\mathbf{D}}_3& =-\mathbf{a}+{\mathbf{R}}_3^{\mathrm{T}}{\mathbf{x}}_{TCP}+\mathbf{b},\hfill \end{array}$$

(13)

where

$$\begin{array}{cc}\hfill {\mathbf{R}}_1& =\left[\begin{array}{ccc}cos0& -sin0& 0\\ sin0& cos0& 0\\ 0& 0& 1\end{array}\right];\hfill \end{array}$$

(14)

$$\begin{array}{cc}\hfill {\mathbf{R}}_2& =\left[\begin{array}{ccc}cos\frac{3\pi }{4}& -sin\frac{3\pi }{4}& 0\\ sin\frac{3\pi }{4}& cos\frac{3\pi }{4}& 0\\ 0& 0& 1\end{array}\right];\hfill \end{array}$$

(15)

$$\begin{array}{cc}\hfill {\mathbf{R}}_3& =\left[\begin{array}{ccc}cos\frac{3\pi }{4}& sin\frac{3\pi }{4}& 0\\ -sin\frac{3\pi }{4}& cos\frac{3\pi }{4}& 0\\ 0& 0& 1\end{array}\right];\hfill \end{array}$$

(16)

$$\begin{array}{cc}\hfill \mathbf{a}& ={\left[\begin{array}{ccc}0.150& 0& 0\end{array}\right]}^{\mathrm{T}};\hfill \end{array}$$

(17)

$$\begin{array}{cc}\hfill \mathbf{b}& ={\left[\begin{array}{ccc}0.065& 0& 0\end{array}\right]}^{\mathrm{T}}.\hfill \end{array}$$

(18)

Here, ${\mathbf{x}}_{TCP}$ represented the desired position of the robot’s TCP, $\mathbf{R}$ represented the rotation matrices describing the rotation of each arm about the z axis of the base coordinate system, and $\mathbf{a}$ and $\mathbf{b}$ represented translations from the base coordinate system to the first joint of the arm ${\mathbf{p}}_1$ (a translation of 150 mm along the x axis), and from the TCP coordinate system to the last joint of the arm ${\mathbf{p}}_3$ (a translation of 65 mm along the x axis). By employing the aforementioned method, all three joint angles could be calculated, thereby yielding the joint angles corresponding to the desired robot position.

• Trajectory planning

During asparagus harvesting, a delta robot attached to a mobile platform must compensate for platform motion. In other words, while the platform is in motion, the robot grippers must remain stationary, relative to the global coordinate system: to accomplish this, the delta robot’s reference position must be dynamically determined in each control loop.

First, we determined the coordinate with the greatest range of motion, denoted $x_f$, in order to reach the desired position; on the basis of this, and the maximum velocity of the TCP ${\dot{x}}_{\max }$, we calculated the time necessary to complete the motion, denoted by $t_f$:

$$t_f=t+\frac{\left|x_f-x\right|}{{\dot{x}}_{\max }}.$$

(19)

As the desired position could change while the platform was in motion, this parameter had to be constantly updated, to ensure that the robot’s trajectory was maintained accurately.

In order to meet this requirement, the robot’s trajectory was generated using a minimum jerk interpolation method [36]. The expression (20) describes the calculation of the TCP’s jerk $\stackrel{⃛}{x}$:

$$\begin{array}{cc}\hfill \stackrel{⃛}{x}& =\frac{-9\ddot{x}}{\Delta t}+\frac{-36\dot{x}}{\Delta t^2}+\frac{60\left(x_f-x\right)}{\Delta t^3};\hfill \end{array}$$

(20)

$$\begin{array}{cc}\hfill \Delta t& =t_f-t\hfill \end{array}$$

(21)

Here, $\Delta t$ represented the difference between the final time $t_f$ and the current time t.

Figure 7 is an illustration of the minimum jerk interpolation for the x coordinate in the system of the delta robot during asparagus harvesting. The reference values for regions (b) and (c), in which the asparagus was virtually moving in relation to the mobile platform, were modified based on the platform’s velocity. In the idle state (case a), and during the deposition of harvested spears into the container, the reference remained constant, as these positions were fixed in the mobile platform’s coordinate system.

We acquired the TCP’s acceleration $\ddot{x}$, velocity $\dot{x}$, and position x by employing the three-step Euler integration method. Inverse kinematics was utilized, to convert the desired robot position into joint coordinates. Subsequently, an external proportional–derivative controller used these joint values. The controller computed and transmitted to the low-level controller the desired joint velocities via ADS communication.

• Manual control

A Logitech F710 wireless gamepad was used as the manual control interface. The buttons and joysticks on the gamepad were assigned specific functions, such as controlling the movement and velocity of the mobile platform, and manipulating the delta robot and gripper. We implemented a dedicated safety button that required continuous activation during platform movement, to ensure operational safety. This safety mechanism allowed for the swift cessation of platform motion in the event of uncontrolled movements or loss of connection to the controller.

2.3. Asparagus Spears Detection

A SICK LMS111 outdoor 2D LiDAR sensor was used to detect the asparagus spears. The working range of the LMS111 was $0.5$ m to 20 m, and it had a scanning resolution of $0.{25}^{\circ }$ and a scanning frequency of 25 Hz. The sensor was attached to the front of a mobile platform, pointing in the opposite direction of the vehicle’s motion. The sensor was positioned at an angle of ${45}^{\circ }$ relative to the y axis of the mobile platform, and at an appropriate height, to ensure that the spears fell within the LMS111’s minimum working range. The linear resolution of the sensory system was $1.1$ mm. Figure 8 illustrates an example of the scanned data: as can be seen, the operation principle of the sensor produced noticeable shadows.

A laser scanner only provides distance data on a single plane; therefore, a 3D measurement is required, to determine the position and height of an object, such as an asparagus spear. To obtain this measurement, we created a joint point cloud, by combining data from the laser scanner with odometry data from the mobile platform. We converted the data from the cylindrical coordinate system utilized by the LMS111 to a global Cartesian coordinate system. The laser data was triggered by position, and sampled every 1 mm of mobile platform movement, resulting in a measurement independent of speed. In each direction, the spatial resolution of the measurement system was approximately 1 mm.

The obtained point cloud was altered, to include only the region of interest between two tracks. This updated point cloud was then discretized in the x–y plane, with a 20 mm step, resulting in the creation of a grid matrix for z values. As shown in Figure 9, for each discrete field, the sum of the z axis points was computed. To minimize the impact of potential weeds or other objects on the asparagus detection, only z values above $0.05$ m were used at this stage. Afterward, the macro-locations of the spears were determined, by comparing the obtained values to a predetermined threshold.

Based on the previously detected field within the x–y grid matrix, the micro-location of the spear was determined. Figure 10a,b depict the z values below 50 mm above the soil level, which were used to confirm the spear’s potential growth location. The aggregate of these z values had to exceed a predetermined threshold. The center of the spear was then determined, by calculating the median of the x and y values in the field of interest in the grid matrix (Figure 10c).

As the final step, we examined the z values within a 30 mm radius of the spear’s position, to confirm that it was of harvestable height (the threshold height for harvested spears was set to 150 mm). The positions of confirmed spears were converted to the coordinate system of the delta robot, and sent to a trajectory planning algorithm, to generate a harvesting path.

2.4. Harvesting Procedure

The asparagus harvesting algorithm optimized the process, by taking a number of variables into account, to ensure controlled and efficient harvesting. The algorithm began by detecting the locations where asparagus was growing, and, based on its height, determining whether or not it was suitable for harvesting. These growth locations, obtained in a fixed global coordinate system, were essential for harvesting purposes.

The algorithm initiated linear motion of the platform, and employed a laser scanner to generate a point cloud, in order to plan the gripper’s trajectory. Then, an algorithm for point cloud segmentation was used, to detect the presence of asparagus, and to confirm the availability of growth locations. If no suitable growth locations were identified, or if the asparagus detected was too small, the delta robot returned to its initial position.

After a suitable asparagus had been identified, the algorithm planned the trajectory, using a FIFO system, harvesting the asparagus in the order in which it was identified. To prevent duplicate harvesting, the algorithm compared the distances between harvested and detected asparagus in the x–y plane, using the Euclidean distance formula.

As an additional safety measure, the algorithm verified whether adjacent asparagus were too close to the target asparagus, which could impede successful harvesting, due to the gripper’s size. This also ensured sufficient picking space in front of the desired asparagus. Upon detecting an obstruction, the algorithm modified the picking angle, until a clear path was found.

The resulting trajectory for harvesting asparagus included three points: the initial point for approaching the asparagus at picking height; the final point, where the asparagus was growing; and an intermediate point halfway between the initial and final points.

Once the picking points had been determined, the manipulator executed its movements, using interpolation with jerk minimization. The sequence of movements included the mobile platform moving along the asparagus row, opening and closing the gripper, moving to the starting point, reaching the desired points, compensating for velocity during gripper closure, lifting the harvested asparagus, storing it in a container, and releasing the gripper.

The algorithm optimized the asparagus harvesting process, by taking into account constraints, such as the size of the asparagus, its proximity to other asparagus, and the space available for picking. By incorporating these variables, the algorithm ensured efficient and controlled harvesting, thereby boosting overall output.

3. Testing of Asparagus Harvesting

The robotic system for asparagus harvesting underwent testing in a controlled laboratory environment, employing plastic 3D-printed models of asparagus (see Figure 11a). Subsequently, the robot was further tested outdoors, on an actual asparagus field, as depicted in Figure 11b.

3.1. Laboratory Testing

We carried out 10 harvesting repetitions on a $2.5$-m-long row containing 16 asparagus spears of various shapes (straight, curved) and heights (10 spears > 15 cm, 6 spears < 15 cm). To replicate real-world conditions as closely as possible, we spread garden soil on the ground. In total, a 25-m-long test polygon, with 100 harvest-ready spears and 60 spears below harvesting height, was used. Each repetition involved the random placement of spears in a non-uniform pattern. A test setup example is depicted in Figure 12.

The speed of the platform was manually set to approximately 60 mm/s for each repetition: at that speed, the robotic system was able to detect and harvest spears, even if two spears were placed next to one another, while maintaining continuous motion. Due to hardware limitations, the mobile platform included stop-and-go motion, in order to collect all spears if the platform’s speed increased.

The laboratory test results, as shown in

Table 1. Results of laboratory and outdoor field tests: spears ready to harvest; recognized/unrecognized spears; harvested spears; missed spears, length of the test field; and total time required. The relative rate, expressed as a percentage of the total number of spears ready to harvest, is indicated within the brackets.

	Spears Ready to Harvest					Total Length/m	Total Time/s
	Total	Rec. a	Unrec. b	Harvested	Missed
laboratory	100	94 (94%)	6 (6%)	88 (88%)	6 (6%)	25	423
outdoor field	43	39 (91%)	4 (9%)	33 (77%)	6 (14%)	25	437

^a recognized, ^b unrecognized.

, indicate a 94% detection rate for all the asparagus, with only 6 of the 100 harvest-ready spears going undetected, due to their close proximity, causing them to be treated as a single spear (see Figure 12). Of the harvest-ready spears, 88% were successfully detected and collected; however, there were six instances in which spears were detected but not harvested, due to inaccurate harvesting point determination. No small spears were harvested.

The total completion time for all 10 repetitions was 423 s, excluding robot return times. The harvesting cycle cost $3.44\mathrm{s}\pm 0.14\mathrm{s}$ (mean ± standard deviation), from the detection of the spear to the transfer and release of the spear into the storage container.

3.2. Field Testing

Testing was conducted on an asparagus field at the Biotechnical Faculty of the University of Ljubljana, Slovenia, that is used solely for research. The robot was tested on a single row measuring 25 m in length and 1 m in width. Prior to harvesting, there were a total of 57 asparagus spears, of which 43 were ripe for picking, and 14 were deemed insufficiently developed.

The settings of the robot remained identical to those used during the laboratory testing. The robot traversed the length of the field in a single pass at a speed of approximately 60 mm/s. The robot successfully harvested 33 of the 43 mature asparagus spears, accounting for 77% of the total harvest. In addition, 6 out of the 43 correctly identified spears were not harvested (14%). The clustering of missed spears led to the identification of a single spear with the harvesting point positioned in the middle. A further 4 out of the 43 spears ripe for picking were not identified at all (9%): the growth of these spears was poor, as they were either severely twisted or aligned parallel to the ground. In such cases, the detection algorithm classified the spears unsuitable for harvest, because they did not meet the predetermined height threshold. There were no too-small spears harvested.

4. Discussion

This article describes a lightweight agricultural robot designed specifically for green asparagus harvesting. For efficient spear harvesting, the robot system consists of a tracked mobile platform and a delta robot. The delta robot significantly reduces picking times (below 3.5 s) when compared to serial mechanisms [24,28].

The results from the study indicate that the recognition rate in laboratory tests was 94%, which is comparable to [33]; however, six spears were not recognized, because there were instances in which spears were placed too closely together, causing the algorithm to mistake two separate spears for a single spear. This issue, of detecting a single spear as opposed to multiple spears, was also responsible for the failure to collect some spears. In the laboratory experiment, the overall rate of harvest was 88%. The incorrect determination of the picking point resulted in a few misses. When two or more spears were grouped together, the harvesting point was incorrectly determined to be in the middle of the spears, resulting in harvesting next to the spear rather than at the spear. Another potential error was the density of spears, causing the robot to be too slow in picking all of them. At this stage, the stop-and-go method must be utilized to address this issue.

In an open field, the spear recognition rate was 91%, while the harvesting success rate was 77%. The lower success rate compared to the laboratory tests was due to the various growth patterns and spear shapes. The issues with harvesting are attributable to the detection method that is currently in use: when spears are closely spaced, overlap, or grow at varying angles, this method becomes ineffective. Because the grasping point is determined in the middle, two asparagus spears that are close together may be misidentified as a single spear, leading to a missed harvest. In addition, if spears are twisted or growing at an angle, the point cloud segmentation method generates inaccurate gripping points, due to an increased number of detected points where the spear is straight, typically at the spear’s tip, as opposed to its base. Furthermore, severely tilted spears may not meet the height requirement for harvest readiness, and are therefore incorrectly processed as not harvest-ready.

The majority of harvesting mechanisms [24,26,28] can only approach spears from one side, preventing them from avoiding small spears or other obstacles. Funami et al. [25] presented a partial solution, by introducing a SCARA-type robot that allows the tool’s orientation to be set; however, the issue persists, as the mechanism’s segments can still harm other spears. In our design, the robot’s gripper incorporates an additional degree-of-freedom, enabling orientation changes and the harvesting of asparagus from different angles. This feature prevents damage to young spears, which would otherwise inhibit plant growth, because the spear releases growth-stimulating enzymes when it reaches a height of 12 cm or more.

For spear detection, a laser scanner was employed, similar to the approach used by Sakai et al. [31], where spears were scanned vertically from the ground to the top. The point cloud generated by the combination of 2D laser scans and odometry eliminated the need for an RGBD camera. This system is anticipated to operate reliably throughout the day, irrespective of lighting conditions. Utilizing a straightforward method of point cloud segmentation, the optimal picking points were determined. This approach is suitable for systems with limited computational capability, such as Raspberry Pi, operating in real time.

Based on the robot’s test speed of $0.06$ m/s, a single robot is capable of harvesting approximately 2000 m of asparagus rows in 10 h. Taking into account the operational limitations of the robot, this corresponds to an estimated harvest of approximately 8.000 spears, which exceeds the results of Sakai et al. [31], who estimated a harvest of 6.500 spears in 18 h. Notably, this estimation assumes that the spears are growing approximately 0.2 m apart, that the robot operates continuously, and that approximately 20% of the time is allocated for platform maneuvering, such as turning between rows.

The small size of the robotic system provides numerous benefits. Smaller and lighter autonomous robots are more environmentally friendly than large and heavy farm machinery, because they cause less damage to arable land. In addition, smaller robots are typically more cost-effective, enabling the deployment of multiple small robots for the price of a single large robot.

On the basis of test-identified deficiencies, future work should involve enhancing the detection system with additional low-cost modalities, to further enhance asparagus detection. As suggested by Leu et al. [28], a combination of RGB and laser technologies could be utilized, to improve the system’s ability to detect and identify asparagus spears. Priority should also be given to the implementation of AI methods, in order to increase the detection success of asparagus with varying shapes. Efforts should be made to improve design and functionality, in order to make harvesting operations faster and more efficient. In instances where there is a large cluster of spears, speed adaptation should be implemented, to ensure thorough picking, with the mobile platform adjusting its speed, or even coming to a complete stop, to collect all spears within the cluster. In addition, an RTK GPS (real-time kinematics GPS) system should be considered, for mapping the locations of small asparagus spears, which would provide valuable information for future harvesting. The system could be expanded to harvest a variety of stem vegetables, including leeks and celery, with minimal modifications; furthermore, it is adaptable to tasks other than vegetable harvesting, such as flower picking.

In addition, the mobile robot system could be augmented by additional capabilities, to address common issues in asparagus fields, such as weed control. Mechanical weeding is not feasible, as it can damage underground spears. Integrating non-contact weeding options, such as laser weed [9] or flame weeders [37], into the robot system is one potential solution.

The short harvesting season for asparagus (typically up to nine weeks) necessitates alternative business models, in light of the substantial financial investment required for agricultural robots. Exploring additional modalities, or offering the robot as a service in which the system can be rented for a fee, would increase its economic viability. Therefore, the presented platform serves as the basis for the development of a multifunctional robot that can be used throughout the entire growing season.

5. Conclusions

This paper introduced a specialized asparagus harvesting robot system that effectively addresses the challenges posed by spear growth variability. The design consists of a tracked mobile platform outfitted with a delta robot for harvesting individual spears. The additional rotation of the gripper enables the robot to harvest spears from the optimal direction. The system demonstrated a high rate of success in both laboratory and field settings. The robot’s diminutive size provides environmental and financial benefits, while potential enhancements extend its utility. In conclusion, this research significantly improves asparagus harvesting’s productivity and sustainability, ensuring the availability of high-quality produce.