Evaluation of End Effectors for Robotic Harvesting of Mango Fruit

Abstract

The task of gripping has been identified as the rate-limiting step in the development of tree-fruit harvesting systems. There is, however, no set of universally adopted ‘specifications’ with standardized measurement procedures for the characterization of gripper performance in the harvest of soft tree fruit. A set of metrics were defined for evaluation of the performance of end effectors used in soft tree-fruit harvesting based on (i) laboratory-based trials using metrics termed ‘picking area’, which was the cross-sectional area in a plane normal to the direction of approach of the gripper to the fruit in which a fruit was successfully harvested by the gripper; ‘picking volume’, which was the volume of space in which fruit was successfully harvested by the gripper; and ‘grasp force’, which was the peak force involved in removing a fruit from the grasp of a gripper; (ii) orchard-based trials using metrics termed ‘detachment success’ and ‘harvest success’, i.e., the % of harvest attempts of fruit on tree (of a given canopy architecture) that resulted in stalk breakage and return of fruit to a receiving area, respectively; and (iii) postharvest damage in terms of a score based on the percentage of fruit and severity of the damage. Evaluations were made of external (skin) damage visible 1 h after gripping and of internal (flesh) damage after ripening of the fruit. The use of the metrics was illustrated in an empirical evaluation of nine gripper designs in the harvest of mango fruit in the context of fruit weight and orientation to the gripper. A design using six flexible fingers achieved a picking area of ~150 cm² and a picking volume of 467 cm³ in laboratory trials involving a 636 g phantom fruit as well as detachment and harvest efficiency rates of 74 and 65%, respectively, in orchard trials with no postharvest damage associated with the harvest of unripe fruit. Additional metrics are also proposed. Use of these metrics in future studies of fruit harvesting is recommended for literature–performance comparisons.

1. Introduction

1.1. Evaluation of Harvesters

The quest to develop aids for the harvest of tree fruit is as old as human civilisation. This quest is illustrated by an 1886 patent for a ‘new and improved fruit picker’ by Arnold [1] for an end effector mounted to a pole carried by a human operator. There have also been many attempts in recent decades to further mechanise this process by placing end effectors on autonomously guided manipulators, e.g., as reviewed by Bac et al. [2] and Tang et al. [3]. However, no widely adopted commercial solution exists yet for the mechanical harvest of soft tree fruit.

The harvesting of fruit requires the fruit to be located in a 3D space, grasped, detached from the tree, and placed in a receptacle. Automation of soft fruit harvesting requires a mobile platform carrying a manipulator operating with at least three degrees of freedom (DoF) to reach any position within its work environment. However, a manipulator with 6 DoF are required to reach any pose, that is any combination of position and orientation. Control complexity and cost increase rapidly with DOF number. In a mechanical fruit harvester, the manipulator carries an end effector on its last link, which is used to interact with the fruit. Soft tree fruits are relatively low-value items, and harvest involves large numbers of fruit; for example, there are 35,000 pieces of mango fruit per ha (for a yield of 15 t/ha and an average fruit weight of 0.43 kg) in a typical Australian mango orchard. Associated harvest equipment must be relatively low cost and robust for field use.

There are numerous reports on the development of robotic tree-fruit harvesting systems. For example, Silwal et al. [4] documented the performance of an apple harvester in terms of the accuracy of the machine vision system in fruit detection and average localisation time, harvest success rate (successful picks as a % of attempted picks), and average picking time per fruit. Williams et al. [5] documented the performance of a kiwi-fruit harvester in terms of the successful harvest of both reachable fruit and of all fruit in the canopy and cycle time.

Comparison of technologies is, however, severely compromised due to a lack of comparable performance data [6]. Performance metrics with clearly documented criteria are therefore required. In a comparison of harvesting systems, Zhou et al. [6] adopted the metrics of Bac et al. [2]: (i) ‘harvest success rate’ (harvested fruit as a proportion of all fruit in the canopy); (ii) ‘cycle time’ (the average time taken for the harvest of a single fruit by a single arm, including for missed fruit and movement time); and (iii) ‘fruit damage rate’. However, these metrics reflect both the performance of the machine and the architecture of the canopy in terms of density of the fruit, foliage, and branches. This confounds the inter-study comparison of equipment performance. A machine-centric interpretation of these metrics is (i) ‘harvest success rate’, defined as harvested fruit count as a proportion of harvest attempts; (ii) ‘cycle time’, defined as the average time taken for the harvest of a single fruit by a single arm from initiation of arm movement to release of fruit into harvest receptacle; and (iii) ‘fruit damage rate’ with a definition for each fruit type, e.g., loss of peduncle in apple, which results in reduced shelf life [7].

1.2. Gripper Evaluation

Performance metrics can be set on a component of a harvesting system. In the current study, we focus on a consideration of end-effector performance. There are performance benchmarks for robotic grasping. Many reports use a protocol for assessing gripper performance based on scoring for the maintenance of a firm grasp of objects of varying size and shape (typically ‘daily objects’, such as pencils, coins, and tennis balls) from different offset positions through a sequence of events, e.g., rotations and translations [8,9,10,11]. Other reports document grasping force in the evaluation of gripper performance. For example, Kragten et al. [12] and Wang et al. [13] measured the force required to pull a grasped object out of the gripper applied perpendicular to the palm of the gripper. Ma et al. [14] designed a testing platform to measure the force applied to the grasped object at the fingertips of a pneumatic gripper during actuation.

There is no set of universally adopted ‘specifications’ with standardised measurement procedures for the characterization of gripper performance in the harvest of soft tree fruit. In most reports, integrated tests are undertaken in which the gripper is assessed during the robot’s normal operating cycle. Chen et al. [15] ‘compartmentalized’ the performance of a soft gripper in the harvest of apples in terms of ‘picking success rate’ and ‘skin damage rate’. In addition to the pick and damage rate, Xiong et al. [16] also evaluated ‘fruit isolation’, i.e., the ability to isolate the targeted fruit from its neighbours, in a strawberry harvest application.

1.3. Gripper Designs

The choice of end effector is driven by consideration of both harvesting effectiveness and life-cycle cost. Grippers are the most commonly used type of end effector in fruit harvesting robots [17], though there are alternatives. For example, Abundant Robots (Santa Monica, CA, USA) employs a tube as an end effector, using low air pressure to draw apple fruit into the system [18]. Grippers can be characterised by the number of fingers used. Two-fingered grippers are the most common in general use, but three-fingered grippers are argued to be the most stable for use with approximately spherical fruit [19]. Generally, multi-fingered grippers are more effective for grasping fragile objects due to their increased area of contact with the object, requiring less contact pressure at a given point [20]. Kim et al. [21] developed a three-fingered gripper capable of grasping objects of various sizes with a well-distributed force and pressure that was also able to transition between pinching and grasping modes.

Grippers can also be characterised according to the form of the finger used. A simple design consists of two firm surfaces, i.e., fingers, moved towards each other to grasp an object [22]. An example is provided by the Twister Fruit Picker (Spokane, WA, USA), a gripper commercially used in manual fruit picking. As described in Hanford [23], this gripper consists of a scissor-like arrangement of two ‘spoon’-like ends with closure actuated by a cord and opening controlled by the action of a tension spring.

A more complex design involves separating each finger into a series of under-actuated hinged links (phalanges). An example of this design is the EZGripper™ from SAKE Robotics (Redwood City, CA, USA) described in Thiem et al. [24] (who also provide a simulation model). In this under-actuated design, the fingers remain straight when grasping small objects but wrap around larger objects. Each finger has two spring-loaded phalanges actuated by Dyneema tendons running through aluminium oxide ceramic eyelets powered by a Dynamixel MX-64 servo motor under the base plate of the gripper. The phalanges are 30 mm in height and 50 mm in length with a grasp opening of 170 mm at 50% open and 285 mm at 100% open (70° and 180° between fingers, respectively). An abrasive coating on the phalanges increases friction with the contacted object. This end-effector has been employed in a wide range of applications; for example, Bajracharya et al. [25] modified this gripper by adding hooks for the opening of drawers in a home-assist function.

Rigid grippers such as the EZGripper™ are precise and can handle heavy objects; however, they are unable to conform autonomously to objects of different shapes. Soft grippers constructed from compliant materials, e.g., elastomers, sacrifice grasping precision and payload capacity for compliance while also typically being simpler, cheaper, and lighter than rigid grippers [26]. Flexible fingers provide greater area of contact between finger and object, distributing the force exerted on the object being grasped and reducing risk of damage to the object [21]. Several designs are commercially available. For example, the Soft Robotics mGrip™ finger consists of pneumatically actuated silicone compartments [27]. The Festo adaptive gripper finger is based on the Fin Ray^® effect. This mechanism uses a principle described by Kniese [28] based on observations of the fins of bony fish, which bend in the direction of a compressive force [29]. A three-fingered arrangement of Fin Ray^®-type fingers was used by Chen et al. [15] for apple harvesting.

New gripper designs are constantly being reported in the literature. For example, Becker et al. [30] designed an end effector composed of tentacles made of hollow elastomeric filament actuated by fluid, which can successfully grasp objects with varying shape and size. However, the design is not fit for fruit harvesting as the targeted object must be placed below this gripper and the long tentacles prevent it from navigating in dense canopy. Kellaris et al. [31] developed a gripper with fingers composed of spider-inspired electrohydraulic joints, while Yap et al. [32] described a design that used the actual leg of a spider as the end effector.

The design of a gripper for the fruit-harvesting task is challenging as fruit are relatively delicate and vary in size, shape, weight, and placement on trees relative to branches. The gripping task was identified as the rate-limiting step in the development of tree-fruit harvesting systems in general [19]. Bespoke solutions are effectively required for each fruit type given differences in fruit size, shape, and the required harvesting motions. For example, mango fruit is harvested using a twisting motion, requiring a different design than that for an Imperial mandarin, which requires the stalk to be cut, as twisting to break the stem results in tearing of the skin.

1.4. Mango Harvest

With the exception of one conference presentation [33], there is no scientific literature on automation trials for mango harvesting, as revealed by a Scopus search of the string ‘mango AND robotic OR mechanical AND harvest’. However, several foundation technologies are now in place. Deep-learning-based detection of mango fruit in RGB images, e.g., as developed by Koirala et al. [34], has been scaled to practical application in the form of fruit counts of whole orchards, e.g., as documented in Anderson et al. [35], and low-cost depth cameras have been deployed in orchards for the estimation of mango fruit size on trees, e.g., as described in Neupane et al. [36].

Konam [33] presented a conceptual design of a mango harvest using a quadcopter equipped with a blade as an end effector to cut the fruit stalk, with fruit falling into nets under the tree. This concept does not seem feasible because fruit will be damaged falling through the tree canopy and hitting fruit already in the net, and exuded sap will deteriorate the quality of fruit if it is left sitting on the capture net for more than a few seconds. Human labour is also required to collect fruit from the capture nets. The battery life of the free-flying quadcopter described would also limit the harvesting operation, as would service life. A similar concept was adopted (and patented) by Tevel Aerobotics Technologies (Tel Aviv, Israel) for apple harvesting with the key difference being the usage of a fleet of tethered drones with power provided through a cable instead of a battery within the drone [18]. This system initially employed a three-fingered gripper but currently uses a suction gripper.

Several factors relevant to harvest of mango fruit should be considered when selecting an appropriate design of a gripper for this application. First, the fruit is relatively large and located at the end of a long (inflorescent) stalk, making approach with an end effector easier than, picking apples, which are clustered on short stalks, for example. Second, the fruit is not symmetrical, which may pose challenges in grasping compared to apples or citrus fruits. Third, fruit size at harvest varies markedly with growing conditions and cultivars (

Table 1. Typical fruit lineal dimensions for several mango cultivars. Mean ± standard deviation, with range in brackets, is presented for fruit collected across multiple orchards at harvest maturity (n > 300 per cultivar). Data from Amaral et al. [39], except Kensington Pride average lineal dimensions, which are from DAF [40].

Cultivar	Length (mm)	Width (mm)	Thickness (mm)	Mass (g)
Kensington Pride	106	79	83	396 ± 80 (295–543)
Calypso	100 ± 8 (78–120)	86 ± 8 (63–113)	76 ± 7 (62–96)	369 ± 82 (155–678)
Honey Gold	106 ± 8 (82–128)	105 ± 8 (81.0–122.0)	89 ± 7 (71–104)	554 ± 124 (141–812)
R2E2	117 ± 11 (90–135)	109 ± 9 (82–124)	100 ± 8 (82–113)	754 ± 147 (339–1042)
Keitt	114 ± 12 (85–142)	97 ± 9 (72–124)	84 ± 8 (68–105)	511 ± 141 (234–897)

). Fourth, harvesting mango fruit involves a sharp rotation of the fruit to break the stalk at the point of attachment to the fruit, requiring a tangential peak force at the stalk-fruit interface of at least 64 ± 10 N [37]. Fifth, as released sap burns the fruit skin, the fruit should be conveyed in an inverted orientation to allow sap to drip away from the fruit. Lastly, all materials used in the harvester system should be resistant to the corrosive sap. For example, the vacuum harvesting systems described by Zhang et al. [38] for use with apples might not be appropriate with mango fruit given the release of acidic sap.

1.5. Aims

In a prequel study, Goulart et al. [37] developed phantom mango fruit to enable comparative assessment of end effectors in the development of a robotic harvester given the limited seasonal availability and perishability of real fruit. In the current study, a set of metrics are proposed and implemented for the comparison of gripper effectiveness in harvesting mango fruit. Novelty is claimed in the development of performance metrics, and, to our knowledge, this is the first report on gripper effectiveness in the harvest of mango fruit.

2. Materials and Methods

2.1. Metrics

Three metrics were defined for comparison of end-effector effectiveness in laboratory trials. ‘Picking area’ was estimated from a plot of all successful grasp positions within a 2D plane for a given design, phantom, and z distance. ‘Picking volume’ was estimated as the volume of all successful harvest positions within a 3D space. ‘Grasp force’ was estimated as the force required to remove a piece of fruit from the grasp of a gripper.

For in-orchard trials, harvest failure was classified as either a failure to detach the fruit from its stalk during rotation, or a failure to retain the grasp of the fruit during manipulator retraction. Two metrics were used: (i) the percentage of attempted fruit harvests in which the fruit was detached from the stalk and retained in the gripper, termed ‘detachment success’; and (ii) the percentage of fruit returned to the arm’s home position, with the placement of the detached mango into an allocated receiving area, termed ‘harvest success’. Harvest success is therefore necessarily equal to or less than detachment success. Failures caused by software issues, e.g., false-positive mango detections, were excluded from the count of attempted fruit harvests.

Two metrics were also defined to describe postharvest damage, namely ‘skin damage’, assessed immediately after harvest of ripened fruit, and ‘flesh damage’, assessed of ripened fruit.

2.2. Gripper Designs

Gripper evaluation was undertaken in the context of a 3 DOF manipulator with a horizontal approach of the end effector to the fruit for grasping followed by a rapid 180° twist to detach the fruit from the stalk and for sap drainage away from the fruit. It is intended that the manipulators of this study be used within a horizontal array that is translated across the face of the mango tree canopy. In this application, the gripper opening must be large enough to accommodate any mango, with larger openings requiring fewer manipulators in the array, but increasing the risk of inclusion of extraneous material such as leaves in the grasp. Based on expected fruit widths (Table 1), a grasp opening of 175 mm was adopted for gripper design. Overall efficiency of the harvesting, i.e., speed of harvest, is improved with increases in the vertical range of fruit positions in which a gripper can successfully reach. The design goal for the gripper was therefore to maximise the vertical range in which fruit could be successfully harvested.

Gripper design varied in terms of finger type, length, spacing, and its orientation within the plane of the ‘palm’, i.e., base plate, of the gripper. In-house grippers were designed through an iterative process influenced by performance observations. Designs were executed in Autodesk Fusion 360 (version 2.0), and components were fabricated, as required, using a 3D printer (FlashForge^® Guider II using FlashForge ABS Filament-1.75 mm diameter, Zhejiang, China).

Designs evolved across the following variants:

• Twister Fruit Picker (Spokane, WA, USA) (Figure 1a) with grasp opening at 155 mm;
• Vertical extensions were added to the phalanges of the EZGripper™; however, this did not improve performance (results not included);
• An in-house under-actuated design, referred to as ‘Under-actuated 2’, in which the height of the phalanges and palm was increased to 100 mm (Figure 1c);
• Several designs were based on use of flexible fingers (20 mm height, 100 mm length) employing the FinRay™ effect as described in [41]. A three-fingered arrangement, as used by Chen et al. [15] for apple harvesting, was trialled but discontinued (and not documented here) as it failed to cope with variation in mango fruit size. Four- and six-finger arrangements were trialled, either in two parallel arrays of two and three fingers, respectively, or with the top and bottom fingers tilted (Figure 1d–g). These designs were coded in terms of finger number, top and bottom finger angles, and palm height and width; for example, 6F_30A_100H_100W describes an end effector with six fingers, 30° angle on top and bottom fingers, and a 100 mm × 100 mm base plate.

2.3. Experimental Platform

Grippers were evaluated in the context of a manipulator with 2 DoF, using a prismatic joint that translated the gripper in a horizontal plane to the fruit and one joint to rotate the gripper. To achieve harvest of fruit, the manipulator’s arm was extended, the gripper closed and then rotated to break the stalk, the arm was retracted, the gripper opened to release the harvested mango into a receptacle, and the cycle ended with rotation of the gripper to its starting position. The duration of a pick cycle was around 6 s. The torque specification of the rotary actuator was 2.6 N m, thus a force of approximately 65 N should be exerted at a fruit-stalk interface 40 mm from the axis of rotation. This estimate was consistent with forces measured using a dynamometer (FT 327, Wagner Instruments, Greenwich, CT, USA) following the method of Goulart et al. [37].

A solitary manipulator with a gripper was used in laboratory trials. For orchard trials, eight manipulators were mounted at 170 mm spacing on an elevated platform either under operator manual control or under control of a vision system. The vision system employed a neural network model, as described by Koirala et al. [34], to detect fruit in images captured by a depth camera (Azure Kinect DK, Microsoft, Redmond, WA, USA). The detected objects were tracked with respect to the robot’s frame of reference. The elevator was moved to position a manipulator such that the top of the detected fruit was at the same height as the top of a gripper. A pick cycle was then executed for the target fruit.

2.4. Laboratory Trials

An artificial but reproducible environment was used to evaluate the grasp efficiency of several end-effector designs. These laboratory trials did not consider the presence of interfering materials such as leaves, branches, or other fruits. A common set of phantom mango fruit (of masses 378, 512, 636, and 836 g (Figure 2)) was used. Phantoms were produced from a starch-silicone mixture as described in Goulart et al. [37]. A frame was used to suspend and translate a phantom fruit within a two-dimensional plane, parallel to the base plate of the gripper (i.e., horizontal and vertical movement (Figure 3a)). A spring-loaded ball and hole latch assembly on the frame allowed reproducible 10 mm steps in horizontal (x) and vertical (y) directions. The frame could also be translated in relation to the distance to the end effector, i.e., depth (z).

Fruit position was defined with reference to the mid-point of the top of the fruit. End-effector horizontal and vertical positions were defined with reference to the mid-point of the top of the base plate of the end effector (Figure 3b). The position of the end effector in the direction of the movement of the manipulator, i.e., in the z plane, was assigned a value of 0 at the point the fruit touched the base plate. As mango fruit is non-symmetrical, trials were undertaken in which the gripper approached either the front, back, or side of the fruit (Figure 2).

The test pick cycle involved a movement of the end effector to the phantom followed by closure and rotation of the gripper. The arm then retracted to its home position and the end effector opened, releasing the phantom fruit (if successfully ‘harvested’). Harvest failure occurred when the fruit was outside the grasp of the end effector or when the fruit slipped from grasp during the rotation or retraction movements. Grasp success/failure per position was scored as 1/0. Three measurements were taken per position with the majority result recorded.

The force required to remove the 626 g phantom fruit from a gripper was assessed by grasping the fruit in a given gripper and retracting the supporting arm with the fruit attached to a dynamometer (FT 327, Wagner Instruments, Greenwich, CT, USA) held in a bench vice. The peak force was recorded.

2.5. Orchard Trials

The two designs with the best grasping efficiency, as identified in the laboratory trials, were trialled in an orchard setting (Figure 4). For each design, harvest attempts were made on approximately 100 fruits in given orchards. Trials were undertaken in a cultivar Keitt orchard in which the trees had been pruned to remove some obstructing branches and foliage and two unpruned orchards (of cultivars Keitt and Calypso) with fruit present in an outer canopy of non-woody branches.

Once a mango was detected within range of the manipulator, the elevator was positioned such that the top of a gripper was at the level of the top of the fruit. A pick cycle was then executed for that fruit and any other fruit within the vertical range as defined by the picking area (at z = 0 mm) identified during laboratory trials.

2.6. Postharvest Evaluation

Trials were based on a ‘worst case’ scenario in which fruit were gripped below the fruit centre, such that the edge of the top fingers contacted the fruit. For each gripper type, fruit cheeks (n = 30) were examined for skin damage 1 h after harvest and for skin and flesh damage after ripening to the eating stage. The fruits were ripened at 20 °C.

In farm practice, fruits are harvested at a so-called ‘hard-green’ stage, not at eating-ripe stage. Eating-ripe-stage fruit are less physically robust than hard-green fruit. Although not of direct relevance to field harvesting, another set of fruit were gripped when at the eating-ripe stage to record damage under a ‘worst case’ scenario.

Skin damage was evaluated 1 h after gripping and flesh evaluation took place after 48 h at 20 °C (n = 30). For flesh-damage evaluation, fruit cheeks were removed and recut transversely at the point of grip to reveal their depth profiles. A rating of 0 (no damage), 1 (minor), or 2 (severe) was assigned to each evaluation, and a score for each treatment (gripper) was calculated as the percentage of the sum of all measurements divided by the maximum possible score.

3. Results and Discussion

3.1. Grasp Efficacy

The grasp efficacy of the various designs was characterised by the metric of picking area in the context of fruit size and the distance of fruit from the base plate when grasping was initiated. Examples of raw data for example situations are provided in Figure 5, and values for the picking area metric are recorded in

Table 2. Grasp success in term of the metric of ‘picking area’ for multiple end-effector designs using four fruit phantoms of varied masses. Mean is the average across all phantoms. Phantoms were placed such that the fruit ‘front’ faced the end effector. Design code refers to number of fingers (F), angle of top and bottom fingers (A), base plate height (H in cm), and grasp opening width (W in mm).

	Picking Area (cm2)
	Phantom Fruit Mass (g)
Design	378	512	636	836	Mean
Twister Fruit Picker	93	83	74	59	77.3
EZGripper™	66	64	45	11	46.5
Under-actuated 2	100	66	17	0	61.0
4F_0A_80H_175W	81	84	86	77	82.0
4F_30A_95H_150W	129	137	132	137	133.8
6F_0A_100H_180W	154	149	145	124	143.0
6F_15A_95H_175W	126	137	129	125	129.3
6F_30A_95H_175W	134	151	151	150	146.5
6F_30A_95H_190W	142	162	169	152	156.3

. The soft grippers outperformed other designs, and six-finger soft grippers outperformed the four-finger designs based on this metric. The 6F_0A_100H_180W-gripper picking area was 74% larger than that of 4F_0A_80H_175W, while it was only 25% greater in the vertical dimension.

Increasing the angle of the top and bottom fingers reduced the grasp performance of the gripper at extreme height positions (>20 or <−50). This was caused by the reduced grasp contact area between the gripper and the fruit (Figure 6). On the other hand, gripper designs with angled fingers achieved a superior grasping contact area at other heights (<20 or >−50, Figure 6). This characteristic increased the gripper tolerance to external disturbances, e.g., inclusion of stalk or leaves in the grasp of the fruit. This outcome was not tested for in the laboratory trials.

An illustration of the position of the fruit at two different distances from the gripper’s base plate (z positions) at the time of grasping and after grasping is provided in Figure 7. According to the metric of harvest success rate, the optimum z distance was −2 cm.

Two designs were further characterised in terms of the impact of fruit orientation to the gripper (as defined in Figure 2) on picking area (Figure 8 and

Table 3. Grasp success using the metric of ‘picking area’ (cm²) for two end-effector designs for fruit phantoms of different masses posed in three orientations facing the end effector. The label ‘na’ signifies ‘not attempted’.

		Fruit Orientation
Design	Fruit Mass (g)	Front	Side	Back
4F_0A_80H_175W	378	81	na	85
	512	84	na	73
	636	86	82	70
	836	77	86	79
Mean		82.0	84.0	76.8
6F_30A_95H_175W	378	158	na	154
	512	151	na	151
	636	151	151	157
	836	150	142	142
Mean		152.5	146.5	151

). Fruit orientation had little impact on grasp success, as defined by picking area and illustrated for two gripper designs (Table 3).

The two designs were clearly distinguished in terms of the metric of picking volume (Figure 9), with 6F_30A_95H_175W at 191% of 4F_0A_80H_175W.

The force required to remove a 626 g phantom mango from the 4F_0A_80H_175W and 6F_30A_95H_175W grippers was 3.1 ± 0.5 (mean ± SD) and 4.1 ± 0.1 kgf, respectively. Variability was higher with the four-finger design, i.e., the grip was less consistent than the six-finger design.

3.2. Orchard Trials

Failure to detach fruit from stalk during rotation occurred when the grasp included many non-fruit materials, i.e., leaves or inflorescence stalks. Most failures were due to the capture of leaves in the approach of the gripper to the fruit. In this situation, the fruit can slide within the encasing leaves as the gripper rotates, without stalk detachment, or with stalk detachment but loss of fruit from the gripper during rotation or retraction.

The metrics of ‘detachment success’ and ‘harvest success’ allowed comparison between orchards for the same gripper design and between different gripper designs for the same orchard. The higher harvest performance of the 4F_0A_80H_175W gripper in the Keitt-1 orchard compared to the Calypso and Keitt-2 orchards was consistent with a lower level of fruit occlusions by leaves, stems, and fruit stalks (panicle rachis) in the Keitt-1 orchard (

Table 4. Detachment and harvest success rate for fruit on tree for two end-effector designs at two orchards. Detachment refers to successful breakage of fruit from stalk. Harvest success refers to successful detachment and retraction of fruit as a percentage of attempted harvests.

Orchard	Design	Pick Attempts	Detachment Success (%)	Harvest Success (%)
Keitt-1	4F_0A_80H_175W	138	84	79
Keitt-2	4F_0A_80H_175W	73	77	56
	6F_30A_95H_175W	72	71	51
Calypso	4F_0A_80H_175W	98	67	54
	6F_30A_95H_175W	139	67	50

).

Use of standard metrics allows for inter-study comparisons. For example, Chen et al. [15] described the performance of a gripper in a harvest of apples (n = 25 fruit) in terms of metrics analogous to those of the current study. A ‘picking success rate’ of 100, 80, and 96% were reported in the use of rigid fingers, soft fingers with slip detection, and soft fingers with no slip detection, respectively. An in-orchard harvest rate of 70% was reported for apples [13]. In comparison, the four-finger design achieved a harvest success rate of 67–84% for mango fruit depending on canopy architecture in the current study.

3.3. Postharvest Evaluation

Fruits were evaluated in terms of skin condition immediately after gripping and in terms of skin and internal flesh conditions after ripening. The only skin condition visible immediately after gripping of un-ripe fruit was associated with the grasp of the EZGripper™ since the edge of the finger created a temporary mark on some of the fruit. These marks were not visible when assessed 1 h after gripping (

Table 5. Evaluation of skin and flesh damage (n = 30) for three gripper types. After unripe (hard green) fruits were gripped, skin damage was assessed after 1 h, and flesh damage was assessed after ripening of the fruit. After ripe (eating-stage) fruit were gripped, skin damage was assessed after 1 h, and flesh damage was assessed after 48 h. A rating of 0 (no damage), 1 (minor), or 2 (severe) was assigned to each evaluation along with a score for each treatment (gripper) calculated as the percentage of the sum of all measurements divided by the maximum possible score.

	Grasp at Hard Green		Grasp at Ripe
Gripper	Skin Damage (%)	Flesh Damage (at Ripe) (%)	Skin Blemish (%)	Internal Damage (%)
Twister Picker™	0	0	17	28
EZGripper™	0	7	17	42
6F_30A_95H_175W	0	0	0	8

). All fruits were also free of skin blemishes after ripening, but some flesh damage was associated with use of the EZGripper™ (Table 5). Flesh damage was manifest as bruising of flesh beginning approximately 3 mm below the skin surface.

Conversely, when ripened fruit were gripped, external markings were noted 1 h after gripping using either the EZGripper™ or Twister Picker™ (Table 5 and Figure 10a). Flesh damage, assessed after a further two days of ripening, was notable with the use of both the EZGripper™ and Twister Picker™ (Table 5 and Figure 10b).

Skin damage rates in fruit harvesting have been reported for other gripper-fruit commodity combinations, e.g., 6.3% (apple [4]), 30% (apple [42]), 25% (sweet pepper [43]), and 26% (kiwifruit [44]). Zhou et al. [45] reported a 17% flesh-bruising rate for a test involving placement of a branch between a gripper finger and (ripened) apple fruit. Chen et al. [15] described skin damage rates of 16, 0, and 8% associated with the use of rigid fingers, soft fingers with slip detection, and soft fingers with no slip detection, respectively. In the current study, skin and flesh damage rates of 0% were achieved for the harvest of mango fruit at harvest maturity.

4. Conclusions

The use of standardized metrics is recommended for use in all reports of the performance of automation of fruit harvesting. Performance metrics for gripper performance were proposed, including (i) ‘picking area’ and ‘picking volume’ in laboratory trials; (ii) ‘detachment success’ and ‘harvest success’ in orchard-harvest trails with (iii) postharvest assessment of ‘skin damage’ and ‘flesh damage’ (Table 6). Laboratory-based metrics allow for comparison under reproducible conditions, while orchard-based metrics confound hardware and canopy factors. By way of example, laboratory metrics were used in comparisons of nine gripper designs with grippers based on the FinRay™ flexible finger recommended for mango harvesting given that they achieved superior picking area and volume. Orchard metrics were used in a comparison of two gripper designs with a six- and a four-finger flexible finger design performing similarly regarding detachment and harvest-success parameters, and both outperformed non-flexible finger designs regarding ‘damage rate’.

The performance difference for the same gripper design used in different orchards is consistent with the need to structure orchards to suit automatic robotic harvesters rather than designing mechanical systems to suit existing mango orchards. This will involve both management, i.e., pruning, and cultivar selection, e.g., for genotypes with fewer fruit per inflorescence, and thus fewer clusters of fruit.

Other metrics can be proposed for inclusion in a specification list for a fruit harvester (Table 6). For example, the laboratory-based tests that were conducted do not consider the impact of inclusion of non-fruit material in the grasp of the fruit on harvest-success measurements. With leaves being the primary interferant, it is recommended that a secondary laboratory evaluation protocol include the presence of ‘standard’ leaves or stem material (number and position) with the grasp. Furthermore, the durability of grippers and arms are important since the harvest of a large farm involves millions of pieces of fruit. A metric measuring the failure rate of grippers and arms per 100,000 fruits harvested is suggested (Table 6). Other metrics can be proposed that are of relevance to farm management rather than engineering-design evaluation.

Table 6. Proposed metrics for evaluation of performance of a robotic fruit harvester.

Gripper Metric	Description
General metrics
Detachment success	Successful detachment of fruit from stalk as % of harvest attempts
Harvest success	Successful detachment and placement of fruit into a receptacle as % of harvest attempts
Laboratory assessment with phantom fruit
Picking area	Area of a plane parallel to and ahead of the gripper palm in which fruit will be successfully harvested at a given gripper palm-to-fruit distance
Picking volume	Volume of space ahead of gripper in which fruit will be successfully harvested
Grasp force	Force to pull fruit out of gripper
Orchard assessment with actual fruit
Cycle time	Time to harvest a single fruit by a single arm
Average cycle time	Average time to harvest a single fruit by a single arm including missed fruit and movement time between fruit
Durability	Gripper or arm breakage per 100,000 fruits harvested
Postharvest
Skin damage	% of harvested fruit with unacceptable level of skin damage
Flesh damage	% of harvested fruit with unacceptable level of flesh damage
Harvester metrics–orchard assessment
Harvest efficiency	% of fruit on trees that are successfully harvested
Harvest time	Time to harvest an orchard on a per-ha or per-bin basis
Power Consumption	kW per 100,000 pick cycles

Table 6. Proposed metrics for evaluation of performance of a robotic fruit harvester.

Gripper Metric	Description
General metrics
Detachment success	Successful detachment of fruit from stalk as % of harvest attempts
Harvest success	Successful detachment and placement of fruit into a receptacle as % of harvest attempts
Laboratory assessment with phantom fruit
Picking area	Area of a plane parallel to and ahead of the gripper palm in which fruit will be successfully harvested at a given gripper palm-to-fruit distance
Picking volume	Volume of space ahead of gripper in which fruit will be successfully harvested
Grasp force	Force to pull fruit out of gripper
Orchard assessment with actual fruit
Cycle time	Time to harvest a single fruit by a single arm
Average cycle time	Average time to harvest a single fruit by a single arm including missed fruit and movement time between fruit
Durability	Gripper or arm breakage per 100,000 fruits harvested
Postharvest
Skin damage	% of harvested fruit with unacceptable level of skin damage
Flesh damage	% of harvested fruit with unacceptable level of flesh damage
Harvester metrics–orchard assessment
Harvest efficiency	% of fruit on trees that are successfully harvested
Harvest time	Time to harvest an orchard on a per-ha or per-bin basis
Power Consumption	kW per 100,000 pick cycles