Tractor-Robot Cooperation: A Heterogeneous Leader-Follower Approach

Abstract

In this paper, we investigated the idea of including mobile robots as complementary machinery to tractors in an agricultural context. The main idea is not to replace the human farmer, but to augment his/her capabilities by deploying mobile robots as assistants in field operations. The scheme is based on a leader–follower approach. The manned tractor is used as a leader, which will be taken as a reference point for a follower. The follower then takes the position of the leader as a target, and follows it in an autonomous manner. This will allow the farmer to multiply the working width by the number of mobile robots deployed during field operations. In this paper, we present a detailed description of the system, the theoretical aspect that allows the robot to autonomously follow the tractor, in addition to the different experimental steps that allowed us to test the system in the field to assess the robustness of the proposed scheme.

1. Introduction

The agricultural sector is facing an ever-growing pressure to produce enough food for an increasing population, while maintaining a low level of environmental impact. The combination of both goals necessitates the inclusion of the latest advances in remote-sensing techniques to collect the data as well as the latest developments in agricultural machinery to mitigate the problems that the world is facing at present regarding having a sufficient food supply for everyone. Adding to this, the aging population in the agricultural sector is another factor that increases the need to look for newer alternatives to the conventional practices.

Robotics and artificial intelligent (AI) systems have contributed to the revolution of industrial processes for decades; however, the agricultural sector is still lagging behind, although it is a vital sector that is the foundation of human beings’ existence. The latest advances in robotics and AI should be gradually included, where these tools first assist the farmers in their daily tasks, before completely replacing them in a farmer-free scenario. The main purpose of this is to provide a solution that can be adopted in the near future, in which the farmer is still in touch with the daily activities of the field operations, and to work within the frame of the current legislation regarding the autonomous operations of robotic platforms, namely, the necessity of having a human supervisor that can take back control of the autonomous robot at all times.

Building upon the idea of using robots to assist farmers in their daily tasks, the cooperative nature of these type of systems needs to be addressed. Inspired mainly from nature (e.g., ants [1]), cooperation in robotic systems has attracted the attention of the mobile robotic research community in recent years, due to the great advantages that can be obtained by deploying multiple robots, where each robot can overcome the limitations of the other robots (perception, calculation abilities, payload, etc). Thus, the efficiency of the mission is increased without the need to have a single, complete, complex mobile robot. Although there is a large amount of related works dealing with the inclusion of mobile robots in an agricultural context [2], there is still a need for more efforts in the direction of cooperative robotic schemes in agriculture [3].

Autonomous navigation for single robots is the basis of any further development of fleet management. In the agricultural context, the authors in [4] present a hybrid approach to improving the autonomous navigation of mobile robots for field operations. The approach combines both geometric and semantic information to create a more comprehensive representation of the environment. By doing so, the method improves the accuracy of robot navigation and reduces the risk of collisions. In [5], the authors used a laser range finder model to allow for the autonomous navigation of a mobile robot in a maize field. The model uses a particle filter algorithm to estimate the position of the robot. The approach is effective in handling the uncertainty associated with the environment and the robot’s motion. In [6], the authors illustrated the development of an autonomous navigation controller for agricultural vehicles. The controller relies on a combination of sensors, including GPS and inertial measurement units, to guide the vehicle. The controller was tested and evaluated in different agricultural scenarios, showing promising results.

The use of different sensors to estimate the surrounding environment status is crucial when navigating in semi-clustered environments; the authors in [7] propose a local motion planner for autonomous navigation in grape yards using a red–green–blue depth (RGB-D) camera-based algorithm and deep learning. The approach combines the RGB-D camera and deep learning to recognize and navigate around obstacles. A light detection and ranging device (lidar) was used in [8] to develop an autonomous navigation system for under-canopy environments.The approach uses a combination of LiDAR and GPS sensors to accurately estimate the position of the robot. In [9], an autonomous navigation scheme of an agricultural robot is presented. The approach combines the global navigation satelite system (GNSS), real-time kinematic (RTK) and a Pixhawk [10] to improve the accuracy and reliability of the robot’s navigation. Another interesting work on autonomous navigation in an agricultural context can be found in [11]. The authors present an autonomous navigation system for agricultural machinery based on Linux-ARM technology. The system combines different sensors, including GNSS, compass, and gyroscope, to accurately guide the machinery. In [12], the authors developed a small-scale agricultural mobile robot prototype named Helvis for precision agriculture. The robot is designed to navigate autonomously in agricultural fields, collect data, and perform targeted agricultural tasks such as spraying or fertilizing. The authors in [13] discuss the development of a spraying robot designed for precision agriculture. This robot utilizes an edge-following approach to navigate through crop fields and apply pesticides with accuracy. The edge-following approach allows for the robot to adjust its path based on the irregular shape of crop fields.Equipped with sensors and a control system, the robot is capable of autonomously navigating through fields and applying pesticides with precision.

The recent advances in agricultural robots, whether in navigation, sensing, or overall scheme, have paved the way for the inclusion of multi-vehicles in the same field. The authors in [14] present a simulation environment for fleets of robots used in precision agriculture. The simulation environment allows for the testing and optimization of various fleet configurations and control algorithms. In [15], the authors describe a method used to assign tasks and planning trajectories for a group of cooperative agricultural robots. The method uses a combination of a genetic algorithm (GA) and multi-agent system (MAS) to optimize task assignments and trajectories for the robots. The proposed method was tested using simulated scenarios, and the results showed that it is able to effectively assign tasks and plan trajectories for the robots, which leads to an increase in the efficiency of the overall task performance. In [16] a multi-robot tractor system was designed and developed for use in agriculture field work. The system consists of multiple tractors that are capable of working together to perform tasks such as plowing, planting, and harvesting. The tractors are equipped with sensors and communication devices to enable them to coordinate their actions and share information. The system is designed to improve the efficiency of agricultural field work by allowing multiple tractors to work together to complete tasks. The authors of the paper have implemented and tested the system in a simulated environment, and the results show that it is capable of performing tasks effectively and efficiently. The authors in [17] present the development of an automatic navigation system for agricultural machinery. The system is designed to guide agricultural machinery, such as tractors and harvesters, along a predefined path without the need for human intervention. The proposed system uses a combination of GPS, inertial sensors, and machine vision to determine the position and orientation of the vehicle and to track the path to be followed. Communication is key to successful multi-vehicle navigation; the authors in [18] describe the development of a multi-vehicle cooperative navigation communication system based on TD-LTE. The system uses a combination of GPS and TD-LTE to achieve high-precision positioning, communication, and data transmission between vehicles. The system was tested in an agricultural field, focusing on evaluating the latency of the communication scheme. Precise pose estimation is another key to achieving desirable behavior when driving in the field; the authors in [19] proposed an integrated navigation system for agricultural machinery based on RTK-BDS/INS. The system used an RTK-BDS satellite navigation system and an inertial navigation system (INS) for accurate positioning and orientation. The experimental results show that the proposed system has good positioning accuracy and stability, making it suitable for agricultural machinery applications. The authors in [20] developed a leader–follower structure for harvesting purposes.This approach allows the group to navigate efficiently and effectively, particularly in challenging environments. The work in [21] proposes an approach to multi-vehicle navigation based on a dynamic inter-target distance matrix, which provides a stable and flexible solution for complex scenarios. The method combines centralized and decentralized control approaches to optimize the navigation task. The simulation results show that the proposed method achieved a better performance in terms of stability and flexibility compared to traditional methods. In [22], the authors present a method for controlling the formation of a group of agricultural mobile robots. The method uses a bidirectional weighted constraints approach, which allows for the robots to maintain a specific formation while also considering obstacles and other constraints in the environment. The proposed method was tested using simulations and experiments with real robots, and the results showed that it is able to effectively control the formation of the robots while also maintaining safety and avoiding collisions. A leader–follower tracking system was presented in [23]. The system allows for a lead vehicle to be followed by one or more follower vehicles, with the aim of improving the efficiency of agricultural field work. The system is based on a combination of a control algorithm and a communication system to enable the vehicles to track the lead vehicle and maintain a safe distance. The authors of the paper have implemented and tested the system in a simulated environment, and the results have shown that it is capable of effectively tracking the lead vehicle and maintaining a safe distance between the vehicles.

Although interesting, most of the related works assessed using simulated experiments, which limits the applicability of the proposed schemes in real-life conditions, especially when it comes to communication issues, controller design, navigation on uneven terrains, and the cooperative aspects of fleet navigation. In this paper, we cover the aforementioned aspects by taking the inclusion of a mobile robot alongside a conventional tractor as an example, and present the theoretical aspects, as well as the experimental results that were achieved in the fields of our research station, located in Kapp, Norway.

The rest of the paper is organized as follows: in Section 2 we provide a description of the different elements of the system. In Section 3, we present the control laws that allow for autonomous navigation inside the field to fulfill the leader–follower scheme requirements. In Section 4, we show the experiments that were preformed, followed by a discussion to assess the robustness of the proposed method. In Section 5, we conclude the paper and provide some guidelines regarding future possibilities to enhance the overall cooperation approach.

2. Materials and Methods

Hardware Description

The study-case that we are describing in this paper utilizes an electrical tractor (manned, leader) and an electrical robot-tractor (unmanned, follower). The tractor (Solectrac, CA, USA [24]) is powered by a 48 V lithium-ion battery with a capacity of 30 kWh. It can be fully charged in 7–8 h using a 240 v/30 A charger. The electric motor is a 48 V PM brushless type with a continuous output of 25 horsepower and a peak output of 50 horsepower. The tractor’s chassis is based on the Ford 7610 2001 model (Figure 1).

To obtain an accurate pose estimation (position and heading) of the tractor, we equipped it with a high-resolution lobal navigation satellite system (GNSS) receiver and a real-time kinematic (RTK) module (Piksi Multi Evaluation kit, SwiftNav, CA, USA [25]), and a high-precision inertial measurement unit (IMU) (VN-100, Vectornav, TX, USA [26]). The pose estimation was performed on an embedded computer (Jetson AGX Xavier, Nvidia, CA, USA [27]) running Ubuntu 20.04 LTS and the robot operating system (ROS Noetic) [28]. The results of the pose estimation were broadcast to the robot through wireless user datagram protocol (UDP) packages via the installed wireless router (Asus 4G-AC68U, ASUS, Taipei, Taiwan [29]). Figure 2 illustrates the hosting case for the tractor’s embedded computer, GNSS-RTK board, and the wireless router.

The mobile robot (referred to as “robot” in the remainder of the paper) is a differential drive vehicle (Figure 3) with a payload of maximum 750 kg. It is equipped with a three-point mount system similar to the one used by conventional tractors, which opens for a multi-purpose utilization. Eight (8) 12 V batteries power the robot. We redesigned the robot to meet our requirements. The driving wheels and their gearbox have were upgraded, and all the electronic parts were remade. The robot was run on ROS (Noetic) and equipped with the same RTK-GNSS receiver as in the one found on the tractor.

Table 1. Technical specifications of the robot.

Specification	Value
Dimension (L × W × H)	(2.13 × 2.6 × 1.16) m
Wheelbase	2.0 m
Wheels size (diameter/width)	0.54/0.2 m
Maximum linear velocity	2.78 m/s
Maximum angular velocity	9.87 rad/s
Instantaneous center of rotation	The intersection of the front wheelbase.
Onboard sensors	Lidar, RGB camera, IMU, GNSS-RTK receivers

illustrates the different technical specifications of the robot:

We initially relied on the onboard IMU of the robot to obtain the heading estimation; however, the data were not reliable and the drift was too big to be used as a reliable source for the autonomous navigation. To this end, we equipped the robot with two additional GNSS-RTK receivers (Figure 4) in order to obtain a more robust heading estimation with no drift over time.

3. Leader–Follower Scheme

Regardless of the domain of application, the inclusion of a human supervisor is necessary when dealing with autonomous robots [30] in real conditions. The main purpose of the present study is to propose a scheme that can be applied in the near future, even with the current regulations. The farmer will have access to the controlling unit of the robot at all times and, in the case of an incident, can take back control of the robot or halt its operations. With these constraints in mind, the field operations will be performed as a team composed of a leader (the manned tractor) and a follower (the robot).

The tractor is driven in a conventional way (moving from one row to another during field operations). As the leader (tractor) navigates inside the field, the follower takes the leader as a target and follows it, maintaining a predefined configuration; this allows the working width inside the field to be doubled with only one manned vehicle. In [31], we presented a leader–follower scheme for multi-vehicle navigation, where the leader was taken as a visual reference to the followers, and the group performed object transportation in an industrial context. Building on that work, in this paper we replaced the visual referencing aspects with a direct communication scheme which allowed for a more robust navigation, especially in outdoor environments, where the efficiency of vision can easily be reduced or even obstructed by natural elements.

Control Law Design

Autonomous fleet navigation in a leader–follower configuration suggests that the robot can take a reference point to the position of the tractor as the following target and follow it in an autonomous manner (Figure 5). To this end, the controller of the robot has to generate both the linear v and angular w velocities, which ensure that the error between the position of the robot and the tractor is reduced to the predefined values.

The reference frame of both the tractor and robot is represented in the universal transverse mercator (UTM) coordinates. We denote $F_{utm}$ as the reference frame where the planar navigation of the tractor and the robot occurred. The formation shape for the field navigation was decided prior to the mission, combining both the distance that separates the tractor from the robot and the side on which the farmer desires the robot to follow (right or left). The main objective of the control law is to ensure that the robot is navigating at the desired side, while holding the predefined distance at acceptable lower boundaries.

We consider the following errors along the X and Y axis in the reference frame $F_{utm}$. They represent the difference between the actual robot pose, and the reference pose of the desired configuration:

$$\begin{array}{c}\hfill E_d=\sqrt{{(x_t-x_r)}^2+{(y_t-y_r)}^2}\\ \hfill E_{\theta }={\theta }_t-{\theta }_r\end{array}$$

(1)

where $x_t$ and $y_t$ represent the UTM coordinates of the reference point of the tractor, while $x_r$ and $y_r$ are the UTM coordinates of the robot. ${\theta }_t$ and ${\theta }_r$ represent the heading of the tractor and the robot in the reference frame, respectively $F_{utm}$.

The aim of the controller is to regulate the distance and the angle that separate the robot from the reference point as follows:

$$\begin{array}{c}\hfill \underset{t\to \infty }{lim}{E}_d\left(t\right)=L_0\\ \hfill \underset{t\to \infty }{lim}{E}_{\theta }\left(t\right)={\theta }_0\end{array}$$

(2)

In Equation (2), the desired position of the reference point is defined by the polar coordinates ($L_0$, ${\theta }_0$). ${\theta }_0$ represents the predefined linear distance that separates the robot and tractor. ${\theta }_0$ represents the predefined bearing angle that is selected according to the following mode. In our experiments, ${\theta }_0=+\frac{\pi }{6}$ if we want the robot to follow the tractor from a position that is considered to be south-east to the position of the tractor, or ${\theta }_0=-\frac{\pi }{6}$ if we want the robot to follow the tractor from a position that is considered to be south-west to the position of the tractor.

From Equation (1), the error representation of both the distance $E_d$ and the angle $E_{\theta }$ are directly linked to the linear v and angular w velocities of the robot. To ensure that the conditions in Equation (2) are fulfilled, we propose the following non-linear controller that considers these errors and the required velocities:

$$\begin{array}{c}\hfill v=k_{pv}\ast E_d\\ \hfill w=k_{pw}\ast E_{\theta }+k_{dw}\ast \frac{E_{\theta }}{dt}\end{array}$$

(3)

where $k_{pv}$, $k_{pw}$, and $k_{dw}$ are positive gains.

4. Experimental Results

4.1. Static GNSS Waypoint Following

The tuning of the positive gains in Equation (3) was carried out manually. The tractor was parked so the broadcasted pose to the tractor was considered a static waypoint (this could have been replaced by providing a static GNSS waypoint, but we wanted to test the communication robustness before performing the field navigation). The process is as follows: each time the robot was driven manually to a random location, we switched on the autonomous navigation and let the robot navigate to the reference point while observing its behavior. Once the robot reached the reference location, the gains were tuned and the process was repeated (drive the robot to a different starting point, etc.).

In Figure 6, two examples of the tests performed to finetune the positive gains of the control law are presented (Equation (3)). The robot has two different starting values (in UTM coordinates): start pose 1 (602,037.1 North, 6,731,060 East), and start pose 2 (602,025.5 North, 6,731,065.8 East).

The robot used the pose of the tractor as the following goal, and tried to reduce the pose errors (Equation (2)) to the predefined values. The gains were then tuned to reduce the undesired behavior of the robot while driving and reduce the converging time to the desired pose (tractor location). The trajectory of the robot during these trials (starting positions 1 and 2) can be seen in Figure 7.

The distance and orientation of the robot to the tractor’s pose were recorded in order to assess the evolution of the errors of both the distance $E_d$ and the angle $E_{\theta }$ (Figure 8).

We can see from Figure 8 that, during the first example of the navigation from starting position 1, the distance shown in Figure 8a converged to the desired pose from the tractor (7 m from the back of the tractor); however, we can see from Figure 8b that the orientation is not stable. The second starting point was initiated after adjusting the gains of the controller and, this time, we can see that both the distance (Figure 8c) and orientation (Figure 8d) converged more smoothly to the desired values.

Table 2. Selected gains for the control law.

Parameter	Significance	Values
$k_{pv}$	positive proportional gain for the linear velocity	0.3
$k_{pw}$	positive proportional gain for the angular velocity	0.6
$k_{dw}$	positive derivative gain for the angular velocity	2.2

illustrates the selected gains for the control law (Equation (3)):

4.2. Dynammic GNSS Waypoint Following: Leader–Follower Configuration

Once we ensured that the robot showed acceptable behavior when navigating towards the fixed location of the tractor, we moved to the second phase. We performed field driving, where the tractor’s pose is no longer static, but dynamic. To this end, the tractor was driven manually throughout the field while broadcasting its position continuously to the robot. The robot, taking the tractor’s pose as a target, used the control law established in Equation (3) to reduce both the distance $E_d$ and the angle $E_{\theta }$ and fulfill the predefined configuration (driving to the right of the tractor at a distance of 7 m). Figure 9 illustrates a sequence where the tractor and robot performs a U turn while driving inside the field.

The aim of this experiment was to demonstrate the fleet navigation scheme inside the field. The tractor and the robot were both connected to same subnet via the wireless router, which was mounted on the the tractor. The tractor broadcast its position and heading continuously as UDP packeted. The robot listened to these packets and extracted the necessary linear and angular velocities that reduce both the distance and the angle that separates its current position from the desired following position. We performed several driving rounds in order to assess the robustness of both the communication scheme and the control law of the robot.

As the tractor directly sent the coordinates of the reference point (and not its own coordinates), the robot only had access to reference point coordinates and not the actual coordinates of the tractor. Since we only recorded the data from the robot, we only had the reference path (the evolution of the reference point during the navigation) and not the actual path of the tractor. Figure 10 shows the path of the robot and its reference path while navigating inside the field.

During field navigation, the tractor was driven without using cruise control for constant speed, and the navigation trajectory was chosen to emulate driving back and forth with U-turns in between. Figure 11 and Figure 12 show, at T1 = 87 s, T2 = 163 s, and T3 = 261 s, spikes in the distance that separates the robot from the tractor. These occurred when the tractor driver performed a u-turn (the spikes are the turning point at which the tractor’s driver is already performing the other half of the u-turn). Since the robot, at the time of writing this paper, has no special instructions when it comes to performing u-turns, the distance that separates it from the tractor increases because of the upper-velocity limit that imposed on the motors’ speed controllers for security reasons. At the end of the previously mentioned time spikes (t1, t2, t3), we can see that the distance converges again, which means that the robot is caught up with tractor, and followed it according to the predefined scheme.

The experimental results, whether following a static waypoint or driving in the field in a leader–follower scheme, state that the developed controller in this paper is able to satisfy the control objective (Equation (2)). However, we believe that adding some filtering functions [32] could contribute to achieving a much smoother navigation scheme and more efficient following of the robot.

5. Conclusions

In this paper, we presented a leader–follower scheme for an agricultural use-case. A tractor and a robot were used to demonstrate the ability of multi-vehicle cooperation in field tasks. The tractor was manned and driven manually, while the robot was unmanned and completely autonomous. The scheme relies on direct communication between the tractor and the robot, which allows for the continuous broadcasting of the pose of the tractor (position and heading) to the robot. The robot uses this information as input and follows the tractor in a predefined configuration. In this paper, we developed the necessary hardware and software (communication and control scheme) to allow the robot to fulfill this task and experimentally showed the viability of the proposed scheme. The work presented in this paper can be extended to include multiple followers, and a future direction may be the inclusion of more robots, developing the necessary layers to deal with navigation inside the field, and the management of the different tasks, such as reconfiguration, charging, and failure-proofing.