Next Article in Journal
Effect of Vanadium and Rare Earth on the Structure, Phase Transformation Kinetics and Mechanical Properties of Carbide-Free Bainitic Steel Containing Silicon
Next Article in Special Issue
Multi-Robot Leader Grouping Consistent Formation Control Method Research with Low Convergence Time Based on Nonholonomic Constraints
Previous Article in Journal
Bound-State Soliton and Noise-like Pulse Generation in a Thulium-Doped Fiber Laser Based on a Nonlinear Optical Loop Mirror
Previous Article in Special Issue
Intelligent Trust-Based Utility and Reusability Model: Enhanced Security Using Unmanned Aerial Vehicles on Sensor Nodes
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Development of a Control System for Multilink Manipulators on Unmanned Underwater Vehicles Dynamically Positioned over Seafloor Objects

Institute of Marine Technology Problems, Far Eastern Branch, Russian Academy of Sciences (FEB RAS), 690091 Vladivostok, Russia
*
Author to whom correspondence should be addressed.
Appl. Sci. 2022, 12(3), 1666; https://doi.org/10.3390/app12031666
Submission received: 30 December 2021 / Revised: 3 February 2022 / Accepted: 4 February 2022 / Published: 5 February 2022
(This article belongs to the Special Issue New Trends in Robotics, Automation and Mechatronics (RAM))

Abstract

:
This article considers an approach to synthesizing a multilevel system to control movements of a multilink manipulator (MM) mounted on an unmanned underwater vehicle (UUV) for performing autonomous manipulative operations in the mode of dynamic positioning over various objects on the seafloor. The system is based on an upgraded method that, using point clouds received from the machine vision systems (MVS), provides high-accuracy determination of the shape and location of the work object relative to UUV. The preset trajectories of the MM working tool are overlaid on the identified surface of the object, with possible silting, fouling or deformation of the latter taken into account. To execute the programmed trajectories with the MM working tool, the following methods have been implemented in software: stabilization of UUV in hovering mode near the object, high-precision control of working tool’s movements, and also corrections of its trajectory taking into account UUV’s displacements relative to the object. The synthesized system has been developed in the C++ programming language. The operation of the system has been numerically simulated using a model of UUV with MM, as well as models of the environment and the target object, in the Matlab/Simulink and V-REP software packages. The results of the study show a high efficiency of the system both in processing sensor information and in providing the dynamic control of movements of an UUV with a MM.

1. Introduction

The necessity for increasing the efficiency of deployment of autonomous or hybrid unmanned underwater vehicles (UUV) with multilink manipulators (MM) for surveying large areas, detecting marine objects and performing manipulative operations with them has spurred the development of a multilevel system that would provide control over complex dynamic objects. The operation of robotic vehicles with such a system does not require a special support vessel or numerous maintenance personnel; additionally, their movements are not limited to the length of the umbilical cable, and they can travel dozens of kilometers in search of target objects [1,2,3]. A MM mounted on an UUV significantly extends the range of its functional capabilities to also include sediment sampling from specified areas, collection of marine organisms, installation of beacons and various equipment and their cleaning and maintenance [3,4,5,6].
Researchers have designed a number of methods and algorithms for automating various manipulative operations of UUV. As shown in Figure 1, these methods can be divided into a few major groups.
The methods explained in the studies [7,8] provide an evaluation of the roughness of the bottom topography in the working area and make a decision as to whether the bottom is suitable for safe manipulation by the time the UUV approaches the object. Their efficiency has been confirmed experimentally. Prior to the operations, the object is identified on the basis of data received from the onboard sonars [9,10], and then the UUV is sent to the object. Simultaneously, a site for stabilized hovering of the UUV is selected in such a way that the working zone of the manipulator allows correct execution of the required operations and avoidance of collisions of the vehicle with the bottom or object [8].
Efficient methods have been developed to keep the UUV in the hovering (or dynamic positioning) mode near the object [11,12,13,14] using the vehicle’s thrusters, these methods provide compensation for the negative dynamic effects from unknown underwater currents and from the working manipulator. In this case, the force and torque effects from the MM are calculated in real time mode by the algorithm for solving the inverse dynamics problem (IDP) for underwater MM [13]. The parameters of manipulator links’ interaction with the aquatic environment included in this algorithm are determined experimentally by the method [15].
The shape and location of the target object relative to the UUV are determined with high accuracy using the data received from the multibeam echosounder sonars and stereo cameras, which constitute an onboard machine vision system (MVS) [16,17,18]. Various MVS that produce output data in the form of point clouds have long and widely been used in both industrial and mobile robotics [19]. The major advantages of such systems are the high-speed response, high accuracy and high scanning density, which allow reconstructing a 3D model of the workspace on the vehicles’ onboard computer. However, the underwater environment somewhat restricts the use of laser scanners or stereo cameras as MVS, due to optical distortions and possible high turbidity of the water. Nevertheless, with special algorithms for filtering and processing distorted information applied, optical MVS can also be used underwater [20]. Before processing the obtained point clouds, the photo and video images pass through an image processing system [21,22]. The latter makes it possible to recognize the object with approximately known dimensions and color in an obtained image using special color filters. This significantly accelerates the process of search and identification of a previously known object by devices with lower computing capacities.
To build trajectories of movements of the MM working tool, special methods are required for various manipulative operations over the bottom surface and certain objects with already known shapes [8,18,23,24,25]. These methods take into account the positions of the object’s surface, the MM’s working zone and the angle of approach to the object relative to each other. In particular, the method [25] allows building complex spatial trajectories of the MM working tool on the object surface using point clouds obtained from MVS. However, it is unsuitable for automated manipulations, since the type of trajectory and its location on the object are determined by the operator and set by pointing the UUV camera optical axis.
The MM working tool should execute the programmed trajectories with specified accuracy. This necessitates using the method [26] that, thanks to the introduction of self-adjusting correction devices (SCD) of the manipulator’s electric drives, compensates for the negative interactions between all its degrees of freedom that occur during movement at a high speed in the aquatic environment, and also for the coulomb and viscous friction torques in the drives. These interactions are calculated by the recurrent algorithm for solving IDP [13]. The introduction of additional observers provides identification of unknown external torques on the shafts of the respective electric drives.
During a manipulative operation, the UUV inevitably, although slightly, shifts from its initial position, even if a system of stabilization in the hovering mode is used [11,12,13,14]. As a result, the MM working tool deviates from the programmed trajectories, thus, reducing the operation accuracy. To maintain the specified accuracy in the work [27], a method for continuous automatic correction of programmed trajectories of MM’s movements is proposed, which takes into account actual displacements of UUV relative to its initial position.
When the MM grips a massive object, the UUV acquires undesirable list and trim angles under the effect of its weight. A lot of studies have addressed the issue of controlling movements of UUV exposed to external disturbances ([11,28], etc.). In particular, a method has been proposed in the work [28], which allows adjustment of thrusters’ propulsions depending on the information about the difference between the programmed and current UUV movement vectors that come from a gyroscope and a Doppler velocity log.
An analysis of the above-mentioned methods and algorithms shows that a substantial basis has already been created for synthesizing an efficient multilevel control system of UUV to perform automatic manipulative operations. However, the known methods for determining the shape and location of an object relative to UUV have a number of disadvantages that restrict the range of their applications. In particular, the method [18] suggests alignment of the point clouds received from MVS with the clouds formed on the basis of previously constructed 3D models of the object. However, the point clouds of the scanned object obtained in the model were evenly distributed over the object’s surface, which is impossible in actual conditions due to unidirectional scanning. Furthermore, the alignment of the point cloud of the constructed model with that of the actual object scanned needs to be tested for accuracy. A decision about the further execution of the trajectory built on the object’s surface should be made on the basis of the test results. However, when forming desired trajectories of the MM working tool, this method does not take into account the probability of silting, fouling and deformation of the object, which can negatively affect the accuracy of manipulative operations.
In this article, we propose an upgrade of the method [18] to eliminate these weaknesses. Based on this method for manipulations over an object (Figure 1), as well as on the previously considered ones [13,15,18,26,27], a goal is set to synthesize, implement and study, through numerical modeling, a control system of UUV that would provide accurate automatic manipulative operations.

2. Improving the Accuracy of Determination of Work Object’s Shape and Location

As noted in the Introduction, the high-accuracy determination of the shape and location of an object relative to UUV is carried out in the stabilized hovering mode [11,12,13,14]. The undersea surroundings are scanned by the MVS units [16,17] to form clouds of points distributed over the bottom and object surfaces. The point clouds are set in a rectangular coordinate system (CS) Oxyz rigidly fixed to the UUV hull, with the origin located at the center of buoyancy of the vehicle, the Ox axis coincides with the horizontal, longitudinal axis of the UUV, the Oz axis coincides with its vertical axis and is directed downward, and the Oy axis complements the other two axes to form the right-hand CS.
To identify a certain known object in the point cloud obtained from MVS, the work [18] suggests using a 3D model of this object, preprocessed and transformed into a point cloud set in the CS Oxyz. This cloud, consisting of m points K q R 3 ( q = 1 , m ¯ ), first, is virtually approximated to the cloud of points belonging to the actual object through the linear transfer in the direction of the vector p 1 R 3 , whose magnitude and direction are determined by the systems of detection of underwater objects that stand out from the surrounding [9,10]. Then the displaced cloud, defined by the points P q = ( K q + p 1 ) , is aligned with the point cloud of the object using the iterative closest point (ICP) algorithm [29] implemented in the open-access Point Cloud Library (PCL) [30].
In the case the point clouds are aligned by the standard PCL tools (see Figure 2), the vector p 2 R 3 and the matrix M r 1 R 3 × 3 are calculated. These are required for the cloud alignment as they describe, respectively, the linear displacement and rotation of the model point cloud relative to the axes of the CS O x y z . Consequently, for any point P q belonging to the point cloud of the object model, a point G q can be obtained that defines the position of the point P q after the two clouds are aligned [18]:
G q = H P + M r 1 l q + p 2
where H P = 1 m q = 1 m P q is the midpoint (center of mass) of the point cloud of the object model, and l q = ( P q H P ) is the vector that defines the position of any point P q relative to the point H p of this cloud.
Accuracy of cloud alignment is evaluated by the method of least squares for the nearest points of these clouds at each iteration of the ICP algorithm. Therefore, the presence of “excess” points in the displaced object model (which are absent from the scan of the actual object due to the unidirectional scanning by MVS) often causes errors in alignment of the centers of mass of the two clouds. Nevertheless, studies have shown that correct spatial orientation of the object model’s cloud is usually achieved even in such cases (Figure 2).
To increase accuracy of the point clouds alignment, some of the points G q of the object model lying on the surface of this model and invisible to MVS should be discarded. Thus, we assume that a preliminary alignment of clouds allows setting a spatial orientation of the object model that would match quite precisely the orientation of the actual object. To do this, for each point G q , calculate the angle φ q = arccos ( n q v q | n q | | v q | ) lying between the normal vector n q = ( G q H G ) , connecting the center of mass of the cloud H G = 1 m q = 1 m G q with the point G q , and the vector v q = ( G q S ) , where S R 3 are the coordinates of the CS origin of MVS in the CS Oxyz. Discard the points G q for which the inequality φ q π / 2 is true (Figure 3).
After discarding the excess points, the clouds are aligned again by the ICP algorithm, and the vector p 3 R 3 and matrix M r 2 R 3 × 3 are calculated that describe the subsequent linear displacement and rotation of the model’s point cloud relative to the SC O x y z . As a result, similarly to Equation (1), the coordinates of the points D q of the aligned point cloud of the object model are found by the following expression:
D q = H G + M r 2 n q + p 3

3. Testing the Accuracy of Point Clouds’ Alignment and Building the Trajectories of Movement of MM Working Tool over Object Surfaces

To test the accuracy of the point clouds’ alignment performed, it is suggested to compare two projections of the same test trajectory: a projection onto the triangulation surface of the scanned object and a projection of this trajectory onto the triangulation surface of the object model aligned. To do this, the point cloud D q and the point cloud of the object obtained from MVS should be represented as triangulation surfaces [17,31] consisting of a multitude of triangular polygons, where adjacent ones have common edges and vertices. In the CS O x y z , each triangle of the object model surface is defined by a set of three vertices V k 0 , V k 1 , V k 2 R 3 , where k = 1 , g ¯ , g is the number of triangles, and each of the w triangles of the scanned object surface is defined by vertices V h 0 , V h 1 , V h 2 R 3   ( h = 1 , w ) ¯ .
To project the test trajectory built in the CS O x y z onto both triangulation surfaces, it is necessary, first, to define the direction of the projecting rays by a single vector o = ( H D S ) / H D S from the point S of the CS origin of MVS to the center of mass H D = 1 m q = 1 m D q of the object model’s point cloud (Figure 4). The test trajectory should run on a plane perpendicular to the vector o , and its configuration can be selected depending on the known dimensions and shape of the object. As an example, define the trajectory as a sequence of n points lying on the same straight line c i , ( i = 1 , n ¯ ) and find intersections of the rays extending from these points in the direction of the vector o with both triangulation surfaces (Figure 4).
As a result of the intersections, sets of points C i and C i are formed, which are the projections of the test trajectory onto the surfaces of the actual and simulated objects, respectively.
The coordinates of the points C i and C i can be calculated using the modified Möller–Trumbore algorithm [25] in the following form:
C i = c i + ( ( c i V k 0 ) × ( V k 1 V k 0 ) ) ( V k 2 V k 0 ) ( o × ( V k 2 V k 0 ) ) ( V k 1 V k 0 ) o C i = c i + ( ( c i V h 0 ) × ( V h 1 V h 0 ) ) ( V h 2 V h 0 ) ( o × ( V h 2 V h 0 ) ) ( V h 1 V h 0 ) o
where (·) is the scalar product of vectors; (×) is the vector product of vectors.
The trajectory projections consist only of the coordinates of the points C i and C i and, therefore, they can be transmitted to the operator’s control console even via a hydroacoustic communication channel with a low bandwidth. The operator can visually evaluate the locations of these trajectories relative to each other, and then either confirm execution of the manipulative operation or give a command to re-scan and re-align the clouds.
For a UUV having no communication channel with its operator, criteria should be set out to evaluate adequacy of the previously performed alignment. Standard deviation of the coordinates of the projected trajectories’ points can be used as such a criterion: Q = i = 1 n E i 2 , E i = C i C i (Figure 5).
The threshold value of the criterion Q depends on the dimensions of the object and the resolving power of MVS. Studies have shown that the threshold value for the simulated MVS and object is Q m a x ; if it is exceeded, the alignment can be considered erroneous. In this example of successful alignment of point clouds, the value amounts to Q = 0.000346 .
When the performed test shows that the object model’s point cloud is precisely aligned with the object’s cloud, the trajectory of the MM working tool built on the original object model can be transferred to the actual object. This trajectory can be initially defined both in an analytical form and as a sequence of points X j R 3 ( j = 1 , s ¯ , where s is the number of points) with the respective single vectors a j R 3 and b j R 3 defining the desired orientation of the working tool at the jth point. With Equation (2) taken into account, the trajectory transferred to an actual object can be expressed as follows:
X j = H D + M r 2 M r 1 ( X j H K )
a j = M r 2 M r 1 a j , b j = M r 2 M r 1 b j ,
where Xj are the points with the respective orientation vectors a j and b j that define the desired positions of the MM working tool on the object surface, and H K = 1 m q = 1 m K q is the center of mass of the object model’s cloud of initial points K q .
In cases where the exact shape of an object cannot be known in advance, or there is a high probability of its deformation, fouling or silting, the desired trajectory of the MM working tool (4) can additionally be projected onto the triangulation surface of this object constructed using MVS. To do this, similarly to Equation (3), points Xj are projected in the positive and negative directions of the vector o , because these points can be located both outside and inside the triangulation surface of the object. As a result, a sequence of Tj R 3 points is formed to be located on the surface of the actual object:
T j = X j ± ( ( X j V h 0 ) × ( V h 1 V h 0 ) ) ( V h 2 V h 0 ) ( ± o × ( V h 2 V h 0 ) ) ( V h 1 V h 0 ) o .
To provide smooth movements of the MM working tool along the trajectory, the sequences of points Xj or Tj can be smoothed by parametric 3rd-order B-splines using the algorithm [32].

4. Software-Based Implementation of the System and the Mathematical Models of UUV and MM

The above-considered upgraded methods for processing sensor-derived information and building trajectories of a MM working tool were implemented in the C++ programming language. The created program makes it possible to build triangulation models of surfaces of seafloor objects by the algorithm [31]. The input data for the program are a multitude of points lying on the bottom and object surfaces obtained using MVS, and also the point cloud of the object model. Based on the input data, the program starts a cycle (iterations) of object identification and, as a result, values of H D , M r 1 , M r 2 are determined. Then the alignment of point clouds is tested, and, in case of successful identification of the object, the program transfers the sequence of points X j pre-set in the CS O x y z and their respective vectors a j and b j onto the surface of the actual object.
A graphical interface using PCL has been created to display and test alignment of point clouds and build MM working tool trajectories (Figure 6). Using visual information, the operator can additionally evaluate the results of the system operation.
A system operability test was performed through numerical simulation in the virtual environment of the V-Rep robot simulation software. A scene of one manipulative operation to clean an object from fouling is shown in Figure 7. As MVS, we used a model of multibeam echosounder sonar with a resolution of 32 × 32 dots and unidirectional scanning. This resolution is sufficient to perform operations with the target object. The sequence of points X j (or T j ) and their respective vectors a j and b j of the MM working tool, formed by the software, were smoothed by parametric 3rd-order B-splines [32] and sent to the Matlab/Simulink software package via the TCP protocol. The programmed signals to control the MM electric drives of the respective degrees of freedom were formed on the basis of these sequences.
In the simulation, we used the well-tested mathematical model of UUV [33] and MM with the PUMA kinematic scheme having the first five degrees of freedom (three in translation and two in orientation). The basic parameters of the UUV and the manipulator are shown in Table 1. A complete list of parameters of the UUV model is provided in the works [34,35]. The MM model used and its parameters are described in detail in [26]. This model includes a recurrent algorithm for solving IDP [13], which takes into account not only mutual influences in the MM degrees of freedom, but also specifics of the effect of a viscous fluid on the moving links of MM and the experimentally determined parameters of this effect [15]. The MM attachment point in the SC O x y z had coordinates M = (0.6, 0.6, 0.45)T (Figure 7).
In our work, we used the dynamic models of UUV and MM implemented in Matlab/Simulink. Therefore, in order to simplify interactions of the subsystems included in the created multilevel control system of UUV (Figure 1), the methods for stabilizing the UUV in the hovering mode near the object, controlling its movements and correcting the MM working tool trajectory, with undesirable displacements of the UUV taken into account, were also implemented in Matlab/Simulink.
To stabilize the UUV in the hovering mode, we used a combined system [13,14] that provides compensation for the effect of sea currents and the dynamic effects of the moving manipulator, calculated in real time, using the propulsion thrusters of the vehicle. To maintain the specified accuracy of MM working tool’s movement along the trajectory in conditions of small but unavoidable displacements of the UUV in the stabilization mode, the method [27] was implemented in the system. On the basis of information about linear and angular displacements of the UUV from the initial position, this method allows finding coordinates of the vector x R 3 that defines the current desired position of the MM working tool on the object surface in the CS Oxyz as follows:
x * = R T ( x r ) ,
where   R = [ C θ C ψ C θ S ψ S φ S θ C φ S θ S φ + C θ S ψ C φ S θ C ψ C θ C φ + S θ S ψ S φ S θ S ψ C φ C θ S φ S ψ C ψ S φ C ψ C φ ]
is the matrix of rotations of the CS Oxyz relative to the absolute CS, calculated taking into account the directions of its axes; S() = sin(); C() = cos(); φ , ψ , θ are the roll, yaw and pitch angles of the UUV, respectively; r R 3 is the vector defining linear displacements of the UUV in the absolute CS; x * R 3 is the vector defining the current desired position of the working tool’s characteristic point in the CS Oxyz.
To provide accurate movements of the working tool along programmed trajectories, SCD [26] have been synthesized. These SCD generate control signals of the MM electric drives taking into account the estimated external torques in the respective drives, and also the variations in the viscous and coulomb friction torques in the drives that are identified by the additional observers.
Figure 8 shows a block diagram of the subsystem for the dynamic control of movements of an UUV and its manipulator implemented in Matlab/Simulink. The acronyms used in the figure are as follows: TCB is the block for correcting the trajectory of the MM working tool [27]; MM and its SC are the model for the manipulator and the system of its control [26]; f R 3 , m R 3 are the vectors of, respectively, force and torque effects of the manipulator on the UUV calculated by the IDP solving block [13]; UUV SS is the stabilization system [13] that forms the resulting vector of thrusts and torques τ R 6 created by the UUV thrusters to compensate for the vectors of the force f ˜ R 3 and torque m ˜ R 3 effects from the manipulator; UUV NS is the navigation system of the UUV; x ˜ is the vector of the actual position of the MM working tool in the CS Oxyz.

5. Results of Numerical Modeling

When determining the work object’s shape and location, we compared the results of using the standard ICP algorithm with those of the proposed method based on discarding points invisible to the sonar. The quality of alignment of the point clouds for these two cases was tested by projecting the test trajectory onto the triangulation surfaces of the object and its model (Figure 9). This trajectory consisted of a set of points c n , ( n = 1 , 100 ¯ ) spaced at 0.01 m intervals along two perpendicular straight lines lying in a plane perpendicular to the vector o . The directions of these straight lines were defined by the vectors F 1 , 2 R 3 :
F 1 , 2 = o × k o × k ± o × k × o o × k × o ,   k = ( 0 , 0 , 1 ) , c n = ± 0.01 i F 1 , 2 F 1 , 2 ,   i = 1 , 25 ¯ .
Then we found the intersections of the rays extending from the points c n in the direction of the vector o with both triangulation surfaces (Figure 9) using Equation (3). An assessment of the accuracy of point clouds alignment showed values Q 1 = 0.093826 and Q 2 = 0.013175 , respectively, for the standard ICP algorithm and the proposed method. According to this assessment criterion, the accuracy increased 7-fold.
A sinusoidal trajectory with an amplitude of 0.2 m was preliminarily overlaid on the object model (Figure 7). After accurate identification of the object (Figure 9), the trajectory was transferred to its surface and smoothed by 3rd-order B-splines (Figure 10). The origins of the absolute CS and the CS Oxyz were assumed to coincide with the initial moment of time ( t = 0 ). The MM working tool was moving at a constant speed of 0.3 m/s along the resulting trajectory; the total movement time was 8.4 s.
Figure 11 (curve 1) shows the Euclidean norm r of the vector r of the UUV’s linear displacement from the initial point when the working tool is moving along the programmed trajectory without the combined stabilization system; curve 2 represents values of the same parameter with this system employed. As can be seen from the figure, the introduction of a stabilization system allows accurately holding the UUV in a desired position near the object, which reduces unwanted UUV’s displacements from 65 mm to 7 mm. Thus, the deviation of the angular coordinates of the UUV without a dynamic positioning system reaches 0.05 rad; with this system used, the value is no greater than 0.015 rad.
Figure 12 shows graphs of variations in the Euclidean norm ε of the vectors ε R 3 of MM working tool’s deviations from a programmed trajectory in the absolute CS for three cases of simulation: without the UUV stabilization system (curve 1), with the UUV stabilization system employed (curve 2), and with the systems of UUV stabilization and correction of programmed working tool’s trajectories (curve 3). In the first case (Figure 12, curve 1), the ε value reached 7 cm. After the introduction of the UUV stabilization system, the ε values were no greater than 2 cm (Figure 12, curve 2). The use of an additional correction of programmed working tool’s trajectories (Figure 12, curve 3) reduced deviations almost 10-fold in most segments of the trajectory. However, in curved segments, where the pattern of manipulator links’ movements changed, the ε value reached 1 cm for a short time due to an increase in the dynamic error of tracking the working tool along the trajectory. With the synthesized SCD of the MM drives employed, the error was no greater than 9 mm in the CS of the MM.
Thus, the results of numerical simulation have confirmed the operability of the synthesized system that provides high accuracy of operations executed by the MM working tool of a UUV in the automatic mode. Stable functioning of the dynamic control subsystem (Figure 9) is provided by known and well-tested methods and algorithms [13,26,27]. These systems allow quite efficient compensation of external disturbing effects on the accuracy of UUV stabilization and MM operation.
It should be noted that the synthesized system needs geometric shapes of target objects to be known prior to the operation. However, for many manipulative operations, this is much easier than setting special markers on an object in advance, as suggested in [22]. Unlike the known methods based on image processing [4,21], the proposed system allows accurate identification of objects that are subject to silting, fouling and deformation. Furthermore, MM trajectories can be built even on the surfaces of objects invisible to the sonar.
As compared to the method [25], which uses point clouds to identify objects in a similar way, the synthesized system makes it possible to perform operations that do not require an operator’s intervention. A significant advantage of the system is also the opportunity to use the underwater acoustic channel of communication with the UUV to transmit to the operator the information necessary for assessing the accuracy of object identification and building the MM trajectories to the operator.

6. Conclusions

The designed multilevel control system for automatic manipulative operations of a UUV with a MM allows, on the basis of data received from onboard MVS, accurate overlaying of any already known trajectory of the MM working tool onto the scanned surface of the target object. This does not require an operator’s direct control over the system operation. Instead, the operator has an opportunity to make a decision as to whether the preset trajectories, transmitted via an underwater acoustic channel of communication with the UUV, are correctly overlaid onto the object. The specified accuracy of the automatic execution of the programmed trajectories by the MM working tool is provided by the use of the methods for stabilizing the UUV in the hovering mode near the object, high-precision control of the working tool’s movements and also correction of the working tool’s trajectory to compensate for undesirable displacements of the UUV relative to the object. The software-based implementation will facilitate the introduction of the system in currently existing UUV.

Author Contributions

Conceptualization and methodology, A.K.; software, A.Y. and P.P.; formal analysis, A.K. and N.K.; investigation, A.Y., N.K. and P.P.; writing—original draft preparation, A.K., A.Y. and N.K.; writing—review and editing, A.K., A.Y., N.K and P.P.; visualization, A.Y., N.K. and P.P.; project administration, validation, supervision and funding acquisition, A.K. All authors have read and agreed to the published version of the manuscript.

Funding

This study was funded by the Ministry of Science and Higher Education of the Russian Federation, grant no. 13.1902.21.0012 “Fundamental Problems of Study and Conservation of Deep-Sea Ecosystems in Potentially Ore-Bearing Areas of the Northwestern Pacific” (agreement no. 075-15-2020-796).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Request to corresponding author of this article.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Yuh, J.; Marani, G.; Blidberg, D.R. Applications of marine robotic vehicles. Intell. Serv. Robot. 2011, 4, 221–231. [Google Scholar] [CrossRef]
  2. Konoplin, A.; Borovik, A.; Mikhailov, D.; Vaulin, Y.; Scherbatyuk, A.; Boreiko, A.; Babaev, R.; Bolovin, D.; Tregubenko, D. Application of Autonomous Underwater Vehicles for Research of Ecosystems in the Southern Ocean. In Antarctic Peninsula Region of the Southern Ocean: Oceanography and Ecology, 1st ed.; Advances in Polar Ecology Series; Springer: Berlin/Heidelberg, Germany, 2021; ISBN 978-3030789268. [Google Scholar]
  3. Simpkins, C.A. Introduction to Autonomous Manipulation: Case Study with an Underwater Robot, SAUVIM [On the Shelf]. IEEE Robot. Autom. Mag. 2014, 21, 109–110. [Google Scholar] [CrossRef]
  4. Marani, G.; Choi, S.K.; Yuh, J. Underwater autonomous manipulation for intervention missions AUVs. Ocean Eng. 2009, 36, 15–23. [Google Scholar] [CrossRef]
  5. Juan, J.J.F.M.P.; Pedro, C.G.R.M.; Penyalver, J.S.A.; Fresnada, F.; Fresnada, J.F.; Prats, M.; Sánchez, J.C.G.; Marin, R.; Sanz, P.; Peñalver, A. Manipulation in the Seabed: A New Underwater Manipulation System for Shallow Water Intervention. IFAC Proc. Vol. 2012, 45, 314–319. [Google Scholar] [CrossRef]
  6. Cieslak, P.; Ridao, P.; Giergiel, M. Autonomous underwater panel operation by GIRONA500 UVMS: A practical approach to autonomous underwater manipulation. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA, 26–30 May 2015; pp. 529–536. [Google Scholar] [CrossRef]
  7. Dukan, F.; Sørensen, A.J. Sea floor geometry approximation and altitude control of ROVs. Control Eng. Pract. 2014, 29, 135–146. [Google Scholar] [CrossRef]
  8. Filaretov, V.F.; Konoplin, N.Y.; Konoplin, A.Y. System for automatic soil sampling by AUV equipped with multilink ma-nipulator. Int. J. Energy Technol. Policy 2019, 15, 208–223. [Google Scholar] [CrossRef]
  9. Inzartsev, A.; Pavin, A.; Panin, M.; Tolstonogov, A.; Eliseenko, G. Detection and Inspection of Local Bottom Objects with the Help of a Group of Special-Purpose AUVs. In Proceedings of the 2018 OCEANS-MTS/IEEE Kobe Techno-Oceans (OTO), Kobe, Japan, 28–31 May 2018; pp. 1–6. [Google Scholar] [CrossRef]
  10. Guerneve, T.; Subr, K.; Petillot, Y. Three-dimensional reconstruction of underwater objects using wide-aperture imaging SONAR. J. Field Robot. 2018, 35, 890–905. [Google Scholar] [CrossRef]
  11. Antonelli, G. Underwater Robots; Springer: Berlin/Heidelberg, Germany, 2014. [Google Scholar] [CrossRef]
  12. Vu, M.T.; Le Thanh, H.N.N.; Huynh, T.-T.; Thang, Q.; Duc, T.; Hoang, Q.-D.; Le, T.-H. Station-Keeping Control of a Hovering Over-Actuated Autonomous Underwater Vehicle Under Ocean Current Effects and Model Uncertainties in Horizontal Plane. IEEE Access 2021, 9, 6855–6867. [Google Scholar] [CrossRef]
  13. Filaretov, V.F.; Konoplin, A.Y. System of automatic stabilization of underwater vehicle in hang mode with working multilink manipulator. In Proceedings of the 2015 International Conference on Computer, Control, Informatics and Its Applications (IC3INA), Bandung, Indonesia, 5–7 October 2015; pp. 132–137. [Google Scholar] [CrossRef]
  14. Vu, M.T.; Le, T.-H.; Thanh, H.L.N.N.; Huynh, T.-T.; Van, M.; Hoang, Q.-D.; Do, T.D. Robust Position Control of an Over-actuated Underwater Vehicle under Model Uncertainties and Ocean Current Effects Using Dynamic Sliding Mode Surface and Optimal Allocation Control. Sensors 2021, 21, 747. [Google Scholar] [CrossRef]
  15. Filaretov, V.; Konoplin, A. Experimental Definition of the Viscous Friction Coefficients for Moving Links of Multilink Underwater Manipulator. In Proceedings of the 26th DAAAM International Symposium, Zadar, Croatia, 15–25 October 2016; Volume 1, pp. 762–767. [Google Scholar] [CrossRef]
  16. Joe, H.; Kim, J.; Yu, S.-C. Sensor Fusion-based 3D Reconstruction by Two Sonar Devices for Seabed Mapping. IFAC-PapersOnLine 2019, 52, 169–174. [Google Scholar] [CrossRef]
  17. Marton, Z.C.; Rusu, R.B.; Beetz, M. On fast surface reconstruction methods for large and noisy point clouds. In Proceedings of the 2009 IEEE International Conference on Robotics and Automation, Kobe, Japan, 22–25 August 2009; pp. 3218–3223. [Google Scholar] [CrossRef] [Green Version]
  18. Konoplin, A.Y.; Konoplin, N.Y.; Shuvalov, B.V. Technology for Implementation of Manipulation Operations with Different Underwater Objects by AUV. In Proceedings of the 2019 International Conference on Industrial Engineering, Applications and Manufacturing (ICIEAM), Sochi, Russia, 25–29 March 2019; pp. 1–5. [Google Scholar] [CrossRef]
  19. Pomerleau, F.; Colas, F.; Siegwart, R. A Review of Point Cloud Registration Algorithms for Mobile Robotics. In Foundations and Trends R© in Robotics; NOW Publishers: Boston, MA, USA, 2013; Volume 4, pp. 1–104. [Google Scholar] [CrossRef] [Green Version]
  20. Bruno, F.; Lagudi, A.; Barbieri, L.; Rizzo, D.; Muzzupappa, M.; De Napoli, L. Augmented reality visualization of scene depth for aiding ROV pilots in underwater manipulation. Ocean Eng. 2018, 168, 140–154. [Google Scholar] [CrossRef]
  21. Rizzini, D.L.; Kallasi, F.; Aleotti, J.; Oleari, F.; Caselli, S. Integration of a stereo vision system into an autonomous underwater vehicle for pipe manipulation tasks. Comput. Electr. Eng. 2017, 58, 560–571. [Google Scholar] [CrossRef]
  22. Sivčev, S.; Rossi, M.; Coleman, J.; Dooly, G.; Omerdic, E.; Toal, D. Fully automatic visual servoing control for work-class marine intervention ROVs. Control Eng. Pract. 2018, 74, 153–167. [Google Scholar] [CrossRef]
  23. Youakim, D.; Ridao, P.; Palomeras, N.; Spadafora, F.; Ribas, D.; Muzzupappa, M. MoveIt!: Autonomous Underwater Free-Floating Manipulation. IEEE Robot. Autom. Mag. 2017, 24, 41–51. [Google Scholar] [CrossRef] [Green Version]
  24. Youakim, D.; Ridao, P. Motion planning survey for autonomous mobile manipulators underwater manipulator case study. Robot. Auton. Syst. 2018, 107, 20–44. [Google Scholar] [CrossRef]
  25. Konoplin, A.; Filaretov, V.; Yurmanov, A. A Method for Supervisory Control of Manipulator of Underwater Vehicle. J. Mar. Sci. Eng. 2021, 9, 740. [Google Scholar] [CrossRef]
  26. Filaretov, V.F.; Konoplin, A.; Zuev, A.; Krasavin, N. System of High-precision Movements Control of Underwater Manipulator. In Proceedings of the 31st DAAAM International Symposium, Mostar, Bosnia and Herzegovina, 21–24 October 2020; DAAAM International: Vienna, Austria, 2020; pp. 0752–0757. [Google Scholar] [CrossRef]
  27. Filaretov, V.; Konoplin, A. System of Automatically Correction of Program Trajectory of Motion of Multilink Manipulator Installed on Underwater Vehicle. Procedia Eng. 2015, 100, 1441–1449. [Google Scholar] [CrossRef] [Green Version]
  28. Filaretov, V.; Konoplin, A. Method of Synthesis of Automatic Correction Systems of Underwater Vehicles Linear Displacements. Procedia Eng. 2015, 100, 1434–1440. [Google Scholar] [CrossRef] [Green Version]
  29. Besl, P.J.; McKay, N.D. Method for registration of 3-D shapes. In Proceedings of the Sensor Fusion IV: Control Paradigms and Data Structures, Boston, MA, USA, 14–15 November 1991; pp. 586–606. [Google Scholar]
  30. Interactive Iterative Closest Point: Documentation of Point Cloud Library. Available online: http://pointclouds.org/documentation/tutorials/interactive_icp.php (accessed on 1 November 2021).
  31. Point Cloud Library: Fast Triangulation of Unordered Point Clouds. Available online: http://ns50.pointclouds.org/ (accessed on 1 November 2021).
  32. Filaretov, V.; Gubankov, A.; Gornostaev, I. The formation of motion laws for mechatronics objects along the paths with the desired speed. In Proceedings of the 2016 International Conference on Computer, Control, Informatics and Its Applications (IC3INA), Jakarta, Indonesia, 3–5 October 2016; Institute of Electrical and Electronics Engineers (IEEE): Manhattan, NY, USA, 2016; pp. 93–96. [Google Scholar]
  33. Fossen, T.I. Handbook of Marine Craft Hydrodynamics and Motion Control; John Wiley & Sons: Hoboken, NJ, USA, 2011. [Google Scholar]
  34. Herman, P. Numerical Test of Several Controllers for Underactuated Underwater Vehicles. Appl. Sci. 2020, 10, 8292. [Google Scholar] [CrossRef]
  35. Vervoort, J. Modeling and control of an unmanned underwater vehicle. Master Traineesh. Rep. 2009, 5–15. [Google Scholar]
Figure 1. Block diagram of a multilevel control system of UUV for automated manipulative operations.
Figure 1. Block diagram of a multilevel control system of UUV for automated manipulative operations.
Applsci 12 01666 g001
Figure 2. Preliminary alignment of point clouds.
Figure 2. Preliminary alignment of point clouds.
Applsci 12 01666 g002
Figure 3. Filtered (green) and discarded (red) point clouds near the point cloud of the object surface.
Figure 3. Filtered (green) and discarded (red) point clouds near the point cloud of the object surface.
Applsci 12 01666 g003
Figure 4. General view of the trajectory, projection vectors and triangulation surfaces.
Figure 4. General view of the trajectory, projection vectors and triangulation surfaces.
Applsci 12 01666 g004
Figure 5. Trajectories projected onto the surfaces of the object (red) and its model (green).
Figure 5. Trajectories projected onto the surfaces of the object (red) and its model (green).
Applsci 12 01666 g005
Figure 6. A graphical interface displays the processes of object identification and overlaying a working tool trajectory onto the object surface.
Figure 6. A graphical interface displays the processes of object identification and overlaying a working tool trajectory onto the object surface.
Applsci 12 01666 g006
Figure 7. A scene in the V-REP simulator.
Figure 7. A scene in the V-REP simulator.
Applsci 12 01666 g007
Figure 8. Block diagram of the subsystem for the dynamic control of UUV and its manipulator.
Figure 8. Block diagram of the subsystem for the dynamic control of UUV and its manipulator.
Applsci 12 01666 g008
Figure 9. Results of point clouds alignment.
Figure 9. Results of point clouds alignment.
Applsci 12 01666 g009
Figure 10. A trajectory of movement of the MM working tool over the object surface.
Figure 10. A trajectory of movement of the MM working tool over the object surface.
Applsci 12 01666 g010
Figure 11. Magnitudes r of UUV’s linear displacements.
Figure 11. Magnitudes r of UUV’s linear displacements.
Applsci 12 01666 g011
Figure 12. Magnitudes ε of MM working tool’s deviations from the programmed trajectory.
Figure 12. Magnitudes ε of MM working tool’s deviations from the programmed trajectory.
Applsci 12 01666 g012
Table 1. Parameters of UUV and MM.
Table 1. Parameters of UUV and MM.
ParametersValues (Unit)
UUVL × H × W 1.2 × 1.5 × 0.9   ( m 3 )
Mass170 (kg)
Link 1Length0.05 (m)
Mass0.4 (kg)
Link 2Length0.5 (m)
Mass4 (kg)
Link 3Length0.5 (m)
Mass4 (kg)
Link 4Length0.1 (m)
Mass0.5 (kg)
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Konoplin, A.; Yurmanov, A.; Krasavin, N.; Piatavin, P. Development of a Control System for Multilink Manipulators on Unmanned Underwater Vehicles Dynamically Positioned over Seafloor Objects. Appl. Sci. 2022, 12, 1666. https://doi.org/10.3390/app12031666

AMA Style

Konoplin A, Yurmanov A, Krasavin N, Piatavin P. Development of a Control System for Multilink Manipulators on Unmanned Underwater Vehicles Dynamically Positioned over Seafloor Objects. Applied Sciences. 2022; 12(3):1666. https://doi.org/10.3390/app12031666

Chicago/Turabian Style

Konoplin, Alexander, Alexander Yurmanov, Nikita Krasavin, and Pavel Piatavin. 2022. "Development of a Control System for Multilink Manipulators on Unmanned Underwater Vehicles Dynamically Positioned over Seafloor Objects" Applied Sciences 12, no. 3: 1666. https://doi.org/10.3390/app12031666

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop