Smart Port Shuttle: Sensor-Based Navigation for Inland Waterway Transportation ^†

Abstract

Germany has a wide-meshed waterway network. Although inland navigation is one of the most efficient transportation methods, its share of transport is steadily decreasing. The research project Smart-Port-Shuttle Hildesheim aims to strengthen the waterway system both from the aspect of inland navigation and from the aspect of ports. For this purpose, a holistic approach is chosen, which includes technological innovations on the one hand and integration into a logistical concept on the other. This paper focuses on the first point: a technological approach for a (partially) automated inland vessel. It will present the hardware setup and an evaluation of the sensors at a selection of critical points, such as in a lock, at a bridge passage, or during the passing of an obstacle. Different methods, such as Galileo E5a measurements for positioning or Simultaneous Localization and Mapping (SLAM) for constructing and updating the map of the branch canal, are analyzed. The data structure of the Robot Operating System (ROS) is presented, as the raw data are available to the audience.

1. Introduction

Germany’s inland waterways are an extensive network of rivers and canals that provide a crucial transportation infrastructure. They play a vital role in linking Germany’s industrial centers with its ports and other transportation hubs. Although inland navigation is one of the most efficient transportation methods, its share of transport is steadily decreasing [1]. In recent years, there has been a growing trend toward the use of partially automated transportation. This is seen as a promising solution to the challenges faced by the shipping industry, such as a shortage of skilled workers, increasing fuel costs, and the need to reduce emissions.

With their reduced complexity—no flowing water, only navigable in one lane, easily separable by the locks—demarcated branch canals in particular offer ideal conditions for the gradual automation of ship navigation. The challenges in planned (partially) autonomous shipping with regard to positioning are similar in the context of sensors for autonomous driving. The research project Smart-Port-Shuttle Hildesheim (SPS) [2] aims for a holistic approach that includes technological innovations on the one hand and the integration of a logistical concept on the other. This paper focuses on the first point: a technological approach to a (partially) automated inland vessel. The aim is to adapt existing sensors from the automotive industry to a vessel and to show technological ways that allow for partially autonomous shipping on German canals. For this purpose, the ship’s own control systems have to be analyzed, which include other timing aspects, for example. Therefore, a sensor setup was integrated on a test ship, see Figure 1, in order to run various test scenarios and to record and analyze the data.

Currently, the German inland navigation industry is undergoing a transformation, driven by factors such as digitization, automation, and sustainability. One of the main trends in the industry is the development of partially automated and autonomous inland vessels, which can help to increase efficiency, safety, and environmental sustainability. The Central Commission for the Navigation of the Rhine (CCNR) currently lists 40 pilot and research projects in the field of automation in inland navigation [3]. They cover a variety of aspects of the automation task, like positioning, navigation, motion control, control center development, establishing digital test fields, or the mobility chain.

The feasibility study “Autonomes Fahren in der Binnenschifffahrt” [4] showed the primarily associated opportunities for automated and (partially) autonomous driving in bin shipping. The researchers conclude that it can help to overcome the shortage of skilled workers, reduce costs, increase the level of safety, better link transport carriers to form intermodal and integrated transport chains, and include harbours more in the automatization while developing new processes. A test field is proposed to showcase and test automated and (partially) autonomous driving for ships and to use substantial synergy effects from other research projects. Regarding the sensor setup, they stated that the following sensors are suitable: echo sounders, radar, Global Navigation Satellite System (GNSS) receivers, cameras, Light Detection and Ranging (LiDAR), compasses, inertial sensors, and ultrasonic sensors. This is similar to the SPS setup, which utilizes radar, LiDAR, cameras, and GNSS receivers.

The SciPPPer project [5] aims to develop an assistance system for automated lock entry based on Precise Point Positioning (PPP) and a VHF Data Exchange System (VDES) for inland waterways. Their sensor setup includes the GNSS and near-field sensors (LiDAR), as well as reference systems on the waterway. To compensate for the shadowing effects and guarantee highly accurate positioning, the project team developed a new PPP protocol with a VDES. The SciPPPer technologies are now being further developed and the team is currently building a digital test field for inland vessels on the Spree–Oder waterway [6]. The project encountered some of the same difficulties that SPS was facing, but had a different approach due to their use of reference stations.

The SPS project did not plan to autonomously steer a vessel, but rather to define and showcase a sensor setup which is practical (since ships can be upgraded with it) and enables vessels to navigate in inland waterways and detect obstacles in the water as well as height restrictions, e.g., due to very low bridges. The recorded sensor data add a great value, as they can be reused and different approaches can be tested further.

This paper presents the hardware setup and the evaluation of the sensors at a lock in the Hildesheim canal. Different methods for positioning with GNSS or a SLAM for constructing and updating a map of the branch canal are analyzed. A discussion of the results and their implications and the outlook for the future conclude the paper.

2. Hardware Description and Design

This section provides an overview of the hardware specification and description, as well as the system architecture.

2.1. Hardware Specification and Description

In order to calculate a global position and time, the GNSS is the appropriate solution. Most vessels are already equipped with GNSS receivers to broadcast their position over the Automatic Identification System (AIS). In the scope of this work, a GNSS Receiver with Open Software Interface (GOOSE)© (software version: 1.26.0-71) [7] is used, which was developed by Fraunhofer IIS. This platform records raw GNSS measurements and calculates the corresponding Position, Velocity, and Time (PVT) solution. This receiver allow users to test different configurations according to the requirements.

During the measurement tests, two different data streams were recorded simultaneously. One contains the position and timing solution recorded via an ROS node and the PVT data in an Open GNSS Receiver Protocol (OGRP) format developed by Fraunhofer IIS. This approach allows us to visualize and analyze the data in post-processing. A multi-system and multi-frequency configuration of GPS L1/L5 and Galileo E1/E5a was used in testing. In order to determine the heading of the vessel, two GOOSE© [7] receivers were used, one placed on the head barge and one on the pusher. Each of them was connected to a Tallysman VSS6037L VeroStar™ [8] GNSS antenna.

In order to provide distances to potential obstacles during the ride as well as for docking and coupling maneuvers, LiDAR was selected as the sensor technology. These sensors allow not only the provision of 2D information like monocular cameras, but output a complete 3D representation of the sensor’s surroundings. The chosen Velodyne Puck VLP-16 [9] provides a 360° field of view and has a maximum range of approximately 100 m. Since a complete coverage of at least the entire bow of the vessel is required, one sensor was placed at either side of the bow. Using slightly tilted mounting angles for the two sensors, the detection accuracy in the area inside both fields of view was enhanced. After fusion of the individual point clouds, this representation could be used to either generate/enhance a map of the vessel’s surrounding and thus improve the vessel’s navigation or to detect obstacles on the planned route and thus enable an emergency response.

A radar system was deployed on the front and back of the vessel to detect obstacles and map the surroundings, such as bridges and walls. An AVR Quad DigiMMIC radar from Symeo GmbH [10] was mounted at the front of the vessel. This is a 77 GHz multiple-input-multiple-output (MIMO) evaluation platform based on four cascaded monolithic microwave-integrated circuits (MMICs) and has a total of 12 transmitter and 16 receiver channels. The propagation characteristics of electromagnetic waves at around 77 GHz come in especially handy in difficult visibility conditions like fog and drizzle. Depending on the operation mode, maximum measurement ranges of 250 m or more in a field of view of ±40° are possible. It is capable of deriving the distance from the round-trip time-of-flight to the target, the velocity from the Doppler shift, and the angle in the x-plane by evaluating the incident wave at the antenna array of the module. Furthermore, a Halo20 radar from SIMRAD [11] was deployed at the back of the vessel. It is a compact dome radar with pulse compression technology that detects collision hazards and other targets at close range and up to 24 nautical miles away.

In addition to the sensors from above, a monocular camera was installed at the front of the vessel, enabling visual inspection of the situation during the measurement campaign.

2.2. Hardware Architecture

Figure 2 shows the sensor setup on the test vessel. At the pusher, one Car-PC is connected to the HALO20 radar, to one GOOSE© receiver, and one of the Wireless Fidelity (WiFi) bridges. The GNSS antenna is centrally mounted on the rooftop of the pusher. The setup on the head barge consists of another Linux computer (Car-PC) for data processing, using a second WiFi bridge to connect the two PCs and thus all sensors to one network. The Symeo radar, a second GNSS antenna, and the camera were placed in the front-center of the head barge, whereas two LiDARs were mounted on the left and right of the head barge to enable a 360° view. All sensors on the head barge were aligned in the direction of motion. The Car-PC of the head barge as well as the GOOSE receiver were mounted in a waterproof box. The energy supply of the setup was ensured with a gasoline generator, since the head barge used for the measurement campaign does not provide electricity by itself.

2.3. System Architecture

Individual ROS nodes were developed for each sensor to convert raw sensor readings into ROS-compliant message types. The ROS framework [12] is a combination of a messaging middleware, predefined messages for various information related to sensor data and actor commands, and an expanding set of software modules (nodes) capable of generating, processing, and visualizing messages geared towards real-time robotic applications. As this open-source framework eliminates the need for implementing all required components from scratch, the development speed is greatly enhanced and project resources can be used for tuning the performance or enhancing critical components with alternative approaches.

One benefit of using an ROS is the ability to record all messages being passed in the system with temporal consistency in archives that can be used to replay the actual situation and test or debug each component, at least in an open-loop scenario where the scenario is not influenced by the processing in an ROS. One key concept of an ROS is the use of multiple coordinate systems called frames that allow for easy integration of different sensor measurements if the transformations between coordinate frames are known. Since the scenarios in the measurement campaign exclude dynamic vessel configurations, the transformations between sensors are assumed to be static and dynamic transformations are used to locate the vessel in its environment. The setup uses the latest version of an ROS, the Noetic Ninjemys [13] ROS; the main components are shown in Figure 3.

In the initial phase of semi-automatic operation, visualization of sensor data can assist the captain for an improved situational awareness by providing information about obstacles on a map. Towards the intermediate stages, sensors can not only visualize but also use the real-time data to track the current vessel state and compare it to its ideal trajectory to assist in steering the vessel. With full automation, the system not only collects data but also interprets and acts upon them without human intervention. This includes a full scan of the surroundings, identifying obstacles to avoid collisions, steering the vessel to follow an ideal trajectory on the canal, interpreting signs, connecting with locks and ports, and more.

3. Measurement Campaign

Based on the hardware architecture mentioned in Section 2.2, the setup was prepared and installed on the vessel as shown in Figure 4. The sub-figures clearly show how the various sensors were mounted on the vessel.

For the measurement campaign, different test scenarios were defined to cover the most challenging aspects for a cruise on the canal. The following enumeration summarizes the experiments and their goals.

• Lock: Detect the position where the vessel has to be secured.
  The lock, especially when its not flooded, is a challenging environment for GNSS due to the high multipath and low satellite visibility. Local sensors need to precisely detect where the vessel has to stop to be secured for the lock process.
• Bridge: Detection of bridge heights.
  Bridges on the canal have different heights. Due to the water level, the ship, and the cargo of the ship, it is necessary to detect the height to adjust the cabin on the main ship. This needs to be carried out as soon as possible to stop the vessel if it is to high to pass under the bridge.
• Canal: Detect the shoreline and possible obstacles.
  The vessel ideally cruises in the middle of the channel for efficiency. Therefore, the detection of the shoreline with sensors is required in order to keep the vessel in the center of the channel. Additionally, any obstacles, like barrels or other vessels, need to be detected.

In this paper, just a few examples are presented in order to show the possibilities of visualization and fusion of sensor data. However, the recordings of all experiments and the complete Hildesheim canal, from the harbour to the lock, are available and analyzed.

4. Results

A functioning hardware setup and data recording was one of the goals of the SPS project. Additionally, the recorded data were analyzed and different post processing methods were applied to showcase some approaches for future positioning in inland waterway canals. First, the GNSS data were processed and different frequencies were used to calculate the position. Second, all sensor data were used in a SLAM approach to fuse them for an overview of the vessel’s position and surroundings.

4.1. GNSS Positioning

Figure 5a shows the horizontal position solutions of the GOOSE© receiver using GPS L1 C/A and Galileo E1 combined measurements, and Galileo E5a. E5a results have fewer outliers compared to L1/E1 because of its signal characteristics [14]; it is more robust to multipath signals and interference. Here, the vessel moves into the lock from the south (bottom of the figure). The vessel enters the lock with the water level raised. Due to the high satellite visibility and little to no multipath effects, the position solution has no major outliers. After the vessel enters the lock, it is tied up to the left side of the lock. The lock process begins, and the water levels slowly drops. Due to the higher outer walls of the lock, interference and multipath effects occur, leading to outliers in the position solution. This scenario also has lower satellite availability since the walls limit the direct path of the satellites to the GNSS antenna. This effect continues to be observed when the vessel starts to move out of the lock.

A comparison of the skyplots in Figure 5b,c explains the low visibility of the satellites. The upper skyplot shows the satellites tracked for the whole cruise in the lock, whereas the lower one only shows the satellites tracked when the vessel is at the bottom of the lock. Low-elevation satellites in the upper plot can be seen as missing from the lower plot due to blockage from the walls. Nevertheless, for the whole duration, the GNSS position solution is available.

4.2. SLAM

Figure 6 shows the sub-meter resolution of the LiDAR point cloud when entering the lock, where small structures in the wall can be identified and the concrete shoreline is clearly visible. The grid for point cloud visualization uses 1 × 1 m cells; the sensor distance resolution is well below 10 cm.

Figure 7 displays a dynamic 3D map generated during cruising via OctoMap [15] on top of static map information. The gray rectangle in the center represents the vessel; the colors are mapped to the height information. Such a map can be refined by multiple passages and provides a high-resolution image of the canal and its surroundings, including vegetation and infrastructure. By using this map, control algorithms are able to plan well beyond the actual sensor range.

In the case of previously unknown obstacles in the water, the vessel can only adapt its course as soon as the sensors detect the objects. Depending on the vessel’s agility, additional long-range sensors might be needed to ensure successful collision avoidance. The sensor set used during the test campaign was reliably able to detect buoys used to mark narrow passages in the canal up to a distance of 75 m, see Figure 8.

The vessel’s speed and braking distance are highly dependent on the length and weight of the vessel and on the canal (width, depth, flow). In still waters—like the Hildesheim canal—the standstill will be against the water over a distance, measured against the land, of no more than 350 m for vessel convoys longer than 110 m and wider than 11.45 m, and 305 m for shorter and narrower convoys. Hence, a mixed sensor setup with, e.g., long range LiDAR, is needed to detect obstacles a higher distances.

5. Discussion

In order to fulfill the requirements for automated inland vessels, it is important to precisely and reliably locate the position of a vessel in a channel. Furthermore, the timely detection of surroundings and obstacles plays a vital role. For this project, sensors were used that have also been used in the development of systems for autonomous driving of cars. The used hardware setup is highly flexible and can be integrated and tested in any kind of vessel. The final hardware setup presented in this paper is the result of several iterations of tests in the canal and for each test cruise, a different vessel was provided. Each test cruise was used to optimize the setup and methodology. As far the project team knows, the sensors connected to one ROS have not been not mentioned by others. This setup enables synchronized recording on one centralized node for future data usage.

The test cruise recordings were used to evaluate positioning solutions with complex Galileo E5a signals. A dynamic 3D map was generated by fusing all sensor data. This generated map can be used in addition to a static map, in which the vessel is located purely via the GNSS. The exact requirements for using the dynamic map for later control purposes remain unclear, since, e.g., accuracy constraints heavily depend on the approach to controlling the vessel. For tasks like automated docking, a full map might not be needed, and instead, the free space or objects detected by LiDAR sensors can be used directly, since the required surroundings of the vessel can be observed by the sensors. This is also true for automated assistance in situations where dynamic/floating objects are present on the waterways and need to be avoided. Further analysis and visualization were performed for all defined experiments. An examination of how the data can be used for autonomous control of the vessel was outside the scope of this paper.

To keep inland shipping attractive and competitive, automation is an important step. Similar challenges to autonomous driving on roads also apply to inland navigation. Sensors may fail, the required infrastructure may not be available, or interfering signals may interrupt the reception of the GNSS or other signals. During the test cruises, a poor to non-existent network reception and interference in the E5 frequency band were the main problems encountered. However, it must be possible to reliably cover all these problems in the future. The sensor setup used in this project is modular and can be enhanced to have fall-back solutions, to fuse inertial sensors, and to use possible existing methods, like correction data from a reference station or Digital Audio Broadcasting (DAB)+, including 5G if available, e.g., at the port. The team is aiming to test the method to showcase the visualization in real time.