The science control room on R/V Falkor is the center of action, where you will find 18 individual computer screens (ranging in size and setup) displaying everything the ship is doing. Members of the science party keep an eye on two large screens that show the impressive software toolchain that controls all the autonomous assets on this expedition. It is the exact toolchain that researchers from the Underwater Systems and Technology Lab (LSTS) at Porto University have come to use out at sea.
Monitoring multiple AUV’s, UAV’s, a wave-glider, a sail drone and satellite data takes a unique tool box. Imagine controlling a jet airplane, a cargo ship, and a submarine—all at the same time. To make matters more complicated, every type of asset is built by different companies with different operating systems. And these machines need to send real time location data, planned routes, and precise times of when they will arrive at a certain point.
“Being aware of everything going on in the air, underwater, and making sense of data coming from models and actual surveys requires the special power of understanding what is going on in 4D – from the small, towards the big picture,” says Jose Pinto, a researcher at LSTS. “And that’s exactly what the Toolchain does.”
Key Ingredients
The LSTS Toolchain contains four important building blocks. (1) Dune is the back-seat driver for most of the vehicles. It is installed on a computer in each vehicle and works with the main operating system of the vehicle to send science data and receive mission plans from the control room. Dune is able to accept high level objectives and create mission plans and contingencies that take in to account conditions of the vehicle, boundaries of the mission, and conditions of the other vehicles.
(2) The IMC Protocol (Inter-Module Communication), is the language the vehicles use to speak with the rest of the Toolchain. “This is one of the important strengths of the LSTS Toolchain,” says Trent Lukaczyk, post-doctoral fellow at LSTS. “By building a common message dictionary we can control heterogeneous robotic fleets.” IMC can work over multiple types of links with multiple types of vehicles. On this expedition alone, the robots are sending IMC messages over links like long range Wi-Fi, serial telemetry, Iridium satellite, and satellite spot tracking. Other missions in the past have also used underwater acoustic modems.
(3) Neptus is the local command and control software for the scientists. “It’s the closest thing we get to a video game on this expedition. We set go-to points to and review the data that comes back from the vehicles overlaid on a map of the ocean,” says Trent. Using the Falkor’s matrix video system, the Neptus map is being displayed all around the ship – from bedrooms to the bridge. It helps keep everyone onboard aware of where everything is in the water.
(4) Ripples is the cloud communication hub that archives the mission data and makes it available via a web interface to anyone with an internet connection. While it’s possible for the scientists to use Ripples on the Falkor, the real power is seen when collaborating in real-time with researches on shore around the globe. As Jose Pinto puts it, “Ripples is the cloud repository for data, and it allows our colleagues back in Portugal to view underway data at the same time as us.”
Operating from the Falkor the LSTS tool chain lets one operator control multiple vehicles with complete situational awareness, in order to collect a comprehensive view of the ocean, and disseminate it in global collaboration. This is how we are advancing the way we conduct ship-based robotic operations.
