Exploring Fronts with Multiple Robots

When you build a house, you do not use only a hammer, but a variety of specialized tools in an organized and collaborative manner to construct a complex structure. The same idea holds true in oceanography, and as such, Principal Investigator João Tasso de Figueiredo Borges de Sousa of the Laboratório de Sistemas e Tecnologias Subaquáticas (LSTS) from Porto University and his team are working to construct a network that manages multiple autonomous vehicles simultaneously. At the Subtropical Front, located approximately 1,000 nautical miles off the coast of Southern California, the multidisciplinary team will work towards establishing a new method for observing dynamic, constantly changing ocean systems and processes with autonomous vehicles that are constantly in contact with themselves and a remote control center.

In oceanography, “Fronts” are sites where drastic changes occur within the properties of waters, such as areas where warm masses of water meet cold currents. In this area of vastly different conditions, the team will deploy several autonomous vehicles to find, track, and sample various physical, chemical, and biological features of the ocean, on time-space scales not previously possible by ship or aircraft studies alone.

The planned course of action builds on innovative open-source communication and autonomy software developed by LSTS that enables networked vehicle systems to operate independently while still working towards a common goal. A major component is on-board Artificial Intelligence (AI) based planning and execution control, which will be used to coordinate with autonomous underwater vehicles (AUV), unmanned air vehicle vehicles (UAV), and autonomous surface vehicles (ASV) to perform complex sampling tasks. At times, multiple vehicles will work in a combined effort.

A diagram of anticipated communication systems for multiple autonomous vehicles working at the same time.

The international team will be made up of researchers from LSTS and CIIMAR from Porto University in Portugal, Technical University of Cartagena in Spain, Norwegian University of Science and Technology, in Norway, and Harvard, University of Rhode Island, and Lamond-Doherty Observatory in the United States.

Research vessel Falkor will be an important part of the expedition as well, not just a passive launch pad. A serious obstacle in ocean robotics work is sending instructions and receiving feedback over long distances underwater – delays and disruptions are common using traditional transmission methods. Therefore, another goal is to develop more effective means of correspondence in such communications-challenged environments.

The Flightwave Edge Aerial Unmanned Vehicle (AUV) has a visible light camera, an infrared camera as well as running time of two hours. It also can achieve Vertical Take-Off and Landing (VTOL).

Falkor will act as a “nerve center” permitting mixed initiative planning and execution control by having human operators in the planning and execution of the control loops. This will happen by extending Falkor’s communications systems with underwater and radio communications (including satellite communications) to form networks over a wide area of operations. A shipboard Ocean Space Center (OSC) will be interfaced with Falkor as part of a ship-based integrated networked vehicle system (SINVS). This will allow several capabilities, such as real time updates of coordination and communication software and opportunities for the High Performance Computing System (HPCS) to be used for data assimilation and modeling. Last, there will be communications “nodes” formed by robotic vehicles that route data dynamically to other nodes based on network connectivity and the routing algorithm in use. The aerial, surface, and underwater vehicles will periodically connect these local networks to Falkor using Disruptive Tolerant Network (DTN) protocols.

Bruno Loureiro and Kanna Rajan getting ready to launch the Xplore -1 and -2 in a previous study, called Rapid Environment Picture (REP).

Unique to this mission, an anthropologist will be onboard looking at how best to organize the interactions among the engineers, scientists, and vehicles as the robots gain higher levels of intelligence and autonomy. This expedition will foster a deeper interdisciplinary understanding of oceanic fronts, eddies, and other hydrographic features on temporal-spatial scales not previously possible by using autonomous tools. Additionally, the team will be able to document, digest and report changes using vehicles with deliberative planning capabilities to accommodate intermittent communications in the field.

Special thanks and acknowledgment goes to these partners and collaborators:

MBARI – Provided technical support for the launch and recovery of a Wave Glider from Monterey bay and for in-water testing of all AUVs to be used in the cruise.
Portuguese Navy – Provided logistics support for at sea testing taking place in Portugal during the first five months of 2018.
Universidade de Aveiro – Contributed modeling and analysis techniques.
SINTEF – In-situ particle images from the top 100m of the water column were obtained by profiling the SINTEF SilCam.
NASA-Ames – Contributed a Dimethyl sulfide (DMS) sensor for airborne measurement of concentrations of this gas.

Ripples is a software tool we are using to monitor and control our underwater, surface, and air vehicles.

Ripples provides a web-based interface to interact with a team of autonomous vehicles. Ripples can be accessed by any compatible device (laptops, tablets or phones) and provides realtime information from all deployed assets.

By using Ripples, scientists (both on Falkor and on shore) can follow a mission’s execution. Researchers can monitor data being collected by the vehicles and other assets, in addition to data coming in from remote sensing and/or computer models. This simplified version of Ripples enables public access to all the data and plans in realtime.

lamont-doherty-earth-observatory-collaborator

Cruise Log

June 19, 2018

Data & Publications

Wave Glider and Saildrone data, hosted by University of Porto, can be accessed here.

Waveglider data hosted by NOAA’s ERRDAP server can be accessed here.

Saildrone data hosted by NOAA’s ERRDAP server can be accessed here.

The resulting shipboard dataset is being archived at Rolling Deck to Repository and is now available.

Water temperature, salinity, and other physical, biological, and remote sensing data collected by Falkor, waveglider, and aircraft during the cruise are archived with NOAA here.

Data, including microplankton analysis, from 126 CTD casts performed during the cruise is archived at Pangaea.

Genetic (amplicon) sequencing data from the water samples collected is archived at the European Nucleotide Archive.

Acoustic backscatter, navigation, and temperature and bathymetry from Falkor, and Fluid Chemistry, Conductivity, Fluoresence, Pressure, Temperature, and Turbidity from AUV is archived at Marine Geoscience Data System.

Nutrient Analysis Data from water samples are available through Pangaea.

ADCP data us curated and processed by University of Hawaii.

Tasso de Figueiredo Borges de Sousa, J. (2018). Mobile connectivity and mobile locality in networked vehicle systems: are we missing something? Oral Presentation at Control 2018, Sheffield, UK.
Pinto, J., Dias, P., Borges de Sousa, J., (2018). Coordinated operation of multiple AUVs, ASVs, and UAVs using the LSTS tool chain. Oral Presentation, IEEE OES AUV Conference, Porto, POR.
Belkin, I., Borges de Sousa, J., Pinto, J., Mendes, R., Lopez-Castejon, F. (2018). Marine robotics exploration of a large-scale open-ocean front. Oral Presentation, IEEE OES AUV Conference, Porto, POR.
Belkin, I., Borges de Sousa, J., Pinto, J., Mendes, R., and Lopez-Castejon, F. (2018). A new front-tracking algorithm for AUVs. Oral Presentation, IEEE OES AUV Conference, Porto, POR.
Costa, M., Pinto, J., Sousa Dias, P., Pereira, J., Lima, K., Ribeiro, M., Borges de Sousa, J., Lukaczyk, T., Mendes, R., Tomasino, and et. al. (2018). Field Report: Exploring Fronts with Multiple Robots. Oral Presenation, IEEE OES AUV Conference, Porto, POR.
30 day Preliminary Cruise Report: Exploring Fronts with Multiple Robots
Borges de Sousa, J. (2019). Adventures in Mobile Connectivity and Locality: Persistent Robotic Surveys. Invited Oral Presentation to Ericsson Research, Stockholm, Sweden.
Borges de Sousa, J. (2019). Adventures in Mobile Connectivity and Locality: Persistent Robotic Surveys. Invited Oral Presentation to Royal Institute of Technology, Stockholm, Sweden.
Borges de Sousa, J. (2019). Exploring Dynamic Features of the Ocean with Coordinated Robots. Oral Presentation at Breaking the Surface Event, Biograd, Croatia.
Borges de Sousa, J. (2019). Exploring Dynamic Features of the Ocean with Multiple Robots. Oral Presentation at EuroGEOSS workshop, Lisbon, Portugal.
Borges de Sousa, J. (2019). Exploring the Pacific Subtropical Front: Adventures in Coordinated Ship-robotic Surveys. Oral Presentation at Robots in the Wild: Challenges in Deploying Robust Autonomy for Robotic Exploration, Freiburg, Germany.
Borges de Sousa, J. (2019). From Coasts to Depths: Strengthening our Autonomous Presence in the Ocean. Oral Presentation at Workshop on European-funded Marine Robotics and Applications, Toulon, France.
Borges de Sousa, J. (2019). Multi-domain Vehicles for Earth Observation: What, Why, and How? Oral Presentation at GNSS International Summer School, Cerveira, Portugal.
Borges de Sousa, J. (2019). Sustainable Presence in the Ocean: Yes, we can! So Why Don't We Make It Happen? Oral Presentation at Ciencia Portugese Conference, Lisbon, Portugal.
Borges de Sousa, J. (2019). Veículos não tripulados para operações marítimas: passado, presente e futuro. Oral Presentation at IDN Seminarios, Faro, Portugal.
Borges de Sousa. (2019). Developing Networked Vehicle Systems for Naval Operations. Oral Presentation at Chiefs of the European Navies Conference, Lisbon, Portugal.
Borges de Sousas, J. (2019). Exploring the STF with Multiple Robots. Oral Presentation at Norwegian University of Science and Technology, Trondheim, Norway.
Borges de Sousas, J. (2019). Mobile Connectivity and Mobile Locality in Networked Vehicle Systems: Are we Missing Something? Oral Presentation at Norwegian University of Science and Technology, Trondheim, Norway.
Borges de Sousas, J., Mendes, R., Costa, M., Pinto, J., Dias, P., et. al. (2019). Field Report and Lessons-learned: Exploring Fronts with Multiple Robots, Oral Presentation at Encontro de Oceanografia, Peniche, Portugal.
Pinto, J. (2019). Networked Vehicle Systems: From Vision to Reality. Oral Presentation at International Conference on Communication, Computing and Digital Systems, Islamabad, Pakistan.
Mirmalek, Z. (2019). Meshing Remote and Co-Located Work: Human-Technology Relationships in Space and Ocean Exploration. Oral Presentation at MIT Media Lab, Cambridge, MA, USA.
Pinto, J., Costa, M., Lima, K., Dias, P., Pereira, J., Riberio, M., et al. (2021). To Boldly Dive Where No One Has Gone Before: Experiments in Coordinated Robotic Ocean Exploration. Conference Paper, International Symposium on Experimental Robotics, doi: 10.1007/978-3-030-71151-1_42. [This article has been published as OPEN ACCESS].

Porto researchers study Pacific with “an unprecedented detail”

TSF Radio Noticas • August 29th, 2018