Conducting research with multiple underwater vehicles takes a huge amount of deliberation and coordination that is based on data gathered well before the work even takes place. But what happens if the environment you work on is constantly changing? Then the information you are basing your research on isn’t up to date. Additionally, there is a mismatch between the time it takes to digest collected data and the daily cycle of robotics deployment, limiting how well informed in-the-field decisions can be…until now.
Dr. Blair Thornton of the University of Southampton, and his international team of scientists and engineers are trying to address this problem using underwater robots and algorithms that are smarter than before. The group will perform deployments of multiple different types of underwater platform, along with concurrent in-situ chemical and biological sensing. While on board Falkor, the team plans to demonstrate how large volumes of data can be efficiently viewed and interpreted as part of a daily operational workflow, allowing for better informed decision making and increasing the scientific value of an at-sea expedition. The underwater robots used will not be just mindless gathers of data, but will make important decisions based on an increased awareness of their surroundings.
Not just intelligent but adaptive robots
The data collected will be used to update plans for underwater robot deployments on the fly, taking strategic approaches to gathering data in complex, dynamically changing environments like cold seeps. The information will be compiled onto a single geo referenced frame so humans, as well as machines, can recognize patterns that might be of interest and biological hot spots that we would want to know more about.
Hot spots in the deep
By developing tools for rapid visualization and machine-assisted interpretation, Dr. Thornton and his team will be able to focus observational efforts and generate a new framework that can cover larger ground with better resolution where it is most needed. This means biological hot spots will be easily identified providing a representative picture of the benthic ecosystem. The team will apply this concept by studying environmental aspects that shape the distribution of animal life in gas hydrate fields or “cold seeps.”
Gas hydrate fields and cold seeps sustain some of the richest known ecosystems on the seafloor. They only exist in cold, deep waters where the pressure is high. At the same time, hydrate fields cover vast regions, so the only way to study these systems and grasp an understanding of their influence on deep-sea ecology, is to make high-resolution observations over large areas. Understanding these systems is important for both benthic ecology, climate, and geo hazards research. Unfortunately, in the past we have not had the ability to intelligently investigate these systems in the time frames relevant to their dynamic changing environment.
Innovative Instrumentation
Barkley Canyon and Hydrate Ridge are the two sites that will be explored during this expedition. Since these are continuously changing environments, the AUV will be sent out first to complete a screening survey of the seafloor. Using three AUVs and ROV Subastian, the vehicles will undertake data-gathering deployments in real time looking at how the chemical, physical and biological properties in the water column influence (or is influenced by) the distribution of animals on the seafloor. An AUV will first produce 1 cm resolution 3D visual reconstructions of vast regions flying at a higher distance in the water column looking for patterns of interest.
Once biological and chemical hot spots are found, artificial intelligence (AI) will be used to tell another vehicle where to do higher resolution photo mosaicking, generating submillimeter resolution 3D visual reconstructions, allowing scientists to gather data in the most interesting regions and interpret them in a way that would not be possible otherwise. Hovering closer to the seafloor, the vehicles will cover a smaller, targeted area, making in-situ chemical and biological measurements using instruments such as laser spectrometers.
Having this level of detail where it matters most will help to illuminate relationships and advance the understanding of these environments. The deep-ocean is one of the most challenging ecosystems to deploy technology, and this work will be a small step towards improving our understanding of these environments, and a big step towards improving how effective we are a gathering information from these ecosystems.
Data & Publications
The resulting shipboard dataset is being archived at Rolling Deck to Repository and is now available.
Navigation, CTD and EventLogger Documentation, as well as Falkor Navigation are now available on MGDS.
Publicly annotated images can be found on Squidle+ here and here. [You must create an account (free) to view these images].
Point labels of Imagery are also publicly available in Squidle+.
Map of EM302 Bathymetry, Backscatter, and Navigation (combined with data from Hunting Bubbles cruise) can be viewed here. Data processed by UNH in partnership with SOI.
- 30 day Preliminary Cruise Report: Adaptive Robotics at Barkley Canyon
- Thornton, B. (2018). #Adaptive Robotics. Invited Lecture, 62nd Ocean and Seafloor Engineering Forum, Tokyo, Japan.
- Thornton, B. (2018). Whole site multi-resolution photogrammetric surveys of deep-sea vents and cold seeps. Invited Lecture, Beyond Challenger: A new age of deep-sea science and exploration meeting hosted by the Royal Society, London, UK.
- Thornton, B. (2019). 3D Visual Mapping and Rapid Understanding of Multi-hectare benthic habitats. Oral Presentation at Underwater Intervention, New Orleans, LA, USA.
- Thornton, B. (2019). Adaptive Robotics 2.0. Oral Presentation at Oceanology International, San Diego, CA, USA.
- Thornton, B., Yamada T., Bodenmann, A., Massot-Campus, M., Walker, J., et. al. (2019). 3D Visual Mapping and Rapid Understanding of Multi-hectare Benthic Habitats. Oral Presentation at Marine Autonomy Technology Showcase, Southampton, UK.
- Cappelletto, J., Massot-Campos, M., Bodenmann, A., Das, S., and B. Thornton. (2019). Micro-ballast Dispenser for Long Endurance Underwater Mapping Platforms. 2019 IEEE Underwater Technology, 1-8, doi.10.1109/UT.2019.8734317.
- Lim, J., Thornton B., and A. Prugel-Bennet. (2019). Unsupervised Clustering of Multi-resolution Seafloor Images for Efficient Scene Understanding, Oral Presentation at Marine Imaging Workshop, British Columbia, Canada.
- Walker, J., Thornton, B., Ruhl, H., Huvenne, V., and A. Prugel-Bennet. (2019). The Automation of Extracting Useful Statistics from Seafloor Imagery: A New Workflow. Oral Presentation at Marine Imaging Workshop, Victoria, British Columbia, Canada.
- Walker, J., Yamada, T., Prugel-Bennett, A., and B. Thornton. (2019). The Effect of Physics-based Corrections and Data Augementation of Transfer Learning for Segmentation of Benthic Imagery. 2019 IEEE Underwater Technology, 1-8, doi: 10.1109/UT.2019.8734463
- Yamada, T., Prugel-Bennett, A., and B. Thornton. (2020). Learning Features from Georeferenced Seafloor Imagery with Location Guided Autoencoders. Journal of Field Robotics, doi: 10.1002/rob.21961. [This article has been published as OPEN ACCESS].
- Edwards, J. (2020). Signal Processing Advances Undersea Research. IEEE Signal Processing pgs 7-10, Magazine, doi:10.1109/MSP.2020.2982856.
- Final Cruise Report: Adaptive Robotics at Barkley Canyon and Hydrate Ridge
- Marcon, Y., Kelley, D., Thornton, B., Manalang, D., and Bohrmann, G. (2021). Variability of Natural Methane Bubble Release at Southern Hydrate Ridge. Geochemistry, Geophysics, Geosystems, 22, doi: 10.1029/2021GC009894. [This article has been published as OPEN ACCESS].
- Walker, J., Prugel-Bennett, A., and Thornton, B. (2021). Towards Observation Condition Agnostic Fauna Detection and Segmentation in Seafloor Imagery for Biomass Estimation. Conference Technical Paper, OCEANS 2021, San Diego, CA, USA.
- Yamada, T., Massot-Campus, M., Prugel-Bennett, A., Pizarro, O., Williams, S., and Thornton, B. (2022). Guiding Labelling Effort for Efficient Learning with Georeferenced Images, IEEE Transactions on Pattern Analysis and Machine Learning, doi: 10.1109/TPAMI.2021.3140060. (Early Access). [This article is published as OPEN ACCESS with support from SOI].
In the News
Public offered free tours of research vessel
The Daily Astorian • August 9th, 2018
Artificial Intelligence Guides Rapid Data-Driven Exploration of Changing Underwater Habitats Mapped
Crossroads Today • August 29th, 2018
AI Guides Rapid Data-Driven Exploration of Changing Underwater Habitats
Marine Technology News • August 30th, 2018
Artificial intelligence guides rapid data-driven exploration of underwater habitats
Science Daily • August 30th, 2018
AI Helps Create One of the Largest 3D Maps of the Seafloor
Technology Networks • August 30th, 2018
Artificial Intelligence Guides Rapid Data-Driven Exploration of Changing Underwater Habitats Mapped
Portland Business Journal • August 30th, 2018
Artificial intelligence guides rapid data-driven exploration of underwater habitats
EurekAlert! • August 30th, 2018
Research: Mapped onto one of the world’s largest multiresolution 3D photogrammetric reconstruction of the seafloor
Tunisie Soir News • August 30th, 2018
Artificial intelligence guides rapid data-driven exploration of underwater habitats
PHYS.org • August 30th, 2018
A.I. Guides Data-driven Exploration of Underwater Habitats
Laboratory Equipment • August 31st, 2018
Artificial Intelligence Guides Rapid Data-Driven Exploration of Changing Underwater Habitats Mapped onto one of the World’s Largest Multiresolution 3D Photogrammetric Reconstruction of the Seafloor
Institute of Industrial Science, University of Tokyo • August 31st, 2018
Artificial Intelligence Guides Rapid Data-Driven Exploration of Changing Underwater Habitats Mapped
NBC 2 • August 31st, 2018
Artificial Intelligence Guides Rapid Data-Driven Exploration of Changing Underwater Habitats Mapped
News on 6 • August 31st, 2018
Using Artificial Intelligence and Autonomous Robotics for Rapid Exploration of Deep-Sea Ecosystems
AZO Robotics • September 3rd, 2018
AI Leads Data-Driven Survey of Changing Underwater Habitats
Subsea World News • September 4th, 2018
Rapid data analysis at sea leads to largest 3D color reconstruction of the seafloor
University of Southampton News • September 13th, 2018
Artificial Intelligence Guides Rapid Data-Driven Exploration of Changing Underwater Habitats
ECO Magazine • September 17th, 2018
Rapid data analysis at sea leads to largest 3D color reconstruction of the seafloor
UK-RAS Network • September 26th, 2018
Artificial Intelligence Guides Rapid Data-Driven Exploration of Changing Underwater Habitats
Age of Robotics • September 28th, 2018
Data-Driven Exploration of Changing Underwater Habitats (Full magazine link)
ECO Magazine • September/October 2018
Underwater Photonics (That technology aims for the universe! What is marine photonics that illuminates the unknown seabed)
Optronics Magazine (ISSN 0286-9659) • October 2018
AUVs provide Largest-Known 3D Colour Reconstruction of the Seafloor
Ocean News & Technology • October 2018
Send in the Drones: New Tools for Coral Reef Research
Coral Magazine • November 6th, 2018