Non-Cooperative Vessel Detection Using Multispectral Satellite Imaging and Machine Learning Applied Towards Onboard Satellite Dark Vessel Detection

Abstract

Dark vessels are a major source of economic and environmental damage, estimated to cost the global economy tens of billions (USD) a year. These vessels hide or obscure their presence on the ocean to conduct unlawful activity. Current satellite non-cooperative vessel detection methods rely on terrestrial processing of data from large synthetic aperture radar satellites, and small visible imaging satellites. Multispectral imaging could bridge gaps in the capabilities offered by these methods. Furthermore, the development of powerful onboard computers for small satellites has enabled applied onboard machine vision to automate and optimize the vessel detection process.

The study investigates whether the additional spectral bands in satellite multispectral imagery, beyond traditional RGB ones, provide detection benefits for marine vessels in the seas or oceans. Additionally, it also examines the suitability of multispectral imagery to detect vessels with object detection convolutional neural network (CNN) models and evaluating onboard processing of the models on representative single board computers. The process included the data fusion of multispectral satellite imagery and automatic identification system (AIS) data, to create a dataset which was used to train machine vision object detection models. The models were trained upon 39 permutations of spectral bands and model size and evaluated within the context of four simulated orbits and imaging constraints based on terrain elevation and coordinates.

For the trained CNN models, it was found that using multispectral imaging improved the ability to detect vessels present by up to 10% compared to using only RGB imaging. This improvement was not uniform with different spectral band permutations varying considerably. The inference time penalty for using multispectral imaging was found to be no more than 4 ms per image compared to RGB inferencing time. Despite this, the multispectral models were found to be suitable for near real-time processing when imaging constraints relevant to vessel detection were utilized.