Marine mammals
Developing a broadly applicable, drone-based system for monitoring seal colonies along the entire Norwegian coastline.
Charlotte Garmann-Johnsen | October 18th, 2022 | Applications
In SeaBee, we develop a broad system for monitoring seal colonies along the Norwegian coastline, by moving beyond small-scale local operations towards longer-range operations using autonomous flight operations. This approach can greatly improve the coverage and frequency of seal assessments such as those carried out by IMR in support of the national coastal seal management plan. This is highly relevant, also in view of increased developments of human maritime activities along the coast (e.g. offshore wind, offshore aquaculture, fishing), and the associated risks of negative impacts on ecosystems, including seals.
One of the great advantages of using long-range drone operations for seal monitoring is the increased coverage, compared to today’s surveys which are carried out with small drones within visual line of sight from a home vessel. One alternative is to instead use manned aircraft, but this is associated with substantially greater cost and safety concerns.
Our data pipeline includes data acquisition, data archiving and image classification using machine learning, enabling automatic seal detection from drone imagery.
SeaBee Technology
For local operations, we use small drones (for example DJI Mavic 2 Enterprise Dual), which can be hand-launched and retrieved from small boats. This allows testing of infrared sensor technology to facilitate seal detection. For longer-range operations, we have used a Mugin-2 Pro 2930MM H-Tail Full Carbon Fiber VTOL (Vertical Take-Off and Landing), operated from a ground station and obtaining RGB images with an RGB camera (initially a Sony DSC-RX100 III ~20 MP, later upgraded to a Sony ILX-LR1 61 MP full-frame camera) together with an infrared sensor (FLIR Vue Pro) along pre-programmed transects covering entire groups of skerries where harbour seals are known to haul out.
Key results
As part of IMR’s ongoing efforts to monitor coastal seal colonies, a large dataset of drone-based images has been gathered over the past 7 years. This dataset is used to update existing machine learning algorithms for seal detection on ice floes, enabling the detection of seals on rocky shores.
In 2022, we tested long-range BVLOS operations at Tarva, Trøndelag, covering two seal colonies and obtaining a total of 1196 images. Flight operations were highly efficient, with reliable data telemetry, demonstrating the feasibility of large-scale BVLOS operations. Imagery from the 2022 campaign in Tarva are published at the GeoNode Server.
Example image showing seals on a skerry. Seals can be relatively easily seen in the original image (left). However, in the inset showing a close-up, it becomes clear that the image resolution is insufficient for clearly distinguishing some individuals from the mottled background. Image from Martin Biuw.
Fieldwork in 2022 identified some issues with the camera that was initially used (Sony DSC-RX100 III ~20 MP), in terms of poor resolution, making it challenging to distinguish seals from rocky backgrounds. This led to an upgrade in camera sensors for the 2024 field season to a Sony ILX-LR1 61 MP full-frame camera, which greatly improved the pixel resolution in the images. In order to improve the detection accuracy, we also included a FLIR Vue Pro infrared sensor in addition to the standard RGB camera. The IR video greatly decreases detection ambiguity uncertainty, thereby aiding in seal detection and classification.
Example images showing seals on a skerry taken in 2024. Seals can be relatively easily seen in the original image (left). However, the righthand image shows the IR image of the same skerry, illustrating the great improvement in seal detection using both RGB and IR sensors. The bottom row shows a close-up of the top lefthand corner of the top images. Note the improved detectability of seals of similar colouration as the seaweed. Images: Pål Kvaløy, Martin Biuw.
Ongoing work
Images from the two Tarva campaigns have been manually annotated, and are now being used to refine the machine learning algorithms for seal detections developed in Seabee WP3. In order to be as informative as possible for the machine learning training, annotations include known seals as well as known non-seal features, some of which could easily be misidentified as seals.
Example images showing manual annotations of seals from one of the images taken in 2024. Green rectangles represent known seals, whereas orange rectangles are manually selected features that are not seals. Some of these are relatively easily distinguishable from seals, while others have been selected to represent features that could be misidentified as seals. In the close-up to the right, one feature (to the bottom left represents an easily distinguishable round rock, while the other two non-seal annotations represent features that have characteristics that make them difficult to distinguish from seals. Images: Pål Kvaløy, John-André Henden & Martin Biuw
Contact
Interested in learning more about our work marine mammals or opportunities for collaboration? Please contact us:
Lead of work on marine mammals in SeaBee: Martin Biuw (Institute of Marine Research, IMR).
Operations Coordinator of SeaBee: Trygve Heide (NIVA)
Related work
STORIES
PUBLICATIONS