KELPMAP – Upscaling drone-based maps using satellite images shows promise

Helgelandskysten is one of Norway’s most beautiful coastlines, holding World Heritage Status (UNESCO) and ‘Outstanding Universal Value’.

It has thousands of small islands, islets and skerries, mountains, fjords and a great deal of life above and under the water. It must be managed well to safeguard the valuable cultural history and ecosystems. Kelp forests are a key part of the ecosystems in this area. 

Helgelandskysten (above) and kelp forest (below). Photo from H. Gundersen, NIVA.

KELPMAP (NIVA and NR) is investigating if it is possible to map kelp forests using drones, and then upscale the information collected using satellite data. The project is financed by Miljødirektoratet and Norsk Romsenter, 2022 – 2024 (NIVA field report).  

SeaBee is being used to map the benthic habitats. Both rotor drones and fixed-wing drones with RGB and MSI sensors (SeaBee equipment) are used to collect the data which is uploaded to the SeaBee Data Pipeline.

Ground truth data are also sampled in the field, for training of algorithms and validation of data products. The drone images are then annotated, guided by the ground truth, to show which parts of the drone images are kelp forest and other species.

SeaBee has defined 42 different habitat classes in three analysis levels (shared on GitHub), many of them compatible with the NiN classifications. These results are analysed, quality controlled and presented as high-resolution maps of habitat classes for the whole study area – which show the kelp forests and other species. 

Another new aspect is upscaling these results to cover a larger area using satellite imagery (using Sentinel-2 with 10m resolution and Pleiades with 2m resolution). The upscaling showed promising results, important for environmental management on larger scales.  

The results will be delivered to Miljødirektoratet this year to be used as a tool for measuring progress and implementation of existing national policies and management plans.

Miljødirektoratet are pleased with progress so far, and with the results that were received. These advances also have potential applications in other national and European research, not the least the Kunming-Montreal Global Nature Agreement (CBD). 

The drone view of Helgelandskysten during field work. Photo from G. Medyan, NIVA.

SeaBee shines at GeoHab 2024

SeaBee Research Infrastructure team members proudly presented at the international GeoHab Conference 2024 (6th – 10th May) in Arendal, Norway.

SeaBee contributed four presentations, and was mentioned in the keynote by Terje Thorsnes (NGU) on marine habitat mapping programs in Norway.    

Kristina Kvile (NIVA) presented the latest news on drones, updated methods and protocols for marine habitat mapping. She shared how habitats are classified and how drones can be used to identify marine vegetation at various hierarchical levels ranging from habitat classes to species level using high-resolution multispectral sensors and AI classification tools.

Collecting direct observations from the ground and boat (above) and the possible resolutions of collected data. Photos from K. Kvile (NIVA).

Håvard Løvas (NTNU) told the audience about hyperspectral solutions for detailed identification of shallow-water species and objects with distinct optical ‘fingerprints’.

Hyperspectral imaging is a novel, powerful tool for coastal drone mapping. You can characterise water quality, classify bottom substrates, and identify simple species based on their optical characteristics. However, interpretating images of underwater features and species is challenging, thus careful, systematic data handling and calibration routines are essential to achieve good results.  

Øyvind Tangen Ødegaard (NIVA) described how to tame a surface drone (Otter Pro USV), and how its sophisticated on-board optical  and acoustic sensors can support the information from flying drones to produce even better maps and classifications of seafloor species and features.

The SeaBee Otter operates with a high-accuracy positioning system (better than 2-3 cm on measured data) which enables precise georeferencing of collected data – be it underwater photo and video,  acoustic data of the seafloor or optical data of the water quality. 

Views of how the SeaBee Otter is handled in the field. Photo by Ø. Odegaard (NIVA).

Kasper Hancke (NIVA) evaluated the use of drones and artificial intelligence for kelp forest and seagrass mapping, highlighting the main advantages of SeaBee Research Infrastructure as well: 

– Drones with RGB and MSI sensors combined are powerful tools for high resolution mapping and monitoring of coastal habitats and species 
– Artificial Intelligence models are a powerful method for classifying benthic habitats and are cost-effective
– High resolution habitat maps are essential for coastal carbon accounting and more sustainable management and for guiding marine preservation and restoration initiatives. 
 

Great days of science and exciting discussions. Thank you to the organizing committee! – Kasper Hancke, SeaBee project coordinator

For more information on SeaBee Research Infrastructure visit seabee.no, or follow us on LinkedIn: SeaBee Research Infrastructure.

 

SeaBee Out and About

The SeaBee experts have been busy, out and about sharing how SeaBee Research Infrastructure can be used, testing new possibilities and implementing the SeaBee Data Pipeline in coastal research and environmental monitoring activities.  

SeaBee at C-BLUES EU kick-off meeting   

There is a lot to discover about blue carbon ecosystems – seagrass meadows, tidal marshes, mangroves and macroalgae. The C-BLUES project aims to significantly advance knowledge and understanding of blue carbon ecosystems to reduce scientific uncertainty, improve reporting of blue carbon, and promote the role of blue carbon in delivering climate policy commitments.  

At the kick-off meeting (held 14-17th April in Barcelona), SeaBee coordinator, Kasper Hancke (NIVA), presented how SeaBee and drones can contribute to efficient mapping and monitoring of blue carbon habitats across a range of coastal environments and how drone data further can be developed into tools for assessing:

  1. ecological status of coastal systems,
  2. species and biomass of marine vegetation,
  3. development for estimating stocks and content of blue carbon, with relevance to sustainable management and research on climate regulation. 

C-BLUES will join forces with the already running Horizon Europe project, OBAMA-NEXT.  

C-BLUES is a Horizon Europe Framework project running from 2024-2028, funded under the call for EU-China international cooperation on blue carbon (HORIZON-CL5-2023-D1-02). 

Group picture at C-BLUES kickoff meeting, 14th – 17th April in Barcelona.
SeaBee at remote sensing seminar at MDIR 

Miljødirektoratet (MDIR) hosted a seminar on remote sensing for Norwegian environmental monitoring and mapping on the 23rd April, 2024.

SeaBee co-coordinator, Hege Gundersen (NIVA), presented the KELPMAP project on using drones for mapping and identification of kelp and underwater vegetation.

KELPMAP develops novel tools for kelp forests mapping using drone products and machine learning-based image classification, and the results demonstrate that drone images can be used to identify klep forests to a water depth of at least 10 m.

MDIR and Norwegian Space Center funded SeaBee to developing the tools to map kelp forests and other habitats for improving marine management actions. 

For more information on SeaBee Research Infrastructure visit seabee.no, or follow us on LinkedIn: SeaBee Research Infrastructure.

 

New SeaBee Pilots Ready to Fly

Preparations are well underway for the main field season in SeaBee for 2024. 

SeaBee Research Infrastructure relies on drone-based data collection by experienced pilots and marine biology experts. Ensuring drone pilots are trained in the latest techniques and are up-to-date with new regulations and certifications is a key part of collecting data for mapping, monitoring and research along the Norwegian coastline.  

Drones-eye-view of the NIVA team getting ready to learn new skills in drone flying at Ølbergholmen,Larvik.
Drone Pilot Training 

At the end of March, a team from NIVA headed out to Ølbergholmen (Larvik) to train new pilots, and test the latest sensors and new EVO drones (New EVO drones added to SeaBee Family)  

“We had bright conditions, around 10°C, so relatively good for March, but the weather forecast was not so accurate, and there was more wind than we would have hoped for and expected. However, this was not a show-stopper, as the very lower levels of air were calm enough (less than 10m/s). Our larger drones were able to map at 60m. During lulls in the wind, we were able to test one of our NEW EVO R VTOL (Vertical Take Off and Landing) fixed-wing vehicles” – Medyan Ghareeb, Drone Operations (NIVA) 

  

Kristina Kvile (Data Validation lead) and Debhasish Bakta (Web interface development) went through the training manual and the practical procedures around drone flying. Both did exceptionally well flying two DJI Mavic Mini drones, with skills and confidence increasing throughout the day. Kristina and Debhasish are now certified for A2 and A1/3  (respectively). They were supervised by SeaBee pilots Medyan and Øyvind, who later ran some test flights. 

A panorama-like view of the field site where the team went through practical certifications and tested new SeaBee drones.
Testing new drones and sensors 

Øyvind Herman Torp (Drone Pilot) used the breaks between high winds to test the ReefEdge Blue, Altum PT and RGB sensors. He collected datasets using these covering the whole Ølbergholmen area at an altitude of 60 m.  

The wind subsided a little in the afternoon, and some test flights using the EVO R with RGB and Rededge-P dual setup were flown. The pilots were able to collect a complete data set with the Rededge-P sensor, but challenges with the RGB camera means another attempt is needed to fully test both EVOs and collect a full dual sensor data set. 

For details about the technology we use, visit SeaBee Equipment.

Trained to fly – Drone pilots in SeaBee Research Infrastructure

Drone pilots are a crucial part of the data collection for the SeaBee Research Infrastructure. We sat down with Tor Arne Johansen (NTNU) and Medyan Ghareeb (NIVA) who work with this part of SeaBee. They shared how drone pilots get certified, what they do to practice, and how SeaBee uses drone pilots from the partner institutions to fly the variety of missions needed for different scientific applications.

Medyan Ghareeb (NIVA) flies a drone as part of a training mission for SeaBee Reseach Infrastructure

Why does SeaBee Research Infrastructure need drone pilots?

SeaBee needs drone pilots to fly the state-of-the-art drones equipped with assorted sensors, during field missions to collect data on different species and environments in coastal areas of Norway. Many coastal areas have complex terrains to work in, so flying drones enables covering larger areas that would be a struggle to reach on foot.

Drone and sensor technology is constantly developing and improving, therefore SeaBee needs pilots who have the appropriate training, certifications and knowledge in operating drones to collect high quality data that can be used in research and coastal management.

How many drone pilots does SeaBee have?

There are multiple drone pilots at the partner institutes who are involved in SeaBee. 

At NIVA, under the umbrella of SeaBee Research Infrastructure, there are 7 pilots for Visual Line of Sight (VLOS) flights and 2 who can fly drones on Beyond Line of Sight (BLOS) flights under Tiepoint.

The regulations covering drone flights are complex, therefore other partners are obliged to fly under their respective operations manuals.

NTNU has 2 pilots flying under the NTNU operations manual, and NINA had 6 drone pilots working on a recent summer campaign (read more in the news about 440 drone missions and 100 thousand images) that can fly VLOS under NINA operations manual.

How does one become a drone pilot in SeaBee?

Pilot training and operating licenses are managed directly by the partner institutes. There are different levels of training and certification needed before one can fly drones, and the partners are constantly updating procedures to ensure all pilots remain compliant.

Medyan Ghareeb (NIVA) flying a drone during a training mission on the coast of Norway

When considering whether to take on drone pilot certification, Medyan suggests keeping the following in mind: 

“… potential Pilots are enthusiastic or at least willing for all the processes for example: training, regulations, hms, data, flight preparation, updating or buying in hardware and software and not just flying drones – the flying is just a small part of being a pilot.” 

 

What is the most exciting, and the most difficult moment during a drone flight? 

“It is exciting when it all comes together, a [drone] flight gets good data and the drone lands! Keeping pace with new regulations, training and hardware requirements can be difficult, and integrating/using new sensors can also offer many challenges” – Medyan Ghareeb, drone pilot at NIVA. 

 

What are the next steps for drone pilot operations in SeaBee Research Infrastructure? 

As the available SeaBee Research Infrastructure hardware becomes larger and more capable we need to keep up with training and regulations. This has different implications for the different partners. For example, at NIVA this means becoming capable in the Specific and PDRA, later SORA classes of operations. These are operations that require further training, more rigorous risk assessments, hardware needs and notices or permissions.  

Annual Meeting 2023 – From outcomes to impact

All partners and industry partners met in Bergen for the SeaBee Annual Meeting (25 – 27th October, 2023) to update and discuss the year’s progress in delivering the SeaBee Research Infrastructure.

SeaBee aims to create a cohesive national infrastructure, enabling sharing and use of data, collected by state-of-the-art drones and technology, to better understand Norway’s natural environment. This was the fourth Annual Meeting.

Main outcomes 

During the Annual Meeting, SeaBee partners shared the progress that happened in 2023:

  • New expert pilots trained in latest certification requirements 
  • Two state-of-art drones and new sensors purchased 
  • Different models built for annotation and processing the drone images, and tested using field examples 
  • SeaBee Data Pipeline implemented for processing and visualising results and found to work well 
  • New methods and techniques validated through in-situ tests in the different SeaBee applications 
  • Began paving the way for a SeaBee2…

 

What happens next? 

We received valuable input and feedback and SeaBee has some clear focus areas for the coming year. Building impact through applications and use by decision-makers in Norway will be highlighted in 2024, as will sustainability of the infrastructure in the future. 

 

SeaBee project consortium gathered atop Ulriken in Bergen for the Annual Meeting (25-27th October, 2023).

For more information on SeaBee Research Infrastructure visit seabee.no, or follow us on LinkedIn: SeaBee Research Infrastructure.

 

Increasing flight range and monitoring capacity – Two New EVO Drones added to SeaBee family!

At last, a long-awaited, new arrival has come to SeaBee. SeaBee has received two new EVO drones.

The new drones will increase SeaBees operational area limit from a few kilometres to tens of kilometres. The drones also increase the ‘payload’ capacity (what equipment the drone can carry) with more, better sensors, which can be mounted all at the same time. This gives more effective flight times. 

SeaBee Drone Operator, Medyan Ghareeb, and Project Coordinator, Kasper Hancke, are very excited to start working with the newest member of the SeaBee family, one of two EVO VTOL drones

The EVO drones are capable of Vertical Take-Off and Landing (VTOL), which means they do not require a runway, and can lift off and land similar to a helicopter. This gives SeaBee a strong advantage in researching, mapping and environmental monitoring along the difficult terrain of the norwegian coastline. Not to mention in mountainous and forested regions of Norway! 

Curious about what data SeaBee has collected so far? Visit geonode.seabee.sigma2.no to see the range of locations and data types.

The EVO drones are state-of-the-art, fixed wing drones, that look like a small plane. The have a wingspan of 269 cm, a take-off capacity of 10 kg and a flight time of up to 3 hours. They are equipped with a ‘standard’ RGB camera (62 megapixels), as well as multispectral and thermal cameras.  

Read more about the technical details at DeltaQuad Evo – DeltaQuad VTOL UAV 

Watch this space for more updates once we get these drones out into the field! 

One of the new EVO VTOL drones in the SeaBee family on display in the NIVA social zone in Oslo.

Using drones for mapping seagrass and coastal carbon inventories

SeaBee is being proven in the field through different applications and subprojects. ZosMap is a SeaBee subproject with focus on developing applications for seagrass mapping and monitoring using drones (Unoccupied Aerial Vehicles, UAVs), high-resolution aerial imaging, and Machine Learning (ML) technology for image analysis and thematic mapping.  

The aim is to explore the possibilities of measuring seagrass distribution, biomass, carbon content, and health status using flying drones. Not only are seagrass meadows essential to sustain marine biodiversity and provide nursery grounds for fish, but they also host large amount of organic carbon bound in living biomass and in the sediments below the seagrass meadows.   

Mapping seagrass mission

In mid-June, a team of NIVA researchers and drone pilots once again went to map seagrass meadows and explore the amount of organic carbon that is stored in these vital coastal habitats.

The fieldwork is part of an annual monitoring campaign with drone missions flown every month. Five researchers from NIVA participated in this fieldwork. In addition, engineering researchers and two Master students from NTNU are involved in the project work.

A bit outside Larvik, around Ølbergholmen in the Oslofjord, ZosMap has a test ground for developing methods and testing out new hypotheses. Two small bays define the test area that hosts both dense and sparse seagrass meadows, rockweed beds, sandy sediments with microalgae, and a few scattered kelp occurrences. 

The two adjacent field sites close to Larvik. Named "Ølberg S" the southern bay and "Ølberg N" the northern bay (named after the peninsula on the outside of the two bays). N59.007, E10.132.
NIVA partners working in SeaBee, in the field and ready to to map seagrass meadows and explore the amount of organic carbon that is stored in these vital coastal habitats. Photo Kasper Hancke.

A 360 degree panorama of the field area in Larvik, taken using a Mavic mini Pro3. Panorama by Kasper Hancke.

Flying drones

Over four days in June the team flew multiple drone missions for collecting imaging data covering both a larger overview of the region (at high altitude) and high-resolution images for detailed studies (at low altitude). To assess the meaning of pixel resolution for seagrass and organic carbon mapping drone mission were flown at altitudes from 20 to 100 m.  

The drones used in the field mission (top and lower right), and the seaweed species that were mapped (lower left). Photo by Kasper Hancke.

A DJI Matrice 300 drone was deployed equipped with an RGB camera (SeaBee Tech) and a multispectral spectral (MSI) camera (Micasense Altum). In addition a DJI M600 drone was deployed with a hyperspectral (HSI) sensor (SPECIM AFX-10). The RGB images will be used primarily for image annotation and provide good true color overview image, while MSI and HIS image data will be used for generating thematic vegetation maps and quantifying seagrass biomass, organic carbon content and ecosystem health status.  

In addition to drone data collection, the team was busy collecting ground truth data using traditional techniques in order to annotate (categorise) image data and to build a database for training of machine learning algorithms. 

Measuring of seagrass using ground-based methods. Photo by Kasper Hancke.

The algorithms will in turn be able to identify seagrass and other coastal vegetated habitats automatically from drone data, in the same way as a smart phone identifies the face of its owner.

In June we found the seagrass meadows of the Larvik region to be both well developed, in good ecological condition and rich in organic carbon hosted in its dense seagrass canopies – Kasper Hancke, NIVA

 

Importance for seagrass meadows 

Seagrass meadows in the Oslofjord are often under strong pressure from anthropogenic stressors, so it was great to experience healthy and live seagrass meadows in these areas. These areas are not only a SeaBee/ZosMap test ground, but are also a well-visited recreational area for beach drifters, picnic groups, kayakers and many others.

Next Steps

Next steps are to analyse the images further in the SeaBee data pipeline. The results will be published when the data and laboratory samples have been processed.

For more information on how SeaBee works with coastal habitat mapping, visit ‘What We Do’

Two weeks, 440 drone missions, 100 thousand images – The SeaBee Research Infrastructure is taking off!

SeaBee is creating a comprehensive national research infrastructure, enabling sharing and use of data, collected with drones, to better understand Norway’s natural environment. SeaBee trains machine learning algorithms to analyse drone data and facilitate automated data analysis. This allows researchers to do their work faster, better, and more efficiently.  

 
SeaBee and SeaBirds – collecting drone images in the field 

Two field teams from NINA were out in May 2023. Led by Sindre Molværsmyr (NINA), the teams mapped sea gull and cormorant breeding colonies along the southern Norwegian coast from the Swedish border to Karmøy (more than ~500 km of coastline). The goal was to count seabirds through collecting drone images of seabird populations. The data collected from the drone images can then be compared with historical datasets, and data collected using traditional methods.

Traditional methods of data collection are people going to the seabird colony and counting the seabirds, or taking thousands of photos with a handheld camera from the window of a small airplane. The seabird population counting work also contributes to SEAPOP (Om SEAPOP – SEAPOP). 

Map of missions flown to map breeding seabird colonies, during field work by NINA. Photo Sindre Molværsmyr

In total, 440 drone missions were flown by two teams during two weeks from 15 – 29th May, 2023, each using a DJI MAVIC 3 equipped with a RGB camera (SeaBee tech). The surveys mostly covered open-nesting species, like gulls and cormorants.

With very nice weather for most of the field mission, the teams collected around 100 thousand drone images of seabird populations during the 440 missions. The largest missions collected around 2000 images to cover larger islands along the coast. Both teams covered about 150 km of coastline each day, one team with their own boat and the other with good help from Skjærgårdstjenesten or other local boat drivers.

The teams covered most known bird populations in the mission area. All missions were logged and added to the map and data visualisations. This will help for next year’s field mission planning, as colonies that were missed this year are easily seen and located in the map visualisations.

 

Our field work and data collection could not have been done at this scale without SeaBee – sending data for processing by pressing enter on my computer in the evenings was very efficient. The amount of data collected would be impossible to analyse manually, it would take many months to analyse drone images from 400 colonies by hand – Sindre Molværsmyr, NINA

Preparing for drone flying mission from the boat. Photo by Sine Dagsdatter Hagestad
Pilot flying drone out from the boat to capture images for counting seabird populations during NINA's field mission in May 2023. Photo by Sine Dagsdatter Hagestad.
 
Large scale data upload

NINA’s field campaign is one of the first examples of the SeaBee infrastructure operating on large-scale data handling, with drone images uploading directly to the SeaBee data server directly from the field at the end of each day. This is a great achievement for SeaBee, which not only saves time in the field, but allows more efficient mapping and monitoring of large geographic areas with better data quality.

James Sample (NIVA) and the SeaBee Data Platform team have worked hard to get to this point (SeaBee Data Platform components and how it works). Using code deployed on Sigma2, an automated script runs every hour to identify, process and publish new datasets uploaded by scientists in the field.

It was an exciting first test of SeaBee as “data infrastructure”. I really enjoyed being able to explore the new mission data published each morning to see what I could find – James Sample, NIVA

Ready for reviewing drone data collected during the day, before uploading to the SeaBee Data Pipeline
SeaBee Data Pipeline

The SeaBee data infrastructure was valuable during this mission, as having data processed in near real-time allowed the field teams to review data and adjust settings and drone flying strategies during the mission, to ensure the best quality drone images were collected.

In total, around 1 TB of raw images from drone flights were collected and uploaded to the SeaBee data repository, which led to around 15 TB of data once it had been processed (Digital Terrain models (DTMs), orthophotos, point clouds, etc).

The SeaBee data infrastructure ran well for processing such a large amount of data. On average, missions took around 90 minutes to process. Missions with approximately 2000 images took around 1 day to process. After processing, the final products were automatically published to SeaBees GeoNode server. Currently, there are more than 500 data resources published on the server.

This project has been an important achievement for SeaBee as it demonstrates how useful a drone-based infrastructure is for seabird research and mapping, and it serves as a good example of how efficient SeaBee can be for large-scale mapping projects and handling of big data. – Kasper Hancke, NIVA

 

Drone image of seabird population, collected during NINA's field work in May 2023. Photo by Sine Dagsdatter Hagestad
Next steps

The next steps for this seabird mapping project include analysing the drone images that have been collected to estimate seabird population numbers. This will involve training a new AI model, and annotating drone images work as bird species were encountered that are not in the current AI models (for more information, check using Artifical Intelligence and drone data). Also, some comparisons will be made between the data collected using drones and that collected using traditional methods.

In terms of what will happen next for the SeaBee data infrastructure, this field mission provided some very useful lessons for processing large scale datasets. Better error handling systems will be developed, and the machine learning from NR will be integrated to further automate the infrastructure.