Woldstad T. 2023.

This thesis explores a novel application of Self-Organizing Maps (SOMs) for the purpose of multimodal image fusion in remote sensing. The research builds upon the work of the HYPSO satellite project and the SeaBee project, both aiming to enhance the monitoring and mapping of coastal environments for ecological preservation.

The thesis addresses a key challenge in the field: the processing of vast amounts of high-dimensional data collected by different imaging modalities, including RGB, multispectral, and hyperspectral cameras. The motivation behind the task of fusing images emanates from the distinct advantages of both hyperspectral and RGB images. While hyperspectral images possess high spectral resolution, facilitating precise identification of materials, they often suffer from low spatial resolution. Conversely, RGB images possess high spatial resolution but lack in spectral information.Traditional techniques for data fusion often struggle with real-world complexities, resulting in inefficiencies in time and computational power.

To overcome these limitations, the thesis proposes the use of SOMs, a type of shallow neural network. SOMs are advantageous due to their simplicity, reliability, and computational efficiency. With a unique capability of visualizing uncertainties, a deeper understanding of the underlying patterns and errors in the data set can be obtained. Key features of SOMs that are explored include their topological preservation property, which retains distances from the high-dimensional space in the outputted 2-dimensional space.

The thesis discusses potential challenges in implementing SOMs in the application of image fusion. The study includes testing and evaluation on both simulated and real data of the proposed SOM. The results demonstrate that the fusion of HSI and RGB data enables the SOM to project and infer spectral information beyond the limitations of the HSI, where only RGB data is available. The results and their potential impacts are expected to contribute significantly to the ongoing efforts of NIVA and NTNU, as well as the broader field of remote sensing.