MSFA-Net: A convolutional neural network based on multispectral filter arrays for texture feature extraction

Anis Amziane (1) , Olivier Losson (1) , Benjamin Mathon (1) , Ludovic Macaire (1)

1 CRIStAL - Centre de Recherche en Informatique, Signal et Automatique de Lille - UMR 9189

Multispectral snapshot cameras fitted with a multispectral filter array (MSFA) acquire several spectral bands in one shot and provide a raw mosaic image in which a single channel value is available at each pixel. Texture features are classically extracted from fully-defined images that are estimated by demosaicing. Such an estimation may however cause spatio-spectral artifacts. Moreover, texture feature extraction becomes computationally inefficient and yields to high-dimensional features as the number of bands increases. In this paper, we propose an original approach based on a convolutional neural network called MSFA-Net to capture spatio-spectral interactions in raw images at reduced computation costs. Experiments of multispectral image classification and outdoor image segmentation show that the proposed approach outperforms several hand-crafted and deep learning-based feature extractors.

Mots clés: Multispectral Imaging; deep learning

Domaines: Traitement du signal et de l'image [eess.SP]

Anis Amziane, Olivier Losson, Benjamin Mathon, Ludovic Macaire. MSFA-Net: A convolutional neural network based on multispectral filter arrays for texture feature extraction. Pattern Recognition Letters, 2023, 168, pp.93-99. ⟨10.1016/j.patrec.2023.03.004⟩. ⟨hal-04018052⟩