Prof. Richard Gloaguen
Helmholtz-Zentrum Dresden-Rossendorf HZDR
Spectroscopy vs. image processing: beyond the clash of the Titans
Abstract:
With the recent development of imaging spectrometers and the concomitant rise of computing solutions (GPU-HPC, machine learning, computer vision), we can observe two divergent trends in the processing of spectral data. The input data is the same: an array of spectra with implicit spatial organization, usually called a hypercube. The data science and machine learning communities mostly consider hyperspectral data as images and are processed using architectures modified from those developed for RGB images (e.g. CNNs, transformers). On the other hand, traditional spectrometry usually only considers the spectral content of the data and ignores the spatial relationships. Our recent work has clearly demonstrated that both approaches are important and might independently be suitable for specific tasks. But there is added value in combining both approaches. Spectrally aware machine learning approaches or spectrally tuned deep learning architectures provide both the big-data advantages of neural networks but allow a better explainability of the results, ensure a better use of the data (e.g. band selection instead of feature reduction), and preserve the physical/chemical relationships. For example, it is important to understand the specific spectral signatures of polymers (e.g. PP, PE, ABS) to map them using hyperspectral sensors on a conveyor belt running at 1 m/s. With a few examples, I will highlight the power of hyperspectral imaging and illustrate current ML/CV strategies for the characterization of complex material streams or the Earth’s surface.
