IASI PC Compression - Searching for sinal in the residuals

Hyper-spectral data, such as IASI L1C radiances, can be seen as a sum of atmospheric signal and random instrument noise. While the spectral correlation in the signal allows for substantial compression, the high entropy of the noise imposes small limits on the maximum possible compression rate. Therefore high compression rates can only be achieved if a large part of the noise can be identified and removed from the spectra before dissemination. Principal component (PC) compression, as used for EUMETSAT's dissemination of compressed IASI spectra, offers a method for doing this. There is a legitimate concern that some amount of genuine atmospheric signal is discarded together with the noise. This happens, in particular, for rare features, not well represented in the training data, and for spectrally narrow features, for which it becomes difficult to distinguish between signal and noise.

In this talk, we present the details of EUMETSAT's PC compression of IASI spectra and how an initial set of randomly selected base spectra was systematically extended by iteratively adding selected outlier spectra in order to obtain a PC training data set with good representation of rare situations, such as volcanic eruptions and heavy fires. Furthermore, we analyse the residuals from this compression method with the purpose of identifying manifestations of atmospheric signal, which might have been removed from the spectra before dissemination.

Other topics touched upon in this talk include:
• The use of canonical angles between the signal and forward model subspaces to obtain a rotation of the signal space basis, which can be used for suppression of certain instrument artefacts.
• The implications of using two different sets of PC's, one for dissemination and another for retrieval/assimilation.
• The use of PC's based on IASI-A spectra for compression of IASI-B spectra.
• A plea for further use of reconstructed radiances, which often provide a simple way to get better results from IASI data.