IASI Cloud Detection Optimisation
IASI Cloud Tests
The IASI Level 2 processing includes 10 logical tests to detect cloud contamination.
These tests exploit different physical processes modifying the infrared radiation, such as thresholds applied to the brightness temperature, spectral shape, spatial coherence, spectral match with clear sky radiances, and special tests over elevated polar regions or detection of desert dust storms. A dedicated test has been developed to detect thin cirrus clouds, which are present within an IASI IFOV.
All tests include parameters, and these parameters may depend on geographical location as well as on the time within a year. Not all tests are applied over all areas on earth.
The result of all tests is at least a binary decision whether the IFOV is cloudy or clear.
In this study 7 of the 10 tests shall be optimised using the collected sample data:
Test |
Name |
Description |
Parameters |
Cloud/ clear |
Partly cloudy |
B |
Inter channel regression tests |
The emissivity of clouds is known, and the BT in a window channel can be estimated from the BT in other window channels by a linear equation. Comparison of the difference between the measured BT in 3 channels and the linear combination estimate with thresholds. |
16 parameters x 2 surface types (land/water) of the regression;
3 thresholds for temperature differences |
yes |
no |
D |
Horizontal coherence tests |
The BT in one window channel (3.7µm) of the IFOV is compared with the mean BT of the 4 IFOVs in the EFOV. If the difference exceeds a threshold, the IFOV is declared cloudy. |
1 temperature threshold, which depends on location and time;
Test is applied only over water surfaces and not in polar regions. |
yes |
unclear |
E |
Threshold on IASI EOF residuals |
The IASI spectrum is compared with clear sky spectra. This is done by mapping the measurement vector on EOFs derived from precalculated clear sky radiances. If the difference exceeds a threshold, the IFOV is declared cloudy. |
A threshold vector with 8461 values.
The collected sample also will be used to construct a new set of clear sky EOFs |
yes |
unclear |
F |
Window cross correlation test |
Comparison of the measured window channels spectra with synthetic spectra. If the correlation (homomorphic degree) exceeds a threshold, the IFOV is declared cloudy.
The window region in which the test is applied is defined by the user. |
1 threshold, depends on surface conditions; synthetic spectrum and window region (size) could be considered as parameters subject to tuning as well. |
yes |
unclear |
G |
Test for clouds over polar regions |
Test on the BT difference between 11µm and 6.7µm channels. |
1 temperature threshold; depends on location |
yes |
no |
H |
Detection of dust storms |
Test on the BT difference between 11µm and 12µm channels |
1 temperature threshold; depends on location. The test is applied over deserts and adjacent areas |
yes |
no |
Optimization Methodology
The optimization of the parameters of these tests is a classical problem where standard methods exist. In this case the result of the function to be optimized is a discrete function and its derivate is non-continuous. This constraint the mathematical procedure used for the optimization. Methods such as Conjugate Gradient or Levenberg-Marquard algorithm cannot be applied, because they require continuous derivatives.
For this study we propose to consider the Simplex method and a Genetic Algorithm (GA).
The Simplex algorithm is a method for function minimization which is able to crawl out of some local minima to find better minima. It is not the fastest method, but rather robust (performs also well with noisy functions).
GAs are adaptive heuristic search algorithms premised on the evolutionary ideas of natural selection and genetics. The basic concept of GAs is designed to simulate processes in natural system necessary for evolution, specifically those that follow the principles first laid down by Charles Darwin of survival of the fittest.
As such they represent an intelligent exploitation of a random search within a defined search space to solve a problem. GA work well also for high dimensional search spaces.
Both methods allow constraining the search region, which will be useful here.
|
