What if you have a perfect algorithm and want to apply it to thousands of satellite images? What if you want to automate your processing chain and convert it into an operational service? Or what if you are part of a team and want to share computing resources and also methods for processing, development, and validation? In these cases a processing system can help.
Our processing system for medium to large scale automated EO data processing
We use our processing system Calvalus for all these purposes: We process 130,000 images for a yearly time series of global burned area products. Every day, we automatically process new Sentinel-3 OLCI data to water quality products and add them to a continuously growing data cube for the Cyanoalert service and other water quality services. We use our processing system for several projects at the same time, sharing parts of the software, and using a common way to specify workflows and to run processing jobs, systematically and on-demand.
Data and processing management for projects, institutions, and teams
Other organizations use Calvalus for the same reasons. CalFin operated by the FMI and SYKE in Finland and ESTHub as part of the Estonian Collaborative Ground Segment are two processing systems where scientists process datasets and develop new processing chains. Another example is CalLand in the land monitoring department of DLR where Calvalus serves as the processing system for a working group.
Apache Hadoop clusters and cloud solutions for Earth Observation
Technically, Calvalus is based on Apache Hadoop as Cluster scheduler and “batch system”, with several additional functions to use a Hadoop for Earth Observation data processing.