SMOS NetCDF Conversion

The NetCDF Conversion Command can be used to convert the data of SMOS products in Earth Explorer file format to NetCDF. This conversion process involves a re-sampling of the data from the original ISEA4H-R9 hexagonal data cells to a rectangular raster file format.
The conversion tool supports the following SMOS product types:

MIR_SM_BWSD1C
MIR_SM_BWLD1C
MIR_SM_BWLF1C
MIR_SM_BWSF1C
MIR_SM_SCLD1C
MIR_SM_SCSD1C
MIR_SM_SCLF1C
MIR_SM_SCSF1C
MIR_SM_OSUDP2
MIR_SM_SMUDP2

The NetCDF Conversion Command is integrated into the BEAM framework in two different places. It can be either invoked from within VISAT or be used as a GPT-Operator for batch-processing using the cmd-line.

Note: The conversion tool converts the L1C data using the browse-view data representation as described in the chapter SMOS Data in BEAM. The target product does not contain snapshot information anymore.

NetCDF Output File Format

The target file format for the conversion process is NetCDF-4 (compressed) with Climate and Forecast (CF) conventions. General information about the NetCDF-4 format can be found at:

UNIDATA NetCDF (Network Common Data Form)

The CF format convention is described at:

CF Metadata

The metadata of the original Earth-Explorer file is converted to a CF-compliant metadata record and stored as global NetCDF attribute.

All fields in the SMOS EE grid point data records are represented as single bands on a rectangular grid raster. The raster is a Plate-Carree projection with a pixel size of 0.02197 degrees in both axes. Each hexagonal ISEA data cell is represented by approximately 30 image pixels for equatorial grid points of the SMOS discrete global grid. For details on the conversion process, please refer to the Data Model description.
Note that the band data used for converting science products corresponds to the browse view.

The size of the target NetCDF product covers the geometric bounding box of the original SMOS file. This can lead to large products, especially when the SMOS dataset crosses the anti-meridian. All pixels not covering a SMOS cell are filled with "fill" values.
It is highly recommended to make use of the possibility to create regional subset products. This reduces the size of the resulting NetCDF file and increases the conversion speed significantly.

The figure below shows a converted MIR_SM_BWSD1C product, band BT_Value_X without any subsetting applied.

It can clearly be seen that most parts of the product contain no data.

NetCDF Conversion in VISAT

NetCDF Conversion GUI

The NetCDF Conversion dialog can be invoked from within VISAT using the "Tools" menu.

Source Products: Either the SMOS product selected in VISAT, or all SMOS products residing in a directory of the file system can be used for conversion. When a directory is selected, the conversion process will convert all SMOS files contained in the directory. The software can handle zip-compressed products.

Region of Interest: There are three possibilities for specifying a region of interest (ROI) considered for export:

Whole Product: the conversion process will automatically perform a subset that contains the geometric boundary of the input file, i.e. all measurement data is converted.
Geometry: the target product covers all pixels contained in a polygon defined in VISAT. Polygons can be selected using a drop-down list. Only products intersecting with the polygon will be converted. The target product will be tailored to the intersecting geometry.
Area: the target product covers all pixels contained in a rectangular area. The area can be defined by either entering the bounding coordinates in the text fields or by using the mouse to adjust the bounding rectangle on the map. Only products intersecting with the polygon will be converted. The target product will be tailored to the intersecting geometry.

Target Directory: all output files of the conversion process will be stored in the directory selected.

NetCDF Conversion as GPT operator

The NetCDF conversion tool is implemented as a GPT operator. This allows to use the converter in batch mode using the GPT command line tool. Information about the Graph Processing Tool can be found in the VISAT main documentation, chapter Graph Processing Framework and online in the BEAM-Wiki: GPT bulk processing.

Overview

Name:	`SmosEE2NetCDF`
Full name:	`org.esa.beam.smos.ee2netcdf.ConverterOp`
Purpose:	Convert SMOS products from Earth Explorer Format to NetCDF4.
Version:	2.3

Description

This operator is used to convert SMOS data in Earth Explorer format to NetCDF-CF 4. It accepts either single input products or wildcard path expressions as input parameter. The conversion process can optionally take a bounding region into account. Only data contained in this region is used for the conversion process. If no bounding region is supplied, the operator automatically calculates the bounding box of the input SMOS data and applies the resulting geometry during the processing.

Sources

Name	Description	Type
`sourceProduct`	The source products to be converted. If not given, the parameter 'sourceProductPaths' must be provided.	MIR_BW[LS][DF]1C\|MIR_SC[LS][DF]1C\|MIR_OSUDP2\|MIR_SMUDP2

Parameters

Name	Data Type	Default	Description
`sourceProductPaths`	`String[]`		Comma-separated list of file paths specifying the source products. Each path may contain the wildcards '*' (matches recursively any directory),'' (matches any character sequence in path names) and '?' (matches any single character).
`targetDirectory`	`File`	`.`	The target directory for the converted data. If not existing, directory will be created.
`region`	`Geometry`		The geographical region as a polygon-geometry in well-known text format (WKT), i.e. POLYGON((...)). If not given, the bounding box of the data is used.
`overwriteTarget`	`boolean`	`false`	Set true to overwrite already existing target files.

GPT Conversion Examples

The following section gives some examples on the usage of the SmosEE2NetCDF operator, covering the most important use cases. The GPT tool is invoked from the command-line. This section uses the tag <BEAM-HOME> for the installation directory of BEAM. The following paragraphs use Windows specific notation; for Linux, please replace "gpt.bat" with "gpt.sh".

Note: Region definitions in the examples below define polygon objects as WKT. A concise description and links to in-depth documentation can be found at:
WKT at Wikipedia
In short: the points of a polygon are ordered lon/lat, the last point in the polygon must match the first to close the polygon.

Conversion of a single product

The command:

    <BEAM-HOME>/bin>gpt.bat SmosEE2NetCDF <file-path>

converts a single file located at <file-path> to NetCDF, using no geometric boundary. The target file will be located in the current working directory. i.e. <BEAM-HOME>/bin.

Conversion of a single product with ROI

To introduce a ROI into the conversion process, this example uses a so called graph-xml file. This file contains the description of a processing graph and the parametrization of each node in the graph. An example that can be used to convert a single SMOS-EE file to a specific target directory using a ROI is displayed below:

    <graph id="SMOS test conversion">
        <version>1.0</version>
        <node id="smos-conversion">
            <operator>SmosEE2NetCDF</operator>

            <parameters>
                <targetDirectory>C:/Data/results</targetDirectory>
                <region>POLYGON((-80 20, -50 20,-50 -20,-80 -20, -80 20))</region>
            </parameters>
        </node>
    </graph>

Assuming this XML is written to a file named roi-example.xml, the command:

    <BEAM-HOME>/bin>gpt.bat roi-example.xml <file-path>

converts a single file located at <file-path> to NetCDF, tailoring the target product to the polygon supplied. The target file will be located in C:/Data/results. If the input file does not intersect with the given geometry, no target file will be created.

Note: GPT either uses the operator name (as in the previous example) or a graph-xml file - as shown in this example.
Note: The parameters for region and target directory can also be supplied by other means. Parameters can either be supplied on the command line:

    <BEAM-HOME>/bin>gpt.bat SmosEE2NetCDF  "-PtargetDirectory=C:/Data/results"  "-Pregion=POLYGON((-80 20, -50 20,-50 -20,-80 -20, -80 20))"

or using a parameter file:

    <BEAM-HOME>/bin>gpt.bat SmosEE2NetCDF -p convparam.txt

where the file convparam.txt contains key/value pairs of the parameters:

    targetDirectory = C:/Data/results
    region  = POLYGON((-80 20, -50 20,-50 -20,-80 -20, -80 20))

Batch Conversion of multiple directories with ROI and wildcards

To batch convert all SMOS-EE files of type MIR_SMUDP2 located in multiple directories to a target directory using a ROI, the following graph-xml file can save as template:

    <graph id="SMOS test conversion">
        <version>1.0</version>
        <node id="smos-conversion">
            <operator>SmosEE2NetCDF</operator>

            <parameters>
                <sourceProductPaths>C:/Data/SMOS/*SMUDP2*,E:/Data/SMOS/*SMUDP2*</sourceProductPaths>
                <targetDirectory>C:/Data/results</targetDirectory>
                <region>POLYGON((-80 20, -50 20,-50 -20,-80 -20, -80 20))</region>
            </parameters>
        </node>
    </graph>

Assuming this XML is written to a file named directories-example.xml, the command:

    <BEAM-HOME>/bin>gpt.bat directories-example.xml

converts all SMOS-EE Soil Moisture L2 files contained in the directories C:/Data/SMOS and E:/Data/SMOS to NetCDF, tailoring the target products to the polygon supplied. The target files will be located in C:/Data/results. If the input file does not intersect with the given geometry, no target file will be created.