Dear All,

We have been using the MERIS Surface Directional Reflectance (SDR) plug-in processor for BEAM 4.9.0.1 and trying to process significant volumes of MER_FRS_1P data from 2003 over Europe. We have found that there is an ~12.5% failure rate (29/231 failures) for the MERIS SDR processing due to an issue with the coordinates of the input MER_FRS_1P and/or matching MER_RR__2P product . In each failure case, the following Java stack trace is output:

"org.esa.beam.framework.gpf.OperatorException: Coordinate out of bounds!
at org.esa.beam.framework.gpf.graph.GraphProcessor$GPFImagingListener.errorOccurred(GraphProcessor.java:362)
at com.sun.media.jai.util.SunTileScheduler.sendExceptionToListener(SunTileScheduler.java:1646)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:921)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
at org.esa.beam.framework.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:384)
at org.esa.beam.framework.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:362)
at org.esa.beam.framework.gpf.Operator.getSourceTile(Operator.java:472)
at org.esa.beam.meris.sdr.SdrOp.computeTileStack(SdrOp.java:153)
at org.esa.beam.framework.gpf.internal.OperatorImageTileStack.computeRect(OperatorImageTileStack.java:114)
at org.esa.beam.framework.gpf.internal.OperatorImageTileStack.computeTile(OperatorImageTileStack.java:87)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at javax.media.jai.PlanarImage.getData(PlanarImage.java:2085)
at com.bc.ceres.glevel.MultiLevelImage.getData(MultiLevelImage.java:64)
at org.esa.beam.framework.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:384)
at org.esa.beam.framework.gpf.internal.OperatorContext.getSourceTile(OperatorContext.java:362)
at org.esa.beam.framework.gpf.internal.OperatorImage.computeRect(OperatorImage.java:69)
at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
at javax.media.jai.OpImage.getTile(OpImage.java:1129)
at com.sun.media.jai.util.RequestJob.compute(SunTileScheduler.java:247)
at com.sun.media.jai.util.WorkerThread.run(SunTileScheduler.java:468)
Caused by: java.lang.ArrayIndexOutOfBoundsException: Coordinate out of bounds!
at java.awt.image.ComponentSampleModel.getSampleDouble(Unknown Source)
at java.awt.image.Raster.getSampleDouble(Unknown Source)
at org.esa.beam.framework.gpf.internal.TileImpl.getSampleDouble(TileImpl.java:455)
at org.esa.beam.gpf.operators.meris.RRToFRSOp.computeTile(RRToFRSOp.java:124)
at org.esa.beam.framework.gpf.internal.OperatorImage.computeRect(OperatorImage.java:75)
at javax.media.jai.SourcelessOpImage.computeTile(SourcelessOpImage.java:137)
at com.sun.media.jai.util.SunTileScheduler.scheduleTile(SunTileScheduler.java:904)
... 21 more

Error: Coordinate out of bounds!"

The processing seems to abort part way through due to the Java ArrayIndexOutOfBoundsException and we are left with a partially generated output product. An example of the command line that we have been using is as follows:

/opt/beam-4.9.0.1/jre/bin/java -Xmx2048M
-Dceres.context=beam
-Dbeam.envisat.usePixelGeoCoding=false
-Dbeam.mainClass=org.esa.beam.framework.gpf.main.Main
-Dbeam.home=/opt/beam-4.9.0.1
-Dncsa.hdf.hdflib.HDFLibrary.hdflib=/opt/beam-4.9.0.1/modules/lib-hdf-2.7/lib/linux64/libjhdf.so
-Dncsa.hdf.hdf5lib.H5.hdf5lib=/opt/beam-4.9.0.1/modules/lib-hdf-2.7/lib/linux64/libjhdf5.so
-jar /opt/beam-4.9.0.1/bin/ceres-launcher.jar /home/pfsc/hiprogen/scripts/config/BEAM_SDR.xml -e
-PinputFile1=~/SDR_processing/working/MER_FRS_1PNUPA20030706_080236_000003152017_00479_07046_5458.N1
-PinputFile2=~/SDR_processing/working/MER_RR__2PRACR20030706_074541_000026402017_00479_07046_0000.N1
-PoutputFile1=~/SDR_processing/working/MER_FRS_1PNUPA20030706_080236_000003152017_00479_07046_5458.N1_SDR.dim

I have attached the content of the BEAM_SDR.xml that we have been using. We would be most grateful for any advice on what might be the cause of the Java ArrayIndexOutOfBoundsException and if there might be some workaround to resolve this issue.

Thanks for your help and support.

Kind regards,
Steven Hubbard

Attachments:

BEAM_SDR.xml (5.4k)

Dear Thomas,

Thanks for your feedback on this issue. I have been doing some further investigation of my own.

I found that the observed problem seems to be linked to the tiling that is performed and the calculation of the RR pixel position from the FRS pixel position. In the method computeTile() in RRToFRSOp.java, the RR starting point is calculated thus:

xStart = Math.round(rrPixelPos.x);
yStart = Math.round(rrPixelPos.y);

and there is then a 2-D for loop in which the RR position (rrX, rrY) is calculated based on the FRS position (x,y). The RR position (rrX, rrY) is then used to calculate the following:

double d = srcTile.getSampleDouble(rrX, rrY);

Using the debugger, I found that the failure occurred for a tile at the very end of the RR swath. For this tile, rrPixelPos.x was evaluated to be = 1020.59674, which resulted in xStart being rounded up to 1021. Then, in the 2-D for loop, with frsRectangle.width = 401, rrX was incremented from 1021 every 4th loop in x until finally the rrX value reached 1121. This caused the observed ArrayIndexOutOfBoundsException, since the RR pixel position array extends in x from 0 to 1120.

I could prevent this error occurring by checking and constraining the value of rrX used to calculate d thus:

if (rrX > srcTile.getMaxX()) {
rrX = srcTile.getMaxX();
}
double d = srcTile.getSampleDouble(rrX, rrY);

Interestingly, I then read the BEAM forum thread: BEAM performance issue and, following the advice, uncommented the following lines in the beam.config file:

beam.envisat.tileWidth = *
beam.envisat.tileHeight = 64

This change not only seemed to reduce the BEAM SDR processing time by at least 4-5 times, it also reduced the incidence of the reported "Coordinate out of bounds!" issue. I suppose that this is because the computed tile now extends across the whole width of the swath and the rounding error is thus less likely to occur.

I have been running the BEAM SDR processing with -XmX2048M. I tried to increase beam.envisat.tileHeight = 128, but then I was observing java.lang.OutOfMemoryErrors. However, our Linux platform has 16GB memory, so I could try to increase the memory setting and experiment further.

Many thanks for your help and support.

Steven