RE: Is there an efficient way to apply a kernel

RE: beam-meris-icol rel 2.9 issues

2012-05-16T14:54:08Z

Hi Tyn,

Valentijn Venus:

I'd be happy to try, can you give me an example for one of the other processors? Here is the output i get from the terminal on Macosx Lion:

Valentijn-Venuss-MacBook-Pro:bin venus$ pwd
/Applications/beam/bin
Valentijn-Venuss-MacBook-Pro:bin venus$ gpt -h icol.ThematicMapper
gpt: illegal option -- h
usage: gpt [-rv] [-p nparts] command [options] device ...

Obviously, you have some other program called 'gpt' in your path. You need to make sure that you call (on MacOS):
/Applications/beam/bin/gpt.command -h

On Windows/Linux, you have gpt.bat/gpt.sh, respectively.

I think i match the required hardware requirements and have the opencl-jars in the lib folder, can you suggest how to turn this experimental feature back on. I will not ask further questions if i do get error messages, promise...

Well, maybe you are lucky if you say you meet the requirements... It is as you already assumed: use the command-line parameter '-PopenclConvolution=true'.

Cheers,
Olaf

RE: gpt Error: Null, when using Meris.N1Patcher over an ICOL2.9 with cloud

2012-05-16T14:41:10Z

Dear Jose,

Jose M. Beltran:

Dear Olaf,

Although, ICOL still have a previously reported issue that if I display and save the request file, ii only displays and saves the default options. Not the ones that a user might have changed.

Jose

I fixed this issue as well, it will come with the next ICOL update. If you find further issues in the meantime, please let us know...

Cheers,
Olaf

RE: beam-meris-icol rel 2.9 issues

2012-05-16T11:06:32Z

Dear Olaf,

[/quote]
I cannot reproduce this, gpt -h icol.ThematicMapper works fine here in various enviromnents. Does your 'gpt -h' work for other processors?

In principle you can, but you will likely get error messages.

I forward to you the answer of my colleague who did the implementation of this land-water database:[/quote]

thanks, the reply was helpful indeed.

Cheers, Tyn

RE: band math gives bad result

2012-05-16T10:05:11Z

Hi Evgenia,

Sorry for not catching up for so long. Do you still suffer from your problem? If so, please attach the XML file, there must be something wrong with the file format, since "CDOM_Kow" is not part of your expression.

Norman

RE: beam-meris-icol rel 2.9 issues

2012-05-16T09:54:26Z

Dear Valentijn,

please see my comments below.

Best regards,
Olaf

Valentijn Venus:

I cannot reproduce this, gpt -h icol.ThematicMapper works fine here in various enviromnents. Does your 'gpt -h' work for other processors?

2- The openclConvolutionCheckBox is (by design) disabled from the user (greyed out) in this release.

Can I somehow still turn this boolean to ‘true’ using some configuration (xml) file? Is it still accessible from the command-line as parameter -PopenclConvolution=boolean?

In principle you can, but you will likely get error messages. The ICOL processing with OpenCL technology is strongly dependent on the hardware and configuration of the particular computer, this is why it is meant to be an experimental option. It is impossible for us to remotely give individual support on this for the whole community of ICOL users. We hope for your understanding.

3- As—per the release notes, this ICOL version uses an improved land-water database which is based on SRTM data. I have noticed the extension has a new dependency (beam-watermask-1.2) also, which I guess provides this improved land-water database. Can you indicate which of the following files is used by ICOL (I guess GC_water_mask.zip)?

file://localhost/Users/venus/Downloads/beam-watermask-1.2/auxdata/images/50m.zip
file://localhost/Users/venus/Downloads/beam-watermask-1.2/auxdata/images/150m.zip
file://localhost/Users/venus/Downloads/beam-watermask-1.2/auxdata/images/GC_water_mask.zip
file://localhost/Users/venus/Downloads/beam-watermask-1.2/auxdata/images/MODIS_north_water_mask.zip
file://localhost/Users/venus/Downloads/beam-watermask-1.2/auxdata/images/MODIS_south_water_mask.zip

I forward to you the answer of my colleague who did the implementation of this land-water database:
"You are right, the dependency to 'beam-watermask-1.2' provides the improved land-water database. In fact, all of the files you mention are used in ICOL apart from the MODIS_north and MODIS_south; the '50m.zip' contains the high-resolution database for regions between latitude 60°S and 60°N, the '150m.zip' contains the low-resolution database for the same regions, and the 'GC_water_mask.zip' contains the water mask used for regions north of 60° N. Below 60° S, there is no database yet."

RE: BEAM SDR processing failures: Coordinate out of bounds!

2012-05-16T09:26:23Z

Dear Thomas,

Thanks for your feedback on this issue. I have been doing some further investigation of my own.

I found that the observed problem seems to be linked to the tiling that is performed and the calculation of the RR pixel position from the FRS pixel position. In the method computeTile() in RRToFRSOp.java, the RR starting point is calculated thus:

xStart = Math.round(rrPixelPos.x);
yStart = Math.round(rrPixelPos.y);

and there is then a 2-D for loop in which the RR position (rrX, rrY) is calculated based on the FRS position (x,y). The RR position (rrX, rrY) is then used to calculate the following:

double d = srcTile.getSampleDouble(rrX, rrY);

Using the debugger, I found that the failure occurred for a tile at the very end of the RR swath. For this tile, rrPixelPos.x was evaluated to be = 1020.59674, which resulted in xStart being rounded up to 1021. Then, in the 2-D for loop, with frsRectangle.width = 401, rrX was incremented from 1021 every 4th loop in x until finally the rrX value reached 1121. This caused the observed ArrayIndexOutOfBoundsException, since the RR pixel position array extends in x from 0 to 1120.

I could prevent this error occurring by checking and constraining the value of rrX used to calculate d thus:

if (rrX > srcTile.getMaxX()) {
rrX = srcTile.getMaxX();
}
double d = srcTile.getSampleDouble(rrX, rrY);

Interestingly, I then read the BEAM forum thread: BEAM performance issue and, following the advice, uncommented the following lines in the beam.config file:

beam.envisat.tileWidth = *
beam.envisat.tileHeight = 64

This change not only seemed to reduce the BEAM SDR processing time by at least 4-5 times, it also reduced the incidence of the reported "Coordinate out of bounds!" issue. I suppose that this is because the computed tile now extends across the whole width of the swath and the rounding error is thus less likely to occur.

I have been running the BEAM SDR processing with -XmX2048M. I tried to increase beam.envisat.tileHeight = 128, but then I was observing java.lang.OutOfMemoryErrors. However, our Linux platform has 16GB memory, so I could try to increase the memory setting and experiment further.

Many thanks for your help and support.

Steven

RE: gpt Error: Null, when using Meris.N1Patcher over an ICOL2.9 with cloud

2012-05-16T09:09:26Z

Dear Olaf,

Thank you ! the problem has been fixed. And I saw as well that this fix did not dropped AMORGOS correction bands.

Although, ICOL still have a previously reported issue that if I display and save the request file, ii only displays and saves the default options. Not the ones that a user might have changed.

Cheers!

Jose

RE: gpt Error: Null, when using Meris.N1Patcher over an ICOL2.9 with cloud

2012-05-15T15:42:00Z

Dear Jose,

thanks for the sources. I found and fixed this problem. Then I checked your example, it worked fine with both Visat and gpt command line. A new version of ICOL (v2.9.1) is available for installation, please see Module Manager --> Module Updates in BEAM/Visat. - Please let us know if you still observe any problems.

Thanks and best regards,
Olaf

beam-meris-icol rel 2.9 issues

2012-05-15T15:39:08Z

Dear support,

In the latest version (2.9) of the beam-meris-icol processor I have found the following issues:

1- I cannot invoke the command-line processor help using:
${BEAM-INSTALL-DIR}/bin/gpt -h icol.ThematicMapper

Any ideas?

2- The openclConvolutionCheckBox is (by design) disabled from the user (greyed out) in this release.

Can I somehow still turn this boolean to ‘true’ using some configuration (xml) file? Is it still accessible from the command-line as parameter -PopenclConvolution=boolean?

3- As—per the release notes, this ICOL version uses an improved land-water database which is based on SRTM data. I have noticed the extension has a new dependency (beam-watermask-1.2) also, which I guess provides this improved land-water database. Can you indicate which of the following files is used by ICOL (I guess GC_water_mask.zip)?

file://localhost/Users/venus/Downloads/beam-watermask-1.2/auxdata/images/50m.zip
file://localhost/Users/venus/Downloads/beam-watermask-1.2/auxdata/images/150m.zip
file://localhost/Users/venus/Downloads/beam-watermask-1.2/auxdata/images/GC_water_mask.zip
file://localhost/Users/venus/Downloads/beam-watermask-1.2/auxdata/images/MODIS_north_water_mask.zip
file://localhost/Users/venus/Downloads/beam-watermask-1.2/auxdata/images/MODIS_south_water_mask.zip

Cheers, Valentijn

request for example of xml file of a Merge operator

2012-05-15T15:37:17Z

Dear forum,

Could you please share an example of how to set up correctly the xml graph for the new Merge operator?
I am not sure what the productId in the include section refers to. It is to the sourceProd name? or "Bands" if I want to select only some bands.

Thanks,

Jose

ICOL 2.9 drops MER_FSG related bands after processing

2012-05-15T13:20:52Z

Dear forum,

After processing MERIS FSG products with ICOL 2.9, I notice that the new ICOL does not keep the AMORGOS (FSG) related bands. i.e, corr_longitude, corr_latitude and altitude, in the output file.

Anyone facing the same issue? I have reported in the issue tracker and I hope it can be fixed soon, if it is a bug.

Jose

RE: Is there an efficient way to apply a kernel (solved)

2012-05-14T15:24:48Z

Dear Thomas,

I have found my problem. It was a stupid bug. Just invert the two first loops. arrrrrr !

1
2 for (int y = offset; y < tileHeight - offset; y++) {
3                      for (int x = offset; x < tileWidth - offset; x++) {
4                      int value = 0;
5                      for (int xk = -offset; xk <= offset; xk++) {
6                         for (int yk = -offset; yk <= offset; yk++) {
7                               int posIndex = (y + yk) * tileWidth + (x + xk);
8                               value += ((int) srcSamples[posIndex] & 0xff)
9                                     * (1.f / (float) (kernelSize * kernelSize));
10
11                         }
12                      }
13                      destSamples[destPixelPos] = value;
14                      destPixelPos++;
15                   }
16             }

Best regards

Erwan

RE: gpt Error: Null, when using Meris.N1Patcher over an ICOL2.9 with cloud

2012-05-14T14:11:38Z

Dear Jose,

it seems that from your preprocessing chain (equalize, smile etc.) you have additional bands in the FSG input product which the N1 patcher does not correctly take into account. Could you provide me your input product for testing, or at least give me a list of all bands and tie point grids in this product? - I just tried the N1 patcher with a 'normal' L1b input product, which has the input bands 'radiance_n' (n=1,...,15), l1_flags, and detector_index. In this case, it works fine.

Thanks,
Olaf

RE: gpt Error: Null, when using Meris.N1Patcher over an ICOL2.9 with cloud

2012-05-14T12:57:21Z

Hi Jose, did you really attach the graph XML etc.?

RE: gpt Error: Null, when using Meris.N1Patcher over an ICOL2.9 with cloud

2012-05-14T11:11:35Z

I have the same problem within BEAM4.10. Not using the gpt call at all. Just a simple ICOL processing with the output format as ENVISAT N1.
A screen shoot of the error message in beam is attached.
Thanks for your help,

Jose

RE: recommended minimum subset size for ICOL to run over a MER FR image

2012-05-14T09:35:47Z

Thanks Olaf,
That was the answer that I was looking for!
Cheers!

Jose

gpt Error: Null, when using Meris.N1Patcher over an ICOL2.9 with cloud mask

2012-05-14T09:31:20Z

Dear Forum,

I received and "error: Null". when I tried to save the result of my gpt graph back to ENVISAT N1 format. My source image is a MERIS L1b FSG product in ENVISAT N1 format. I am using the Correct Radiometry first to Equalize and Smile Correct my images, then I run ICOL2.9 using a using a cloud mask and as input the node results of correct radiometry, and I am trying to save the end product back to ENVISAT N1 format using the Meris.N1Patcher. If I omit the N1Patcher and save the end result as a BEAM-DIMAP format I received no errors. The errors are raised when I try to save it back to N1Patcher. If I omit the ICOL node, I can save the results with no problem with the ENVISAT N1 format. If I run the ICOL node alone I have the same problem.

Here is the call to the GPT and the xml graph is attached. The output that I got follows bellow, and the gpt graph is attached.

/

1home/jose/beam-4.10/bin/gpt.sh /home/jose/EOData3/scripts/gpt/current_gpt_graph.xml -SsourceProd=/home/jose/EOData3/Instrument/MERIS/V3/L1B/FSG/MER_FSG_1PNEPA20080725_092241_000000252070_00351_33471_3811.N1 -ScloudMaskProduct=/home/jose/EOData3/cloud_prob/V3_cloud_prob_MER_FSG_1PNEPA20080725_092241_000000252070_00351_33471_3811.nc
2
3
4SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
5SLF4J: Defaulting to no-operation (NOP) logger implementation
6SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
7Error: Could not load mediaLib accelerator wrapper classes. Continuing in pure Java mode.
8Occurs in: com.sun.media.jai.mlib.MediaLibAccessor
9com.sun.media.jai.mlib.MediaLibLoadException
10kernel sum = 0.08745136
11kernel sum = 0.08745136
12kernel sum = 0.8996342
13kernel sum = 0.8996342
14x, y = 73,101
15x, y = 73,101
16
17Error: null

Any hints? or advice? Is it a bug?

RE: recommended minimum subset size for ICOL to run over a MER FR image

2012-05-14T09:15:44Z

Dear Jose,

from theory, the adjacency effect (which ICOL shall correct for) is caused by two main contributions, the Rayleigh and the aerosol scattering. If you have land and water (ocean or inner lakes) in your image, the Rayleigh contribution is significant on a range of about 30km distance from a coastline, and the aerosol contribution on a range of about 10km. Therefore, 30*30 and 10*10km are the sizes of the ICOL "correction windows" for each pixel. So, if you consider a specific region of interest, we would recommend to use a subset which exceeds your region of interest by at least 30 kilometres to each direction (no matter if you use FR or RR images). In this way you make sure that sufficient information from the image is used in the ICOL correction.

I hope this helps.

Cheers,
Olaf

RE: Is there an efficient way to apply a kernel

2012-05-14T08:55:17Z

Dear Erwan,

Looking into your code, I have observed that your code can only work if a band with data type 'byte' is chosen as input; if you choose an input band with a different data type, a NullPointerException is thrown. But that's not the issue, is it?

Can you tell me what's wrong with the target image? Are the values ok in principle but wrongly distributed? Or are the values not ok?

Best regards,
Thomas

RE: Is there an efficient way to apply a kernel

2012-05-11T14:56:51Z

Dear Erwan,

could you please be a bit more specfic on what goes wrong, e.g. provide an error message?

Best regards,
Thomas