Welcome to the BEAM Forum!

We encourage you to sign in our forum and participate in the BEAM community. The forum is maintained by the BEAM project team who will most likely answer your questions within 24 hours (except during common holidays) - if not done by other community members. Collaborate, share your knowledge and learn from other users!

If you don't find what you are looking for, please also consider the following external forums:

Combination View Flat View Tree View
Threads [ Previous | Next ]
RE: gpf from inside java code not utilizing multiple CPU cores
toggle
Hello everyone,

I'm seeing a difference in performance when running a CPU intensive pipeline using the command-line tool gpt and from inside Java code.

With gpt, graph execution takes about 9 minutes and htop shows all 8 CPU cores at 100% utilization.
With the Java code, the same graph takes 15 minutes. According to htop, on average 2 CPU cores are being used at 100% (average of 25% CPU usage).

I've tried setting the system properties beam.parallelism and seadas.parallelism with no noticeable improvement.

I'm invoking the GPF from Java using the following code:
1
2ProgressMonitor pm;
3...
4Graph graph = GraphIO.read(new StringReader(graphStringI));
5GraphProcessor processor = new GraphProcessor();
6processor.executeGraph(graph, pm);


Some further information which might be useful:
  • I'm running Ubuntu 12.04 on a 64-bit machine with 8GB RAM and a Corei7-3770 CPU.
  • Using SeaDAS 7.0.2 and Open JDK (1.7.0_55).
  • The graph XML produces 7 day averages of MODIS aqua chlorophyll data over the Southern GBR region for a month.
  • The pipeline has been made artificially CPU intensive, by performing extra unrelated calculations inside computeTile() of the band averaging operator.
  • The Java code is run as a junit test case from Maven.


(My question is fairly similar to this forum question, but with some differences as mentioned above.)

What could I be missing here?

thanks,
Kutila
Flag Flag
RE: gpf from inside java code not utilizing multiple CPU cores
Answer Answer (Unmark)
6/11/14 2:04 PM as a reply to Kutila Gunasekera.
Dear Kutila,

when executing a graph using the Java API of GPF you have to initialize geotools and especially JAI by invoking:
1SystemUtils.init3rdPartyLibs


This initializes JAI, which is used for scheduling multiple computational threads in parallel, to the use all available cores. By default only 2 threads are used.

Best regards,
Marco
Flag Flag
RE: gpf from inside java code not utilizing multiple CPU cores
6/12/14 7:19 AM as a reply to Marco Zühlke.
That fixed it. Thanks Marco!
Flag Flag