Slow create_job() + warning "preflight process graph validation failed"

openEO Platform
1

Hi,

This is related to this discussion post, but the context seems different to me, since I am not working with a UDP.

I recently upgraded the openeo Python package from version 0.22.0 to 0.27.0.

Since then, create_job() seems to go a lot slower than before. I compared the elapsed time for the creation of a batch job (with the same script) in two different environments:

  • environment with openeo 0.22.0 → 848 s (job-id: j-2401267b53df487f8d4cf46aa4941f0f)
  • environment with openeo 0.27.0 → 2 s (job-id: j-2401269f896444ffa8919dbb4d5b3a2d)

The job is created in both cases, but in the latter case, the following warning was printed:

Preflight process graph validation failed: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))

However, I’ve seen different variations of this warning, but it always starts with “Preflight process graph validation failed”.

Since it’s just a warning, it’s mainly the significant slow-down that concerns me. Is there any way to avoid it?

Kind regards,
Margot

2

Recent version of the openeo python library indeed do automatic validation of the process graph before sending it for actual processing. This might indeed cause job creation to be a bit slower.

If this is a problem, you can disable this auto validation feature by setting a flag when connecting, e.g.

connection = openeo.connect(..., auto_validate=False)

as documented at API (General) — openEO Python Client 0.28.0a1 documentation

3

Ok, thank you.