UDF correlation

Luckily it is not, but it is good to know :slight_smile:

minor update:

I have he impression that the issue only occurs when the CLP based mask is in play. If you don’t include CLP in the masking, it seems that I get a perfect match between “local” and “remote openeo” UDF evaluation.

The other way around is also interesting: only doing the CLP based masking, (no CLD, no water/land) .
You get more output pixels and you can see that the “local” way is more consistent than the “remote openeo” way.
For example: with a slightly larger region of interest:

The difference image on the bottom shows an inherent difference between the two ways: the local way works on the whole region as a whole, while openeo executes the UDF on different chunks (of 256x256 pixels) separately. However, I still don’t understand how the chunking could impact the result, because the UDF only works in the time dimension

Very well to know. Sadly, the CLP filter seems to be the one that works better, to my experience.

In the meanwhile, I have a quick additional general question: I am tryng to run my code to obtain the last results for the LPS. Sadly, I cannot run the code since as soon as I request to execute a part of the code I obtain this error:

('Connection aborted.', ConnectionResetError(10054, "Connessione in corso interrotta forzatamente dall'host remoto", None, 10054, None))

Is this error “openeo side”? Is there a way to solve it aside “try again”?

what openeo connection URL are you using at the moment?

Is it still https://openeo-dev.vito.be ? That still works normally from my end.

I was actually connected to https://openeo.cloud
But I can try with dev

No, I still can’t get anything most of the time. Maybe there is too much traffic with the servers or my operation are too computational demanding?

I think I found the bug that caused the whole local vs openeo mismatch:

ID=np.where(np.in1d(D, IP[:,1]))[0]

should be (both in the UDF and local calculation):

ID=np.where(np.in1d(IP[:,1], D))[0]

If I change that in my test notebook, local and openeo results match perfectly.

Thanks @stefaan.lippens. That is a very good news, even if I’m sorry to have done a so stupid mistake. I’ll try to fix this on my code and verify if everything works fine. But I don’t know if I’ll be able to obtain the results in time for LPS since I am still getting the error

('Connection aborted.', ConnectionResetError(10054, "Connessione in corso interrotta forzatamente dall'host remoto", None, 10054, None))

Maybe my code preoperation (masking of clouds and other things) has become too computationally heavy…

When do you get this error exactly? Immediately on a .download() call, or after like 5 or 10 minutes? Can you also provide a full stack trace?
It’s also bit strange that the error message is in Italian for you, do you have a special setup that localizes error messages?

That is indeed possible, there is a limit on synchronous download() calls. If you hit it, you could switch to batch jobs (e.g. execute_batch(), get_results, …)

I get the error after some minutes of operation. But I don’t get it always: sometimes the code work, and sometimes the error is raised. It seems to me that it appear more often when I try to run more session of python together in different console (to analyze different river sections at the same time). This operation usually works only by night (probably because the servers are more free?).

I can indeed try to swith to batch jobs in order to make the code work, thanks for the advice :slight_smile: