Hi,
I notice that a significant portion of the total runtime of my job is in “queue”. An estimate of time per job would be useful when considering development time required.
What expectations should I have regarding this portion of the job? Are there targets/ estimates of the queue time per job? How does the infrastructure adapt to periods of high usage across the platform?
Thanks!
Hi Daniel,
I’m afraid that, at the moment, we can not give guarantees about how long a job will be in status “queued” or how long it will run (status “running”). It indeed heavily depends on the load of our infrastructure. It might be indeed an idea to provide estimates about this in some way.
Note that if you are concerned about the impact on your development time: it is recommended to develop your algorithms on small area’s of interest first and use synchronous execution for inspection of results (e.g. cube.download("result.nc"). Synchronous execution does not have a “queued” phase, so it should be faster than batch job execution, but there is a limit to the execution time (I think max 10 or 15 minutes at the moment), which is why you should work on a small area of interest first.
Maybe a bit more advice, complementary to Stefaan’s answer:
Finally, if you have a specific job that you think was queued for an unreasonable time, do let us know, it can always be the case that something else is going on.
We have tried to make the Terrascope Registration but we got this error:
Could you let us know why we can not complete the registration please?
Hi Andrea,
this is due to a migration that happened at EGI, we’re waiting for an approval from them, which seems to take more time than anticipated.
Will let you know when it should work again!
Hi Andrea,
can you try again? It should be fixed now!
