Memory - overhead problem

[jeroen.dries]

Hi Andrea,

it’s already quite efficient. One trick we can add is to change the data type to something smaller. I’m guessing you now get float32 output, and we can scale this for instance to shorts. If your current range is between 0 and 1, you could do:
cube.linear_scale_range(0.0,1.0,0,10000)

It is however nog guaranteed to work, so simply increasing memory is also an option. You can try the settings below to add 1GB to each worker:

@jeroen.dries

job_options = {
        "executor-memory": "3G",
        "executor-memoryOverhead": "3G",
        "executor-cores": "2"
}
cube.execute_batch(  out_format="GTiff",
        job_options=job_options)