Error when executing process with ERA5 data collection

@daniel.thiex I receive an error when trying to perform basic operation on ERA5_WIND_U / ERA5_WIND_V data collection using openEO platform backend.

OpenEoApiError: [500] Internal: Failed to create job on backend ‘sentinelhub’: OpenEoApiError(“[500] Internal: Server error: ‘NoneType’ object is not subscriptable”) (ref: r-a30d5fd7fe72442ab8a3d981962897d4) ,

I even tested for the same data using the Sentinel hub backend but again received an error:

OpenEoApiError: [500] Internal: Server error: ‘NoneType’ object is not subscriptable

The used code snippet is:

datacube = connection.load_collection( “ERA5_WIND_U”,
spatial_extent={​​​​​​​"west": 16.06, “south”: 48.06, “east”: 16.65, “north”: 48.35}​​​​​​​,
temporal_extent=[“2020-03-01”, “2020-10-01”],
)
era5_job = datacube.create_job(title=“test”)

Running this pg in the editor seems to work fine so I feel this might be connected to the python client. @stefaan.lippens Do you maybe have an idea what’s going on here?

{
  "process_graph": {
    "load1": {
      "process_id": "load_collection",
      "arguments": {
        "id": "ERA5_WIND_U",
        "spatial_extent": {
          "west": 16.06,
          "east": 16.070415502630215,
          "south": 48.3427154929816,
          "north": 48.349999999999994
        },
        "temporal_extent": [
          "2020-03-01T00:00:00Z",
          "2020-10-01T00:00:00Z"
        ],
        "bands": [
          "windu10m"
        ]
      }
    },
    "save2": {
      "process_id": "save_result",
      "arguments": {
        "data": {
          "from_node": "load1"
        },
        "format": "GTIFF"
      },
      "result": true
    }
  },
  "parameters": []
}

That Python reproduction snippet is very simple, I would not expect that the root cause is at the level of the python client.

One difference of the snippet and your process graph is that you have an explicit bands argument on load_collection, could it be related to that?

1 Like

Another thing @pratichhya.sharma is that you don’t have a save_result node in the resulting process graph: you don’t set it explicitly (e.g. with datacube.save_result(format="GTiff")), nor implicitly by setting a out_format in create_job().
So you basically send just this process graph to the backend:

{
  "process_graph": {
    "loadcollection1": {
      "process_id": "load_collection",
      "arguments": {
        "id": "ERA5_WIND_U",
        "spatial_extent": {
          "west": 16.06,
          "south": 48.06,
          "east": 16.65,
          "north": 48.35
        },
        "temporal_extent": [
          "2020-03-01",
          "2020-10-01"
        ]
      },
      "result": true
    }
  }
}
1 Like

@stefaan.lippens You are correct (both about the missing bands as well as the missing save results process), thanks for the pointer!

@pratichhya.sharma Next to the load collection (which should specify the bands you want to use) you need to at least include a save results process. Here a code snippet that should work:

datacube = connection.load_collection(
    "ERA5_WIND_U",
    spatial_extent={"west": 16.06, "south": 48.06, "east": 16.65, 
                    "north": 48.35},
    temporal_extent=["2020-03-01", "2020-10-01"],
    bands=["windu10m"]
)

# Save results
datacube_tiff = datacube.save_result(format="GTiff")

era5_job = datacube_tiff.create_job(title="test")

Yes, it worked well. Thank you both

@daniel.thiex sorry for quick response earlier, it sucessfully created job this time without an error but again failed with:

OpenEoApiError: [500] Internal: Server error: unsupported operand type(s) for *: ‘float’ and ‘NoneType’

In order to be able to better help you can you please share with us the steps you did to come to this error message (and/or the process graph you used).

If you don’t know it yet I would also recommend you to take a look at our web editor (https://editor.openeo.cloud/). It offers a simple UI to created process graphs and to execute synchronous and asynchronous job and might help you debugging your script.

@daniel.thiex just to be sure, what is the cause for the error originally reported (“‘NoneType’ object is not subscriptable”): the missing bands or the missing save_result?
Or maybe @pratichhya.sharma can try this out?

@daniel.thiex
This is my process graph

{
  "process_graph": {
    "loadcollection1": {
      "process_id": "load_collection",
      "arguments": {
        "bands": [
          "windu10m"
        ],
        "id": "ERA5_WIND_U",
        "spatial_extent": {
          "west": 16.06,
          "south": 48.06,
          "east": 16.65,
          "north": 48.35
        },
        "temporal_extent": [
          "2020-03-01",
          "2020-10-01"
        ]
      }
    },
    "saveresult1": {
      "process_id": "save_result",
      "arguments": {
        "data": {
          "from_node": "loadcollection1"
        },
        "format": "GTiff",
        "options": {}
      },
      "result": true
    }
  }
}

which I received by executing this code:

import openeo

connection = openeo.connect("jjdxlu8vwl.execute-api.eu-central-1.amazonaws.com/production").authenticate_oidc()
# connection = openeo.connect("openeocloud.vito.be").authenticate_oidc()

datacube = connection.load_collection(
    "ERA5_WIND_U",
    spatial_extent={"west": 16.06, "south": 48.06, "east": 16.65, 
                    "north": 48.35},
    temporal_extent=["2020-03-01", "2020-10-01"],
    bands=["windu10m"]
)

# Save results
datacube_tiff = datacube.save_result(format="GTiff")

era5_job = datacube_tiff.create_job(title="test")
# era5_result = era5_job.start_and_wait()

I am not sure if I am missing something or what am I doing that creates this error.

@stefaan.lippens I tried using both save_result and bands & also not including bands but including save_result that gave same error:

OpenEoApiError: [500] Internal: Server error: unsupported operand type(s) for *: ‘float’ and ‘NoneType’

whereas specifying bands but neglecting save_result was the cause of initial error I guess i.e:

OpenEoApiError: [500] Internal: Server error: ‘NoneType’ object is not subscriptable

The cause of the error is the missing save_result. If you leave the band away it will work (in this case all bands are loaded). As the collection only has 1 band it’s the same as specifying this one band. For easier reading I always find it though better to be implicit explicit here.

1 Like

I am unfortunately not able to replicate your error (until the create_job) everything works fine for me. Which openEO version are you using?

When I actually try to start the job (line: era5_result = era5_job.start_and_wait().) there seems however to be some bug on our side. We are looking into this and I will report here once this one is fixed.

I guess you meant “explicit” (specifying bands even if it’s equivalent to the default)?

I am using openeo 0.15.0 in this case.

…correct, updated.

Hi @daniel.thiex ,

I was wondering if there is an update on any developments regarding this issue. Nevertheless, if there are other ways to execute the workflow, please guide me with that.