Memory - overhead problem

openEO Platform
21

I’m not sure I understand what you mean here. Some points that might be an answer:

  • instead of the filter_bbox process (which is equivalent to the spatial_extent in load_collection) to pass a west-south-east-nort bounding box, you can also use the filter_spatial process with for example a geojson structure to spatially filter with this geometry directly
  • if you use operations like mask and aggregate_spatial that require a limiting geometry by design, you can leave out a spatial extent in load_collection, as the VITO backend will infer that from the geometry used in mask/aggregate_spatial. Be careful that this feature is not supported on other backends necessarily.
22

Thanks for a good overview!

1. filter_spatial
It seems that it does not want to accept my geojson file.

Code:

s2_cube = connection.load_collection(
    'SENTINEL2_L2A_SENTINELHUB',
    # spatial_extent  = spatial_extent,
    temporal_extent = [start_date_exclusion, end_date],
    bands           = bands)

aoi = "../../aoi/DK/DK.geojson"

# aoi = "https://github.com/DHI-GRAS/worldwater-toolbox/blob/main/openeo_platform_notebooks/DK.geojson"
bbox = dict(zip(["west", "south", "east", "north"], gpd.read_file(aoi).total_bounds))
print(bbox)

s2_cube = s2_cube.filter_spatial(aoi)

s2_cube_save = s2_cube.save_result(format='netCDF') #GTiff #netCDF
my_job  = s2_cube_save.send_job(title="s2_cube")
results = my_job.start_and_wait().get_results()
results.download_files("s2_cube")

Error message: OpenEoApiError: [500] Internal: Server error: NotImplementedError("filter_spatial only supports dict but got DelayedVector('../../aoi/DK/DK.geojson')") (ref: r-1aaf592938bc41c69d022ee03bfb26e0)

The geojson file is uploaded on GitHub . The file is in EPSG:4326.

2. mask_polygon (another option which I have tried)

I have followed this instructions but it seems there is a problem to work with my geojson file. The bbox can be print {‘west’: 8.07638931, ‘south’: 54.55902863, ‘east’: 15.19305611, ‘north’: 57.75152588} so I believe that my file is fine but it can not be used with mask_ polygon or in filter_spatial

Code:

s2_cube = connection.load_collection(
    'SENTINEL2_L2A_SENTINELHUB',
    # spatial_extent  = spatial_extent,
    temporal_extent = [start_date_exclusion, end_date],
    bands           = bands)

aoi = "../../aoi/DK/DK.geojson"
bbox = dict(zip(["west", "south", "east", "north"], gpd.read_file(aoi).total_bounds))


s2_cube = s2_cube.filter_bbox(bbox)
s2_cube = s2_cube.mask_polygon(aoi)

s2_cube_save = s2_cube.save_result(format='netCDF') #GTiff #netCDF
my_job  = s2_cube_save.send_job(title="S2L2A_cube")
results = my_job.start_and_wait().get_results()

Error:
OpenEoApiError: [500] Internal: Server error: FileNotFoundError(2, 'No such file or directory') (ref: r-53be7db1c4934f15a2b6a6dd0c19064e)

Would you advice me what is incorrect in these codes please?

23

We indeed have this size limitation of 1 million sq km for SentinelHub powered collections.

In openEO Platform we are experimenting with features to support these kind of huge jobs, (e.g. by automatically splitting large jobs in smaller digestable jobs).
Some initial documentation can be found at Federation Aspects and Known Issues | openEO Platform Documentation

1 Like
24

You are passing here a local file path that is only valid in your environment, but does not exist on the openEO back-end side. Alternatively you can

1 Like
25

Works, thanks! :slight_smile: