How to get imagery footprint and filter by cloud coverage

dileomargherita · 28 January 2023 16:56

Hi,

I’m trying to load a set of Sentinel 2 over the whole Europe for a year (2018). What I’m trying to do is to calculate some vegetation indeces, so if I could display the footprint of the available imagery over a certain area, I could reduce my time span to include only a subset of the available images. Is it possible to display the footprint in a certain region within a certain time frame?

Furthermore, I would like to filter only images that are cloud-free. I’ve tried to reproduce the command from here Finding and loading data — openEO Python Client 0.14.0a1 documentation connection.load_collection(“SENTINEL2_L2A”,
…,
max_cloud_cover=80)
But I get the following error:
TypeError: load_collection() got an unexpected keyword argument ‘max_cloud_cover’

Thanks in advance to whom might help.

jeroen.dries · 29 January 2023 11:11

Hi @dileomargherita ,
it seems like the documentation needs updating. The property to use for cloud cover filtering is eo:cloud_cover can you try that?

Displaying the footprint is not directly supported by openEO, but it is a feature of the catalogs that we are based on. In the case of SENTINEL2_L2A, we in fact have the whole collection available.

There’s also multiple visual tools to retrieve this. One being the newly launched:

Or similarly:

Does that help?

best regards,
Jeroen

dileomargherita · 29 January 2023 14:40

Thank you Jeroen for your timely reply!
Unfortunately, neither the cloud_cover keyword argument seems to be supported:

TypeError: load_collection() got an unexpected keyword argument ‘cloud_cover’

inspect.signature(con.load_collection)
<Signature (collection_id: str, spatial_extent: Optional[Dict[str, float]] = None, temporal_extent: Optional[List[Union[str, datetime.datetime, datetime.date]]] = None, bands: Optional[List[str]] = None, properties: Optional[Dict[str, Union[str, openeo.internal.graph_building.PGNode, Callable]]] = None, fetch_metadata=True) → openeo.rest.datacube.DataCube>

Thanks

jeroen.dries · 29 January 2023 15:40

Ah wait, I made a mistake, the documentation was in fact correct, but can you check the version of your openEO python client?
It needs to be at least 0.13.0, and then the max_cloud_cover keyword should work.

diregola · 30 January 2023 10:17

I’m working on the JupyterLab provided by the platform itself, not on a remote client. Thanks

jeroen.dries · 30 January 2023 10:58

You can normally use conda to update the openeo package in that environment to the latest version.
I guess it has not been updated after the latest release, right @benjamin.schumacher ?

best regards,
Jeroen

dileomargherita · 30 January 2023 17:46

I’m not sure what I’m doing wrong but when I try

!conda update openeo

I get:

PackageNotInstalledError: Package is not installed in prefix.
prefix: /opt/conda
package name: openeo

I even tried updating conda itself before, but to no avail.

Thanks

jeroen.dries · 30 January 2023 18:40

@benjamin.schumacher @christoph.reimer Can you help with this one? We tried following the suggestion to use conda from our own documentation:

but doesn’t seem to work?

christoph.reimer · 31 January 2023 14:15

The openeo Python Client is only available via PyPI, therefore please use pip to upgrade the package to the latest version using the following command.

“pip install -U openeo”

dileomargherita · 31 January 2023 14:54

Thank you @christoph.reimer , so I updated OpenEO to v. 0.13.0, which seems to be the latest available, and then as @jeroen.dries suggested, I tried again to apply the max_cloud_cover, but it still does not work.

TypeError Traceback (most recent call last)
/tmp/ipykernel_2639/863939069.py in
1 # Load data cube
----> 2 cube = con.load_collection(“SENTINEL2_L2A_SENTINELHUB”,
3 spatial_extent=bbox,
4 temporal_extent=dates,
5 bands=bands,

TypeError: load_collection() got an unexpected keyword argument ‘max_cloud_cover’

Thanks

jeroen.dries · 31 January 2023 15:46

Did you restart the kernel after upgrading?
You can also check the version in your notebook by printing:
openeo.version

If the version is correct, the keyword argument should be there, as this is the relevant code:

github.com

Open-EO/openeo-python-client/blob/v0.13.0/openeo/rest/connection.py#L962


      
              """
              return self.datacube_from_flat_graph(load_json_resource(src), parameters=parameters)
          
          
def load_collection(
                  self,
                  collection_id: str,
                  spatial_extent: Optional[Dict[str, float]] = None,
                  temporal_extent: Optional[List[Union[str, datetime.datetime, datetime.date]]] = None,
                  bands: Optional[List[str]] = None,
                  properties: Optional[Dict[str, Union[str, PGNode, Callable]]] = None,
                  max_cloud_cover: Optional[float] = None,
                  fetch_metadata=True,
          ) -> DataCube:
              """
              Load a DataCube by collection id.
          
          
    :param collection_id: image collection identifier
              :param spatial_extent: limit data to specified bounding box or polygons
              :param temporal_extent: limit data to specified temporal interval
              :param bands: only add the specified bands
              :param properties: limit data by metadata property predicates

thanks,
Jeroen

dileomargherita · 31 January 2023 15:55

OK, I restarted the kernel, then:

print(openeo.version)
0.13.0

Load data cube

cube = con.load_collection(“SENTINEL2_L2A_SENTINELHUB”,
spatial_extent=bbox,
temporal_extent=dates,
bands=bands,
max_cloud_cover=80)

/opt/conda/lib/python3.9/site-packages/openeo/rest/connection.py:980: UserWarning: SENTINEL2_L2A_SENTINELHUB property filtering with properties that are undefined in the collection metadata (summaries): eo:cloud_cover.
return DataCube.load_collection(

michele.claus · 8 February 2023 09:28

@daniel.thiex I am also trying a simple example but I am getting an error from the SentinelHub back-end:

import openeo

conn = openeo.connect("openeo.cloud").authenticate_oidc("egi")

data = conn.load_collection("SENTINEL2_L2A_SENTINELHUB",
    spatial_extent={'west': 4.0, 'east': 4.08, 'south': 51.29, 'north': 51.39},
    temporal_extent=["2021-04-01", "2021-06-01"],
    bands=['B04', 'B03', 'B02'],
    max_cloud_cover=80,
).save_result(format='netCDF')

data.create_job(title='sample_cloud_80')

OpenEoApiError: [500] Internal: Failed to create job on backend 'sentinelhub': OpenEoApiError('[500] unknown: <!doctype html>\n<html lang=en>\n<title>500 Internal Server Error</title>\n<h1>Internal Server Error</h1>\n<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>\n') (ref: r-4856425504cb463eb7d5860b4b510c54)

michele.claus · 8 February 2023 10:22

@dileomargherita this solution works, which specifies to use the back-end from VITO and not the the one from SentinelHub (which currently returns the error showed in the previous message):

import openeo

conn = openeo.connect("openeo.cloud").authenticate_oidc("egi")

data = conn.load_collection("SENTINEL2_L2A_SENTINELHUB",
    spatial_extent={'west': 4.0, 'east': 4.08, 'south': 51.29, 'north': 51.39},
    temporal_extent=["2021-04-01", "2021-06-01"],
    bands=['B04', 'B03', 'B02'],
    max_cloud_cover=30,
    properties={"provider:backend":lambda v: v =="vito"}
    ).save_result(format='netCDF')

data.create_job(title='sample_cloud_30')

stefaan.lippens · 15 February 2023 16:04

Hi all, it’s not completely relevant anymore to this discussion, but since this week the openeo python package is now also available in conda through the conda-forge channel: Openeo :: Anaconda.org

daniel.thiex · 27 February 2023 16:25

@daniel.thiex I am also trying a simple example but I am getting an error from the SentinelHub back-end:

Sorry, about the late reply, I somehow missed this. I am a bit confused why in your example our backend was used as the if nothing specified Vito’s should be used. Running it on our backend I get indeed the same cryptic error which is due to the fact we don’t support netCDF. If you replace the format with format=GTiff it works fine. Will see if we can improve the error message for such cases.