Setting a cloudfilter in load_collections

openEO Platform Clients
1

Hi,

According to the openEO process manual a cloud filter can be set in the properties
argument of load_collection. In the manual the filter is given as JSON string, to be
able to use it with the R client we translated it as seen below.

collections = "SENTINEL2_L2A_SENTINELHUB"
bands = c("B02", "B03", "B04", "B08", "SCL")
period = c("2021-04-01", "2021-06-30")
ext = list(11.12692, 49.74992, 11.15708, 49.76938)
names(ext) = c("west", "south", "east", "north")

# change to login with your credentials
# con = openeo::connect("https://openeo.cloud")
# login(
#     login_type = "oidc"
#     , provider = "egi"
#     , config = list(
#         client_id = "<client_id>"
#         , secret = "<secret>"
#     )
# )

procs = processes()

cloudfilter = list(
    "eo:cloud_cover" = list(
        "process_graph" = list(
            "cc" = list(
                "process_id" = "between"
                , "arguments" = list(
                    "x" = list("from_parameter" = "value")
                    , "max" = 50
                    , "min" = 0
                )
                , "result" = TRUE
            )
        )
    )
)


cube = procs$load_collection(
    id = collections
    , spatial_extent = ext
    , temporal_extent = period
    , bands = bands
    , properties = cloudfilter
)

## create process graph
graph = procs$save_result(
    data = cube
    , format = "GTiff"
)

## create and start job
job = openeo::create_job(graph)
openeo::start_job(job)

The call gives a server Error, saying process “between” is not supported. We have already reported this here. The same error type is thrown when using “lt” or “lte” instead of “between”.

How would a working example for the cloud filter look like?

1 Like
2

The argument sub type metadata-filter is currently missing in the package and was probably included after API version 1.0.0. I have to get some more information regarding the API, first, because it seems it requires a different serialization.
I have also opened an issue on Github about this: Improve meta data filtering on load_collection ¡ Issue #102 ¡ Open-EO/openeo-r-client ¡ GitHub

3

Hey,

the code for the between filter looks mostly good, except that you probably need to use list("from_parameter" = "x") instead of referring to value. Still, the issue seems back-end related as confirmed in the other thread. To me it looks “lte” is also not working on the server-side (“SERVER-ERROR: [500] unknown: process lte is not supported”), we’ll need to check. @jeroen.dries

4

The exact property needed is still a bit collection specific. We’re looking into that and will also fix the filtering on Sentinelhub.
Note that processes for collection filtering are entirely different from everything else, so again a separate code path! (Meaning support for ‘between’ and ‘lt’ is indeed not there.)

5

This now works:

    rgb = connection.load_collection("SENTINEL2_L2A_SENTINELHUB",
            spatial_extent={'west':3.758216409030558,'east':4.087806252,'south':51.291835566,'north':51.3927399},
            temporal_extent=["2020-03-07","2020-04-30"],bands=['B04','B03','B02'],properties={
            "eo:cloud_cover": lambda cc:lte(cc, 50 )
        } )

I would really recommend it in any workflow, as even filtering on for instance 95% cloudcover gets rid of some entirely useless observations, and thus improves performance.

1 Like
6

Hi,
thanks for the update. We fixed our previous translation for the R-client to use lte instead of between. With the following code this now works perfectly. After defining ‘cloudfilter’ we pass it to the properties argument in load_collections.

cloudfilter = list(
        "eo:cloud_cover" = list(
            "process_graph" = list(
                "cc" = list(
                    "process_id" = "lte"
                    , "arguments" = list(
                        "x" = list("from_parameter" = "value")
                        , "y" = p
                    )
                    , "result" = TRUE
                )
            )
        )
    )
7

Hi,

After the update to version 1.2.0 of the R-client this property filter no longer works. The error message reads:
“Error in FUN(X[[i]], …) :
Value for the metadata filter needs to be a list of functions.”

Can you maybe provide an updated version of how above filter could be implemented now?

Thanks as always!

8

Hi,

I included a small sample in the documentation to the meta data filter ?MetadataFilter.

So cloud filtering can be done like this:

data = p$load_collection(id = "SENTINEL2_L2A",
                         bands=c("B04","B08"),
                         spatial_extent = list(west=6.75,south=51.85,east=7.25,north=52.15),
                         temporal_extent = c("2021-03-01","2021-05-15"),
                         properties = list(
                           "eo:cloud_cover" = function(x) x <= 50
                         ))

Best,
Florian

1 Like