Error using exp() process, Sentinel-1 collection

openEO Platform
1

Hi there,
I am trying to apply exp() on SENTINEL1_GRD, however, I am getting the error:
'py4j.protocol.Py4JJavaError: An error occurred while calling ' 'o1028.expressionEnd.\n' ': java.lang.IllegalArgumentException: Unsupported operation: exp ' '(arguments: [p])\n' '\tat

Please can you advice the correct way to use this proces?

This is my code:

import openeo
from openeo.processes import exp, array_element

start_date = '2021-01-01'
end_date = '2021-02-01'
spatial_extent  = {'west': -74.06810760, 'east': -73.90597343, 'south': 4.689864510, 'north': 4.724080996, 'crs': 4326} #colombia


s1_cube = connection.load_collection(
    'SENTINEL1_GRD', 
     spatial_extent = spatial_extent, 
     temporal_extent = [start_date, end_date], 
     bands = ['VH','VV'],
     properties = {"polarization": lambda p: p == "DV"} ##VV/VH only.
).ard_normalized_radar_backscatter()

def log_(x):
  return 10 * log(x, 10)

# s1_median = s1_cube.median_time().apply(log_)
s1_median = s1_cube.apply(log_).median_time()

def s1_water_function(data):
    vv = array_element(data, index = 1)                 ##returns fine.
    water = 1 / (1 + exp(- (-7.17 + (-0.48 * vv))))     ##failing.
    return water

median_water = s1_median.reduce_dimension(reducer = s1_water_function, dimension = "bands")
2

What connection URL are you using?

As mentioned in Error using exp() process - #7 , the exp process is only on our dev instance openeo-dev.vito.be at the moment, not yet on the production instances openeo.cloud or openeo.vito.be

3

we’ve used the openeo-dev.vito.be connection. @dadr
Could you let us know what can be a problem please?

4

When did you try and got the failure?

Because I just tried your code sample against openeo-dev.vito.be and it worked.
We had a deploy of openeo-dev.vito.be somewhere this morning, so maybe you tried before that

5

This minimal code snippet using exp works for me:

import openeo
from openeo.processes import exp

connection = openeo.connect("openeo-dev.vito.be").authenticate_oidc()

def convert(data):
    return 3 + exp(data * 0.0001)

cube = s2_cube.apply_dimension(process=convert, dimension="bands")

# try synchronous download
cube.download("forum397-minimal.nc")

# try batch job
job = cube.execute_batch()
job.get_results().download_files("forum397-minimal-results")

Can you confirm that it does or does not work for you too?

6

It works for the S2 collection well. Will you try it for the S1 collection?

7

hmm apparently I made a copy-paste mistake, and the forum doesn’t allow me to update my previous post. This is the minimal exp example code snippet I meant to send:

import openeo
from openeo.processes import exp

connection = openeo.connect("openeo-dev.vito.be").authenticate_oidc()

s2_cube = connection.load_collection(
    "SENTINEL2_L2A",
    spatial_extent={"west": 4.00, "south": 51.04, "east": 4.10, "north": 51.1},
    temporal_extent=["2020-09-01", "2020-09-30"],
    bands=["B02", "B03", "B04"]
)

def convert(data):
    return 3 + exp(data * 0.0001)

cube = s2_cube.apply_dimension(process=convert, dimension="bands")

# try synchronous download
cube.download("forum397-minimal.nc")

# try batch job
job = cube.execute_batch()
job.get_results().download_files("forum397-minimal-results")
8

Hi Stefaan,

We also got that code snippet to work for Sentinel-2 data. However, the new problem is that it doesn’t seem to be working on Sentinel-1 data. Interestingly it completes succesfully as a synchronous download, but not as a batch download. What is the reason for that?

import openeo
from openeo.processes import exp, array_element

start_date = '2021-01-01'
end_date = '2021-02-01'
spatial_extent  = {'west': -74.06810760, 'east': -73.90597343, 'south': 4.689864510, 'north': 4.724080996, 'crs': 'epsg:4326'} #colombia


s1_cube = connection.load_collection(
    'SENTINEL1_GRD', 
     spatial_extent = spatial_extent, 
     temporal_extent = [start_date, end_date], 
     bands = ['VH','VV'],
     properties = {"polarization": lambda p: p == "DV"} 
).ard_normalized_radar_backscatter()

def log_(x):
  return 10 * log(x, 10)

s1_median = s1_cube.apply(log_).median_time()

def s1_water_function(data):
    vv = array_element(data, index = 1)                
    water = 1 / (1 + exp(- (-7.17 + (-0.48 * vv))))    
    return water

median_water = s1_median.reduce_dimension(reducer = s1_water_function, dimension = "bands")

# try synchronous download
median_water.download("s1-median-water.nc") ## Works fine.

# try batch job
job = median_water.execute_batch()
job.get_results().download_files("s1-median-water-results") ## Fails
9

We think it is caused by an outdated dependency that pops up for batch jobs in some geographical regions. I have a colleague working on that and I will circle back when that is resolved.

10

My colleague updated the dependency so now it should work with the openeo-dev.vito.be back-end. Can you confirm?

11

Hi Stefaan. I just checked this and the batch job succeeded this time. Thanks.

12

great, thanks for confirming