Terrascope backend upgrade

Anther major update of the Terrascope backend:

  • merge_cubes and mask will now try to automatically resample input data cubes if required, reducing the need to always use resample_cube_spatial
  • When catalogs contain 2 versions of the same product, we now detect this better and try to use the most recent version. This improves performance, but also reproducability.
  • sar_backscatter based on sentinelhub has improved defaults for orthorectification

The majority of improvements in this release are smaller performance improvements and bugfixes.

1 Like

The Terrascope production backend received another upgrade.
Main features listed below!

2023-02-27 (0.6.7a1)

2023-02-07 (0.6.7a1)

  • Added initial support for the inspect process. It can be used on datacubes and in callbacks.
  • The size of a single chunk is now automatically increased for larger jobs, to improve IO performance.
  • resample_cube_spatial is no longer needed in all cases when using merge_cubesor mask
  • Better detection of duplicate products in source catalogs
  • The ‘if’ process will no longer evaluate the branch that is not accepted implement lazy `if` branching · Issue #109 · Open-EO/openeo-python-driver · GitHub

The previous update of Terrascope had to be rolled back for a short time, but is now back in effect.
In addition, this is included:

  • improvements to messages that are logged, as part of an ongoing effort to include quality of messages
  • area calculated and reported by batch job metadata should now be more accurate
  • derived-from links when using Sentinelhub based collections have been improved
  • more robustness for functionality that depends on external services, by adding more retries before failing
  • a new ‘filename_prefix’ format option allows more control over the filename of generated assets, which is otherwise set to default values like ‘openEO.tif’.

The Terrascope production backend received another upgrade:

  • a bug in aggregate_temporal is fixed, it could cause the first aggregation to be incorrectly set to nodata
  • logs are being improved, for instance to also show error messages generated by user code in a UDF

In today’s upgrade the Terrascope backend started running on Spark 3.3.1. Generally speaking this should not result in very visible changes, but do let us know if you suddenly see issues!

While some intermediate upgrades with smaller fixes were released over summer, we now again have a pretty major upgrade, featuring new processes for vector cubes and many small fixes.

One major change is the interpretation of time intervals as half-open, which was previously announced. This aligns the Terrascope backend with the specification and ensures compatibility with other backends.

New collections were also added:


The detailed list of changes can be found in our changelog, and contains many fixes and features that were requested by openEO platform users on this forum.