Renaming existing bands bugged?

Hi all,

When renaming some single band datacubes, I think I may have run into a bug, as I keep getting "MNDWI" back as band name, resulting is errors when merging:

dc_merged \
    .filter_bands(["MNDWI"]) \
    .apply(lambda _: processes.int(1)) \
    .rename_labels("bands", target=["water"], source=["MNDWI"])

I also tried the band method of Datacube, getting the same result.

When I do the same thing from the source data, using the same code, I am getting the correct result. Maybe something with UDFs?

Process Graph (warning, very long due to udfs):

{'loadcollection1': {'process_id': 'load_collection',
  'arguments': {'bands': ['B03', 'B08', 'B11', 'CLM', 'CLP'],
   'id': 'SENTINEL2_L1C_SENTINELHUB',
   'spatial_extent': {'west': 589947.873967385,
    'east': 596088.3303869166,
    'south': 5489655.1321837725,
    'north': 5494855.214093619,
    'crs': 'EPSG:32633'},
   'temporal_extent': ['2021-05-01', '2021-08-01']}},
 'renamelabels1': {'process_id': 'rename_labels',
  'arguments': {'data': {'from_node': 'loadcollection1'},
   'dimension': 'bands',
   'source': ['B03', 'B08', 'B11', 'CLM', 'CLP'],
   'target': ['green', 'nir', 'swir', 'cloudmask', 'cloudp']}},
 'chunkpolygon1': {'process_id': 'chunk_polygon',
  'arguments': {'chunks': {'type': 'MultiPolygon',
    'coordinates': [(((16.254265938049592, 49.58860047892567),
       (16.254322774071163, 49.58957887300433),
       (16.25441642107371, 49.58985086572059),
       (16.256000197238357, 49.59143153655345),
       (16.257132884506554, 49.59197557028358),
       (16.259841262431102, 49.59249901145751),
       (16.26101410419378, 49.592454466766604),
       (16.26321719165413, 49.591941110939764),
       (16.26407334982892, 49.591535306060756),
       (16.26557534838399, 49.59018898956556),
       (16.265758124333388, 49.58982330147995),
       (16.26596125811, 49.58898927367213),
       (16.265961225028647, 49.588766693202736),
       (16.266360051472724, 49.58838581956765),
       (16.266758833384223, 49.5880491401048),
       (16.26678119924604, 49.58798362150429),
       (16.267061232360483, 49.58771618411336),
       (16.26717760973574, 49.58762996590407),
       (16.267303870910126, 49.58748445641037),
       (16.267576243026916, 49.587224328985826),
       (16.267604955216125, 49.58713746825679),
       (16.268014343776965, 49.58666565451496),
       (16.26819263371431, 49.58629998697619),
       (16.26838007435019, 49.585333095734455),
       (16.268290918537236, 49.58430307900289),
       (16.268062614939154, 49.58401797516814),
       (16.268080583820495, 49.58401647088828),
       (16.26799139069211, 49.583566089884854),
       (16.267288805676117, 49.58264747424579),
       (16.267675292056342, 49.58214625518044),
       (16.26811675809452, 49.581822144087525),
       (16.268540924060144, 49.58119823302146),
       (16.269077368876758, 49.58076209099866),
       (16.26952768707333, 49.57986581849109),
       (16.269732832777287, 49.579027735012524),
       (16.269732726715574, 49.578170924887864),
       (16.26982689183397, 49.57820220718116),
       (16.27179992330451, 49.578497030126506),
       (16.275131078013363, 49.57845242790952),
       (16.276244123398357, 49.578129663875316),
       (16.27625680210342, 49.57816719391865),
       (16.276881079909494, 49.57807801951678),
       (16.27770233799579, 49.57770678413624),
       (16.27807466753168, 49.57759880510579),
       (16.27832335038518, 49.57742605875032),
       (16.2785934452353, 49.577303961757174),
       (16.27885303426424, 49.577218862819706),
       (16.27895809489197, 49.57713911908995),
       (16.279479359421792, 49.576903473564435),
       (16.279929723767165, 49.57644864749466),
       (16.279889786372472, 49.57643192574231),
       (16.279940466100832, 49.57639345688266),
       (16.280981175257022, 49.576614883600534),
       (16.283772724483413, 49.57656582620624),
       (16.285774802907802, 49.57619024219344),
       (16.287271143491655, 49.57561396086274),
       (16.287763278656563, 49.575526957771814),
       (16.288914485580136, 49.57491919846551),
       (16.289358442952615, 49.57524995118203),
       (16.291962684462302, 49.5762444110619),
       (16.293326341910625, 49.576485504440754),
       (16.29339620112297, 49.57669244957111),
       (16.293578948667182, 49.57696445328856),
       (16.295308430629735, 49.5782215842108),
       (16.29580787851964, 49.578408854781145),
       (16.298356885524687, 49.57875175085352),
       (16.298949966590037, 49.578707181159544),
       (16.300917357852715, 49.5782074193576),
       (16.301773609777655, 49.57780164049675),
       (16.302192969081304, 49.57742740831083),
       (16.303400414108857, 49.577219738492836),
       (16.30372156716775, 49.57708596610233),
       (16.30445035439406, 49.57641960454211),
       (16.30471453597227, 49.576763684775344),
       (16.304887512746753, 49.57734178049774),
       (16.305114935025344, 49.57770743001362),
       (16.30633140191055, 49.578643191175395),
       (16.306364884750355, 49.57868079846174),
       (16.306398847680036, 49.57869507206231),
       (16.306787103277767, 49.57899372454402),
       (16.307455920466705, 49.57926130839331),
       (16.30753851052074, 49.57917402747325),
       (16.308040506147, 49.57938498987665),
       (16.308134815217084, 49.579606812469244),
       (16.308549512954404, 49.580057186631755),
       (16.31189250550981, 49.58136555954167),
       (16.313332884331135, 49.581414619192415),
       (16.31584165009615, 49.58096948283159),
       (16.316162601931175, 49.58083122334818),
       (16.317700129792165, 49.57962121110209),
       (16.317931975674377, 49.579260001159945),
       (16.318128489493887, 49.57862216247138),
       (16.31825046967024, 49.57852505888694),
       (16.318477856779715, 49.57816385875064),
       (16.318785743086895, 49.577071972460985),
       (16.318741056389257, 49.5759482960766),
       (16.318405115419512, 49.57495239805758),
       (16.318222353646163, 49.57467594518698),
       (16.317819773503675, 49.57433808218208),
       (16.31787215601444, 49.57430110450285),
       (16.31895208253255, 49.57272066670656),
       (16.319041178232286, 49.5720919579318),
       (16.318567726230633, 49.57057115205526),
       (16.318153018795943, 49.57006722742494),
       (16.317191859273393, 49.56963598995787),
       (16.31721869520814, 49.56962218777022),
       (16.316942227584132, 49.56939482171704),
       (16.3159142690632, 49.56904435210835),
       (16.31609144705077, 49.56837483251801),
       (16.31604227546401, 49.56779065724178),
       (16.315934210959238, 49.56753562510283),
       (16.315965391549245, 49.56750859932478),
       (16.3160066973351, 49.56741949973316),
       (16.316532661821743, 49.56709489138999),
       (16.317032036440214, 49.56642157456011),
       (16.317444013117054, 49.565249176432445),
       (16.3174438951055, 49.56398720725468),
       (16.31723394289541, 49.56313985771012),
       (16.3170956112865, 49.562867901497356),
       (16.31571144838571, 49.561589305768436),
       (16.315167500472388, 49.56131283600447),
       (16.314641685503197, 49.56124154209579),
       (16.314494041251375, 49.561001352655),
       (16.312947364077083, 49.55975977739661),
       (16.312126877916388, 49.559403073864964),
       (16.308548496717712, 49.55907848798643),
       (16.307464971939154, 49.55930586301233),
       (16.305710326814527, 49.560029747043416),
       (16.305126295024802, 49.56043554805149),
       (16.30431615874004, 49.561256462651656),
       (16.304088617875987, 49.561617641437856),
       (16.303796302541564, 49.562726350098394),
       (16.303236628115112, 49.56313850891986),
       (16.303009170595953, 49.563499682479126),
       (16.30299814696084, 49.5635360904049),
       (16.301938655399344, 49.564227565940826),
       (16.30196069634923, 49.56424185377918),
       (16.30189863543038, 49.56428076871826),
       (16.30091336672279, 49.56453568573841),
       (16.300057270427416, 49.56494140994572),
       (16.298896740379707, 49.565785327174424),
       (16.298697724232362, 49.56601226983211),
       (16.298606716964276, 49.56576951518691),
       (16.29804643879201, 49.564930083984315),
       (16.297413185343963, 49.56430137214952),
       (16.296487035486738, 49.5636486324467),
       (16.29469444583306, 49.562743433002275),
       (16.292569284970256, 49.56219029701919),
       (16.291676349057074, 49.56213136518876),
       (16.290063821574417, 49.561821528611375),
       (16.288133112339146, 49.56186607319271),
       (16.28730846752254, 49.5620873644869),
       (16.28706267473101, 49.56208736947864),
       (16.28432222798225, 49.562760078556316),
       (16.283060362763724, 49.5634824077393),
       (16.282184763007926, 49.564185811367125),
       (16.282110257697504, 49.564272022402086),
       (16.28130826357945, 49.56480894979022),
       (16.280808817344507, 49.565437682359565),
       (16.280345946983456, 49.56668033484936),
       (16.280346188171677, 49.566783198979465),
       (16.280290534870414, 49.56676153778047),
       (16.279813062497045, 49.566741723004384),
       (16.27836738913003, 49.56644388432813),
       (16.276427806344156, 49.56644390977682),
       (16.274001175475018, 49.56695365915437),
       (16.273136118429353, 49.56735940686496),
       (16.27180736071771, 49.568401247402186),
       (16.271321338225604, 49.5690745439099),
       (16.27142822240101, 49.569107179307316),
       (16.27091004704858, 49.56959411723474),
       (16.270339966579787, 49.569478121382666),
       (16.270320465972254, 49.56947090589559),
       (16.27029358032436, 49.5694686828944),
       (16.268664481822523, 49.56913718633598),
       (16.265788494453723, 49.56913719446965),
       (16.264208306533664, 49.56934055429524),
       (16.26370894539383, 49.569474324934994),
       (16.263573611986306, 49.56960272405779),
       (16.263451882342398, 49.56962301526469),
       (16.26240730221672, 49.57020482986607),
       (16.262159976198266, 49.570267701646074),
       (16.261079605787845, 49.57094430235856),
       (16.260669185031034, 49.57117288233143),
       (16.260637206735964, 49.571221354173744),
       (16.259865896254603, 49.571704374845815),
       (16.259415468797386, 49.57237768836951),
       (16.25907101303529, 49.57345373445442),
       (16.259071005098136, 49.57349334565549),
       (16.258870936736198, 49.57366042480293),
       (16.258692470486633, 49.57407064547517),
       (16.25853576657436, 49.57480497503484),
       (16.258535599658725, 49.5763299634909),
       (16.258838346822852, 49.57734187422281),
       (16.259065816299564, 49.577707554544176),
       (16.25908255818756, 49.577718827911546),
       (16.25912817332941, 49.57788012873881),
       (16.258805079729463, 49.5782177560095),
       (16.258919181965748, 49.57826392430513),
       (16.258703287029224, 49.57842259932147),
       (16.258184878433187, 49.57864110459957),
       (16.256505230150967, 49.580052136599996),
       (16.25632240368468, 49.5804088228932),
       (16.256140716362925, 49.58113659223611),
       (16.255841860315247, 49.58207345226046),
       (16.255841794655836, 49.58360291337983),
       (16.25604027294733, 49.584427427648585),
       (16.256218537742274, 49.58478858928104),
       (16.256471441232385, 49.58496261992678),
       (16.25621859927433, 49.585194850872206),
       (16.256040202106014, 49.585556035942204),
       (16.25584146530205, 49.58638096986582),
       (16.255841465001176, 49.586407511783094),
       (16.255290435292565, 49.58686160643115),
       (16.25521233970638, 49.587128171622915),
       (16.25444883741794, 49.58778449298927),
       (16.254265938049592, 49.58860047892567)),)]},
   'context': {'cutoff_percentile': 35,
    'quality_band': 'cloudp',
    'score_percentile': 75},
   'data': {'from_node': 'renamelabels1'},
   'process': {'process_graph': {'runudf1': {'process_id': 'run_udf',
      'arguments': {'data': {'from_parameter': 'data'},
       'runtime': 'Python',
       'udf': 'from typing import Optional\n\nfrom openeo.udf import XarrayDataCube\nimport numpy as np\nfrom xarray import DataArray\n\ndef apply_datacube(cube: XarrayDataCube, context: dict) -> XarrayDataCube:\n    """\n    Filter the datacube based on the quality band and the quantile of the band values.\n    \n    This function assumes a DataCube with Dimension \'t\' as an input.\n    \n    Args:\n        cube (XarrayDataCube): datacube to apply the udf to.\n        context (dict): key-value arguments.\n    """\n\n    # Load kwargs from context\n    cutoff_percentile: Optional[float] = context.get("cutoff_percentile")\n    if not cutoff_percentile:\n        cutoff_percentile = 35\n    cutoff_percentile = cutoff_percentile / 100.\n\n    score_percentile: Optional[float] = context.get("score_percentile")\n    if not score_percentile:\n        score_percentile = 75.\n    score_percentile = score_percentile / 100.\n\n    quality_band: Optional[str] = context.get("quality_band")\n    if not quality_band:\n        quality_band = "cloudp"\n\n    array: DataArray = cube.get_array()\n    # Need to get band index, as bands are not dims here\n    index = np.where(array["bands"].values == quality_band)[0][0]\n    score: DataArray = array.isel(bands=index).quantile([score_percentile], dim=["x", "y"])\n    filtered: DataArray = array.sel(t=score.where(score / np.max(score) < cutoff_percentile, drop=True).t)\n    print(filtered.shape)\n    return XarrayDataCube(\n        array=filtered\n    )\n'},
      'result': True}}}}},
 'chunkpolygon2': {'process_id': 'chunk_polygon',
  'arguments': {'chunks': {'type': 'MultiPolygon',
    'coordinates': [(((16.254265938049592, 49.58860047892567),
       (16.254322774071163, 49.58957887300433),
       (16.25441642107371, 49.58985086572059),
       (16.256000197238357, 49.59143153655345),
       (16.257132884506554, 49.59197557028358),
       (16.259841262431102, 49.59249901145751),
       (16.26101410419378, 49.592454466766604),
       (16.26321719165413, 49.591941110939764),
       (16.26407334982892, 49.591535306060756),
       (16.26557534838399, 49.59018898956556),
       (16.265758124333388, 49.58982330147995),
       (16.26596125811, 49.58898927367213),
       (16.265961225028647, 49.588766693202736),
       (16.266360051472724, 49.58838581956765),
       (16.266758833384223, 49.5880491401048),
       (16.26678119924604, 49.58798362150429),
       (16.267061232360483, 49.58771618411336),
       (16.26717760973574, 49.58762996590407),
       (16.267303870910126, 49.58748445641037),
       (16.267576243026916, 49.587224328985826),
       (16.267604955216125, 49.58713746825679),
       (16.268014343776965, 49.58666565451496),
       (16.26819263371431, 49.58629998697619),
       (16.26838007435019, 49.585333095734455),
       (16.268290918537236, 49.58430307900289),
       (16.268062614939154, 49.58401797516814),
       (16.268080583820495, 49.58401647088828),
       (16.26799139069211, 49.583566089884854),
       (16.267288805676117, 49.58264747424579),
       (16.267675292056342, 49.58214625518044),
       (16.26811675809452, 49.581822144087525),
       (16.268540924060144, 49.58119823302146),
       (16.269077368876758, 49.58076209099866),
       (16.26952768707333, 49.57986581849109),
       (16.269732832777287, 49.579027735012524),
       (16.269732726715574, 49.578170924887864),
       (16.26982689183397, 49.57820220718116),
       (16.27179992330451, 49.578497030126506),
       (16.275131078013363, 49.57845242790952),
       (16.276244123398357, 49.578129663875316),
       (16.27625680210342, 49.57816719391865),
       (16.276881079909494, 49.57807801951678),
       (16.27770233799579, 49.57770678413624),
       (16.27807466753168, 49.57759880510579),
       (16.27832335038518, 49.57742605875032),
       (16.2785934452353, 49.577303961757174),
       (16.27885303426424, 49.577218862819706),
       (16.27895809489197, 49.57713911908995),
       (16.279479359421792, 49.576903473564435),
       (16.279929723767165, 49.57644864749466),
       (16.279889786372472, 49.57643192574231),
       (16.279940466100832, 49.57639345688266),
       (16.280981175257022, 49.576614883600534),
       (16.283772724483413, 49.57656582620624),
       (16.285774802907802, 49.57619024219344),
       (16.287271143491655, 49.57561396086274),
       (16.287763278656563, 49.575526957771814),
       (16.288914485580136, 49.57491919846551),
       (16.289358442952615, 49.57524995118203),
       (16.291962684462302, 49.5762444110619),
       (16.293326341910625, 49.576485504440754),
       (16.29339620112297, 49.57669244957111),
       (16.293578948667182, 49.57696445328856),
       (16.295308430629735, 49.5782215842108),
       (16.29580787851964, 49.578408854781145),
       (16.298356885524687, 49.57875175085352),
       (16.298949966590037, 49.578707181159544),
       (16.300917357852715, 49.5782074193576),
       (16.301773609777655, 49.57780164049675),
       (16.302192969081304, 49.57742740831083),
       (16.303400414108857, 49.577219738492836),
       (16.30372156716775, 49.57708596610233),
       (16.30445035439406, 49.57641960454211),
       (16.30471453597227, 49.576763684775344),
       (16.304887512746753, 49.57734178049774),
       (16.305114935025344, 49.57770743001362),
       (16.30633140191055, 49.578643191175395),
       (16.306364884750355, 49.57868079846174),
       (16.306398847680036, 49.57869507206231),
       (16.306787103277767, 49.57899372454402),
       (16.307455920466705, 49.57926130839331),
       (16.30753851052074, 49.57917402747325),
       (16.308040506147, 49.57938498987665),
       (16.308134815217084, 49.579606812469244),
       (16.308549512954404, 49.580057186631755),
       (16.31189250550981, 49.58136555954167),
       (16.313332884331135, 49.581414619192415),
       (16.31584165009615, 49.58096948283159),
       (16.316162601931175, 49.58083122334818),
       (16.317700129792165, 49.57962121110209),
       (16.317931975674377, 49.579260001159945),
       (16.318128489493887, 49.57862216247138),
       (16.31825046967024, 49.57852505888694),
       (16.318477856779715, 49.57816385875064),
       (16.318785743086895, 49.577071972460985),
       (16.318741056389257, 49.5759482960766),
       (16.318405115419512, 49.57495239805758),
       (16.318222353646163, 49.57467594518698),
       (16.317819773503675, 49.57433808218208),
       (16.31787215601444, 49.57430110450285),
       (16.31895208253255, 49.57272066670656),
       (16.319041178232286, 49.5720919579318),
       (16.318567726230633, 49.57057115205526),
       (16.318153018795943, 49.57006722742494),
       (16.317191859273393, 49.56963598995787),
       (16.31721869520814, 49.56962218777022),
       (16.316942227584132, 49.56939482171704),
       (16.3159142690632, 49.56904435210835),
       (16.31609144705077, 49.56837483251801),
       (16.31604227546401, 49.56779065724178),
       (16.315934210959238, 49.56753562510283),
       (16.315965391549245, 49.56750859932478),
       (16.3160066973351, 49.56741949973316),
       (16.316532661821743, 49.56709489138999),
       (16.317032036440214, 49.56642157456011),
       (16.317444013117054, 49.565249176432445),
       (16.3174438951055, 49.56398720725468),
       (16.31723394289541, 49.56313985771012),
       (16.3170956112865, 49.562867901497356),
       (16.31571144838571, 49.561589305768436),
       (16.315167500472388, 49.56131283600447),
       (16.314641685503197, 49.56124154209579),
       (16.314494041251375, 49.561001352655),
       (16.312947364077083, 49.55975977739661),
       (16.312126877916388, 49.559403073864964),
       (16.308548496717712, 49.55907848798643),
       (16.307464971939154, 49.55930586301233),
       (16.305710326814527, 49.560029747043416),
       (16.305126295024802, 49.56043554805149),
       (16.30431615874004, 49.561256462651656),
       (16.304088617875987, 49.561617641437856),
       (16.303796302541564, 49.562726350098394),
       (16.303236628115112, 49.56313850891986),
       (16.303009170595953, 49.563499682479126),
       (16.30299814696084, 49.5635360904049),
       (16.301938655399344, 49.564227565940826),
       (16.30196069634923, 49.56424185377918),
       (16.30189863543038, 49.56428076871826),
       (16.30091336672279, 49.56453568573841),
       (16.300057270427416, 49.56494140994572),
       (16.298896740379707, 49.565785327174424),
       (16.298697724232362, 49.56601226983211),
       (16.298606716964276, 49.56576951518691),
       (16.29804643879201, 49.564930083984315),
       (16.297413185343963, 49.56430137214952),
       (16.296487035486738, 49.5636486324467),
       (16.29469444583306, 49.562743433002275),
       (16.292569284970256, 49.56219029701919),
       (16.291676349057074, 49.56213136518876),
       (16.290063821574417, 49.561821528611375),
       (16.288133112339146, 49.56186607319271),
       (16.28730846752254, 49.5620873644869),
       (16.28706267473101, 49.56208736947864),
       (16.28432222798225, 49.562760078556316),
       (16.283060362763724, 49.5634824077393),
       (16.282184763007926, 49.564185811367125),
       (16.282110257697504, 49.564272022402086),
       (16.28130826357945, 49.56480894979022),
       (16.280808817344507, 49.565437682359565),
       (16.280345946983456, 49.56668033484936),
       (16.280346188171677, 49.566783198979465),
       (16.280290534870414, 49.56676153778047),
       (16.279813062497045, 49.566741723004384),
       (16.27836738913003, 49.56644388432813),
       (16.276427806344156, 49.56644390977682),
       (16.274001175475018, 49.56695365915437),
       (16.273136118429353, 49.56735940686496),
       (16.27180736071771, 49.568401247402186),
       (16.271321338225604, 49.5690745439099),
       (16.27142822240101, 49.569107179307316),
       (16.27091004704858, 49.56959411723474),
       (16.270339966579787, 49.569478121382666),
       (16.270320465972254, 49.56947090589559),
       (16.27029358032436, 49.5694686828944),
       (16.268664481822523, 49.56913718633598),
       (16.265788494453723, 49.56913719446965),
       (16.264208306533664, 49.56934055429524),
       (16.26370894539383, 49.569474324934994),
       (16.263573611986306, 49.56960272405779),
       (16.263451882342398, 49.56962301526469),
       (16.26240730221672, 49.57020482986607),
       (16.262159976198266, 49.570267701646074),
       (16.261079605787845, 49.57094430235856),
       (16.260669185031034, 49.57117288233143),
       (16.260637206735964, 49.571221354173744),
       (16.259865896254603, 49.571704374845815),
       (16.259415468797386, 49.57237768836951),
       (16.25907101303529, 49.57345373445442),
       (16.259071005098136, 49.57349334565549),
       (16.258870936736198, 49.57366042480293),
       (16.258692470486633, 49.57407064547517),
       (16.25853576657436, 49.57480497503484),
       (16.258535599658725, 49.5763299634909),
       (16.258838346822852, 49.57734187422281),
       (16.259065816299564, 49.577707554544176),
       (16.25908255818756, 49.577718827911546),
       (16.25912817332941, 49.57788012873881),
       (16.258805079729463, 49.5782177560095),
       (16.258919181965748, 49.57826392430513),
       (16.258703287029224, 49.57842259932147),
       (16.258184878433187, 49.57864110459957),
       (16.256505230150967, 49.580052136599996),
       (16.25632240368468, 49.5804088228932),
       (16.256140716362925, 49.58113659223611),
       (16.255841860315247, 49.58207345226046),
       (16.255841794655836, 49.58360291337983),
       (16.25604027294733, 49.584427427648585),
       (16.256218537742274, 49.58478858928104),
       (16.256471441232385, 49.58496261992678),
       (16.25621859927433, 49.585194850872206),
       (16.256040202106014, 49.585556035942204),
       (16.25584146530205, 49.58638096986582),
       (16.255841465001176, 49.586407511783094),
       (16.255290435292565, 49.58686160643115),
       (16.25521233970638, 49.587128171622915),
       (16.25444883741794, 49.58778449298927),
       (16.254265938049592, 49.58860047892567)),)]},
   'context': {'minimum_filled_fraction': 'from typing import Optional\n\nfrom openeo.udf import XarrayDataCube\nimport numpy as np\nfrom xarray import DataArray\n\ndef apply_datacube(cube: XarrayDataCube, context: dict) -> XarrayDataCube:\n    """\n    Filter the datacube based on the quality band and the quantile of the band values.\n    \n    This function assumes a DataCube with Dimension \'t\' as an input.\n    \n    Args:\n        cube (XarrayDataCube): datacube to apply the udf to.\n        context (dict): key-value arguments.\n    """\n\n    # Load kwargs from context\n    cutoff_percentile: Optional[float] = context.get("cutoff_percentile")\n    if not cutoff_percentile:\n        cutoff_percentile = 35\n    cutoff_percentile = cutoff_percentile / 100.\n\n    score_percentile: Optional[float] = context.get("score_percentile")\n    if not score_percentile:\n        score_percentile = 75.\n    score_percentile = score_percentile / 100.\n\n    quality_band: Optional[str] = context.get("quality_band")\n    if not quality_band:\n        quality_band = "cloudp"\n\n    array: DataArray = cube.get_array()\n    # Need to get band index, as bands are not dims here\n    index = np.where(array["bands"].values == quality_band)[0][0]\n    score: DataArray = array.isel(bands=index).quantile([score_percentile], dim=["x", "y"])\n    filtered: DataArray = array.sel(t=score.where(score / np.max(score) < cutoff_percentile, drop=True).t)\n    print(filtered.shape)\n    return XarrayDataCube(\n        array=filtered\n    )\n',
    'quality_check_bands': ['green', 'nir', 'swir']},
   'data': {'from_node': 'chunkpolygon1'},
   'process': {'process_graph': {'runudf2': {'process_id': 'run_udf',
      'arguments': {'data': {'from_parameter': 'data'},
       'runtime': 'Python',
       'udf': 'from typing import List, Optional\n\nfrom openeo.udf import XarrayDataCube\nimport numpy as np\nimport xarray as xr\n\ndef apply_datacube(cube: XarrayDataCube, context: dict) -> XarrayDataCube:\n    """\n    Preprocess reservoir polygons to fill the polygon with nodata values.\n    \n    This function assumes a DataCube with Dimension \'t\' as an input.\n    \n    Args:\n        cube (XarrayDataCube): datacube to apply the udf to.\n        context (dict): key-value arguments.\n    """\n\n    # Load kwargs from context\n    minimum_filled_fraction: Optional[float] = context.get("minimum_filled_fraction")\n    if not minimum_filled_fraction:\n        minimum_filled_fraction = 0.35\n    \n    quality_check_bands: Optional[List[str]] = context.get("quality_check_bands")\n    if not quality_check_bands:\n        quality_check_bands = ["green", "nir", "swir"]\n\n    masked_value: Optional[int] = context.get("masked_value")\n    if not masked_value:\n        masked_value = -999999\n    \n    missing_value: Optional[int] = context.get("missing_value")\n    if not missing_value:\n        missing_value = -888888\n\n    array: xr.DataArray = cube.get_array()\n    # Need to get band index, as bands are not dims here\n    indexes_used: List[int] = [bandname in quality_check_bands for bandname in array["bands"].values]\n    check = array.isel(bands=indexes_used)\n    masked = xr.where(check.sel(bands=quality_check_bands[0], drop=True) == masked_value, 1, 0)\n    fraction_masked = masked.mean(dim=["x", "y"])\n    print(fraction_masked)\n    mean = 1 - np.isnan(check).mean(dim=["x", "y"])\n    # Because we need all bands, use the most scarcely populated band as a filter criterion\n    mean = mean.min(dim="bands")\n    # Filter where the are too few observations.\n    filtered: xr.DataArray = array.sel(t=mean.where(mean / (1 - fraction_masked) > minimum_filled_fraction, drop=True).t)\n    # replace missing observations using missing value\n    return XarrayDataCube(\n        array=filtered\n    )'},
      'result': True}}}}},
 'reducedimension1': {'process_id': 'reduce_dimension',
  'arguments': {'data': {'from_node': 'chunkpolygon2'},
   'dimension': 'bands',
   'reducer': {'process_graph': {'arrayelement1': {'process_id': 'array_element',
      'arguments': {'data': {'from_parameter': 'data'}, 'index': 0}},
     'arrayelement2': {'process_id': 'array_element',
      'arguments': {'data': {'from_parameter': 'data'}, 'index': 2}},
     'subtract1': {'process_id': 'subtract',
      'arguments': {'x': {'from_node': 'arrayelement1'},
       'y': {'from_node': 'arrayelement2'}}},
     'add1': {'process_id': 'add',
      'arguments': {'x': {'from_node': 'arrayelement1'},
       'y': {'from_node': 'arrayelement2'}}},
     'divide1': {'process_id': 'divide',
      'arguments': {'x': {'from_node': 'subtract1'},
       'y': {'from_node': 'add1'}},
      'result': True}}}}},
 'adddimension1': {'process_id': 'add_dimension',
  'arguments': {'data': {'from_node': 'reducedimension1'},
   'label': 'MNDWI',
   'name': 'bands',
   'type': 'bands'}},
 'adddimension2': {'process_id': 'add_dimension',
  'arguments': {'data': {'from_node': 'adddimension1'},
   'label': 'MNDWI',
   'name': 'bands',
   'type': 'bands'}},
 'loadcollection2': {'process_id': 'load_collection',
  'arguments': {'bands': ['occurrence'],
   'id': 'GLOBAL_SURFACE_WATER',
   'spatial_extent': {'west': 589947.873967385,
    'east': 596088.3303869166,
    'south': 5489655.1321837725,
    'north': 5494855.214093619,
    'crs': 'EPSG:32633'},
   'temporal_extent': None}},
 'filtertemporal1': {'process_id': 'filter_temporal',
  'arguments': {'data': {'from_node': 'loadcollection2'},
   'extent': ['2019-12-31', '2020-01-02']}},
 'dropdimension1': {'process_id': 'drop_dimension',
  'arguments': {'data': {'from_node': 'filtertemporal1'}, 'name': 't'}},
 'resamplecubespatial1': {'process_id': 'resample_cube_spatial',
  'arguments': {'data': {'from_node': 'dropdimension1'},
   'method': 'nearest',
   'target': {'from_node': 'chunkpolygon2'}}},
 'dropdimension2': {'process_id': 'drop_dimension',
  'arguments': {'data': {'from_node': 'resamplecubespatial1'},
   'name': 'bands'}},
 'apply1': {'process_id': 'apply',
  'arguments': {'data': {'from_node': 'dropdimension2'},
   'process': {'process_graph': {'multiply1': {'process_id': 'multiply',
      'arguments': {'x': {'from_parameter': 'x'}, 'y': 1.0},
      'result': True}}}}},
 'adddimension3': {'process_id': 'add_dimension',
  'arguments': {'data': {'from_node': 'apply1'},
   'label': 'wo',
   'name': 'bands',
   'type': 'bands'}},
 'mergecubes1': {'process_id': 'merge_cubes',
  'arguments': {'cube1': {'from_node': 'adddimension2'},
   'cube2': {'from_node': 'adddimension3'}}},
 'filterbands1': {'process_id': 'filter_bands',
  'arguments': {'bands': ['MNDWI'], 'data': {'from_node': 'mergecubes1'}}},
 'apply2': {'process_id': 'apply',
  'arguments': {'data': {'from_node': 'filterbands1'},
   'process': {'process_graph': {'int1': {'process_id': 'int',
      'arguments': {'x': 1},
      'result': True}}}}},
 'renamelabels2': {'process_id': 'rename_labels',
  'arguments': {'data': {'from_node': 'apply2'},
   'dimension': 'bands',
   'source': ['MNDWI'],
   'target': ['water']},
  'result': True}}

I’m not completely sure I understand the problem.
What error do you get? Does it happen client side or back-end side?

you get an error when merging (as you comment here) or after merging (as your snippet suggests here)?

Hi @stefaan.lippens,
I am not getting an error, but after downloading the datacube, the bands are not renamed. So in this case the resulting datacube from the snippet still has 1 band called "MNDWI"

When I try to reproduce this error from the connection.load_collection, this snippet works and renames the band dimension.

So I am lost as to what may cause this. I have restarted the notebook to make sure this is not a fluke (had some weird errors when reading netcdfs in a jupyter notebook session). and tried a few other things.

I’m still investigating, just some quick notes:

I can reproduce the issue without the chunk_polygon and UDFs

I also notice you re-add the bands dimension twice on the SENTINEL2_L1C_SENTINELHUB data. Any reason for that?

1 Like

You found my bug there I believe. Apologies @stefaan.lippens, I had missed this when checking before posting the question.

I’m not sure where there yet. I still can reproduce the problem after eliminating one of those add_dimension’s

hmm I’m confused now, I did some more experimentation today and I’m seeing different results compared to yesterday.

Removing the duplicate add_dimension indeed seems to fix the problem now. I’d swear I could still get the naming issue without a duplicate add_dimension node yesterday.
We did a production deploy yesterday as well, so maybe that has something to do with it.

@jaapel can you confirm that the bug disappears for you as well when the duplicate add_dimension is removed?

1 Like

FYI: I reported minimal reproduction use case at `rename_labels` doesn't work with duplicate `add_dimension` · Issue #198 · Open-EO/openeo-geopyspark-driver · GitHub

1 Like

Hi Stefaan! I can confirm that this problem is fixed for me now! On to the next udf :wink: