download_glofas_discharge fails with "ValueError: did not find a match in any of xarray's currently installed IO backends"

[luseverin]

I am trying to download discharge forecasts using download_glofas_discharge directly.

from climada_petals.hazard.rf_glofas.transform_ops import download_glofas_discharge

countries : ["KEN"],
forecast_date = "2024-03-20"
lead_time_days = 5
preproc = lambda x: x.max(dim="step").mean(dim="number")
leadtime_hour = list(
            map(str, (np.arange(1, query_forecast["lead_time_days"] + 1, dtype=np.int_) * 24).flat)
)
forecast = download_glofas_discharge(
    product="forecast",
    date_from=,
    date_to=None,
    countries=countries,
    preprocess=preproc,
    leadtime_hour=leadtime_hour,
    system_version="version_3_1",
)

However, it fails with a ValueError:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[14], [line 8](vscode-notebook-cell:?execution_count=14&line=8)
      [2](vscode-notebook-cell:?execution_count=14&line=2) from climada_petals.hazard.rf_glofas.transform_ops import download_glofas_discharge
      [5](vscode-notebook-cell:?execution_count=14&line=5) leadtime_hour = list(
      [6](vscode-notebook-cell:?execution_count=14&line=6)             map(str, (np.arange(1, query_forecast["lead_time_days"] + 1, dtype=np.int_) * 24).flat)
      [7](vscode-notebook-cell:?execution_count=14&line=7) )
----> [8](vscode-notebook-cell:?execution_count=14&line=8) forecast = download_glofas_discharge(
      [9](vscode-notebook-cell:?execution_count=14&line=9)     product="forecast",
     [10](vscode-notebook-cell:?execution_count=14&line=10)     date_from=pd.Timestamp(query_forecast["forecast_date"].min()).date().isoformat(),
     [11](vscode-notebook-cell:?execution_count=14&line=11)     date_to=pd.Timestamp(query_forecast["forecast_date"].max()).date().isoformat(),
     [12](vscode-notebook-cell:?execution_count=14&line=12)     countries=query_forecast["countries"],
     [13](vscode-notebook-cell:?execution_count=14&line=13)     preprocess=query_forecast["preproc"],
     [14](vscode-notebook-cell:?execution_count=14&line=14)     leadtime_hour=leadtime_hour,
     [15](vscode-notebook-cell:?execution_count=14&line=15) )

File ~/Documents/PhD/workspace/climada_petals/climada_petals/hazard/rf_glofas/transform_ops.py:335, in download_glofas_discharge(product, date_from, date_to, num_proc, download_path, countries, preprocess, open_mfdataset_kw, **request_kwargs)
    [332](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/Documents/PhD/workspace/climada_petals/climada_petals/hazard/rf_glofas/transform_ops.py:332)     open_kwargs.update(open_mfdataset_kw)
    [334](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/Documents/PhD/workspace/climada_petals/climada_petals/hazard/rf_glofas/transform_ops.py:334) # Squeeze all dimensions except time
--> [335](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/Documents/PhD/workspace/climada_petals/climada_petals/hazard/rf_glofas/transform_ops.py:335) arr = xr.open_mfdataset(files, **open_kwargs)["dis24"]
    [336](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/Documents/PhD/workspace/climada_petals/climada_petals/hazard/rf_glofas/transform_ops.py:336) dims = {dim for dim, size in arr.sizes.items() if size == 1} - {"time"}
    [337](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/Documents/PhD/workspace/climada_petals/climada_petals/hazard/rf_glofas/transform_ops.py:337) return arr.squeeze(dim=dims)

File ~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/api.py:1077, in open_mfdataset(paths, chunks, concat_dim, compat, preprocess, engine, data_vars, coords, combine, parallel, join, attrs_file, combine_attrs, **kwargs)
   [1074](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/api.py:1074)     open_ = open_dataset
   [1075](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/api.py:1075)     getattr_ = getattr
-> [1077](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/api.py:1077) datasets = [open_(p, **open_kwargs) for p in paths]
   [1078](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/api.py:1078) closers = [getattr_(ds, "_close") for ds in datasets]
   [1079](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/api.py:1079) if preprocess is not None:

File ~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/api.py:1077, in <listcomp>(.0)
   [1074](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/api.py:1074)     open_ = open_dataset
   [1075](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/api.py:1075)     getattr_ = getattr
-> [1077](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/api.py:1077) datasets = [open_(p, **open_kwargs) for p in paths]
   [1078](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/api.py:1078) closers = [getattr_(ds, "_close") for ds in datasets]
   [1079](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/api.py:1079) if preprocess is not None:

File ~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/api.py:569, in open_dataset(filename_or_obj, engine, chunks, cache, decode_cf, mask_and_scale, decode_times, decode_timedelta, use_cftime, concat_characters, decode_coords, drop_variables, inline_array, chunked_array_type, from_array_kwargs, backend_kwargs, **kwargs)
    [566](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/api.py:566)     kwargs.update(backend_kwargs)
    [568](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/api.py:568) if engine is None:
--> [569](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/api.py:569)     engine = plugins.guess_engine(filename_or_obj)
    [571](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/api.py:571) if from_array_kwargs is None:
    [572](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/api.py:572)     from_array_kwargs = {}

File ~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/plugins.py:197, in guess_engine(store_spec)
    [189](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/plugins.py:189) else:
    [190](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/plugins.py:190)     error_msg = (
    [191](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/plugins.py:191)         "found the following matches with the input file in xarray's IO "
    [192](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/plugins.py:192)         f"backends: {compatible_engines}. But their dependencies may not be installed, see:\n"
    [193](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/plugins.py:193)         "https://docs.xarray.dev/en/stable/user-guide/io.html \n"
    [194](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/plugins.py:194)         "https://docs.xarray.dev/en/stable/getting-started-guide/installing.html"
    [195](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/plugins.py:195)     )
--> [197](https://file+.vscode-resource.vscode-cdn.net/Users/lseverino/Documents/PhD/workspace/IBF_ICPAC/IBF_CI_roads_health/~/miniforge3/envs/climada_nw_osmnx/lib/python3.9/site-packages/xarray/backends/plugins.py:197) raise ValueError(error_msg)

ValueError: did not find a match in any of xarray's currently installed IO backends ['netcdf4', 'scipy', 'cfgrib', 'rasterio']. Consider explicitly selecting one of the installed engines via the ``engine`` parameter, or installing additional IO dependencies, see:
https://docs.xarray.dev/en/stable/getting-started-guide/installing.html
https://docs.xarray.dev/en/stable/user-guide/io.html

Proposed fix: it seems to be working when specifying engine="cfgrib" in download_glofas_discharge:
arr = xr.open_mfdataset(files, **open_kwargs, engine="cfgrib")["dis24"]

I am not sure exactly why this issue arised in the first place, maybe also related to GloFAS upgrade to v4?