Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 31 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
# eobs_environment
[![Build environment](https://github.com/ECA-D/eobs_environment/actions/workflows/build_environment.yml/badge.svg)](https://github.com/ECA-D/eobs_environment/actions/workflows/build_environment.yml)


Software environment for running E-OBS code

*This repository is in a testing phase*

## How to use
This repository is the testbed for testing the installation procedure for `eobs` code.
Expand All @@ -13,6 +11,10 @@ The goal of the repository is to assure one can easily install the required depe

When working with the latest version of `eobs` and `E-OBS_batch` (both from this organization), one should be able to install everything (on a linux host).

There are two use-cases:
- A local installation (see below).
- Installation to a so-called `tykky` environment on ECMWF.

### 1. Install `miniforge`

Run the following commands, and follow the install guide of miniforge:
Expand Down Expand Up @@ -40,7 +42,33 @@ Which first installs the `conda`-packages in `conda_env.txt`. Some packages are
#### 4. Install `eobs` from source
This step is currently not available

### 5. Test for a succesfull installation
### 5. Test for a successful installation
```bash
bash run_minimal_test.sh
```

## `tykky` environment
On ECMWF, we use a so-called `tykky` environment as recommended by the [user documentation](https://confluence.ecmwf.int/display/UDOC/HPC2020%3A+Python+support).
Therefore, on the ECMWF HPC, one may instead follow these steps to create a `tykky`:
```bash
# Start an interactive session with more memory for building the environment
ecinteractive -m 20GB -c 1
# Load python for running the tykky wrapper script, called create_tykky_env.py
module load python3
export TYKKY_PATH=$HPCPERM/tykky
# Build the image, the name after "-n" will be the new environment name.
# See python3 create_tykky_env.py --help for more options.
# This step usually takes about 30 minutes
time python3 create_tykky_env.py -n <NAME>

# To activate the environment, run:
module load tykky
tykky activate <NAME>

# NB! At the time of writing (2025-08-11), there is an issue with tykky on ECMWF, which causes race conditions with sed.
# Therefore change the execution level:
chmod -x $(which sed)

# Check that the software was succesfully installed:
bash run_miniml_test.sh
```
8 changes: 5 additions & 3 deletions create_tykky_env.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
"""
Only works on ATOS, to run:
Only works on ATOS/ECMWF, to run:
module load python3
export TYKKY_PATH=$HPCPERM/tykky
time python3 create_tykky_env.py -n eobs_v25.07
time python3 create_tykky_env.py -n eobs_v25.08

Installing with tykky on atos is a bit tricky, since conda-containerize changes paths during the installation, so we need to have absolute paths.
Installing with tykky on atos is a bit tricky, since conda-containerize changes
paths during the installation, so we need to have absolute paths.
For that reason, we convert the conda-requirements to a yaml file.
"""

Expand All @@ -20,6 +21,7 @@
module load tykky
conda-containerize new --mamba --prefix {tykky_path}/{env_name} {yaml_file}
conda-containerize update {tykky_path}/{env_name} --post-install {abs_path}/install_step1_R.sh
conda-containerize update {tykky_path}/{env_name} --post-install {abs_path}/install_step2_pip.sh
"""
yaml_template = """
channels:
Expand Down
Loading