This repository contains configuration and driver code for running an end-to-end machine learning pipeline for weather prediction. The pipeline is orchestrated with make targets and uwtools-based drivers, and it provisions a self-contained set of conda environments to support each step of the workflow. A typical run follows these steps:
- Environment setup: Creates the runtime environments used by each stage of the pipeline.
- Prepare training and inference data: Provisions required static assets (e.g., grids and meshes) and produces Zarr-formatted datasets via
ufs2arco. - Train an AI model: Trains an Anemoi model using the provisioned datasets, producing checkpoints for inference.
- Generate a forecast: Runs inference from training checkpoints using
anemoi-inferenceto produce forecast output. - Prepare output for verification: Postprocesses forecast output into the formats and directory structure expected by
wxvx. - Verify model performance: Runs
wxvxverification against gridded analysis and/or observations, producing MET statistics and plots.
To learn about EAGLE and how to use the provided workflows, please see our documentation.
- If you encounter a problem using EAGLE that appears to be a bug, please open an issue with us.
- For free-form sharing of ideas, questions, tips and tricks, etc., please start or join a discussion.
- To contribute to the codebase, please see our docs.
ufs2arco: Tim Smith (NOAA Physical Sciences Laboratory)
Anemoi: European Centre for Medium-Range Weather Forecasts
- anemoi-core github
- anemoi-inference github
- Documentation: anemoi-models, anemoi-graphs, anemoi-training, anemoi-inference
wxvx: Paul Madden (NOAA Global Systems Laboratory/Cooperative Institute for Research In Environmental Sciences)
eagle-tools: Tim Smith (NOAA Physical Sciences Laboratory)