From 826324c133dcbdaeb4e46da6503f61500170522f Mon Sep 17 00:00:00 2001
From: Sophie Arana <arana.sophie@gmail.com>
Date: Thu, 2 Oct 2025 13:43:01 +0100
Subject: [PATCH 1/8] docs: add Jupyter Book config and source files for
 initial scaffold

---
 .gitignore                                    |   1 +
 docs/_config.yml                              |  25 ++++
 docs/_toc.yml                                 |  15 ++
 docs/adding-new-models.md                     |  53 ++++++++
 docs/api/data_loaders.rst                     |  88 ++++++++++++
 docs/api/evaluation.rst                       |  17 +++
 docs/api/index.md                             |  43 ++++++
 docs/api/models.rst                           |  17 +++
 docs/api/training.rst                         |  17 +++
 docs/api/types.rst                            |  12 ++
 docs/api/visualisations.rst                   |  15 ++
 docs/cli.md                                   | 128 ++++++++++++++++++
 docs/intro.md                                 |  18 +++
 docs/logo.png                                 | Bin 0 -> 9854 bytes
 .../pipeline-encode-process-decode.png        | Bin
 docs/{assets => }/pipeline-standalone.png     | Bin
 docs/quickstart.md                            |  83 ++++++++++++
 docs/requirements.txt                         |   3 +
 18 files changed, 535 insertions(+)
 create mode 100644 docs/_config.yml
 create mode 100644 docs/_toc.yml
 create mode 100644 docs/adding-new-models.md
 create mode 100644 docs/api/data_loaders.rst
 create mode 100644 docs/api/evaluation.rst
 create mode 100644 docs/api/index.md
 create mode 100644 docs/api/models.rst
 create mode 100644 docs/api/training.rst
 create mode 100644 docs/api/types.rst
 create mode 100644 docs/api/visualisations.rst
 create mode 100644 docs/cli.md
 create mode 100644 docs/intro.md
 create mode 100644 docs/logo.png
 rename docs/{assets => }/pipeline-encode-process-decode.png (100%)
 rename docs/{assets => }/pipeline-standalone.png (100%)
 create mode 100644 docs/quickstart.md
 create mode 100644 docs/requirements.txt

diff --git a/.gitignore b/.gitignore
index c668d847..a0ff7939 100644
--- a/.gitignore
+++ b/.gitignore
@@ -11,3 +11,4 @@ outputs
 wandb
 notebooks/data
 notebooks/*.nc
+docs/_build/
\ No newline at end of file
diff --git a/docs/_config.yml b/docs/_config.yml
new file mode 100644
index 00000000..2791b212
--- /dev/null
+++ b/docs/_config.yml
@@ -0,0 +1,25 @@
+# Book settings
+title: "Ice Station Zebra API Documentation"
+author: The Alan Turing Institute
+
+sphinx:
+  extra_extensions:
+    - sphinx.ext.autodoc
+    - sphinx.ext.autosummary
+  config:
+    autosummary_generate: true
+    autodoc_typehints: "description"   # optional
+    autodoc_member_order: "bysource"   # optional
+    autoclass_content: "init"          # pull class doc from __init__ docstring
+
+# Table of contents
+toc:
+  - file: intro
+  - file: cli
+  - file: api/index
+  - file: api/data_loaders
+  - file: api/models
+  - file: api/training
+  - file: api/evaluation
+  - file: api/types
+  - file: api/visualisations
\ No newline at end of file
diff --git a/docs/_toc.yml b/docs/_toc.yml
new file mode 100644
index 00000000..24354482
--- /dev/null
+++ b/docs/_toc.yml
@@ -0,0 +1,15 @@
+# Table of contents
+format: jb-book
+root: intro
+chapters:
+- file: quickstart
+- file: adding-new-models
+- file: cli
+- file: api/index
+  sections:
+  - file: api/data_loaders
+  - file: api/models
+  - file: api/training
+  - file: api/evaluation
+  - file: api/types
+  - file: api/visualisations
diff --git a/docs/adding-new-models.md b/docs/adding-new-models.md
new file mode 100644
index 00000000..1ebd8cd0
--- /dev/null
+++ b/docs/adding-new-models.md
@@ -0,0 +1,53 @@
+# Adding new models
+
+## Background
+
+An `ice-station-zebra` model needs to be able to run over multiple different datasets with different dimensions.
+These are structured in `NTCHW` format, where:
+- `N` is the batch size,
+- `T` is the number of history (forecast) steps for inputs (outputs)
+- `C` is the number of channels or variables
+- `H` is a height dimension
+- `W` is a width dimension
+
+`N` and `T` will be the same for all inputs, but `C`, `H` and `W` might vary.
+
+Taking as an example, a batch size (`N=2`), 3 history steps and 4 forecast steps, we will have `k` inputs of shape `(2, 3, C_k, H_k, W_k)` and one output of shape `(2, 4, C_out, H_out, W_out)`.
+
+## Standalone models
+
+A standalone model will need to accept a `dict[str, TensorNTCHW]` which maps dataset names to an `NTCHW` Tensor of values.
+The model might want to use one or more of these for training, and will need to produce an output with shape `N, T, C_out, H_out, W_out`.
+
+As can be seen in the example below, a separate instance of the model is likely to be needed for each output to be predicted.
+
+![Standalone Pipeline](pipeline-standalone.png)
+
+Pros:
+- all input variables are available without transformation
+
+Cons:
+- hard to add new inputs
+- hard to add new outputs
+
+## Processor models
+
+A processor model is part of a larger encode-process-decode step.
+Start by defining a latent space as `(C_latent, H_latent, W_latent)` - in the example below, this has been set to `(10, 64, 64)`.
+The encode-process-decode model automatically creates one encoder for each input and one decoder for each output.
+The dataset-specific encoder takes the input data and converts it to shape `(N, C_latent, H_latent, W_latent)`, compressing the time and channels dimensions.
+The `k` encoded datasets can then be combined in latent space to give a single dataset of shape `(N, k * C_latent, H_latent, W_latent)`.
+
+This is then passed to the processor, which must accept input of shape `(N, k * C_latent, H_latent, W_latent)` and produce output of the same shape.
+
+This output is then passed to one or more output-specific decoders which take input of shape `(N, k * C_latent, H_latent, W_latent)` and produce output of shape `(N, T, C_out, H_out, W_out)`, regenerating the time dimension.
+
+![Encode-Process-Decode Pipeline](pipeline-encode-process-decode.png)
+
+Pros:
+- easy to add new inputs
+- easy to add new outputs
+
+Cons:
+- input variables have been transformed into latent space
+- time-step information has been compressed into the latent space
diff --git a/docs/api/data_loaders.rst b/docs/api/data_loaders.rst
new file mode 100644
index 00000000..46f44d6c
--- /dev/null
+++ b/docs/api/data_loaders.rst
@@ -0,0 +1,88 @@
+Data Loaders
+============
+
+The data loaders module provides classes for loading and managing datasets in the Ice Station Zebra framework.
+
+Classes
+-------
+
+CombinedDataset
+~~~~~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.data_loaders.combined_dataset.CombinedDataset
+      :members:
+      :undoc-members:
+      :show-inheritance:
+
+ZebraDataModule
+~~~~~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.data_loaders.zebra_data_module.ZebraDataModule
+      :members:
+      :undoc-members:
+      :show-inheritance:
+
+ZebraDataset
+~~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.data_loaders.zebra_dataset.ZebraDataset
+      :members:
+      :undoc-members:
+      :show-inheritance:
+
+Usage Examples
+--------------
+
+Loading a Dataset
+~~~~~~~~~~~~~~~~~
+
+.. code-block:: python
+
+   from ice_station_zebra.data_loaders.zebra_dataset import ZebraDataset
+
+   # Load a dataset
+   dataset = ZebraDataset("path/to/dataset.zarr")
+
+   # Access data
+   data = dataset[0]  # Get first sample
+
+
+Combining Multiple Datasets
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. code-block:: python
+
+   from ice_station_zebra.data_loaders.combined_dataset import CombinedDataset
+
+   # Create a combined dataset from multiple ZebraDatasets
+   combined = CombinedDataset(
+       datasets=[dataset1, dataset2, dataset3],
+       target="target_dataset_name",
+       n_forecast_steps=4,
+       n_history_steps=3
+   )
+
+   # Access combined data
+   sample = combined[0]  # Returns dict with input and target data
+
+Using ZebraDataModule
+~~~~~~~~~~~~~~~~~~~~~
+
+.. code-block:: python
+
+   from ice_station_zebra.data_loaders.zebra_data_module import ZebraDataModule
+   from omegaconf import DictConfig
+
+   # Initialize with configuration
+   data_module = ZebraDataModule(config)
+
+   # Get data loaders
+   train_loader = data_module.train_dataloader()
+   val_loader = data_module.val_dataloader()
+   test_loader = data_module.test_dataloader()
\ No newline at end of file
diff --git a/docs/api/evaluation.rst b/docs/api/evaluation.rst
new file mode 100644
index 00000000..320ee8c3
--- /dev/null
+++ b/docs/api/evaluation.rst
@@ -0,0 +1,17 @@
+Evaluation
+==========
+
+The evaluation module provides evaluation metrics and utilities for model assessment in the Ice Station Zebra framework.
+
+Classes
+-------
+
+ZebraEvaluator
+~~~~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.evaluation.evaluator.ZebraEvaluator
+      :members:
+      :undoc-members:
+      :show-inheritance:
diff --git a/docs/api/index.md b/docs/api/index.md
new file mode 100644
index 00000000..49be0c99
--- /dev/null
+++ b/docs/api/index.md
@@ -0,0 +1,43 @@
+# API Reference
+
+This section contains detailed documentation for all classes and functions in the Ice Station Zebra framework.
+
+## Modules
+
+### Data Loaders
+Classes for loading and managing datasets:
+- **CombinedDataset** - Combines multiple ZebraDatasets for training
+- **ZebraDataModule** - Lightning DataModule for dataset management
+- **ZebraDataset** - Base dataset class for individual datasets
+
+### Data Processors
+Classes for preprocessing and transforming data:
+- **ZebraDataProcessor** - Main data processing pipeline
+- **ZebraDataProcessorFactory** - Factory for creating processors
+
+### Models
+Neural network models and architectures:
+- **ZebraModel** - Main model wrapper
+- **EncodeProcessDecode** - Encode-process-decode architecture
+- **Persistence** - Baseline persistence model
+
+### Training
+Training utilities and trainers:
+- **ZebraTrainer** - Main training class
+
+### Evaluation
+Evaluation metrics and utilities:
+- **ZebraEvaluator** - Model evaluation class
+
+### Types
+Type definitions and data structures:
+- **ArrayTCHW** - Time-Channel-Height-Width array type
+- **DataSpace** - Data space definition
+- **DataloaderArgs** - DataLoader arguments
+
+### Visualisations
+Plotting and visualization utilities:
+- **PlottingCore** - Core plotting functionality
+- **PlottingMaps** - Map-based visualizations
+- **Layout** - Plot layout utilities
+- **Convert** - Data conversion utilities
diff --git a/docs/api/models.rst b/docs/api/models.rst
new file mode 100644
index 00000000..ae6c6aa1
--- /dev/null
+++ b/docs/api/models.rst
@@ -0,0 +1,17 @@
+Models
+======
+
+The models module provides neural network architectures and model classes for the Ice Station Zebra framework.
+
+Classes
+-------
+
+ZebraModel
+~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.models.zebra_model.ZebraModel
+      :members:
+      :undoc-members:
+      :show-inheritance:
diff --git a/docs/api/training.rst b/docs/api/training.rst
new file mode 100644
index 00000000..e318b947
--- /dev/null
+++ b/docs/api/training.rst
@@ -0,0 +1,17 @@
+Training
+========
+
+The training module provides utilities and trainers for model training in the Ice Station Zebra framework.
+
+Classes
+-------
+
+ZebraTrainer
+~~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.training.trainer.ZebraTrainer
+      :members:
+      :undoc-members:
+      :show-inheritance:
diff --git a/docs/api/types.rst b/docs/api/types.rst
new file mode 100644
index 00000000..875c76b5
--- /dev/null
+++ b/docs/api/types.rst
@@ -0,0 +1,12 @@
+Types
+=====
+
+The types module provides type definitions and data structures for the Ice Station Zebra framework.
+
+Type Definitions
+----------------
+
+.. automodule:: ice_station_zebra.types.typedefs
+   :members:
+   :undoc-members:
+   :show-inheritance:
diff --git a/docs/api/visualisations.rst b/docs/api/visualisations.rst
new file mode 100644
index 00000000..798c5621
--- /dev/null
+++ b/docs/api/visualisations.rst
@@ -0,0 +1,15 @@
+Visualisations
+==============
+
+The visualisations module provides plotting and visualization utilities for the Ice Station Zebra framework.
+
+Functions
+---------
+
+Plotting Core
+~~~~~~~~~~~~~
+
+.. automodule:: ice_station_zebra.visualisations.plotting_core
+   :members:
+   :undoc-members:
+   :show-inheritance:
diff --git a/docs/cli.md b/docs/cli.md
new file mode 100644
index 00000000..bf2028fe
--- /dev/null
+++ b/docs/cli.md
@@ -0,0 +1,128 @@
+# CLI Commands
+
+The Ice Station Zebra CLI provides commands for dataset management, model training, and evaluation.
+
+## Main Entry Point
+
+The main CLI entry point is accessed through:
+
+```bash
+uv run zebra --help
+```
+
+This provides access to all available commands organized into subcommands.
+
+## Available Commands
+
+### Dataset Management
+
+#### Create Dataset
+```bash
+uv run zebra datasets create [CONFIG_NAME] [OVERRIDES...]
+```
+
+Creates a new dataset from configuration files.
+
+**Parameters:**
+- `CONFIG_NAME`: Name of the configuration file (optional, defaults to "zebra")
+- `OVERRIDES`: Space-separated Hydra config overrides
+
+**Example:**
+```bash
+uv run zebra datasets create era5-0d5-south-2019-12-24h-v1
+```
+
+#### Inspect Dataset
+```bash
+uv run zebra datasets inspect [CONFIG_NAME] [OVERRIDES...]
+```
+
+Inspects an existing dataset to show its structure and contents.
+
+**Parameters:**
+- `CONFIG_NAME`: Name of the configuration file (optional, defaults to "zebra")
+- `OVERRIDES`: Space-separated Hydra config overrides
+
+**Example:**
+```bash
+uv run zebra datasets inspect era5-0d5-south-2019-12-24h-v1
+```
+
+### Model Training
+
+#### Train Model
+```bash
+uv run zebra train [CONFIG_NAME] [OVERRIDES...]
+```
+
+Trains a model using the specified configuration.
+
+**Parameters:**
+- `CONFIG_NAME`: Name of the configuration file (optional, defaults to "zebra")
+- `OVERRIDES`: Space-separated Hydra config overrides
+
+**Example:**
+```bash
+uv run zebra train encode_ddpm_decode
+```
+
+### Model Evaluation
+
+#### Evaluate Model
+```bash
+uv run zebra evaluate [CONFIG_NAME] [OVERRIDES...]
+```
+
+Evaluates a trained model on test data.
+
+**Parameters:**
+- `CONFIG_NAME`: Name of the configuration file (optional, defaults to "zebra")
+- `OVERRIDES`: Space-separated Hydra config overrides
+
+**Example:**
+```bash
+uv run zebra evaluate default
+```
+
+## Configuration Overrides
+
+All commands support Hydra configuration overrides using the syntax:
+
+```bash
+uv run zebra [COMMAND] [CONFIG_NAME] key=value key.subkey=value
+```
+
+**Examples:**
+```bash
+# Override model parameters
+uv run zebra train encode_ddpm_decode model.n_forecast_steps=7
+
+# Override data paths
+uv run zebra datasets create era5-0d5-south-2019-12-24h-v1 data.path=/path/to/data
+
+# Multiple overrides
+uv run zebra train encode_ddpm_decode model.n_forecast_steps=7 trainer.max_epochs=100
+```
+
+## Getting Help
+
+For detailed help on any command:
+
+```bash
+uv run zebra --help
+uv run zebra datasets --help
+uv run zebra train --help
+uv run zebra evaluate --help
+```
+
+## Configuration Files
+
+Configuration files are located in `ice_station_zebra/config/` and define:
+
+- **Dataset configurations**: Data sources, preprocessing, and storage
+- **Model configurations**: Architecture, training parameters, and optimization
+- **Training configurations**: Trainer settings, callbacks, and logging
+- **Evaluation configurations**: Metrics, visualization, and output formats
+
+See the [Configuration Guide](configuration.md) for more details on creating and customizing configuration files.
+
diff --git a/docs/intro.md b/docs/intro.md
new file mode 100644
index 00000000..b364c4e1
--- /dev/null
+++ b/docs/intro.md
@@ -0,0 +1,18 @@
+# Weclome to Ice Station Zebra!
+
+Welcome to the Ice Station Zebra API documentation.
+
+## Overview
+
+Ice Station Zebra is a machine learning framework for sea ice forecasting and analysis.
+
+## Documentation Sections
+
+- **[Quickstart](quickstart.md)** - Setting up your environment and running zebra commands
+- **[Adding New Models](adding-new-models.md)** - Guide to creating standalone and processor models
+- **[CLI Commands](cli.md)** - Command-line interface for dataset management, training, and evaluation
+- **[API Reference](api/index.md)** - Detailed information about all classes and functions
+
+
+
+
diff --git a/docs/logo.png b/docs/logo.png
new file mode 100644
index 0000000000000000000000000000000000000000..06d56f40c838b64eb048a63e036125964a069a3a
GIT binary patch
literal 9854
zcmcJ#_ghoX7cGn*K?4W`P^xr9dX-2=s)7_L0g)Pd2}GoYu8}Goq=uqY=^(vJ3q7D9
zgx&<Ch8_s*jo<tJ2lu%@BspiEGy7!E%-*wRtsVJF=Q#x#6B!W^5rxJJRed6&TRVhn
zJHQ>ncihTY58>yQhyHVAq6+lGO~MVagOauq5m9v<`6Yyea8LU7g^33d5#6JIpIaLG
z-1|gCJhU3BN``QY-K@=)`@JW9M^t~b7uLWQYei|mDOJQ5UUg0~vIqbGt~C8wn@$P%
z5pl~zRjG%ihlB(HjNnDEe-dDz2JixSk(`6?==a`q;3sz5kB=vYkB^Usv)VI9k21rD
zJiTz9qguh+nKE8m#+pE4C16PIb7Iqf7uN3q_3Quydk+ycREf|Kaf=g!AT$7Pt5%T^
z8aVDmSdkMNl<xsM>Hc+OU`GfMo(G6M`@b>}|7lC2WNZAX;dG{O$?=D%iOq{^-7HrB
zcK+ZB)!$jy15VseoVK<M*JNvcZ<5ka24CYWzYG?d=b<nCKb2f<z90W4`oDkn`rAUi
zda{la_$I!dL0R}opfe>DTyWDkjAxOOZ*QX8d#=@20sP;i@Xow95<_tP$?zwjiu96e
z>w=n(W1oxV>vfZL+?;M$rHrbr-j~Q4Z0#f{{?B_kL)V~b;Ypj(r`bl+t51=jl6us%
zisP0c%+gd*@NjHx-9P?#e$1%)t<{43ZZ7psNeO=)Y*C@keuU`+V-r`LF5ytZC}IBu
zbG$h|;({qNsYw+4y*-`g9}w-)-1o;4J-XQ1@Hi(x-*u)|<LYooq*=mZ!|Ve)z`f{n
z=@cLS9S70Ec&yoije~wUJ<1%9pQ<{L2Fo0*^k5_R{(u7yo5y(-?}u8YCMOp+EC!jL
zZ2f*YA8<5MV$&B}q7qK2=F-}M&XQ&3Pz=P^dORRZGt>Ne#p@^B%N%Ah2Q;LC<VMbM
zLFJlkP2$TJ`k*6xD-Rc_*=lNXvgBlYs?=@1nNh6u0L5%$^K{-b34c7U3)YSaK@oH8
zO|UK$v~ANlu>G(ZHBQFL?Li-e*gAW=zAB)SnqFmSqA*R7HO(vfNpkV`XWrI=Kemp{
z`XTgmXPPHd1fbknj1MsBI<wtWoc%Vlx+rT((usJ)Z*Xw<%E9LNNr{!jbql2X&hLVk
z^rZ}-FzP*&N4;y!bY7eSKVk7{6QPP}SzF2PNSP9INP>};8fOdfpI;lh4^A@)#qeVu
zJb2)IeR*!gAy`Wti~X5b#3bXH#-tDsvNcs{`2~3O>45+@w=lsB@yx}N+7A*AT1iWo
zCLk(xWbfP7ppKMjUV$FTMNv+WKG*ZuS~AGj-P2i^ah`gNkqv6jA-Y4>M+bYJ1ouAM
zhio_#B1U3u6!)N$?mJ2ZbANVV>Xl|5+3DVVOU$d89?>$dF>ix#X2Zv>SuCqqWS!S>
zomY&g)au`g*ER&}`dKnw-|KyL(Xv>>BAvCz#~c7<2z4i2S3Ea{s$tl4;Fmfrw5uQ6
zdZdFou<ohNb4wDIi+Sd=i?KLqKF2!m3k4h4w3wH(?ztytc>B9Zn$Ox^;m&3&jBs=B
z%AGu-+-3>0hu*7<Q?%|e=|r{cEe1d9l7@o3whX=2wIG_mND=Xts#T$)nY4(JOS?}A
zn2<3^W%TgK-h7d6Vnfv97~EWofvlts`7t7q&UrgtvA8=gqc}%Br${GO`l4I5;O5$-
z!&?=0Vb+j7wL!+U19;lE3=h0Gm9UxW4x=#knfEeA#S$O&Hn`0@7Z&((B*da4?i*5v
zz&Q{?A<LOSC@P3r9m!zLQtL3k5i4t`vDm&9n@Hp3(%?oh3vf?${EG5l2*9arh3jpU
z8f3G*;gJ<vm#cH=VVk&A5CX$?-D&yzK4K?|Yc?5(H9*_*Ee-gCKDMmV;=2h7@cl38
z$iSr^w+`A#IMfMA0OELvh^ca&tryz$6qYj!Ve0Ni<@GY}1&yCL>*Go%ig0F*a+}bQ
z8Cc)R_4Xa}T)gvvu4H@GAS#AA)o|_JsNW8zdTY`Y2EMw$8M6iKf6zLo2}vX5#E`E8
z<e=;gfy!3v2g+0X?^>LfTXySbqo;<XPO`Ep4NCErg|Yej-$*~+l(jj8To(P@Kn7E<
zSsrf}hp@md@>)cm*vz`Y<&q8-QUpyBxlw(wEG~e8ar+<dZzZY;1H_bwO#v!N^gFbV
zb%?Xkq5BET@~tkWbv78by0dHtw?PfpzFJ<kM%;f#c!!x)3)y{|6Tgm*ZyPVKoxA2o
z=YHz9O*DkQ`KlIFL$~Itf-DVezKZixJoSNz=3c?(3q8V5Cvi(pId9AQ>}fF-S+sb&
zDz})rHK~=qsT-W;2Plhi{U0Y!6S$rmj%Lf#_6Q{}o9ON=pa2rAPpn&9&e(qYe-t*V
z@vGCn-F!I$@0)jPw(#1-v_r^JT~5YZW{`r1+085#GB%6IJC?RRv%5q~F>%c&OxynZ
z%xYjt7MVY0BPNAf>4{^p{XbrcwEclTApV;6FC515Ns#UfLZ<a;n{O=KYZy7U_~a`L
z!_`3&^_zQ9!-hgW&UDtSPoj4vPg}s)ILvBbun$aPoRR%Q`A3IlV}7KZ?{|J7ZL_k>
z2YrA=|5>ly8F0Bp+l*6!<>1heHsIR01D`C08YNKz!~*JpVLU<@0QpHnTUW{;>sYp=
z#eU02VX*}f3vp`~7iJXDzx9yXd^Up*0-yHrbarsvW*UH%P49|4$4@)tJgR$?6o6f5
z(}}v|BxI|mgea@2>qg^b#ozLf17HU@5Fa-Fraz@QN%7lZ5lq)Vn7Q=hZ-RdZ+wFlD
zJdw!7J1*GND#_)ys*=%<Stj~{9p?VKY=4jBCrgv-Vq6UEFBWgEHwYs|C-^#k!n;47
zLn9mBPS~A8=K9sDoVMqu>^U*Zr0;^&s<^_q%ds6d3-IC~_amnNV&0xM^1;`=@1xFn
zORQ(tnI35sf7lD%W&%N9E6Y~6qoNr#BAw2aiDiR!7CS8KoW|9)GoJAAS##ZIrQTUl
zBW}q?kb$Tk4JP=7j@W0(Ea^R`$D>gU%nfa^3Dwub5~JS+2Q@c7Z9!yGJ7`P^5kIj$
zg3O{je@?Kt&v;;R&~$K48WR=L6GcxNIc4yw)BgJ8Bb7oLJ2X_3X0F)>>o%A<Q-4=7
zu^;3%_m|)zd>^0m7n(dq+!+P%cBi)cl|I6CzcZF{kC1jgSv|j(+zAw~tn#IxO6lUd
zj3V;e_C=~at8H=>ZGE#9<8QfP>BH9b3(Dp1zuXnN-uc&csJ#%8Q4@!2to4XneWRff
zIaA{h=OO9fyAt`B20gSGZE0+5EGtCJ!AS6zFb)b4RvV0{cMY(`Y<6f+_ign{;I9w2
z?`CxPvQXRE34SVHT0>{c&%(Dx6)wu&)H)_KU+lGLizO9h`wd3$Z?JRA2VVyyEvYkH
zj67X5JX#+y_;{BJ<ne<o=nQf)=P{Z-CB<{m*cW)vOJ6<|q#t{nY8m8S@r^b2P9XkR
z!?EwevK3xEt_<9Xu%f5u#2hFQ%Z`c$#Q+K852NML66O4~G?F1X#O-<@V|_W}Qki~I
zMBO)K8Eh_pim_3Tc0gx3wK}vqS`?)`wK$q<@))^(80BOeA<K!r2ArK8tj09lTpo=g
zL&Q!ktF1#bBphOyo;bcr3?I3X4qJ3Vtrq5{Hwe}BbX?<IibB~j)Sbe@s#U8W0p8hP
z_f8|IxoAR0T;M+eoVhVX`lbGV$G+M#C-<fZn>&ZZ+qCX?k*~w*V_4;9w2D`5E~7T0
zi3~~~jxu(te?CA<_w_{5#uzKO&O9+lj}F{x+F-4r%K9((FwF&kFVse6mP!vDt_>x9
zUx>VaMt_HzSdkN>rs`F|pR*{TM8rR(unZk0EXqrLLqyw#%|A#KhSQVT6e&4<dpNPk
z%bR8;al>@$gbX?Lg3SMXEj8wDG)D;FDQ8q8%^qqz=<=X1rcVpN?A{w?-*WMkRlJWw
z3+-+`iUdC0c&rucqhLSGU;|L-v$MoCUgN^3b8#YnlqTL^wOuSldYIkFOhc9*s!|7C
zZCfJ4{p-Vci9_o*vV1IliLv_q<tPfxGQ5BU(192<*_l>g!ONlaCFU*O(&by>`lq|I
z4hpOFuCt)IyVtJs&2^hgfhWI>P3B?isG8c+o4E?=CTZWp{P7tyGpzM%&=GR+HEzQq
z;QD++XUMPpY$fV5j+c40`CQ?TIK@slTaYNrn;_-|+;Pj|71}f4JSG4)@AI{Y``0;x
zLIAw$!n>UCW{q*q4}sT5IX7vGytw4;e6H4@D|~;@%gt~92mAUO5uvgxOB5{Ep(ELz
z2y^2geQ@Ae;wJm&>(%ce!yCWuih%7rn$vb`hZwIICxbe)vwUsJ`2COHc=-h!)ol1J
zae_fdeqQ#!4Z#;G@7#18om~t^Dkw@;k|8CYnl4`WYjT=O>;V$I*8CVeKahu}oThzI
zby8<sfh^Kwb559;gxN{V6lYwSX0g=dou1+FGNxJbtczLNA~FOxW`!70nHP8;)IId5
zcI5W$xZ9nmn)UFwhM4eSP^%THp4yA`3_g1q<0y`be!NE}5;pd(Om?p^4`SwzKi=ed
zhz^1;MabvweQI5lqRjSJQw#NwJb9=XBv;HML&d1Pt=FlNq|@KTi<!uLE_dY$MYR?E
zaRnx5Jm!6E@BnHrG#7kx#kbfnB}H}44)CeB<-$aWMXg_+j|SJ*_rXWcg`XLoqgNxl
z((_VnX?F&H$CVt??`g?O@^#iWW}94FNi9!yeB2T-F8vi;U(GfP!^<bj|KzK6WUPH<
z+;*cJ__)WLvSZWZonjhIs*n<Ki{FNymtR*-;!J^fn5r@)5<PaSzd$AN6N0oJ`OIFT
z)lqdqTT10MBbG;12g8JAf~a~@8@28bAnVNKP$THVCi*4Ul7(*j^ElQ*u4l@Oed?b{
z6KWJm9pH@c!_1Gn3%E6=%GHFk-uIGiS9-TwPf)BK>mM|V!o$r$h~2x@YYgecvv)44
zVEmn@-71;kO#&Fe!dj}OTkMCigz^2|hDFfuhsUY!IpqW^Rup!q8D*X-)f`dZYB%V(
z+J*fl9B5WrGvpO-E^E$NZT%lAFfaIUD6e?hS2V7Wc__#^DX?+UE*yzRce_CIV*Ig-
z{@Au>EMGnM(+{WpNRX7+Z+dydzM}QZc1KP7jO|yav-WKD37#31CiIdmppsvasoZjJ
zhjMmpSbHGVq~5PpJY6V>>3^|%q!?r@6j>j<0Q>N?Q0ng{%+Hv@V2buU<19&o4fYJ3
zRGPrfikVAIeWWMdn<`28#Px;wwVB2fJsK(!YG{yS2zy&s*m4tR82lID$;!4LN{)!y
zpw)6_si`@A8LBejn^g}Gj<kN*eNx}H@k7Fa?OGjkjs(ix&~%~ZY<w4UTtdcub}-9k
z+VXNgURWH(S*|v1Tr49a8_7XA)fZpZKV&;8$S%la$rCekb<rFzU+=YGGXnclm!u%i
zUUxmrdxhBb4uZ!KLfG;Po~YKn_x`~`735>hqlZDAg{@exDRYNd-J<uD)w>HnKP<H!
z`H$W|>1SG{R>+AL4dGabfD5bgVHmK*ej73ye~XLd@pb%2zq%8KX$Hu7^Z0-YJ;>P`
zMz#ko5|<$pp_sI$-oYj5Rh=9)S^tdB2NesZ#!D?}dqn52D&U`j+g9hxWVkkYBdn4%
zc1N30W|e88l8+P(9tA)ET&$81!y6FlDYdS0di~KK=i%a0-3?AToyPe^X?C+|Opi&`
zbYC5`m0#x7y;R^{@3?t`@KHC#yOZy27qg$X^7DX*k*Y3=r*l?48H^0m@Zwspa2w!=
z8Rzo~D@*^~I(w;37S?CAu65^aOXttu1t0tLL~jVQR1W5BCjS95x7_>(Zn95tLXtC*
zAm8pA%*Ozxz$w46`Mo9f*vB&x+b$=u+Rs;z6TfMxU!%gWE+ByCzxygTL4BDhyv1d}
zD{w^)?0bgm#ph9M!q4%-kFW6k$paVLINgXgd}#yNe44b#J%%(m=NxxGP{<*vw)MiW
z6(l~^rYnHK%O&5WXN!98Ny<2ab6T^H9CrHXik+$sMot2o2Lo_hnsKuJwz^8h{%eED
zr2qYWAmxXK|7vS?)YGY<p;z7stGt@sUzny-FDo`(C|^C{b4eL5{$X<z$C&tjy)qMZ
zPRa%XX)9Hwi{$J!KFS>#yL&+^&tb?CV%7@vu~e0v#UWvx>Telu)*eDs26wvKAAXce
ztkMG*S4qdZc!ua^$*k4(E6U{?$oIskaZkqURK+y3u8q`gKrVl;*Kr~!{`;%OR=SeB
ztg)-z=sVw9%gUM?9nbAWb9|%m;w8yNvf{LmJDY1OcB@=q+=4Avtsm1NlJkLj{9Zl{
zRC#RkkoY@`MSlwW&pUEARg2XK0BFF<0#Y;GEnlJE-CR#m7hqrV%3DU|i@%R^k^PBt
zL9=sbeO)J@Xjbkq>qG>iL5LcW_dHMcNq-6<E~va&_aV>n&mRKy6!O?ZPQK4yWR5ho
z{xPlMGn^<hk?<nyj->?DBvWZmb@A?AY_C}N(gWxoy$S=QS5eTZZNYtHt5=3oKWhj+
zaI1UQC{CL^LFo3hB0Yv-r9m2lrMlI5bVANzvQRAEKf+Mm4kO*IX;k1Odq94dXD2Rs
zWX}=%8D2#S>f=WSng80ZNRQ|oV9VtCLzT;8yEMCSo9+)@Kf$MS9rA)R#TWwx9jD+W
zi=Qw0Y3I9G%tn(aT>or~nGrp+uA%epd$M7pH7C*qpS6v-koOaxCl@1mMC(q!bC(s)
zUgY#LBuJW)rT0r8sRU(ABiG>{p%6yPkp?S?iJpV=r}PnKq7z9&)n=Xca+&dPn@b(&
ze@Ry7r&SX0G7$e$1tfQ_eZVwx$s~3PWZ`c=&{$USD7g>1-9MIsOBgfi&|QFmfGN66
z9#c7bCn;+><g%bn9j7_m8dV0Ray#h?WFv7b&8Ksr$-{x2s_{a5s<bi;WPJBI$lwTH
zJ7EFa0@rlCNu}KV%{$KlhmG|*V<Qy5Imw%r>Nxde;IuKZxgr+*ZjS~=78Slptsx0B
zC(yjsMZl}YK{pqR$m-cI5O-qwWr{63uC0&=YTv<yFpeCAb}UIZnOR)$Cldy#JAJut
z6N2lS*&7if4#N)fyp9K5wB@hmN`lI!Bpx?)a?-dWt!QKI&~BhK_s#UF$E*%3y@jvC
zU;cXdZboiG`8O+CL9;ilCYaywe6GTwZ<+<lDvy<L%*~)TEy`b6`-(P_RU(nKT>T9y
zqo$r(5f9TobpUqSOOL1pntof&8#PdLxxmJ+JLjGv#)W(sdt|m&Pg~Ei>X{9WRM-FL
z1ug=$A>CFfx;j-KXvS4L_(V+6QyE%^N?!-xBP2BBQ&_ebDIcw^RMMR1W<hERto<D2
zD=9GzpY|jJP#>|7&hYIg>2|Km|AZ``=`r~-Lr_^!%2k1B)uvrP6qZ4d`6mPBN>!xZ
zJk**bYPy$w%2j60-W`={AU!!oHcBpiucFvTp~*34H)rMJxvaCFizQ$>@9ORE9?hrY
z_T-JG%a{$#O{{Y(Q@<e0&-ct6;w}@#Uz&B$AqCx)or($UO<4ooWFT(xacJf*Z|mWk
z^c#`3Bay8=(+WuocykZofZVX@eI;XF#EShr7&1-eQhAlHK-VWHei$I(mW8hIjtP-M
z!sBCxpNPUtzhkaK!Awz7<le%oQ^JtA2NOd)!#bGkZPR+e0LiidvU-0isi4nvMsK6^
zs?19|UZ?+IT(e>I#^@Irxli=@R7a-z_}8Dgl&lu4==oQh=QPkJP(*KtS8U5075dF7
zk<6-UO^#Ci_8qvBD}L=^EXWWq<w<?Z_W@Ls{SITt>C7Ki;}V;^B#BGlT;7cAwRaC&
zbWyAQj{@gNy2Gix);R!v_O^)R%4Ip-S@)aIy3x`iPB(2_!mn0a%vs>k4@ENe8|dXK
zHIjH9)!DsyQ_armEk(s_CL(LDUW*)7qrAO%P<3Cqijj$(X$*6}#_C8^v1VmCWJ3)T
z|NZ4>M2bedT9pKY4Q7bvkLx|;@_%6ztsBvH1rl^a`}A}Jw($PS)0VjqRNGVbvSsj1
zeq5c?zFG;a$eXVSb{-Qhrt$Ln4+G5@1M_i1Fd>I!(Z%Ryk{|<_Z0^nWo_yDc#qZRN
zW*Uzoe6ISr;?i&$?@WdN7*w?_e&Bt%7FO_$#Pp-o<B3e30}cNL0?K)l&w9G$gf?os
zI<nKS`4RF{1;SUe$tH0ZjdlG9cD;x5uL@f(s@0TTfYCw|+w_g#<CS9b0$YPN&-yqa
z!YvBMJI;}`lpCFN_HLd<gfQ<)q_b~j?9^^NxD|5+yxQ%~D1Gi<kM<MHtk!51QY5T(
z5cpTCD`Vbb%hL<L#&ga+1PeF?jv+4gv;eK?@o^&VVlD6Sm>%+Bmb?YO52TFJ_X=Y`
zB79{;AGE8w(H+`M-ReL&@+8qpOiubk(5<r5CGUCnCb{L;1qQR<nrR~!y<{Y=qc5tu
z0ZO#<t@+XXe=lAFGG8?Y%Nl`-JpLH8T#~}8&B{y5%7)?m7Lb>AfNWE$Iqg?mQ0-sS
zlV5}N6=QWM-DmG5T$?m-d0zL9tvkD61+==-ZvvE}&!_HEa);T%|5iUNQgr??Arj2v
zYeVDHiT2)^j`CqWEdiHi8rN_or|xDgsM^V2=a8S%L1RY)zkF1Bo+rlV-5C~88P38p
z>}G^G6k1xQ5BXO3n!~$(MW8-5Y&VdjIRXZ<l}H|bSqOPwGMd6;SKWeoA}+(Dm%xp2
zH={z{CC=S=0P5gC%b68(#L~Y{2;Djv)Og~bzme>``{U*C+40wek?4&Psi$FSNhg8N
zU>DZ?ITJ2d!p5txmKpAB-_hjqgY3%%Myhw3#rRoHotWKDxD&LqP=@Yh^miCTC#uU(
z=E!Jmu<%zp1u}I+JV)@$hXbDq!nQdddw1iD+p{!H2R#miIqW_TAeZvSG>?CwQDl<=
zq&r!EMjYv>v=zr(%WfQ4B|0sk7UEOk!}O@D@vX9{>t{X+!7w~tYw!I{;N8C%TN>!M
z>7xX?(GH%vZg|!Xp4X8E(FQ-T=0g3Wlt`Tf|0;>yF&gVyM`s}om7?7plxL!q;@a!V
z{l4{qolEEroMw2oJNmp@)G2ljpK|T#-8cW*{bS2K$Q!%h%5Qx>Yp}*|3=`1IrGYA_
z#3pFJc%b*?vw(G9JA{OJs6Iu?03Z#!9|dMV4WKd?L9VWXkJ|UY=bg3AH;sIrwQD;I
zc&6=zrtXy=AOQHjS}Aa=9}Kkvkysnn7oOmLzpHuu&^6N3?IQXpetcqqXIvVbh9q;e
zZ*y5xQ0j@_UR5}&q#}Q}_z?g~1NZ0~DoWtg{a2c3{5#i|(VB{zs7lh{ns-0}39+eZ
zmuodiGS}9p!Cll;j;+GMld@E%{}vT(vQ^7~sZyUydd{}xs*Gvp`d6JIiVu(*_EN9q
z$Qn5T>zL^jWs2J*dS)WbaTymtJNWt7SCtZNBxs!#HlJZ0Xj4IzF#0FmKaa9WFj*t^
z4$IrEQwQer_l3e3LFZ0uR?w~Qj8tBUW5aL8_8yvFiQH_QgmCjr9apD6&DIQX(SNWv
zb|F@%eNq*cn1*MdhziznN^Xq<tqQSecPx<?=$5bc;xjd(kWy0)bT=>bD54Rd`f42e
z%dkRxE0pw|k;g*Kxz=~~Q^d%iQS|mdZfV%PFuTgKOoQQ2B*Db7CQ{U+%(vqjr<M?7
zogTbPL-1I#URp;6Mt{qy&2I0mtFI`5?3=k{QLplCOq%OktQaFUp+Y7%1Gm2oTKKg=
z3(wpsX_|7Gu9u|t0^DRGmA1w_Tq7Q{Fv`&0W3k0wA4pR&gUXg!4`hc78Bn24E~;9X
zxGTO6W}bk>2@+)eLf{cZscW-ddJS@qnyU6i*!6DF%fms`AZv@>Z`K>M^jl7)Jy^-;
z(0WW!4OGD=qRpx%OsLesm*9)M|LH&G?IjJgE9N?vI~25Tc)^B;`$p7s5P+z)rqsu8
z#LN*-*k4E7rlzJtJp0n5I7ds<0+d9DE{E<ec64xT4okA0l^R!i$7V!ZRxP(++g%tw
zj@Sm!c{1hhm$fxofHwN=4x$X%e?>_46|Ejr244-$k>h0krj6YhP4oX0j<t|rVVt?!
zWZzC3ZPPN<72sn8I`T%mFA(&Ni3JWMyZK++tyOA)(Puhij6RDtuU8q;IR*5RV4a}f
zt-s(AeA?I~Z6Rn1&`w%=yIMSb+pZ9ZHZ+8p=7kO_SY-Y@mM;D;)ol<gj2K~%vpJ{_
zgnFY%L7XX0mrNol*|MvX07(LgeP4MYZ9x*{f7tyo1Gnz_ygvu6VaRH3NH{V^d}u+}
z-hADXB!1uYw_;<R9;$j%-moA(3&jvKODQfpPRcp7Tqpp6t!JdPJZaTS0fBcaQb(jc
zvfNTWL1lOtrbe)#`QM>S7JghDO3@cFKC#AZ`8L30t0U8)M^2*m&M6}$Qp~Q_wmx(G
z<sR?j{A^yM*z+#_&0Km)3jZ-wLbV-VDv5HTyp3?tY{L)c7e1-LM(TX15FwmMr@CqK
zNRM`03*!ROp*_9?Vox&ZFQ@k<FXIvSL3H5R*CJ+|@aMqepU=_1C%RRbut44Z*;rAh
zuWbBE5&Bi;Ky6McxYwRaomvzf&(?b!J&)Gbqhzb6pn!Hyh0V7ZvSL*d7Rs<&FZNg4
zn{k-kG$6ik)$>n(!n3Y)O0=4MK=5P0>QQfvJ@cAL_pnWzfF4g)K|wu=I~FYX(3W-2
zf|@^YU!Ut&*_K+D;p+qF5BVv9?k(CLxvwrM?*$1Y8Q1rO%po-&x{`*9F<g!1;C87Q
z&UU7Y*PU|`F><>gKc8C(qtBR&?*4F>(;9=xmQ<Ho$f+J;V7E!VuFrcO2Ra&wiU|OC
zxAPfwdaJBV3%2m}BoVlvvHXZSE7)A)PxxSvdsLK|&IZovZ^$urQL$KP8&5ZTCOH;q
z#VW158b^0upqCf3C%3K@^#P!Ln;Ob-ulfO0*;T9ZAUt-}b|_WEr=pOx$Dg1%*$>ap
z#=6YaIOw962LhIK=$)s(7ibVg-6j|qt}Gf?spXuC?|60jPfUv_x01+;B7RU=)jPnT
zh|?|SxQAbf65$EmPTvdZzt8N+PABxnwrkm)Y?Tga)nYIMgoc<p;{X4y296(91^;z@
z-7=fwIt)LJ=TJ$U(aDyRHoO%Xpdu+ilV8;sZK+aC3Q~~`>7w4XzRV2$`%ex6z9bNU
zTv2t^8?QF3hskl_jm7(RNCWicsx?yw57HN%DNW%~m{)QN=KZ8m6{!i#T2h!Xg3@O2
zaAK4htobm}2YP9pB2afR<%OFoY%oDA4Bs?^mtDV=I(pY}zRp|(4qBFfrFG}vZP7x$
zMB$pC$#-tpQDWYIddI0^+71N$Q1U1_4^iys=?2}<v`hcg0XR_qr_TwJAjA0vq%9y(
zN!0k`$8>s!KPO2!JcD*HVg#F{oEWIe*nU-%yV<;YJz}09_`Dz?AWLlKA$!=5B7tkp
z9_Ihe&3$NL+wtF@TpDvLR)ihayRpLvH0}o-Zvte|uU@(+0lWT*aU3ZK?GKTLYcJCO
zQ~U7QT6{r3c?3TzU|gX^V}%M%XI;xd_qK-&M9KdV1Sos|8}%17JACDbM!iDforMt*
z<z>7Auxhgv1PQq~6FDWuVifhd=4`Vl2Xr#vI%}S{cQRAwGNOF9zF0RTH&w_q#Z%t4
zGx*92|7e3Cd$W~Y2_l4SU!I)$Ol%$mL(dkfgnhfk3#n<-t!kX(JFLhS_|%>=Fst7u
zheXTT)Vo^bsf*`klEo@*>S0f;%3gz^+m_^rcv(QLan(?cKx9RG{g^Ez;Q<zhD%&(q
z2m-fMsq$b#A_uZhKh6I5a63i{MV;7iIo{My_n7Rhuuud$V1o>tBl6Ph+saou0`c!|
zI8XcO^OnR(X{|3YvOxJr%@#4@tTR!0O7_qzZS`-=iZ3mwL3@L<MM~!gNznT4tn64K
z4u%umSM137if1n0b&)w1^+_81Hl-OdeXe1P*sX9}dmxRfO{eqPU+Sz~-mv};>wASi
z(QvV}`KN5>8$=N+@p9IR8VfQdvSZ+Lf{Fv;$%v%_0s%+RBoafg<S)Y~v(SqZGyTNK
zvQ6A#u%c6e@b_wyA)m=IlR_B=SB2}d)&7{e5c3*WT?c|L^p$i|#Jt&1J6*ijHuj>;
zB^upVY;ewq1QLHeD4y<6Oa4d6hgbOmM;(hw8Y(3@UM)XV_nC91+I{svghGcwL9DQC
zyM&5PhT=%Y7C{j)JyC3slsF1h)J!2ju6cNc?HhXJLHl25entk$v!XYOzK=(rP~Rh!
z*p(T?yl4h)l~Mkkoa1>4%y{DERdSd$UE=vGXLs>#A3q)Cuz$F)ey2S&b!HYf=Mm@i
z$vBfjX|diF=}~}Se`0d%u`^s!Jiz*S>KK$b5mKnTyL{tReI0e>|9%tuAFB^Xu1EqI
z2*~6xoM8t-esZMcNE3x1OqyN-Lp+FtX23bZdIhv1I_L3poeEE12w+x`r3BtK?UgS_
zgjv<CpFFYw1RZbkPq`GMwf6UELSc*czltP}yIl*(%sG++s&bhlDsQ4BJ!+5A&A9JV
z>%6!;CN9dDzM}v3-DxU~SIgW9;-IvqR%OnBm4Ejqg0G}YnQC~*J|RZ=pB5-~vu$W~
z)Un>6^zlydKLzhIZ?b;=zuG68MC1PzKM`{<{lBe>Vh5<j>EBTCLDuB`X-584ml0{G
L>8MsHTOs~GC;Fmw

literal 0
HcmV?d00001

diff --git a/docs/assets/pipeline-encode-process-decode.png b/docs/pipeline-encode-process-decode.png
similarity index 100%
rename from docs/assets/pipeline-encode-process-decode.png
rename to docs/pipeline-encode-process-decode.png
diff --git a/docs/assets/pipeline-standalone.png b/docs/pipeline-standalone.png
similarity index 100%
rename from docs/assets/pipeline-standalone.png
rename to docs/pipeline-standalone.png
diff --git a/docs/quickstart.md b/docs/quickstart.md
new file mode 100644
index 00000000..f3256882
--- /dev/null
+++ b/docs/quickstart.md
@@ -0,0 +1,83 @@
+# Quickstart
+
+## Setting up your environment
+
+### Tools
+
+You will need to install the following tools if you want to develop this project:
+
+- [`uv`](https://docs.astral.sh/uv/getting-started/installation/)
+
+### Creating your own configuration file
+
+Create a file in `config` that is called `<your chosen name here>.local.yaml`.
+You will want this to inherit from `base.yaml` and then apply your own changes on top.
+For example, the following config will override the `base_path` option in `base.yaml`:
+
+```yaml
+defaults:
+  - base
+
+base_path: /local/path/to/my/data
+```
+
+You can then run this with, e.g.:
+
+```bash
+uv run zebra datasets create --config-name <your local config>.yaml
+```
+You can also use this config to override other options in the `base.yaml` file, as shown below:
+
+```yaml
+defaults:
+  - base
+  - override /model: encode_unet_decode # Use this format if you want to use a different config
+
+# Override specific model parameters
+model:
+  processor:
+    start_out_channels: 37 # Use this format to override specific model parameters in the named configs
+
+base_path: /local/path/to/my/data
+```
+
+Alternatively, you can apply overrides to specific options at the command line like this:
+
+```bash
+uv run zebra datasets create ++base_path=/local/path/to/my/data
+```
+
+Note that `persistence.yaml` overrides the specific options in `base.yaml` needed to run the `Persistence` model.
+
+### Running on Baskerville
+
+As `uv` cannot easily be installed on Baskerville, you should install the `zebra` package directly into a virtual environment that you have set up.
+
+```bash
+source /path/to/venv/activate.sh
+pip install -e .
+```
+
+This means that later commands like `uv run X ...` should simply be `X ...` instead.
+
+## Running Zebra commands
+
+### Create
+
+You will need a [CDS account](https://cds.climate.copernicus.eu/how-to-api) to download data with `anemoi`.
+
+Run `uv run zebra datasets create` to download all datasets locally.
+
+### Inspect
+
+Run `uv run zebra datasets inspect` to inspect all datasets available locally.
+
+### Train
+
+Run `uv run zebra train` to train using the datasets specified in the config.
+
+:information_source: This will save checkpoints to `${BASE_DIR}/training/wandb/run-${DATE}$-${RANDOM_STRING}/checkpoints/${CHECKPOINT_NAME}$.ckpt`.
+
+### Evaluate
+
+Run `uv run zebra evaluate --checkpoint PATH_TO_A_CHECKPOINT` to evaluate using a checkpoint from a training run.
diff --git a/docs/requirements.txt b/docs/requirements.txt
new file mode 100644
index 00000000..7e821e45
--- /dev/null
+++ b/docs/requirements.txt
@@ -0,0 +1,3 @@
+jupyter-book
+matplotlib
+numpy

From ff8b8bb9ae854d3a2928868cdf08a615f1ed1c26 Mon Sep 17 00:00:00 2001
From: Sophie Arana <arana.sophie@gmail.com>
Date: Thu, 2 Oct 2025 15:33:42 +0100
Subject: [PATCH 2/8] adding workflows for deploying docs pages on PR and main

---
 .github/workflows/docs-preview.yaml | 40 +++++++++++++++++++++++++++++
 .github/workflows/docs.yaml         | 38 +++++++++++++++++++++++++++
 2 files changed, 78 insertions(+)
 create mode 100644 .github/workflows/docs-preview.yaml
 create mode 100644 .github/workflows/docs.yaml

diff --git a/.github/workflows/docs-preview.yaml b/.github/workflows/docs-preview.yaml
new file mode 100644
index 00000000..a85e602f
--- /dev/null
+++ b/.github/workflows/docs-preview.yaml
@@ -0,0 +1,40 @@
+# .github/workflows/docs-preview.yml
+name: Docs (PR preview)
+
+on:
+  pull_request:
+    branches: [ main ]   # runs for ANY PR targeting main (e.g., your 20-create-docs → main)
+
+permissions:
+  contents: read
+  pages: write
+  id-token: write
+  pull-requests: write
+
+jobs:
+  preview:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      # build your docs (use uv or pip; shown with uv here)
+      - name: Install uv
+        run: |
+          curl -LsSf https://astral.sh/uv/install.sh | sh
+          echo "$HOME/.local/bin" >> $GITHUB_PATH
+      - name: Install project + doc tooling
+        run: |
+          uv venv .venv
+          . .venv/bin/activate
+          uv pip install -e .
+          uv pip install -U jupyter-book "sphinx>=7" sphinx-autodoc-typehints
+      - name: Build Jupyter Book
+        run: |
+          . .venv/bin/activate
+          uv run jupyter-book build docs/
+
+      # Publishes to /pr-<PR number>/ by default and comments the link on the PR
+      - name: Deploy PR Preview
+        uses: rossjrw/pr-preview-action@v1
+        with:
+          source-dir: docs/_build/html
diff --git a/.github/workflows/docs.yaml b/.github/workflows/docs.yaml
new file mode 100644
index 00000000..d2c07e8c
--- /dev/null
+++ b/.github/workflows/docs.yaml
@@ -0,0 +1,38 @@
+name: Docs
+
+on:
+  push:
+    branches: [ main ]
+  pull_request:
+
+jobs:
+  build-docs:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      # Install uv (or use your preferred installer)
+      - name: Install uv
+        run: |
+          curl -LsSf https://astral.sh/uv/install.sh | sh
+          echo "$HOME/.local/bin" >> $GITHUB_PATH
+
+      # Create / use env and install your package + doc deps
+      - name: Install project (editable) and doc tooling
+        run: |
+          uv venv .venv
+          . .venv/bin/activate
+          uv pip install -e .
+          uv pip install jupyter-book "sphinx>=7" sphinx-autodoc-typehints
+
+      # Build the book with the SAME interpreter/env
+      - name: Build docs
+        run: |
+          . .venv/bin/activate
+          uv run jupyter-book build docs/
+
+      # (Optional) publish artifacts or deploy HTML to GitHub Pages
+      - name: Upload site
+        uses: actions/upload-pages-artifact@v3
+        with:
+          path: docs/_build/html

From 74dcd1839f4cb7326e72f7eb016c83b6cc41ff14 Mon Sep 17 00:00:00 2001
From: Sophie Arana <arana.sophie@gmail.com>
Date: Thu, 2 Oct 2025 15:37:57 +0100
Subject: [PATCH 3/8] changes due to pre-commit

---
 .gitignore                | 2 +-
 docs/_config.yml          | 2 +-
 docs/api/data_loaders.rst | 2 +-
 docs/cli.md               | 1 -
 docs/intro.md             | 4 ----
 5 files changed, 3 insertions(+), 8 deletions(-)

diff --git a/.gitignore b/.gitignore
index a0ff7939..71cac5ea 100644
--- a/.gitignore
+++ b/.gitignore
@@ -11,4 +11,4 @@ outputs
 wandb
 notebooks/data
 notebooks/*.nc
-docs/_build/
\ No newline at end of file
+docs/_build/
diff --git a/docs/_config.yml b/docs/_config.yml
index 2791b212..ac67a063 100644
--- a/docs/_config.yml
+++ b/docs/_config.yml
@@ -22,4 +22,4 @@ toc:
   - file: api/training
   - file: api/evaluation
   - file: api/types
-  - file: api/visualisations
\ No newline at end of file
+  - file: api/visualisations
diff --git a/docs/api/data_loaders.rst b/docs/api/data_loaders.rst
index 46f44d6c..f44b8b29 100644
--- a/docs/api/data_loaders.rst
+++ b/docs/api/data_loaders.rst
@@ -85,4 +85,4 @@ Using ZebraDataModule
    # Get data loaders
    train_loader = data_module.train_dataloader()
    val_loader = data_module.val_dataloader()
-   test_loader = data_module.test_dataloader()
\ No newline at end of file
+   test_loader = data_module.test_dataloader()
diff --git a/docs/cli.md b/docs/cli.md
index bf2028fe..e0366c6f 100644
--- a/docs/cli.md
+++ b/docs/cli.md
@@ -125,4 +125,3 @@ Configuration files are located in `ice_station_zebra/config/` and define:
 - **Evaluation configurations**: Metrics, visualization, and output formats
 
 See the [Configuration Guide](configuration.md) for more details on creating and customizing configuration files.
-
diff --git a/docs/intro.md b/docs/intro.md
index b364c4e1..bf2498f1 100644
--- a/docs/intro.md
+++ b/docs/intro.md
@@ -12,7 +12,3 @@ Ice Station Zebra is a machine learning framework for sea ice forecasting and an
 - **[Adding New Models](adding-new-models.md)** - Guide to creating standalone and processor models
 - **[CLI Commands](cli.md)** - Command-line interface for dataset management, training, and evaluation
 - **[API Reference](api/index.md)** - Detailed information about all classes and functions
-
-
-
-

From f9ce5777804965f392f35e48e3d0a5ac8bcc83c1 Mon Sep 17 00:00:00 2001
From: Sophie Arana <arana.sophie@gmail.com>
Date: Thu, 2 Oct 2025 15:42:15 +0100
Subject: [PATCH 4/8] linking docs pages in README

---
 README.md | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/README.md b/README.md
index 0ddf39a9..822a1773 100644
--- a/README.md
+++ b/README.md
@@ -2,6 +2,10 @@
 
 A pipeline for predicting sea ice.
 
+## Documentation
+
+📚 **[View the full documentation](https://alan-turing-institute.github.io/ice-station-zebra/pr-112/)** - Complete API reference, guides, and examples.
+
 ## Setting up your environment
 
 ### Tools

From b2800eb7f0d3802231cfdda60b5a55e6ddd14c75 Mon Sep 17 00:00:00 2001
From: Sophie Arana <arana.sophie@gmail.com>
Date: Thu, 2 Oct 2025 16:17:01 +0100
Subject: [PATCH 5/8] fix permissions

---
 .github/workflows/docs-preview.yaml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/.github/workflows/docs-preview.yaml b/.github/workflows/docs-preview.yaml
index a85e602f..fc789905 100644
--- a/.github/workflows/docs-preview.yaml
+++ b/.github/workflows/docs-preview.yaml
@@ -6,7 +6,7 @@ on:
     branches: [ main ]   # runs for ANY PR targeting main (e.g., your 20-create-docs → main)
 
 permissions:
-  contents: read
+  contents: write
   pages: write
   id-token: write
   pull-requests: write

From 5e2fdd1b8b7db6cf2a6638caada1a6f7093cfd11 Mon Sep 17 00:00:00 2001
From: Sophie Arana <arana.sophie@gmail.com>
Date: Thu, 2 Oct 2025 17:14:30 +0100
Subject: [PATCH 6/8] add model docs to include diffusion, vit etc

---
 docs/_toc.yml       |  5 +++++
 docs/api/index.md   |  9 ++++++---
 docs/api/models.rst | 24 ++++++++++++++++++++++--
 3 files changed, 33 insertions(+), 5 deletions(-)

diff --git a/docs/_toc.yml b/docs/_toc.yml
index 24354482..7c041df2 100644
--- a/docs/_toc.yml
+++ b/docs/_toc.yml
@@ -9,6 +9,11 @@ chapters:
   sections:
   - file: api/data_loaders
   - file: api/models
+  - file: api/models-common
+  - file: api/models-encoders
+  - file: api/models-decoders
+  - file: api/models-processors
+  - file: api/models-diffusion
   - file: api/training
   - file: api/evaluation
   - file: api/types
diff --git a/docs/api/index.md b/docs/api/index.md
index 49be0c99..ab13462b 100644
--- a/docs/api/index.md
+++ b/docs/api/index.md
@@ -17,9 +17,12 @@ Classes for preprocessing and transforming data:
 
 ### Models
 Neural network models and architectures:
-- **ZebraModel** - Main model wrapper
-- **EncodeProcessDecode** - Encode-process-decode architecture
-- **Persistence** - Baseline persistence model
+- **[Main Models](models.md)** - Core model classes (ZebraModel, EncodeProcessDecode, Persistence)
+- **[Common Components](models-common.md)** - Building blocks and utilities
+- **[Encoders](models-encoders.md)** - Input encoding components
+- **[Decoders](models-decoders.md)** - Output decoding components
+- **[Processors](models-processors.md)** - Latent space processing components
+- **[Diffusion Models](models-diffusion.md)** - Diffusion-based forecasting algorithms
 
 ### Training
 Training utilities and trainers:
diff --git a/docs/api/models.rst b/docs/api/models.rst
index ae6c6aa1..6cd9dc11 100644
--- a/docs/api/models.rst
+++ b/docs/api/models.rst
@@ -3,8 +3,8 @@ Models
 
 The models module provides neural network architectures and model classes for the Ice Station Zebra framework.
 
-Classes
--------
+Main Model Classes
+------------------
 
 ZebraModel
 ~~~~~~~~~~
@@ -15,3 +15,23 @@ ZebraModel
       :members:
       :undoc-members:
       :show-inheritance:
+
+EncodeProcessDecode
+~~~~~~~~~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.models.encode_process_decode.EncodeProcessDecode
+      :members:
+      :undoc-members:
+      :show-inheritance:
+
+Persistence
+~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.models.persistence.Persistence
+      :members:
+      :undoc-members:
+      :show-inheritance:
\ No newline at end of file

From b911cf3dfd8748e25b45298373f5141647148144 Mon Sep 17 00:00:00 2001
From: Sophie Arana <arana.sophie@gmail.com>
Date: Thu, 2 Oct 2025 17:15:09 +0100
Subject: [PATCH 7/8] adding missing rst files

---
 docs/api/models-common.rst     | 107 +++++++++++++++++++++++++++++++++
 docs/api/models-decoders.rst   |  27 +++++++++
 docs/api/models-diffusion.rst  |  28 +++++++++
 docs/api/models-encoders.rst   |  27 +++++++++
 docs/api/models-processors.rst |  61 +++++++++++++++++++
 5 files changed, 250 insertions(+)
 create mode 100644 docs/api/models-common.rst
 create mode 100644 docs/api/models-decoders.rst
 create mode 100644 docs/api/models-diffusion.rst
 create mode 100644 docs/api/models-encoders.rst
 create mode 100644 docs/api/models-processors.rst

diff --git a/docs/api/models-common.rst b/docs/api/models-common.rst
new file mode 100644
index 00000000..39a60da4
--- /dev/null
+++ b/docs/api/models-common.rst
@@ -0,0 +1,107 @@
+Common Components
+=================
+
+The common components module provides building blocks and utilities used across different model architectures.
+
+Classes
+-------
+
+CommonConvBlock
+~~~~~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.models.common.conv_block_common.CommonConvBlock
+      :members:
+      :undoc-members:
+      :show-inheritance:
+
+ConvNormAct
+~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.models.common.conv_norm_act.ConvNormAct
+      :members:
+      :undoc-members:
+      :show-inheritance:
+
+ConvBlockDownsample
+~~~~~~~~~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.models.common.conv_block_downsample.ConvBlockDownsample
+      :members:
+      :undoc-members:
+      :show-inheritance:
+
+ConvBlockUpsample
+~~~~~~~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.models.common.conv_block_upsample.ConvBlockUpsample
+      :members:
+      :undoc-members:
+      :show-inheritance:
+
+ConvBlockUpsampleNaive
+~~~~~~~~~~~~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.models.common.conv_block_upsample_naive.ConvBlockUpsampleNaive
+      :members:
+      :undoc-members:
+      :show-inheritance:
+
+PatchEmbedding
+~~~~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.models.common.patchembed.PatchEmbedding
+      :members:
+      :undoc-members:
+      :show-inheritance:
+
+ResizingAveragePool2d
+~~~~~~~~~~~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.models.common.resizing_average_pool_2d.ResizingAveragePool2d
+      :members:
+      :undoc-members:
+      :show-inheritance:
+
+ResizingInterpolation
+~~~~~~~~~~~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.models.common.resizing_interpolation.ResizingInterpolation
+      :members:
+      :undoc-members:
+      :show-inheritance:
+
+TimeEmbed
+~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.models.common.time_embed.TimeEmbed
+      :members:
+      :undoc-members:
+      :show-inheritance:
+
+TransformerEncoderBlock
+~~~~~~~~~~~~~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.models.common.transformerblock.TransformerEncoderBlock
+      :members:
+      :undoc-members:
+      :show-inheritance:
diff --git a/docs/api/models-decoders.rst b/docs/api/models-decoders.rst
new file mode 100644
index 00000000..66f0a4c0
--- /dev/null
+++ b/docs/api/models-decoders.rst
@@ -0,0 +1,27 @@
+Decoders
+========
+
+The decoders module provides classes for decoding latent space representations back to output data.
+
+Classes
+-------
+
+BaseDecoder
+~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.models.decoders.base_decoder.BaseDecoder
+      :members:
+      :undoc-members:
+      :show-inheritance:
+
+CNNDecoder
+~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.models.decoders.cnn_decoder.CNNDecoder
+      :members:
+      :undoc-members:
+      :show-inheritance:
diff --git a/docs/api/models-diffusion.rst b/docs/api/models-diffusion.rst
new file mode 100644
index 00000000..d8405f90
--- /dev/null
+++ b/docs/api/models-diffusion.rst
@@ -0,0 +1,28 @@
+Diffusion Models
+================
+
+.. note::
+   Diffusion models provide the underlying denoising algorithms. To use them within the encode–process–decode pipeline, wrap them via :class:`DDPMProcessor` (see :doc:`Processors <models-processors>`).
+
+Classes
+-------
+
+GaussianDiffusion
+~~~~~~~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.models.diffusion.gaussian_diffusion.GaussianDiffusion
+      :members:
+      :undoc-members:
+      :show-inheritance:
+
+UNetDiffusion
+~~~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.models.diffusion.unet_diffusion.UNetDiffusion
+      :members:
+      :undoc-members:
+      :show-inheritance:
diff --git a/docs/api/models-encoders.rst b/docs/api/models-encoders.rst
new file mode 100644
index 00000000..918c2e28
--- /dev/null
+++ b/docs/api/models-encoders.rst
@@ -0,0 +1,27 @@
+Encoders
+========
+
+The encoders module provides classes for encoding input data into latent space representations.
+
+Classes
+-------
+
+BaseEncoder
+~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.models.encoders.base_encoder.BaseEncoder
+      :members:
+      :undoc-members:
+      :show-inheritance:
+
+CNNEncoder
+~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.models.encoders.cnn_encoder.CNNEncoder
+      :members:
+      :undoc-members:
+      :show-inheritance:
diff --git a/docs/api/models-processors.rst b/docs/api/models-processors.rst
new file mode 100644
index 00000000..c92a5945
--- /dev/null
+++ b/docs/api/models-processors.rst
@@ -0,0 +1,61 @@
+Processors
+==========
+
+The processors module provides classes for processing data in latent space within the encode-process-decode pipeline.
+
+Classes
+-------
+
+BaseProcessor
+~~~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.models.processors.base_processor.BaseProcessor
+      :members:
+      :undoc-members:
+      :show-inheritance:
+
+UNetProcessor
+~~~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.models.processors.unet.UNetProcessor
+      :members:
+      :undoc-members:
+      :show-inheritance:
+
+NullProcessor
+~~~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.models.processors.null.NullProcessor
+      :members:
+      :undoc-members:
+      :show-inheritance:
+
+VitProcessor
+~~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.models.processors.vit.VitProcessor
+      :members:
+      :undoc-members:
+      :show-inheritance:
+
+DDPMProcessor
+~~~~~~~~~~~~~
+
+.. container:: toggle
+
+   .. autoclass:: ice_station_zebra.models.processors.ddpm.DDPMProcessor
+      :members:
+      :undoc-members:
+      :show-inheritance:
+
+   .. note::
+      This processor wraps the diffusion models from the :doc:`diffusion models <models-diffusion>` section.
+      See :class:`GaussianDiffusion` and :class:`UNetDiffusion` for the underlying denoising algorithms.

From fe4964e52160658d437d40b1c4d7e841c898900d Mon Sep 17 00:00:00 2001
From: Sophie Arana <arana.sophie@gmail.com>
Date: Thu, 2 Oct 2025 17:17:39 +0100
Subject: [PATCH 8/8] pre-commit fixes

---
 docs/api/models.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/docs/api/models.rst b/docs/api/models.rst
index 6cd9dc11..5b550f2a 100644
--- a/docs/api/models.rst
+++ b/docs/api/models.rst
@@ -34,4 +34,4 @@ Persistence
    .. autoclass:: ice_station_zebra.models.persistence.Persistence
       :members:
       :undoc-members:
-      :show-inheritance:
\ No newline at end of file
+      :show-inheritance: