Skip to content

Commit 74203ec

Browse files
Add CLI demo & install section to README
1 parent 3c14a50 commit 74203ec

File tree

3 files changed

+23
-3
lines changed

3 files changed

+23
-3
lines changed

README.md

+23-3
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,20 @@
22
The slurm pipeline facilitates the scheduling of [slurm](https://slurm.schedmd.com/overview.html) jobs. It allows to sequentially schedule multiple jobs and supports logging, retrying, dynamic allocation of computing resources, and intelligent parallization via slurm job arrays.
33

44

5+
## Install
6+
Install from local wheel file:
7+
```
8+
pip install dist/slurm_pipeline-*.whl
9+
```
10+
11+
512
## CLI
613
A CLI allows to control the slurm pipeline, such as starting & aborting jobs, inspecting logs, and more. See [CLI docs](#cli-docs) section for more details.
714

15+
<!-- agg --speed=1.5 --cols=100 --rows=25 --font-size=24 docs/slurm-pipeline-demo.cast docs/slurm-pipeline-demo.gif -->
16+
![](./docs/slurm-pipeline-demo.gif)
17+
18+
819
**Commands**:
920

1021
* `abort`: Stops scheduled slurm jobs.
@@ -24,7 +35,7 @@ A CLI allows to control the slurm pipeline, such as starting & aborting jobs, in
2435
The config file is the heart of the slurm-pipeline. Here, all jobs and properties are specified. The job order in the config determines the sequential scheduling order.
2536

2637
The minimal config looks like:
27-
```
38+
```yaml
2839
jobs:
2940
- name: <some-name>
3041
script: </path-to/script-name.py>
@@ -40,11 +51,11 @@ properties:
4051
conda_env: </path-to/.conda/envs/env-name>
4152
account: <slurm-account>
4253
```
43-
A more extensive template can be found at `template-config.yml`. For an exact definition of the schema and all available properties, please see `slurm_pipeline/config.py`.
54+
A more extensive template can be found at [`docs/template-config.yml`](./docs/template-config.yml). For an exact definition of the schema and all available properties, please see [`slurm_pipeline/config.py`](./slurm_pipeline/config.py).
4455

4556
## Jobs
4657
Each job is based on an executable Python script. The path to the `script` must be specified in the slurm config. Additionally, a JSON, YAML, or CSV formatted `param_file` can be provided which includes a list of arguments that will be passed one by one via stdin to the Python script. For example, the following `param_file` will trigger two runs of the job script with different parameterizations:
47-
```json
58+
```yaml
4859
- param_1: some-value
4960
param_2: some-value
5061
@@ -66,6 +77,15 @@ if __name__ == '__main__':
6677
```
6778

6879

80+
## Development
81+
82+
Build from source:
83+
```
84+
poetry build
85+
pip install dist/slurm_pipeline-*.whl --force-reinstall --no-deps
86+
```
87+
88+
6989
## CLI docs
7090

7191

docs/slurm-pipeline-demo.gif

1.27 MB
Loading
File renamed without changes.

0 commit comments

Comments
 (0)