You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardexpand all lines: README.md
+23-3
Original file line number
Diff line number
Diff line change
@@ -2,9 +2,20 @@
2
2
The slurm pipeline facilitates the scheduling of [slurm](https://slurm.schedmd.com/overview.html) jobs. It allows to sequentially schedule multiple jobs and supports logging, retrying, dynamic allocation of computing resources, and intelligent parallization via slurm job arrays.
3
3
4
4
5
+
## Install
6
+
Install from local wheel file:
7
+
```
8
+
pip install dist/slurm_pipeline-*.whl
9
+
```
10
+
11
+
5
12
## CLI
6
13
A CLI allows to control the slurm pipeline, such as starting & aborting jobs, inspecting logs, and more. See [CLI docs](#cli-docs) section for more details.
@@ -24,7 +35,7 @@ A CLI allows to control the slurm pipeline, such as starting & aborting jobs, in
24
35
The config file is the heart of the slurm-pipeline. Here, all jobs and properties are specified. The job order in the config determines the sequential scheduling order.
25
36
26
37
The minimal config looks like:
27
-
```
38
+
```yaml
28
39
jobs:
29
40
- name: <some-name>
30
41
script: </path-to/script-name.py>
@@ -40,11 +51,11 @@ properties:
40
51
conda_env: </path-to/.conda/envs/env-name>
41
52
account: <slurm-account>
42
53
```
43
-
A more extensive template can be found at `template-config.yml`. For an exact definition of the schema and all available properties, please see `slurm_pipeline/config.py`.
54
+
A more extensive template can be found at [`docs/template-config.yml`](./docs/template-config.yml). For an exact definition of the schema and all available properties, please see [`slurm_pipeline/config.py`](./slurm_pipeline/config.py).
44
55
45
56
## Jobs
46
57
Each job is based on an executable Python script. The path to the `script` must be specified in the slurm config. Additionally, a JSON, YAML, or CSV formatted `param_file` can be provided which includes a list of arguments that will be passed one by one via stdin to the Python script. For example, the following `param_file` will trigger two runs of the job script with different parameterizations:
0 commit comments