Skip to content

Commit 53be508

Browse files
authored
feat: partition time handling (#378)
Even though, I do not expect admin folks to enter weird numerical values in the partition configs, the parsing should support the SLURM standard formats for time indicators. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Accept flexible time formats for max_runtime (numeric, unit-suffixed, and SLURM-style) with automatic conversion and rounding to minutes when loading partitions. * Invalid or unrecognized time formats now produce clear validation errors. * **Documentation** * Expanded configuration guide with examples and a clarifying note about accepted time formats and conversion behavior. * **Tests** * Added comprehensive tests covering formats, rounding, invalid inputs, and partition normalization. <sub>✏️ Tip: You can customize this high-level summary in your review settings.</sub> <!-- end of auto-generated comment: release notes by coderabbit.ai -->
1 parent b45709f commit 53be508

File tree

4 files changed

+520
-2
lines changed

4 files changed

+520
-2
lines changed

docs/further.md

Lines changed: 15 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -87,14 +87,14 @@ The following limits can be defined for each partition:
8787

8888
| Parameter | Type | Description | Default |
8989
| ----------------------- | --------- | ---------------------------------- | --------- |
90-
| `max_runtime` | int | Maximum walltime in minutes | unlimited |
90+
| `max_runtime` | int/str | Maximum walltime | unlimited |
9191
| `max_mem_mb` | int | Maximum total memory in MB | unlimited |
9292
| `max_mem_mb_per_cpu` | int | Maximum memory per CPU in MB | unlimited |
9393
| `max_cpus_per_task` | int | Maximum CPUs per task | unlimited |
9494
| `max_nodes` | int | Maximum number of nodes | unlimited |
9595
| `max_tasks` | int | Maximum number of tasks | unlimited |
9696
| `max_tasks_per_node` | int | Maximum tasks per node | unlimited |
97-
| `max_threads` | int | Maximum threads per node | unlimited |
97+
| `max_threads` | int | Maximum threads per node | unlimited |
9898
| `max_gpu` | int | Maximum number of GPUs | 0 |
9999
| `available_gpu_models` | list[str] | List of available GPU models | none |
100100
| `max_cpus_per_gpu` | int | Maximum CPUs per GPU | unlimited |
@@ -103,6 +103,19 @@ The following limits can be defined for each partition:
103103
| `available_constraints` | list[str] | List of available node constraints | none |
104104
| `cluster` | str | Cluster name in multi-cluster setup | none |
105105

106+
Note: the `max_runtime` definition may contain
107+
- Numeric values (assumed to be in minutes): 120, 120.5
108+
- Snakemake-style time strings: "6d", "12h", "30m", "90s", "2d12h30m"
109+
- SLURM time formats:
110+
- "minutes" (e.g., "60")
111+
- "minutes:seconds" (interpreted as hours:minutes, e.g., "60:30")
112+
- "hours:minutes:seconds" (e.g., "1:30:45")
113+
- "days-hours" (e.g., "2-12")
114+
- "days-hours:minutes" (e.g., "2-12:30")
115+
- "days-hours:minutes:seconds" (e.g., "2-12:30:45")
116+
117+
They are all auto-converted to minutes. Seconds are rounded to the nearest value in minutes.
118+
106119
##### Example Partition Configuration
107120

108121
```yaml

snakemake_executor_plugin_slurm/partitions.py

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88
JobExecutorInterface,
99
)
1010
from snakemake_interface_executor_plugins.logging import LoggerExecutorInterface
11+
from .utils import parse_time_to_minutes
1112

1213

1314
def read_partition_file(partition_file: Path) -> List["Partition"]:
@@ -222,6 +223,15 @@ class PartitionLimits:
222223
# Node features/constraints
223224
available_constraints: Optional[List[str]] = None
224225

226+
def __post_init__(self):
227+
"""Convert max_runtime to minutes if specified as a time string"""
228+
# Check if max_runtime is a string or needs conversion
229+
# isinf() only works on numeric types, so check type first
230+
if isinstance(self.max_runtime, str) or (
231+
isinstance(self.max_runtime, (int, float)) and not isinf(self.max_runtime)
232+
):
233+
self.max_runtime = parse_time_to_minutes(self.max_runtime)
234+
225235

226236
@dataclass
227237
class Partition:

snakemake_executor_plugin_slurm/utils.py

Lines changed: 121 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,136 @@
11
# utility functions for the SLURM executor plugin
22

3+
import math
34
import os
45
import re
56
from pathlib import Path
7+
from typing import Union
68

79
from snakemake_interface_executor_plugins.jobs import (
810
JobExecutorInterface,
911
)
1012
from snakemake_interface_common.exceptions import WorkflowError
1113

1214

15+
def round_half_up(n):
16+
return int(math.floor(n + 0.5))
17+
18+
19+
def parse_time_to_minutes(time_value: Union[str, int, float]) -> int:
20+
"""
21+
Convert a time specification to minutes (integer). This function
22+
is intended to handle the partition definitions for the max_runtime
23+
value in a partition config file.
24+
25+
Supports:
26+
- Numeric values (assumed to be in minutes): 120, 120.5
27+
- Snakemake-style time strings: "6d", "12h", "30m", "90s", "2d12h30m"
28+
- SLURM time formats:
29+
- "minutes" (e.g., "60")
30+
- "minutes:seconds" (interpreted as hours:minutes, e.g., "60:30")
31+
- "hours:minutes:seconds" (e.g., "1:30:45")
32+
- "days-hours" (e.g., "2-12")
33+
- "days-hours:minutes" (e.g., "2-12:30")
34+
- "days-hours:minutes:seconds" (e.g., "2-12:30:45")
35+
36+
Args:
37+
time_value: Time specification as string, int, or float
38+
39+
Returns:
40+
Time in minutes as integer (fractional minutes are rounded)
41+
42+
Raises:
43+
WorkflowError: If the time format is invalid
44+
"""
45+
# If already numeric, return as integer minutes (rounded)
46+
if isinstance(time_value, (int, float)):
47+
return round_half_up(time_value) # implicit conversion to int
48+
49+
# Convert to string and strip whitespace
50+
time_str = str(time_value).strip()
51+
52+
# Try to parse as plain number first
53+
try:
54+
return round_half_up(float(time_str)) # implicit conversion to int
55+
except ValueError:
56+
pass
57+
58+
# Try SLURM time formats first (with colons and dashes)
59+
# Format: days-hours:minutes:seconds or variations
60+
if "-" in time_str or ":" in time_str:
61+
try:
62+
days = 0
63+
hours = 0
64+
minutes = 0
65+
seconds = 0
66+
67+
# Split by dash first (days separator)
68+
if "-" in time_str:
69+
parts = time_str.split("-")
70+
if len(parts) != 2:
71+
raise ValueError("Invalid format with dash")
72+
days = int(parts[0])
73+
time_str = parts[1]
74+
75+
# Split by colon (time separator)
76+
time_parts = time_str.split(":")
77+
78+
if len(time_parts) == 1:
79+
# Just hours (after dash) or just minutes
80+
if days > 0:
81+
hours = int(time_parts[0])
82+
else:
83+
minutes = int(time_parts[0])
84+
elif len(time_parts) == 2:
85+
# was: days-hours:minutes
86+
hours = int(time_parts[0])
87+
minutes = int(time_parts[1])
88+
elif len(time_parts) == 3:
89+
# was: hours:minutes:seconds
90+
hours = int(time_parts[0])
91+
minutes = int(time_parts[1])
92+
seconds = int(time_parts[2])
93+
else:
94+
raise ValueError("Too many colons in time format")
95+
96+
# Convert everything to minutes
97+
total_minutes = days * 24 * 60 + hours * 60 + minutes + seconds / 60.0
98+
return round_half_up(total_minutes) # implicit conversion to int
99+
100+
except (ValueError, IndexError):
101+
# If SLURM format parsing fails, try Snakemake style below
102+
pass
103+
104+
# Parse Snakemake-style time strings (e.g., "6d", "12h", "30m", "90s", "2d12h30m")
105+
# Pattern matches: optional number followed by unit (d, h, m, s)
106+
pattern = r"(\d+(?:\.\d+)?)\s*([dhms])"
107+
matches = re.findall(pattern, time_str.lower())
108+
109+
if not matches:
110+
raise WorkflowError(
111+
f"Invalid time format: '{time_value}'. "
112+
f"Expected formats:\n"
113+
f" - Numeric value in minutes: 120\n"
114+
f" - Snakemake style: '6d', '12h', '30m', '90s', '2d12h30m'\n"
115+
f" - SLURM style: 'minutes', 'minutes:seconds', 'hours:minutes:seconds',\n"
116+
f" 'days-hours', 'days-hours:minutes', 'days-hours:minutes:seconds'"
117+
)
118+
119+
total_minutes = 0.0
120+
for value, unit in matches:
121+
num = float(value)
122+
if unit == "d":
123+
total_minutes += num * 24 * 60
124+
elif unit == "h":
125+
total_minutes += num * 60
126+
elif unit == "m":
127+
total_minutes += num
128+
elif unit == "s":
129+
total_minutes += num / 60
130+
131+
return round_half_up(total_minutes)
132+
133+
13134
def delete_slurm_environment():
14135
"""
15136
Function to delete all environment variables

0 commit comments

Comments
 (0)