auto_compute_allocation enhancements - Adds accelerator hours, slurm attrs, slurm account and fairshare naming...

coldfront/config/plugins/auto_compute_allocation.py

@@ -9,6 +9,10 @@
     "coldfront.plugins.auto_compute_allocation",
 ]
 
+AUTO_COMPUTE_ALLOCATION_ACCELERATOR_HOURS = ENV.int("AUTO_COMPUTE_ALLOCATION_ACCELERATOR_HOURS", default=0)
+AUTO_COMPUTE_ALLOCATION_ACCELERATOR_HOURS_TRAINING = ENV.int(
+    "AUTO_COMPUTE_ALLOCATION_ACCELERATOR_HOURS_TRAINING", default=0
+)
 AUTO_COMPUTE_ALLOCATION_CORE_HOURS = ENV.int("AUTO_COMPUTE_ALLOCATION_CORE_HOURS", default=0)
 AUTO_COMPUTE_ALLOCATION_CORE_HOURS_TRAINING = ENV.int("AUTO_COMPUTE_ALLOCATION_CORE_HOURS_TRAINING", default=0)
 AUTO_COMPUTE_ALLOCATION_END_DELTA = ENV.int("AUTO_COMPUTE_ALLOCATION_END_DELTA", default=365)
@@ -18,3 +22,16 @@
 AUTO_COMPUTE_ALLOCATION_CLUSTERS = ENV.tuple("AUTO_COMPUTE_ALLOCATION_CLUSTERS", default=())
 # example auto|Cluster| results in auto|Cluster|CDF0001 where CDF0001 is an example project_code for pk=1
 AUTO_COMPUTE_ALLOCATION_DESCRIPTION = ENV.str("AUTO_COMPUTE_ALLOCATION_DESCRIPTION", default="auto|Cluster|")
+# Tuple for slurm_specs attribute - a tuple is required with each element a slurm attr e.g. AUTO_COMPUTE_ALLOCATION_SLURM_ATTR_TUPLE=('GrpTRESMins=cpu=999;mem=999;gres/gpu=999',)
+# NOTE: the semi-colon is required [as an internal delimiter] instead of comma, this gets converted to comma
+AUTO_COMPUTE_ALLOCATION_SLURM_ATTR_TUPLE = ENV.tuple("AUTO_COMPUTE_ALLOCATION_SLURM_ATTR_TUPLE", default=())
+# Tuple same format as above, see note
+AUTO_COMPUTE_ALLOCATION_SLURM_ATTR_TUPLE_TRAINING = ENV.tuple(
+    "AUTO_COMPUTE_ALLOCATION_SLURM_ATTR_TUPLE_TRAINING", default=()
+)
+AUTO_COMPUTE_ALLOCATION_FAIRSHARE_INSTITUTION_NAME_FORMAT = ENV.str(
+    "AUTO_COMPUTE_ALLOCATION_FAIRSHARE_INSTITUTION_NAME_FORMAT", default=""
+)
+AUTO_COMPUTE_ALLOCATION_SLURM_ACCOUNT_NAME_FORMAT = ENV.str(
+    "AUTO_COMPUTE_ALLOCATION_SLURM_ACCOUNT_NAME_FORMAT", default=""
+)

coldfront/plugins/auto_compute_allocation/README.md

@@ -24,11 +24,24 @@ Further down in the documentation, all variables are described in a table.
 
 As well as controlling the end date of the generated allocation, the changeable and locked attributes can be toggled.
 
-Optionally core hours can be assigned to new projects with ``AUTO_COMPUTE_ALLOCATION_CORE_HOURS`` and specifically training projects with ``AUTO_COMPUTE_ALLOCATION_CORE_HOURS_TRAINING``. These are activated by providing an integer greater than 0.
+Optionally **gauges** for accelerator and core hours can be assigned to new projects by providing an integer greater than 0.
+
+- ``AUTO_COMPUTE_ALLOCATION_ACCELERATOR_HOURS``
+- ``AUTO_COMPUTE_ALLOCATION_CORE_HOURS``
+
+...specifically for training (field of science = training) projects with:
+
+- ``AUTO_COMPUTE_ALLOCATION_ACCELERATOR_HOURS_TRAINING``
+- ``AUTO_COMPUTE_ALLOCATION_CORE_HOURS_TRAINING``
+
 
 A variable can be used to filter which Cluster resources the allocation can work with ``AUTO_COMPUTE_ALLOCATION_CLUSTERS``.
 
-Finally an optional usage, is to enable an institute based fairshare attribute. This requires the _institution feature_ has been enabled correctly, such that a match is found (for the submitting PI). If a match isn't found then this attribute can't be set and the code handles.
+An optional usage, is to enable an **institution based fairshare attribute** - ``AUTO_COMPUTE_ALLOCATION_FAIRSHARE_INSTITUTION``. This requires the _institution feature_ has been enabled correctly, such that a match is found (for the submitting PI). If a match isn't found then this attribute can't be set and the code handles.
+
+The **slurm account can be named** and its naming controlled via ``AUTO_COMPUTE_ALLOCATION_SLURM_ACCOUNT_NAME_FORMAT``. Similarly ``AUTO_COMPUTE_ALLOCATION_FAIRSHARE_INSTITUTION_NAME_FORMAT`` gives some control over the output of the **institutional fairshare naming/value**.
+
+**slurm_attributes can be added** with ``AUTO_COMPUTE_ALLOCATION_SLURM_ATTR_TUPLE`` and ``AUTO_COMPUTE_ALLOCATION_SLURM_ATTR_TUPLE_TRAINING``.
 
 
 ### Design - signals and actions
@@ -86,21 +99,59 @@ All variables for this plugin are currently **optional**.
 
 | Option | Type | Default | Description |
 |--- | --- | --- | --- |
-| `AUTO_COMPUTE_ALLOCATION_CORE_HOURS` | int | 0 | Optional, number of core hours to provide on the allocation, if 0 then this functionality is not triggered and no core hours will be added  |
-| `AUTO_COMPUTE_ALLOCATION_CORE_HOURS_TRAINING` | int | 0 | Optional, number of core hours to provide on the allocation, if 0 then this functionality is not triggered and no core hours will be added. This applies to projects which select 'Training' as their field of science discipline.  |
+| `AUTO_COMPUTE_ALLOCATION_ACCELERATOR_HOURS` | int | 0 | Optional, **not a slurm attribute, enables guage**, number of accelerator hours to provide on the allocation, if 0 then this functionality is not triggered and no accelerator hours will be added  |
+| `AUTO_COMPUTE_ALLOCATION_ACCELERATOR_HOURS_TRAINING` | int | 0 | Optional, **not a slurm attribute, enables guage**, number of accelerator hours to provide on the allocation, if 0 then this functionality is not triggered and no accelerator hours will be added. This applies to projects which select 'Training' as their field of science discipline.  |
+| `AUTO_COMPUTE_ALLOCATION_CORE_HOURS` | int | 0 | Optional, **not a slurm attribute, enables guage**, number of core hours to provide on the allocation, if 0 then this functionality is not triggered and no core hours will be added  |
+| `AUTO_COMPUTE_ALLOCATION_CORE_HOURS_TRAINING` | int | 0 | Optional, **not a slurm attribute, enables guage**, number of core hours to provide on the allocation, if 0 then this functionality is not triggered and no core hours will be added. This applies to projects which select 'Training' as their field of science discipline.  |
 | `AUTO_COMPUTE_ALLOCATION_END_DELTA` | int | 365 | Optional, number of days from creation of the allocation to expiry, default 365 to align with default project duration of 1 year  |
 | `AUTO_COMPUTE_ALLOCATION_CHANGEABLE` | bool | True | Optional, allows the allocation to have a request logged to change - this might be useful for an extension |
 | `AUTO_COMPUTE_ALLOCATION_LOCKED` | bool | False | Optional, prevents the allocation from being modified by admin - this might be useful for an extensions |
 | `AUTO_COMPUTE_ALLOCATION_FAIRSHARE_INSTITUTION` | bool | False | Optional, provides an institution based slurm fairshare attribute, requires that the _institution feature_ is setup correctly |
 | `AUTO_COMPUTE_ALLOCATION_CLUSTERS` | tuple | empty () | Optional, filter for clusters to automatically allocate on - example value ``AUTO_COMPUTE_ALLOCATION_CLUSTERS=(Cluster1,Cluster4)`` |
-| ``AUTO_COMPUTE_ALLOCATION_DESCRIPTION`` | str | "auto\|Cluster\|" | Optionally control the produced description for the allocation and its delimiters within. The _project_code_ will always be appended. Example resultant description: ``auto\|Cluster\|CDF0001`` |
+| `AUTO_COMPUTE_ALLOCATION_DESCRIPTION` | str | "auto\|Cluster\|" | Optionally control the produced description for the allocation and its delimiters within. The _project_code_ will always be appended. Example resultant description: ``auto\|Cluster\|CDF0001`` |
+| `AUTO_COMPUTE_ALLOCATION_SLURM_ATTR_TUPLE` | tuple | empty () | Optional, a tuple of slurm_attributes to add to the allocation. **Note each element needs an internal delimiter of a semi-colon `;` rather than a comma, if a comma is present in your intended element string**. <br><br>An example is `AUTO_COMPUTE_ALLOCATION_SLURM_ATTR_TUPLE=('GrpTRESMins=cpu=999;mem=999;gres/gpu=999',)` which defines a single element tuple and therefore 1x slurm attribute. More could be added. Note the internal semi-colon `;` delimiter instead of a comma with the string. |
+| `AUTO_COMPUTE_ALLOCATION_SLURM_ATTR_TUPLE_TRAINING` | tuple | empty () | Optional, a tuple of slurm_attributes to add to the allocation for a training project. **Note each element needs an internal delimiter of a semi-colon `;` rather than a comma, if a comma is present in your intended element string**. <br><br>An example is `AUTO_COMPUTE_ALLOCATION_SLURM_ATTR_TUPLE_TRAINING=('GrpTRESMins=cpu=999;mem=999;gres/gpu=999',)` which defines a single element tuple and therefore 1x slurm attribute. More could be added. Note the internal semi-colon `;` delimiter instead of a comma with the string. |
+| `AUTO_COMPUTE_ALLOCATION_FAIRSHARE_INSTITUTION_NAME_FORMAT` | str | empty "" | Optional, variable to define how the fairshare attribute will be named. <br><br>**If not defined then the default format `{institution}` will be used**.|
+| `AUTO_COMPUTE_ALLOCATION_SLURM_ACCOUNT_NAME_FORMAT` | str | empty "" | Optional, variable to define how the slurm_account_name attribute will be named. <br><br>**If not defined then the default format `{project_code}_{PI_First_Initial}_{PI_Last_Name_Formatted}_{allocation_id}` will be used**. <br><br> If you just want `project_code` then use `AUTO_COMPUTE_ALLOCATION_SLURM_ACCOUNT_NAME_FORMAT="{project_code}"`.| 
+
+
+#### AUTO_COMPUTE_ALLOCATION_SLURM_ACCOUNT_NAME_FORMAT - detail
+
+This table shows the possible values that can be used for the ENV var ``AUTO_COMPUTE_ALLOCATION_SLURM_ACCOUNT_NAME_FORMAT`` string.
+
+| AUTO_COMPUTE_ALLOCATION_SLURM_ACCOUNT_NAME_FORMAT | value | comment |
+|--- | --- | --- |
+| | `allocation_id` | the allocation id - useful as will be a distinct number (pk) - this will not necessarily be in sequence but is unique |
+| | `institution_abbr_upper_lower` | the institution's capital letters extracted and joined as lowercase - to make an abbreviation |
+| | `institution_abbr_upper_upper` | the institution's capital letters extracted and joined as uppercase - to make an abbreviation |
+| | `institution` | institution with spaces converted to '_' |
+| | `institution_formatted` | institution lowercase with spaces converted to '_' |
+| | `PI_first_initial` | PI last name initial lowercase|
+| | `PI_first_name` | PI first name lowercase |
+| | `PI_last_initial` | PI first initial lowercase |
+| | `PI_last_name_formatted` | PI last name lowercase with spaces converted to '_' |
+| | `PI_last_name` | PI last name lowercase |
+| | `project_code` | the project_code |
+| | `project_id` | the project_id (pk) wont be distinct within multiple allocations in the same project |
+
+
+#### AUTO_COMPUTE_ALLOCATION_FAIRSHARE_INSTITUTION_NAME_FORMAT - detail
+
+This table shows the possible values that can be used for the ENV var ``AUTO_COMPUTE_ALLOCATION_FAIRSHARE_INSTITUTION_NAME_FORMAT`` string.
+
+| AUTO_COMPUTE_ALLOCATION_SLURM_ACCOUNT_NAME_FORMAT | value | comment |
+|--- | --- | --- |
+| | `institution_abbr_upper_lower` | the institution's capital letters extracted and joined as lowercase - to make an abbreviation |
+| | `institution_abbr_upper_upper` | the institution's capital letters extracted and joined as uppercase - to make an abbreviation |
+| | `institution` | institution with spaces converted to '_' |
+| | `institution_formatted` | institution lowercase with spaces converted to '_' |
 
 
 
 ## Example settings
 
-- 10k core hours for new project
-- 100 core hours for a new training project
+- show a gauge for 10k core hours for new project
+- show a gauge for 100 core hours for a new training project
 - default 365 end delta - no need to set variable
 
 ```
@@ -113,5 +164,5 @@ AUTO_COMPUTE_ALLOCATION_CORE_HOURS_TRAINING=100
 
 Future work could include:
 
-- accelerator allocation
-- some more specific logic to map to partitions
+- slurm parent accounts
+- a seperate plugin for storage allocations - ``auto_storage_allocation``

coldfront/plugins/auto_compute_allocation/fairshare_institution_name.py

@@ -0,0 +1,41 @@
+# SPDX-FileCopyrightText: (C) ColdFront Authors
+#
+# SPDX-License-Identifier: AGPL-3.0-or-later
+
+"""Coldfront auto_compute_allocation plugin fairshare_institution_name.py"""
+
+import logging
+
+from coldfront.core.utils.common import import_from_settings
+
+AUTO_COMPUTE_ALLOCATION_FAIRSHARE_INSTITUTION_NAME_FORMAT = import_from_settings(
+    "AUTO_COMPUTE_ALLOCATION_FAIRSHARE_INSTITUTION_NAME_FORMAT"
+)
+
+logger = logging.getLogger(__name__)
+
+
+def generate_fairshare_institution_name(project_obj):
+    """Method to generate a fairshare_institution_name using predefined variables"""
+
+    # Get the uppercase characters from institution
+    institution_abbr_raw = "".join([c for c in project_obj.institution if c.isupper()])
+
+    FAIRSHARE_INSTITUTION_NAME_VARS = {
+        "institution_abbr_upper_lower": institution_abbr_raw.lower(),
+        "institution_abbr_upper_upper": institution_abbr_raw,
+        "institution": project_obj.institution.replace(" ", ""),
+        "institution_formatted": project_obj.institution.replace(" ", "_").lower(),
+    }
+
+    # if a faishare institution name format is defined as a string (use env var), else use format suggested
+    if AUTO_COMPUTE_ALLOCATION_FAIRSHARE_INSTITUTION_NAME_FORMAT:
+        gen_fairshare_institution_name = AUTO_COMPUTE_ALLOCATION_FAIRSHARE_INSTITUTION_NAME_FORMAT.format(
+            **FAIRSHARE_INSTITUTION_NAME_VARS
+        )
+    else:
+        gen_fairshare_institution_name = "{institution}".format(**FAIRSHARE_INSTITUTION_NAME_VARS)
+
+    logger.info(f"Generated fairshare institution name {gen_fairshare_institution_name}")
+
+    return gen_fairshare_institution_name

coldfront/plugins/auto_compute_allocation/slurm_account_name.py

@@ -0,0 +1,56 @@
+# SPDX-FileCopyrightText: (C) ColdFront Authors
+#
+# SPDX-License-Identifier: AGPL-3.0-or-later
+
+"""Coldfront auto_compute_allocation plugin slurm_account_name.py"""
+
+import logging
+
+from coldfront.core.utils.common import import_from_settings
+
+AUTO_COMPUTE_ALLOCATION_SLURM_ACCOUNT_NAME_FORMAT = import_from_settings(
+    "AUTO_COMPUTE_ALLOCATION_SLURM_ACCOUNT_NAME_FORMAT"
+)
+
+logger = logging.getLogger(__name__)
+
+
+def generate_slurm_account_name(allocation_obj, project_obj):
+    """Method to generate a slurm_account_name using predefined variables"""
+
+    # define valid vars for naming slurm account in dictionary
+    SLURM_ACCOUNT_NAME_VARS = {
+        "allocation_id": allocation_obj.pk,
+        "PI_first_initial": project_obj.pi.first_name[0].lower(),
+        "PI_first_name": project_obj.pi.first_name.lower(),
+        "PI_last_initial": project_obj.pi.last_name[0].lower(),
+        "PI_last_name_formatted": project_obj.pi.last_name.replace(" ", "_").lower(),
+        "PI_last_name": project_obj.pi.last_name.lower(),
+        "project_code": project_obj.project_code,
+        "project_id": project_obj.pk,
+    }
+
+    if hasattr(project_obj, "institution"):
+        # Get the uppercase characters from institution
+        institution_abbr_raw = "".join([c for c in project_obj.institution if c.isupper()])
+
+        SLURM_ACCOUNT_NAME_VARS.update(
+            {
+                "institution_abbr_upper_lower": institution_abbr_raw.lower(),
+                "institution_abbr_upper_upper": institution_abbr_raw,
+                "institution": project_obj.institution.replace(" ", "_"),
+                "institution_formatted": project_obj.institution.replace(" ", "_").lower(),
+            }
+        )
+
+    # if a slurm account name format is defined as a string (use env var), else use format suggested
+    if AUTO_COMPUTE_ALLOCATION_SLURM_ACCOUNT_NAME_FORMAT:
+        gen_slurm_account_name = AUTO_COMPUTE_ALLOCATION_SLURM_ACCOUNT_NAME_FORMAT.format(**SLURM_ACCOUNT_NAME_VARS)
+    else:
+        gen_slurm_account_name = "{project_code}_{PI_first_initial}_{PI_last_name_formatted}_{allocation_id}".format(
+            **SLURM_ACCOUNT_NAME_VARS
+        )
+
+    logger.info(f"Generated slurm account name {gen_slurm_account_name}")
+
+    return gen_slurm_account_name