Conversation
- each user gets /tmp/of3-of-$USERNAME - respects TMPDIR and other env vars - all the colabfold and jackhammer code is moved int - tests - fixes #150
jnwei
left a comment
There was a problem hiding this comment.
This is good, thank you for updating the tmp directory throughout the repo.
Two comments:
-
For the jackhmmer workflow, I am not sure if it is preferable to have the results moved to a user subdirectory. If a user is running a large number of alignments with jackhmmer, chances are that they would want to use the alignments with other users / roles, and may prefer to just change the permissions manually for the directory. @vinay-swamy do you have thoughts about this?
-
Could you also update the full configuration reference doc?
I think it should just bemsa_computation_settings
|
Good shout on the docs – for the jackhammer, it seems to me like it was something super custom to run in a setup where you had /tmp in RAM, so indeed might not be appropriate to store that under. it's also not impacting the original test problem. |
jnwei
left a comment
There was a problem hiding this comment.
LGTM. I would suggest that we leave the tmp pipeline for jackhmmer alone for now, since that is a fairly different pipeline. We can change the tmp dir later if needed.
|
Agreed on jackhammer |
|
tbh I think its better to deprecate the old python based jackhmmer workflow - It would be better to instead point users to the snakemake workflow as that will generally be faster than the previous method |
let's mark as deprecated for Q2 in another PR, and not mess around with it in this PR – thanks @vinay-swamy |
Summary
Fix shared
/tmpdirectories that causePermissionErroron multi-user machines.Introduces a centralized
get_of3_tmpdir()utility that namespaces temp dirs per user, following the same convention as pytest (/tmp/pytest-of-alice/→/tmp/of3-of-alice/).Changes
get_of3_tmpdir(subdir)inopenfold3/core/data/tools/utils.py— single source of truth for OF3 temp directories/tmp/of3-of-$USERNAME/<subdir>/— no more cross-user permission conflicts$TMPDIR,$TEMP,$TMPand other platform conventions viatempfile.gettempdir()colabfold_msa_server.py,template.py, andjackhmmer.pyoff hardcoded/tmppathstest_utils.pycovering path structure, isolation, idempotency, and$TMPDIRsupportRelated Issues
Testing
pytest openfold3/tests/core/data/tools/test_utils.py
Other Notes