souporcell_pipeline.py: ensure that logs subdirectory exists#278
Open
carandraug wants to merge 1 commit into
Open
souporcell_pipeline.py: ensure that logs subdirectory exists#278carandraug wants to merge 1 commit into
carandraug wants to merge 1 commit into
Conversation
If out_dir already exists it does not mean that it is restarting a pipeline. Even if it does, it does not mean that a logs subdirectory was successfully created on the previous run. This commit ensures that whether the logs directory is created is based on whether that directory exists and not whether its parent directory exists.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
souporcell looks at whether
out_diralready exists to identify if it is restarting a pipeline. This is then used to know if it needs to create thelogssubdirectory.Why is this an issue for me? I'm using
mktemp -dto create separate directories for parallelsouporcell_pipeline.pyprocesses. So I'm passing an existing, but empy,out_dirto souporcell. Because souporcell uses whetherout_direxists to identify if it is resuming a previous run, it then skips the creation of thelogsdirectory, leading to the following error later on:Actually, the whole condition is unnecessary since
os.makedirs(... exist_ok=True)will already do the check and only create the directories if needed.