-
Notifications
You must be signed in to change notification settings - Fork 15
Project directory structure under Shared allocation in Cheaha #920
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
14 commits
Select commit
Hold shift + click to select a range
8cf1aca
first draft on project directory structure
bdu-birhanu 55910ca
typo and formatting
bdu-birhanu 2c44d29
Merge branch 'main' into feat-proj-org
bdu-birhanu 68a6b6f
fix typos
bdu-birhanu 71bbb86
Merge branch 'main' into feat-proj-org
bdu-birhanu d0aabe8
fix typos on mkdocs
bdu-birhanu d610157
fix typos with lts in mkdocs
bdu-birhanu 7dcc04d
adding the missed redirect (storage to index page in mkdocs
bdu-birhanu 032cd88
adding the missed redirect (storage to index page in mkdocs
bdu-birhanu e9d7ee7
Merge branch 'main' into feat-proj-org
bdu-birhanu b7a8df4
moved project_directory_organization under gpfs tab
bdu-birhanu 1a35e0d
Merge branch 'main' into feat-proj-org
bdu-birhanu 06b9afb
adding overall directory structure
bdu-birhanu 89da66f
Merge branch 'main' into feat-proj-org
bdu-birhanu File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
84 changes: 84 additions & 0 deletions
84
docs/data_management/cheaha_storage_gpfs/project_directory_organization.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,84 @@ | ||
| # Project Directory Organization | ||
|
|
||
| Effective research data management starts with a well organized directory structure, especially for researchers working on multiple projects under shared allocations. A well organized project directory structure ensures scalability, simplifies collaboration, and makes it easier to manage permissions. | ||
|
|
||
| To maintain long-term organization, create a new subdirectory each time your group begins a distinct project. Assign [permissions](#permissions-and-access-control) based on access needs to ensure security while allowing collaboration. | ||
|
|
||
| Below is a recommended directory structure to help keep project files, shared research data, and software environments organized within shared allocations on Cheaha. | ||
|
|
||
| 1. Top-Level Directory | ||
|
|
||
| Suppose the top-level directory is your lab Cheaha project directory, `/data/project/mylab/`. | ||
|
|
||
| 1. Projects' Directory | ||
|
|
||
| Each project gets its own subdirectory. This keeps things organized and isolated from one another, making it easier to manage and avoid confusion. | ||
|
|
||
| Example: | ||
|
|
||
| ```bash | ||
| /data/project/mylab/project/ | ||
| |-- project-1/ | ||
| |-- project-2/ | ||
| |-- project-3/ | ||
| ``` | ||
|
|
||
| 1. Repositories | ||
|
|
||
| If your projects rely on code or software tools that are used across multiple efforts, you can set up a repository directory for shared scripts, tools, or utilities. | ||
|
|
||
| Example: | ||
|
|
||
| ```bash | ||
| /data/project/mylab/repository/ | ||
| |-- common-software-tool-1/ | ||
| |-- common-software-tool-2/ | ||
| ``` | ||
|
|
||
| 1. Shared Research Data | ||
|
|
||
| Having a separate directory for research data that is common across projects helps prevent duplication and ensures that everyone is working with the same dataset. | ||
|
|
||
| Example: | ||
|
|
||
| ```bash | ||
| /data/project/mylab/common-data/ | ||
| |-- some-data-used-by-multiple-projects/ | ||
| |-- other-shared-data/ | ||
| ``` | ||
|
|
||
| 1. Environments | ||
|
|
||
| For projects that require specific software environments (e.g., Conda environments), you can centralize these in a separate directory. This way, you ensure that everyone is using the right tools. | ||
|
|
||
| Example: | ||
|
|
||
| ```bash | ||
| /data/project/mylab/environments/ | ||
| |-- shared-conda-env-1.yml | ||
| |-- shared-conda-env-2.yml | ||
| ``` | ||
|
|
||
| To provide a clearer picture, here is how all the components fit together in the overall structure: | ||
|
|
||
| ```bash | ||
| /data/project/mylab/ | ||
| |-- project/ | ||
| | |-- project-1/ | ||
| | |-- ... | ||
| |-- repository/ | ||
| | |-- common-software-tool-1/ | ||
| | |-- ... | ||
| |-- common-data/ | ||
| | |-- some-data-used-by-multiple-projects/ | ||
| | |-- ... | ||
| |-- environments/ | ||
| | |-- shared-conda-env-1.yml | ||
| | |-- ... | ||
| ``` | ||
|
|
||
| ## Permissions and Access Control | ||
|
|
||
| Permissions for each directory can be managed by either being a member of the group associated with the top-level project directory or by using [Linux permissions](../cheaha_storage_gpfs/index.md#project-directory-permissions) or [Access Control Lists (ACLs)](../../workflow_solutions/shell.md#manage-researcher-access-to-files-and-directories-getfacl-setfacl). Managing proper permissions ensures that team members can access only the resources they need, while protecting sensitive data. | ||
|
|
||
| If you plan managing multiple projects and would like assistance with organizing your project directories or managing permissions, please send us a [support ticket](../../help/support.md/#how-do-i-create-a-support-ticket). | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.