Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
188 commits
Select commit Hold shift + click to select a range
8efa779
Update README.md
heylf Mar 14, 2024
0fe74b5
merging from master
heylf Nov 7, 2024
c8afca3
Template update for nf-core/tools version 3.1.0
nf-core-bot Dec 12, 2024
c3aa28c
Template update for nf-core/tools version 3.1.1
nf-core-bot Dec 20, 2024
99585a3
Template update for nf-core/tools version 3.1.2
nf-core-bot Jan 20, 2025
3d824d5
Template update for nf-core/tools version 3.2.0
nf-core-bot Jan 27, 2025
0f5b5a4
solving merge conflcit
heylf Jan 31, 2025
5ddf503
Merge pull request #56 from nf-core/master
heylf Jan 31, 2025
a8c9395
Merge branch 'dev' into nf-core-template-merge-3.2.0
heylf Jan 31, 2025
7530e12
Merge pull request #50 from nf-core/nf-core-template-merge-3.2.0
heylf Jan 31, 2025
bda1441
removing fastqc
heylf Feb 3, 2025
b1fd0cb
added module baysor with changes to 'modules.config' for handling con…
khersameesh24 Feb 9, 2025
be8df7a
Merge pull request #3 from nf-core/add-local-module-baysor
khersameesh24 Feb 9, 2025
f8e13d8
Merge pull request #4 from khersameesh24/add-local-module-baysor
khersameesh24 Feb 9, 2025
bf99566
Revert "Add local module baysor"
khersameesh24 Feb 9, 2025
5e768fa
Merge pull request #5 from khersameesh24/revert-4-add-local-module-ba…
khersameesh24 Feb 9, 2025
fe12aa1
Merge branch 'nf-core:dev' into dev
khersameesh24 Feb 9, 2025
b190326
addding format paremeter to nextflow.config
heylf Feb 19, 2025
f33393d
changing schema_input.json
heylf Feb 19, 2025
bbf85fd
changing spatialxe.nf and adding spatialdata module
heylf Feb 19, 2025
d8c42f2
adding spatialdatawrite.py scripts
heylf Feb 19, 2025
252390b
adding meta.yml for spatialdata module
heylf Feb 19, 2025
f415f63
adding main.nf for spatialdata module
heylf Feb 19, 2025
8d63d53
adding gunzip to modules.json
heylf Mar 13, 2025
4f85600
adding parameters
heylf Mar 13, 2025
d7b7ffd
adding image field to schema_input.json
heylf Mar 13, 2025
03f8d3d
adding stuff to modules.config
heylf Mar 13, 2025
091b564
adding baysor
heylf Mar 13, 2025
b2e7159
adding ficture
heylf Mar 13, 2025
2286dd1
changes to proseq main.nf and proseg preprocess main.nf
heylf Mar 13, 2025
f1b2e82
first version of utils_nf_core_spatialxe_pipeline
heylf Mar 13, 2025
f6f8cb4
changes to spatialdata module main.nf, meta.yml and template.py
heylf Mar 13, 2025
f38f4f6
adding gunzip module
heylf Mar 13, 2025
863495b
changes to cellpose main.nf and meta.yml
heylf Mar 13, 2025
2c17f77
changes to xeniumranger import-segementation
heylf Mar 13, 2025
1ebd7a4
add segger dockerfile
heylf Mar 16, 2025
4a3ecc8
add segger
heylf Mar 16, 2025
7f454bf
increase singularity pullTimeout because segger container is large
heylf Mar 16, 2025
3a44622
add segger predict to modules.config
heylf Mar 16, 2025
4a514f5
first version of spatialxe.nf
heylf Mar 16, 2025
527bc17
add resolift module with test
kubranarci Mar 25, 2025
ee55bbc
add test dataset from zip
kubranarci Mar 25, 2025
dc52894
Merge branch 'dev' into add_resolift_module
kubranarci Mar 25, 2025
d940d74
Update main.nf
khersameesh24 Mar 25, 2025
c5cec8e
Update .gitignore
khersameesh24 Mar 25, 2025
4e5b52c
Merge pull request #62 from kubranarci/add_resolift_module
khersameesh24 Mar 25, 2025
ee5685c
updating docker file for segger
heylf Mar 27, 2025
4b75627
adding spatialdata mpodules and conversions to modules.config
heylf Mar 27, 2025
6ed4d52
changes to segger predict main.nf
heylf Mar 27, 2025
b3d070a
changes to segger train main.nf
heylf Mar 27, 2025
e75e744
adding spatialconverter module
heylf Mar 27, 2025
0d20b02
bump xenium ranger version
heylf Mar 27, 2025
817b088
adding spatialdata docker file because of numba cache issue on cluster
heylf Mar 28, 2025
d76a0ea
adding spatialdata merge module
heylf Mar 28, 2025
7dde082
adding spatialdata meta module
heylf Mar 28, 2025
f738534
updating spatialdata write module
heylf Mar 28, 2025
4a72341
updating xeniumranger import-segementation module
heylf Mar 28, 2025
b8da198
upating spatialxe.nf
heylf Mar 28, 2025
25851be
Merge branch 'nf-core:dev' into dev
heylf Mar 28, 2025
8dbc0f1
use ficutre nf.core docker container
heylf Mar 31, 2025
d8df60f
Merge branch 'dev' of https://github.com/heylf/spatialxe into dev
heylf Mar 31, 2025
a198bc4
bump proseg docker container version
heylf Mar 31, 2025
a58a28e
remove torch additona debug info from segger traing main.nf
heylf Mar 31, 2025
b9434ef
add TODOs to modules.config
heylf Mar 31, 2025
f650c37
Merge pull request #67 from heylf/dev
khersameesh24 Mar 31, 2025
4eb67eb
Merge branch 'nf-core:dev' into dev
khersameesh24 Mar 31, 2025
283f1ca
added subworkflow for segger
khersameesh24 Mar 31, 2025
0ec763c
handled some TODOs from the updated dev branch, minor enhancements to…
khersameesh24 Apr 1, 2025
998cb9a
minor fix to catch ENV vars
khersameesh24 Apr 1, 2025
09eaf3c
path fix for segger modules
khersameesh24 Apr 1, 2025
d03925c
ädded pytorch install
khersameesh24 Apr 1, 2025
2ffa540
changes to PR as suggested
khersameesh24 Apr 2, 2025
14b6788
minor linting fixes
khersameesh24 Apr 3, 2025
413a64d
Merge pull request #68 from khersameesh24/add-subworkflow-segger
heylf Apr 3, 2025
fcdd6cf
minor changes to module proseg, added subworkflow and fixed directory…
khersameesh24 Apr 3, 2025
1a6f5c8
add proseg subworkflows to spatialxe.nf
khersameesh24 Apr 4, 2025
58f48dd
path fix
khersameesh24 Apr 4, 2025
8c7b5ae
changes to subworkflow name
khersameesh24 Apr 5, 2025
c56fca8
changes to subworkflow name
khersameesh24 Apr 5, 2025
e0e9ca0
added subworkflow for ficture, minor changes to ficture modules
khersameesh24 Apr 6, 2025
5e4598b
minor fixe to handle ENV variables in segger
khersameesh24 Apr 6, 2025
c72dffc
minor fix to use accelerator
khersameesh24 Apr 6, 2025
2239e64
minor
khersameesh24 Apr 6, 2025
fa674c5
revert config
khersameesh24 Apr 6, 2025
5fb0db0
minor changes to proseg dir structure and subworkflow
khersameesh24 Apr 7, 2025
36b735d
Merge pull request #70 from khersameesh24/add-subworkflow-proseg
heylf Apr 9, 2025
48fc031
Merge branch 'dev' into add-subworkflow-ficture
khersameesh24 Apr 10, 2025
e204c1c
Merge pull request #72 from khersameesh24/add-subworkflow-ficture
heylf Apr 14, 2025
bce9cbe
added baysor subworkflow, minor changes to config, resolift
khersameesh24 Apr 14, 2025
fc91a3b
filename change
khersameesh24 Apr 15, 2025
6e04f0f
Merge pull request #76 from khersameesh24/add-subworkflow-baysor
heylf Apr 15, 2025
1561182
added subworkflows to handle xr scenarios
khersameesh24 Apr 28, 2025
2db936c
minor changes
khersameesh24 Apr 28, 2025
817480e
added config options for pipeline run scenarios
khersameesh24 Apr 28, 2025
a515150
param for image alignment file
khersameesh24 Apr 28, 2025
72a7073
added option to generate cell flows for boms method
khersameesh24 Apr 28, 2025
eec9ed1
Merge pull request #79 from khersameesh24/add-subworkflow-xeniumranger
heylf Apr 28, 2025
bed3709
Merge pull request #80 from khersameesh24/add-subworkflow-cellpose
heylf Apr 28, 2025
e6e2d70
mics-subworkflow-module-changes
khersameesh24 Apr 28, 2025
847bee3
revert cellpose subworkflow
khersameesh24 Apr 28, 2025
85418e7
minor
khersameesh24 Apr 28, 2025
c95ab4a
spatialxe workflow layers and other minor changes
khersameesh24 Apr 29, 2025
aa2e6b7
removed gunzip
khersameesh24 Apr 29, 2025
fb5f0cd
Merge pull request #81 from khersameesh24/add-subworkflow-cellpose
heylf Apr 29, 2025
3f2ab9e
added subworklfow for cellpose->bayor->imp-seg, nextflow schema
khersameesh24 Apr 30, 2025
a19870e
Template update for nf-core/tools version 3.2.1
nf-core-bot Apr 30, 2025
b82a77e
Merge pull request #83 from khersameesh24/add-subworkflow-cellpose
heylf Apr 30, 2025
71129ae
schema, config fix, minor changes to spatialxe workflow
khersameesh24 May 3, 2025
64b08b1
major linting fixes and moving to nf-core standards
khersameesh24 May 3, 2025
782a343
Merge branch 'dev' into nf-core-template-merge-3.2.1
khersameesh24 May 3, 2025
5af3689
pre-commit fix
khersameesh24 May 3, 2025
83e3dbc
revert schema draft version
khersameesh24 May 3, 2025
756228f
test config update
khersameesh24 May 4, 2025
69744d5
more linting/warning fixes, generated module patches for cellpose, xr
khersameesh24 May 4, 2025
c75970a
Delete modules/nf-core/custom/dumpsoftwareversions/main.nf
khersameesh24 May 4, 2025
bc3f951
Delete modules/nf-core/custom/dumpsoftwareversions/meta.yml
khersameesh24 May 4, 2025
b56455e
Delete modules/nf-core/custom/dumpsoftwareversions/templates/dumpsoft…
khersameesh24 May 4, 2025
de1a38c
assets, template fixes
khersameesh24 May 4, 2025
0fd31ec
Merge branch 'add-subworkflow-cellpose' of https://github.com/khersam…
khersameesh24 May 4, 2025
ab914b6
update modules.json file to remove custom dumpspftware
khersameesh24 May 4, 2025
152e3e1
moving towards full test profile run
khersameesh24 May 4, 2025
cd58ace
manual testing changes
khersameesh24 May 5, 2025
527d622
Merge pull request #85 from khersameesh24/add-subworkflow-cellpose
heylf May 5, 2025
223834c
Merge pull request #84 from nf-core/nf-core-template-merge-3.2.1
heylf May 5, 2025
90a242a
logo fix
khersameesh24 May 7, 2025
77aed6a
Merge pull request #7 from khersameesh24/add-subworkflow-cellpose
khersameesh24 May 7, 2025
9f857fd
parameter validation
khersameesh24 May 7, 2025
8f0801b
fixed warning generated while running baysor, xenium.toml included in…
khersameesh24 May 7, 2025
1d7b6d6
Manual testing changes, spatialdata subworkflow needs more testing wi…
khersameesh24 May 8, 2025
bd3433c
fix ci errors
khersameesh24 May 8, 2025
193ab2b
Merge branch 'dev' of https://github.com/khersameesh24/spatialxe into…
khersameesh24 May 8, 2025
a57f9ac
manual testing changes
khersameesh24 May 9, 2025
8a78bad
manual testing changes, config, schema, new pipeline mode option
khersameesh24 May 13, 2025
4b7291f
updated readme and precommit fix
khersameesh24 May 13, 2025
83bd0cd
removed idle config lines
khersameesh24 May 13, 2025
2055cb0
fix for CI test
khersameesh24 May 14, 2025
b534877
PR fixes and updated usage readme
khersameesh24 May 15, 2025
d5ffc67
fix for CI test profile
khersameesh24 May 15, 2025
6763695
subworkflow for testdata staging
khersameesh24 May 15, 2025
ac950f1
test profile fix
khersameesh24 May 16, 2025
894f19c
fix for cellpose CI test failure
khersameesh24 May 16, 2025
6b95f2e
Merge pull request #86 from khersameesh24/dev
khersameesh24 May 16, 2025
842af6b
variable uniformity, main workflow to subworkflows, fixed stub runs
khersameesh24 May 17, 2025
0b4b945
manual testing changes
khersameesh24 May 17, 2025
49ce4ee
manual testing changes
khersameesh24 May 17, 2025
811fe5b
manual testing changes
khersameesh24 May 18, 2025
6792bce
test profile fixes
khersameesh24 May 18, 2025
248b7b2
changed transcripts handling to parquet file from csv
khersameesh24 May 19, 2025
0703cd2
cpu num fix for cellpose
khersameesh24 May 19, 2025
a487b60
memory fix for cellpose
khersameesh24 May 19, 2025
81d1960
fix for imp-seg pipestance errors
khersameesh24 May 19, 2025
cd6ba06
fixed time for CI test with reduced resources
khersameesh24 May 19, 2025
50431e8
validator function for input xenium bundle
khersameesh24 May 19, 2025
2fd92b8
xenium bundle validator, minor changes for ci
khersameesh24 May 20, 2025
5389b80
xenium bundle validator, minor changes for ci
khersameesh24 May 20, 2025
261a56e
xenium bundle validator, minor changes for ci
khersameesh24 May 20, 2025
cd984ac
time limit changes for ci
khersameesh24 May 20, 2025
4cdb980
added updated metromap for spatialxe
khersameesh24 May 21, 2025
62aa1c6
removed gunzip, changes to ci test pipeline, ci changed to stub runs
khersameesh24 May 21, 2025
e54e368
removed gunzip, changes to ci test pipeline, ci changed to stub runs
khersameesh24 May 21, 2025
9d6710d
added preview and segfree mode
khersameesh24 May 23, 2025
dc84908
stub run on ci
khersameesh24 May 23, 2025
8468855
testing segfree methods
khersameesh24 May 23, 2025
67d45c1
testing segfree methods
khersameesh24 May 23, 2025
eca34f3
stub run on ci
khersameesh24 May 23, 2025
13bc492
Merge branch 'dev' of https://github.com/khersameesh24/spatialxe into…
khersameesh24 May 23, 2025
5c97a67
updated pipeline docs
khersameesh24 May 23, 2025
5351a51
Merge branch 'nf-core:dev' into dev
khersameesh24 May 30, 2025
21135b2
Merge pull request #88 from khersameesh24/dev
heylf Jun 2, 2025
d172d0b
Changes to metromap
heylf Jun 3, 2025
ec62b2a
deleting metromap svg
heylf Jun 3, 2025
ff0aaa6
Merge pull request #90 from heylf/dev
heylf Jun 3, 2025
e8168b4
changes to spatialxe.nf
heylf Jun 6, 2025
2c63a3c
changes for cellpose in module.config
heylf Jun 6, 2025
3f3aac0
changes for spatialdata write.py and spatialdatawrite main.nf
heylf Jun 6, 2025
adbea19
changes in cellpose main.nf
heylf Jun 6, 2025
4fd2c1a
changes in cellpose_resolift subworlfow main.nf
heylf Jun 6, 2025
8c86861
changes in spatialdata_write_meta_merge subworlfow main.nf
heylf Jun 6, 2025
653ee24
changes in utils_nfcore_spatialxe main.nf
heylf Jun 6, 2025
488989d
Merge pull request #92 from heylf/dev
khersameesh24 Jun 13, 2025
749aab4
changes for cell or nucleus segmentation
khersameesh24 Jun 16, 2025
9da39b1
run segmentation on cell, nuclei or both
khersameesh24 Jul 16, 2025
80e1842
run segmentation on cell, nuclei or both
khersameesh24 Jul 16, 2025
baa2dae
fixes for baysor runs, included transcript filtering, new preprocessi…
khersameesh24 Jul 16, 2025
8c828ef
parallel processing for baysor runs
khersameesh24 Jul 17, 2025
939d225
linting fix
khersameesh24 Jul 17, 2025
e3c3ebc
updated metromap
khersameesh24 Jul 20, 2025
0eeb8de
spatialdata and baysor fixes
khersameesh24 Jul 20, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 2 additions & 9 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,15 @@
"name": "nfcore",
"image": "nfcore/gitpod:latest",
"remoteUser": "gitpod",
"runArgs": ["--privileged"],

// Configure tool-specific properties.
"customizations": {
// Configure properties specific to VS Code.
"vscode": {
// Set *default* container specific settings.json values on container create.
"settings": {
"python.defaultInterpreterPath": "/opt/conda/bin/python",
"python.linting.enabled": true,
"python.linting.pylintEnabled": true,
"python.formatting.autopep8Path": "/opt/conda/bin/autopep8",
"python.formatting.yapfPath": "/opt/conda/bin/yapf",
"python.linting.flake8Path": "/opt/conda/bin/flake8",
"python.linting.pycodestylePath": "/opt/conda/bin/pycodestyle",
"python.linting.pydocstylePath": "/opt/conda/bin/pydocstyle",
"python.linting.pylintPath": "/opt/conda/bin/pylint"
"python.defaultInterpreterPath": "/opt/conda/bin/python"
},

// Add the IDs of extensions you want installed when the container is created.
Expand Down
17 changes: 15 additions & 2 deletions .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ trim_trailing_whitespace = true
indent_size = 4
indent_style = space

[*.{md,yml,yaml,html,css,scss,js,cff}]
[*.{md,yml,yaml,html,css,scss,js}]
indent_size = 2

# These files are edited and tested upstream in nf-core/modules
Expand All @@ -18,7 +18,20 @@ end_of_line = unset
insert_final_newline = unset
trim_trailing_whitespace = unset
indent_style = unset
indent_size = unset
[/subworkflows/nf-core/**]
charset = unset
end_of_line = unset
insert_final_newline = unset
trim_trailing_whitespace = unset
indent_style = unset

[/assets/email*]
indent_size = unset

# ignore python and markdown
[*.{py,md}]
indent_style = unset

# ignore ro-crate metadata files
[**/ro-crate-metadata.json]
insert_final_newline = unset
34 changes: 20 additions & 14 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# nf-core/spatialxe: Contributing Guidelines
# `nf-core/spatialxe`: Contributing Guidelines

Hi there!
Many thanks for taking an interest in improving nf-core/spatialxe.
Expand All @@ -9,6 +9,7 @@ Please use the pre-filled template to save time.
However, don't be put off by this template - other more general issues and suggestions are welcome!
Contributions to the code are even more welcome ;)

> [!NOTE]
> If you need help using or modifying nf-core/spatialxe then the best place to ask is on the nf-core Slack [#spatialxe](https://nfcore.slack.com/channels/spatialxe) channel ([join our Slack here](https://nf-co.re/join/slack)).

## Contribution workflow
Expand All @@ -18,13 +19,19 @@ If you'd like to write some code for nf-core/spatialxe, the standard workflow is
1. Check that there isn't already an issue about your idea in the [nf-core/spatialxe issues](https://github.com/nf-core/spatialxe/issues) to avoid duplicating work. If there isn't one already, please create one so that others know you're working on this
2. [Fork](https://help.github.com/en/github/getting-started-with-github/fork-a-repo) the [nf-core/spatialxe repository](https://github.com/nf-core/spatialxe) to your GitHub account
3. Make the necessary changes / additions within your forked repository following [Pipeline conventions](#pipeline-contribution-conventions)
4. Use `nf-core schema build` and add any new parameters to the pipeline JSON schema (requires [nf-core tools](https://github.com/nf-core/tools) >= 1.10).
4. Use `nf-core pipelines schema build` and add any new parameters to the pipeline JSON schema (requires [nf-core tools](https://github.com/nf-core/tools) >= 1.10).
5. Submit a Pull Request against the `dev` branch and wait for the code to be reviewed and merged

If you're not used to this workflow with git, you can start with some [docs from GitHub](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests) or even their [excellent `git` resources](https://try.github.io/).

## Tests

You have the option to test your changes locally by running the pipeline. For receiving warnings about process selectors and other `debug` information, it is recommended to use the debug profile. Execute all the tests with the following command:

```bash
nf-test test --profile debug,test,docker --verbose
```

When you create a pull request with changes, [GitHub Actions](https://github.com/features/actions) will run automatic tests.
Typically, pull-requests are only fully reviewed when these tests are passing, though of course we can help out before then.

Expand All @@ -33,7 +40,7 @@ There are typically two types of tests that run:
### Lint tests

`nf-core` has a [set of guidelines](https://nf-co.re/developers/guidelines) which all pipelines must adhere to.
To enforce these and ensure that all pipelines stay in sync, we have developed a helper tool which runs checks on the pipeline code. This is in the [nf-core/tools repository](https://github.com/nf-core/tools) and once installed can be run locally with the `nf-core lint <pipeline-directory>` command.
To enforce these and ensure that all pipelines stay in sync, we have developed a helper tool which runs checks on the pipeline code. This is in the [nf-core/tools repository](https://github.com/nf-core/tools) and once installed can be run locally with the `nf-core pipelines lint <pipeline-directory>` command.

If any failures or warnings are encountered, please follow the listed URL for more documentation.

Expand All @@ -48,27 +55,27 @@ These tests are run both with the latest available version of `Nextflow` and als

:warning: Only in the unlikely and regretful event of a release happening with a bug.

- On your own fork, make a new branch `patch` based on `upstream/master`.
- On your own fork, make a new branch `patch` based on `upstream/main` or `upstream/master`.
- Fix the bug, and bump version (X.Y.Z+1).
- A PR should be made on `master` from patch to directly this particular bug.
- Open a pull-request from `patch` to `main`/`master` with the changes.

## Getting help

For further information/help, please consult the [nf-core/spatialxe documentation](https://nf-co.re/spatialxe/usage) and don't hesitate to get in touch on the nf-core Slack [#spatialxe](https://nfcore.slack.com/channels/spatialxe) channel ([join our Slack here](https://nf-co.re/join/slack)).

## Pipeline contribution conventions

To make the nf-core/spatialxe code and processing logic more understandable for new contributors and to ensure quality, we semi-standardise the way the code and other contributions are written.
To make the `nf-core/spatialxe` code and processing logic more understandable for new contributors and to ensure quality, we semi-standardise the way the code and other contributions are written.

### Adding a new step

If you wish to contribute a new step, please use the following coding standards:

1. Define the corresponding input channel into your new process from the expected previous process channel
1. Define the corresponding input channel into your new process from the expected previous process channel.
2. Write the process block (see below).
3. Define the output channel if needed (see below).
4. Add any new parameters to `nextflow.config` with a default (see below).
5. Add any new parameters to `nextflow_schema.json` with help text (via the `nf-core schema build` tool).
5. Add any new parameters to `nextflow_schema.json` with help text (via the `nf-core pipelines schema build` tool).
6. Add sanity checks and validation for all relevant parameters.
7. Perform local tests to validate that the new code works as expected.
8. If applicable, add a new test command in `.github/workflow/ci.yml`.
Expand All @@ -77,15 +84,15 @@ If you wish to contribute a new step, please use the following coding standards:

### Default values

Parameters should be initialised / defined with default values in `nextflow.config` under the `params` scope.
Parameters should be initialised / defined with default values within the `params` scope in `nextflow.config`.

Once there, use `nf-core schema build` to add to `nextflow_schema.json`.
Once there, use `nf-core pipelines schema build` to add to `nextflow_schema.json`.

### Default processes resource requirements

Sensible defaults for process resource requirements (CPUs / memory / time) for a process should be defined in `conf/base.config`. These should generally be specified generic with `withLabel:` selectors so they can be shared across multiple processes/steps of the pipeline. A nf-core standard set of labels that should be followed where possible can be seen in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/master/nf_core/pipeline-template/conf/base.config), which has the default process as a single core-process, and then different levels of multi-core configurations for increasingly large memory requirements defined with standardised labels.
Sensible defaults for process resource requirements (CPUs / memory / time) for a process should be defined in `conf/base.config`. These should generally be specified generic with `withLabel:` selectors so they can be shared across multiple processes/steps of the pipeline. A nf-core standard set of labels that should be followed where possible can be seen in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/main/nf_core/pipeline-template/conf/base.config), which has the default process as a single core-process, and then different levels of multi-core configurations for increasingly large memory requirements defined with standardised labels.

The process resources can be passed on to the tool dynamically within the process with the `${task.cpu}` and `${task.memory}` variables in the `script:` block.
The process resources can be passed on to the tool dynamically within the process with the `${task.cpus}` and `${task.memory}` variables in the `script:` block.

### Naming schemes

Expand All @@ -96,7 +103,7 @@ Please use the following naming schemes, to make it easy to understand what is g

### Nextflow version bumping

If you are using a new feature from core Nextflow, you may bump the minimum required version of nextflow in the pipeline with: `nf-core bump-version --nextflow . [min-nf-version]`
If you are using a new feature from core Nextflow, you may bump the minimum required version of nextflow in the pipeline with: `nf-core pipelines bump-version --nextflow . [min-nf-version]`

### Images and figures

Expand All @@ -116,4 +123,3 @@ To get started:
Devcontainer specs:

- [DevContainer config](.devcontainer/devcontainer.json)
- [Dockerfile](.devcontainer/Dockerfile)
5 changes: 2 additions & 3 deletions .github/ISSUE_TEMPLATE/bug_report.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ body:

- [nf-core website: troubleshooting](https://nf-co.re/usage/troubleshooting)
- [nf-core/spatialxe pipeline documentation](https://nf-co.re/spatialxe/usage)

- type: textarea
id: description
attributes:
Expand Down Expand Up @@ -42,9 +41,9 @@ body:
attributes:
label: System information
description: |
* Nextflow version _(eg. 22.10.1)_
* Nextflow version _(eg. 23.04.0)_
* Hardware _(eg. HPC, Desktop, Cloud)_
* Executor _(eg. slurm, local, awsbatch)_
* Container engine: _(e.g. Docker, Singularity, Conda, Podman, Shifter or Charliecloud)_
* Container engine: _(e.g. Docker, Singularity, Conda, Podman, Shifter, Charliecloud, or Apptainer)_
* OS _(eg. CentOS Linux, macOS, Linux Mint)_
* Version of nf-core/spatialxe _(eg. 1.1, 1.5, 1.8.2)_
2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/feature_request.yml
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
name: Feature request
description: Suggest an idea for the nf-core/spatialxe pipeline. Please write in the title as the first word *module* (suggestin a new module for spatialxe) or *implement* for general feature request.
description: Suggest an idea for the nf-core/spatialxe pipeline
labels: enhancement
body:
- type: textarea
Expand Down
6 changes: 4 additions & 2 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,11 @@ Learn more about contributing: [CONTRIBUTING.md](https://github.com/nf-core/spat

- [ ] This comment contains a description of changes (with reason).
- [ ] If you've fixed a bug or added code that should be tested, add tests!
- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/spatialxe/tree/master/.github/CONTRIBUTING.md)- [ ] If necessary, also make a PR on the nf-core/spatialxe _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
- [ ] Make sure your code lints (`nf-core lint`).
- [ ] If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/spatialxe/tree/master/.github/CONTRIBUTING.md)
- [ ] If necessary, also make a PR on the nf-core/spatialxe _branch_ on the [nf-core/test-datasets](https://github.com/nf-core/test-datasets) repository.
- [ ] Make sure your code lints (`nf-core pipelines lint`).
- [ ] Ensure the test suite passes (`nextflow run . -profile test,docker --outdir <OUTDIR>`).
- [ ] Check for unexpected warnings in debug mode (`nextflow run . -profile debug,test,docker --outdir <OUTDIR>`).
- [ ] Usage Documentation in `docs/usage.md` is updated.
- [ ] Output Documentation in `docs/output.md` is updated.
- [ ] `CHANGELOG.md` is updated.
Expand Down
38 changes: 26 additions & 12 deletions .github/workflows/awsfulltest.yml
Original file line number Diff line number Diff line change
@@ -1,34 +1,48 @@
name: nf-core AWS full size tests
# This workflow is triggered on published releases.
# This workflow is triggered on PRs opened against the main/master branch.
# It can be additionally triggered manually with GitHub actions workflow dispatch button.
# It runs the -profile 'test_full' on AWS batch

on:
workflow_dispatch:
pull_request_review:
types: [submitted]
release:
types: [published]
workflow_dispatch:

jobs:
run-tower:
run-platform:
name: Run AWS full tests
if: github.repository == 'nf-core/spatialxe'
# run only if the PR is approved by at least 2 reviewers and against the master/main branch or manually triggered
if: github.repository == 'nf-core/spatialxe' && github.event.review.state == 'approved' && (github.event.pull_request.base.ref == 'master' || github.event.pull_request.base.ref == 'main') || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-latest
steps:
- name: Launch workflow via tower
uses: nf-core/tower-action@v3
- name: Set revision variable
id: revision
run: |
echo "revision=${{ (github.event_name == 'workflow_dispatch' || github.event_name == 'release') && github.sha || 'dev' }}" >> "$GITHUB_OUTPUT"

- name: Launch workflow via Seqera Platform
uses: seqeralabs/action-tower-launch@v2
# TODO nf-core: You can customise AWS full pipeline tests as required
# Add full size test data (but still relatively small datasets for few samples)
# on the `test_full.config` test runs with only one set of parameters
with:
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
compute_env: ${{ secrets.TOWER_COMPUTE_ENV }}
workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/spatialxe/work-${{ github.sha }}
revision: ${{ steps.revision.outputs.revision }}
workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/spatialxe/work-${{ steps.revision.outputs.revision }}
parameters: |
{
"outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/spatialxe/results-${{ github.sha }}"
"hook_url": "${{ secrets.MEGATESTS_ALERTS_SLACK_HOOK_URL }}",
"outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/spatialxe/results-${{ steps.revision.outputs.revision }}"
}
profiles: test_full,aws_tower
- uses: actions/upload-artifact@v3
profiles: test_full

- uses: actions/upload-artifact@v4
with:
name: Tower debug log file
path: tower_action_*.log
name: Seqera Platform debug log file
path: |
seqera_platform_action_*.log
seqera_platform_action_*.json
20 changes: 12 additions & 8 deletions .github/workflows/awstest.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,25 +5,29 @@ name: nf-core AWS test
on:
workflow_dispatch:
jobs:
run-tower:
run-platform:
name: Run AWS tests
if: github.repository == 'nf-core/spatialxe'
runs-on: ubuntu-latest
steps:
# Launch workflow using Tower CLI tool action
- name: Launch workflow via tower
uses: nf-core/tower-action@v3
# Launch workflow using Seqera Platform CLI tool action
- name: Launch workflow via Seqera Platform
uses: seqeralabs/action-tower-launch@v2
with:
workspace_id: ${{ secrets.TOWER_WORKSPACE_ID }}
access_token: ${{ secrets.TOWER_ACCESS_TOKEN }}
compute_env: ${{ secrets.TOWER_COMPUTE_ENV }}
revision: ${{ github.sha }}
workdir: s3://${{ secrets.AWS_S3_BUCKET }}/work/spatialxe/work-${{ github.sha }}
parameters: |
{
"outdir": "s3://${{ secrets.AWS_S3_BUCKET }}/spatialxe/results-test-${{ github.sha }}"
}
profiles: test,aws_tower
- uses: actions/upload-artifact@v3
profiles: test

- uses: actions/upload-artifact@v4
with:
name: Tower debug log file
path: tower_action_*.log
name: Seqera Platform debug log file
path: |
seqera_platform_action_*.log
seqera_platform_action_*.json
Loading
Loading