From f7b967e284676b4197374507d588b12ebc526631 Mon Sep 17 00:00:00 2001 From: "James A. Fellows Yates" Date: Sun, 16 Nov 2025 07:06:08 +0100 Subject: [PATCH 1/3] Move contributing to `docs/` and overhaul/review and add pipline-specific conventions --- .../pipeline-template/.github/CONTRIBUTING.md | 142 ------------- .../pipeline-template/docs/CONTRIBUTING.md | 186 ++++++++++++++++++ 2 files changed, 186 insertions(+), 142 deletions(-) delete mode 100644 nf_core/pipeline-template/.github/CONTRIBUTING.md create mode 100644 nf_core/pipeline-template/docs/CONTRIBUTING.md diff --git a/nf_core/pipeline-template/.github/CONTRIBUTING.md b/nf_core/pipeline-template/.github/CONTRIBUTING.md deleted file mode 100644 index b8157a7f4e..0000000000 --- a/nf_core/pipeline-template/.github/CONTRIBUTING.md +++ /dev/null @@ -1,142 +0,0 @@ -# `{{ name }}`: Contributing Guidelines - -Hi there! -Many thanks for taking an interest in improving {{ name }}. - -We try to manage the required tasks for {{ name }} using GitHub issues, you probably came to this page when creating one. -Please use the pre-filled template to save time. - -However, don't be put off by this template - other more general issues and suggestions are welcome! -Contributions to the code are even more welcome ;) - -{% if is_nfcore -%} - -> [!NOTE] -> If you need help using or modifying {{ name }} then the best place to ask is on the nf-core Slack [#{{ short_name }}](https://nfcore.slack.com/channels/{{ short_name }}) channel ([join our Slack here](https://nf-co.re/join/slack)). - -{% endif -%} - -## Contribution workflow - -If you'd like to write some code for {{ name }}, the standard workflow is as follows: - -1. Check that there isn't already an issue about your idea in the [{{ name }} issues](https://github.com/{{ name }}/issues) to avoid duplicating work. If there isn't one already, please create one so that others know you're working on this -2. [Fork](https://help.github.com/en/github/getting-started-with-github/fork-a-repo) the [{{ name }} repository](https://github.com/{{ name }}) to your GitHub account -3. Make the necessary changes / additions within your forked repository following [Pipeline conventions](#pipeline-contribution-conventions) -4. Use `nf-core pipelines schema build` and add any new parameters to the pipeline JSON schema (requires [nf-core tools](https://github.com/nf-core/tools) >= 1.10). -5. Submit a Pull Request against the `dev` branch and wait for the code to be reviewed and merged - -If you're not used to this workflow with git, you can start with some [docs from GitHub](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests) or even their [excellent `git` resources](https://try.github.io/). - -## Tests - -{% if test_config -%} -You have the option to test your changes locally by running the pipeline. For receiving warnings about process selectors and other `debug` information, it is recommended to use the debug profile. Execute all the tests with the following command: - -```bash -nf-test test --profile debug,test,docker --verbose -``` - -{% endif -%} -When you create a pull request with changes, [GitHub Actions](https://github.com/features/actions) will run automatic tests. -Typically, pull-requests are only fully reviewed when these tests are passing, though of course we can help out before then. - -{% if test_config %}There are typically two types of tests that run:{% endif %} - -### Lint tests - -`nf-core` has a [set of guidelines](https://nf-co.re/developers/guidelines) which all pipelines must adhere to. -To enforce these and ensure that all pipelines stay in sync, we have developed a helper tool which runs checks on the pipeline code. This is in the [nf-core/tools repository](https://github.com/nf-core/tools) and once installed can be run locally with the `nf-core pipelines lint ` command. - -If any failures or warnings are encountered, please follow the listed URL for more documentation. - -{%- if test_config %} - -### Pipeline tests - -Each `nf-core` pipeline should be set up with a minimal set of test-data. -`GitHub Actions` then runs the pipeline on this data to ensure that it exits successfully. -If there are any failures then the automated tests fail. -These tests are run both with the latest available version of `Nextflow` and also the minimum required version that is stated in the pipeline code. -{%- endif %} - -## Patch - -:warning: Only in the unlikely and regretful event of a release happening with a bug. - -- On your own fork, make a new branch `patch` based on `upstream/main` or `upstream/master`. -- Fix the bug, and bump version (X.Y.Z+1). -- Open a pull-request from `patch` to `main`/`master` with the changes. - -{% if is_nfcore -%} - -## Getting help - -For further information/help, please consult the [{{ name }} documentation](https://nf-co.re/{{ short_name }}/usage) and don't hesitate to get in touch on the nf-core Slack [#{{ short_name }}](https://nfcore.slack.com/channels/{{ short_name }}) channel ([join our Slack here](https://nf-co.re/join/slack)). - -{% endif -%} - -## Pipeline contribution conventions - -To make the `{{ name }}` code and processing logic more understandable for new contributors and to ensure quality, we semi-standardise the way the code and other contributions are written. - -### Adding a new step - -If you wish to contribute a new step, please use the following coding standards: - -1. Define the corresponding input channel into your new process from the expected previous process channel. -2. Write the process block (see below). -3. Define the output channel if needed (see below). -4. Add any new parameters to `nextflow.config` with a default (see below). -5. Add any new parameters to `nextflow_schema.json` with help text (via the `nf-core pipelines schema build` tool). -6. Add sanity checks and validation for all relevant parameters. -7. Perform local tests to validate that the new code works as expected. -8. If applicable, add a new test in the `tests` directory. - {%- if multiqc %} -9. Update MultiQC config `assets/multiqc_config.yml` so relevant suffixes, file name clean up and module plots are in the appropriate order. If applicable, add a [MultiQC](https://https://multiqc.info/) module. -10. Add a description of the output files and if relevant any appropriate images from the MultiQC report to `docs/output.md`. - {%- endif %} - -### Default values - -Parameters should be initialised / defined with default values within the `params` scope in `nextflow.config`. - -Once there, use `nf-core pipelines schema build` to add to `nextflow_schema.json`. - -### Default processes resource requirements - -Sensible defaults for process resource requirements (CPUs / memory / time) for a process should be defined in `conf/base.config`. These should generally be specified generic with `withLabel:` selectors so they can be shared across multiple processes/steps of the pipeline. A nf-core standard set of labels that should be followed where possible can be seen in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/main/nf_core/pipeline-template/conf/base.config), which has the default process as a single core-process, and then different levels of multi-core configurations for increasingly large memory requirements defined with standardised labels. - -The process resources can be passed on to the tool dynamically within the process with the `${task.cpus}` and `${task.memory}` variables in the `script:` block. - -### Naming schemes - -Please use the following naming schemes, to make it easy to understand what is going where. - -- initial process channel: `ch_output_from_` -- intermediate and terminal channels: `ch__for_` - -### Nextflow version bumping - -If you are using a new feature from core Nextflow, you may bump the minimum required version of nextflow in the pipeline with: `nf-core pipelines bump-version --nextflow . [min-nf-version]` - -### Images and figures - -For overview images and other documents we follow the nf-core [style guidelines and examples](https://nf-co.re/developers/design_guidelines). -{%- if codespaces %} - -## GitHub Codespaces - -This repo includes a devcontainer configuration which will create a GitHub Codespaces for Nextflow development! This is an online developer environment that runs in your browser, complete with VSCode and a terminal. - -To get started: - -- Open the repo in [Codespaces](https://github.com/{{ name }}/codespaces) -- Tools installed - - nf-core - - Nextflow - -Devcontainer specs: - -- [DevContainer config](.devcontainer/devcontainer.json) - {%- endif %} diff --git a/nf_core/pipeline-template/docs/CONTRIBUTING.md b/nf_core/pipeline-template/docs/CONTRIBUTING.md new file mode 100644 index 0000000000..00e7723817 --- /dev/null +++ b/nf_core/pipeline-template/docs/CONTRIBUTING.md @@ -0,0 +1,186 @@ +--- +title: Contributing +markdownPlugin: checklist +--- + +# `{{ name }}`: Contributing Guidelines + +Hi there! +Many thanks for taking an interest in improving {{ name }}. + +This page describes the recommended nf-core way for contributing to both {{ name }} and nf-core pipelines in general. + +- [General contribution guidelines](#general-contribution-guidelines): for common procedures or guides across all nf-core pipelines. +- [Pipeline specific contribution guidelines](#pipeline-specific-contribution-guidelines): for any procedures or guides specific to the development conventions of {{ name }}. + +{% if is_nfcore -%} + +> [!NOTE] +> If you need help using or modifying {{ name }} then the best place to ask is on the nf-core Slack [#{{ short_name }}](https://nfcore.slack.com/channels/{{ short_name }}) channel ([join our Slack here](https://nf-co.re/join/slack)). + +{% endif -%} + +## General contribution guidelines + +### Contribution quick start + +If you'd like to write some code for {{ name }}, the standard workflow is as follows: + +- [ ] Make sure you have Nextflow, nf-core tools, and nf-test installed - see the [nf-core/tools repository](https://github.com/nf-core/tools). +- [ ] Check that there isn't already an issue about your idea in the [{{ name }} issues](https://github.com/{{ name }}/issues) to avoid duplicating work. If there isn't one already, please create one so that others know you're working on this. +- [ ] [Fork](https://help.github.com/en/github/getting-started-with-github/fork-a-repo) the [{{ name }} repository](https://github.com/{{ name }}) to your GitHub account. +- [ ] Make the necessary changes / additions within your forked repository following [Pipeline conventions](#pipeline-contribution-conventions), ideally on a branch +- [ ] - If you are fixing a major bug, please using a branch nmmed `patch` and see the [Patch](#patch) section below. +- [ ] Update any relevant documentation within the `docs/` folder and use nf-core/tools to update `nextflow_schema.json` using nf-core/tools. +- [ ] [Lint](#lint-tests) your code using nf-core/tools +- [ ] Run and/or update tests as appropriate (see [Tests](#tests) section below). +- [ ] Submit a Pull Request against the `dev` branch and request for the code to be reviewed and merged. + +If you're not used to this workflow with git, you can start with some [docs from GitHub](https://help.github.com/en/github/collaborating-with-issues-and-pull-requests) or even their [excellent `git` resources](https://try.github.io/). + +{% if is_nfcore -%} + +### Getting help + +For further information/help, please consult the [{{ name }} documentation](https://nf-co.re/{{ short_name }}/usage) and don't hesitate to get in touch on the nf-core Slack [#{{ short_name }}](https://nfcore.slack.com/channels/{{ short_name }}) channel ([join our Slack here](https://nf-co.re/join/slack)). + +{% endif -%} + +{%- if codespaces %} + +### GitHub Codespaces + +This repo includes a devcontainer configuration which will create a GitHub Codespaces for Nextflow development! This is an online developer environment that runs in your browser, complete with VSCode and a terminal. + +To get started: + +- Open the repo in [Codespaces](https://github.com/{{ name }}/codespaces) +- Tools installed + - nf-core + - Nextflow + +Devcontainer specs: + +- [DevContainer config](.devcontainer/devcontainer.json) + {%- endif %} + +### Tests + +{% if test_config -%} +You have the option to test your changes locally by running the pipeline using nf-test. +We recommend to use the `--verbose` flag to see the Nextflow console log output in addition to nf-test's own output. + +```bash +nf-test test --tag test --profile +docker --verbose +``` + +{% endif -%} + +When you create a pull request with changes, [GitHub Actions](https://github.com/features/actions) will run automatic tests. +Typically, pull-requests are only fully reviewed when these tests are passing, though of course we can help out before then. + +{% if test_config %}There are typically two types of tests that run:{% endif %} + +#### Lint tests + +`nf-core` has a [set of guidelines](https://nf-co.re/docs/contributing/guidelines) which all pipelines must adhere to. +To enforce these and ensure that all pipelines stay in sync, you can use the nf-core/tools package to run the linting locally with the command. + +```bash +nf-core pipelines lint +``` + +If any failures or warnings are encountered, please follow the listed URL for more documentation. +The full list of tests can be seen [here](https://nf-co.re/docs/nf-core-tools/api_reference/latest/pipeline_lint_tests/actions_awsfulltest). + +{%- if test_config %} + +#### Pipeline tests + +Each `nf-core` pipeline should be set up with a minimal set of test-data. +`GitHub Actions` then runs the pipeline on this data to ensure that it exits successfully. +If there are any failures then the automated tests fail. +These tests are run both with the latest available version of `Nextflow` and also the minimum required version that is stated in the pipeline code. +{%- endif %} + +### Patch + +:warning: Only in the unlikely and regretful event of a release happening with a bug. + +- [ ] On your own fork, make a new branch `patch` based on `upstream/main` or `upstream/master`. +- [ ] Fix the bug, and use nf-core tools to bump version to the next semantic version e.g., `1.2.3` -> `1.2.4`. +- [ ] Open a pull-request from `patch` directly to `main`/`master` with the changes. + +### Pipeline contribution conventions + +To make the `{{ name }}` code and processing logic more understandable for new contributors and to ensure quality, we semi-standardise the way the code and other contributions are written. + +#### Adding a new step + +If you wish to contribute a new step to the pipeline, please use the following general nf-core coding procedure. +Please also refer to the [pipeline-specific contribution guidelines](#pipeline-specific-contribution-guidelines): + +- [ ] Define the corresponding [input channel](#channel-naming-schemes) into your new process from the expected previous process channel. +- [ ] Install a module using nf-core tools, or write a local module (check [info about resources](#default-processes-resource-requirements)), and add it to the target `.nf`. +- [ ] Define the output channel if needed (see below), mixing both the version output channel into `ch_versions` and relevant files into `ch_multiqc`. +- [ ] Add any new/update parameters to `nextflow.config` with a [default](#default-values) (see below). +- [ ] Add any new/update parameters to `nextflow_schema.json` with help text using [nf-core/tools](#default-values). +- [ ] Add validation for all relevant parameters to the pipeline`utilisation section of`utils*nfcore*{{ shortname }}\_pipeline/main.nf` subworkflow. +- [ ] Perform local tests to validate that the new code works as expected. +- [ ] If applicable, add a new test in the `tests` directory. +- [ ] Update `usage.md`, `output.md`, `citation.md` docs as appropriate. +- [ ] [Lint](lint) the code using nf-core/tools +- [ ] Update any diagrams or pipeline images as necessary + {%- if multiqc %} +- [ ]. Update MultiQC config `assets/multiqc_config.yml` so relevant suffixes, file name clean up and module plots are in the appropriate order. - [ ] applicable, add a [MultiQC](https://seqera.io/multiqc/) module. +- [ ]. Add a description of the output files and if relevant any appropriate images from the MultiQC report to `docs/output.md`. + {%- endif %} + +If you need to update the minimum required Nextflow version, please see the [Nextflow version bumping](#nextflow-version-bumping) section below. + +#### Channel naming schemes + +Please use the following naming schemes for channels, to make it easy to understand what is going where. + +- initial process channel: `ch_output_from_` +- intermediate and terminal channels: `ch__for_` + +#### Default values + +Parameters should be initialised / defined with default values within the `params` scope in `nextflow.config`. + +Once there, use: + +```bash +nf-core pipelines schema build +``` + +To update `nextflow_schema.json`. + +#### Default processes resource requirements + +If writing a local module, you should specify a default set of resource requirements for the process. + +Sensible defaults for process resource requirements (CPUs / memory / time) for a process should be defined in `conf/base.config`. +These should generally be specified with generic `withLabel:` selectors, so they can be shared across multiple processes/steps of the pipeline. + +nf-core provides a set of standard labels that should be followed where possible, and can be seen in the [nf-core pipeline template](https://github.com/nf-core/tools/blob/main/nf_core/pipeline-template/conf/base.config). +These labels have resource defaults for a single core-process, a module requiring use of a GPU, and then different levels of multi-core configurations for increasingly large memory requirements defined with standardised labels. + +The process resources should be passed on to the tool dynamically within the process using the `${task.cpus}` Nextflow variable, and where necessary the `${task.memory}` variable, in the command of the `script:` block (see example [here](https://github.com/nf-core/modules/blob/bd1b6a40f55933d94b8c9ca94ec8c1ea0eaf4b82/modules/nf-core/samtools/bam2fq/main.nf#L30)). + +#### Images and figures + +For overview images and other documents we follow the nf-core [style guidelines and examples](https://nf-co.re/docs/guidelines/graphic_design/overview). + +#### Nextflow version bumping + +If you are using a new feature from core Nextflow, you may bump the minimum required version of nextflow in the pipeline with: + +```bash +nf-core pipelines bump-version --nextflow . [min-nf-version] +``` + +## Pipeline specific contribution guidelines + + From 17604768374bac24f204b5d5a0219ce6a27d5cea Mon Sep 17 00:00:00 2001 From: "James A. Fellows Yates" Date: Sun, 16 Nov 2025 07:15:53 +0100 Subject: [PATCH 2/3] Update CHANGELOG --- CHANGELOG.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index a04aea7af9..6106197297 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -31,6 +31,7 @@ - Change GitHub Codespaces badge style ([#3869](https://github.com/nf-core/tools/pull/3869) and [#3873](https://github.com/nf-core/tools/pull/3873)) - Update multiqc to 1.32 ([#3878](https://github.com/nf-core/tools/pull/3878)) +- Move CONTRIBUTING.md to docs/, review and update text, and add pipeline-specific conventions section ([#3903](https://github.com/nf-core/tools/pull/3903)) ### Linting @@ -2157,6 +2158,7 @@ making a pull-request. See [`.github/CONTRIBUTING.md`](.github/CONTRIBUTING.md) - Docs are automatically built by Travis CI and updated on the nf-co.re website. - Introduced test for filtering remote workflows by keyword. - Build tools python API docs + - Use Travis job for api doc generation and publish - `nf-core bump-version` now stops before making changes if the linting fails From a320047d595222e5d803b77ef2cfb7a4e2e3cc9a Mon Sep 17 00:00:00 2001 From: "James A. Fellows Yates" Date: Sun, 16 Nov 2025 07:16:44 +0100 Subject: [PATCH 3/3] Update CHANGELOG.md --- CHANGELOG.md | 1 - 1 file changed, 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 6106197297..f7c0e910c0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2158,7 +2158,6 @@ making a pull-request. See [`.github/CONTRIBUTING.md`](.github/CONTRIBUTING.md) - Docs are automatically built by Travis CI and updated on the nf-co.re website. - Introduced test for filtering remote workflows by keyword. - Build tools python API docs - - Use Travis job for api doc generation and publish - `nf-core bump-version` now stops before making changes if the linting fails