Repository navigation
feat: add colabfold module #13066
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
feat: add colabfold module #13066
Changes from 4 commits
5504259
ab93c8e
a260a90
25135c1
a5aaff6
e22eedc
d4abca6
ca34b3b
198f9fb
c34377f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| --- | ||
| # yaml-language-server: $schema=https://raw.githubusercontent.com/nf-core/modules/master/modules/environment-schema.json | ||
| channels: | ||
| - conda-forge | ||
| - bioconda | ||
| dependencies: | ||
| - "bioconda::colabfold=1.5.5" | ||
| # CUDA 12.0 is the lowest CUDA whose jaxlib build ships ptxas (via cuda-nvcc) | ||
| - "conda-forge::cuda-version=12.0" | ||
| # colabfold 1.5.5 needs jax <0.4.26; newer dm-haiku releases need a newer jax | ||
| - "conda-forge::dm-haiku=0.0.12" | ||
| - "conda-forge::flax=0.8.2" | ||
| - "conda-forge::jax=0.4.25" | ||
| - "conda-forge::jaxlib=0.4.25=*cuda*" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| --- | ||
| # yaml-language-server: $schema=https://raw.githubusercontent.com/nf-core/modules/master/modules/environment-schema.json | ||
| channels: | ||
| - conda-forge | ||
| - bioconda | ||
| dependencies: | ||
| - "bioconda::colabfold=1.5.5" | ||
| # colabfold 1.5.5 needs jax <0.4.26; newer dm-haiku releases need a newer jax | ||
| - "conda-forge::dm-haiku=0.0.12" | ||
| - "conda-forge::flax=0.8.2" |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| process COLABFOLD { | ||
| tag "${meta.id}" | ||
| label 'process_high' | ||
| label 'process_gpu' | ||
|
|
||
| conda "${task.accelerator ? "${moduleDir}/environment.gpu.yml" : "${moduleDir}/environment.yml"}" | ||
| container "${workflow.containerEngine in ['singularity', 'apptainer'] && !task.ext.singularity_pull_docker_container | ||
| ? (task.accelerator ? 'https://community-cr-prod.seqera.io/docker/registry/v2/blobs/sha256/ee/eefb564f01b59ec8f7d85de34df208ba245d830ed9fecf1eb5bf8625cfbd68ef/data' : 'https://community-cr-prod.seqera.io/docker/registry/v2/blobs/sha256/d2/d2de6276b0c23cc1aaab4af2aa1336fa943b0c45d43ce4ed483911fb99a3db33/data') | ||
| : (task.accelerator ? 'community.wave.seqera.io/library/colabfold_cuda-version_dm-haiku_flax_pruned:ccfbc77af1b9acd9' : 'community.wave.seqera.io/library/colabfold_dm-haiku_flax:50a22b7e73876edb')}" | ||
|
|
||
| input: | ||
| tuple val(meta), path(fasta) | ||
| tuple val(meta2), path(weights) | ||
|
|
||
| output: | ||
| tuple val(meta), path("${prefix}/*_rank_*.pdb") , emit: pdb , optional: true | ||
| tuple val(meta), path("${prefix}/*_scores_rank_*.json"), emit: json, optional: true | ||
| tuple val(meta), path("${prefix}/*.a3m") , emit: a3m , optional: true | ||
| tuple val(meta), path("${prefix}/*.png") , emit: png , optional: true | ||
| tuple val(meta), path("${prefix}/log.txt") , emit: log , optional: true | ||
| tuple val("${task.process}"), val('colabfold'), eval("python -c 'import importlib.metadata as m; print(m.version(\"colabfold\"))'"), topic: versions, emit: versions_colabfold | ||
| tuple val("${task.process}"), val('cuda'), eval('python -c "import glob, json, sys; f = glob.glob(sys.prefix + \'/conda-meta/cuda-version-*.json\'); print(json.load(open(f[0]))[\'version\'] if f else \'no CUDA available\')"'), topic: versions, emit: versions_cuda | ||
|
|
||
| when: | ||
| task.ext.when == null || task.ext.when | ||
|
|
||
| script: | ||
| def args = task.ext.args ?: '' | ||
| prefix = task.ext.prefix ?: "${meta.id}" | ||
| def data = weights ? "--data ${weights}" : '' | ||
| """ | ||
| # Under Docker the container runs as the host UID, which has no home | ||
| # directory; colabfold_batch needs a writable HOME for its cache. | ||
| export HOME="\$PWD" | ||
|
|
||
| if [ ! -s ${fasta} ]; then | ||
| echo "WARN: ${fasta} is empty, no structure predicted" >&2 | ||
| else | ||
| colabfold_batch \\ | ||
| ${fasta} \\ | ||
| ${prefix} \\ | ||
| ${data} \\ | ||
| ${args} | ||
| fi | ||
| """ | ||
|
|
||
| stub: | ||
| prefix = task.ext.prefix ?: "${meta.id}" | ||
| """ | ||
| mkdir ${prefix} | ||
| touch ${prefix}/${prefix}_unrelaxed_rank_001_alphafold2_ptm_model_1_seed_000.pdb | ||
| touch ${prefix}/${prefix}_scores_rank_001_alphafold2_ptm_model_1_seed_000.json | ||
| touch ${prefix}/${prefix}.a3m | ||
| touch ${prefix}/${prefix}_plddt.png | ||
| touch ${prefix}/log.txt | ||
| """ | ||
| } | ||
| Original file line number | Diff line number | Diff line change | ||||
|---|---|---|---|---|---|---|
| @@ -0,0 +1,166 @@ | ||||||
| name: colabfold | ||||||
| description: Predicts protein structure with AlphaFold2, taking the multiple sequence alignment from an MMseqs2 server rather than local databases | ||||||
| keywords: | ||||||
| - protein structure | ||||||
| - structure prediction | ||||||
| - AlphaFold2 | ||||||
| - MSA | ||||||
| - folding | ||||||
| tools: | ||||||
| - colabfold: | ||||||
| description: | | ||||||
| ColabFold runs AlphaFold2 (and other folding models) while sourcing its | ||||||
| multiple sequence alignment from a remote MMseqs2 server, which removes | ||||||
| the ~2.6 TB of reference databases a local AlphaFold2 install needs. | ||||||
| The public server (api.colabfold.com) is used by default; pass | ||||||
| `--host-url` through `ext.args` to use your own instance. A GPU speeds | ||||||
| inference up but is not required; Amber relaxation is enabled with | ||||||
| `--amber` (and `--use-gpu-relax` on a GPU). | ||||||
| homepage: https://github.com/sokrypton/ColabFold | ||||||
| documentation: https://github.com/sokrypton/ColabFold | ||||||
| doi: 10.1038/s41592-022-01488-1 | ||||||
| licence: ["MIT"] | ||||||
| identifier: biotools:colabfold | ||||||
| args_id: "$args" | ||||||
|
|
||||||
| input: | ||||||
| - - meta: | ||||||
| type: map | ||||||
| description: | | ||||||
| Groovy Map containing sample information | ||||||
| e.g. [ id:'sample1' ] | ||||||
| - fasta: | ||||||
| type: file | ||||||
| description: Protein sequences to fold, one prediction per record (an empty file produces no output) | ||||||
| pattern: "*.{fa,faa,fasta}" | ||||||
| ontologies: | ||||||
| - edam: http://edamontology.org/format_1929 # FASTA | ||||||
| - - meta2: | ||||||
| type: map | ||||||
| description: | | ||||||
| Groovy Map containing weights information | ||||||
| e.g. [ id:'alphafold2_params' ] | ||||||
| - weights: | ||||||
| type: directory | ||||||
| description: Optional ColabFold data directory with the AlphaFold2 parameters in `params/`; pass `[]` to download them | ||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Applied as suggested. |
||||||
| ontologies: [] | ||||||
| output: | ||||||
| pdb: | ||||||
| - - meta: | ||||||
| type: map | ||||||
| description: | | ||||||
| Groovy Map containing sample information | ||||||
| e.g. [ id:'sample1' ] | ||||||
| - ${prefix}/*_rank_*.pdb: | ||||||
| type: file | ||||||
| description: Predicted structures (rank_001 is the best model; per-residue pLDDT in the B-factor column) | ||||||
| pattern: "*_rank_*.pdb" | ||||||
| ontologies: | ||||||
| - edam: http://edamontology.org/format_1476 # PDB | ||||||
| json: | ||||||
| - - meta: | ||||||
| type: map | ||||||
| description: | | ||||||
| Groovy Map containing sample information | ||||||
| e.g. [ id:'sample1' ] | ||||||
| - ${prefix}/*_scores_rank_*.json: | ||||||
| type: file | ||||||
| description: Per-model scores (per-residue pLDDT, PAE, pTM) | ||||||
| pattern: "*_scores_rank_*.json" | ||||||
| ontologies: | ||||||
| - edam: http://edamontology.org/format_3464 # JSON | ||||||
| a3m: | ||||||
| - - meta: | ||||||
| type: map | ||||||
| description: | | ||||||
| Groovy Map containing sample information | ||||||
| e.g. [ id:'sample1' ] | ||||||
| - ${prefix}/*.a3m: | ||||||
| type: file | ||||||
| description: Multiple sequence alignment used for each query | ||||||
| pattern: "*.a3m" | ||||||
| ontologies: [] | ||||||
| png: | ||||||
| - - meta: | ||||||
| type: map | ||||||
| description: | | ||||||
| Groovy Map containing sample information | ||||||
| e.g. [ id:'sample1' ] | ||||||
| - ${prefix}/*.png: | ||||||
| type: file | ||||||
| description: pLDDT, PAE and MSA coverage plots | ||||||
| pattern: "*.png" | ||||||
| ontologies: | ||||||
| - edam: http://edamontology.org/format_3603 # PNG | ||||||
| log: | ||||||
| - - meta: | ||||||
| type: map | ||||||
| description: | | ||||||
| Groovy Map containing sample information | ||||||
| e.g. [ id:'sample1' ] | ||||||
| - ${prefix}/log.txt: | ||||||
| type: file | ||||||
| description: colabfold_batch log | ||||||
| pattern: "log.txt" | ||||||
| ontologies: | ||||||
| - edam: http://edamontology.org/format_2330 # Textual format | ||||||
| versions_colabfold: | ||||||
| - - ${task.process}: | ||||||
| type: string | ||||||
| description: The name of the process | ||||||
| - colabfold: | ||||||
| type: string | ||||||
| description: The name of the tool | ||||||
| - python -c 'import importlib.metadata as m; print(m.version("colabfold"))': | ||||||
| type: eval | ||||||
| description: The expression to obtain the version of the tool | ||||||
| versions_cuda: | ||||||
| - - ${task.process}: | ||||||
| type: string | ||||||
| description: The name of the process | ||||||
| - cuda: | ||||||
| type: string | ||||||
| description: Name of the runtime | ||||||
| - python -c "import glob, json, sys; f = glob.glob(sys.prefix + '/conda-meta/cuda-version-*.json'); print(json.load(open(f[0]))['version'] if f else 'no CUDA available')": | ||||||
| type: eval | ||||||
| description: CUDA version of the task's environment, or `no CUDA available` on the CPU path | ||||||
|
|
||||||
| topics: | ||||||
| versions: | ||||||
| - - ${task.process}: | ||||||
| type: string | ||||||
| description: The name of the process | ||||||
| - colabfold: | ||||||
| type: string | ||||||
| description: The name of the tool | ||||||
| - python -c 'import importlib.metadata as m; print(m.version("colabfold"))': | ||||||
| type: eval | ||||||
| description: The expression to obtain the version of the tool | ||||||
| - - ${task.process}: | ||||||
| type: string | ||||||
| description: The name of the process | ||||||
| - cuda: | ||||||
| type: string | ||||||
| description: Name of the runtime | ||||||
| - python -c "import glob, json, sys; f = glob.glob(sys.prefix + '/conda-meta/cuda-version-*.json'); print(json.load(open(f[0]))['version'] if f else 'no CUDA available')": | ||||||
| type: eval | ||||||
| description: CUDA version of the task's environment, or `no CUDA available` on the CPU path | ||||||
|
|
||||||
| authors: | ||||||
| - "@Bboy010" | ||||||
| maintainers: | ||||||
| - "@Bboy010" | ||||||
| containers: | ||||||
| docker: | ||||||
| linux/amd64: | ||||||
| name: community.wave.seqera.io/library/colabfold_dm-haiku_flax:50a22b7e73876edb | ||||||
| build_id: bd-50a22b7e73876edb_1 | ||||||
| scan_id: sc-7d6a4f09ec9a8b34_1 | ||||||
| singularity: | ||||||
| linux/amd64: | ||||||
| name: oras://community.wave.seqera.io/library/colabfold_dm-haiku_flax:797eb5b15587ba43 | ||||||
| build_id: bd-797eb5b15587ba43_1 | ||||||
| https: https://community-cr-prod.seqera.io/docker/registry/v2/blobs/sha256/d2/d2de6276b0c23cc1aaab4af2aa1336fa943b0c45d43ce4ed483911fb99a3db33/data | ||||||
| conda: | ||||||
| linux/amd64: | ||||||
| lock_file: modules/nf-core/colabfold/.conda-lock/linux_amd64-bd-50a22b7e73876edb_1.txt | ||||||
| Original file line number | Diff line number | Diff line change | ||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,65 @@ | ||||||||||||||||||||||||||||||||
| nextflow_process { | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| name "Test Process COLABFOLD (GPU)" | ||||||||||||||||||||||||||||||||
| script "../main.nf" | ||||||||||||||||||||||||||||||||
| process "COLABFOLD" | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| tag "gpu" | ||||||||||||||||||||||||||||||||
| tag "modules" | ||||||||||||||||||||||||||||||||
| tag "modules_nfcore" | ||||||||||||||||||||||||||||||||
| tag "colabfold" | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| // Weights are passed as [] so colabfold_batch downloads them (~3.5 GB); | ||||||||||||||||||||||||||||||||
| // they cannot be shipped in nf-core/test-datasets. nextflow.gpu.config | ||||||||||||||||||||||||||||||||
| // runs a single-sequence prediction, so no MSA server is queried. | ||||||||||||||||||||||||||||||||
| test("proteomics - faa - single sequence - gpu") { | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| config "./nextflow.gpu.config" | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| when { | ||||||||||||||||||||||||||||||||
| process { | ||||||||||||||||||||||||||||||||
| """ | ||||||||||||||||||||||||||||||||
| input[0] = [ | ||||||||||||||||||||||||||||||||
| [ id:'test' ], | ||||||||||||||||||||||||||||||||
| file(params.modules_testdata_base_path + 'proteomics/interproscan/l_arginase.faa', checkIfExists: true) | ||||||||||||||||||||||||||||||||
| ] | ||||||||||||||||||||||||||||||||
| input[1] = [ [:], [] ] | ||||||||||||||||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No test passes
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done: the weights come from the setup block, and two new GPU tests cover |
||||||||||||||||||||||||||||||||
| """ | ||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| then { | ||||||||||||||||||||||||||||||||
| assert process.success | ||||||||||||||||||||||||||||||||
| assertAll( | ||||||||||||||||||||||||||||||||
| { assert snapshot(sanitizeOutput(process.out, unstableKeys: ['pdb', 'json', 'png', 'log'])).match() }, | ||||||||||||||||||||||||||||||||
| { assert path(process.out.pdb[0][1]).readLines().any { it.startsWith('ATOM') } }, | ||||||||||||||||||||||||||||||||
| { assert path(process.out.log[0][1]).text.contains('Done') } | ||||||||||||||||||||||||||||||||
| ) | ||||||||||||||||||||||||||||||||
|
Comment on lines
+33
to
+36
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Please put the stable checks for the unstable channels inside the snapshot, so a change shows up as a snapshot diff, and add one for
Suggested change
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Done in a5aaff6: the ATOM count, the plddt check on the scores JSON and the |
||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| test("proteomics - faa - stub - gpu") { | ||||||||||||||||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Stub test names should end with
Suggested change
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Renamed to |
||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| config "./nextflow.gpu.config" | ||||||||||||||||||||||||||||||||
| options "-stub" | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| when { | ||||||||||||||||||||||||||||||||
| process { | ||||||||||||||||||||||||||||||||
| """ | ||||||||||||||||||||||||||||||||
| input[0] = [ | ||||||||||||||||||||||||||||||||
| [ id:'test' ], | ||||||||||||||||||||||||||||||||
| file(params.modules_testdata_base_path + 'proteomics/interproscan/l_arginase.faa', checkIfExists: true) | ||||||||||||||||||||||||||||||||
| ] | ||||||||||||||||||||||||||||||||
| input[1] = [ [:], [] ] | ||||||||||||||||||||||||||||||||
| """ | ||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||
|
|
||||||||||||||||||||||||||||||||
| then { | ||||||||||||||||||||||||||||||||
| assert process.success | ||||||||||||||||||||||||||||||||
| assertAll( | ||||||||||||||||||||||||||||||||
| { assert snapshot(sanitizeOutput(process.out)).match() } | ||||||||||||||||||||||||||||||||
| ) | ||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||
| } | ||||||||||||||||||||||||||||||||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Several files that
colabfold_batchwrites are not emitted:config.json,cite.bibtex,*_predicted_aligned_error_v1.json, the*.picklefiles (--msa-only,--save-all), the*_repr_*.npyfiles (--save-single-representations,--save-pair-representations),*.result.zip(--zip) and*_template_domain_names.json(--templates). Please emit them as optional outputs, and updatemeta.ymland the stub to match.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done in a5aaff6, with your suggested outputs (pae, pickle, npy, zip, templates, config, bibtex), plus meta.yml entries and EDAM terms where they exist (ZIP format_3987, NumPy format_4003, pickle format_4002;
[]for BibTeX). I checked the file names against colabfold 1.5.5's source: representations are*_single_repr_*.npy/*_pair_repr_*.npyand the pickles are*_all_*.pickle(--save-all) or<query>.pickle(--msa-only), so the globs match.