EBI nf-core config

[priyanka-surana]

Details below:

[nuno-agostinho]

To include in the config:

• Add configuration for LSF/SLURM/cloud (mind job limits?)
• Support for the LSF/SLURM queues in Codon
• Singularity profile
• Default options for trace, dag, timeline and report?
• Generic environment variables using env
• Global variables/parameters to define (such as default directories: scratch, software, production, research)
• Default temporary directory?
• GPU support
• Generic errorStrategy (example)
• Add a manifest (example)
• Modularisation if possible using includeConfig
• Option to enable notification

To discuss:

• Should we create a unique profile for each option? This would allow to choose which profiles to run in Nextflow, such as -profile lsf,singularity,gpu
• Create a README/tutorial:
  • Simple commands on how to use Nextflow and the EBI nf-core config in Codon
  • How to run a Nextflow main job (using tmux/screen or as a LSF/SLURM job itself)
  • How to use a custom config on top of the EBI config
  • How to optimise Nextflow
  • How to cleanup after Nextflow (nextflow clean or manual steps)
  • Cleanup for temporary directories
  • Further resources / help / contact
• In which directory to put the EBI config? Who would have write permissions?
• Add all information to Confluence or embl.service-now.com?

More information:

• https://embl.service-now.com/kb_view.do?sysparm_article=KB0010969
• https://embl.service-now.com/sp/?id=kb_article_view&sysparm_article=KB0010982

[biomadeira]

We have already made good progress on this, but looking forward to continuing working on it.
See https://github.com/biomadeira/EBI-nextflow-config

[likhitha-surapaneni]

It was a great start @biomadeira , thank you to everyone who has participated! Looking forward to continue the work.