Handle extra header for reads#246
Draft
sainsachiko wants to merge 20 commits into
Draft
Conversation
Contributor
|
Warning Newer version of the nf-core template is available. Your pipeline is using an old version of the nf-core template: 3.5.2. For more documentation on how to update your pipeline, please see the nf-core documentation and Synchronisation documentation. |
|
muffato
reviewed
Jun 3, 2026
muffato
left a comment
muffato
left a comment
Member
There was a problem hiding this comment.
If you pay attention to the final headers, you'll notice that the insertion of the extra @PG headers creates a parallel program chain. From that point on, each command (samtools reheader, samtools view, samtools reheader) is inserted twice: one connected to the pre-existing chain, one connected to the inserted one.
The workaround is to insert a PP tag on the first @PG line that is in the BAM file, and link it to the last ID value from the header being inserted.
Secondly, there are now two reheader jobs happening from CRAM output. Each one creates a new file which makes a lot of I/O. I would suggest the following:
- Make
SAMTOOLS_CRAM(SAMTOOLS_VIEW) accept an optional header to be able embedSAMTOOLS_REHEADER_CRAM. Here, the principle is that the complete output header is the input header without its@SQlines and adding the@SQlines from the extra header. I'd like that because this reheader is essentially about fixing the@SQlinessamtools viewis created wrong. - Make the sanger-tol
FASTX_MAP_LONG_READSaccept an optional header file of@PGlines and slot asamtools reheadercommand betweenminimap2andsamtools sort.subworkflows/local/align_long.nfcould prepare that file (add the.extra_headersuffix etc) beforehand in a new module. Connecting this to my first comment, you could use the fact minimap2 inserts a new root@PGand connect the PG chains there.
It might make sense to expand thePREPARE_READ_GROUPSsub-workflow into aPREPARE_HEADERSfor this ? - Similar fix for the sanger-tol short read sub-workflows
Think about it, see if it makes sense
Member
There was a problem hiding this comment.
Test a samplesheet without the extra_header column
sainsachiko
commented
Jun 3, 2026
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: Matthieu Muffato <cortexspam-github@yahoo.fr> Co-authored-by: Hanh Hoang <134130358+sainsachiko@users.noreply.github.com>
muffato
reviewed
Jun 4, 2026
Member
There was a problem hiding this comment.
oh I didn't think it'd make the module that complex ! Some comments:
- the whole point is to avoid having multiple copies of the BAM/CRAM files written on disk where just the header differs. I was hoping
samtools viewcould pipe intosamtools reheader. If they have to be separate commands, they might as well be different modules, there's no difference I/O wise (there's a difference LSF wise but not worth losing the nf-core modularity). - I can see you're running
samtools indexseparately. Can it not be kept insamtools view? - Do we do reheader + unselected at the same time ? Seeing all those
if, I wonder if it'd be cleaner to keep the original nf-core/samtools/view module and create a new one local/samtools/viewreheader
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixed this #229:
PR checklist
nf-core pipelines lint).nextflow run . -profile test,docker --outdir <OUTDIR>).nextflow run . -profile debug,test,docker --outdir <OUTDIR>).docs/usage.mdis updated.docs/output.mdis updated.CHANGELOG.mdis updated.README.mdis updated (including new tool citations and authors/contributors).