I think I found another bug regarding the GATK4_BASERECALIBRATOR image used to run that process.
and I verified this by running the image with singularity and then trying to import the library in java after running jshell
Because of this the pipeline crashed at the base recalibration step.
Attached below the complete error in nextflow + the nextflow log + the commands used to verify the lack of the common lang3 library
#Nextflow error:
Monitor the execution with Seqera Platform using this URL: https://seqera.ieo.it/orgs/IEO/workspaces/DIMA/watch/3Rn25aAs63pzJT
executor > slurm (3762)
[- ] NFC…EPARE_GENOME:BWAMEM1_INDEX -
[25/bce5e9] NFC…_sapiens_assembly38.fasta) | 1 of 1 ✔
[- ] NFC…E_GENOME:DRAGMAP_HASHTABLE -
[- ] NFC…4_CREATESEQUENCEDICTIONARY -
[- ] NFC…PARE_GENOME:SAMTOOLS_FAIDX -
[- ] NFC…PREPARE_GENOME:TABIX_DBSNP -
[- ] NFC…ME:TABIX_GERMLINE_RESOURCE -
[- ] NFC…RE_GENOME:TABIX_KNOWN_SNPS -
[- ] NFC…_GENOME:TABIX_KNOWN_INDELS -
[- ] NFC…S:PREPARE_GENOME:TABIX_PON -
[1a/ba56a8] NFC…_sapiens_assembly38.fasta) | 1 of 1 ✔
[8c/26d248] NFC…egions_noseconds.hg38.bed) | 1 of 1 ✔
[d1/7a0001] NFC…LIT (chrY_9055175-9057608) | 21 of 21 ✔
[5d/157bde] NFC…ng_regions_noseconds.hg38) | 1 of 1 ✔
[- ] NFC…NPUT:SAMTOOLS_VIEW_MAP_MAP -
[- ] NFC…:SAMTOOLS_VIEW_UNMAP_UNMAP -
[- ] NFC…UT:SAMTOOLS_VIEW_UNMAP_MAP -
[- ] NFC…UT:SAMTOOLS_VIEW_MAP_UNMAP -
[- ] NFC…INPUT:SAMTOOLS_MERGE_UNMAP -
[- ] NFC…_INPUT:COLLATE_FASTQ_UNMAP -
[2d/0d1a34] NFC…p_85411558507499-T-DNA-L1) | 150 of 150 ✔
[50/91781a] NFC…p_85411558507499-T-DNA-L1) | 150 of 150 ✔
[78/962d1e] NFC…M (p_85411558507499-T-DNA) | 412 of 412 ✔
[1d/905c52] NFC…GN (p_8431998253920-T-RNA) | 200 of 200 ✔
[31/f0bd50] NFC…RT (p_8431998253920-T-RNA) | 200 of 200 ✔
[58/9e6343] NFC…EX (p_8431998253920-T-RNA) | 200 of 200 ✔
[- ] NFC…TS_SAMTOOLS:SAMTOOLS_STATS | 0 of 200
[- ] NFC…SAMTOOLS:SAMTOOLS_FLAGSTAT | 0 of 200
[- ] NFC…SAMTOOLS:SAMTOOLS_IDXSTATS | 0 of 200
[9a/858887] NFC…ES (p_8431998253920-T-RNA) | 150 of 150 ✔
[7d/d3eb3c] NFC… (p_271070591634223-T-RNA) | 96 of 150
[9f/ee377b] NFC… (p_271070591634223-T-RNA) | 98 of 150
[3b/5635ce] NFC… (p_271070591634223-T-RNA) | 42 of 1050
[- ] NFC…_INDEX_SAMTOOLS:MERGE_CRAM | 0 of 2
[ea/2fc870] NFC…R (p_80673775312053-T-DNA) | 2006 of 2100
[4d/9531e3] NFC… (p_276496020777021-N-DNA) | 10 of 94
[- ] NFC…_APPLYBQSR:GATK4_APPLYBQSR | 0 of 210
Plus 33 more processes waiting for tasks…
Execution cancelled -- Finishing pending tasks before exit
-[nf-core/rnadnavar] Pipeline completed with errors-
WARN: Killing running tasks (20)
ERROR ~ Error executing process > 'NFCORE_RNADNAVAR:RNADNAVAR:BAM_PROCESSING:BAM_GATK_PREPROCESSING:BAM_BASERECALIBRATOR:GATK4_BASERECALIBRATOR (p_75013800252932-T-DNA)'
Caused by:
Process `NFCORE_RNADNAVAR:RNADNAVAR:BAM_PROCESSING:BAM_GATK_PREPROCESSING:BAM_BASERECALIBRATOR:GATK4_BASERECALIBRATOR (p_75013800252932-T-DNA)` terminated with an error exit status (1)
Command executed:
gatk --java-options "-Xmx26214M -XX:-UsePerfData" \
BaseRecalibrator \
--input p_75013800252932-T-DNA.md.cram \
--output p_75013800252932-T-DNA_chr5_139453660-155760324.recal.table \
--reference Homo_sapiens_assembly38.fasta \
--intervals chr5_139453660-155760324.bed \
--known-sites dbsnp_146.hg38.vcf.gz \
--tmp-dir . \
cat <<-END_VERSIONS > versions.yml
"NFCORE_RNADNAVAR:RNADNAVAR:BAM_PROCESSING:BAM_GATK_PREPROCESSING:BAM_BASERECALIBRATOR:GATK4_BASERECALIBRATOR":
gatk4: $(echo $(gatk --version 2>&1) | sed 's/^.*(GATK) v//; s/ .*$//')
END_VERSIONS
Command exit status:
1
Command output:
(empty)
Command error:
01:08:21.839 INFO ProgressMeter - Current Locus Elapsed Minutes Reads Processed Reads/Minute
01:08:31.864 INFO ProgressMeter - chr5:146408704 0.2 202000 1208977.6
01:08:41.911 INFO ProgressMeter - chr5:157360284 0.3 423000 1264448.0
01:08:51.923 INFO ProgressMeter - chr5:171525914 0.5 629000 1254487.4
01:09:01.969 INFO ProgressMeter - chr5:180115338 0.7 849000 1269374.5
01:09:11.984 INFO ProgressMeter - chr6:10400647 0.8 1052000 1258749.6
01:09:21.988 INFO ProgressMeter - chr6:22901995 1.0 1259000 1255881.2
01:09:31.993 INFO ProgressMeter - chr6:35637143 1.2 1451000 1240984.1
01:09:42.012 INFO ProgressMeter - chr6:43206450 1.3 1651000 1235593.5
01:09:52.051 INFO ProgressMeter - chr6:54137953 1.5 1872000 1245067.2
01:10:02.059 INFO ProgressMeter - chr6:70536243 1.7 2069000 1238687.3
01:10:12.095 INFO ProgressMeter - chr6:83895816 1.8 2274000 1237483.7
01:10:20.057 INFO BaseRecalibrator - 712140 read(s) filtered by: MappingQualityNotZeroReadFilter
0 read(s) filtered by: MappingQualityAvailableReadFilter
0 read(s) filtered by: MappedReadFilter
0 read(s) filtered by: NotSecondaryAlignmentReadFilter
12336424 read(s) filtered by: NotDuplicateReadFilter
0 read(s) filtered by: PassesVendorQualityCheckReadFilter
0 read(s) filtered by: WellformedReadFilter
13048564 total reads filtered out of 15485788 reads processed
01:10:20.057 INFO ProgressMeter - chr6:94986409 2.0 2437224 1236981.2
01:10:20.057 INFO ProgressMeter - Traversal complete. Processed 2437224 total reads in 2.0 minutes.
01:10:20.094 INFO BaseRecalibrator - Calculating quantized quality scores...
01:10:20.101 INFO BaseRecalibrator - Writing recalibration report...
01:13:20.453 INFO BaseRecalibrator - Shutting down engine
[January 27, 2026 at 1:13:20 AM GMT] org.broadinstitute.hellbender.tools.walkers.bqsr.BaseRecalibrator done. Elapsed time: 5.00 minutes.
Runtime.totalMemory()=1493172224
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/lang3/math/NumberUtils
at org.broadinstitute.hellbender.utils.report.GATKReportColumn.isRightAlign(GATKReportColumn.java:70)
at org.broadinstitute.hellbender.utils.report.GATKReportColumn.updateFormat(GATKReportColumn.java:126)
at org.broadinstitute.hellbender.utils.report.GATKReportColumn.updateFormatting(GATKReportColumn.java:115)
at org.broadinstitute.hellbender.utils.report.GATKReportTable.set(GATKReportTable.java:367)
at org.broadinstitute.hellbender.utils.report.GATKReportTable.addRowIDMapping(GATKReportTable.java:286)
at org.broadinstitute.hellbender.utils.report.GATKReportTable.addRowID(GATKReportTable.java:261)
at org.broadinstitute.hellbender.utils.recalibration.RecalibrationArgumentCollection.generateReportTable(RecalibrationArgumentCollection.java:153)
at org.broadinstitute.hellbender.utils.recalibration.RecalUtils.outputRecalibrationReport(RecalUtils.java:299)
at org.broadinstitute.hellbender.tools.walkers.bqsr.BaseRecalibrator.generateReport(BaseRecalibrator.java:221)
at org.broadinstitute.hellbender.tools.walkers.bqsr.BaseRecalibrator.onTraversalSuccess(BaseRecalibrator.java:200)
at org.broadinstitute.hellbender.engine.GATKTool.doWork(GATKTool.java:1123)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.runTool(CommandLineProgram.java:150)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMainPostParseArgs(CommandLineProgram.java:203)
at org.broadinstitute.hellbender.cmdline.CommandLineProgram.instanceMain(CommandLineProgram.java:222)
at org.broadinstitute.hellbender.Main.runCommandLineProgram(Main.java:166)
at org.broadinstitute.hellbender.Main.mainEntry(Main.java:209)
at org.broadinstitute.hellbender.Main.main(Main.java:306)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang3.math.NumberUtils
at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641)
at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:525)
... 17 more
Work dir:
/beegfs/scratch/ieo7224/__softwares/rnadnavar/pipeline/dev/work/27/d7b3dfd78106960499f9ac1e109d32
Container:
/hpcnfs/data/DIMA/singularity_cache/community-cr-prod.seqera.io-docker-registry-v2-blobs-sha256-b2-b28daf5d9bb2f0d129dcad1b7410e0dd8a9b087aaf3ec7ced929b1f57624ad98-data.img
Tip: you can replicate the issue by changing to the process work dir and entering the command `bash .command.run`
-- Check '.nextflow.log' file for details
ERROR ~ Pipeline failed. Please refer to troubleshooting docs: https://nf-co.re/docs/usage/troubleshooting
-- Check '.nextflow.log' file for details
Join mismatch for the following entries:
- key=[patient:p_8431998253920, sample:p_8431998253920-N-DNA, status:0, lane:L1, normal_id:[], n_fastq:4, data_type:cram, id:p_8431998253920-N-DNA] values=
- key=[patient:p_8374025177592, sample:p_8374025177592-T-DNA, status:1, lane:L1, normal_id:[], n_fastq:4, data_type:cram, id:p_8374025177592-T-DNA] values=
- key=[patient:p_53130606084809, sample:p_53130606084809-N-DNA, status:0, lane:L1, normal_id:[], n_fastq:4, data_type:cram, id:p_53130606084809-N-DNA] values=
- key=[patient:p_83702486114145, sample:p_83702486114145-N-DNA, status:0, lane:L1, normal_id:[], n_fastq:4, data_type:cram, id:p_83702486114145-N-DNA] values=
- key=[patient:p_55824097563636, sample:p_55824097563636-N-DNA, status:0, lane:L1, normal_id:[], n_fastq:4, data_type:cram, id:p_55824097563636-N-DNA] values=
- key=[patient:p_48526896159257, sample:p_48526896159257-T-DNA, status:1, lane:L1, normal_id:[], n_fastq:4, data_type:cram, id:p_48526896159257-T-DNA] values=
- key=[patient:p_69015818905746, sample:p_69015818905746-T-DNA, status:1, lane:L1, normal_id:[], n_fastq:4, data_type:cram, id:p_69015818905746-T-DNA] values=
- key=[patient:p_42680145080740, sample:p_42680145080740-T-DNA, status:1, lane:L1, normal_id:[], n_fastq:4, data_type:cram, id:p_42680145080740-T-DNA] values=
- key=[patient:p_62645628290215, sample:p_62645628290215-N-DNA, status:0, lane:L1, normal_id:[], n_fastq:4, data_type:cram, id:p_62645628290215-N-DNA] values=
- key=[patient:p_53832531377606, sample:p_53832531377606-T-DNA, status:1, lane:L1, normal_id:[], n_fastq:8, data_type:cram, id:p_53832531377606-T-DNA] values=
(more omitted)
WARN: Seqera Platform request field `workflow.errorMessage` exceeds expected size | offending value: `01:08:21.839 INFO ProgressMeter - Current Locus Elapsed Minutes Reads Processed R(truncated)`, size: 4555 (max: 255)
ERROR ~ Pipeline failed. Please refer to troubleshooting docs: https://nf-co.re/docs/usage/troubleshooting
-- Check '.nextflow.log' file for details
WARN: Failed to render execution report -- see the log file for details
WARN: Failed to render execution timeline -- see the log file for details
WARN: Seqera Platform request field `workflow.errorMessage` exceeds expected size | offending value: `01:08:21.839 INFO ProgressMeter - Current Locus Elapsed Minutes Reads Processed R(truncated)`, size: 4555 (max: 255)
#Java test:
(base) [3367243][vizu02][ieo7224@vizu02 ~]$ singularity exec \
/hpcnfs/data/DIMA/singularity_cache/community-cr-prod.seqera.io-docker-registry-v2-blobs-sha256-b2-b28daf5d9bb2f0d129dcad1b7410e0dd8a9b087aaf3ec7ced929b1f57624ad98-data.img \
bash
Singularity> which gatk
Illegal option --
Usage: /usr/bin/which [-as] args
Singularity> gatk --java-options "-XshowSettings:properties" BaseRecalibrator --help 2>&1 | grep -i classpath
Singularity> ps -ef | grep gatk
ieo7224 2326298 2326141 0 09:56 pts/5 00:00:00 grep gatk
Singularity> jshell
Jan 27, 2026 8:56:21 AM java.util.prefs.FileSystemPreferences$1 run
INFO: Created user preferences directory.
| Welcome to JShell -- Version 17.0.11-internal
| For an introduction type: /help intro
jshell> import org.apache.commons.lang3.math.NumberUtils;
| Error:
| package org.apache.commons.lang3.math does not exist
| import org.apache.commons.lang3.math.NumberUtils;
| ^---------------------------------------^
Description of the bug
Dear rnadnavar developers,
I think I found another bug regarding the GATK4_BASERECALIBRATOR image used to run that process.
The image that is present in the nextflow module is:
container "${ workflow.containerEngine == 'singularity' && !task.ext.singularity_pull_docker_container ?
'https://community-cr-prod.seqera.io/docker/registry/v2/blobs/sha256/b2/b28daf5d9bb2f0d129dcad1b7410e0dd8a9b087aaf3ec7ced929b1f57624ad98/data':
'community.wave.seqera.io/library/gatk4_gcnvkernel:e48d414933d188cd' }"
But apparently this image is missing the java library:
org.apache.commons.lang3.math
or
commons-lang3
and I verified this by running the image with singularity and then trying to import the library in java after running jshell
Because of this the pipeline crashed at the base recalibration step.
Attached below the complete error in nextflow + the nextflow log + the commands used to verify the lack of the common lang3 library
Command used and terminal output
Relevant files
nextflow.log
System information
No response