Repeat runs do not produce reproducible results

[calizilla]

I ran ctat-lr-fusion v 1.3.0 on the same BAM file with identical script 3 times, only changing the output path between runs. I did this first at 12 CPU and then at 1 CPU, 6 runs in total. Each run has slightly different results. Differences occur at both the fusion-ID level as well as breakpoint positon within the same fusion IDs.

For example a simple line count shows a different number of final predictions for each of the runs:

   541 ./TEST-NONPATCH-v1-3-0-run1-1CPU/ctat-LR-fusion.fusion_predictions.tsv
   539 ./TEST-NONPATCH-v1-3-0-run2-1CPU/ctat-LR-fusion.fusion_predictions.tsv
   546 ./TEST-NONPATCH-v1-3-0-run3-1CPU/ctat-LR-fusion.fusion_predictions.tsv
   539 ./TEST-NONPATCH-v1-3-0-run2-12CPU/ctat-LR-fusion.fusion_predictions.tsv
   542 ./TEST-NONPATCH-v1-3-0-run1-12CPU/ctat-LR-fusion.fusion_predictions.tsv
   539 ./TEST-NONPATCH-v1-3-0-run3-12CPU/ctat-LR-fusion.fusion_predictions.tsv

Comparing with jvenn to show the differences, looking only at fusion IDs (ie not considering fusion IDs that may be the same but have different positions, so this is an underestiamte of the differences between runs):

I could not find a flag or parameter within the tool documentation to control for this, eg a starting seed value. Please correct me if I am mistaken.

It would be great if tool authors could advise how we can obtain reproducible runs from this tool.

Many thanks
Cali

[brianjohnhaas]

Hi, Is the data set you're testing with something you can share? I could use it to explore this variability. Or, if you want to compare the size of the different intermediate output files, it might help point to which step is contributing to it. I'm away till next week, but can look into it when I return. best, Brian

…

On Thu, Feb 19, 2026 at 7:28 PM calizilla ***@***.***> wrote: *calizilla* created an issue (TrinityCTAT/CTAT-LR-fusion#18) <#18> I ran ctat-lr-fusion v 1.3.0 on the same BAM file with identical script 3 times, only changing the output path between runs. I did this first at 12 CPU and then at 1 CPU, 6 runs in total. Each run has slightly different results. Differences occur at both the fusion-ID level as well as breakpoint positon within the same fusion IDs. For example a simple line count shows a different number of final predictions for each of the runs: 541 ./TEST-NONPATCH-v1-3-0-run1-1CPU/ctat-LR-fusion.fusion_predictions.tsv 539 ./TEST-NONPATCH-v1-3-0-run2-1CPU/ctat-LR-fusion.fusion_predictions.tsv 546 ./TEST-NONPATCH-v1-3-0-run3-1CPU/ctat-LR-fusion.fusion_predictions.tsv 539 ./TEST-NONPATCH-v1-3-0-run2-12CPU/ctat-LR-fusion.fusion_predictions.tsv 542 ./TEST-NONPATCH-v1-3-0-run1-12CPU/ctat-LR-fusion.fusion_predictions.tsv 539 ./TEST-NONPATCH-v1-3-0-run3-12CPU/ctat-LR-fusion.fusion_predictions.tsv Comparing with jvenn <https://jvenn.toulouse.inrae.fr/app/index.html> to show the differences, looking only at fusion IDs (ie not considering fusion IDs that may be the same but have different positions, so this is an underestiamte of the differences between runs): image.png (view on web) <https://github.com/user-attachments/assets/66e4881f-b843-4410-a3b8-3ef9c5bc2490> I could not find a flag or parameter within the tool documentation to control for this, eg a starting seed value. Please correct me if I am mistaken. It would be great if tool authors could advise how we can obtain reproducible runs from this tool. Many thanks Cali — Reply to this email directly, view it on GitHub <#18>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABZRKX2BOFGXQUJO4CWTVND4MZIKNAVCNFSM6AAAAACVYOUFS2VHI2DSMVQWIX3LMV43ASLTON2WKOZTHE3DKOBVG4YTGOI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

-- -- Brian J. Haas The Broad Institute http://broadinstitute.org/~bhaas <http://broad.mit.edu/~bhaas>

[calizilla]

Hi Brian

Thanks for your fast reply. I can't share the test data as it's patient data, but I expect the same results could be generated using any input BAM produced with the --eqx minimap2 flag.

The command for each run:

REF_DIR=./ctat_LR-fusion/GRCh38_gencode_v44_CTAT_lib_Oct292023.plug-n-play/ctat_genome_lib_build_dir
IMAGE=./ctat_lr_fusion_v1.3.0.sif

singularity run ${IMAGE} \
        ctat-LR-fusion \
        --genome_lib_dir ${REF_DIR} \
        --LR_bam ${BAM} \
        --prep_reference \
        --CPU ${PBS_NCPUS} \
        --vis \
        --output ${OUTDIR}

I am not sure which specific step within the codebase produces the predictions.tsv file, I took this to be the main results file from the collection of outputs from the above run command:

-rw-r--r-- 39M Feb 14 22:32 sample.fasta.gz
-rw-r--r-- 488 Feb 14 22:35 sample.bam.primary_read_counts
-rw-r--r-- 1.7M Feb 14 22:55 ctat-LR-fusion.fusion_predictions.preliminary.tsv
-rw-r--r-- 1.3M Feb 14 22:55 ctat-LR-fusion.fusion_predictions.preliminary.abridged.tsv
-rw-r--r-- 128K Feb 14 22:55 ctat-LR-fusion.fusion_predictions.tsv
-rw-r--r-- 76K Feb 14 22:55 ctat-LR-fusion.fusion_predictions.abridged.tsv
drwxr-sr-x 4.0K Feb 14 22:56 fusion_intermediates_dir
-rw-r--r-- 137M Feb 14 22:57 ctat-LR-fusion.fusion_inspector_web.html

[brianjohnhaas]

Hi Cali, Thanks for the info. When I get back next week, I'll run some tests and try to figure out where the issue is coming from. Will be in touch. best, Brian

…

On Thu, Feb 19, 2026 at 8:28 PM calizilla ***@***.***> wrote: *calizilla* left a comment (TrinityCTAT/CTAT-LR-fusion#18) <#18 (comment)> Hi Brian Thanks for your fast reply. I can't share the test data as it's patient data, but I expect the same results could be generated using any input BAM produced with the --eqx minimap2 flag. The command for each run: REF_DIR=./ctat_LR-fusion/GRCh38_gencode_v44_CTAT_lib_Oct292023.plug-n-play/ctat_genome_lib_build_dir IMAGE=./ctat_lr_fusion_v1.3.0.sif singularity run ${IMAGE} \ ctat-LR-fusion \ --genome_lib_dir ${REF_DIR} \ --LR_bam ${BAM} \ --prep_reference \ --CPU ${PBS_NCPUS} \ --vis \ --output ${OUTDIR} I am not sure which specific step within the codebase produces the predictions.tsv file, I took this to be the main results file from the collection of outputs from the above run command: -rw-r--r-- 39M Feb 14 22:32 sample.fasta.gz -rw-r--r-- 488 Feb 14 22:35 sample.bam.primary_read_counts -rw-r--r-- 1.7M Feb 14 22:55 ctat-LR-fusion.fusion_predictions.preliminary.tsv -rw-r--r-- 1.3M Feb 14 22:55 ctat-LR-fusion.fusion_predictions.preliminary.abridged.tsv -rw-r--r-- 128K Feb 14 22:55 ctat-LR-fusion.fusion_predictions.tsv -rw-r--r-- 76K Feb 14 22:55 ctat-LR-fusion.fusion_predictions.abridged.tsv drwxr-sr-x 4.0K Feb 14 22:56 fusion_intermediates_dir -rw-r--r-- 137M Feb 14 22:57 ctat-LR-fusion.fusion_inspector_web.html — Reply to this email directly, view it on GitHub <#18 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABZRKX7RLV72WSGRFQCZ5P34MZPLTAVCNFSM6AAAAACVYOUFS2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTSMZRGA3DGMJTGU> . You are receiving this because you commented.Message ID: ***@***.***>

-- -- Brian J. Haas The Broad Institute http://broadinstitute.org/~bhaas <http://broad.mit.edu/~bhaas>

[brianjohnhaas]

Hi Cali, I tracked down some sources of non-determinism and verified the fix on one of our large data sets. I put out a new release: https://github.com/TrinityCTAT/CTAT-LR-fusion/releases/tag/ctat-LR-fusion-v1.4.0 Please let me know if you still encounter anything unexpected. best, Brian

…

On Thu, Feb 19, 2026 at 9:54 PM Brian Haas ***@***.***> wrote: Hi Cali, Thanks for the info. When I get back next week, I'll run some tests and try to figure out where the issue is coming from. Will be in touch. best, Brian On Thu, Feb 19, 2026 at 8:28 PM calizilla ***@***.***> wrote: > *calizilla* left a comment (TrinityCTAT/CTAT-LR-fusion#18) > <#18 (comment)> > > Hi Brian > > Thanks for your fast reply. I can't share the test data as it's patient > data, but I expect the same results could be generated using any input BAM > produced with the --eqx minimap2 flag. > > The command for each run: > > REF_DIR=./ctat_LR-fusion/GRCh38_gencode_v44_CTAT_lib_Oct292023.plug-n-play/ctat_genome_lib_build_dir > IMAGE=./ctat_lr_fusion_v1.3.0.sif > > singularity run ${IMAGE} \ > ctat-LR-fusion \ > --genome_lib_dir ${REF_DIR} \ > --LR_bam ${BAM} \ > --prep_reference \ > --CPU ${PBS_NCPUS} \ > --vis \ > --output ${OUTDIR} > > I am not sure which specific step within the codebase produces the > predictions.tsv file, I took this to be the main results file from the > collection of outputs from the above run command: > > -rw-r--r-- 39M Feb 14 22:32 sample.fasta.gz > -rw-r--r-- 488 Feb 14 22:35 sample.bam.primary_read_counts > -rw-r--r-- 1.7M Feb 14 22:55 ctat-LR-fusion.fusion_predictions.preliminary.tsv > -rw-r--r-- 1.3M Feb 14 22:55 ctat-LR-fusion.fusion_predictions.preliminary.abridged.tsv > -rw-r--r-- 128K Feb 14 22:55 ctat-LR-fusion.fusion_predictions.tsv > -rw-r--r-- 76K Feb 14 22:55 ctat-LR-fusion.fusion_predictions.abridged.tsv > drwxr-sr-x 4.0K Feb 14 22:56 fusion_intermediates_dir > -rw-r--r-- 137M Feb 14 22:57 ctat-LR-fusion.fusion_inspector_web.html > > — > Reply to this email directly, view it on GitHub > <#18 (comment)>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/ABZRKX7RLV72WSGRFQCZ5P34MZPLTAVCNFSM6AAAAACVYOUFS2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTSMZRGA3DGMJTGU> > . > You are receiving this because you commented.Message ID: > ***@***.***> > -- -- Brian J. Haas The Broad Institute http://broadinstitute.org/~bhaas <http://broad.mit.edu/~bhaas>

-- -- Brian J. Haas The Broad Institute http://broadinstitute.org/~bhaas <http://broad.mit.edu/~bhaas>

[calizilla]

Hi Brian, thanks so much for your prompt resolution of this issue and the new release including extended CIGAR support. We are using CTAT-LR in a rare disease project so this is wonderful. My colleague @amarinderthind has tested the new version on our data and the results from 3 duplicate runs are identical.

[brianjohnhaas]

Great to hear! all the best, Brian

…

On Wed, Mar 11, 2026 at 8:25 PM calizilla ***@***.***> wrote: *calizilla* left a comment (TrinityCTAT/CTAT-LR-fusion#18) <#18 (comment)> Hi Brian, thanks so much for your prompt resolution of this issue and the new release including extended CIGAR support. We are using CTAT-LR in a rare disease project so this is wonderful. My colleague @amarinderthind <https://github.com/amarinderthind> has tested the new version on our data and the results from 3 duplicate runs are identical. — Reply to this email directly, view it on GitHub <#18 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABZRKXY62ZTS4PXIOR5L4I34QH7W5AVCNFSM6AAAAACVYOUFS2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHM2DANBTGA3DQMZTGQ> . You are receiving this because you commented.Message ID: ***@***.***>

-- -- Brian J. Haas The Broad Institute http://broadinstitute.org/~bhaas <http://broad.mit.edu/~bhaas>