Merge master, fix unrelated VDS-making breakage [VS-1777] #9302
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… names on output for B37 aligned files (#8539)
* hmer ondel must have mon length * Revert "hmer ondel must have mon length" This reverts commit 7852871. * remove superfluous variant type condition * fix error message to actually reflect missing argument * fixed unittest to include variant type * Remove conflict
* Additional fix + logging fixes * Added missing initialization
New Tool: GroundTruthScorer Update: FlowFeatureMapper
…dependency on the ADAM library (#8606) * Add a native GATK implementation for 2bit references, with comprehensive unit tests * For now, this is only hooked up to the Spark codepath, but it could easily be hooked up to ReferenceDataSource and the Walker codepath as well * Remove the dependency on the ADAM library, to resolve conflicts with future dependency upgrades
…curity scanner to build.gradle (#8607) * Updated many GATK dependencies to address known security vulnerabilities * Added a security scanner to build.gradle * There are still some remaining vulnerabilities in GATK dependencies, but this eliminates most of them
* Update http-nio and wire it so it's configured at startup along with GCS setttings.
* New experimental tool to print out human readable file diagnostics for cram/crai/bai files.
…#8438) * GATK's lack of support for az:// URIs means that although GenomicsDB can natively read them, parts of the java code crash when interacting with them * Adding --avoid-nio and --header arguments These allow disabling all of the java interaction with the az:// links and simply passing them through to genomicsdb This disables some safeguards but allows operating on files in azur * Update GenomicsDB version to 1.5.1 for azure improved support * There are no direct tests on azure since we do not yet have any infrastructure to generate the necessary tokens, there is a disabled test which requires #8612 before we can enable it. --------- Co-authored-by: Nalini Ganapati <[email protected]> Co-authored-by: Nalini Ganapati <[email protected]>
For having variable ploidy in different regions, like making haploid calls outside the PAR on chrX or chrY, there is now a --ploidy-regions flag. The -ploidy flag sets the default ploidy to use everywhere, and --ploidy-regions should be a .bed or .interval_list with "name" column containing the desired ploidy to use in that region when genotyping. Note that variants near the boundary may not have the matching ploidy since the ploidy used will be determined using the following precedence: * ploidy given in --ploidy-regions for all intervals overlapping the active region when calling your variant with ties broken by using largest ploidy); note ploidy interval may only overlap the active region and determine the ploidy of your variant even if the end coordinate written for your variant lies outside the given region * ploidy given via global -ploidy flag * ploidy determined by the default global built-in constant for humans (2). --------- Co-authored-by: Ty Kay <[email protected]> Co-authored-by: rickymagner <[email protected]>
* Update the GATK base image to the latest Ubuntu LTS release (22.04) * Add some additional useful utilities to the base image * Switch to a newer conda version with a much faster solver * Update the scripts and documentation for building the base image * Update the VETS integration tests to allow for a small epsilon during numeric comparisons, and include the full diff output in exception messages when a mismatch is detected
…oud-based docker build, and add a release script (#8247) * Added a -r argument to build_docker_remote.sh to toggle the RELEASE flag during docker builds * Added a release_prebuilt_docker_image.sh to release a prebuilt docker image to the official repos
* update to htsjdk 4.1.0 which enables http-nio in more cases * remove several test cases handling genomicsdb path parsing which were testing nonsensical paths that are now illegal
…nstant in build.gradle (#8625)
* This should make http access seamless in many places * The way this handles query parameters is not ideal for signed url cases so we'll need to revisit that
…ervals output (#8621) * Write gCNV interval output ID=GT header as Type=String Incorrectly writing this as Type=Integer causes bcftools to misparse the genotype field. * Use correct header types and numbers in test VCF file
* include normal seq error log likelihood in Permutect dataset * handle different alelle representations in multiallelic / indel variants for Permutect training data mode * set the default artifact to non-artifact ratio to 1 in Permutect training data mode
* Add SQ threshold for DRAGEN-derived somatic GVCFs, featuring tests with generously provided user data
Bumps commons-beanutils:commons-beanutils from 1.9.4 to 1.11.0. --- updated-dependencies: - dependency-name: commons-beanutils:commons-beanutils dependency-version: 1.11.0 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
- Correctly remove the version suffix from transcripts that include an underscore character - Allow ENSEMBL GTF files to use the --prefer-mane-transcripts option
* htsjdk 4.2.0 -> 4.3.0 * enables cram 3.1
* Check whether a variant is contained within at least one MANE transcript before returning the list of selected transcripts to avoid a "no_transcript" output * Add MANE status comparator for transcript selection
…9231) * Draft a tool for converting a counts type file to a depth type file. * Update expected integration test files. * Improve argument definition & make sample label optional. * Improve integration test. * Update expected outputs of the integration test. * Add tool & argument documentation, & clean up. * Fix a typo. * Fix a typo. * Refactor variables & add tool documentation. * Add the class attribute needed for doc generation.
- Updated GKL dependency from v0.9.0 to v0.9.1 for a small but important bug fix for the Smith-Waterman implementation (moving a static global variable `D_MAX_SEQ_LEN` to an internal scope to avoid potential conflicts).
* CodeQL was enabled on all public Broad repositories. Since this repository contains no JavaScript / TypeScript, CodeQL always fails. Because of this we have to add a workaround here so it does not fail and our checks all come up nice and green.
mcovarr
force-pushed
the
vs_1777_build_failure
branch
from
3ff1a7d to
99c5b26
Compare
mcovarr
changed the title
There was a problem hiding this comment.
Pull request overview
This PR merges changes from master and fixes VDS-making breakage related to ticket VS-1777. The primary focus is on fixing issues with structural variant stratification and adding support for MANE transcript preference in Funcotator.
Key changes:
- Fixed TabixFormat parameters in multiple evidence codec files (changed 4th parameter from 0 to 2)
- Added a new test for SVStratify to handle redundant configurations without failing
- Enhanced Funcotator to support MANE transcript preferences with proper variant context checks
- Added new ConvertCountsToDepthFile tool for converting count files to depth evidence format
Reviewed changes
Copilot reviewed 23 out of 27 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/test/java/org/broadinstitute/hellbender/tools/walkers/sv/SVStratifyIntegrationTest.java | Split redundant test into passing and failing cases, added assertions for output validation |
| src/test/java/org/broadinstitute/hellbender/tools/sv/ConvertCountsToDepthFileIntegrationTest.java | New integration test for counts to depth file conversion |
| src/test/java/org/broadinstitute/hellbender/tools/funcotator/dataSources/gencode/GencodeFuncotationFactoryUnitTest.java | Updated tests to pass VariantContext and preferMANETranscripts parameter |
| src/main/resources/placate_codeql.js | Added JavaScript file to prevent CodeQL analysis failures |
| src/main/java/org/broadinstitute/hellbender/utils/codecs/*.java | Fixed TabixFormat end position parameter from 0 to 2 |
| src/main/java/org/broadinstitute/hellbender/tools/walkers/sv/SVStratify.java | Enhanced to support non-split output mode and unbounded STRATUM_INFO_KEY |
| src/main/java/org/broadinstitute/hellbender/tools/walkers/haplotypecaller/*.java | Added activeRegionAltMultiplier parameter for weighted active region detection |
| src/main/java/org/broadinstitute/hellbender/tools/sv/stratify/SVStatificationEngine.java | Deleted obsolete file (renamed to SVStratificationEngine) |
| src/main/java/org/broadinstitute/hellbender/tools/sv/*.java | Added DocumentedFeature annotations and new ConvertCountsToDepthFile tool |
| src/main/java/org/broadinstitute/hellbender/tools/funcotator/**/*.java | Added MANE transcript tracking and variant-context-aware filtering |
| scripts/variantstore/wdl/*.wdl | Updated Docker image versions and added apt-get update |
| build_docker_remote.sh | Removed duplicate cloudbuild.yaml steps block |
| .dockstore.yml | Updated branch reference from vs_1739_cmrg_learnings to vs_1777_build_failure |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| final String args = | ||
| " --" + ConvertCountsToDepthFile.COUNTS_FILE_PATH_ARG_FULL_NAME + " " + testCase.inputFilename + | ||
| " --" + StandardArgumentDefinitions.SEQUENCE_DICTIONARY_NAME + " " + FULL_HG38_DICT + | ||
| ((testCase.inputFilename == null) ? |
There was a problem hiding this comment.
The condition checks testCase.inputFilename == null but should check testCase.sampleLabel == null. When sampleLabel is null, the sample name argument should be omitted to test reading from the file header.
Suggested change
| ((testCase.inputFilename == null) ? | |
| ((testCase.sampleLabel == null) ? |
Comment on lines
131
to
132
| echo "- name: 'gcr.io/cloud-builders/docker'" >> cloudbuild.yaml | ||
| echo " args: [ 'push', '${DOCKER_IMAGE_TAG}' ]" >> cloudbuild.yaml |
There was a problem hiding this comment.
Duplicate block of cloudbuild.yaml steps was removed, but the remaining block doesn't include the build step. The removed lines 134-144 contained the build command that should have been kept, while lines 131-132 (the push-only command) should have been removed instead.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Master merge + fix in VDS-making. Mostly successful integration run here, delta some angry cloud issues with pulling Docker images.