Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
59fb35b
wip
mcovarr Nov 20, 2025
3f1ecfc
fixes
mcovarr Nov 21, 2025
11cc37c
wire everything up
mcovarr Nov 21, 2025
f722835
dockstore
mcovarr Nov 22, 2025
9acd0be
oops
mcovarr Nov 22, 2025
d183c11
oops
mcovarr Nov 22, 2025
3fbd180
vep cache
mcovarr Nov 22, 2025
ab10180
stage vep reference
mcovarr Nov 22, 2025
a61a206
try to get rid of ellipsis
mcovarr Nov 22, 2025
3844cc6
try preemptible
mcovarr Nov 22, 2025
c56cea6
oops
mcovarr Nov 22, 2025
d0cb27b
pull load out of loop to avoid rate limits
mcovarr Nov 22, 2025
b35a1ee
oops
mcovarr Nov 23, 2025
e52cd1b
oops
mcovarr Nov 23, 2025
3e4f224
oops
mcovarr Nov 23, 2025
e1ba7bb
update Docker
mcovarr Nov 23, 2025
3db29e3
big oops
mcovarr Nov 23, 2025
6d896dc
do not error on empty input
mcovarr Nov 23, 2025
e12f69f
fix empty file handling
mcovarr Nov 23, 2025
fbab0fb
oops
mcovarr Nov 23, 2025
8b318f2
maybe
mcovarr Nov 23, 2025
54f8333
geez
mcovarr Nov 24, 2025
76a6072
more
mcovarr Nov 24, 2025
58c0603
My robot friend catches and fixes my bugs
mcovarr Nov 25, 2025
2b884f0
attempt to fix coordinates
mcovarr Nov 26, 2025
e1e8a5e
fix parens
mcovarr Nov 26, 2025
491335a
oops
mcovarr Nov 26, 2025
d587750
whoops
mcovarr Nov 26, 2025
9009322
fix
mcovarr Nov 26, 2025
8aa3ba4
argh
mcovarr Nov 26, 2025
1df94e1
arghhhh
mcovarr Nov 26, 2025
41a46ac
more
mcovarr Nov 26, 2025
5b61a20
relax transcript version number matching
mcovarr Dec 2, 2025
aca846a
dockstore
mcovarr Dec 4, 2025
917e3b1
scale testing improvements: TTL on load tables, creating if necessary…
mcovarr Dec 5, 2025
b9786c2
update Docker
mcovarr Dec 5, 2025
c3bf9d5
monitoring log
mcovarr Dec 5, 2025
b53f12a
oops
mcovarr Dec 5, 2025
8e08539
adjust size expectations
mcovarr Dec 5, 2025
a9e55f6
adjusting memory
mcovarr Dec 5, 2025
4120841
more retries plz
mcovarr Dec 6, 2025
36b3a99
manually adjust memory
mcovarr Dec 7, 2025
c36a277
comments
mcovarr Dec 7, 2025
c42d390
adjust TTL, dedup
mcovarr Dec 8, 2025
67d0dda
data loading rework
mcovarr Dec 8, 2025
870487a
fix arg order
mcovarr Dec 9, 2025
d8d060a
adjust mem and disk back down
mcovarr Dec 9, 2025
e3f010f
Silence CodeQL JavaScript / TypeScript errors [VS-1777] (#9304)
mcovarr Dec 8, 2025
767fd8b
use ulimit to try to get retry with more memory working
mcovarr Dec 9, 2025
4e47b7d
a dash of logging
mcovarr Dec 9, 2025
846b3f4
no Python 3 on that image, only Python 2 but hey it works
mcovarr Dec 9, 2025
6625463
fix
mcovarr Dec 9, 2025
e5c0bd8
change ulimit arg as limiting -m seems to force super slow paging
mcovarr Dec 9, 2025
0c33013
revert ulimit stuff, vep just seems to hang immediately
mcovarr Dec 10, 2025
91525ed
force offline
mcovarr Dec 10, 2025
9c25376
noAddress
mcovarr Dec 10, 2025
e8366f8
copilot suggestions
mcovarr Dec 10, 2025
33358a5
VEP + LOFTEE Dockerfile and docs
mcovarr Dec 10, 2025
8803613
warnings files not always generated
mcovarr Dec 11, 2025
09d31ba
Revert "revert ulimit stuff, vep just seems to hang immediately"
mcovarr Dec 11, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .dockstore.yml
Original file line number Diff line number Diff line change
Expand Up @@ -240,6 +240,7 @@ workflows:
branches:
- master
- ah_var_store
- vs_1520_loftee
tags:
- /.*/
- name: GvsCreateVATFilesFromBigQuery
Expand Down Expand Up @@ -343,7 +344,7 @@ workflows:
branches:
- master
- ah_var_store
- vs_1739_cmrg_learnings
- vs_1520_loftee
tags:
- /.*/
- name: GvsIngestTieout
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -431,6 +431,48 @@
"type": "String",
"mode": "Nullable"
},
{
"description": "HGNC (HUGO Gene Nomenclature Committee) Symbol",
"name": "hgnc_symbol",
"type": "String",
"mode": "Nullable"
},
{
"description": "HGNC_ID (HUGO Gene Nomenclature Committee) Identifier",
"name": "hgnc_id",
"type": "Integer",
"mode": "Nullable"
},
{
"description": "LOFTEE Loss-of-function annotation (HC = High Confidence; LC = Low Confidence)",
"name": "LoF",
"type": "String",
"mode": "Nullable"
},
{
"description": "LOFTEE Reason for LoF not being HC",
"name": "LoF_filter",
"type": "String",
"mode": "Repeated"
},
{
"description": "LOFTEE Possible warning flags for LoF",
"name": "LoF_flags",
"type": "String",
"mode": "Repeated"
},
{
"description": "LOFTEE Info used for LoF annotation",
"name": "LoF_info",
"type": "String",
"mode": "Repeated"
},
{
"description": "GERP (Genomic Evolutionary Rate Profiling) Scores",
"name": "GERP",
"type": "Float",
"mode": "Repeated"
},
{
"description": "gnomAD: 'Total' frequency",
"name": "gnomad_all_af",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -431,6 +431,48 @@
"type": "String",
"mode": "Nullable"
},
{
"description": "HGNC (HUGO Gene Nomenclature Committee) Symbol",
"name": "hgnc_symbol",
"type": "String",
"mode": "Nullable"
},
{
"description": "HGNC_ID (HUGO Gene Nomenclature Committee) Identifier",
"name": "hgnc_id",
"type": "Integer",
"mode": "Nullable"
},
{
"description": "LOFTEE Loss-of-function annotation (HC = High Confidence; LC = Low Confidence)",
"name": "LoF",
"type": "String",
"mode": "Nullable"
},
{
"description": "LOFTEE Reason for LoF not being HC",
"name": "LoF_filter",
"type": "String",
"mode": "Repeated"
},
{
"description": "LOFTEE Possible warning flags for LoF",
"name": "LoF_flags",
"type": "String",
"mode": "Repeated"
},
{
"description": "LOFTEE Info used for LoF annotation",
"name": "LoF_info",
"type": "String",
"mode": "Repeated"
},
{
"description": "GERP (Genomic Evolutionary Rate Profiling) Scores",
"name": "GERP",
"type": "Float",
"mode": "Repeated"
},
{
"description": "gnomAD: 'Total' frequency",
"name": "gnomad_all_af",
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
[
{
"name": "vid",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "Uploaded_variation",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "Location",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "Allele",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "Gene",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "Feature",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "Feature_type",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "Consequence",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "cDNA_position",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "CDS_position",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "Protein_position",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "Amino_acids",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "Codons",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "Existing_variation",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "IMPACT",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "DISTANCE",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "STRAND",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "FLAGS",
"type": "STRING",
"mode": "REPEATED"
},
{
"name": "HGNC_SYMBOL",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "SYMBOL_SOURCE",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "HGNC_ID",
"type": "INTEGER",
"mode": "NULLABLE"
},
{
"name": "SOURCE",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "LoF",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "LoF_filter",
"type": "STRING",
"mode": "REPEATED"
},
{
"name": "LoF_flags",
"type": "STRING",
"mode": "REPEATED"
},
{
"name": "LoF_info",
"type": "STRING",
"mode": "REPEATED"
},
{
"name": "GERP",
"type": "FLOAT",
"mode": "REPEATED"
}
]
Original file line number Diff line number Diff line change
@@ -0,0 +1,132 @@
[
{
"name": "Uploaded_variation",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "Location",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "Allele",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "Gene",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "Feature",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "Feature_type",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "Consequence",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "cDNA_position",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "CDS_position",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "Protein_position",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "Amino_acids",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "Codons",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "Existing_variation",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "IMPACT",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "DISTANCE",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "STRAND",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "FLAGS",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "SYMBOL",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "SYMBOL_SOURCE",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "HGNC_ID",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "SOURCE",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "LoF",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "LoF_filter",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "LoF_flags",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "LoF_info",
"type": "STRING",
"mode": "NULLABLE"
},
{
"name": "GERP",
"type": "STRING",
"mode": "NULLABLE"
}
]
Loading