Skip to content

Make normalise work chunk by chunk - #46

Merged
tomwhite merged 2 commits into
sgkit-dev:mainfrom
tomwhite:normalise-chunked
Apr 20, 2026
Merged

tomwhite merged 2 commits into
sgkit-dev:mainfrom
tomwhite:normalise-chunked

Conversation

@tomwhite

Copy link
Copy Markdown
Member

Add unit test for remap_genotypes

Parameterize test_normalise with more values of variants_chunk_size

Update README with mention of vczstore normalise

Add unit test for remap_genotypes

Parameterize test_normalise with more values of variants_chunk_size

Update README with mention of vczstore normalise
@tomwhite

Copy link
Copy Markdown
Member Author

Part of #30

@jeromekelleher jeromekelleher left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM generally

Comment thread vczstore/normalise.py
data[local_idx, ...] = arr[match_sl, ...]

def remap_genotypes(gt, allele_mappings):
if var == "call_genotype":

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need to implement right away, but it's more general than just genotypes. We'd need to remap any field with dimensions alleles, alt_alleles, or genotypes, as well as local alleles. We should assert on finding any of these for now, I guess.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I've added a check for this and a test.

@tomwhite
tomwhite merged commit cb53a12 into sgkit-dev:main Apr 20, 2026
10 checks passed
@tomwhite
tomwhite deleted the normalise-chunked branch April 20, 2026 12:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants