Skip to content

[2/5] Refactor: make the fragment parsing helpers private - #435

Merged
GeorgWa merged 13 commits into
mainfrom
refactor/private-fragment-parsing-helpers
Sep 15, 2026
Merged

GeorgWa merged 13 commits into
mainfrom
refactor/private-fragment-parsing-helpers

Conversation

@GeorgWa

@GeorgWa GeorgWa commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

Pure refactor. Rename only, no functional change.

parse_fragment, fill_in_indices and calculate_fragment_numbers only serve flatten_fragments, and nothing in alphabase, alphadia or peptdeep calls them. A leading underscore states that, and frees their signatures for the refactoring that follows.

Stacked on #434. Splitting #429.

🤖 Generated with Claude Code

GeorgWa and others added 2 commits August 31, 2026 15:56
flatten_fragments had no unit test. Add black-box tests that pin its contract, so
the refactoring that follows can be checked step by step.

`_dense_library` builds dense frames with mz == 0 padding from both sources that
produce it: unmodified precursors carry no modloss fragments, and some fragments
fall outside the mz range. Intensities are distinct, so the top-k selection is
unambiguous.

`_expected_keep_mask` derives the expected mask per precursor, without
flatten_fragments. The tests cover the kept fragments over five combinations of
`keep_top_k_fragments` and `min_fragment_intensity`, every annotation column, the
reannotated precursor pointers, `custom_df` and `custom_columns`, the path
without intensities, an empty library, and a precursor long enough to need more
than a uint8 fragment number.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`parse_fragment`, `fill_in_indices` and `calculate_fragment_numbers` only serve
`flatten_fragments`, and nothing in alphabase, alphadia or peptdeep calls them.
A leading underscore states that, and frees their signatures for the refactoring
that follows.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@GeorgWa GeorgWa changed the title Refactor: make the fragment parsing helpers private [2/5] Refactor: make the fragment parsing helpers private Aug 31, 2026

@mschwoer mschwoer left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

number_of_fragment_types: int,
) -> Tuple[np.ndarray, np.ndarray, np.ndarray]:
"""
Parse fragments to get fragment numbers, fragment positions and not top k excluded indices in one hit

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"fragment numbers" is kinda ambigous.. is this counts? or enumeration? not sure if it should be renamed now (depends on the radius of this change), but at last a comment would help :-)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, it's the MS convention fragment numbering. I will add a comment :D

@mo-sameh

mo-sameh commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

PS: I'll be creating a PR soon to optimize the translation modules (translate.py, translate_diann.py). They each have their own fragment-flattening code (merge_precursor_fragment_df) that does very similar task as flatten_fragments, just emitting the type/loss/charge columns as strings for DIA-NN/Spectronaut rather than numeric codes.

I think it will be better to reuse the much more optimized helpers here instead.
So maybe worth keeping them public? Not a blocker, just flagging since there'll be a caller shortly.

@GeorgWa

GeorgWa commented Sep 15, 2026

Copy link
Copy Markdown
Collaborator Author

PS: I'll be creating a PR soon to optimize the translation modules (translate.py, translate_diann.py). They each have their own fragment-flattening code (merge_precursor_fragment_df) that does very similar task as flatten_fragments, just emitting the type/loss/charge columns as strings for DIA-NN/Spectronaut rather than numeric codes.

I think it will be better to reuse the much more optimized helpers here instead. So maybe worth keeping them public? Not a blocker, just flagging since there'll be a caller shortly.

very nice!

GeorgWa and others added 10 commits September 15, 2026 09:57
_expected_keep_mask reimplemented the filtering logic, so it could agree
with a broken flatten_fragments. Add a minimal_library fixture of two
precursors over a forward and a reverse fragment type, with the kept
slots written out per filter setting, and assert both the helper and
flatten_fragments against it.

Derive the long precursor row count from the uint8 boundary it tests
instead of picking a number near the fill_in_indices limit, and guard
the limit explicitly so exceeding it fails on an assertion rather than
on a broadcast error inside numba.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
"fragment number" read as ambiguous between a count and an enumeration.
Spell out that it is the MS series numbering, and contrast it with the
fragment position that all series share.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The library was generated from a seeded rng and the expectations came
from _expected_keep_mask, which reimplemented the filtering logic and so
could agree with a broken flatten_fragments.

Replace both with one library fixture over hand-written mz and intensity
grids, and hand-written keep masks per filter setting. Two fragment types
carry the whole annotation surface: y_z1 is reverse without loss and
b_modloss_z1 is forward with loss, so direction, series and loss all vary
over two columns.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Comparing column by column left the column set, the column order and the
dtypes unchecked. Assert the whole annotated frame against an expected
frame instead, and fold the row count of the filter cases into a frame
comparison of mz and intensity.

type and charge come out as int8, so the expected frame spells that out.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The merged tests named fill_in_indices, which this branch renamed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Replace the remaining column comparisons. Each one now also pins the
column set, the column order and the dtypes:

- without_intensity subsumes its separate missing-column assertion
- selects_custom_columns checked only the column names
- filters_custom_df_columns now pins where cardinality lands
- reannotates_precursor_pointers compares hardcoded flat pointers
  instead of reslicing mz per precursor, which was the last place the
  tests recomputed an expectation
- empty_precursor_df compares both frames instead of their lengths

Five tests assert against the same unfiltered library, so it moves into
a flat_unfiltered fixture.

test_flatten_fragments_long_precursor keeps its property assertions. It
builds 257 rows to push fragment numbers past 255, so a frame of 1542
generated values would bury what it tests.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The four scalar assertions become two equality comparisons. The dtypes
need their own comparison, because .max() over columns of different
dtypes upcasts to a common one and would hide a narrowed column.

Also drop the tuple rebuild in the parse test, expected is already the
tuple being compared.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Base automatically changed from test/flatten-fragments-characterization to main September 15, 2026 09:52
@GeorgWa
GeorgWa requested a review from mschwoer September 15, 2026 09:53
Main carries PR #434, the characterization tests of this stack, so the
test file arrives identical apart from the helper names this branch
makes private. Kept the private names in both conflicts.

Main also dropped Python 3.8 and rewrote the typing generics of the whole
file. The annotations of the extracted helpers came from main unchanged,
so nothing here still references the removed typing imports.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@GeorgWa
GeorgWa merged commit 52e5821 into main Sep 15, 2026
5 checks passed
@GeorgWa
GeorgWa deleted the refactor/private-fragment-parsing-helpers branch September 15, 2026 11:44
GeorgWa pushed a commit that referenced this pull request Sep 15, 2026
Main carries PR #435, so the helper renames of this stack arrive from
main and the test file now matches it exactly.

One conflict in flatten_fragments. Main still builds frag_directions
inline and passes it to _parse_fragment, while this branch takes the
directions from _annotate_charged_frag_types. Kept this branch's call,
since the variable main passes no longer exists here.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants