feat(extractor): expose the raw font resource tag on TextItem - #449
Merged
Conversation
1.16.0 replaced TextItem::font's resource tag with the /BaseFont family name (#415). That is the right default, but it erased the only signal that distinguishes two font programs sharing a family: consumers that partition items by font program (style grouping, training-time feature parity) lost their key when same-family subsets merged. Add font_tag, carrying exactly what font carried before 1.16.0: the raw per-page resource tag, first-run's tag on merged items, empty for items that don't come from a content-stream show operator (images, links, form fields, OCR). Both content-stream parsers stamp it from the same resource name, derived items (numeric splits, merges) propagate it, and it is exposed through --items-json and the Node/Python bindings.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
All reported issues were addressed across 31 files
Shadow auto-approve: would not auto-approve because issues were found.
Fix all with cubic | Re-trigger cubic
Member
Author
|
@abimaelmartell I have started the AI code review. It will take a few minutes to complete. |
There was a problem hiding this comment.
No issues found across 31 files
Shadow auto-approve: would require human review. This PR adds a font_tag field to TextItem to restore data parity. It is a source-breaking change for Rust crate consumers and modifies the public API contract across multiple bindings.
Re-trigger cubic
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
1.16.0 replaced
TextItem::font's resource tag with the/BaseFontfamily name (#415). That is the right default, but it erased the only signal distinguishing two font programs that share a family name — consumers that partition items by font program (style grouping, feature parity with pre-1.16.0 extractions) lost their key when same-family subsets merged.This adds a
font_tagfield carrying exactly whatfontcarried before 1.16.0:F2,T22), stamped by both content-stream parsers from the same resource nameEMCfor ActualText spans, matching the siblingfont/is_bold/is_italic/font_sizefields, which all sample the same state at the same point — legacy parity is the contract, so any change to that sampling must move all five fields togetherExposed through
--items-json, the Node binding (fontTag), and the Python binding (font_tag+.pyistub). The wasm binding doesn't surface positioned items, so no change there.Compatibility
TextItemliterally or destructure it exhaustively (field readers are unaffected).TextItemis an output type, so literal construction outside tests is expected to be rare, but the release that ships this should be versioned with that caveat in mind.Notes
items_carry_family_name_and_resource_tag,form_items_carry_family_name_and_resource_tag).