feat(markdown): opt-in inline-formula recovery as LaTeX from native math glyphs - #414
Draft
abimaelmartell wants to merge 2 commits into
Draft
feat(markdown): opt-in inline-formula recovery as LaTeX from native math glyphs#414abimaelmartell wants to merge 2 commits into
abimaelmartell wants to merge 2 commits into
Conversation
…ath glyphs
Digital TeX and word-processor PDFs draw formulas as native text, with
sub/superscripts encoded purely by baseline offset and font-size drop,
and math fonts (CMMI, CMSY, MSBM, OpenType Math) self-identifying when
producers preserve family names. Rewrite such glyph runs into $...$
LaTeX during markdown conversion — no recognition model, no downloads,
no inference cost. Font names disambiguate alphabets an image model has
to guess: MSBM is \mathbb, CMSY capitals are \mathcal, EUFM is
\mathfrak. Off by default behind MarkdownOptions::formula_latex and
pdf2md --formula-latex.
Detection: items classify as strong (math font, or majority mapped math
symbols excluding prose punctuation like dot-leader ellipses and
footnote daggers), connective (digits, brackets, relations, short
italic identifiers, and mixed identifier-punctuation like "F(q,r)=" —
join but never anchor), or prose (terminates the run). Detached
script-fragment lines are stitched back to their base line first: the
line grouper's 3pt baseline tolerance is smaller than TeX's superscript
rise, so scripts often arrive as their own lines, superscripts sorting
before their base in reading order.
Reconstruction: dominant baseline and size by median; smaller items
raised >=0.25em become ^{}, dropped >=0.12em become _{}; adjacent
same-kind script groups merge (E₄ then a geometric x4 gives E_{4x4},
never the invalid E_{4}_{4x4}); standalone combining accents wrap the
preceding glyph (\hat{Q}, \hat{\alpha}); trailing sentence
punctuation moves outside the delimiters.
Confidence gating keeps it conservative — a wrong $...$ is worse than
none. Rejections: stacked structure (fractions, matrices, bounded big
operators), unbalanced delimiters, dangling accents, unmapped glyphs,
LaTeX-active ASCII. Rewritten items drop emphasis flags so markdown
markers cannot split the delimiters.
Why opt-in: on olmOCR-bench's render-equivalence math tests the flat
reconstruction converts 2.4% of arxiv_math (from zero — both this and
other local parsers score 0 without LaTeX output), while LaTeXifying
text costs fuzzy text-presence tests more than the math gains. The
default flips when net-positive. Known next steps: resolve BaseFont
family names into extraction (item.font is often an opaque resource
name like "F2", so font anchoring rarely fires today), and assemble
display equations across full lines rather than intra-line runs.
The unicode map extends #42's with codepoint-variant siblings (MICRO
SIGN for \mu, MINUS SIGN, double-struck and script letters) and ~60
additional relations and arrows. Supersedes the approach in #40/#42:
detection is native and inline, and emission goes into the markdown
stream rather than a separate API.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
9 issues found across 6 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/markdown/convert.rs">
<violation number="1" location="src/markdown/convert.rs:707">
P2: When `formula_latex` is enabled, rewriting runs before heading discovery lets isolated or large display equations pass the heading heuristics and emit `# $...$` instead of equation content. Exclude display-equation lines from every heading-promotion path, while retaining short expressions such as `A + B` and `(R-12)`.
(Based on your team's feedback about gating equation heading promotion.) .</violation>
</file>
<file name="src/formula/mod.rs">
<violation number="1" location="src/formula/mod.rs:428">
P2: An unmapped alphabetic glyph can still pass the confidence gate and become LaTeX. Give unmapped letters a rejecting penalty, or otherwise reject the reconstruction whenever one is encountered.</violation>
<violation number="2" location="src/formula/mod.rs:485">
P2: This rejects every oversized big operator, including valid unbounded operators with no limits. Detect the presence of unsupported bounds before applying the rejection penalty.</violation>
<violation number="3" location="src/formula/mod.rs:536">
P1: When a mapped command is followed by a geometrically separated item, reconstruction emits two spaces because the command already has a guard space. Add the geometric gap only when `latex` does not already end in whitespace.</violation>
<violation number="4" location="src/formula/mod.rs:616">
P2: A detached line is stitched without requiring a genuine script offset. Require a nontrivial baseline displacement and a tighter upper bound before merging it into the formula.</violation>
</file>
<file name="src/formula/unicode_map.rs">
<violation number="1" location="src/formula/unicode_map.rs:113">
P2: The second U+22A5 entry overwrites the earlier `\bot` mapping, so the map loses the UP TACK form and never maps U+27C2. Use U+27C2 for the `\perp` entry.</violation>
<violation number="2" location="src/formula/unicode_map.rs:196">
P2: In prose units such as `5 µm`, this mapping can anchor a formula run and rewrite the unit as `$5 \mu m$`. Make MICRO SIGN non-anchoring or reject unit context before using it as a math anchor.</violation>
<violation number="3" location="src/formula/unicode_map.rs:240">
P2: When formula recovery sees `∟`, this entry emits generic `\angle` and changes the symbol's meaning and shape. Preserve the Unicode glyph or use a renderer-supported right-angle representation instead of aliasing it to U+2220.</violation>
<violation number="4" location="src/formula/unicode_map.rs:281">
P2: When formula recovery sees `⋕`, this mapping emits `\#`, which is a number sign rather than EQUAL AND PARALLEL TO. Remove the unsupported mapping or replace it with a representation of that relation.</violation>
</file>
Shadow auto-approve: would not auto-approve because issues were found.
Tip: instead of fixing issues one by one fix them all with cubic
Re-trigger cubic
| if role == ScriptRole::Base && previous_role == ScriptRole::Base { | ||
| if let Some(end_x) = previous_end_x { | ||
| if item.x - end_x > item.font_size * 0.2 && !latex.is_empty() { | ||
| latex.push(' '); |
There was a problem hiding this comment.
P1: When a mapped command is followed by a geometrically separated item, reconstruction emits two spaces because the command already has a guard space. Add the geometric gap only when latex does not already end in whitespace.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/formula/mod.rs, line 536:
<comment>When a mapped command is followed by a geometrically separated item, reconstruction emits two spaces because the command already has a guard space. Add the geometric gap only when `latex` does not already end in whitespace.</comment>
<file context>
@@ -0,0 +1,1115 @@
+ if role == ScriptRole::Base && previous_role == ScriptRole::Base {
+ if let Some(end_x) = previous_end_x {
+ if item.x - end_x > item.font_size * 0.2 && !latex.is_empty() {
+ latex.push(' ');
+ }
+ }
</file context>
|
|
||
| // Rewrite confident native math glyph runs into $...$ LaTeX | ||
| let lines = if options.formula_latex { | ||
| crate::formula::rewrite_math_runs(lines) |
There was a problem hiding this comment.
P2: When formula_latex is enabled, rewriting runs before heading discovery lets isolated or large display equations pass the heading heuristics and emit # $...$ instead of equation content. Exclude display-equation lines from every heading-promotion path, while retaining short expressions such as A + B and (R-12).
(Based on your team's feedback about gating equation heading promotion.) .
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/markdown/convert.rs, line 707:
<comment>When `formula_latex` is enabled, rewriting runs before heading discovery lets isolated or large display equations pass the heading heuristics and emit `# $...$` instead of equation content. Exclude display-equation lines from every heading-promotion path, while retaining short expressions such as `A + B` and `(R-12)`.
(Based on your team's feedback about gating equation heading promotion.) .</comment>
<file context>
@@ -702,6 +702,13 @@ pub(super) fn to_markdown_from_lines_with_tables_and_images(
+ // Rewrite confident native math glyph runs into $...$ LaTeX
+ let lines = if options.formula_latex {
+ crate::formula::rewrite_math_runs(lines)
+ } else {
+ lines
</file context>
| .chars() | ||
| .any(|c| matches!(c, '∑' | '∏' | '∫' | '√' | '∮' | '⋃' | '⋂')) | ||
| }); | ||
| if has_huge_operator { |
There was a problem hiding this comment.
P2: This rejects every oversized big operator, including valid unbounded operators with no limits. Detect the presence of unsupported bounds before applying the rejection penalty.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/formula/mod.rs, line 485:
<comment>This rejects every oversized big operator, including valid unbounded operators with no limits. Detect the presence of unsupported bounds before applying the rejection penalty.</comment>
<file context>
@@ -0,0 +1,1115 @@
+ .chars()
+ .any(|c| matches!(c, '∑' | '∏' | '∫' | '√' | '∮' | '⋃' | '⋂'))
+ });
+ if has_huge_operator {
+ penalties.add("huge_operator", 0.5);
+ }
</file context>
| // Scripts are smaller and within one line-height of the base baseline. | ||
| let all_script_sized = fragment.items.iter().all(|item| { | ||
| item.font_size < base_size * SCRIPT_SIZE_RATIO | ||
| && (item.y - base.y).abs() < base_size * 0.9 |
There was a problem hiding this comment.
P2: A detached line is stitched without requiring a genuine script offset. Require a nontrivial baseline displacement and a tighter upper bound before merging it into the formula.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/formula/mod.rs, line 616:
<comment>A detached line is stitched without requiring a genuine script offset. Require a nontrivial baseline displacement and a tighter upper bound before merging it into the formula.</comment>
<file context>
@@ -0,0 +1,1115 @@
+ // Scripts are smaller and within one line-height of the base baseline.
+ let all_script_sized = fragment.items.iter().all(|item| {
+ item.font_size < base_size * SCRIPT_SIZE_RATIO
+ && (item.y - base.y).abs() < base_size * 0.9
+ && item.text.trim().chars().count() <= 6
+ && classify_item(item) != MathEvidence::None
</file context>
| // Non-ASCII letters without a mapping (accented identifiers) | ||
| // pass through; KaTeX accepts them in text-ish positions but | ||
| // they are a mild risk. | ||
| penalties.add("unmapped_letter", 0.15); |
There was a problem hiding this comment.
P2: An unmapped alphabetic glyph can still pass the confidence gate and become LaTeX. Give unmapped letters a rejecting penalty, or otherwise reject the reconstruction whenever one is encountered.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/formula/mod.rs, line 428:
<comment>An unmapped alphabetic glyph can still pass the confidence gate and become LaTeX. Give unmapped letters a rejecting penalty, or otherwise reject the reconstruction whenever one is encountered.</comment>
<file context>
@@ -0,0 +1,1115 @@
+ // Non-ASCII letters without a mapping (accented identifiers)
+ // pass through; KaTeX accepts them in text-ish positions but
+ // they are a mild risk.
+ penalties.add("unmapped_letter", 0.15);
+ out.push(c);
+ } else {
</file context>
| ('\u{22A4}', r"\top"), | ||
| ('\u{22A5}', r"\bot"), | ||
| ('\u{2225}', r"\parallel"), | ||
| ('\u{22A5}', r"\perp"), |
There was a problem hiding this comment.
P2: The second U+22A5 entry overwrites the earlier \bot mapping, so the map loses the UP TACK form and never maps U+27C2. Use U+27C2 for the \perp entry.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/formula/unicode_map.rs, line 113:
<comment>The second U+22A5 entry overwrites the earlier `\bot` mapping, so the map loses the UP TACK form and never maps U+27C2. Use U+27C2 for the `\perp` entry.</comment>
<file context>
@@ -0,0 +1,391 @@
+ ('\u{22A4}', r"\top"),
+ ('\u{22A5}', r"\bot"),
+ ('\u{2225}', r"\parallel"),
+ ('\u{22A5}', r"\perp"),
+ // ── Arrows ──────────────────────────────────────────────────
+ ('\u{2190}', r"\leftarrow"),
</file context>
Suggested change
| ('\u{22A5}', r"\perp"), | |
| ('\u{27C2}', r"\perp"), |
| // TeX/legacy fonts frequently decode to the "wrong" sibling | ||
| // codepoint: MICRO SIGN for mu, GREEK LUNATE EPSILON for epsilon, | ||
| // RING OPERATOR vs MASCULINE ORDINAL. Map every sibling. | ||
| ('\u{00B5}', r"\mu"), // MICRO SIGN |
There was a problem hiding this comment.
P2: In prose units such as 5 µm, this mapping can anchor a formula run and rewrite the unit as $5 \mu m$. Make MICRO SIGN non-anchoring or reject unit context before using it as a math anchor.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/formula/unicode_map.rs, line 196:
<comment>In prose units such as `5 µm`, this mapping can anchor a formula run and rewrite the unit as `$5 \mu m$`. Make MICRO SIGN non-anchoring or reject unit context before using it as a math anchor.</comment>
<file context>
@@ -0,0 +1,391 @@
+ // TeX/legacy fonts frequently decode to the "wrong" sibling
+ // codepoint: MICRO SIGN for mu, GREEK LUNATE EPSILON for epsilon,
+ // RING OPERATOR vs MASCULINE ORDINAL. Map every sibling.
+ ('\u{00B5}', r"\mu"), // MICRO SIGN
+ ('\u{2212}', "-"), // MINUS SIGN → ASCII hyphen-minus
+ ('\u{2044}', "/"), // FRACTION SLASH
</file context>
| ('\u{25A1}', r"\square"), | ||
| ('\u{25A0}', r"\blacksquare"), | ||
| ('\u{2662}', r"\diamondsuit"), | ||
| ('\u{22D5}', r"\#"), |
There was a problem hiding this comment.
P2: When formula recovery sees ⋕, this mapping emits \#, which is a number sign rather than EQUAL AND PARALLEL TO. Remove the unsupported mapping or replace it with a representation of that relation.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/formula/unicode_map.rs, line 281:
<comment>When formula recovery sees `⋕`, this mapping emits `\#`, which is a number sign rather than EQUAL AND PARALLEL TO. Remove the unsupported mapping or replace it with a representation of that relation.</comment>
<file context>
@@ -0,0 +1,391 @@
+ ('\u{25A1}', r"\square"),
+ ('\u{25A0}', r"\blacksquare"),
+ ('\u{2662}', r"\diamondsuit"),
+ ('\u{22D5}', r"\#"),
+ ];
+
</file context>
| ('\u{2216}', r"\setminus"), | ||
| ('\u{2234}', r"\therefore"), | ||
| ('\u{2235}', r"\because"), | ||
| ('\u{221F}', r"\angle"), |
There was a problem hiding this comment.
P2: When formula recovery sees ∟, this entry emits generic \angle and changes the symbol's meaning and shape. Preserve the Unicode glyph or use a renderer-supported right-angle representation instead of aliasing it to U+2220.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/formula/unicode_map.rs, line 240:
<comment>When formula recovery sees `∟`, this entry emits generic `\angle` and changes the symbol's meaning and shape. Preserve the Unicode glyph or use a renderer-supported right-angle representation instead of aliasing it to U+2220.</comment>
<file context>
@@ -0,0 +1,391 @@
+ ('\u{2216}', r"\setminus"),
+ ('\u{2234}', r"\therefore"),
+ ('\u{2235}', r"\because"),
+ ('\u{221F}', r"\angle"),
+ ('\u{2220}', r"\angle"),
+ ('\u{22C8}', r"\bowtie"),
</file context>
3 tasks
Latin Modern splits its math faces across LMMathItalic, LMMathSymbols, LMMathExtension, and LMMathOperators; only the italic family matched the table. arXiv pdfTeX output uses all four.
There was a problem hiding this comment.
0 issues found across 1 file (changes from recent commits).
Shadow auto-approve: would not auto-approve. Auto-approval blocked by 9 unresolved issues from previous reviews.
Re-trigger cubic
Member
Author
|
Measured with #415 merged in (font family names on items): font-evidence anchoring now fires on arXiv PDFs (LMMathItalic/MSBM/LMMathSymbols), and arxiv_math conversion rises from 2.4% to 6.4% (188/2,927 render-equivalence tests, from a hard 0 before this PR). Remaining headroom is display-equation assembly across full lines — the render-equivalence bar requires whole expressions, and intra-line runs still fragment the left/right sides of large equations. |
abimaelmartell
marked this pull request as draft
August 18, 2026 03:51
Member
Author
|
Converting to draft. The inline reconstruction is measured and stable behind its flag, but the complete version of this feature depends on assembling display equations across lines, which may reshape parts of this module. Parking until that work is scheduled rather than carrying a dormant flag on main. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Model-free formula extraction: rewrites native math glyph runs into
$...$LaTeX during markdown conversion, using font evidence and item geometry — no recognition model, no downloads, no inference cost. Opt-in viaMarkdownOptions::formula_latex/pdf2md --formula-latex(default off).Supersedes the approach in #40/#42: detection is native and inline (no external layout-model bboxes), and emission goes into the markdown stream rather than a separate API. #42's unicode map is salvaged and extended (codepoint-variant siblings like MICRO SIGN →
\mu, double-struck/script letters, ~60 more relations and arrows).How it works
F(q,r)=— join but never anchor), prose (terminates). Detached script-fragment lines are stitched back to their base first: the line grouper's 3pt tolerance is smaller than TeX's superscript rise.E_{4x4}, never invalidE_{4}_{x4}); combining accents wrap the preceding glyph (\hat{\alpha}); font names select alphabets (MSBM →\mathbb, CMSY capitals →\mathcal, EUFM →\mathfrak); trailing sentence punctuation stays outside the delimiters.$...$is worse than none.Why opt-in
Measured on olmOCR-bench's render-equivalence math tests: converts 2.4% of arxiv_math from a hard zero (no local parser scores above zero there without LaTeX output), but LaTeXifying text currently costs fuzzy text-presence tests more than the math gains. Regression corpus with the flag on: all pass, with rewrites confined to math-heavy documents and all emitted LaTeX valid. Default stays off until net-positive; two known unlocks are queued:
TextItem::fontis often an opaque resource name ("F2"), so font anchoring rarely fires today and detection leans on symbol anchoring.Test plan
cargo fmt/cargo clippy -- -D warnings(both feature sets) / full suites (977 unit + 165 integration)🤖 Generated with Claude Code
Summary by cubic
Adds opt‑in inline LaTeX recovery for native math glyph runs during Markdown conversion. Previously output used raw Unicode only; with
MarkdownOptions.formula_latexorpdf2md --formula-latexset, confident math runs emit$...$while prose stays unchanged. Default off keeps output identical.formula::rewrite_math_runsin both Markdown paths; adds--formula-latexandMarkdownOptions::formula_latex.LMMathItalic,LMMathSymbols,LMMathExtension,LMMathOperators).$...$.$...$.--formula-latexor set the option; downstream consumers should handle LaTeX in output when enabled.Written for commit a4efd0a. Summary will update on new commits.