Skip to content

fix(ast): preserve positions and fence data through rawHtml reparse - #286

Draft
Princesseuh wants to merge 7 commits into
mainfrom
fix/rawhtml-root-position
Draft

Princesseuh wants to merge 7 commits into
mainfrom
fix/rawhtml-root-position

Conversation

@Princesseuh

@Princesseuh Princesseuh commented Aug 24, 2026 •

Copy link
Copy Markdown
Member

Closes #322.

Changes

  • Preserve the original source and root span through the raw-HTML reparse, including UTF-16 offsets and MDX component overrides used by optimizeStatic.
  • Share tokenizer-level provenance tracking for positions and fence metadata: serialized byte ranges identify original tokens, and parser handles retain their association through relocation. No HTML markers are added for provenance, and elements are not matched by content or output order.
  • Preserve data.lang and data.meta on surviving original code-fence elements, without attaching fence metadata to user-supplied raw HTML or parser-created formatting clones.
  • Retain Markdown-derived positions and self-contained raw-block spans where the source association is unambiguous. Split/coalesced nodes, uncertain raw contexts, and nodes parsed within a raw block remain unpositioned. Missing source positions are not invented; arena IDs are rebuilt.

Updated against current main, retaining its SVG/MathML behavior. The previous direct-token feeding approach is replaced with serialization/tokenization so swallowed tags and raw-text contexts keep their existing parsing semantics. Includes separate changesets and updated API/website documentation.

Validation

  • cargo test --all
  • cargo test -p satteri-ast --no-default-features --features from-html
  • Full npm package suite: 3,395 passed, 23 expected failures, one skipped
  • Release native binding and TypeScript builds; TypeScript checking
  • pnpm lint, all-target Clippy, and formatting checks

Regressions cover Markdown/MDX HAST plugin observability and compilation, repeated/lookalike raw HTML, foster-parented fences, swallowed or malformed tags, formatting clones, nested MDX, source positions, UTF-16 offsets, coalesced text, and conservative raw-block spans.

@codspeed

codspeed Bot commented Aug 24, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 38 untouched benchmarks
⏩ 10 skipped benchmarks1


Comparing fix/rawhtml-root-position (5607aa3) with main (0c2c869)

Open in CodSpeed

Footnotes

  1. 10 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@Princesseuh Princesseuh changed the title fix(ast): keep source positions through the rawHtml reparse fix(ast): keep source positions through rawHtml reparse Aug 24, 2026
Integrate PR #286 with the fix for #322 on current main. Keep serialization and tokenization semantics, associate original nodes with parser handles, and preserve positions only when source associations remain unambiguous. Preserve the source document for MDX static component overrides and UTF-16 offsets.
@Princesseuh Princesseuh changed the title fix(ast): keep source positions through rawHtml reparse fix(ast): preserve positions and fence data through rawHtml reparse Oct 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

rawHtml: code fences lose data.lang and data.meta in HAST

1 participant