Header detection collapses on flattening renderers: single-word all-caps unknown header (e.g. INTERNSHIPS) absorbed into prior section

[s-annam]

Summary

On font-flattening renderers (Google Docs/Skia, WeasyPrint/Cairo, and similar), section-header detection effectively loses its font signal — measured header font ratios sit at r = 1.00–1.09, below the VISUAL_HEADER_FONT_RATIO = 1.15 gate. Detection then rests entirely on the vocabulary rules (keyword/anchor) plus the all-caps pattern rule. A header that misses both — specifically a single-word, ALL-CAPS, unknown-vocabulary header like INTERNSHIPS — is matched by nothing and gets silently absorbed into the preceding section, losing the boundary.

This is a confirmed, measured repro (see Evidence), not a theoretical one.

Repro

Fixtures: tests/fixtures/pdfs/google-docs/google-docs-skia-proxy-nonstandard-headers.pdf and tests/fixtures/pdfs/unknown/weasyprint-cairo-nonstandard-headers.pdf.

Run extractFromPdfBytes → groupIntoLines → splitIntoSections and inspect the INTERNSHIPS line:

kw=-  r=1.09  caps=1  w=1  | INTERNSHIPS

It opens no section. Its content folds into the prior experience bucket.

Root cause

classifyLine (src/lib/heuristics/sections.ts:746) is a disjunction of binary accept-rules. Categorize the header rules by the gate each imposes:

Rule	Vocab gate	Casing gate	Font gate
matchSectionHeader exact keyword	known alias	—	—
split-letter rejoin	known alias	—	—
matchAnchorFallback (head-noun anchor)	anchor set	Title-case	—
isVisualHeader 2a (font+short)	free	free	needs 1.15×
isTextPatternHeader 2b (all-caps)	free	ALL-CAPS	—

The only vocab-free and casing-free rule is 2a, and it's the one font-gated rule. On flattening renderers, real headers measure r≈1.0–1.09, so 2a never fires. INTERNSHIPS then needs 2b — which requires ≥2 words (isTextPatternHeader, sections.ts:599, the single-token FP guard from #112) — so a single-word all-caps unknown header falls through to APPEND.

Net: nothing positively detects "this is a header" once font is flat and the word is unknown vocabulary; the line is absorbed and the section boundary is lost.

Why it matters

Header detection on the most common flattening renderers (Google Docs especially) has no font-independent, vocabulary-independent positive signal. Any unknown-vocabulary header that also misses the casing/word-count rules silently pollutes its neighbor. INTERNSHIPS is the reproduced instance; the class is broader (e.g. a single-word AFFILIATIONS, CERTIFICATIONS variant, etc.).

Harm in the current fixtures is mild — INTERNSHIPS absorbs into experience, a semantically adjacent section, so its entries still reach the experience-bullet pool. The latent risk is the same mechanism absorbing a header into an unrelated prior section, mis-attributing or dropping its content from the score.

Proposed direction

Add font-independent positive signals so header detection doesn't collapse when font is flat, rather than widening the keyword list (exact-match coverage will never generalize):

• Vertical gap above — gap to the previous line vs. body line-height. Geometric, survives flattening. Strong header cue.
• Left-alignment / isolation — line at the body left margin, alone on its band (hasEOL).
• Graded casing — ALL-CAPS strong, Title-case moderate (credit, not a hard gate).
• Keep font prominence as a positive feature (credit when present, zero when absent — never negative).
• Revisit the ≥2 words gate in isTextPatternHeader for the single-word all-caps case, paired with the gap/isolation signals to keep the L3 — Visual-primary boundary detection + name/contact disambiguation #112 FP class closed.

This is the narrow, evidence-backed slice of the larger "positive header classifier" idea: rather than a disjunction of binary rules where each failure cascades, score a line from orthogonal features and threshold. Detection (is-this-a-boundary) stays separate from naming (matchSectionHeader → which section, or other).

Calibration / risk

• This retunes a heavily corpus-tuned surface (Letter-spaced section headers ('E XPERIENCE') parse with zero experiences, disabling per-bullet grouping #56, L2 — Head-noun anchor fallback for qualified section headers (closes #108) #111, L3 — Visual-primary boundary detection + name/contact disambiguation #112, feat: section-recognition L2/L3 — anchor fallback + visual bounds #115, Skills under "Additional" section not recognized as skills #122, Multi-line (wrapped) bullets truncated at the wrap point in the per-bullet pool, then mis-grouped as "Other" #162 were all regression fixes here). A threshold change can re-open a closed false-positive class.
• Safety net is the corpus.test.ts golden snapshots — but golden-regen can mask regressions, so hand-verify snapshot diffs, never blind-regen.
• No line-level header unit fixtures exist today (isVisualHeader / isTextPatternHeader / isHeaderShort have no covering unit tests). A labeled header/non-header line set is a prerequisite and makes the signals tunable in isolation.

Acceptance criteria

• INTERNSHIPS (and the single-word all-caps unknown-header class) opens a section boundary on the two nonstandard-headers fixtures, instead of being absorbed.
• Header detection uses at least one font-independent positive signal (vertical gap and/or left-alignment).
• Font prominence absence is never a negative contribution.
• Full corpus.test.ts snapshot diff hand-verified — no regression in the L3 — Visual-primary boundary detection + name/contact disambiguation #112 single-token false-positive class.
• Line-level header/non-header unit fixtures added for the touched signals.

Out of scope / follow-on

• The Title-case unknown-vocabulary header class is not reproduced by any current fixture (the Title-case lines in these PDFs are body/role lines, correctly appended). Confirming that class needs a new fixture exercising a title-case unknown header + an unrelated-neighbor pollution case — separate fixture-gap task before treating it as real.
• Name-inference for confident-but-unnamed (other) headers — content-shape classification of an unnamed boundary's lines — is a separate issue (overlaps the post-section sanity pass).

[s-annam]

Implementation Complete — 9ede2e1

Acceptance Criteria

• Single-word ALL-CAPS unknown header opens a boundary — INTERNSHIPS now terminates the prior section on both google-docs-skia-proxy-nonstandard-headers and weasyprint-cairo-nonstandard-headers fixtures instead of being absorbed.
• Font-independent positive signal — added HEADER_GAP_RATIO (1.4) in sections.ts: a line whose vertical gap to the previous line is ≥1.4× the computed body line-height reads as a section boundary. Survives font flattening.
• Font-prominence absence never negative — gap is purely additive; font ratio only ever adds.
• Corpus snapshot diff hand-verified, no L3 — Visual-primary boundary detection + name/contact disambiguation #112 regression — only the 2 target fixtures changed; every delta traced (INTERNSHIPS → other, overall 88→80). A prevLineOpenedBoundary adjacency guard ("a header never directly follows a header") closed a L3 — Visual-primary boundary detection + name/contact disambiguation #112 single-token FP the gap path initially re-opened (openresume-react-pdf back to byte-identical).
• Line-level header unit fixtures — new #216 describe block in sections.test.ts (fires on gapped single-word ALL-CAPS; rejects inline acronym / post-header gap / Title-Case).

Notes

• INTERNSHIPS opens the other sink (not named experience) — boundary-naming for unnamed headers is the deferred follow-on, out of scope here.
• Full suite green: 1051 tests, typecheck + lint clean.