fix(heuristics): coursework section + multiline-bullet + column deinterleave - #165
Conversation
…erleave - "Relevant Coursework"/"Coursework" now opens an education-type section boundary (added aliases + anchor); isVisualHeader gains an ALL-CAPS text-pattern fallback independent of font metadata; VISUAL_HEADER_FONT_RATIO lowered 1.2→1.15 — stops unknown headers swallowing the prior entry's description (#163) - add mergeWrappedContinuations() in entry-blocks.ts, run per-section in toSectionedResume before byName flatten; folds wrapped-bullet continuation lines onto their parent so bullet pool and reconstructed-role description agree — no more truncation or "Other" mis-grouping (#162) - add reorderEmbeddedColumns pass upstream of line grouping; de-interleaves localized multi-column blocks (e.g. coursework grid) to column-major reading order before sectionizing (#164) Resolves #163 Resolves #162 Resolves #164
Extract groupItemsIntoRows + reorderColumnRun helpers so the main loop drops below the fallow cognitive-complexity threshold (was 27, limit 15). Behavior-preserving — same column-major de-interleave, 756 tests green. Refs #164
Vaishnavi1709
left a comment
There was a problem hiding this comment.
Nice composition — the three sub-fixes (#163/#162/#164) read cleanly in the order claimed, and the helper extraction (groupItemsIntoRows, reorderColumnRun) resolves the fallow complexity flag without changing behavior. Tests are well-rationalized; the "does NOT promote Title-Case job title" pin on isTextPatternHeader and the single-row date-rail pin on reorderEmbeddedColumns are exactly the right invariants to lock down.
One must-address before merge, plus one thing worth a sanity check.
🔴 Must address — cascade regression on student-projects-activities-singlecol.expected.json
That snapshot delta isn't only a score uplift — it flips:
cascade.confidence: 0.88 → 0
cascade.suggestedEscalation: "none" → "ocr"
…which contradicts the PR body's "no section/field/count loss." Reproduced on both branches:
| branch | rawChars | extractedChars | ratio | confidence | escalation |
|---|---|---|---|---|---|
main |
2279 | 1288 | 0.565 | 0.88 | "none" |
| this PR | 2279 | 1086 | 0.476 | 0 | "ocr" |
Root cause: the bleed you correctly removed used to count toward experience.description (its char sum drops 819 → 617 — exactly the 202-char delta). countExtractedChars (cascade.ts:497) only sums typed fields, so the now-correctly-terminated coursework block — which lands in sections.byName rather than a typed schema field — isn't counted. The extracted/raw ratio drops below EXTRACTION_RATIO_FLOOR = 0.5, collectHardFailures raises low_extraction_ratio, and chooseEscalation routes to "ocr" (confidence.ts:78–82, :148–153, :292).
User-visible score got better (58 → 72), but the cascade now reports this resume as failed and would escalate to OCR — the opposite of the intent.
The other shifted fixtures (weasyprint-cairo-two-column, two-column-achievements-sidebar, awesome-cv-{cv,resume}) all keep their pre-PR cascade.confidence, so the regression is isolated. But the root cause is general: every future "correctly terminated unknown section" you ship has the same risk of dragging the extraction ratio toward the floor.
Pick one:
- Recommended: extend
countExtractedCharsto also tally chars fromsections.byNamebeyond the typed-field set — small, scoped, no API change, and protects future fixes too. - Have the coursework block populate
education[](e.g. a comma-joined course list asdegree) so its content lands in an already-counted field. - Acknowledge the regression in the PR body + a comment on the fixture and open a follow-up issue — only if you'd rather not expand scope here.
🟡 Worth a quick look — awesome-cv structure score declined
awesome-cv-cv: structure 20 → 18, goodBullets 35 → 32 (totalBullets 52)
awesome-cv-resume: structure 23 → 20, goodBullets 23 → 20 (totalBullets 30)
Same total bullets, fewer "good" ones. Specificity ↑ (metricBullets ↑) is expected — wrap-merge revealing metrics in the tail. But the goodBullets drop suggests merged longer bullets are tripping a length cap. Can you confirm this is the expected trade (longer merged bullets correctly reclassify as too-verbose) rather than a downstream rule that should be retuned? Not blocking.
Nits (non-blocking)
sections.ts↔entry-blocks.tsis now a type-only circular import; runtime-safe today, but a value import on the entry-blocks side would make it real. ColocatingmergeWrappedContinuationsnext totoSectionedResumeinsections.ts(its only non-test caller) would preempt this.isVisualHeader's split&&reads oddly — an early return on the font path would mirror the text-pattern path.mergeWrappedContinuationsdoes`${prev.text} ${line.text.trim()}`without trimmingprev.text—.trimEnd()on prev would avoid double-space on the off chance a line has a trailing space.
Verdict: request changes on the cascade regression; everything else is approve-modulo-nits.
Review fix for PR #165: countExtractedChars only summed typed parsed fields, so a correctly-terminated unknown section (coursework #164) that lands in sections.byName went uncounted. The extracted/raw ratio fell below EXTRACTION_RATIO_FLOOR, raising low_extraction_ratio and routing to OCR with confidence 0 — the opposite of the fix's intent. Tally byName entries outside the typed-field set (projects, certifications, achievements, other). Restores student-projects to 0.88/none and clears four pre-existing false-OCR escalations (additional-skills, multiline-bullets-coursework, deedy x2) that were already 0/ocr on main. Score blocks unchanged — extractedChars feeds confidence/escalation only. Also trimEnd prev.text in mergeWrappedContinuations to avoid a mid-string double space (review nit). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BcsLGiQZxtq9VpbjiVzGb4
|
Thanks for the thorough review, @Vaishnavi1709 — all addressed in 🔴 Cascade regression — fixed (option 1)Extended
Your "root cause is general" call was right — four other fixtures were already
Score blocks are unchanged across all five — 🟡 awesome-cv goodBullets — confirmed expectedThese fixtures aren't in the changed set, so behavior is identical to before this comment. The drop is the wrap-merge trade you predicted: merged longer bullets correctly reclassify against the length cap (verbosity), while specificity rises as the merge reveals tail metrics. Not a downstream rule that needs retuning. Nits
Re-requesting review. |
The byName tally pushed countExtractedChars over fallow's thresholds (cyclomatic 21/20, cognitive 22/15). Extract scalarFieldChars, listFieldChars, and untypedSectionChars; the host is now a three-term sum. Behavior unchanged — 756 tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BcsLGiQZxtq9VpbjiVzGb4
…sework-bullets # Conflicts: # src/lib/heuristics/entry-blocks.test.ts # tests/fixtures/pdfs/google-docs/google-docs-skia-proxy-multiline-bullets-coursework.expected.json
…erleave (#165) Closes #162, #163, #164. Coursework/unknown-section termination, wrapped multi-line bullet folding, and embedded-column de-interleave. Untyped sections now count toward the extraction ratio so correctly-terminated unknown sections no longer trip a false OCR escalation. Merged main (#166 wrapped-header fold) — conflicts resolved, snapshot rebaked, 758 tests green.
…erleave (#165) Closes #162, #163, #164. Coursework/unknown-section termination, wrapped multi-line bullet folding, and embedded-column de-interleave. Untyped sections now count toward the extraction ratio so correctly-terminated unknown sections no longer trip a false OCR escalation. Merged main (#166 wrapped-header fold) — conflicts resolved, snapshot rebaked, 758 tests green.
Summary
Fixes the full coursework/multiline-bullet parse path on multi-column student résumés (reproducer:
google-docs-skia-proxy-multiline-bullets-coursework.pdf). Three ordered fixes that compose end-to-end:education-type section (added aliases + anchor);isVisualHeadergains an ALL-CAPS, font-metadata-independent text-pattern fallback;VISUAL_HEADER_FONT_RATIOlowered 1.2→1.15. Stops unknown headers swallowing the prior entry's description.mergeWrappedContinuations()inentry-blocks.ts, run per-section intoSectionedResumebefore thebyNameflatten; folds wrapped-bullet continuation lines onto their parent so the bullet pool and reconstructed-role description agree — no more truncation or "Other" mis-grouping.reorderEmbeddedColumnspass upstream of line grouping; de-interleaves localized multi-column blocks (e.g. the coursework grid) to column-major reading order before sectionizing.Pipeline order is load-bearing: #164 de-interleaves columns → #163's boundary captures the block as
education→ #162 folds each column's wrap onto its true parent.Pre-existing Docent tri-line-header mis-parse (bullets in "Other") is orthogonal and left untouched.
Closes #163
Closes #162
Closes #164
Test plan
npm run typecheckcleannpm run testgreen (756 passed, 54 files)npm run lint(eslint) clean