feat(webllm): per-bullet "Suggest a rewrite" pilot — WebGPU-only, Qwen2-1.5B

[Vaishnavi1709]

Summary

First in-browser AI capability: a per-bullet "Suggest a rewrite" CTA that runs Qwen2-1.5B-Instruct via WebLLM on the user's GPU. Bytes never leave the browser. Resolves #3.

Library — src/lib/webllm/

• types.ts — narrow WebLlmEngine contract (only the surface rewrite-bullet consumes) so tests stub without importing the real lib.
• capability.ts — detectWebGpu() returns "available" | "no-webgpu" | "unsupported-os", cached as a module-level promise so the discriminator fires exactly once per page.
• web-llm.ts — pinned MODEL_ID = "Qwen2-1.5B-Instruct-q4f16_1-MLC". loadEngine() dynamic-imports @mlc-ai/web-llm and caches the engine for the page lifetime — concurrent clicks across bullets share one download.
• rewrite-bullet.ts — pure rewriteBulletWithLlm(bullet, engine). Single-source-of-truth BULLET_REWRITE_SYSTEM_PROMPT; post-processing strips a "Rewritten:" prefix, wrapping quotes, and trims to the first non-empty line.

UI — src/components/RewriteButton.tsx

• On mount: detectWebGpu(); if not "available", the component returns null. Hidden, not greyed. Header comment documents the stance so a future contributor doesn't "improve" it with a fallback banner.
• On click: progress panel (percent + status text + green progress bar + "What's happening?" disclosure that mirrors the prompt rules).
• On done: inline green box with rewritten text + "Use this — copy to clipboard".
• Wired into Result.tsx::BulletRow next to the existing check pills via <RewriteButton bullet={bullet.text} />.

Telemetry — src/lib/analytics.ts

• Four new snake_case events through the existing env-gated track() helper: webllm_capability_detected (payload {capability}), webllm_download_started, webllm_loaded, webllm_first_rewrite. Each fires at most once per page.
• Compiles to no-op when VITE_POSTHOG_KEY is unset — verified no posthog string in built assets.

Bundle — the @mlc-ai/web-llm dynamic import keeps the library in its own ~5.7 MB chunk (dist/assets/index-CereMTgS.js); the entry chunk holds only the tiny dynamic-import reference. Per AC, main chunk does not grow >5 KB.

Out of scope (explicit per issue)

• WASM fallback for non-WebGPU browsers
• Cover letters, full-resume rewrite, JD-tailored rewrite
• Cloud rewrite paths, runtime model switching
• Telemetry beyond the four events above

Test plan

• npm run typecheck clean
• npm run test — 194 tests pass (13 new in src/lib/webllm/: 6 capability + 7 rewrite-bullet)
• npm run build clean; @mlc-ai/web-llm lives in its own chunk; grep -r posthog dist/assets/ returns nothing with VITE_POSTHOG_KEY unset
• Chrome (WebGPU available) — dropped a synthetic PDF, all 6 parsed bullets show "Suggest a rewrite" below their check pills
• Chrome — clicked rewrite → confirm progress UI, model download in DevTools Network, rewritten bullet inline, "copy to clipboard" works, second click on a different bullet skips the download
• Firefox — confirm the button is absent (not greyed) under every bullet
• PostHog Live Events show the four events firing in the documented order (only with VITE_POSTHOG_KEY set)

Notes for review

• The narrow WebLlmEngine interface intentionally duplicates the shape of MLCEngineInterface rather than re-exporting it, so the type graph for non-WebLLM call-sites stays light and tests don't pull the library.
• Component placement is flat (src/components/RewriteButton.tsx) matching the existing DropZone.tsx / PdfPreview.tsx / Result.tsx — the ui/ + features/ tiers from CLAUDE.md don't exist yet on main. Happy to move once a primitive folder lands.

[s-annam]

PR Review: feat(webllm) per-bullet "Suggest a rewrite" pilot

Updated — reclassified the hardcoded-color and component-placement findings from Suggestion to Blocking. Both are enforced rules in CLAUDE.md (the styling "What NOT to do" list and the 3-tier component architecture), not stylistic preferences.

Summary

Clean, well-tested WebLLM pilot. Capability detection, lazy engine load, the pure rewrite function, and telemetry all match issue #3's spec. Verified locally on the branch: 207 tests pass (13 new), typecheck clean, build code-splits @mlc-ai/web-llm into its own ~6 MB chunk (index-CereMTgS.js) with the entry chunk barely moving, and no posthog string in dist/ when VITE_POSTHOG_KEY is unset. Three blocking items to address before merge: a broken retry path and two CLAUDE.md rule violations on the new component.

Spec Alignment (issue #3)

Requirement	Status	Notes
WebGPU detect, cache, 3-state	✅ Implemented	capability.ts, cached promise, telemetry fires once
Pinned Qwen2-1.5B MODEL_ID	✅ Implemented	single const, no inline string
Lazy load on click, own chunk	✅ Verified	index-CereMTgS.js 6 MB; entry +5 KB
Hide CTA (not grey) w/o WebGPU	✅ Implemented	returns null; header documents the stance
4 snake_case events, env-gated	✅ Verified	no posthog in dist without key
Tests pass	✅ Verified	207 (6 capability + 7 rewrite-bullet new)

Highlights

• Narrow WebLlmEngine contract so tests stub without importing the 6 MB lib — right call, keeps the type graph and test deps light.
• BULLET_REWRITE_SYSTEM_PROMPT as single source of truth, paraphrased in the user-facing "What's happening?" disclosure. Nice honesty.
• Capability + telemetry caching via module-level promise/flags is correct and well-commented.

Key Findings

1. [Blocking] Failed engine load caches a rejected promise — "Try again" can never recover. web-llm.ts:41. cached is assigned the async IIFE with no .catch. If CreateMLCEngine throws (OOM, network drop), cached holds a rejected promise for the page lifetime; every later click hits if (cached) return cached and re-rejects instantly. The UI's "Try again" button (the error branch of labelFor) is therefore non-functional — only a page reload recovers. Reset the cache on failure so retry can re-attempt. This path is also untested (web-llm.ts has no test file).

2. [Blocking] Hardcoded raw Tailwind palette in feature code. RewriteButton.tsx (L95, L119, L186, L193). border-neutral-300 bg-white text-red-700 border-emerald-200 bg-emerald-50 etc. CLAUDE.md lists this under "What NOT to do → ❌ Hardcoded colors" for feature components — enforced, not advisory. The token system already ships feedback-success-{bg,border,text} and feedback-error-* purpose-built for the green-result and red-error boxes here, and they auto-adapt to dark mode via @media (prefers-color-scheme) in styles.css — so the manual dark: pairs are redundant parallel theming. Chip/FeedbackControl/ContactCard are the pattern to follow.

3. [Blocking] RewriteButton placed in the wrong architecture tier. It's wired to bullet domain data, the CLAUDE.md definition of a Feature component, so it belongs in src/components/features/ next to the siblings Result.tsx already imports (ScoreRing, VerdictHeader, ContactCard, FeedbackControl). The 3-tier architecture is enforced, and the PR description's justification — "the ui/ + features/ tiers don't exist yet on main" — is factually wrong; features/ is right there. Move to features/RewriteButton.tsx.

4. [Suggestion] Raw <button> vs the <Button> primitive. CLAUDE.md says use the primitive — but none exists yet (only Chip in ui/). Keeping the raw element is fine for this PR; flagging so inline button styles don't proliferate before a Button primitive lands. Pairs with finding 2.

5. [Nit] Entry chunk +5.01 KB (559.46 → 564.47), a hair over the PR's "≤5 KB" claim. Trivial.

6. [Nit] webllm_first_rewrite fires even on empty output (rewrite-bullet.ts) when the model returns null content and postProcess yields "". Also copied never resets to false. Both minor.

Verdict

Action: REQUEST_CHANGES
Rationale: One blocking correctness bug (advertised retry can't recover) plus two blocking CLAUDE.md rule violations on the new component (hardcoded palette, wrong tier). All three are cheap to fix and avoid baking drift into the first AI-capability component. Spec coverage, tests, bundle split, and telemetry gating are all in great shape.

[s-annam]

[Blocking]: If the engine load rejects (OOM, dropped network), cached permanently holds a rejected promise — every later click returns it via if (cached) return cached and re-rejects instantly. The "Try again" button (error branch of labelFor) then can't recover; only a reload does. Reset the cache on failure:

cached = (async () => { ... })();
cached.catch(() => { cached = null; }); // allow a real retry

This path is untested too — worth a web-llm.ts test for the reject-then-retry case.

[s-annam]

[Blocking]: Hardcoded raw Tailwind palette here (border-neutral-300 bg-white text-neutral-700 …, plus text-red-700 L119 and border-emerald-200 bg-emerald-50 L186/193). CLAUDE.md lists this under "What NOT to do → ❌ Hardcoded colors" for feature code — it's an enforced rule, not a preference. Use semantic tokens: error → feedback-error-*, result box → feedback-success-{bg,border,text}, all already defined and auto-adapting to dark via @media (prefers-color-scheme) in styles.css (so the dark: pairs become dead weight). Chip/FeedbackControl/ContactCard are the pattern.

Relatedly (non-blocking): the raw <button className> trips CLAUDE.md's "use the <Button> primitive" rule — but no Button primitive exists yet (only Chip). Fine to keep the raw element for this PR; flagging so we don't multiply inline button styling before a primitive lands.

[s-annam]

[Blocking]: RewriteButton is wired to bullet domain data — a Feature component by CLAUDE.md's 3-tier definition — so it belongs in src/components/features/, alongside the siblings imported just above (ScoreRing, VerdictHeader, ContactCard, FeedbackControl). This is the enforced architecture, and the PR description's justification ("the ui/ + features/ tiers don't exist yet on main") is factually wrong — features/ is right here and you're importing from it. Move to features/RewriteButton.tsx. (At 199 LOC it's also right at the ~200 split threshold — worth keeping an eye on.)

[s-annam]

[Nit]: trackWebllmFirstRewrite() fires before checking whether cleaned is non-empty, so a null-content / empty rewrite still counts as a "first rewrite". Consider gating the event on a non-empty result.

[Vaishnavi1709]

Thanks for the careful review @s-annam — all three blocking items + the empty-output nit are addressed. Rebased onto latest main (which had moved 4 commits, including the new <tr>/<td> BulletRow). PR is now MERGEABLE.

1. Engine retry path — web-llm.ts now clears cached on rejection so the UI's "Try again" branch can re-attempt. downloadStartedFired deliberately stays true after a failure so a retry of the same logical attempt doesn't double-fire webllm_download_started. New web-llm.test.ts mocks @mlc-ai/web-llm via a hoisted vi.mock and covers:

• reject-then-retry (the bug you flagged)
• concurrent-call dedupe (three parallel loadEngine calls share one CreateMLCEngine invocation)
• MODEL_ID is the value passed to the library
• initProgressCallback reports forward to the supplied onProgress

2. Hardcoded palette — every raw color in RewriteButton.tsx swapped for a semantic token (border-border-light, bg-surface-card, bg-surface-subtle, text-content-{secondary,muted,tertiary,primary}, bg-feedback-success-bg, border-feedback-success-border, text-feedback-success-text, bg-feedback-success-icon for the progress-bar fill, text-feedback-error-text for the error row). All dark: overrides removed — the token CSS auto-adapts via prefers-color-scheme. Pattern follows Chip / FeedbackControl / ContactCard.

3. Component tier — git mv to src/components/features/RewriteButton.tsx, imports rewritten as ../../lib/webllm/…, Result.tsx now imports from ./features/RewriteButton.tsx. You're right that the PR description's justification was factually wrong — features/ was already there in the same Result.tsx import block, I should have looked harder.

4. Empty-output telemetry — rewrite-bullet.ts now gates trackWebllmFirstRewrite() on cleaned.length > 0 so a null model response isn't counted as a funnel step. Two new mock-based tests assert no-fire on empty and fire-once on first non-empty output (uses vi.hoisted so the mock factory can reference the spy).

One nit I left as-is: copied resetting to false. Re-read the code and setCopied(false) is the first line of onClick, so it does reset on every new rewrite attempt. Post-copy state stays "Copied" until the next click — I read that as intended UX (the user has confirmed the action). Happy to add a timeout-based reset if you'd rather see it auto-revert.

Rebase note

Main's new table-row BulletRow drops CheckPill for inline <td> cells. Conflict resolved by taking main's structure and placing <RewriteButton bullet={bullet.text} /> inside the bullet-text <td> so the button sits under the bullet text in column 1 — the same visual relationship as before, in the new layout.

Verification

• npm run typecheck clean
• npm run test — 213/213 pass (was 207; +4 web-llm.test.ts, +2 telemetry-gating in rewrite-bullet.test.ts)
• npm run build clean; @mlc-ai/web-llm still in its own ~6 MB chunk; posthog still absent from dist/ with VITE_POSTHOG_KEY unset
• Manual click-flow + Firefox-hide verification still outstanding on my side; happy to take a follow-up review pass after that

Ready for re-review whenever you have a minute.

[s-annam]

PR Review: feat(webllm) per-bullet "Suggest a rewrite" pilot — re-review

Follow-up commit a70a642 resolves all three blocking items from the prior review, plus both nits. Verified locally on the PR head.

Prior blocking items — all fixed

Item	Status
#1 Rejected-promise cache made "Try again" a no-op (web-llm.ts)	✅ pending.catch(() => { if (cached === pending) cached = null }) — identity-guarded so a slow failure can't null a later success's cache. New web-llm.test.ts covers reject-then-retry (re-invokes CreateMLCEngine, resolves).
#2 Hardcoded Tailwind palette	✅ All semantic tokens now (feedback-success-{bg,border,text}, feedback-error-text, surface/content/border-*). No raw palette, no redundant dark: pairs.
#3 Wrong architecture tier	✅ Moved to src/components/features/RewriteButton.tsx; Result.tsx imports ./features/RewriteButton.tsx. No dangling old-path refs.
Nit: webllm_first_rewrite fired on empty output	✅ Gated on cleaned.length > 0; test asserts no-fire on null content.
Nit: copied never reset	✅ setCopied(false) at click start.

Local verification (PR head a70a642)

• npm test → 213 passed (19 webllm: 6 capability + 9 rewrite-bullet + 4 web-llm)
• npm run typecheck → clean
• npm run build → @mlc-ai/web-llm code-split into its own ~6 MB chunk (index-CereMTgS.js); entry chunk 566 kB
• No posthog string in dist/ with VITE_POSTHOG_KEY unset → env-gating + DCE hold

Highlights

• Telemetry one-shot flags survive a retry by design (downloadStartedFired stays true) so a single logical attempt doesn't double-fire webllm_download_started — and that reasoning is in the doc comment. Good.
• Narrow WebLlmEngine contract keeps the 6 MB lib out of the test graph; concurrent-click sharing of one engine/one download is tested.

Remaining — non-blocking nits only

1. [Nit] LoadingPanel progress bar has no role="progressbar" / aria-valuenow — screen-reader users get no load progress.
2. [Nit] Error <p> isn't announced (role="alert").
3. [Nit] Component is 197 LOC — right at the ~200 split threshold; worth watching as the rewrite UX grows.

Verdict

Action: APPROVE
Rationale: Zero blocking. The retry-recovery bug is fixed and now covered by a regression test, palette is token-clean, the component sits in the correct tier, and telemetry gating is sane. Tests green, typecheck clean, bundle split confirmed. Nits are optional polish.

[s-annam]

[Nit]: This progress bar conveys state purely visually. Consider role="progressbar" with aria-valuenow={pct} / aria-valuemin={0} / aria-valuemax={100} on the track div so screen-reader users get the ~1-minute download progress, not silence.

[s-annam]

[Nit]: The error message isn't announced to assistive tech. role="alert" (or aria-live="polite") on this <p> would surface a failed load to SR users without a visual cue.

[s-annam]

Re-approve after rebase onto main + a11y nits (progressbar/alert roles). Conflicts resolved, verify CI green, 243 tests pass.