fix(intelligence): replace Workers AI model deprecated 2026-05-30

[chitcommit]

What broke

@cf/meta/llama-3.1-8b-instruct aliases @cf/meta/infire-llama-3.1-8b-instruct, which Cloudflare deprecated on 2026-05-30. Every call to it now fails with error 5028. Six intelligence-layer call sites hardcoded that id, so each has been silently dead for three months — the failures land in catch blocks that log a warning and return a placeholder, so nothing surfaced.

Found while chasing why MemoryCloude returned "Failed to generate summary." for every one of 248 recovered session records. Summarization was only the visible symptom; anomaly detection, task decomposition, cognitive synthesis, relationship summaries and intent refinement were all failing the same way.

Live evidence

Probed through the deployed diagnostic route (POST connect.chitty.cc/api/thirdparty/cloudflare/ai/run):

$ ... -d '{"model":"@cf/meta/llama-3.1-8b-instruct", ...}'
{"error":"5028: @cf/meta/infire-llama-3.1-8b-instruct was deprecated on 2026-05-30.
  See the model catalog for alternatives: https://developers.cloudflare.com/workers-ai/models/"}

Context length was ruled out first: session session-1777931666-130901 has exactly one interaction and failed identically.

Both replacement candidates verified live through the same route, both returning "ok":

model	result
@cf/meta/llama-4-scout-17b-16e-instruct	✅
@cf/meta/llama-3.3-70b-instruct-fp8-fast	✅

The change

• src/lib/ai-model.js (new) — resolveAiModel(env) reads env.AI_MODEL_PRIMARY, defaulting to @cf/meta/llama-4-scout-17b-16e-instruct. This generalises the pattern already live at src/api/routes/prompts.js:359 and adopts it at all seven sites, so the next deprecation is a config value rather than a code change.

  Which model production will actually use: this diff makes AI_MODEL_PRIMARY load-bearing at six sites that previously ignored it, so its current value matters. It is already declared in all three wrangler.jsonc env blocks (:114, :225, :371) as @cf/meta/llama-4-scout-17b-16e-instruct — identical to the new default, and the exact model verified live above. So the six sites converge on a model proven working, not on an untested one. Caveat: that is the value in the repo config; a dashboard-side override would not be visible here.

• extractAiText(result) — the current envelope carries the generated text both as top-level response and as choices[0].message.content. Reading both means a model that drops either shape cannot degrade to undefined. It returns "", never undefined.

• memory-cloude.js now throws on an empty summary instead of caching it. Persisting "" into KV for 90 days would convert a loud failure into a silent one.

Seven call sites changed: memory-cloude, context-consciousness, cognitive-coordination (×2), relationship-engine, intent-predictor, mcp.js sampling default, plus prompts.js routed through the shared resolver. The two AI.run sites left untouched are correct as-is — thirdparty.js:600 takes its model from the request body (that's the diagnostic probe), and mcp.js:469 consumes the resolved variable.

Validation

• npx vitest run — 563 passed, 1 skipped, 0 failed (full suite)
• npm run lint — 0 errors (30 pre-existing no-unused-vars warnings, none in changed files)
• New tests exercise the parser against verbatim recorded production envelopes, not hand-written fixtures — the payloads in tests/lib/ai-model.test.js are the actual bytes Workers AI returned on 2026-09-04.

prettier -c flags prompts.js and memory-cloude.js, but they fail identically on origin/main — pre-existing drift, deliberately not reformatted to keep the diff at 32 lines instead of ~500.

Separated adversarial review

Reviewed by claude-sonnet-4-5 on a different host and provider (reviewer ≠ implementer). It raised two CRITICALs; both were refuted against the source, and the refutations are the reason no code changed:

1. "JSON.parse('') crashes at 3–4 sites." All three sit inside try/catch with fallbacks (context-consciousness try@186/catch@215; cognitive-coordination try@351/catch@390 and try@473/catch@518). More decisively, the old code was JSON.parse(response.response) — on an empty envelope that parsed undefined and threw the same SyntaxError into the same catch. No new crash path.
2. "Missed call sites." Enumerated every AI.run in src/ — nine total, seven changed, two correct untouched (above).

Its remaining points (extractAiText(...) || null in relationship-engine, resolveAiModel env-safety) describe deliberate preservation of existing contracts. Its request for mocked integration tests was declined: adding vi.mock-style coverage is barred by the repo's no-mocks rule.

CI

All 23 required checks pass — Lint (20, 22), Test (20, 22), Build, CodeQL/Analyze ×5, Security Scan, governance, gates, compliance, identity-onboarding, pr-validation.

Not in this PR — deploy is a human gate

Merging deploys via Workers Builds, so auto-merge is deliberately not enabled. Post-deploy verification should be: re-probe the sampling route, then GET /memory/session/:sessionId on any of the 248 recovered records and confirm a real summary replaces "Failed to generate summary."

The second half of the MemoryCloude defect is not addressed here: hasVectorize: false because ai_search_namespaces is commented out in all three wrangler.jsonc env blocks (:140, :278, :417), with no commit explaining why. So the 248 records were never embedded and are not semantically searchable. That needs an account-side namespace check plus a backfill decision, and is filed separately.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DZbwet4A5CENYvuS1KAbSX

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

[coderabbitai]

Warning

Review limit reached

Next included review available in 44 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Team

Run ID: 0b5c8e71-6920-421a-8ec0-c1e44677cdc7

📥 Commits

Reviewing files that changed from the base of the PR and between 8adfefe and 0324b68.

📒 Files selected for processing (11)

• src/api/routes/mcp.js
• src/api/routes/prompts.js
• src/intelligence/cognitive-coordination.js
• src/intelligence/context-consciousness.js
• src/intelligence/intent-predictor.js
• src/intelligence/memory-cloude.js
• src/intelligence/relationship-engine.js
• src/lib/ai-model.js
• tests/intelligence/memory-cloude.test.js
• tests/intelligence/relationship-engine.test.js
• tests/lib/ai-model.test.js

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[Copilot]

🟡 Changes recommended

Two small parsing/caching edge cases should be fixed to ensure fallback text extraction works when response is present-but-empty and to prevent caching whitespace-only summaries.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Fixes intelligence-layer Workers AI failures by removing reliance on the deprecated @cf/meta/llama-3.1-8b-instruct model and centralizing model selection + response-text extraction.

Changes:

• Introduces src/lib/ai-model.js to resolve the Workers AI model via AI_MODEL_PRIMARY (with a safe default) and to extract generated text from multiple envelope shapes.
• Updates all impacted Workers AI call sites (intelligence + routes) to use the shared resolver/extractor instead of hardcoded model IDs and direct .response reads.
• Adds Vitest coverage for model resolution and envelope parsing using recorded production envelopes.

File summaries

File	Description
tests/lib/ai-model.test.js	Adds tests for model resolution defaults/overrides and robust text extraction across envelope shapes.
src/lib/ai-model.js	New shared resolver (resolveAiModel) and envelope parser (extractAiText) to avoid hardcoded model IDs and brittle response reads.
src/intelligence/relationship-engine.js	Switches to resolved model and shared text extractor for relationship summaries.
src/intelligence/memory-cloude.js	Switches to resolved model + extractor and avoids caching empty AI summaries.
src/intelligence/intent-predictor.js	Switches to resolved model and shared text extractor before JSON extraction.
src/intelligence/context-consciousness.js	Switches to resolved model and parses JSON from extracted text.
src/intelligence/cognitive-coordination.js	Switches to resolved model and parses JSON from extracted text in multiple AI paths.
src/api/routes/prompts.js	Routes prompt execution through shared model resolver and extractor.
src/api/routes/mcp.js	Updates MCP sampling default model and response extraction via shared utilities.

Review details

• Files reviewed: 9/9 changed files
• Comments generated: 2
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[chitcommit]

Post-merge verification: this merged but did not deploy. Production is still running pre-#290 code.

Ran the verification this PR specified. Summaries still return "Failed to generate summary.":

POST /api/intelligence/memory/summarize {"sessionId":"session-1777931666-130901"}
-> {"success":true,"summary":"Failed to generate summary."}

That alone is ambiguous — it could be an undeployed merge or a bad fix — so I probed a route that distinguishes them. /mcp/sampling/sample with no model hint falls back to a hardcoded model in the old code and to resolveAiModel(c.env) after this PR:

POST /mcp/sampling/sample {"messages":[...],"maxTokens":16}
-> {"error":"5028: @cf/meta/infire-llama-3.1-8b-instruct was deprecated on 2026-05-30..."}

The deployed worker is still reaching for the dead model by default. That code path only exists before this PR, so the code serving connect.chitty.cc predates this merge. The fix itself is not in question — it is simply not running.

Corroborating, not conclusive on its own: wrangler deployments list --name chittyconnect shows the most recent deployment as 2026-08-08, a month before this merged. I'm flagging that as corroboration rather than proof because this repo uses service environments, so the script actually serving the route may carry a different name than the one I queried. The sampling probe is the authoritative signal — it tests the running code regardless of which script name serves it.

Why it did not deploy: there is no deploy workflow in .github/workflows/ (checked for wrangler deploy, deploy:production, and cloudflare/wrangler-action — none present), which is correct per CLAUDE.md, since deployment is meant to run through Cloudflare Workers Builds. So the likely cause is a Workers Builds trigger that is missing, misconfigured, or failing for this worker. Confirming that needs an account token with Workers Builds Configuration access, which is credential-gated — routing through /chico rather than fetching one.

Consequence worth stating plainly: every intelligence-layer AI path in this repo remains dead in production, and will stay dead after any further merge, until the deploy path is repaired. That includes #296. Merging is currently not sufficient to ship.