Skip to content

fix: declare long-context Claude and GPT metadata - #223

Open
dgokeeffe wants to merge 3 commits into
databricks:mainfrom
dgokeeffe:fix/1m-context-claude-gpt-pi
Open

fix: declare long-context Claude and GPT metadata#223
dgokeeffe wants to merge 3 commits into
databricks:mainfrom
dgokeeffe:fix/1m-context-claude-gpt-pi

Conversation

@dgokeeffe

@dgokeeffe dgokeeffe commented Jul 18, 2026

Copy link
Copy Markdown

Summary

Fixes long-context metadata for Claude and GPT gateway models and centralizes Claude capability policy across agents.

Closes #222.

Changes

  • Pi emits explicit Claude contextWindow, maxTokens, reasoning, image, and adaptive-thinking metadata instead of accepting Pi's 128k/4k custom-model defaults.
  • Pi emits model-specific GPT limits, including the opt-in 1.05M windows for supported GPT tiers.
  • Claude family/version parsing and capability thresholds now live in one shared helper in databricks.py.
  • Pi metadata, Pi adaptive-thinking compatibility, and Claude Code's [1m] selector consume the same capability result.
  • Handles dotted, system.ai.*, databricks-*, provider-qualified, [1m]-suffixed, and future major-version IDs.
  • Opus 4.5, Haiku, and unknown models retain the conservative 200k fallback; Fable 5 uses its verified native 1M/128k capability without an added [1m] suffix.

Claude Code behavior

This PR intentionally changes Claude Code, not only Pi: Sonnet 4.5+ receives the [1m] suffix, while Opus receives it from 4.6 onward. Sonnet 4.5 has the 1M beta window with a 64k output cap. The PR description calls this out explicitly so the Claude agent behavior is reviewed as part of the change.

Review follow-up

The duplicated version policy previously split between agents/claude.py and databricks.py has been removed. Table-driven coverage now checks Opus 4.5/4.6/5, Sonnet 4.4/4.5/4.6/5, Haiku, Fable, malformed IDs, prefixes, and existing [1m] suffixes.

Validation

  • uv run pytest tests/test_databricks.py tests/test_agent_claude.py tests/test_agent_pi.py -q — passed in the final review loop
  • uv run ruff check src/ucode/databricks.py src/ucode/agents/claude.py src/ucode/agents/pi.py tests/test_databricks.py tests/test_agent_claude.py tests/test_agent_pi.py

Stacking / review scope

This PR is stacked on #217. Review the incremental commit/diff from 65b4609 to 636c3b6:

65b4609...636c3b6

After #217 merges, this branch can be rebased directly onto main.

Comment thread src/ucode/databricks.py Outdated
Comment thread src/ucode/agents/claude.py Outdated
@dgokeeffe

Copy link
Copy Markdown
Author

@rohita5l I refreshed this stacked PR: Claude capability/version policy is now shared by Pi and Claude Code, the Sonnet 4.5 Claude Code behavior is explicit in the description, and both threads have detailed replies and are resolved. Could you re-review and approve the fork CI run when convenient?

@dgokeeffe
dgokeeffe force-pushed the fix/1m-context-claude-gpt-pi branch from f2fbb83 to 636c3b6 Compare July 26, 2026 04:38
@dgokeeffe

Copy link
Copy Markdown
Author

Post-review update: shared capability policy now reflects Sonnet 4.6 at 1M/64k and Fable 5 at native 1M/128k with adaptive thinking but no [1m] suffix. GPT matching is boundary-safe/case-insensitive, excludes GPT-OSS from Responses selection, and preserves generic Responses endpoints while preferring the newest semantic GPT. PR body/comparison link updated.

@dgokeeffe

Copy link
Copy Markdown
Author

Pushed 8d66253 to fix a 400 this PR would otherwise introduce.

_pi_gpt_model_entry set reasoning: True without a thinkingLevelMap. Pi's Responses builder treats a missing off-state as effort: "none" (pi-ai/dist/api/openai-responses.js, the thinkingLevelMap?.off !== null branch):

else if (model.provider !== "github-copilot" && model.thinkingLevelMap?.off !== null) {
    params.reasoning = { effort: (model.thinkingLevelMap?.off ?? "none") };
}

"none" is only valid on gpt-5.1+, so with thinking off, four ids rejected every request:

BAD_REQUEST: Unsupported value: 'none' is not supported with the 'gpt-5' model.
Supported values are: 'minimal', 'low', 'medium', and 'high'.

Measured against /ai-gateway/codex/v1/responses:

Model effort:"none" effort:"minimal" omitted
gpt-5, gpt-5-mini, gpt-5-nano 400 200 200
gpt-5-5-pro 400 400 200
gpt-5-1/-2/-3-codex/-4/-4-mini/-4-nano/-5/-6-luna/-6-sol/-6-terra 200 200 200

Fix is entry["thinkingLevelMap"] = {"off": None}, which makes Pi omit reasoning entirely — 200 on all 14 ids. {"off": "minimal"} was the other option but gpt-5-5-pro rejects it too, so it would need a hardcoded per-model exclusion list against a dynamically discovered model set. Same {"off": None} pattern we already use for the Gemini 3.x entries.

Also bisected the rest of Pi's Responses payload against the gateway; all 200, no other compat flags needed: store:false, prompt_cache_key, prompt_cache_retention:"24h", prompt_cache_options, include:["reasoning.encrypted_content"], developer role, flat tool schemas with additionalProperties:false, and the underscore session_id / x-client-request-id affinity headers.

Added two regression tests in tests/test_agent_pi.py (every gpt-5 entry pins {"off": None}; non-gpt codex entries stay bare) plus a docstring note on why the override is mandatory whenever reasoning: True is declared.

uv run pytest --ignore=tests/test_e2e.py --ignore=tests/test_e2e_tracing.py → 1102 passed, 6 skipped. The two test_e2e_user_agent failures I see locally are pre-existing on this branch (identical at 4984d37 without my commit) and need local agent binaries, so they skip in the test job.

Two things for reviewers:

  1. _pi_claude_model_entry in this same diff has the same shapereasoning: True, no thinkingLevelMap. It doesn't bite today because Claude goes through anthropic-messages, which never reaches the Responses reasoning branch. Left alone deliberately, but it's a latent trap if that ever moves.
  2. The trade in this fix: thinking-off is now delegated to the gateway default rather than explicitly requested as zero reasoning. That seemed clearly better than 400ing, but flagging it in case exact off-semantics matter.

Worth noting this was painful to diagnose because the gateway returns {"error_code","message"} instead of OpenAI's {"error":{...}}, so Pi's error-body.js recovery no-ops and every 400 renders as OpenAI API error (400): 400 status code (no body). Reported upstream: earendil-works/pi#7748. Tracking issue: #286.

@dgokeeffe

dgokeeffe commented Aug 7, 2026

Copy link
Copy Markdown
Author

Rebased this PR stack onto current main (67429ec) and force-pushed the existing branch. New head: 94fe107.

Stack dependency

This is 2 of 3 and must land after #217:

  1. feat(pi): add validated GLM/Kimi MLflow provider #217 — Pi MLflow OSS provider
  2. fix: declare long-context Claude and GPT metadata #223 — shared Claude/GPT metadata + Pi GPT thinking-off fix (this PR)
  3. fix(opencode): add GPT routing alongside OSS discovery #239 — OpenCode GPT routing + OSS discovery fallback

The rebase preserves main's #282 contract: Pi pins codex_models[0] verbatim; it does not re-sort for the newest GPT. The preferred_gpt_model helper is not part of this PR anymore—it is introduced with its OpenCode consumer in #239.

94fe107 contains the thinkingLevelMap: {"off": None} fix discussed above and tracked in #286. It prevents Pi from sending unsupported reasoning: {effort: "none"} to gpt-5/-mini/-nano/-5-5-pro.

If #217 is squash-merged, please rebase this branch onto the updated main before merging.

Validation

  • CI-parity suite: 1,648 passed, 6 skipped
  • Only local failures: the two pre-existing CLI capture-server tests (TestClaudeUserAgent, TestPiUserAgent), which skip in CI's unit-test job
  • ruff check and ruff format --check: clean
  • No committed uv.lock churn or conflict markers

Managed allowlist × live-discovery intersection is deliberately outside this PR and tracked in #290.

Expose the GLM and Kimi coding-model cohort through Pi and OpenCode with shared token limits and reasoning metadata. Keep unsupported chat models out of discovery, including Inkling until gateway issue databricks#215 is fixed, and retain the GPT-OSS Responses API routing guard.
Centralize Claude family/version parsing so Pi metadata, adaptive-thinking compatibility, and Claude Code's [1m] selector cannot drift. Cover Sonnet 4.5, Opus 4.6, future major versions, Fable fallback, and prefixed model IDs.
`_pi_gpt_model_entry` declared `reasoning: True` without an off-state, so for
the thinking-off case Pi's Responses builder fell back to
`reasoning: {effort: "none"}` (pi-ai openai-responses.js, the
`thinkingLevelMap?.off !== null` branch). `"none"` is only valid on gpt-5.1+,
so every request to gpt-5, gpt-5-mini, gpt-5-nano and gpt-5-5-pro was rejected:

  BAD_REQUEST: Unsupported value: 'none' is not supported with the 'gpt-5'
  model. Supported values are: 'minimal', 'low', 'medium', and 'high'.

Setting `thinkingLevelMap: {"off": None}` makes Pi omit `reasoning` entirely,
which the gateway accepts for all 14 codex ids. Verified against
/ai-gateway/codex/v1/responses: effort="none" 400s on gpt-5/-mini/-nano/-5-5-pro
and 200s on gpt-5-1..-5-6; omitting `reasoning` is 200 everywhere.
`{"off": "minimal"}` was rejected as an alternative because gpt-5-5-pro 400s on
it too. Same pattern already used for the Gemini 3.x entries.

The rest of Pi's Responses payload was bisected against the gateway and is
fine: store:false, prompt_cache_key, prompt_cache_retention:"24h",
prompt_cache_options, include:["reasoning.encrypted_content"], developer role,
flat tool schemas, and the session_id / x-client-request-id affinity headers.

Regression was hard to spot because the gateway returns
{"error_code","message"} rather than OpenAI's {"error":...}, so Pi's
error-body.js recovery no-ops and every 400 renders as
"OpenAI API error (400): 400 status code (no body)". Reported upstream as
earendil-works/pi#7748.

Refs databricks#286
@dgokeeffe
dgokeeffe force-pushed the fix/1m-context-claude-gpt-pi branch from 15ad5c0 to 94fe107 Compare August 7, 2026 10:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1M-token context not applied to Sonnet 4.5 and GPT-5 (pi)

2 participants