Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/gpt-5.6-codex-preset-benchmark.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ Built-in role assignments are product judgments. The selected TypeScript edit ev
- **Eco**: `terra:low` default, `luna:low` executor, `luna:high` planner, `terra:xhigh` critic, and `terra:high` architect.
- **Medium**: `sol:low` default, `terra:low` executor, `terra:high` planner, `sol:xhigh` critic, and `sol:high` architect.
- **Pro**: `sol:medium` default, `terra:medium` executor, `sol:high` planner, `sol:max` critic, and `sol:xhigh` architect.
- **Combos**: `opus-codex` uses the Medium Codex executor, critic, and architect roles, with the durable `anthropic/claude-sonnet-5` planner override; `codex-opencodego` uses Medium Codex default and architect roles; and `fable-opus-codex` uses Pro Codex executor and architect roles with `anthropic/claude-opus-4-8:medium` as planner.
- **Combos**: `opus-codex` uses the Medium Codex executor, critic, and architect roles, with the durable `anthropic/claude-sonnet-5` planner override; `codex-opencodego` uses Medium Codex default and architect roles; and `fable-opus-codex` uses Pro Codex executor and architect roles with `anthropic/claude-opus-5:medium` as planner.

The edit benchmark does not measure default-agent interpretation, orchestration, explanation, or routing, and it does not measure planner, architect, or critic work. Those non-executor assignments are product judgments, not benchmark findings.

Expand Down Expand Up @@ -119,9 +119,9 @@ The selected-task data show that Luna xhigh used more reported tokens than Luna
| `codex-eco` | `openai-codex/gpt-5.6-terra:low` | `openai-codex/gpt-5.6-luna:low` | `openai-codex/gpt-5.6-luna:high` | `openai-codex/gpt-5.6-terra:xhigh` | `openai-codex/gpt-5.6-terra:high` |
| `codex-medium` | `openai-codex/gpt-5.6-sol:low` | `openai-codex/gpt-5.6-terra:low` | `openai-codex/gpt-5.6-terra:high` | `openai-codex/gpt-5.6-sol:xhigh` | `openai-codex/gpt-5.6-sol:high` |
| `codex-pro` | `openai-codex/gpt-5.6-sol:medium` | `openai-codex/gpt-5.6-terra:medium` | `openai-codex/gpt-5.6-sol:high` | `openai-codex/gpt-5.6-sol:max` | `openai-codex/gpt-5.6-sol:xhigh` |
| `opus-codex` | `anthropic/claude-opus-4-8:xhigh` | `openai-codex/gpt-5.6-terra:low` | `anthropic/claude-sonnet-5` | `openai-codex/gpt-5.6-sol:xhigh` | `openai-codex/gpt-5.6-sol:high` |
| `opus-codex` | `anthropic/claude-opus-5:xhigh` | `openai-codex/gpt-5.6-terra:low` | `anthropic/claude-sonnet-5` | `openai-codex/gpt-5.6-sol:xhigh` | `openai-codex/gpt-5.6-sol:high` |
| `codex-opencodego` | `openai-codex/gpt-5.6-sol:low` | `opencode-go/deepseek-v4-pro` | `opencode-go/kimi-k2.6` | `opencode-go/mimo-v2.5-pro` | `openai-codex/gpt-5.6-sol:high` |
| `fable-opus-codex` | `anthropic/claude-fable-5:high` | `openai-codex/gpt-5.6-terra:medium` | `anthropic/claude-opus-4-8:medium` | `anthropic/claude-opus-4-8:high` | `openai-codex/gpt-5.6-sol:xhigh` |
| `fable-opus-codex` | `anthropic/claude-fable-5:high` | `openai-codex/gpt-5.6-terra:medium` | `anthropic/claude-opus-5:medium` | `anthropic/claude-opus-5:high` | `openai-codex/gpt-5.6-sol:xhigh` |

## Limitations

Expand Down
4 changes: 2 additions & 2 deletions docs/models.md
Original file line number Diff line number Diff line change
Expand Up @@ -231,11 +231,11 @@ Built-in profiles are grouped by provider mix and tier:

- `codex-{eco,medium,pro}` — GPT-5.6 Sol/Terra/Luna role mixes tuned by tier and reasoning effort
- `opencodego` — single OpenCode Go preset (Kimi default, DeepSeek executor/architect, Qwen planner, MiMo critic)
- `claude-opus` — Anthropic OAuth preset centered on `claude-opus-4-8`
- `claude-opus` — Anthropic OAuth preset centered on `claude-opus-5`
- Single-provider tiers: `glm-{eco,medium,pro}`, `kimi-coding-plan-{eco,medium,pro}`, `mimo-{eco,medium,pro}`, `grok-{eco,medium,pro}`, `cursor-{eco,medium,pro}`, `minimax-{eco,medium,pro}`
- Combos: `opus-codex`, `codex-opencodego`, and `fable-opus-codex`

The `eco`, `medium`, and `pro` Codex profile mappings are current product judgments: Eco assigns Terra low/Luna low/Luna high/Terra xhigh/Terra high to default/executor/planner/critic/architect; Medium assigns Sol low/Terra low/Terra high/Sol xhigh/Sol high; and Pro assigns Sol medium/Terra medium/Sol high/Sol max/Sol xhigh. `opus-codex` retains the Medium Codex executor, critic, and architect roles but uses `anthropic/claude-sonnet-5` for planner; `codex-opencodego` retains the Medium Codex default and architect roles; and `fable-opus-codex` uses the Pro Codex executor and architect roles with `anthropic/claude-opus-4-8:medium` for planner. The descriptive repeated local exact-edit evidence informs only selected executor-style TypeScript tasks; it does not evaluate or prove default, planner, architect, or critic performance. See [GPT-5.6 Codex preset benchmark](./gpt-5.6-codex-preset-benchmark.md). Effort suffixes are clamped to each model's supported thinking range at preview and activation time. Single-provider tiers pin each provider's current flagship (`zai/glm-5.2`, `kimi-code/kimi-k2.7-code`, `xiaomi/mimo-v2.5-pro`, `xai/grok-4.3`, `cursor/composer-1.5`, `minimax-code/minimax-m3`). User-defined profiles override built-ins by exact profile name.
The `eco`, `medium`, and `pro` Codex profile mappings are current product judgments: Eco assigns Terra low/Luna low/Luna high/Terra xhigh/Terra high to default/executor/planner/critic/architect; Medium assigns Sol low/Terra low/Terra high/Sol xhigh/Sol high; and Pro assigns Sol medium/Terra medium/Sol high/Sol max/Sol xhigh. `opus-codex` retains the Medium Codex executor, critic, and architect roles but uses `anthropic/claude-sonnet-5` for planner; `codex-opencodego` retains the Medium Codex default and architect roles; and `fable-opus-codex` uses the Pro Codex executor and architect roles with `anthropic/claude-opus-5:medium` for planner. The descriptive repeated local exact-edit evidence informs only selected executor-style TypeScript tasks; it does not evaluate or prove default, planner, architect, or critic performance. See [GPT-5.6 Codex preset benchmark](./gpt-5.6-codex-preset-benchmark.md). Effort suffixes are clamped to each model's supported thinking range at preview and activation time. Single-provider tiers pin each provider's current flagship (`zai/glm-5.2`, `kimi-code/kimi-k2.7-code`, `xiaomi/mimo-v2.5-pro`, `xai/grok-4.3`, `cursor/composer-1.5`, `minimax-code/minimax-m3`). User-defined profiles override built-ins by exact profile name.


Use `gjc --mpreset <name>` to activate a profile for the current session only. Activation hard-blocks when any provider listed in `required_providers` lacks credentials. Add `--default` to persist the selected profile as `modelProfile.default` in `config.yml`, so it applies at startup:
Expand Down
30 changes: 15 additions & 15 deletions docs/multi-vendor-profiles.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

A practical guide to picking models for GJC's roles, for every subscription situation — one vendor, two vendors, or the full multi-vendor set. It adds curated cross-vendor `profiles:` for `~/.gjc/agent/models.yml` and verified selector notes on top of the mechanism in [Model profiles](./models.md#model-profiles---mpreset). Everything here is **user config**; it complements the built-in `--mpreset` presets and overrides a built-in only when it shares its exact name.

> Selectors, prices, and "axis leaders" are catalog- and time-sensitive (observed 2026-06 on the current bundled catalog). Re-verify any selector with `gjc -p --no-session --no-tools --model <selector> "Reply OK"`.
> Selectors, prices, and "axis leaders" are catalog- and time-sensitive (selectors and prices observed 2026-07 on the current bundled catalog; the measured latency and single-message-limit notes below were observed 2026-06 on `claude-opus-4-8` and have not been re-measured on `claude-opus-5`). Re-verify any selector with `gjc -p --no-session --no-tools --model <selector> "Reply OK"`.

## The five roles

Expand Down Expand Up @@ -36,7 +36,7 @@ profiles:
daily: # everyday balance
required_providers: [anthropic, openai-codex, google-antigravity, xai]
model_mapping:
default: anthropic/claude-opus-4-8:medium
default: anthropic/claude-opus-5:medium
executor: openai-codex/gpt-5.4:high
planner: google-antigravity/gemini-3.1-pro-low:high
architect: google-antigravity/gemini-3.1-pro-low:high
Expand All @@ -45,16 +45,16 @@ profiles:
ultimate: # cost-no-object, best per role
required_providers: [anthropic, openai-codex, google-antigravity, xai]
model_mapping:
default: anthropic/claude-opus-4-8:high
executor: anthropic/claude-opus-4-8:max
default: anthropic/claude-opus-5:high
executor: anthropic/claude-opus-5:max
planner: openai-codex/gpt-5.5:xhigh
architect: google-antigravity/gemini-3.1-pro-low:high
critic: xai/grok-4.3:high

eco: # cheapest delegated work; main loop stays on Opus
required_providers: [anthropic, opencode-go, google-antigravity, xai]
model_mapping:
default: anthropic/claude-opus-4-8:low
default: anthropic/claude-opus-5:low
executor: opencode-go/deepseek-v4-flash
planner: xai/grok-4-1-fast:high
architect: google-antigravity/gemini-3.1-pro-low
Expand All @@ -63,19 +63,19 @@ profiles:
monorepo: # huge codebases (openai-codex excluded: 272k context cap)
required_providers: [anthropic, google-antigravity, opencode-go]
model_mapping:
default: anthropic/claude-opus-4-8:medium
executor: anthropic/claude-opus-4-8:high
default: anthropic/claude-opus-5:medium
executor: anthropic/claude-opus-5:high
planner: google-antigravity/gemini-3.1-pro-low:high
architect: anthropic/claude-opus-4-8:high
architect: anthropic/claude-opus-5:high
critic: opencode-go/glm-5.2

reviewer: # review/audit stance — the author-mode role split, inverted
required_providers: [anthropic, openai-codex, google-antigravity]
model_mapping:
default: anthropic/claude-opus-4-8:high # aggregator restraint: preserve raw reviewer verdicts
default: anthropic/claude-opus-5:high # aggregator restraint: preserve raw reviewer verdicts
executor: openai-codex/gpt-5.5:high # support — repro PoCs, failing tests, harnesses
planner: google-antigravity/gemini-3.1-pro-low:high # review checklists / audit scoping
architect: anthropic/claude-opus-4-8:high # lead 1 — primary code-review judge (effective long-context)
architect: anthropic/claude-opus-5:high # lead 1 — primary code-review judge (effective long-context)
critic: openai-codex/gpt-5.5:high # lead 2 — merge gate, cross-family vs Claude-authored code
```

Expand All @@ -101,22 +101,22 @@ Current axis leaders and the cheaper second option, with metered price ($/1M in/

| Need | First pick | Cheaper option |
| --- | --- | --- |
| Router / tool-calling (`default`) | `anthropic/claude-opus-4-8` (5/25) | `anthropic/claude-sonnet-5` (3/15) |
| Coding (`executor`) | `anthropic/claude-opus-4-8` SWE-bench Verified ~88.6 (5/25) | `openai-codex/gpt-5.4` (2.5/15) · `opencode-go/deepseek-v4-flash` (0.14/0.28) |
| Router / tool-calling (`default`) | `anthropic/claude-opus-5` (5/25) | `anthropic/claude-sonnet-5` (3/15) |
| Coding (`executor`) | `anthropic/claude-opus-5` (5/25) — the prior `claude-opus-4-8` scored SWE-bench Verified ~88.6; no Opus 5 measurement yet | `openai-codex/gpt-5.4` (2.5/15) · `opencode-go/deepseek-v4-flash` (0.14/0.28) |
| Reasoning (`planner`) | `openai-codex/gpt-5.5` (ARC-AGI-2) / `google-antigravity/gemini-3.1-pro-low:high` (GPQA) | `xai/grok-4-1-fast` (0.2/0.5) |
| Large context (`architect`) | `anthropic/claude-opus-4-8` (effective long-context) | `xai/grok-4-fast` (2M nominal, 0.2/0.5) |
| Large context (`architect`) | `anthropic/claude-opus-5` (effective long-context) | `xai/grok-4-fast` (2M nominal, 0.2/0.5) |
| Multimodal review (`architect`) | `google-antigravity/gemini-3.1-pro-low:high` | `google-antigravity/gemini-3.5-flash` |
| Independent critic | `xai/grok-4.3` (1.25/2.5) | `opencode-go/glm-5.2` · `google-antigravity/gemini-3.5-flash` |

On standard tasks, all current frontier models in the catalog are accurate; **pick by cost, latency, and role fit, not by raw accuracy on easy prompts.** As an indicative GJC-routed latency reference (`gjc -p`, identical coding + reasoning prompts, all correct): `grok-4.3` and `glm-5.2` ≈ 2–3s, `deepseek-v4-pro` ≈ 3–4s, `claude-opus-4-8` / `gpt-5.5` ≈ 4–7s, `gemini-3.1-pro-low:high` ≈ 7s.
On standard tasks, all current frontier models in the catalog are accurate; **pick by cost, latency, and role fit, not by raw accuracy on easy prompts.** As an indicative GJC-routed latency reference (`gjc -p`, identical coding + reasoning prompts, all correct): `grok-4.3` and `glm-5.2` ≈ 2–3s, `deepseek-v4-pro` ≈ 3–4s, `claude-opus-4-8` / `gpt-5.5` ≈ 4–7s, `gemini-3.1-pro-low:high` ≈ 7s. `claude-opus-5` shares Opus 4.8's published context/output envelope but has not been latency-measured here.

## Verified selector notes (current catalog)

Observed via live `gjc -p` calls; useful when wiring the profiles above:

- **Antigravity Gemini, high reasoning** → use `google-antigravity/gemini-3.1-pro-low:high`. The id `gemini-3.1-pro-high` returns HTTP 400 (no matching backend model); `thinkingLevel` is a per-request parameter, so raising it on `gemini-3.1-pro-low` invokes the model's native high-reasoning mode rather than a degraded one.
- **openai-codex on a ChatGPT account** serves base GPT only (`gpt-5.5`, `gpt-5.4`). Standalone `-codex` variants (`gpt-5.3-codex`, `gpt-5.2-codex`, `gpt-5.1-codex-max` / `-mini`) return `not supported when using Codex with a ChatGPT account`.
- **Single-message input limit is separate from the context window.** `claude-opus-4-8` runs with a 1M window via multi-turn accumulation, but a single `@file` message above ~400k tokens returns 400 on `anthropic` / `google-antigravity`; `xai` / `opencode-go` accept larger single messages. Chunk very large inputs across turns instead of pasting one block.
- **Single-message input limit is separate from the context window.** Measured on `claude-opus-4-8` (not yet re-measured on `claude-opus-5`, which publishes the same 1M window): the model runs with a 1M window via multi-turn accumulation, but a single `@file` message above ~400k tokens returns 400 on `anthropic` / `google-antigravity`; `xai` / `opencode-go` accept larger single messages. Chunk very large inputs across turns instead of pasting one block.
- **Some selectors come from a provider's live catalog, not the bundled snapshot.** `opencode-go/glm-5.2` and `google-antigravity/gemini-3.5-flash` resolved in `gjc -p` tests but are **not** in `packages/ai/src/models.json`; they appear only after the provider's online model discovery has populated the registry. `required_providers` verifies credentials at activation — it does **not** guarantee fresh, non-stale discovery — so activation can still fail with `selector did not resolve` until discovery runs (re-login or retry to refresh). If you hit that, substitute a bundled id: `opencode-go/deepseek-v4-pro` for the critic, or `zai/glm-5.2` (add `zai` to `required_providers`) for GLM 5.2.

## Activation
Expand Down
46 changes: 43 additions & 3 deletions packages/ai/scripts/generate-models.ts
Original file line number Diff line number Diff line change
Expand Up @@ -281,14 +281,54 @@ function applyCodexPricingFallback(models: readonly Model[]): Model[] {
});
}

// Catalog sources occasionally omit image input for Claude Opus 4.8 variants
// Catalog sources occasionally omit image input for recent Claude Opus variants
// (e.g. kilo/venice "-fast" entries) even though every Claude Opus model is
// vision-capable. Correct those so capability advertising stays consistent
// across providers. Runs after the dynamic merge so it survives regeneration.
//
// The list is an explicit allowlist of reviewed generations rather than a
// `claude-opus-*` prefix match: a future generation must be reviewed before we
// assert capabilities for it. `claude-opus-vision.test.ts` imports this list and
// fails when the catalog bundles a newer Opus generation than any declared here.
export const VISION_CORRECTED_CLAUDE_OPUS_GENERATIONS: readonly number[] = [4.8, 5];

/**
* Known separator-less generation aliases. Upstream normally writes
* `claude-opus-4-5`, but a few catalogs collapse it to `claude-opus-45`. This is
* an explicit list so a future two-digit major (`claude-opus-10`) is read as
* generation 10 rather than silently as 1.0.
*/
const COMPACT_CLAUDE_OPUS_ALIASES: Readonly<Record<string, number>> = {
"41": 4.1,
"45": 4.5,
"46": 4.6,
"47": 4.7,
"48": 4.8,
};

/**
* Extract the Claude Opus generation from a model id, ignoring provider
* prefixes, region prefixes, and trailing aliases or date suffixes:
* `claude-opus-4-8` and `anthropic.claude-opus-4-8` -> 4.8, `claude-opus-5-fast`
* -> 5, `claude-opus-45` -> 4.5, `claude-opus-4-20250514` -> 4,
* `claude-opus-10` -> 10. Returns undefined when the id is not a Claude Opus
* model.
*/
export function claudeOpusGeneration(modelId: string): number | undefined {
const match = modelId
.toLowerCase()
.replace(/\./g, "-")
.match(/claude-opus-(\d+)(?:-(\d)(?![\d]))?/);
if (!match) return undefined;
const [, major, minor] = match;
if (minor !== undefined) return Number(major) + Number(minor) / 10;
return COMPACT_CLAUDE_OPUS_ALIASES[major] ?? Number(major);
}

function applyClaudeOpusVisionCorrections(models: readonly Model[]): Model[] {
return models.map(model => {
const normalizedId = model.id.toLowerCase().replace(/\./g, "-");
if (!normalizedId.includes("claude-opus-4-8")) {
const generation = claudeOpusGeneration(model.id);
if (generation === undefined || !VISION_CORRECTED_CLAUDE_OPUS_GENERATIONS.includes(generation)) {
return model;
}
if (model.input.includes("image")) {
Expand Down
Loading
Loading