fix(models): 6 ModelRegistry alias and credential-resolution tests are red on dev

[probepark]

Summary

packages/coding-agent/test/model-registry.test.ts has 6 failing tests on current dev. They are not caused by any open PR - I attributed them while reviewing #4561 and confirmed they reproduce on an unmodified base checkout.

Four cover alias/provider-selection contracts and two cover credential resolution, so provider selection is meaningfully unguarded at the moment.

Reproduction

Clean detached worktree at dev @ 2e3ccb589, fresh bun install --frozen-lockfile, natives rebuilt:

$ bun test packages/coding-agent/test/model-registry.test.ts
(fail) ModelRegistry > active provider resolution > keeps credentialless discovery active with an irrelevant dangling selector
(fail) ModelRegistry > generic local OpenAI-compatible provider config > uses stored credentials for OpenAI-compatible providers without inline auth
(fail) ModelRegistry > provider selection policy and alias resolution > alias A cannot select a sibling variant that only has alias B in the same record
(fail) ModelRegistry > provider selection policy and alias resolution > fails closed for known aliases with no eligible variants
(fail) ModelRegistry > provider selection policy and alias resolution > preserves full model and wire ids when resolving via alias
(fail) ModelRegistry > provider selection policy and alias resolution > resolves final-slash-segment aliases through canonical records

Sample failure:

packages/coding-agent/test/model-registry.test.ts:1399
expect(resolved?.provider).toBe("demo");

Expected: "demo"
Received: "github-copilot"

Why it matters

These are not cosmetic assertions:

• fails closed for known aliases with no eligible variants - a fail-open alias resolution silently routes a request to some other provider instead of erroring. The sample failure above is exactly that shape: a fixture expecting demo resolves to github-copilot.
• alias A cannot select a sibling variant that only has alias B in the same record - alias isolation within a record. Without it, one alias can reach a variant it was never meant to address.
• uses stored credentials for OpenAI-compatible providers without inline auth - credential resolution for local/compatible providers.
• keeps credentialless discovery active with an irrelevant dangling selector - discovery must not be disabled by an unrelated stale selector.

A provider-selection change landing today would not be caught by any of these.

Acceptance

• All six pass on dev, or each one that encodes an intent the product deliberately no longer has is rewritten to the current contract with the reasoning recorded - not deleted silently.
• If the demo -> github-copilot resolution is now correct behaviour, the alias fixtures need updating and the fail-closed assertion needs to be re-expressed against whatever the new fallback rule is, since "fails closed" and "falls back to another provider" cannot both be true.

Context

Found during exact-head review of #4561 (47da2b9b4), which introduces zero new failures - head and base both report the same 6 in the same file set. Filing separately so it is not mistaken for that PR's problem.

[Yeachan-Heo]

Investigation and fix execution started on dedicated branch fix/issue-4582-model-registry-dev-red (worktree at exact origin/dev head f024ded614). Plan: (1) independently reproduce all six failures, (2) classify each as implementation regression vs. intentionally changed contract, (3) implement the smallest correct fix preserving fail-closed alias isolation, canonical/wire ids, final-segment aliases, stored credentials for OpenAI-compatible providers, and credentialless discovery under irrelevant dangling selectors, (4) run focused + adjacent suites and package checks, (5) open one PR targeting dev linked here with exact-head CI. Will report findings on this issue.

—
[repo owner's gaebal-gajae (clawdbot) 🦞]

[Yeachan-Heo]

Investigation complete; fix posted as #4584 (targets dev, head 75f86a3fcc, base 45885ea287 — exact current dev).

Reproduction and root cause. All six reproduce deterministically on unmodified dev (f35cf0e3a2 and 45885ea287) — but only when the launching shell exports provider credentials. With a scrubbed env and fresh HOME, the full file is 269 pass / 0 fail on both heads, and CI's shard harness (which scrubs credential env) passes too. Adding back just OPENAI_API_KEY + GH_TOKEN produces exactly the reported six:

• The four alias/provider-selection failures: GH_TOKEN/GITHUB_TOKEN make bundled github-copilot models credential-active, so resolveModelByLookupAlias(…, { candidates: registry.getAll() }) legitimately ranks the copilot claude-sonnet-4.5 variant over the demo fixture (your sample: expected demo, received github-copilot). The fail-closed test stops failing closed because an unrelated provider's variant becomes eligible through getAll().
• The two credential/discovery failures: ambient OPENAI_API_KEY adds { provider: "openai", connectionKind: "credential" } to getActiveProviders(), breaking the exact toEqual rows.

Classification: fixture non-hermeticity, not an implementation regression. Product alias isolation, fail-closed behavior, canonical/wire ids, final-segment aliases, stored-credential use, and credentialless-discovery persistence are all correct once ambient credentials are excluded — this is the same defect class #3207 fixed for the canonical-equivalence cases.

Fix (#4584, test-only, +63/−21, no assertion deleted or weakened): pin alias/canonical candidate sets to the fixture providers (#3207 pattern), scope the two active-provider row assertions to the fixture provider, and unset ambient anthropic credential vars in the two tests that race them. Verified: 269/269 clean, 269/269 under the reporter's pollution signature, CI shard-4 harness 175/175 files, 206 adjacent provider/model tests pass, package check and build clean.

One residual is documented rather than worked around: an ambient ANTHROPIC_API_KEY cannot be suppressed in-process (\$inheritedEnv pins the launch-time value), so ranks expired OAuth ahead of an environment key still needs a clean ANTHROPIC_API_KEY locally; CI scrubs it. Requesting exact-head review from @probepark and @HaD0Yun on #4584.

—
[repo owner's gaebal-gajae (clawdbot) 🦞]

[Yeachan-Heo]

Linked-PR hold — evidence pointer for #4584 (this issue tracks the PR, no separate mutation):

• PR head reconstructed onto current dev: a2f3ac3b3e (base 804314081f), test-file-only +63/−21, canonical digest 87748eacd90810a4e8299eb5bee4a7fef364df6ea03eebec15bd4436cf480e8a. The body now carries exactly one canonical needs-human verdict line; bootstrap 31884312121 parses it and is red only on Verdict needs-human intentionally blocks merge — the honest pending-review state. Product jobs green.
• Hermetic fail-today/green evidence (fresh-HOME env -i matrices): base reproduces exactly the six reported failures under OPENAI_API_KEY+GH_TOKEN; head is 269/0 clean and under every individual polluter; adjacent suites 0-fail clean and polluted; coding-agent check/build green; state-writer gate 0 violations. Full signed matrix on the PR.
• Fresh exact-head reviews requested from @probepark and @HaD0Yun. Verdict promotes to merge-approved only from an actual write-authorized non-author approval; then squash merge to dev and this issue closes via Fixes #4582.

—
[repo owner's gaebal-gajae (clawdbot) 🦞]

[probepark]

Correction - my diagnosis in this issue was wrong. Closing.

#4584 identifies the actual cause, and it is not what I wrote here.

What I claimed

That six ModelRegistry tests were red on dev, that alias/provider-selection and credential
resolution were "meaningfully unguarded", and that the Expected: "demo" / Received: "github-copilot"
signature suggested a fail-open alias resolution routing requests to another provider.

What is actually happening

The fixtures were not hermetic. They absorbed ambient host credentials. My machine carries
BROWSER_USE_LLM_API_KEY, CPA_LOCAL_API_KEY, JUNIE_API_KEY and others in the environment, plus
state under ~/.gjc/agent/, so provider resolution correctly preferred a credentialed provider over
the fixture's demo. The product behaved properly; the test environment was the variable.

Verified on the same machine, same command, only the head differing:

# dev @ 804314081, unmodified
 263 pass  6 fail    <- the exact six reported here

# #4584 @ a2f3ac3b3
 269 pass  0 fail

What I should have done

The signature was the tell. A fixture expecting demo and receiving github-copilot is far more
likely to be credential resolution finding something real than an alias table silently rerouting -
a genuine fail-open would not have picked a provider that happens to be configured on my laptop.
I should have run with a scrubbed environment before asserting a product defect, and I should not
have escalated to "provider selection is unguarded" from a single-machine observation.

What was legitimate

Non-hermetic tests are a real defect - a suite whose result depends on the developer's credentials
is one that cannot be trusted locally, and it hid nothing but also proved nothing. #4584 fixes that
properly, including try/finally restore so a failing assertion cannot leave credentials unset
for later suites in the same run.

No product change is needed. Closing as resolved by #4584.