Skip to content

fix(routing): rank duplicate model ids; make Google discovery not allowlist-bound - #6

Merged
aliihsaad merged 1 commit into
mainfrom
fix/model-id-collision-and-google-scout
Jul 22, 2026
Merged

fix(routing): rank duplicate model ids; make Google discovery not allowlist-bound#6
aliihsaad merged 1 commit into
mainfrom
fix/model-id-collision-and-google-scout

Conversation

@aliihsaad

Copy link
Copy Markdown
Owner

Fixes the two follow-ups identified in #5.

1. Cross-provider model_id collision

resolveRoutableModel matched on model_id with no platform filter and used a bare .get(), so whichever row had the lowest id won. An aggregator's auto-discovered duplicate could shadow a routable model.

Observed: gemma-4-31b-it returned "Model 'gemma-4-31b-it' is disabled" — it resolved to a disabled BazaarLink row (id 23428) while Google's copy (id 26665) was enabled with 16 keys.

Candidates are now ranked: enabled → provider has a usable key → free → id. Verified live: gemma-4-31b-it now routes to google/gemma-4-31b-it.

Guard tests cover the shadowing cases and confirm the existing verdicts still hold (all-disabled → disabled, all-paid → paid_blocked, unknown → not_found).

2. Google discovery was allowlist-bound

GOOGLE_FREE_CHAT_MODELS gated what the scout could return, so every newly released Google model was structurally invisible — the reason gemini-3.6-flash and gemini-3.5-flash-lite had to be added by hand in #5.

Google's list endpoint carries no pricing, so cost can't be classified the way it can for providers that publish it. Instead, classifyGoogleModel():

  • confirmed-free id → include, enabled, free (unchanged behaviour)
  • unrecognised chat model → include but disabled + marked paid — visible in the dashboard for review, can never auto-route or spend
  • non-chat family (embedding / veo / lyria / imagen / robotics / computer-use / deep-research) → skipped as catalog noise

gemini-3.6-flash + gemini-3.5-flash-lite added to the confirmed-free map.

V18: Gemma disabled

Gemma 4 is free and now reachable, but it answers with raw chain-of-thought ("The user said 'say ok'. The user wants me to output...") rather than the answer — poor behaviour for a model in the auto-route fallback chain. Rows are disabled, not deleted; operators can enable from the dashboard. This also removes drift for databases that briefly had them enabled.

Verification

  • 235/235 passing (up from 225 — 10 new assertions), build green.
  • Live through /v1/chat/completions: gemini-3.6-flash, gemini-3.5-flash-lite, gemini-2.5-flash all still answer and route correctly after the resolver change.

…owlist-bound

Two related catalog/routing defects surfaced while adding the new Google models.

1. Cross-provider model_id collision. resolveRoutableModel matched on model_id
   with no platform filter and took a bare .get(), so whichever row had the
   lowest id won. An aggregator's auto-discovered duplicate (disabled/paid)
   could shadow a perfectly routable model — 'gemma-4-31b-it' resolved to a
   disabled BazaarLink row and reported "Model is disabled" even though
   Google's copy was enabled with keys. Candidates are now ranked: enabled
   first, then providers holding a usable key, then free, then id.

2. Google discovery was allowlist-bound. GOOGLE_FREE_CHAT_MODELS gated which
   models the scout could return, so every newly released Google model was
   structurally invisible. classifyGoogleModel() now surfaces an unrecognised
   chat model DISABLED and marked paid — visible for review, unable to
   auto-route or spend — while confirmed-free ids still auto-enable.
   gemini-3.6-flash and gemini-3.5-flash-lite added to the confirmed-free map.

V18 disables the Google Gemma rows: reachable now that the collision is fixed,
but it answers with raw chain-of-thought, which is poor behaviour for a model
in the auto-route chain. Disabled, not deleted; operators can turn it on.

Verified live: gemma-4-31b-it routes to google/gemma-4-31b-it (was 'disabled'),
and gemini-3.6-flash / 3.5-flash-lite / 2.5-flash still answer via /v1. 235/235.
@coderabbitai

coderabbitai Bot commented Jul 22, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@aliihsaad, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 46 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 2e7277e1-2994-41d0-89cf-944368c31c8d

📥 Commits

Reviewing files that changed from the base of the PR and between 0a6cfc9 and a833800.

⛔ Files ignored due to path filters (7)
  • server/dist/db/index.d.ts.map is excluded by !**/dist/**, !**/*.map
  • server/dist/db/index.js is excluded by !**/dist/**
  • server/dist/db/index.js.map is excluded by !**/dist/**, !**/*.map
  • server/dist/services/model-scout.d.ts is excluded by !**/dist/**
  • server/dist/services/model-scout.d.ts.map is excluded by !**/dist/**, !**/*.map
  • server/dist/services/model-scout.js is excluded by !**/dist/**
  • server/dist/services/model-scout.js.map is excluded by !**/dist/**, !**/*.map
📒 Files selected for processing (5)
  • server/src/__tests__/lib/resolve-model.test.ts
  • server/src/__tests__/services/google-scout.test.ts
  • server/src/db/index.ts
  • server/src/lib/resolve-model.ts
  • server/src/services/model-scout.ts
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/model-id-collision-and-google-scout

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@aliihsaad
aliihsaad merged commit 253bdf9 into main Jul 22, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant