fix(routing): rank duplicate model ids; make Google discovery not allowlist-bound - #6
Conversation
…owlist-bound Two related catalog/routing defects surfaced while adding the new Google models. 1. Cross-provider model_id collision. resolveRoutableModel matched on model_id with no platform filter and took a bare .get(), so whichever row had the lowest id won. An aggregator's auto-discovered duplicate (disabled/paid) could shadow a perfectly routable model — 'gemma-4-31b-it' resolved to a disabled BazaarLink row and reported "Model is disabled" even though Google's copy was enabled with keys. Candidates are now ranked: enabled first, then providers holding a usable key, then free, then id. 2. Google discovery was allowlist-bound. GOOGLE_FREE_CHAT_MODELS gated which models the scout could return, so every newly released Google model was structurally invisible. classifyGoogleModel() now surfaces an unrecognised chat model DISABLED and marked paid — visible for review, unable to auto-route or spend — while confirmed-free ids still auto-enable. gemini-3.6-flash and gemini-3.5-flash-lite added to the confirmed-free map. V18 disables the Google Gemma rows: reachable now that the collision is fixed, but it answers with raw chain-of-thought, which is poor behaviour for a model in the auto-route chain. Disabled, not deleted; operators can turn it on. Verified live: gemma-4-31b-it routes to google/gemma-4-31b-it (was 'disabled'), and gemini-3.6-flash / 3.5-flash-lite / 2.5-flash still answer via /v1. 235/235.
|
Warning Review limit reached
Next review available in: 46 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (7)
📒 Files selected for processing (5)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Fixes the two follow-ups identified in #5.
1. Cross-provider
model_idcollisionresolveRoutableModelmatched onmodel_idwith no platform filter and used a bare.get(), so whichever row had the lowest id won. An aggregator's auto-discovered duplicate could shadow a routable model.Observed:
gemma-4-31b-itreturned "Model 'gemma-4-31b-it' is disabled" — it resolved to a disabled BazaarLink row (id 23428) while Google's copy (id 26665) was enabled with 16 keys.Candidates are now ranked: enabled → provider has a usable key → free → id. Verified live:
gemma-4-31b-itnow routes togoogle/gemma-4-31b-it.Guard tests cover the shadowing cases and confirm the existing verdicts still hold (all-disabled →
disabled, all-paid →paid_blocked, unknown →not_found).2. Google discovery was allowlist-bound
GOOGLE_FREE_CHAT_MODELSgated what the scout could return, so every newly released Google model was structurally invisible — the reasongemini-3.6-flashandgemini-3.5-flash-litehad to be added by hand in #5.Google's list endpoint carries no pricing, so cost can't be classified the way it can for providers that publish it. Instead,
classifyGoogleModel():gemini-3.6-flash+gemini-3.5-flash-liteadded to the confirmed-free map.V18: Gemma disabled
Gemma 4 is free and now reachable, but it answers with raw chain-of-thought (
"The user said 'say ok'. The user wants me to output...") rather than the answer — poor behaviour for a model in the auto-route fallback chain. Rows are disabled, not deleted; operators can enable from the dashboard. This also removes drift for databases that briefly had them enabled.Verification
/v1/chat/completions:gemini-3.6-flash,gemini-3.5-flash-lite,gemini-2.5-flashall still answer and route correctly after the resolver change.