Skip to content

fix(providers): share the error reader, cover the last swallowed message, raise timeouts - #11

Merged
aliihsaad merged 2 commits into
mainfrom
fix/provider-errors-and-timeouts
Aug 23, 2026
Merged

fix(providers): share the error reader, cover the last swallowed message, raise timeouts#11
aliihsaad merged 2 commits into
mainfrom
fix/provider-errors-and-timeouts

Conversation

@aliihsaad

Copy link
Copy Markdown
Owner

Finishes the provider-error and timeout work started in #9. All counts are from the VPS request log (4618 requests, 2205 errors, 47.7% failure rate).

1. One shared error reader

readProviderErrorText moves from openai-compat.ts into base.ts and gains the Cloudflare errors[0].message shape, so all four provider classes use one implementation instead of five hand-rolled expressions. Shapes confirmed by probing each provider with an invalid token on 2026-08-23:

Shape Providers
{error:{message}} Groq, OpenRouter, SambaNova, Zhipu, Vercel, Kilo, Google, GitHub
{type,title,status,detail} NVIDIA NIM (RFC 7807)
{message} Cerebras, Cohere
{detail} Mistral
{error:"string"} Hugging Face
{errors:[{message}]} Cloudflare

2. Cohere — the last swallowed message

cohere.ts is its own class, so #9's fix never reached it. Cohere answers {"id":"…","message":"…"}, so every failure displayed as bare status text: Cohere API error 401: Unauthorized.

3. Nested 2xx error bodies

OpenRouter reports upstream failures as HTTP 200 with {error:{code,…}} and frequently no message, putting the reason in metadata.raw and the upstream's name in metadata.provider_name. throwIfOpenAIErrorBody defaulted straight to 'Provider returned error' — which is why 531 of the 682 logged OpenRouter failures read OpenRouter API error 429: Provider returned error and told us nothing.

4. Timeouts

BaseProvider gains an overridable defaultTimeoutMs; previously only OpenAICompatProvider could set one, leaving Google, Cohere and Cloudflare stuck on 15s with no way out.

Raised to 60s — logged aborts in parentheses: google (67), sambanova (31), mistral (11), zhipu (7), llm7 (4), plus openrouter, kilo, pollinations, cohere, cloudflare on catalog risk. Pollinations was measured at 29.9s across three trials, so it aborted every single time.

An abort also costs a 120s key cooldown in classifyProviderError, so a slow provider benches itself for two minutes per timeout — the multiplier that made this worth fixing.

Groq and Cerebras deliberately stay at 15s: both run custom inference silicon and logged zero aborts across 161 and 30 failures. A test pins that so nobody "fixes" it later.

Verification

  • 302/302 server tests pass, tsc clean
  • Each fix verified by reverting it and confirming the new tests fail. The reverted output reproduces the exact production strings — OpenRouter API error 429: Provider returned error and Cohere API error 401: Unauthorized
  • Reverting checks the TypeScript source, not dist: vitest resolves the source, so patching dist proves nothing

🤖 Generated with Claude Code

…age, raise timeouts

Finishes the provider-error and timeout work started in #9. Counts below are
from the VPS request log (4618 requests, 2205 errors).

SHARED ERROR READER. readProviderErrorText moved from openai-compat.ts into
base.ts and given the Cloudflare `errors[0].message` shape, so all four
provider classes use one implementation. Documented shapes confirmed by
probing each provider with an invalid token on 2026-08-23:

  {error:{message}}          Groq, OpenRouter, SambaNova, Zhipu, Vercel,
                             Kilo, Google, GitHub
  {type,title,status,detail} NVIDIA NIM (RFC 7807)
  {message}                  Cerebras, Cohere
  {detail}                   Mistral
  {error:"string"}           Hugging Face
  {errors:[{message}]}       Cloudflare

COHERE was the last provider still reading only error.message. It answers
{"id":"...","message":"..."}, so every failure displayed as bare statusText
("Cohere API error 401: Unauthorized"). Its own class meant #9's fix did not
reach it.

NESTED 2xx ERROR BODIES. OpenRouter reports upstream failures as HTTP 200
with {error:{code,...}} and often no `message`, putting the reason in
metadata.raw and the upstream's name in metadata.provider_name.
throwIfOpenAIErrorBody defaulted straight to 'Provider returned error',
which is why 531 of the 682 logged OpenRouter failures read "OpenRouter API
error 429: Provider returned error" and told us nothing.

TIMEOUTS. BaseProvider gained an overridable defaultTimeoutMs so the
hand-written classes can raise it — previously only OpenAICompatProvider
could. Raised to 60s: google (67 aborts), sambanova (31), mistral (11),
zhipu (7), llm7 (4), plus openrouter, kilo, pollinations, cohere and
cloudflare on catalog risk. Pollinations was measured at 29.9s across three
trials, so it aborted every single time.

Groq and Cerebras deliberately stay at 15s: both run custom inference
silicon and logged zero aborts across 161 and 30 failures.

Each fix was verified by reverting it and confirming the new tests fail —
the reverted output reproduces the exact production strings.
@coderabbitai

coderabbitai Bot commented Aug 23, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 94b3653f-9b19-4165-9ea1-87a4277f1519


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@aliihsaad
aliihsaad merged commit e86b781 into main Aug 23, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant