Skip to content

Rewrite BitRouter vs LiteLLM around claims that hold up - #63

Merged
SPIKESPIGEL404 merged 3 commits into
mainfrom
claude/bitrouter-litellm-rewrite-afa23a
Aug 7, 2026
Merged

Rewrite BitRouter vs LiteLLM around claims that hold up#63
SPIKESPIGEL404 merged 3 commits into
mainfrom
claude/bitrouter-litellm-rewrite-afa23a

Conversation

@SPIKESPIGEL404

@SPIKESPIGEL404 SPIKESPIGEL404 commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

Why

LiteLLM's GitHub tagline is now "The fastest, litest AI Gateway. Rust core with Python SDK" and they publish benchmarks at ~0.05ms added overhead / 31.7MB memory. Our comparison page claimed they were a Python gateway at ~20ms+ versus our ~5ms p50.

That's a fight we lose by two orders of magnitude, using their published numbers, on the one page whose entire job is credibility. This reframes the argument onto ground that survives their roadmap.

The new argument, in order

  1. All-in-one proxy vs. a router that learns — promoted to lead. Feature coverage is a race they win by definition; a router that folds outcomes back into policy is the one difference they can't ship past.
  2. Self-serve cloud vs. enterprise-only — verified against litellm.ai/pricing: quote-only, no public price, SSO/audit logs/RBAC behind it.
  3. Rust-accelerated vs. Rust-native — demoted and reframed off language speed. Their recommended mode is a PyO3 bridge inside the Python wheel; Python still owns auth, routing, callbacks, spend tracking. The standalone Axum server is beta with fewer routes and no Docker support. Their own Stage 3 endgame keeps Python in a sidecar for plugins, guardrails, and SSO — which is why this claim doesn't expire when the Dec 1 2026 transition lands.

Concessions added

The page previously read as a strawman. It now concedes, in the prose and the table and the migration guide:

  • LiteLLM's in-process library (from litellm import completion) — we have no answer, and the page says so
  • 100+ providers plus embeddings / rerank / audio, which BitRouter doesn't ship out of the box
  • Enterprise checkboxes they have shipped and we haven't
  • Their sub-millisecond Rust overhead, stated plainly as their result

On the security paragraph

Describes the 2026 unauthenticated RCE by incident class — command injection in MCP endpoints chained with a host-header bypass upstream — without CVE numbers or the CVSS score, notes both were disclosed and patched responsibly, and explicitly disclaims immunity: our MCP gateway executes tools too, and Rust prevents none of that bug class. The claim is smaller blast radius, not safety.

Deliberate call. Citing a competitor's CVSS 10.0 reads as FUD and ages badly the first time we ship our own advisory. Easy to harden if you disagree.

Also fixed

Compare table 11 rows → 5; routing-overhead row dropped entirely rather than kept with a number we can't defend
Platform fee 2% stablecoin / 5% card0% markup on the OpenRouter and Portkey tables
Terminal animations Asserted ~85ms p99 · Python GIL and LiteLLM: none of the above (false — they ship MCP). Rewritten with no numbers; marks changed red ✗ → amber ⚠
CompareTerminal Registered in mdx-components.tsx but missing from COMPONENT_WHITELIST, so any page using it failed lint:docs. Added
Migration guide Claimed LiteLLM was "self-hosted only" — flatly wrong, and it undercut our own self-serve argument

The terminal programs are corrected but not rendered on the page — no <CompareTerminal> blocks, same as before this PR. The whitelist entry stays because the registry/whitelist desync is real either way.

Numbers

No hard performance numbers are asserted for BitRouter. The Rust section reads "cold-starting in well under a second and holding tens of megabytes resident" — conservative qualitative bands, chosen because the section argues dependency surface rather than speed. Worth replacing with real measurements if this page ever needs to carry a benchmark claim.

Known follow-ups (pre-existing, not touched here)

  • The migration guide uses 127.0.0.1:4356 while the rest of the site uses :8787 — and it's in a copy-paste path.
  • The 0% markup fix may collide with a parallel branch that already renders none · 0% markup.

Verification

pnpm build clean · pnpm test 83/83 · pnpm lint:docs 49 docs pass · page rendered and checked locally

🤖 Generated with Claude Code

SPIKESPIGEL404 and others added 3 commits August 7, 2026 11:46
LiteLLM now ships a Rust core ("Rust core with Python SDK") and publishes
benchmarks at ~0.05ms added overhead. Our page claimed they were a Python
gateway at ~20ms+ against our ~5ms p50 — a comparison we lose on their own
numbers, on a page whose whole job is credibility.

Reframed the argument onto ground that survives their December roadmap:

- Lead with all-in-one vs. focused-and-extensible around the act → observe
  → evaluate → learn loop. It's the one difference they can't ship past.
- Promote self-serve cloud vs. quote-only Enterprise (verified against
  litellm.ai/pricing).
- Demote Rust to "Rust-accelerated vs. Rust-native": their recommended mode
  is a PyO3 bridge in the Python wheel, the standalone server is beta with
  fewer routes, and their own endgame keeps Python in a sidecar for plugins
  and SSO. Argue dependency surface, not language speed.
- Concede properly: LiteLLM's in-process library and its embeddings/rerank/
  audio coverage are real advantages we don't match. Both now appear in the
  table and the migration guide, not just the prose.

The security paragraph describes the 2026 RCE by incident class rather than
CVE number, and explicitly disclaims immunity — our MCP gateway executes
tools too, and Rust prevents none of that bug class.

Also:
- Trim the compare table 11 rows → 5; drop the routing-overhead row.
- Correct platform fee to 0% markup on the OpenRouter and Portkey tables.
- Rewrite the LiteLLM terminal animations, which asserted ~85ms p99 / GIL
  and "LiteLLM: none of the above" (false — they ship MCP). No numbers.
- Wire two terminals into the page and whitelist CompareTerminal, which was
  registered in mdx-components but missing from the docs lint whitelist.
- migrate-from-litellm claimed LiteLLM was "self-hosted only". It isn't.

Numbers left as TK- placeholders pending real benchmarks.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…page

The page can't ship with TK- markers rendering literally, so the three
numbers are now qualitative:

- LiteLLM's overhead is "sub-millisecond" — their published 0.05ms, stated
  as their result. Conceding it plainly is stronger than hedging, since the
  paragraph's point is that the distinction is where the Rust stops, not
  speed.
- BitRouter "cold-starting in well under a second and holding tens of
  megabytes resident" — conservative bands, still worth replacing with
  measurements before this is load-bearing anywhere.

Also removes both <CompareTerminal> blocks from the page. The rewritten
terminal programs stay in compare-programs.tsx (correct now, just unused
again), as does the CompareTerminal whitelist entry — the component is
registered in mdx-components.tsx, so the whitelist was out of sync
regardless of whether a page uses it today.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PR #62 landed the OpenRouter rewrite and its own platform-fee fix, hitting
the same registry file. Resolution:

- LiteLLM rows: ours (the 5-row trim is what this PR is for)
- OpenRouter rows: main's — #62 rewrote that comparison deliberately, and
  it is not this PR's business
- Portkey fee row: main's "none · 0% markup" phrasing, now the house style,
  superseding the equivalent change made here

Net effect: this branch's diff against main is now only the LiteLLM entry.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@SPIKESPIGEL404
SPIKESPIGEL404 merged commit 6bb6010 into main Aug 7, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant