docs: routing — the task_v1 router and the built-in judge - #446
Open
bbqiu wants to merge 1 commit into
Open
Conversation
The routing page described a single "external routing API" pointed at
task_v0, and framed the built-in judge and the external service as
alternatives you pick between. The shipped implementation has two routers
and one selector that chooses between them per decision, so the page
described neither side accurately.
Rewrite it around what routing actually does today:
- the two routers — the built-in judge from the server's llm: block (the
pure-OSS router) and the Databricks AI Gateway task_v1 router — and the
per-decision selector: task_v1 answers when it is configured and every
model family the decision involves is AI-Gateway-backed, the judge
answers otherwise, and a server with neither has no routing
- complete copy-pasteable configs for all four postures: routing off,
judge only, task_v1, and both, including that the routing: block is
auto-derived from a Databricks provider when omitted and that
routing: {provider: none} is the only true off-switch
- how to verify: /v1/info smart_routing_sources, /v1/hosts
gateway_inference, and the per-family credential rules that decide it
- what users see: decision chips and their rationale, the Databricks mark
keyed on router_source, per-family degradation to the judge rather than
hidden surfaces, the --smart-routing flag, and first-message routing in
a bare TUI
Also corrects two things the old page got wrong regardless of version:
llm: and routing: are read only from the file passed to `omni server -c`,
not picked up from ~/.omnigent/config.yaml, and provider: none disables
the judge too, not just the external router.
Co-authored-by: Isaac
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The routing implementation now ships two routers with a per-decision
selector. The docs page described neither: it documented a single "external
routing API" pointed at
task_v0, and framed the built-in judge and theexternal service as alternatives you choose between. This rewrites
/docs/build/routingaround what routing actually does today.What the page now covers:
llm:block; the pure-OSS router, works on any harness and any credential) and the
Databricks AI Gateway
task_v1router (routing.provider: external, whosepicks are gateway catalog ids).
task_v1answers when it is configured and every modelfamily the decision involves is AI-Gateway-backed on the target host; the
judge answers otherwise; a server with neither has no routing. Gateway backing
selects the source, it does not hide the feature.
task_v1,and both. Includes the two behaviors that surprise people: the
routing:blockis auto-derived from a Databricks provider when omitted, so deleting it does
not disable routing, and
routing: {provider: none}is the only trueoff-switch (it disables the judge too).
GET /v1/info→smart_routing_sources,GET /v1/hosts→per-family
gateway_inference, and the per-family credential rules that decideit (Claude needs
ANTHROPIC_BASE_URL+apiKeyHelper; Codex needs an AIGateway
/codex/v1base URL; subscriptions and Bedrock are nevergateway-backed).
Databricks mark keyed on
router_source: "databricks-aigw", per-familydegradation to the judge with a one-line CLI notice instead of hidden surfaces,
the
--smart-routingflag, and first-message routing in a bare TUI.It also corrects two things the old page got wrong independent of version:
llm:androuting:are read only from the file passed toomni server -c(they are not picked up from~/.omnigent/config.yamlthe wayproviders:is), andprovider: nonedisables the built-in judge as well as theexternal router.
The page stays doc-sized — it links Models & Credentials
and Harnesses for the deeper journeys
rather than inlining the full setup walkthrough.
Provenance
Documents omnigent PR #4074 (
routing-mvp-v3). Every claim was verifiedagainst that branch's implementation — the selector in
omnigent/server/routing_backend.py, config parsing inomnigent/cli.py, thecredential checks in
omnigent/gateway_inference.py, and the chip rendering inthe web UI — not against
main. This should land with or after #4074.Test plan
bun run fmt:check— cleanbun run lint:links— 0 broken linksbun run build— production build passeslib/og-manifest.jsonupdated for the new page description(An unrelated pre-existing drift the OG generator wants to make to
app/releases/0.5.1/page.mdxwas deliberately left out of this PR.)This pull request and its description were written by Isaac.