Description
Implement a cascading model strategy that auto-routes trivial or tool-only turns (simple reads, formatting checks) to a small/fast/cheap model, escalating to the user's "main" strong model only when task complexity warrants it.
Use Case
Saves cost and reduces latency on easy tasks (like view_file) without sacrificing quality on hard reasoning tasks.
Proposed Solution
Create source/ai-sdk-client/smart-router.ts with a classifier based on prompt length, code-block count, etc. Add a /smartroute command to toggle this behavior.
Alternatives Considered
Forcing users to manually swap models (which creates friction).
Additional Context
Description
Implement a cascading model strategy that auto-routes trivial or tool-only turns (simple reads, formatting checks) to a small/fast/cheap model, escalating to the user's "main" strong model only when task complexity warrants it.
Use Case
Saves cost and reduces latency on easy tasks (like
view_file) without sacrificing quality on hard reasoning tasks.Proposed Solution
Create
source/ai-sdk-client/smart-router.tswith a classifier based on prompt length, code-block count, etc. Add a/smartroutecommand to toggle this behavior.Alternatives Considered
Forcing users to manually swap models (which creates friction).
Additional Context