You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[FEATURE] Model canary: smoke-test every model in the context-window tables nightly #5634
The nightly integration run of main (#5503) includes the per-SDK model smoke tables added in #5452, so every model in those tables gets a real tool-using turn and a context-window check each night. The tables only cover the October 2026 catalog batch. Models already in the context-window tables before that batch, including the Claude, Nova and GPT families, have no row, so a provider-side change to one of them is found by users rather than by the canary.
Proposed Solution
Add a smoke row for every model in _CONTEXT_WINDOW_LIMITS (Python) and CONTEXT_WINDOW_LIMITS (TypeScript) that the integration accounts can reach, and add a check that fails when a model is in the context-window table without a smoke row. Failures surface through the canary issues from #5577.
Activity