Skip to content

[FEATURE] Automated model onboarding and updates #5803

Open
0 of 2 issues completed
Open

Description

@mkmeral

Human Overview

SDK Language

Python, TypeScript

Problem Statement

New models ship every week. Today we learn about them from a scheduled bot run outside this repo or from bug reports, and each one is added by hand: a shared/model-info.json entry, a smoke-test row, an end-to-end run, harness effort levels. Changes to existing models (new limits, caching support, deprecations) go unnoticed until something breaks.

Proposed Solution

A scheduled job in this repo keeps shared/model-info.json current and opens a PR for every new, changed or retired model:

  • Seed from models.dev and the providers' own model APIs where they exist (Anthropic, Gemini, Mistral), with a curated overrides file on top for early-access models, Bedrock ids and anything we've verified ourselves.
  • Validate the data in CI (schema, output limit ≤ context window, a diff against the previous version) and record each entry's source and check date.
  • Each new model gets the same checklist: model info entry, end-to-end test run ([FEATURE] Model canary: smoke-test every model in the context-window tables nightly #5634), and any capability that features read (effort levels, caching, thinking).
  • No runtime fetch: the data ships with each release.

Per-model onboarding epics are tracked as sub-issues of this one.

Use Case

A developer picks up a model released this week and it works with the right limits, caching and reasoning settings, without waiting for someone to patch a table.

Additional Context

Part of #5636. Per-model onboarding in flight: #5558 (GPT-6.1 Terra), #5711 (GPT-6 Rosalind).

Metadata

Metadata

Assignees

No one assigned

    Labels

    area-modelRelated to models or model providersenhancementNew feature or request

    Fields

    Language

    None yet

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions