Skip to content

Expose model cost rates in the models contract (AgentModelRow) #5

Description

@alexanderkreidich

Split out of appx-org/agent-client#2 (per-session usage/cost metrics), implemented in appx-org/agent-client#4.

AssistantMessage.usage.cost is often zero for custom LiteLLM-routed models, so consumers recalculate cost from per-million-token rates. AgentModelRow (GET /v1/sessions/models) and the model row embedded in GET .../sessions/{id}/settings expose contextWindow but no cost rates, which forces every consumer to maintain its own rates map (create-appx-app currently uses LITELLM_COST_*_PER_MILLION env vars plus a hard-coded gpt-5.5 default).

Proposal: add an optional cost object to AgentModelRow:

cost?: {
  input: number;      // USD per 1M input tokens
  output: number;     // USD per 1M output tokens
  cacheRead: number;  // USD per 1M cache-read tokens
  cacheWrite: number; // USD per 1M cache-write tokens
}

pi's models.json already carries per-model cost for built-in providers, so for those this is just a passthrough; custom providers could accept the same shape in CustomProviderModel. Consumers would then drop their env-based rates maps and pass model.cost straight into agent-client's aggregateSessionUsage as costRates.

🤖 Generated with Claude Code

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions