Skip to content

[FEATURE] Gate server-side compaction through model metadata capabilities #5466

Description

@jimbrub

Human Overview

Whether a model can compact server-side, and whether its thinking is bound to history, should come from model metadata rather than a hardcoded list in AnthropicModel.

SDK Language

Both / Not language-specific

Problem Statement

#5401 gates native compaction on a private list of model IDs and a temporary AnthropicModelConfig.serverCompaction override. That has to be kept by hand, is specific to Anthropic, and doesn't let other providers declare the capability.

Proposed Solution

Once #5305 lands:

  • Gate on model.metadata.capabilities.serverCompaction (server_compaction in Python) and capabilities.reasoning.historyBound.
  • Move the model ID list into AnthropicModel.declareMetadata(), using the defaults from [FEATURE] Test native compaction on other providers that serve preserved-thinking Claude models #5465.
  • Replace the temporary config key with the metadata config key, for example metadata: { capabilities: { serverCompaction: true } }. Keep the old key as a deprecated alias for one release.
  • BedrockModel declares serverCompaction: false.

Done when:

  • No compaction code reads the private ID list or the temporary key.
  • Unit tests cover declared, overridden, and unknown capability values.

Use Case

A gateway user sets serverCompaction: false through model metadata, the same way they override any other capability.

Additional Context

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area-contextSession or context relatedarea-modelRelated to models or model providersenhancementNew feature or request

Fields

Language

None yet

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions