Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Client Content capabilities #223

Draft
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

evalstate
Copy link
Contributor

@evalstate evalstate commented Mar 24, 2025

This PR adds a new contentTypes capability to the ClientCapabilities interface, allowing clients to advertise which MIME types they can render to users and which they can tokenize for LLM consumption.

This enhancement works with the existing annotations system to optionally enable MCP Servers to adapt their content delivery to best match Host capabilities.

Motivation and Context

Different Host application/LLM pairs have different content handling requirements and capabilities (e.g. Chat Applications, IDEs, Video/Content Editing Suite, Agentic Applications).

This addition allows MCP Servers to make informed decisions to:

  • Select optimal formats for content
  • Use audience annotations more effectively
  • Gracefully enhance or degrade based on Host needs/preferences.

How Has This Been Tested?

The extension has not been directly tested, however some example scenarios are:

  • A Host application that can render but not tokenize audio/video can receive adapted Tool Results and Text Content for the LLM.
  • An MCP Server can choose to return either a application/pdf or downgrade to text/plain based on LLM capabilities.
  • An MCP Server can provide additional instructions in the Tool Result to guide the User to obtain content that could not otherwise be rendered/processed.

Breaking Changes

The change is backwards compatible.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

Checklist

  • I have read the MCP Documentation
  • My code follows the repository's style guidelines
  • New and existing tests pass locally
  • I have added appropriate error handling
  • I have added or updated documentation as needed

Additional context

This is not intended to be a complicated content-type negotiation protocol - but to provide a simple way for participating Hosts and Servers to provide better User Experiences across a range of deployment scenarios. The list of mime-types is intended to be indicative and neither restrictive nor exhaustive.

An agreed convention for Resources where audience: [User], priority: 1 is used to indicate content that should be rendered and not tokenized would further enhance the proposal. For example a PDF could be sent for rendering, with the main content sent as text/plain for the LLM.

I do not think a reciprocal server capability is necessary, as "Roots" provide the ability for the Host to provide arbitrary content to the server.

This PR has been opened for discussion and refinement, with additional documentation to be prepared if there is agreement in principle.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant