Skip to content

[Feature] Add protocol-based AI model connections for OpenAI and Anthropic #4451

Open

Description

@MasamiYui

Feature Request

Add named, protocol-based AI model connections for enterprise gateways, proxy services, and local models, with three explicitly selectable protocols:

  • OpenAI-compatible Chat Completions (openai-chat)
  • OpenAI-compatible Responses (openai-responses)
  • Anthropic-compatible Messages (anthropic-messages)

Implementation can be phased, initially targeting text generation, streaming, and custom tool calling.

Is your feature request related to a problem? Please describe

The current provider presets share an OpenAI-compatible Chat Completions implementation. They do not let users explicitly select Responses or Anthropic Messages, even when their gateway supports those APIs.

Configuration handling also differs between the Agent Gateway and legacy chat/SOP paths: the Gateway uses the active entry in ModelProviderConfigState, while LlmConfig and ChatClientProviderServiceImpl deserialize the same persisted content as a single ModelProviderConfig. A saved and activated connection can therefore be unavailable to the legacy path.

Describe the solution you'd like

  1. Add a connection configuration option with a name, protocol, base URL, user-supplied model ID, API key, and optional custom headers. Vendor examples can remain input helpers.
  2. Support custom URL prefixes and show the final request URL, avoiding duplicated paths such as /v1/v1. Do not silently substitute vendor endpoints or model IDs.
  3. Support protocol-default authentication, custom authentication headers, and an explicit no-authentication option. Preserve an existing key when editing without providing a replacement, and protect stored credentials and sensitive headers from disclosure.
  4. Share effective configuration resolution and protocol-specific client creation across the AI workspace, chat, and SOP. Switching the active connection should take effect without a restart, with streaming and tool calling supported.
  5. Keep YAML as the default when no saved connection is selected, and display its effective settings and credential status. An invalid active connection should produce a clear error instead of silently selecting another connection.
  6. Add a connection test that reports which capabilities were verified and returns sanitized provider errors without changing the active connection.

Existing saved connections and YAML settings should remain usable after upgrading.

Describe alternatives you've considered

Adding more vendor presets alone would still leave the API protocol implicit. Explicit protocol selection also avoids the ambiguity of automatically switching protocols after a failed request.

Automatic model discovery and new reasoning controls can be discussed separately.

Additional context

Related: #4391, which discusses provider defaults and hardcoded generation parameters. This proposal focuses on explicit protocol support and consistent configuration across the AI paths.

I am willing to contribute to the design and implementation. Feedback and discussion on the scope, approach, and compatibility are welcome.

Metadata

Metadata

Assignees

Type

No type

Projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions