Is your feature request related to a problem? Please describe.
DataBlock is documented as the forward-looking unified container for all binary modalities (image / audio / video / file), replacing the legacy ImageBlock / AudioBlock / VideoBlock subclasses. However, none of the provider message converters currently handle it:
DashScopeMessageConverter — convertToMultimodalContent() and hasMediaContent() skip DataBlock
GeminiMessageConverter — no DataBlock branch; extractSourceFromBlock() throws on DataBlock
AnthropicMessageConverter — no DataBlock branch
OpenAIMessageConverter — no DataBlock branch
As a result, when a user builds a message with DataBlock, every provider silently drops it. The LLM receives no media content and responds as if no resource was attached.
There is also a related gap in URLSource: it only stores a URL string, with no field to carry a MIME type hint. When the URL has no file extension (CDN URLs, signed URLs, API-generated links), MediaUtils.determineMediaType() falls back to application/octet-stream, making it impossible for converters to route the block to the correct image / audio / video slot.
Describe the solution you'd like
-
Add an optional mimeType field to URLSource — fully backward-compatible (null means fall back to extension-based inference). This lets callers explicitly declare the media type when the URL has no extension.
-
Add DataBlock support to all provider converters — using the following resolution order for MIME type:
Base64Source → use getMediaType() directly
URLSource → use mimeType field if set, otherwise fall back to MediaUtils.determineMediaType(url)
- If MIME type cannot be resolved, throw a clear exception instead of silently skipping
Route to the appropriate provider-native slot based on the resolved MIME prefix (image/*, audio/*, video/*).
Describe alternatives you've considered
- Adding a
mimeType field directly to DataBlock — rejected because Base64Source already carries mediaType and URLSource can carry it too; duplicating the field on DataBlock would be redundant.
- Keeping
DataBlock as an internal-only type and requiring callers to use legacy block types — contradicts the documented intent ("new code should prefer DataBlock").
Additional context
Affected files:
agentscope-core/.../message/URLSource.java — needs optional mimeType field
agentscope-core/.../formatter/dashscope/DashScopeMediaConverter.java
agentscope-core/.../formatter/dashscope/DashScopeMessageConverter.java
agentscope-core/.../formatter/gemini/GeminiMediaConverter.java
agentscope-core/.../formatter/gemini/GeminiMessageConverter.java
agentscope-core/.../formatter/anthropic/AnthropicMediaConverter.java
agentscope-core/.../formatter/anthropic/AnthropicMessageConverter.java
agentscope-extensions/.../openai/formatter/OpenAIMessageConverter.java
Is your feature request related to a problem? Please describe.
DataBlockis documented as the forward-looking unified container for all binary modalities (image / audio / video / file), replacing the legacyImageBlock/AudioBlock/VideoBlocksubclasses. However, none of the provider message converters currently handle it:DashScopeMessageConverter—convertToMultimodalContent()andhasMediaContent()skipDataBlockGeminiMessageConverter— noDataBlockbranch;extractSourceFromBlock()throws onDataBlockAnthropicMessageConverter— noDataBlockbranchOpenAIMessageConverter— noDataBlockbranchAs a result, when a user builds a message with
DataBlock, every provider silently drops it. The LLM receives no media content and responds as if no resource was attached.There is also a related gap in
URLSource: it only stores a URL string, with no field to carry a MIME type hint. When the URL has no file extension (CDN URLs, signed URLs, API-generated links),MediaUtils.determineMediaType()falls back toapplication/octet-stream, making it impossible for converters to route the block to the correct image / audio / video slot.Describe the solution you'd like
Add an optional
mimeTypefield toURLSource— fully backward-compatible (null means fall back to extension-based inference). This lets callers explicitly declare the media type when the URL has no extension.Add
DataBlocksupport to all provider converters — using the following resolution order for MIME type:Base64Source→ usegetMediaType()directlyURLSource→ usemimeTypefield if set, otherwise fall back toMediaUtils.determineMediaType(url)Route to the appropriate provider-native slot based on the resolved MIME prefix (
image/*,audio/*,video/*).Describe alternatives you've considered
mimeTypefield directly toDataBlock— rejected becauseBase64Sourcealready carriesmediaTypeandURLSourcecan carry it too; duplicating the field onDataBlockwould be redundant.DataBlockas an internal-only type and requiring callers to use legacy block types — contradicts the documented intent ("new code should preferDataBlock").Additional context
Affected files:
agentscope-core/.../message/URLSource.java— needs optionalmimeTypefieldagentscope-core/.../formatter/dashscope/DashScopeMediaConverter.javaagentscope-core/.../formatter/dashscope/DashScopeMessageConverter.javaagentscope-core/.../formatter/gemini/GeminiMediaConverter.javaagentscope-core/.../formatter/gemini/GeminiMessageConverter.javaagentscope-core/.../formatter/anthropic/AnthropicMediaConverter.javaagentscope-core/.../formatter/anthropic/AnthropicMessageConverter.javaagentscope-extensions/.../openai/formatter/OpenAIMessageConverter.java