Repository navigation
Python: fix(contents): accept RFC 6838 media types and parameters in data URIs - #14663
Chen Yufeiyang (feiiiiii5) wants to merge 4 commits into
Conversation
The data URI parser only allowed letters and hyphens in the media type and treated everything after the first semicolon as the data format, so image/svg+xml was rejected and charset=utf-8;base64 left the base64 payload undecoded. Parse the media type and parameters with the RFC 6838 token character set and keep the optional data format as a token.
There was a problem hiding this comment.
🟡 Changes recommended
The revised parser rejects valid parameter characters and shorthand URIs that omit the media type.
2 open issues
What changed in this PR
This PR updates Python’s data URI parser to accept additional media-type characters and correctly separate parameters from the base64 marker.
Changes:
- Expands the parser’s media-type and parameter character classes.
- Adds regression tests for SVG, vendor media types, and charset parameters before base64.
- Tests
ImageContentconstruction from an SVG data URI.
| File | Description |
|---|---|
python/tests/unit/contents/test_image_content.py |
Adds SVG data URI construction coverage. |
python/tests/unit/contents/test_data_uri.py |
Adds media-type and parameter parsing cases. |
python/semantic_kernel/contents/utils/data_uri.py |
Revises the data URI parsing expression. |
Review effort: Balanced
| # Parameters use the RFC 2045 token alphabet, which is broader than a | ||
| # media type's restricted-name alphabet. RFC 2397 allows parameters even | ||
| # when the default text/plain media type is omitted. | ||
| parameter_token = r"[a-zA-Z0-9!#$%&'*+.^_`|~-]+" |
There was a problem hiding this comment.
The current pre-commit job fails on this assignment with Bandit B105:hardcoded_password_string (https://github.com/microsoft/semantic-kernel/actions/runs/40984502209). This is a false positive: the value is a regex, not a credential, but the variable name parameter_token triggers the check. Please rename it to something like parameter_pattern so the existing hook passes without weakening the security checks.
There was a problem hiding this comment.
Done — renamed the local regex variable to parameter_pattern (both the definition and the two references in the f-string). Bandit B105 no longer fires; I left the token alphabet and behavior unchanged and did not add # nosec. Verified locally with the same pre-commit hook (bandit passes) and pytest tests/unit/contents/test_data_uri.py (48 passed).
The local regex variable was named parameter_token, which Bandit flags as B105 (possible hardcoded password) even though it is a character-class pattern. Rename it to parameter_pattern instead of suppressing the check. Same behavior; pre-commit bandit hook and tests/unit/contents/test_data_uri.py pass.

Motivation and Context
Valid RFC 2397 data URIs are rejected or silently mis-decoded when the media type contains
+(for exampleimage/svg+xml) or when a parameter precedesbase64(for exampledata:text/plain;charset=utf-8;base64,...).ImageContent(data_uri="data:image/svg+xml;base64,PHN2Zy8+")raisesContentInitializationError: Invalid data uri format.DataUri.from_data_uri("data:text/plain;charset=utf-8;base64,SGVsbG8=")does not raise, but returns the base64 text as bytes instead of decoding it, becausecharset=utf-8;base64is treated as one data-format string. Both are valid data URIs, and the second is a silent data-corruption path.Description
The parser now uses RFC 6838/RFC 2045 token characters for the media type and parameters, and keeps the optional data format as a token. This accepts
image/svg+xml, vendor types such asapplication/vnd.ms-excel, and;charset=utf-8;base64without treating the parameter as part of the base64 marker.Tests added for the media-type
+, a parameter beforebase64, a vendor media type, andImageContentconstruction from an SVG data URI. On the base commit483ffb4, the four new tests fail; on this branch the related contents/prompt/template/text suites are 1,111 passed.ruff checkandruff format --checkare clean.Contribution Checklist
ruff checkandruff format --checkpass on the changed files.