Spec
See specs/072-multimodal-mcp-passthrough/ (added in PR #6228 on branch feat/issue-5366/multimodal-mcp-passthrough).
Summary
Implements true multimodal passthrough of MCP ContentBlock::Image content to vision-capable LLM providers. Currently zeph_mcp::render_content_blocks renders images only as a [kind: mime, N bytes] text placeholder. This adds an opt-in (per MCP server, default OFF) path that decodes, validates, and attaches the image as a sibling MessagePart::Image so vision-capable providers can actually see it — gated on vision-capable tier routing (degrades to the existing text placeholder rather than a 400/422 when unavailable), with binary content validated by a new MediaSanitizer and stripped from all persistence/embedding surfaces before it reaches SQLite, Qdrant, or the durable JSONL replay log.
Audio/blob passthrough is explicitly out of scope (deferred — adding a MessagePart variant is an Ask-First decision per system invariant #4).
Acceptance criteria
Full testable list in specs/072-multimodal-mcp-passthrough/spec.md §6 (AC-1 through AC-15). Highlights:
- AC-5: no
Image part bytes appear in SQLite parts_json, Qdrant embeddings, or the durable JSONL replay log — verified by an integration test asserting all three surfaces.
- AC-3:
ToolOutput/ImageData custom redacting Debug impl — no raw bytes in logs/debug-dumps.
- AC-6: vision-tier routing gate — media-bearing turns select a vision-capable tier or degrade to the text placeholder; never a runtime 400/422.
- AC-9:
MediaSanitizer enforces magic-byte sniff, format allowlist, and size/dimension/pixel caps (5 MiB / 8192px / 64MP / 4-per-result / 8-per-turn defaults, tunable via config).
- AC-15: the three pre-assembly passes (
run_causal_ipi_post_probe, record_shadow_event, apply_acon_compression) remain Image-part-safe under refactor (invariant C5).
Per project rules, this touches provider request builders and MessagePart construction — a live cascade+MCP-image session test is mandatory before merge (LLM Serialization Gate, .claude/rules/continuous-improvement.md).
Two open questions remain in the spec (OQ-1, OQ-2) — resolve or explicitly re-affirm during implementation.
Suggested next step
Run /rust-agents:team-develop new-feature on this issue. The architect and developer will pick up specs/072-multimodal-mcp-passthrough/ automatically.
Spec
See
specs/072-multimodal-mcp-passthrough/(added in PR #6228 on branchfeat/issue-5366/multimodal-mcp-passthrough).Summary
Implements true multimodal passthrough of MCP
ContentBlock::Imagecontent to vision-capable LLM providers. Currentlyzeph_mcp::render_content_blocksrenders images only as a[kind: mime, N bytes]text placeholder. This adds an opt-in (per MCP server, default OFF) path that decodes, validates, and attaches the image as a siblingMessagePart::Imageso vision-capable providers can actually see it — gated on vision-capable tier routing (degrades to the existing text placeholder rather than a 400/422 when unavailable), with binary content validated by a newMediaSanitizerand stripped from all persistence/embedding surfaces before it reaches SQLite, Qdrant, or the durable JSONL replay log.Audio/blob passthrough is explicitly out of scope (deferred — adding a
MessagePartvariant is an Ask-First decision per system invariant #4).Acceptance criteria
Full testable list in
specs/072-multimodal-mcp-passthrough/spec.md§6 (AC-1 through AC-15). Highlights:Imagepart bytes appear in SQLiteparts_json, Qdrant embeddings, or the durable JSONL replay log — verified by an integration test asserting all three surfaces.ToolOutput/ImageDatacustom redactingDebugimpl — no raw bytes in logs/debug-dumps.MediaSanitizerenforces magic-byte sniff, format allowlist, and size/dimension/pixel caps (5 MiB / 8192px / 64MP / 4-per-result / 8-per-turn defaults, tunable via config).run_causal_ipi_post_probe,record_shadow_event,apply_acon_compression) remain Image-part-safe under refactor (invariant C5).Per project rules, this touches provider request builders and
MessagePartconstruction — a live cascade+MCP-image session test is mandatory before merge (LLM Serialization Gate,.claude/rules/continuous-improvement.md).Two open questions remain in the spec (OQ-1, OQ-2) — resolve or explicitly re-affirm during implementation.
Suggested next step
Run
/rust-agents:team-develop new-featureon this issue. The architect and developer will pick upspecs/072-multimodal-mcp-passthrough/automatically.