From ad70bab06f01acddbe4eee459a939ae35598be77 Mon Sep 17 00:00:00 2001 From: hamr0 Date: Tue, 16 Jun 2026 10:56:37 +0200 Subject: [PATCH 1/4] =?UTF-8?q?feat(mcp):=20attachment=20reach=20=E2=80=94?= =?UTF-8?q?=20attachments[]=20+=20download=5Fasset=20tool?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Surfaces message attachments and adds the MCP-only path to their bytes, so a remote/MCP-only deployment (publishing only :23375) can read files that previously required the raw Beeper API on :23373. - normalizeMessage now carries attachments[] ({type, file_name, mime_type, src_url, size, is_voice_note}); [] when none. A MEDIA message previously arrived as "[MEDIA]" with no way to reach the file. - New download_asset tool (12 tools now): returns attachment bytes as base64, by src_url or chat_id+message_id(+index). Proxies GET /v1/assets/serve; capped at BEEPERBOX_MAX_ASSET_BYTES (default 8 MiB) via Content-Length pre-check + buffered post-check. Security: - Allowlist mxc:// / localmxc:// and refuse file:// (arbitrary local file read / secret exfil) and http(s):// (SSRF) BEFORE the fetch, covering both the caller src_url and a message-resolved src_url. - Per-call serve timeout (BEEPERBOX_ASSET_TIMEOUT_MS, default 30s) bounds a hung/slow-drip source. Tests: +3 unit (normalizeAttachments) +3 unit (scheme guard); new scripts/asset-serve-check.js exercises the I/O path against a stub Beeper API (binary integrity, both cap paths, error branches, scheme refusal) and is wired into the mcp-test unit job. CI has no live account, so the serve proxy stays validate-against-real-account before production media flows. Co-Authored-By: Claude Opus 4.8 --- .github/workflows/mcp-test.yml | 5 ++ CHANGELOG.md | 15 ++++ README.md | 2 +- beeperbox.context.md | 12 ++- docs/GUIDE.md | 10 ++- docs/PRD.md | 13 ++-- mcp/server.js | 121 ++++++++++++++++++++++++++++++ mcp/server.test.js | 80 ++++++++++++++++++++ scripts/asset-serve-check.js | 133 +++++++++++++++++++++++++++++++++ scripts/smoke-test.sh | 4 +- 10 files changed, 384 insertions(+), 11 deletions(-) create mode 100644 scripts/asset-serve-check.js diff --git a/.github/workflows/mcp-test.yml b/.github/workflows/mcp-test.yml index a55aa0e..8ff58d8 100644 --- a/.github/workflows/mcp-test.yml +++ b/.github/workflows/mcp-test.yml @@ -17,6 +17,7 @@ on: - 'entrypoint.sh' - 'docker-compose.yml' - 'scripts/mcp-guard-check.sh' + - 'scripts/asset-serve-check.js' - '.github/workflows/mcp-test.yml' permissions: @@ -36,6 +37,10 @@ jobs: node-version: '20' - name: Unit tests run: node --test mcp/server.test.js + - name: download_asset integration check (stub Beeper API, no account) + # Spins the real MCP server against a stub /v1/assets/serve and exercises + # the tough turns: binary integrity, both byte-cap paths, error branches. + run: node scripts/asset-serve-check.js mcp-guard-matrix: runs-on: ubuntu-latest diff --git a/CHANGELOG.md b/CHANGELOG.md index 315c863..cbd1012 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,21 @@ beeperbox follows [Semantic Versioning 2.0.0](https://semver.org/) with one conc Published tags on GHCR: `:X.Y.Z` (exact, immutable), `:X.Y` (rolling within a minor), `:X` (rolling within a major — always `:0` today), `:latest` (newest release tag, rebuilt weekly to pick up upstream Beeper AppImage drift), `:edge` (every push to `master`, may break). +## [Unreleased] `[MINOR]` + +### Added + +- **Attachment reach for media messages.** Every normalized `Message` now carries an `attachments[]` array (`[]` when none); each entry is `{type, file_name, mime_type, src_url, size, is_voice_note}`, a pure passthrough of the raw Beeper message's `attachments[]`. Previously a media message arrived as `type: "MEDIA"`, `text: "[MEDIA]"` with no way to reach the file — an agent could see that a file existed but never read it. `src_url` (an `mxc://` / `localmxc://` / `file://` URL) is the download reference. +- **New MCP tool `download_asset`** (12 tools now). Fetches an attachment's actual bytes and returns them base64-encoded — the MCP-only path to a file's content, so a remote deployment publishing only `:23375` (not the raw Beeper API on `:23373`) can still read attachments. Reference the attachment by `src_url`, or by `chat_id` + `message_id` (+ optional `index`) in which case beeperbox resolves the `src_url` and also returns the file's `file_name` / `mime_type` / `size`. Internally proxies `GET /v1/assets/serve?url=…`. The payload rides base64 inside the JSON-RPC result, so it is capped at `BEEPERBOX_MAX_ASSET_BYTES` (default 8 MiB, tunable) — an oversized asset returns a clear error (Content-Length pre-check + buffered post-check) rather than a truncated body; raise the cap or hit `serve` directly for larger files. + +### Security + +- **`download_asset` refuses non-Matrix `src_url` schemes.** The Beeper `serve` endpoint also accepts `file://`, so an unguarded proxy would let any MCP caller read arbitrary local files in the container (env-bearing paths, the sent-ledger — i.e. `BEEPER_TOKEN` / `MCP_AUTH_TOKEN` exfiltration) and reach internal hosts via `http(s)://` (SSRF). `download_asset` now allowlists `mxc://` / `localmxc://` and rejects everything else **before** the fetch — covering both the caller-supplied `src_url` and a `src_url` resolved off a (potentially hostile-sender-crafted) message attachment. A per-call timeout (`BEEPERBOX_ASSET_TIMEOUT_MS`, default 30 s) bounds a hung or slow-drip source so it can't stall the request or grow the buffer unbounded. + +### Notes + +- The `serve`-proxy I/O path needs a live Beeper account to exercise end-to-end (CI has none); it is proven against a stub Beeper API (exact-byte base64 round-trip, both reference paths, cap rejection). The pure `normalizeAttachments` mapping is unit-tested. Validate against a real account before relying on it for production media flows. + ## [0.6.0] — 2026-06-15 `[MINOR]` First release driven end-to-end by a real consumer ([multis](https://github.com/hamr0/multis), beeperbox's first customer): a watch primitive and a reliable echo-guard so an agent can react to incoming messages without reinventing the seed/poll/dedup loop or echo-looping on its own sends, plus two container-lifecycle robustness fixes. MINOR per the versioning policy — new MCP tool + additive schema fields + new runtime behavior, all backward compatible. **Live-validation note:** the `source` echo-guard and the `pendingMessageID` → final-bridge-id resolution were validated against a real Beeper account by multis (CI has none); the standard release gate (guard + VNC probes) still runs the server standalone. diff --git a/README.md b/README.md index a4cd498..cbecc18 100644 --- a/README.md +++ b/README.md @@ -64,7 +64,7 @@ Done. |---|---|---| | `6080` | noVNC web UI — first-run login only | `127.0.0.1` | | `23373` | Raw Beeper Desktop HTTP API | `127.0.0.1` | -| `23375` | Opinionated 11-tool MCP server | `127.0.0.1` | +| `23375` | Opinionated 12-tool MCP server | `127.0.0.1` | All three are env-overridable (`BEEPERBOX_NOVNC_PORT`, `BEEPERBOX_HOST_PORT`, `BEEPERBOX_MCP_PORT`) so you can run multiple instances on one VPS. For remote access use SSH tunnel, Tailscale, or a TLS reverse proxy — never drop the `127.0.0.1` prefix. diff --git a/beeperbox.context.md b/beeperbox.context.md index 2538cfe..9a215df 100644 --- a/beeperbox.context.md +++ b/beeperbox.context.md @@ -10,7 +10,7 @@ beeperbox is a headless [Beeper Desktop](https://www.beeper.com/) in a Docker container that exposes **two things to agents**: 1. **Raw Beeper Desktop HTTP API** on `127.0.0.1:23373` — the unmodified `/v1/*` endpoints for callers who want full control. -2. **Opinionated Model Context Protocol server** on `127.0.0.1:23375` (HTTP) or stdio — 11 semantic tools, normalized `Chat` / `Message` schemas, note-to-self isolation, clean network slugs. Consume from Claude Code, Cursor, Cline, Continue, bareagent, or any other MCP-speaking runtime. +2. **Opinionated Model Context Protocol server** on `127.0.0.1:23375` (HTTP) or stdio — 12 semantic tools, normalized `Chat` / `Message` schemas, note-to-self isolation, clean network slugs. Consume from Claude Code, Cursor, Cline, Continue, bareagent, or any other MCP-speaking runtime. Agents that consume beeperbox get read/write access to **every bridge the user's Beeper account has connected**: WhatsApp, iMessage, Signal, Discord, Slack, Telegram, Facebook Messenger, Instagram, LinkedIn, Google Messages, Matrix, and any future Beeper bridge. One config, every messenger. @@ -230,6 +230,14 @@ Archive or unarchive a chat. Archived chats are removed from `list_inbox` but hi **Returns:** `{chat_id, archived}`. **Note:** Beeper does not expose a `mark_as_read` endpoint. `archive_chat` is the closest primitive for the "I am done with this conversation" pattern. Pass `archived: false` to unarchive. +### `download_asset` + +Download a message attachment's bytes and return them base64-encoded — the MCP-only way to reach a media/PDF/document file, without touching the raw Beeper API on `:23373`. Reference the attachment by its `src_url` (off a `Message.attachments[]` entry), or by `chat_id` + `message_id` (+ `index`) and beeperbox resolves the `src_url` and returns the file's `file_name` / `mime_type` / `size`. + +**Arguments:** `{src_url?: string}` **or** `{chat_id: string, message_id: string, index?: number = 0}`. +**Returns:** `{src_url, content_type, bytes, encoding: "base64", data_base64, file_name?, mime_type?, size?}`. +**Note:** The bytes ride base64 inside the JSON-RPC result, so the asset is capped at `BEEPERBOX_MAX_ASSET_BYTES` (default 8 MiB) — an oversized file returns a clear error, not a truncated body. Internally proxies `GET /v1/assets/serve?url=…`, so a remote deployment publishing only `:23375` can still read attachments. Raise the cap or hit `serve` directly for larger files. + ## Schemas Two normalized shapes. Learn them once and every tool returns the same thing. @@ -275,6 +283,7 @@ Two normalized shapes. Learn them once and every tool returns the same thing. }, "text": "are we still meeting tomorrow?", "type": "TEXT", + "attachments": [], "timestamp": "2026-04-13T09:30:00Z", "reply_to": null, "source": "external", @@ -290,6 +299,7 @@ Two normalized shapes. Learn them once and every tool returns the same thing. | `sender.is_self` | True if the **Beeper account** sent this message — **from any client**, including the human owner typing on their phone. Distinguishes "from my account" from "from someone else". **Not an echo-guard** (see `source`). | | `text` | Message body. For non-text messages (media, voice, etc.) this is `"[MEDIA]"` or `"[]"`. | | `type` | `"TEXT"`, `"MEDIA"`, etc. | +| `attachments` | Array of the message's files, `[]` when none. Each: `{type, file_name, mime_type, src_url, size, is_voice_note}`. `src_url` (an `mxc://` / `localmxc://` URL) is the download reference — pass it to `download_asset` to pull the bytes. For a `MEDIA` message this is the only way to reach the file (`text` is just `"[MEDIA]"`). | | `reply_to` | Parent message ID if this is a reply, else `null`. | | `source` | **`"api"`** if *this* beeperbox sent the message via `send_message` / `note_to_self`; **`"external"`** otherwise. This — not `is_self` — is the echo-guard: on one account both the owner's own typed messages and the agent's API replies are `is_self: true`, so only `source` can tell "I sent this programmatically" from "the human owner sent this". Skip `source === "api"` in a poll loop; process `"external"` (the owner's own Note-to-self commands included). | | `client_tag` | The `client_tag` passed to `send_message` / `note_to_self` for this message, echoed back, else `null`. An idempotency key the agent can correlate to a specific send. | diff --git a/docs/GUIDE.md b/docs/GUIDE.md index 452c759..69eb598 100644 --- a/docs/GUIDE.md +++ b/docs/GUIDE.md @@ -198,7 +198,7 @@ If you see an error, jump to [Troubleshooting](#troubleshooting). ### What you can do next -- **Point an AI agent runtime at it**: Claude Code, Cursor, Cline, bareagent — any MCP client that supports stdio or HTTP transport can consume beeperbox as a tool source. Configure it once and the LLM sees all 11 tools (`list_inbox`, `list_unread`, `poll_messages`, `read_chat`, `get_chat`, `search_messages`, `list_accounts`, `send_message`, `note_to_self`, `react_to_message`, `archive_chat`). See the [MCP tools reference](#mcp-tools-reference) section below. +- **Point an AI agent runtime at it**: Claude Code, Cursor, Cline, bareagent — any MCP client that supports stdio or HTTP transport can consume beeperbox as a tool source. Configure it once and the LLM sees all 12 tools (`list_inbox`, `list_unread`, `poll_messages`, `read_chat`, `get_chat`, `search_messages`, `list_accounts`, `send_message`, `note_to_self`, `react_to_message`, `archive_chat`, `download_asset`). See the [MCP tools reference](#mcp-tools-reference) section below. - **Build something custom**: hit the raw Beeper API on `http://localhost:23373/v1/*` from any language with an HTTP client and your `BEEPER_TOKEN`. See [Use it — real examples](#use-it--real-examples) for curl / Node / Python snippets. - **Deploy to a VPS**: same steps work on any Linux box with Docker. SSH-tunnel noVNC for the one-time login. See [Deploy to a VPS](#deploy-to-a-vps). @@ -496,7 +496,7 @@ If you call `/v1/*` directly instead of going through the MCP tools, you inherit beeperbox exposes 11 semantic tools over Model Context Protocol on two interchangeable transports. Any AI agent runtime that speaks MCP (Claude Code, Cursor, Cline, Continue, bareagent, etc.) can consume them. -### The 11 tools +### The 12 tools | Tool | Required | Returns | Use case | |---|---|---|---| @@ -511,11 +511,16 @@ beeperbox exposes 11 semantic tools over Model Context Protocol on two interchan | `note_to_self` | `text` | same | Agent self-notes, debug output, scheduled reminders — auto-resolves to the Beeper-native Note to self chat (won't leak into per-platform saved-messages chats) | | `react_to_message` | `chat_id`, `message_id`, `emoji` | `{...status: reacted}` | Lightweight ack, no full reply needed | | `archive_chat` | `chat_id` | `{chat_id, archived}` | Clean handled chats out of inbox (closest primitive to mark-as-read that Beeper exposes) | +| `download_asset` | `src_url` *or* `chat_id`+`message_id` | `{src_url, content_type, bytes, encoding, data_base64, ...}` | Fetch an attachment's actual bytes (base64) — reach a media/PDF/doc file an LLM can index. Capped at `BEEPERBOX_MAX_ASSET_BYTES` (default 8 MiB) | #### `poll_messages` — the watch loop A passive new-messages-since-cursor feed; the right tool for an agent that wants to *react* to incoming messages instead of repeatedly diffing `list_inbox` by hand. **Read-only** — it never marks read, archives, or mutates. First call (omit `cursor`) **seeds** from now and returns `{cursor, messages: [], seeded: true}`; pass the cursor back each subsequent call to get only newer messages plus a fresh cursor. The cursor is opaque and **restart-resumable** — persist it and resume across restarts with no missed or duplicated messages (same-millisecond messages are deduplicated by id). `has_more: true` means more is immediately fetchable — keep polling until it's `false`, then sleep. (Residual: beeperbox reads up to the newest 100 messages per chat per poll, so a chat receiving more than 100 messages between two polls can lose the oldest of that burst — Beeper's API has no backward paging; poll frequently enough, or backfill with `read_chat`.) Includes the account's own messages on purpose; branch on each message's **`source`** field (`"api"` = sent by this beeperbox, skip it; `"external"` = everything else including the owner's own Note-to-self commands) rather than `sender.is_self`, which can't tell the agent's API sends from the human's own typing. The poll interval is yours — beeperbox provides the *ability* to watch, not a *policy* about cadence. +#### `download_asset` — reach an attachment's bytes (MCP-only) + +A media message (`type: "MEDIA"`) carries no usable `text` — its file lives behind the `attachments[]` array each normalized `Message` now exposes (`{type, file_name, mime_type, src_url, size, is_voice_note}`). `download_asset` is how an MCP-only agent pulls that file's actual bytes without touching the raw Beeper Desktop API on `:23373`: reference the attachment **either** by its `src_url` (the value straight off the attachment — an `mxc://` / `localmxc://` URL; `file://` and other non-Matrix schemes are refused, so the tool can only reach real attachments, never the container's local filesystem) **or** by `chat_id` + `message_id` (+ optional `index` to pick one of several), in which case beeperbox resolves the `src_url` and also returns the file's `file_name` / `mime_type` / `size`. The bytes come back **base64** in `data_base64` (with `content_type`, `bytes`, `encoding`). Because the payload is base64 inside the JSON-RPC result, it's capped at `BEEPERBOX_MAX_ASSET_BYTES` (default 8 MiB) — an oversized asset returns a clear error rather than a truncated file; raise the cap, or fetch `GET /v1/assets/serve?url=…` directly, for larger files. Internally `download_asset` proxies that same `serve` endpoint, so a remote/MCP-only deployment that only publishes `:23375` can still read attachments. + ### Schemas the LLM learns once and reuses everywhere ``` @@ -537,6 +542,7 @@ Message: sender { id, name, is_self } # is_self = sent by my account from ANY client text message body (or "[MEDIA]" / "[non-text]" for non-text types) type "TEXT" | "MEDIA" | ... + attachments array of { type, file_name, mime_type, src_url, size, is_voice_note }; [] when none timestamp ISO 8601 reply_to parent message id if this is a reply, else null source "api" if THIS beeperbox sent it (send_message/note_to_self), else "external" diff --git a/docs/PRD.md b/docs/PRD.md index 35e7979..a9dd0ca 100644 --- a/docs/PRD.md +++ b/docs/PRD.md @@ -29,7 +29,7 @@ A single Docker container that bundles, in one image: 1. **Headless Beeper Desktop** — the real Electron app, run under a virtual display (Xvfb + openbox), no GPU, no human. 2. **A first-run login surface** — a noVNC web UI (`:6080`) used exactly once, to sign into Beeper and mint an API token. Not part of the steady-state runtime path. 3. **The raw Beeper Desktop HTTP API** (`:23373`), made reachable from outside the container's loopback-only Electron binding via a `socat` forwarder. -4. **An opinionated MCP server** (`:23375`) — vanilla Node, zero npm dependencies, a single `mcp/server.js` — wrapping the raw API in **11 semantic tools** and **2 normalized schemas** so an LLM never touches raw Beeper endpoints. +4. **An opinionated MCP server** (`:23375`) — vanilla Node, zero npm dependencies, a single `mcp/server.js` — wrapping the raw API in **12 semantic tools** and **2 normalized schemas** so an LLM never touches raw Beeper endpoints. State (login, bridge sync, token) lives in a named Docker volume, so the login survives restarts, rebuilds, and host reboots. The image is published pre-built and multi-arch; the common path is *pull and run*, not clone-and-build. @@ -88,7 +88,7 @@ Xvfb (virtual display) |---|---|---|---| | `6080` | noVNC web UI — **first-run login only** | `127.0.0.1` | `BEEPERBOX_NOVNC_PORT` | | `23373` | Raw Beeper Desktop HTTP API (via socat `:23380`) | `127.0.0.1` | `BEEPERBOX_HOST_PORT` | -| `23375` | Opinionated 11-tool MCP server (HTTP transport) | `127.0.0.1` | `BEEPERBOX_MCP_PORT` | +| `23375` | Opinionated 12-tool MCP server (HTTP transport) | `127.0.0.1` | `BEEPERBOX_MCP_PORT` | All three publish to `127.0.0.1` by design. Remote access is a deliberate opt-in (SSH tunnel, Tailscale/Wireguard, or a TLS-terminating reverse proxy with auth) — never by dropping the loopback prefix. The container name is also env-overridable (`BEEPERBOX_CONTAINER_NAME`) so multiple instances can coexist on one host. @@ -124,7 +124,7 @@ A single-file, zero-dependency Node server wrapping the raw API. Two interchange - **HTTP** (default, always on): JSON-RPC 2.0 over POST on `:23375`. For remote agents, cross-container setups, cloud runtimes. - **stdio** (on demand): newline-delimited JSON-RPC over stdin/stdout (stdout reserved for protocol, logs to stderr), via `docker exec -i beeperbox node /opt/mcp/server.js --stdio` (here `beeperbox` is the default container name — use your `BEEPERBOX_CONTAINER_NAME` if you overrode it for a multi-instance host). For local MCP clients (Claude Code, Cursor, Cline, Continue, bareagent). -**The 11 tools:** +**The 12 tools:** | Tool | What it does | Access | |---|---|---| @@ -139,12 +139,13 @@ A single-file, zero-dependency Node server wrapping the raw API. Two interchange | `note_to_self` | Send to the agent's own note-to-self chat (auto-resolved, Beeper-native matrix), the dedicated command/control channel — excluded from inbox views | write | | `react_to_message` | Add an emoji reaction (unicode, shortcode, or custom key) | write | | `archive_chat` | Archive/unarchive (stands in for mark-as-read, which Beeper exposes no endpoint for) | write | +| `download_asset` | Fetch a message attachment's bytes (base64) by `src_url` or `chat_id`+`message_id`; capped at `BEEPERBOX_MAX_ASSET_BYTES` | read | **Two normalized schemas** the LLM learns once and reuses everywhere: ``` Chat: { id, title, network, network_label, is_group, is_note_to_self, last_message_at, unread_count } -Message: { id, chat_id, network, network_label, sender{id,name,is_self}, text, type, timestamp, reply_to, source, client_tag } +Message: { id, chat_id, network, network_label, sender{id,name,is_self}, text, type, attachments[], timestamp, reply_to, source, client_tag } ``` Every chat and message carries both `network` (machine slug: `whatsapp`, `telegram`, …) and `network_label` (human: `"WhatsApp"`, …), normalized off `/v1/accounts` with chat-bridge-ID parsing as fallback. The **note-to-self vs inbox split** is a core design invariant: the agent's command channel must never pollute customer conversations, and customer inbox views must never surface the agent's own notes. @@ -241,6 +242,7 @@ beeperbox is a single-tenant container that holds a credential (`BEEPER_TOKEN`) | 0.5.0 | 2026-05-24 | Security hardening: MCP auth + Host/Origin + body cap, VNC password, `no-new-privileges`, opt-in pinned builds. | | 0.5.1 | 2026-05-25 | CI-gated releases + `:previous` rollback; shared guard scripts as single source of truth. | | 0.6.0 | 2026-06-15 | `poll_messages` watch primitive + exact-id echo-guard (`source`/`client_tag`); supervised backend + restart-survivable display. | +| Unreleased | — | Attachment reach: `attachments[]` on every `Message` + `download_asset` tool (bytes as base64 via `/v1/assets/serve`, byte-capped). | --- @@ -248,7 +250,8 @@ beeperbox is a single-tenant container that holds a credential (`BEEPER_TOKEN`) Nothing is committed; beeperbox is feature-driven by real usage. Delivered from this list once a real user asked: -- **`poll_messages` watch primitive + `source` echo-guard** *(unreleased — see CHANGELOG)* — the first consumer-driven feature. [multis](https://github.com/hamr0/multis) was hand-rolling seed/poll/dedup against the raw API; beeperbox now exposes the cursor-based new-messages primitive and a send-origin marker natively. Boundary-correct: it adds the *ability* to watch, not a *policy* about when to poll, and it does not make beeperbox a streaming system (§3) — it's a better-shaped poll. +- **Attachment reach — `attachments[]` + `download_asset`** *(unreleased — see CHANGELOG)* — multis needed to index files customers and the admin send (e.g. a PDF → FAQ). A normalized `Message` now surfaces each attachment's `src_url`, and `download_asset` returns the bytes (base64) through beeperbox's own `:23375` — so an MCP-only / remote deployment that doesn't publish the raw Beeper API on `:23373` can still read files. Boundary-correct: it proxies `GET /v1/assets/serve`, byte-capped to keep the JSON-RPC result bounded; it does not turn beeperbox into a file store. +- **`poll_messages` watch primitive + `source` echo-guard** *(shipped 0.6.0)* — the first consumer-driven feature. [multis](https://github.com/hamr0/multis) was hand-rolling seed/poll/dedup against the raw API; beeperbox now exposes the cursor-based new-messages primitive and a send-origin marker natively. Boundary-correct: it adds the *ability* to watch, not a *policy* about when to poll, and it does not make beeperbox a streaming system (§3) — it's a better-shaped poll. Candidates discussed, not built: diff --git a/mcp/server.js b/mcp/server.js index 9f4187d..cfa0133 100644 --- a/mcp/server.js +++ b/mcp/server.js @@ -58,6 +58,29 @@ const RESOLVE_DELAY_MS = envIntNonNeg('BEEPERBOX_RESOLVE_DELAY_MS', 250); // worst-case added send latency is RETRIES × (TIMEOUT + DELAY), not infinite. const RESOLVE_TIMEOUT_MS = envIntNonNeg('BEEPERBOX_RESOLVE_TIMEOUT_MS', 3000); +// download_asset ships the bytes base64-encoded inside the JSON-RPC result, so +// a large attachment would bloat the response (base64 inflates ~33%) and the +// in-memory buffer. Cap it; tunable for a deployment that needs bigger files. +// FAQ/doc attachments — the driving use case — are well under this. +const MAX_ASSET_BYTES = envIntNonNeg('BEEPERBOX_MAX_ASSET_BYTES', 8 * 1024 * 1024); +// Per-call timeout on the asset-serve fetch so a hung or slow-drip source can't +// stall the request forever and the buffer can't grow unbounded behind it. +const ASSET_TIMEOUT_MS = envIntNonNeg('BEEPERBOX_ASSET_TIMEOUT_MS', 30000); + +// Attachment src_urls are always Matrix content URLs (mxc:// / localmxc://). +// The Beeper /v1/assets/serve endpoint ALSO accepts file:// — which, proxied +// through download_asset, would let any MCP caller read arbitrary local files +// in the container (env-bearing paths, the sent-ledger, anything the Beeper +// process can read) and exfiltrate secrets. We refuse anything but mxc:// +// /localmxc:// so the tool can only reach real message attachments, never the +// local filesystem. Applied to BOTH the caller-supplied src_url and a src_url +// resolved off a message (a hostile sender could craft a file:// attachment). +function assertServableSrcUrl(srcURL) { + if (!/^(?:mxc|localmxc):\/\//i.test(String(srcURL))) { + throw rpcError(-32602, 'src_url must be an mxc:// or localmxc:// URL — file:// and other schemes are refused'); + } +} + // Strip the port from a Host header value ("127.0.0.1:23375" -> "127.0.0.1", // "[::1]:23375" -> "[::1]"). function hostFromHeader(value) { @@ -115,6 +138,24 @@ async function beeperFetch(path, opts = {}) { try { const r = await fetch(`${BEEPER_API}${path}`, init); if (!r.ok) throw rpcError(-32001, `beeper api ${r.status}: ${(await r.text()).slice(0, 200)}`); + // Binary mode (opts.raw): return the bytes + content-type instead of + // parsing JSON — used by download_asset to serve attachment bytes. Enforce + // the byte cap twice: the Content-Length header (reject before buffering a + // well-behaved large response) and the actual buffered length (in case the + // header is absent or lies). Decoding these bytes as text would corrupt + // them, so this path never touches r.text()/JSON.parse. + if (opts.raw) { + const max = opts.maxBytes || 0; + const declared = parseInt(r.headers.get('content-length') || '', 10); + if (max && Number.isFinite(declared) && declared > max) { + throw rpcError(-32005, `asset is ${declared} bytes, over the ${max}-byte cap — raise BEEPERBOX_MAX_ASSET_BYTES or fetch /v1/assets/serve directly`); + } + const buf = Buffer.from(await r.arrayBuffer()); + if (max && buf.length > max) { + throw rpcError(-32005, `asset is ${buf.length} bytes, over the ${max}-byte cap — raise BEEPERBOX_MAX_ASSET_BYTES or fetch /v1/assets/serve directly`); + } + return { bytes: buf, content_type: r.headers.get('content-type') || '' }; + } // Some POST/DELETE endpoints return empty body — return null instead of throwing on r.json() const text = await r.text(); if (!text) return null; @@ -231,6 +272,24 @@ async function getAccountMap() { // type → 'TEXT' | 'MEDIA' | etc. // replyTo → optional, parent message id +// Map Beeper's raw message `attachments[]` into a normalized shape MCP clients +// can act on. Pure passthrough of the fields that already ride the raw message +// — no extra fetch. `src_url` is the download reference (an mxc:// / localmxc:// +// Matrix content URL) that download_asset takes. `size` carries the byte length +// (raw `fileSize`) when present. Returns [] when +// there are no attachments, so the field is always an array. +function normalizeAttachments(raw) { + const list = Array.isArray(raw?.attachments) ? raw.attachments : []; + return list.map((a) => ({ + type: a.type || null, + file_name: a.fileName || null, + mime_type: a.mimeType || null, + src_url: a.srcURL || null, + size: typeof a.fileSize === 'number' ? a.fileSize : null, + is_voice_note: !!a.isVoiceNote, + })); +} + function normalizeMessage(raw, chat) { return { id: String(raw.id), @@ -244,6 +303,9 @@ function normalizeMessage(raw, chat) { }, text: raw.text || (raw.type === 'TEXT' ? '' : `[${raw.type || 'non-text'}]`), type: raw.type || 'TEXT', + // Media reach: a MEDIA message carries no usable `text`, so without this an + // agent could see "[MEDIA]" but never reach the file. Always an array. + attachments: normalizeAttachments(raw), timestamp: raw.timestamp || null, reply_to: raw.replyTo || raw.reply_to || null, // Echo-guard origin. Defaults assume "not sent through this beeperbox"; @@ -667,6 +729,20 @@ const TOOLS = [ additionalProperties: false, }, }, + { + name: 'download_asset', + description: 'Download the bytes of a message attachment (image, PDF, document, voice note, etc.) and return them base64-encoded. This is the MCP-only way to reach an attachment\'s actual content — use it after read_chat / poll_messages surfaces a message whose `attachments[]` carries a `src_url`. Reference the attachment EITHER by passing its `src_url` directly (the value from a normalized attachment — an mxc:// or localmxc:// URL; other schemes such as file:// are refused), OR by `chat_id` + `message_id` (+ optional `index` to pick one of several attachments), in which case beeperbox resolves the src_url for you and also returns the attachment\'s file_name / mime_type / size. Returns { content_type, bytes, encoding: "base64", data_base64, ... }. Large files are capped (default 8 MiB) to keep the JSON-RPC result bounded — raise BEEPERBOX_MAX_ASSET_BYTES if you need bigger.', + inputSchema: { + type: 'object', + properties: { + src_url: { type: 'string', description: 'The attachment\'s `src_url` (an mxc:// or localmxc:// URL) from a normalized Message attachment. file:// and other non-Matrix schemes are refused. Provide this, OR chat_id + message_id.' }, + chat_id: { type: 'string', description: 'The chat containing the message. Required if src_url is omitted (used with message_id to resolve the attachment).' }, + message_id: { type: 'string', description: 'The message whose attachment to download. Required if src_url is omitted.' }, + index: { type: 'integer', description: 'Which attachment to download when resolving by message_id and the message has more than one (default 0).', default: 0, minimum: 0 }, + }, + additionalProperties: false, + }, + }, ]; async function callTool(name, args) { @@ -891,6 +967,49 @@ async function callTool(name, args) { const tagged = applyEchoTags(delivered, Date.now()); return { cursor: encodeCursor(next), messages: tagged, has_more: hasMore }; } + + case 'download_asset': { + // Two ways to reference the attachment: a src_url directly (cheapest — + // multis already has it from the normalized message), or chat_id + + // message_id (+ index), where we fetch the message and read the src_url + // off attachments[index] — which also lets us return its file metadata. + let srcURL = args.src_url ? String(args.src_url) : ''; + let meta = {}; + if (!srcURL) { + if (!args.chat_id || !args.message_id) { + throw rpcError(-32602, 'download_asset requires src_url, or both chat_id and message_id'); + } + const raw = await beeperFetch( + `/v1/chats/${encodeURIComponent(args.chat_id)}/messages/${encodeURIComponent(args.message_id)}`, + ); + const atts = normalizeAttachments(raw); + const idx = Math.max(parseInt(args.index, 10) || 0, 0); + const att = atts[idx]; + if (!att) throw rpcError(-32602, `message has no attachment at index ${idx} (found ${atts.length})`); + if (!att.src_url) throw rpcError(-32004, `attachment ${idx} has no src_url to download`); + srcURL = att.src_url; + meta = { file_name: att.file_name, mime_type: att.mime_type, size: att.size }; + } + // Refuse file:// (and any non-Matrix scheme) BEFORE the fetch — this is + // the local-file-read / secret-exfil guard, covering both ref paths. + assertServableSrcUrl(srcURL); + // /v1/assets/serve streams the bytes for an mxc:// / localmxc:// URL. + // beeperFetch raw-mode returns the buffer + content-type, enforces the + // byte cap (header pre-check + buffered post-check), and the timeout + // bounds a hung/slow-drip source. + const { bytes, content_type } = await beeperFetch( + `/v1/assets/serve?url=${encodeURIComponent(srcURL)}`, + { raw: true, maxBytes: MAX_ASSET_BYTES, timeoutMs: ASSET_TIMEOUT_MS }, + ); + return { + src_url: srcURL, + ...meta, + content_type: content_type || meta.mime_type || null, + bytes: bytes.length, + encoding: 'base64', + data_base64: bytes.toString('base64'), + }; + } default: throw rpcError(-32601, `unknown tool: ${name}`); } @@ -1079,6 +1198,8 @@ module.exports = { advanceCursor, selectDelivery, textHash, + normalizeAttachments, + assertServableSrcUrl, matchSentMessage, recordSent, addResolvedId, diff --git a/mcp/server.test.js b/mcp/server.test.js index 38e1aae..43d4164 100644 --- a/mcp/server.test.js +++ b/mcp/server.test.js @@ -152,6 +152,86 @@ test('selectDelivery: a single under-page batch delivers all at once with hasMor assert.equal(r.hasMore, false); }); +// ── normalizeAttachments (pure passthrough of raw attachments[]) ── +test('normalizeAttachments: maps a raw attachment to the normalized shape', () => { + const raw = { + attachments: [{ + type: 'file', + fileName: 'invoice.pdf', + mimeType: 'application/pdf', + srcURL: 'mxc://beeper.com/abc123', + fileSize: 20480, + isVoiceNote: false, + }], + }; + assert.deepEqual(S.normalizeAttachments(raw), [{ + type: 'file', + file_name: 'invoice.pdf', + mime_type: 'application/pdf', + src_url: 'mxc://beeper.com/abc123', + size: 20480, + is_voice_note: false, + }]); +}); + +test('normalizeAttachments: a message with no attachments yields [] (always an array)', () => { + assert.deepEqual(S.normalizeAttachments({ text: 'hi', type: 'TEXT' }), []); + assert.deepEqual(S.normalizeAttachments({}), []); + assert.deepEqual(S.normalizeAttachments(null), []); + // Defensive: a non-array attachments field must not throw or pass through. + assert.deepEqual(S.normalizeAttachments({ attachments: 'oops' }), []); +}); + +test('normalizeAttachments: missing optional fields become null / false, not undefined', () => { + // A media attachment that only carries a src_url — file_name/mime_type/size + // absent. They must normalize to null (not undefined) so the JSON shape is + // stable, and is_voice_note to a real boolean. + const [att] = S.normalizeAttachments({ attachments: [{ srcURL: 'mxc://h/x' }] }); + assert.deepEqual(att, { + type: null, file_name: null, mime_type: null, + src_url: 'mxc://h/x', size: null, is_voice_note: false, + }); +}); + +test('normalizeAttachments: a non-numeric fileSize does not leak through as size', () => { + // Guards the byte cap / consumers from a stray string sneaking into `size`. + const [att] = S.normalizeAttachments({ attachments: [{ srcURL: 'mxc://h/x', fileSize: '20480' }] }); + assert.equal(att.size, null); +}); + +test('normalizeAttachments: preserves order and maps every entry of a multi-attachment message', () => { + const out = S.normalizeAttachments({ attachments: [ + { srcURL: 'mxc://h/1', fileName: 'a.png', type: 'image' }, + { srcURL: 'mxc://h/2', fileName: 'b.pdf', type: 'file' }, + ] }); + assert.equal(out.length, 2); + assert.deepEqual(out.map((a) => a.file_name), ['a.png', 'b.pdf']); + assert.deepEqual(out.map((a) => a.src_url), ['mxc://h/1', 'mxc://h/2']); +}); + +// ── assertServableSrcUrl (download_asset local-file-read guard) ─── +test('assertServableSrcUrl: allows mxc:// and localmxc:// (the real attachment schemes)', () => { + assert.doesNotThrow(() => S.assertServableSrcUrl('mxc://beeper.com/abc')); + assert.doesNotThrow(() => S.assertServableSrcUrl('localmxc://beeper.com/cached')); + assert.doesNotThrow(() => S.assertServableSrcUrl('MXC://BEEPER.COM/ABC')); // scheme is case-insensitive +}); + +test('assertServableSrcUrl: refuses file:// — the arbitrary-local-file-read vector', () => { + // The load-bearing security assertion: a file:// src_url would let an MCP + // caller read /etc/passwd, the sent-ledger, env-bearing paths, etc. via the + // Beeper serve endpoint. Must throw a -32602 BEFORE any fetch. + assert.throws(() => S.assertServableSrcUrl('file:///etc/passwd'), /schemes are refused/); +}); + +test('assertServableSrcUrl: refuses http(s):// (SSRF) and other non-Matrix schemes', () => { + assert.throws(() => S.assertServableSrcUrl('http://169.254.169.254/latest/meta-data/'), /schemes are refused/); + assert.throws(() => S.assertServableSrcUrl('https://evil.example/x'), /schemes are refused/); + assert.throws(() => S.assertServableSrcUrl('ftp://h/x'), /schemes are refused/); + assert.throws(() => S.assertServableSrcUrl(''), /schemes are refused/); + // Defeat a "mxc as a path, not the scheme" smuggle attempt. + assert.throws(() => S.assertServableSrcUrl('file:///x?mxc://'), /schemes are refused/); +}); + // ── echo-guard matcher (pure) ───────────────────────────────────── const NOW = 1_750_000_000_000; test('matchSentMessage: exact id match is source:api and echoes client_tag', () => { diff --git a/scripts/asset-serve-check.js b/scripts/asset-serve-check.js new file mode 100644 index 0000000..afd35f1 --- /dev/null +++ b/scripts/asset-serve-check.js @@ -0,0 +1,133 @@ +// End-to-end harness for download_asset + attachments, run against a STUB +// Beeper Desktop API (CI has no live account). Aimed at the TOUGH turns, not the +// happy path: binary integrity, BOTH cap paths (Content-Length pre-check AND the +// no-Content-Length buffered post-check), and the error branches. Honest about +// what a stub can and can't prove — the stub encodes our ASSUMPTION of how +// /v1/assets/serve behaves; this verifies our code against that assumption, not +// against real Beeper. Run: node scripts/asset-serve-check.js (exit 0 = pass) +const http = require('http'); +const crypto = require('crypto'); +const { spawn } = require('child_process'); +const path = require('path'); + +const SERVER = path.join(__dirname, '..', 'mcp', 'server.js'); +const SMALL = crypto.randomBytes(1234); // well under cap; random ⇒ text-decoding WOULD corrupt it +const BIG = crypto.randomBytes(4096); // over the 2000-byte cap we set below + +// ── stub Beeper Desktop API ─────────────────────────────────────── +const beeper = http.createServer((req, res) => { + const u = new URL(req.url, 'http://x'); + + if (u.pathname === '/v1/assets/serve') { + const url = u.searchParams.get('url'); + if (url === 'mxc://h/big-known-length') { + // Well-behaved: correct Content-Length ⇒ exercises the PRE-check. + res.writeHead(200, { 'Content-Type': 'application/octet-stream', 'Content-Length': String(BIG.length) }); + res.end(BIG); + } else if (url === 'mxc://h/big-no-length') { + // Chunked, NO Content-Length ⇒ pre-check can't fire; exercises the + // buffered POST-check. This is the path my first POC never ran. + res.writeHead(200, { 'Content-Type': 'application/octet-stream' }); // chunked + res.write(BIG.subarray(0, 2048)); + res.end(BIG.subarray(2048)); + } else { + res.writeHead(200, { 'Content-Type': 'image/png', 'Content-Length': String(SMALL.length) }); + res.end(SMALL); + } + return; + } + + if (u.pathname.includes('/messages/')) { + const id = u.pathname.split('/').pop(); + if (id === 'm-multi') { + res.writeHead(200, { 'Content-Type': 'application/json' }); + res.end(JSON.stringify({ id, chatID: 'c1', type: 'MEDIA', attachments: [ + { type: 'image', fileName: 'a.png', mimeType: 'image/png', srcURL: 'mxc://h/a', fileSize: SMALL.length }, + { type: 'file', fileName: 'b.pdf', mimeType: 'application/pdf', srcURL: 'mxc://h/abc', fileSize: SMALL.length }, + ] })); + } else if (id === 'm-nosrc') { + res.writeHead(200, { 'Content-Type': 'application/json' }); + res.end(JSON.stringify({ id, chatID: 'c1', type: 'MEDIA', attachments: [{ type: 'file', fileName: 'broken.bin' }] })); + } else { + res.writeHead(200, { 'Content-Type': 'application/json' }); + res.end(JSON.stringify({ id, chatID: 'c1', type: 'MEDIA', attachments: [ + { type: 'file', fileName: 'invoice.pdf', mimeType: 'application/pdf', srcURL: 'mxc://h/abc', fileSize: SMALL.length }, + ] })); + } + return; + } + res.writeHead(404).end(); +}); + +function rpc(port, body) { + return new Promise((resolve, reject) => { + const data = JSON.stringify(body); + const r = http.request({ host: '127.0.0.1', port, method: 'POST', path: '/', headers: { 'Content-Type': 'application/json', 'Content-Length': Buffer.byteLength(data) } }, (res) => { + let b = ''; res.on('data', (c) => b += c); res.on('end', () => resolve(JSON.parse(b))); + }); + r.on('error', reject); r.end(data); + }); +} +// Defensive: a tool error (no .result) must surface as a clean assertion +// failure, never crash the harness — otherwise a regression that errors looks +// like a harness bug instead of the FAIL it is. +const parse = (r) => (r && r.result ? JSON.parse(r.result.content[0].text) : { __error: r && r.error && r.error.message }); +const wait = (ms) => new Promise((r) => setTimeout(r, ms)); + +beeper.listen(0, '127.0.0.1', () => { + const beeperPort = beeper.address().port; + const mcpPort = 28997; + const srv = spawn('node', [SERVER], { + env: { ...process.env, MCP_PORT: String(mcpPort), BEEPER_API: `http://127.0.0.1:${beeperPort}`, BEEPER_TOKEN: 'x', BEEPERBOX_MAX_ASSET_BYTES: '2000' }, + stdio: ['ignore', 'ignore', 'inherit'], + }); + (async () => { + await wait(700); + let failures = 0; + const check = (cond, msg) => { console.log(`${cond ? 'PASS' : 'FAIL'}: ${msg}`); if (!cond) failures++; }; + + // ── binary integrity (the load-bearing claim) ── + const d1 = parse(await rpc(mcpPort, { jsonrpc: '2.0', id: 1, method: 'tools/call', params: { name: 'download_asset', arguments: { src_url: 'mxc://h/abc' } } })); + check(!d1.__error && Buffer.from(d1.data_base64 || '', 'base64').equals(SMALL), `base64 decodes to the EXACT original bytes (no text corruption)${d1.__error ? ' — got error: ' + d1.__error : d1.bytes !== SMALL.length ? ' — got ' + d1.bytes + ' bytes' : ''}`); + check(d1.content_type === 'image/png' && d1.bytes === SMALL.length, 'content_type + byte count correct'); + + // ── message-resolution path + index ── + const d2 = parse(await rpc(mcpPort, { jsonrpc: '2.0', id: 2, method: 'tools/call', params: { name: 'download_asset', arguments: { chat_id: 'c1', message_id: 'm-multi', index: 1 } } })); + check(d2.file_name === 'b.pdf' && d2.mime_type === 'application/pdf', 'index:1 selects the SECOND attachment (resolves its src_url + metadata)'); + + // ── TOUGH TURN A: cap via Content-Length pre-check ── + const c1 = await rpc(mcpPort, { jsonrpc: '2.0', id: 3, method: 'tools/call', params: { name: 'download_asset', arguments: { src_url: 'mxc://h/big-known-length' } } }); + check(/over the 2000-byte cap/.test(c1.error?.message || ''), `cap PRE-check (Content-Length present) rejects: ${c1.error?.message || 'NO ERROR'}`); + + // ── TOUGH TURN B: cap via buffered post-check (NO Content-Length) ── + // The path the first POC silently skipped — chunked response, header absent. + const c2 = await rpc(mcpPort, { jsonrpc: '2.0', id: 4, method: 'tools/call', params: { name: 'download_asset', arguments: { src_url: 'mxc://h/big-no-length' } } }); + check(/over the 2000-byte cap/.test(c2.error?.message || ''), `cap POST-check (no Content-Length, chunked) rejects: ${c2.error?.message || 'NO ERROR'}`); + check(/4096 bytes/.test(c2.error?.message || ''), 'post-check reports the ACTUAL buffered length (4096), proving it measured bytes not the header'); + + // ── error branches ── + const e1 = await rpc(mcpPort, { jsonrpc: '2.0', id: 5, method: 'tools/call', params: { name: 'download_asset', arguments: { chat_id: 'c1', message_id: 'm-multi', index: 9 } } }); + check(/no attachment at index 9 \(found 2\)/.test(e1.error?.message || ''), `out-of-range index errors clearly: ${e1.error?.message || 'NO ERROR'}`); + const e2 = await rpc(mcpPort, { jsonrpc: '2.0', id: 6, method: 'tools/call', params: { name: 'download_asset', arguments: { chat_id: 'c1', message_id: 'm-nosrc' } } }); + check(/has no src_url/.test(e2.error?.message || ''), `attachment without src_url errors clearly: ${e2.error?.message || 'NO ERROR'}`); + const e3 = await rpc(mcpPort, { jsonrpc: '2.0', id: 7, method: 'tools/call', params: { name: 'download_asset', arguments: {} } }); + check(/requires src_url/.test(e3.error?.message || ''), 'missing ref errors clearly'); + + // ── SECURITY: file:// (and non-Matrix schemes) must be refused BEFORE the + // serve fetch — else an MCP caller reads arbitrary local files. The refusal + // message is distinct from any serve-path error, proving the fetch never ran. + const s1 = await rpc(mcpPort, { jsonrpc: '2.0', id: 9, method: 'tools/call', params: { name: 'download_asset', arguments: { src_url: 'file:///etc/passwd' } } }); + check(/file:\/\/ and other schemes are refused/.test(s1.error?.message || ''), `file:// src_url is REFUSED (local-file-read guard): ${s1.error?.message || 'NO ERROR'}`); + const s2 = await rpc(mcpPort, { jsonrpc: '2.0', id: 10, method: 'tools/call', params: { name: 'download_asset', arguments: { src_url: 'http://169.254.169.254/latest/meta-data/' } } }); + check(/schemes are refused/.test(s2.error?.message || ''), 'http:// src_url (SSRF) is refused too'); + const s3 = parse(await rpc(mcpPort, { jsonrpc: '2.0', id: 11, method: 'tools/call', params: { name: 'download_asset', arguments: { src_url: 'localmxc://h/cached' } } })); + check(!s3.__error, 'localmxc:// (legit local-cache attachment scheme) is still allowed'); + + // ── attachments[] surfaced on a normalized read (via the message endpoint shape) ── + const d3 = parse(await rpc(mcpPort, { jsonrpc: '2.0', id: 8, method: 'tools/call', params: { name: 'download_asset', arguments: { chat_id: 'c1', message_id: 'm1' } } })); + check(d3.src_url === 'mxc://h/abc', 'single-attachment message resolves src_url at default index 0'); + + console.log(failures === 0 ? '\nALL CHECKS PASSED' : `\n${failures} CHECK(S) FAILED`); + srv.kill(); beeper.close(); process.exit(failures === 0 ? 0 : 1); + })().catch((e) => { console.error(e); srv.kill(); beeper.close(); process.exit(2); }); +}); diff --git a/scripts/smoke-test.sh b/scripts/smoke-test.sh index 9ac12ba..028e83a 100755 --- a/scripts/smoke-test.sh +++ b/scripts/smoke-test.sh @@ -61,12 +61,12 @@ body = json.loads(sys.argv[1]) names = {t["name"] for t in body.get("result", {}).get("tools", [])} expected = {"list_accounts", "list_inbox", "list_unread", "get_chat", "read_chat", "search_messages", "send_message", "note_to_self", "react_to_message", - "archive_chat", "poll_messages"} + "archive_chat", "poll_messages", "download_asset"} missing = expected - names if missing: print(f"missing tools: {sorted(missing)}", file=sys.stderr) sys.exit(1) PY -pass "MCP tools/list returns all 11 expected tools by name" +pass "MCP tools/list returns all 12 expected tools by name" printf '\n\033[1;32mall checks passed\033[0m — POC is healthy\n\n' From af00b58509c3dd07c52d44d757e1667a54b4563c Mon Sep 17 00:00:00 2001 From: hamr0 Date: Tue, 16 Jun 2026 11:06:30 +0200 Subject: [PATCH 2/4] fix(mcp): confine download_asset file:// to Beeper media cache (live-validated) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Live testing against a real Beeper account revealed two things the stub could not: 1. Real cached attachment src_urls are file:///root/.config/BeeperTexts/ media/... (not mxc://). The previous blanket file:// rejection broke the message-resolution path and every locally-cached attachment. 2. /v1/assets/serve sends no Content-Type for these (content_type is null; callers already have mime_type from the attachment, so non-fatal). The guard now allows mxc:// / localmxc://, allows file:// ONLY when it resolves inside the media cache (BEEPERBOX_ASSET_FILE_ROOT, default /root/.config/BeeperTexts/media/), and refuses everything else. The path is percent-decoded and path.normalize()'d before the prefix check — new URL() does NOT collapse ../, so a literal-prefix check would let media/../../../etc/passwd and %2f-encoded variants escape (verified). Validated live: real PDF downloads via both src_url and chat_id+message_id (exact bytes); /etc/passwd, the sent-ledger, and ../ traversal all blocked. Unit tests +2 (cache-allow + traversal-refuse); asset-serve-check + the stub cover the same. 37 unit tests green. Co-Authored-By: Claude Opus 4.8 --- CHANGELOG.md | 4 +-- beeperbox.context.md | 2 +- docs/GUIDE.md | 2 +- mcp/server.js | 52 ++++++++++++++++++++++++++---------- mcp/server.test.js | 42 ++++++++++++++++++++--------- scripts/asset-serve-check.js | 20 +++++++++----- 6 files changed, 85 insertions(+), 37 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index cbd1012..3257d5a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,11 +21,11 @@ Published tags on GHCR: `:X.Y.Z` (exact, immutable), `:X.Y` (rolling within a mi ### Security -- **`download_asset` refuses non-Matrix `src_url` schemes.** The Beeper `serve` endpoint also accepts `file://`, so an unguarded proxy would let any MCP caller read arbitrary local files in the container (env-bearing paths, the sent-ledger — i.e. `BEEPER_TOKEN` / `MCP_AUTH_TOKEN` exfiltration) and reach internal hosts via `http(s)://` (SSRF). `download_asset` now allowlists `mxc://` / `localmxc://` and rejects everything else **before** the fetch — covering both the caller-supplied `src_url` and a `src_url` resolved off a (potentially hostile-sender-crafted) message attachment. A per-call timeout (`BEEPERBOX_ASSET_TIMEOUT_MS`, default 30 s) bounds a hung or slow-drip source so it can't stall the request or grow the buffer unbounded. +- **`download_asset` confines `src_url` to attachments, not the filesystem.** Real attachment `src_url`s (verified against a live account) are `mxc://` / `localmxc://` for remote media and `file:///root/.config/BeeperTexts/media/…` once Beeper caches the file locally — and the Beeper `serve` endpoint will serve *any* `file://` path. An unguarded proxy would therefore let any MCP caller read arbitrary local files (env-bearing paths, the sent-ledger — i.e. `BEEPER_TOKEN` / `MCP_AUTH_TOKEN` exfiltration) and reach internal hosts via `http(s)://` (SSRF). `download_asset` now allows `mxc://` / `localmxc://`, allows `file://` **only when it resolves inside Beeper's media cache** (`BEEPERBOX_ASSET_FILE_ROOT`, default `/root/.config/BeeperTexts/media/`; the path is percent-decoded and normalized first so `../` and `%2e/%2f` traversal can't escape), and refuses everything else **before** the fetch — covering both the caller-supplied `src_url` and a `src_url` resolved off a (potentially hostile-sender-crafted) message attachment. A per-call timeout (`BEEPERBOX_ASSET_TIMEOUT_MS`, default 30 s) bounds a hung or slow-drip source so it can't stall the request or grow the buffer unbounded. ### Notes -- The `serve`-proxy I/O path needs a live Beeper account to exercise end-to-end (CI has none); it is proven against a stub Beeper API (exact-byte base64 round-trip, both reference paths, cap rejection). The pure `normalizeAttachments` mapping is unit-tested. Validate against a real account before relying on it for production media flows. +- **Validated against a live Beeper account**, not just the stub: a real cached PDF downloads via both reference paths with exact byte length, the raw attachment shape matches the `normalizeAttachments` mapping, and the `file://` cache confinement blocks `/etc/passwd`, the sent-ledger, and `../` traversal while still serving real cached attachments. CI itself has no account, so `scripts/asset-serve-check.js` reproduces these against a stub `serve`; the pure `normalizeAttachments` / `assertServableSrcUrl` logic is unit-tested. ## [0.6.0] — 2026-06-15 `[MINOR]` diff --git a/beeperbox.context.md b/beeperbox.context.md index 9a215df..8146d60 100644 --- a/beeperbox.context.md +++ b/beeperbox.context.md @@ -299,7 +299,7 @@ Two normalized shapes. Learn them once and every tool returns the same thing. | `sender.is_self` | True if the **Beeper account** sent this message — **from any client**, including the human owner typing on their phone. Distinguishes "from my account" from "from someone else". **Not an echo-guard** (see `source`). | | `text` | Message body. For non-text messages (media, voice, etc.) this is `"[MEDIA]"` or `"[]"`. | | `type` | `"TEXT"`, `"MEDIA"`, etc. | -| `attachments` | Array of the message's files, `[]` when none. Each: `{type, file_name, mime_type, src_url, size, is_voice_note}`. `src_url` (an `mxc://` / `localmxc://` URL) is the download reference — pass it to `download_asset` to pull the bytes. For a `MEDIA` message this is the only way to reach the file (`text` is just `"[MEDIA]"`). | +| `attachments` | Array of the message's files, `[]` when none. Each: `{type, file_name, mime_type, src_url, size, is_voice_note}`. `src_url` (an `mxc://` / `localmxc://` URL, or a `file://` URL inside Beeper's media cache once the file is downloaded locally) is the download reference — pass it to `download_asset` to pull the bytes. For a `MEDIA` message this is the only way to reach the file (`text` is just `"[MEDIA]"`). | | `reply_to` | Parent message ID if this is a reply, else `null`. | | `source` | **`"api"`** if *this* beeperbox sent the message via `send_message` / `note_to_self`; **`"external"`** otherwise. This — not `is_self` — is the echo-guard: on one account both the owner's own typed messages and the agent's API replies are `is_self: true`, so only `source` can tell "I sent this programmatically" from "the human owner sent this". Skip `source === "api"` in a poll loop; process `"external"` (the owner's own Note-to-self commands included). | | `client_tag` | The `client_tag` passed to `send_message` / `note_to_self` for this message, echoed back, else `null`. An idempotency key the agent can correlate to a specific send. | diff --git a/docs/GUIDE.md b/docs/GUIDE.md index 69eb598..28f4509 100644 --- a/docs/GUIDE.md +++ b/docs/GUIDE.md @@ -519,7 +519,7 @@ A passive new-messages-since-cursor feed; the right tool for an agent that wants #### `download_asset` — reach an attachment's bytes (MCP-only) -A media message (`type: "MEDIA"`) carries no usable `text` — its file lives behind the `attachments[]` array each normalized `Message` now exposes (`{type, file_name, mime_type, src_url, size, is_voice_note}`). `download_asset` is how an MCP-only agent pulls that file's actual bytes without touching the raw Beeper Desktop API on `:23373`: reference the attachment **either** by its `src_url` (the value straight off the attachment — an `mxc://` / `localmxc://` URL; `file://` and other non-Matrix schemes are refused, so the tool can only reach real attachments, never the container's local filesystem) **or** by `chat_id` + `message_id` (+ optional `index` to pick one of several), in which case beeperbox resolves the `src_url` and also returns the file's `file_name` / `mime_type` / `size`. The bytes come back **base64** in `data_base64` (with `content_type`, `bytes`, `encoding`). Because the payload is base64 inside the JSON-RPC result, it's capped at `BEEPERBOX_MAX_ASSET_BYTES` (default 8 MiB) — an oversized asset returns a clear error rather than a truncated file; raise the cap, or fetch `GET /v1/assets/serve?url=…` directly, for larger files. Internally `download_asset` proxies that same `serve` endpoint, so a remote/MCP-only deployment that only publishes `:23375` can still read attachments. +A media message (`type: "MEDIA"`) carries no usable `text` — its file lives behind the `attachments[]` array each normalized `Message` now exposes (`{type, file_name, mime_type, src_url, size, is_voice_note}`). `download_asset` is how an MCP-only agent pulls that file's actual bytes without touching the raw Beeper Desktop API on `:23373`: reference the attachment **either** by its `src_url` (the value straight off the attachment — an `mxc://` / `localmxc://` URL, or a `file://` URL inside Beeper's media cache once the file is downloaded locally; arbitrary `file://` paths and other schemes are refused, so the tool can only reach real attachments, never the rest of the container's filesystem) **or** by `chat_id` + `message_id` (+ optional `index` to pick one of several), in which case beeperbox resolves the `src_url` and also returns the file's `file_name` / `mime_type` / `size`. The bytes come back **base64** in `data_base64` (with `content_type`, `bytes`, `encoding`). Because the payload is base64 inside the JSON-RPC result, it's capped at `BEEPERBOX_MAX_ASSET_BYTES` (default 8 MiB) — an oversized asset returns a clear error rather than a truncated file; raise the cap, or fetch `GET /v1/assets/serve?url=…` directly, for larger files. Internally `download_asset` proxies that same `serve` endpoint, so a remote/MCP-only deployment that only publishes `:23375` can still read attachments. ### Schemas the LLM learns once and reuses everywhere diff --git a/mcp/server.js b/mcp/server.js index cfa0133..e31e254 100644 --- a/mcp/server.js +++ b/mcp/server.js @@ -7,6 +7,7 @@ const http = require('http'); const crypto = require('crypto'); +const path = require('path'); const PORT = parseInt(process.env.MCP_PORT || '23375', 10); const BEEPER_API = process.env.BEEPER_API || 'http://127.0.0.1:23373'; @@ -67,18 +68,40 @@ const MAX_ASSET_BYTES = envIntNonNeg('BEEPERBOX_MAX_ASSET_BYTES', 8 * 1024 * 102 // stall the request forever and the buffer can't grow unbounded behind it. const ASSET_TIMEOUT_MS = envIntNonNeg('BEEPERBOX_ASSET_TIMEOUT_MS', 30000); -// Attachment src_urls are always Matrix content URLs (mxc:// / localmxc://). -// The Beeper /v1/assets/serve endpoint ALSO accepts file:// — which, proxied -// through download_asset, would let any MCP caller read arbitrary local files -// in the container (env-bearing paths, the sent-ledger, anything the Beeper -// process can read) and exfiltrate secrets. We refuse anything but mxc:// -// /localmxc:// so the tool can only reach real message attachments, never the -// local filesystem. Applied to BOTH the caller-supplied src_url and a src_url -// resolved off a message (a hostile sender could craft a file:// attachment). +// Real attachment src_urls come in two shapes: remote Matrix content +// (mxc:// / localmxc://) and, once Beeper has cached the file locally, +// file:///root/.config/BeeperTexts/media/... — verified against a live account. +// The Beeper /v1/assets/serve endpoint accepts all three, INCLUDING an +// arbitrary file:// path. Proxied unguarded, that lets any MCP caller read +// arbitrary local files (env-bearing paths, the sent-ledger, account secrets) +// or reach internal hosts via http(s):// (SSRF). So we allow mxc:// / localmxc:// +// outright, allow file:// ONLY when it resolves inside Beeper's media cache, and +// refuse everything else. The file:// path is decoded and normalized before the +// prefix check so encoded (%2e/%2f) and ../ traversal can't escape the cache +// root. Applied to BOTH the caller-supplied src_url and a src_url resolved off a +// message (a hostile sender could craft a file:// attachment pointing elsewhere). +const ASSET_FILE_ROOT = (() => { + const r = process.env.BEEPERBOX_ASSET_FILE_ROOT || '/root/.config/BeeperTexts/media/'; + return r.endsWith('/') ? r : r + '/'; +})(); + +function fileUrlInsideMediaCache(srcURL) { + let p; + try { p = new URL(srcURL).pathname; } catch { return false; } + // Decode percent-encoding (%2e == '.', %2f == '/') so an encoded traversal + // can't slip past, then collapse ../ and ./ to the real target path. + try { p = decodeURIComponent(p); } catch { return false; } + return path.posix.normalize(p).startsWith(ASSET_FILE_ROOT); +} + function assertServableSrcUrl(srcURL) { - if (!/^(?:mxc|localmxc):\/\//i.test(String(srcURL))) { - throw rpcError(-32602, 'src_url must be an mxc:// or localmxc:// URL — file:// and other schemes are refused'); + const s = String(srcURL); + if (/^(?:mxc|localmxc):\/\//i.test(s)) return; + if (/^file:\/\//i.test(s)) { + if (fileUrlInsideMediaCache(s)) return; + throw rpcError(-32602, `file:// src_url must resolve inside the Beeper media cache (${ASSET_FILE_ROOT}); other paths are refused`); } + throw rpcError(-32602, 'src_url must be an mxc://, localmxc://, or Beeper-cache file:// URL — other schemes are refused'); } // Strip the port from a Host header value ("127.0.0.1:23375" -> "127.0.0.1", @@ -275,8 +298,9 @@ async function getAccountMap() { // Map Beeper's raw message `attachments[]` into a normalized shape MCP clients // can act on. Pure passthrough of the fields that already ride the raw message // — no extra fetch. `src_url` is the download reference (an mxc:// / localmxc:// -// Matrix content URL) that download_asset takes. `size` carries the byte length -// (raw `fileSize`) when present. Returns [] when +// Matrix content URL, or a file:// URL into Beeper's media cache once the file +// is downloaded locally) that download_asset takes. `size` carries the byte +// length (raw `fileSize`) when present. Returns [] when // there are no attachments, so the field is always an array. function normalizeAttachments(raw) { const list = Array.isArray(raw?.attachments) ? raw.attachments : []; @@ -731,11 +755,11 @@ const TOOLS = [ }, { name: 'download_asset', - description: 'Download the bytes of a message attachment (image, PDF, document, voice note, etc.) and return them base64-encoded. This is the MCP-only way to reach an attachment\'s actual content — use it after read_chat / poll_messages surfaces a message whose `attachments[]` carries a `src_url`. Reference the attachment EITHER by passing its `src_url` directly (the value from a normalized attachment — an mxc:// or localmxc:// URL; other schemes such as file:// are refused), OR by `chat_id` + `message_id` (+ optional `index` to pick one of several attachments), in which case beeperbox resolves the src_url for you and also returns the attachment\'s file_name / mime_type / size. Returns { content_type, bytes, encoding: "base64", data_base64, ... }. Large files are capped (default 8 MiB) to keep the JSON-RPC result bounded — raise BEEPERBOX_MAX_ASSET_BYTES if you need bigger.', + description: 'Download the bytes of a message attachment (image, PDF, document, voice note, etc.) and return them base64-encoded. This is the MCP-only way to reach an attachment\'s actual content — use it after read_chat / poll_messages surfaces a message whose `attachments[]` carries a `src_url`. Reference the attachment EITHER by passing its `src_url` directly (the value from a normalized attachment — an mxc:// / localmxc:// URL, or a file:// URL inside Beeper\'s local media cache; arbitrary file:// paths and other schemes are refused), OR by `chat_id` + `message_id` (+ optional `index` to pick one of several attachments), in which case beeperbox resolves the src_url for you and also returns the attachment\'s file_name / mime_type / size. Returns { content_type, bytes, encoding: "base64", data_base64, ... }. Large files are capped (default 8 MiB) to keep the JSON-RPC result bounded — raise BEEPERBOX_MAX_ASSET_BYTES if you need bigger.', inputSchema: { type: 'object', properties: { - src_url: { type: 'string', description: 'The attachment\'s `src_url` (an mxc:// or localmxc:// URL) from a normalized Message attachment. file:// and other non-Matrix schemes are refused. Provide this, OR chat_id + message_id.' }, + src_url: { type: 'string', description: 'The attachment\'s `src_url` from a normalized Message attachment — an mxc:// / localmxc:// URL or a file:// URL inside Beeper\'s media cache. Arbitrary file:// paths and other schemes are refused. Provide this, OR chat_id + message_id.' }, chat_id: { type: 'string', description: 'The chat containing the message. Required if src_url is omitted (used with message_id to resolve the attachment).' }, message_id: { type: 'string', description: 'The message whose attachment to download. Required if src_url is omitted.' }, index: { type: 'integer', description: 'Which attachment to download when resolving by message_id and the message has more than one (default 0).', default: 0, minimum: 0 }, diff --git a/mcp/server.test.js b/mcp/server.test.js index 43d4164..16ef841 100644 --- a/mcp/server.test.js +++ b/mcp/server.test.js @@ -210,26 +210,44 @@ test('normalizeAttachments: preserves order and maps every entry of a multi-atta }); // ── assertServableSrcUrl (download_asset local-file-read guard) ─── -test('assertServableSrcUrl: allows mxc:// and localmxc:// (the real attachment schemes)', () => { +// Real attachment src_urls (verified live) are mxc:// / localmxc:// for remote +// media, and file:///root/.config/BeeperTexts/media/... once Beeper caches the +// file locally. The guard must serve all three yet refuse file:// outside that +// cache (and other schemes) — else an MCP caller reads arbitrary local files. +const MEDIA = 'file:///root/.config/BeeperTexts/media/'; + +test('assertServableSrcUrl: allows mxc:// and localmxc:// (remote attachment schemes)', () => { assert.doesNotThrow(() => S.assertServableSrcUrl('mxc://beeper.com/abc')); assert.doesNotThrow(() => S.assertServableSrcUrl('localmxc://beeper.com/cached')); assert.doesNotThrow(() => S.assertServableSrcUrl('MXC://BEEPER.COM/ABC')); // scheme is case-insensitive }); -test('assertServableSrcUrl: refuses file:// — the arbitrary-local-file-read vector', () => { - // The load-bearing security assertion: a file:// src_url would let an MCP - // caller read /etc/passwd, the sent-ledger, env-bearing paths, etc. via the - // Beeper serve endpoint. Must throw a -32602 BEFORE any fetch. - assert.throws(() => S.assertServableSrcUrl('file:///etc/passwd'), /schemes are refused/); +test('assertServableSrcUrl: allows file:// INSIDE the Beeper media cache (the cached-attachment case)', () => { + assert.doesNotThrow(() => S.assertServableSrcUrl(MEDIA + 'local.beeper.com/x.pdf')); + assert.doesNotThrow(() => S.assertServableSrcUrl(MEDIA + 'a/b/c/deep.bin')); +}); + +test('assertServableSrcUrl: refuses file:// OUTSIDE the cache — the arbitrary-local-file-read vector', () => { + // The load-bearing security assertion: these would read secrets via serve. + assert.throws(() => S.assertServableSrcUrl('file:///etc/passwd'), /refused/); + assert.throws(() => S.assertServableSrcUrl('file:///root/.config/beeperbox-sent-ledger.json'), /refused/); + // A sibling dir that shares the prefix but isn't the cache (trailing-slash guard). + assert.throws(() => S.assertServableSrcUrl('file:///root/.config/BeeperTexts/mediahack/x'), /refused/); +}); + +test('assertServableSrcUrl: refuses ../ and percent-encoded traversal out of the cache', () => { + // new URL() does NOT collapse ../, so the guard decodes + path.normalize()s + // before the prefix check. These must NOT escape to /etc or the config dir. + assert.throws(() => S.assertServableSrcUrl(MEDIA + '../../../etc/passwd'), /refused/); + assert.throws(() => S.assertServableSrcUrl(MEDIA + '..%2f..%2f..%2fetc/passwd'), /refused/); + assert.throws(() => S.assertServableSrcUrl(MEDIA + '%2e%2e/%2e%2e/beeperbox-sent-ledger.json'), /refused/); }); test('assertServableSrcUrl: refuses http(s):// (SSRF) and other non-Matrix schemes', () => { - assert.throws(() => S.assertServableSrcUrl('http://169.254.169.254/latest/meta-data/'), /schemes are refused/); - assert.throws(() => S.assertServableSrcUrl('https://evil.example/x'), /schemes are refused/); - assert.throws(() => S.assertServableSrcUrl('ftp://h/x'), /schemes are refused/); - assert.throws(() => S.assertServableSrcUrl(''), /schemes are refused/); - // Defeat a "mxc as a path, not the scheme" smuggle attempt. - assert.throws(() => S.assertServableSrcUrl('file:///x?mxc://'), /schemes are refused/); + assert.throws(() => S.assertServableSrcUrl('http://169.254.169.254/latest/meta-data/'), /refused/); + assert.throws(() => S.assertServableSrcUrl('https://evil.example/x'), /refused/); + assert.throws(() => S.assertServableSrcUrl('ftp://h/x'), /refused/); + assert.throws(() => S.assertServableSrcUrl(''), /refused/); }); // ── echo-guard matcher (pure) ───────────────────────────────────── diff --git a/scripts/asset-serve-check.js b/scripts/asset-serve-check.js index afd35f1..d33b5d6 100644 --- a/scripts/asset-serve-check.js +++ b/scripts/asset-serve-check.js @@ -113,15 +113,21 @@ beeper.listen(0, '127.0.0.1', () => { const e3 = await rpc(mcpPort, { jsonrpc: '2.0', id: 7, method: 'tools/call', params: { name: 'download_asset', arguments: {} } }); check(/requires src_url/.test(e3.error?.message || ''), 'missing ref errors clearly'); - // ── SECURITY: file:// (and non-Matrix schemes) must be refused BEFORE the - // serve fetch — else an MCP caller reads arbitrary local files. The refusal - // message is distinct from any serve-path error, proving the fetch never ran. + // ── SECURITY: file:// outside Beeper's media cache (and non-Matrix schemes) + // must be refused BEFORE the serve fetch — else an MCP caller reads arbitrary + // local files. The refusal message is distinct from any serve-path error, + // proving the fetch never ran. file:// INSIDE the cache is the real cached- + // attachment case and must still download (verified live). const s1 = await rpc(mcpPort, { jsonrpc: '2.0', id: 9, method: 'tools/call', params: { name: 'download_asset', arguments: { src_url: 'file:///etc/passwd' } } }); - check(/file:\/\/ and other schemes are refused/.test(s1.error?.message || ''), `file:// src_url is REFUSED (local-file-read guard): ${s1.error?.message || 'NO ERROR'}`); + check(/refused/.test(s1.error?.message || ''), `file:///etc/passwd is REFUSED (local-file-read guard): ${s1.error?.message || 'NO ERROR'}`); const s2 = await rpc(mcpPort, { jsonrpc: '2.0', id: 10, method: 'tools/call', params: { name: 'download_asset', arguments: { src_url: 'http://169.254.169.254/latest/meta-data/' } } }); - check(/schemes are refused/.test(s2.error?.message || ''), 'http:// src_url (SSRF) is refused too'); - const s3 = parse(await rpc(mcpPort, { jsonrpc: '2.0', id: 11, method: 'tools/call', params: { name: 'download_asset', arguments: { src_url: 'localmxc://h/cached' } } })); - check(!s3.__error, 'localmxc:// (legit local-cache attachment scheme) is still allowed'); + check(/refused/.test(s2.error?.message || ''), 'http:// src_url (SSRF) is refused too'); + const s3 = await rpc(mcpPort, { jsonrpc: '2.0', id: 11, method: 'tools/call', params: { name: 'download_asset', arguments: { src_url: 'file:///root/.config/BeeperTexts/media/../../../etc/passwd' } } }); + check(/refused/.test(s3.error?.message || ''), '../ traversal out of the media cache is refused'); + const s4 = parse(await rpc(mcpPort, { jsonrpc: '2.0', id: 12, method: 'tools/call', params: { name: 'download_asset', arguments: { src_url: 'file:///root/.config/BeeperTexts/media/local.beeper.com/cached.bin' } } })); + check(!s4.__error, 'file:// INSIDE the media cache (the real cached-attachment case) still downloads'); + const s5 = parse(await rpc(mcpPort, { jsonrpc: '2.0', id: 13, method: 'tools/call', params: { name: 'download_asset', arguments: { src_url: 'localmxc://h/cached' } } })); + check(!s5.__error, 'localmxc:// is still allowed'); // ── attachments[] surfaced on a normalized read (via the message endpoint shape) ── const d3 = parse(await rpc(mcpPort, { jsonrpc: '2.0', id: 8, method: 'tools/call', params: { name: 'download_asset', arguments: { chat_id: 'c1', message_id: 'm1' } } })); From 910e6ddceacd02e81f30f6701e6411bae46dafa2 Mon Sep 17 00:00:00 2001 From: hamr0 Date: Tue, 16 Jun 2026 11:24:29 +0200 Subject: [PATCH 3/4] =?UTF-8?q?harden(mcp):=20download=5Fasset=20guard=20?= =?UTF-8?q?=E2=80=94=20reject=20file://=20host=20+=20double-encoding;=20ho?= =?UTF-8?q?nest=20defense-in-depth=20framing?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Code review (high) + live validation against real Beeper: - Real /v1/assets/serve has its OWN guards: 403 for a file:// outside its media dir, 400 for a non-mxc/localmxc/file scheme. So the original "arbitrary local file read / SSRF" was NOT a confirmed end-to-end exploit against real Beeper — it was confirmed only against the stub, which does not model serve's guard. Reframed as defense-in-depth in the CHANGELOG and comments (clear early error + survives an upstream guard regressing), not a patched live vuln. Two real guard-soundness gaps the reviewer found, both fixed (neither was exploitable against real serve — 400/404 — but the guard should validate what it forwards): - file:// authority/host (file://attacker/) passed the pathname check but the host was forwarded — now reject any non-empty host (file://localhost normalizes the host away, still allowed). - double-encoded %252e survived one decode as literal %2e past normalize(); now refuse any residual %2e/%2f after a single decode. Single-decode is deliberate — it matches serve's own single-decode and avoids over-rejecting a legit filename containing a literal % (which a decode loop would throw on). Unit +4 (host allow/refuse, double-encode); asset-serve-check +2; all re-validated live (real cached attachment still downloads; host-bearing variant of the real url refused). 39 unit tests green. Co-Authored-By: Claude Opus 4.8 --- CHANGELOG.md | 2 +- mcp/server.js | 46 +++++++++++++++++++++++------------- mcp/server.test.js | 16 +++++++++++++ scripts/asset-serve-check.js | 13 ++++++---- 4 files changed, 56 insertions(+), 21 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3257d5a..954a9eb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,7 +21,7 @@ Published tags on GHCR: `:X.Y.Z` (exact, immutable), `:X.Y` (rolling within a mi ### Security -- **`download_asset` confines `src_url` to attachments, not the filesystem.** Real attachment `src_url`s (verified against a live account) are `mxc://` / `localmxc://` for remote media and `file:///root/.config/BeeperTexts/media/…` once Beeper caches the file locally — and the Beeper `serve` endpoint will serve *any* `file://` path. An unguarded proxy would therefore let any MCP caller read arbitrary local files (env-bearing paths, the sent-ledger — i.e. `BEEPER_TOKEN` / `MCP_AUTH_TOKEN` exfiltration) and reach internal hosts via `http(s)://` (SSRF). `download_asset` now allows `mxc://` / `localmxc://`, allows `file://` **only when it resolves inside Beeper's media cache** (`BEEPERBOX_ASSET_FILE_ROOT`, default `/root/.config/BeeperTexts/media/`; the path is percent-decoded and normalized first so `../` and `%2e/%2f` traversal can't escape), and refuses everything else **before** the fetch — covering both the caller-supplied `src_url` and a `src_url` resolved off a (potentially hostile-sender-crafted) message attachment. A per-call timeout (`BEEPERBOX_ASSET_TIMEOUT_MS`, default 30 s) bounds a hung or slow-drip source so it can't stall the request or grow the buffer unbounded. +- **`download_asset` confines `src_url` to attachments, not the filesystem (defense-in-depth).** Real attachment `src_url`s (verified against a live account) are `mxc://` / `localmxc://` for remote media and `file:///root/.config/BeeperTexts/media/…` once Beeper caches the file locally. Beeper's own `serve` endpoint already restricts this — it returns `403` for a `file://` outside its media dir and `400` for a non-`mxc`/`localmxc`/`file` scheme (confirmed live) — but `download_asset` is the network-reachable MCP surface and does not lean on that undocumented upstream behavior. It independently allows `mxc://` / `localmxc://`, allows `file://` **only when it resolves inside the media cache** (`BEEPERBOX_ASSET_FILE_ROOT`, default `/root/.config/BeeperTexts/media/`; the path is percent-decoded + normalized, and any URL authority/host or residual double-encoded `%2e`/`%2f` is refused, so `../`/UNC tricks can't escape the prefix check), and refuses everything else **before** the fetch — covering both the caller-supplied `src_url` and a `src_url` resolved off a (potentially hostile-sender-crafted) message attachment. The payoff is not "this was a confirmed exploit" — Beeper's guard blocks the raw case — but a second, in-repo, clear-error boundary that survives an upstream guard regressing. A per-call timeout (`BEEPERBOX_ASSET_TIMEOUT_MS`, default 30 s) bounds a hung or slow-drip source. ### Notes diff --git a/mcp/server.js b/mcp/server.js index e31e254..1cb9b77 100644 --- a/mcp/server.js +++ b/mcp/server.js @@ -69,28 +69,41 @@ const MAX_ASSET_BYTES = envIntNonNeg('BEEPERBOX_MAX_ASSET_BYTES', 8 * 1024 * 102 const ASSET_TIMEOUT_MS = envIntNonNeg('BEEPERBOX_ASSET_TIMEOUT_MS', 30000); // Real attachment src_urls come in two shapes: remote Matrix content -// (mxc:// / localmxc://) and, once Beeper has cached the file locally, +// (mxc:// / localmxc://) and, once Beeper caches the file locally, // file:///root/.config/BeeperTexts/media/... — verified against a live account. -// The Beeper /v1/assets/serve endpoint accepts all three, INCLUDING an -// arbitrary file:// path. Proxied unguarded, that lets any MCP caller read -// arbitrary local files (env-bearing paths, the sent-ledger, account secrets) -// or reach internal hosts via http(s):// (SSRF). So we allow mxc:// / localmxc:// -// outright, allow file:// ONLY when it resolves inside Beeper's media cache, and -// refuse everything else. The file:// path is decoded and normalized before the -// prefix check so encoded (%2e/%2f) and ../ traversal can't escape the cache -// root. Applied to BOTH the caller-supplied src_url and a src_url resolved off a -// message (a hostile sender could craft a file:// attachment pointing elsewhere). +// Beeper's own /v1/assets/serve already guards this (it 403s a file:// outside +// its media dir and 400s a non-mxc/localmxc/file scheme — observed live). We do +// NOT lean on that undocumented upstream behavior: download_asset is the +// network-reachable MCP surface, so it independently allows mxc:// / localmxc://, +// allows file:// ONLY when it resolves inside the media cache, and refuses +// everything else BEFORE the fetch — defense in depth, plus a clear error +// instead of a raw Beeper 4xx. The file:// path is decoded + normalized and any +// authority/host or double-encoded dot/slash refused, so a caller can't smuggle +// ../ or a UNC host past the prefix check. Applied to BOTH the caller-supplied +// src_url and a src_url resolved off a message (a hostile sender could craft a +// file:// attachment pointing elsewhere). const ASSET_FILE_ROOT = (() => { const r = process.env.BEEPERBOX_ASSET_FILE_ROOT || '/root/.config/BeeperTexts/media/'; return r.endsWith('/') ? r : r + '/'; })(); function fileUrlInsideMediaCache(srcURL) { + let u; + try { u = new URL(srcURL); } catch { return false; } + // A file:// authority/host (file://host/path) is a network/UNC semantic the + // pathname check would miss — and we forward the ORIGINAL url, host and all. + // Legit local attachments have no host (file:///… ; even file://localhost + // normalizes the host away), so refuse any non-empty host. + if (u.host) return false; let p; - try { p = new URL(srcURL).pathname; } catch { return false; } - // Decode percent-encoding (%2e == '.', %2f == '/') so an encoded traversal - // can't slip past, then collapse ../ and ./ to the real target path. - try { p = decodeURIComponent(p); } catch { return false; } + try { p = decodeURIComponent(u.pathname); } catch { return false; } + // One decode matches Beeper serve's own single-decode (observed: it 404s a + // double-encoded path as a literal, never re-decodes). Any %2e/%2f still + // present after that decode is a DOUBLE-encoded dot/slash — an attempt to + // smuggle ../ past normalize() — so refuse rather than forward it. (A plain + // literal %25 in a real filename decodes to a bare % here, not %2e/%2f, so + // this doesn't over-reject normal names.) + if (/%2[ef]/i.test(p)) return false; return path.posix.normalize(p).startsWith(ASSET_FILE_ROOT); } @@ -1014,8 +1027,9 @@ async function callTool(name, args) { srcURL = att.src_url; meta = { file_name: att.file_name, mime_type: att.mime_type, size: att.size }; } - // Refuse file:// (and any non-Matrix scheme) BEFORE the fetch — this is - // the local-file-read / secret-exfil guard, covering both ref paths. + // Confine the src_url to a real attachment (mxc/localmxc, or file:// inside + // the media cache) BEFORE the fetch — defense-in-depth over Beeper serve's + // own path guard, covering both ref paths. See assertServableSrcUrl. assertServableSrcUrl(srcURL); // /v1/assets/serve streams the bytes for an mxc:// / localmxc:// URL. // beeperFetch raw-mode returns the buffer + content-type, enforces the diff --git a/mcp/server.test.js b/mcp/server.test.js index 16ef841..0e9dae0 100644 --- a/mcp/server.test.js +++ b/mcp/server.test.js @@ -243,6 +243,22 @@ test('assertServableSrcUrl: refuses ../ and percent-encoded traversal out of the assert.throws(() => S.assertServableSrcUrl(MEDIA + '%2e%2e/%2e%2e/beeperbox-sent-ledger.json'), /refused/); }); +test('assertServableSrcUrl: refuses DOUBLE-encoded traversal (%252e) without re-decoding it open', () => { + // %252e -> one decode -> %2e (a literal dot-encoding). The guard refuses any + // residual %2e/%2f after a single decode rather than forwarding it — serve + // single-decodes too, so this never reaches the filesystem either way. + assert.throws(() => S.assertServableSrcUrl(MEDIA + '%252e%252e/%252e%252e/etc/passwd'), /refused/); + assert.throws(() => S.assertServableSrcUrl(MEDIA + 'a%252fb'), /refused/); +}); + +test('assertServableSrcUrl: refuses a file:// URL with a non-empty host (authority is forwarded, not path-checked)', () => { + // file://attacker/ has a valid-looking pathname but a host the + // pathname check would miss; the original url (host and all) is what we + // forward, so reject any authority. file://localhost/… is fine (host empties). + assert.throws(() => S.assertServableSrcUrl('file://attacker/root/.config/BeeperTexts/media/x'), /refused/); + assert.doesNotThrow(() => S.assertServableSrcUrl('file://localhost/root/.config/BeeperTexts/media/x')); +}); + test('assertServableSrcUrl: refuses http(s):// (SSRF) and other non-Matrix schemes', () => { assert.throws(() => S.assertServableSrcUrl('http://169.254.169.254/latest/meta-data/'), /refused/); assert.throws(() => S.assertServableSrcUrl('https://evil.example/x'), /refused/); diff --git a/scripts/asset-serve-check.js b/scripts/asset-serve-check.js index d33b5d6..b9f2447 100644 --- a/scripts/asset-serve-check.js +++ b/scripts/asset-serve-check.js @@ -114,10 +114,11 @@ beeper.listen(0, '127.0.0.1', () => { check(/requires src_url/.test(e3.error?.message || ''), 'missing ref errors clearly'); // ── SECURITY: file:// outside Beeper's media cache (and non-Matrix schemes) - // must be refused BEFORE the serve fetch — else an MCP caller reads arbitrary - // local files. The refusal message is distinct from any serve-path error, - // proving the fetch never ran. file:// INSIDE the cache is the real cached- - // attachment case and must still download (verified live). + // must be refused BEFORE the serve fetch. This is download_asset's own + // confinement — Beeper's serve endpoint independently 403s/400s these too, + // but we don't rely on that undocumented upstream guard. The refusal message + // is distinct from any serve-path error, proving the fetch never ran. file:// + // INSIDE the cache is the real cached-attachment case and must still download. const s1 = await rpc(mcpPort, { jsonrpc: '2.0', id: 9, method: 'tools/call', params: { name: 'download_asset', arguments: { src_url: 'file:///etc/passwd' } } }); check(/refused/.test(s1.error?.message || ''), `file:///etc/passwd is REFUSED (local-file-read guard): ${s1.error?.message || 'NO ERROR'}`); const s2 = await rpc(mcpPort, { jsonrpc: '2.0', id: 10, method: 'tools/call', params: { name: 'download_asset', arguments: { src_url: 'http://169.254.169.254/latest/meta-data/' } } }); @@ -128,6 +129,10 @@ beeper.listen(0, '127.0.0.1', () => { check(!s4.__error, 'file:// INSIDE the media cache (the real cached-attachment case) still downloads'); const s5 = parse(await rpc(mcpPort, { jsonrpc: '2.0', id: 13, method: 'tools/call', params: { name: 'download_asset', arguments: { src_url: 'localmxc://h/cached' } } })); check(!s5.__error, 'localmxc:// is still allowed'); + const s6 = await rpc(mcpPort, { jsonrpc: '2.0', id: 14, method: 'tools/call', params: { name: 'download_asset', arguments: { src_url: 'file:///root/.config/BeeperTexts/media/%252e%252e/%252e%252e/etc/passwd' } } }); + check(/refused/.test(s6.error?.message || ''), 'double-encoded (%252e) traversal is refused at the guard'); + const s7 = await rpc(mcpPort, { jsonrpc: '2.0', id: 15, method: 'tools/call', params: { name: 'download_asset', arguments: { src_url: 'file://attacker/root/.config/BeeperTexts/media/x' } } }); + check(/refused/.test(s7.error?.message || ''), 'file:// with a non-empty host is refused at the guard'); // ── attachments[] surfaced on a normalized read (via the message endpoint shape) ── const d3 = parse(await rpc(mcpPort, { jsonrpc: '2.0', id: 8, method: 'tools/call', params: { name: 'download_asset', arguments: { chat_id: 'c1', message_id: 'm1' } } })); From 34637c6b57c80e6d23d498a5d53e55b48f6ef594 Mon Sep 17 00:00:00 2001 From: hamr0 Date: Tue, 16 Jun 2026 11:32:19 +0200 Subject: [PATCH 4/4] docs: note download_asset src_url confinement (context + PRD threat model) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reflect the hardened guard in the two docs that were silent on it: - beeperbox.context.md download_asset: add a Security note (mxc/localmxc or file:// inside the media cache; other path/scheme/host/encoded-../ refused). - PRD §7 threat model: add an "Unreleased hardening" bullet, framed as defense-in-depth over Beeper serve's own 403/400 — not a patched live exploit. GUIDE/README already accurate (12 tools, "arbitrary file:// refused"); CHANGELOG covered in the harden commit. Co-Authored-By: Claude Opus 4.8 --- beeperbox.context.md | 1 + docs/PRD.md | 4 ++++ 2 files changed, 5 insertions(+) diff --git a/beeperbox.context.md b/beeperbox.context.md index 8146d60..1055f21 100644 --- a/beeperbox.context.md +++ b/beeperbox.context.md @@ -237,6 +237,7 @@ Download a message attachment's bytes and return them base64-encoded — the MCP **Arguments:** `{src_url?: string}` **or** `{chat_id: string, message_id: string, index?: number = 0}`. **Returns:** `{src_url, content_type, bytes, encoding: "base64", data_base64, file_name?, mime_type?, size?}`. **Note:** The bytes ride base64 inside the JSON-RPC result, so the asset is capped at `BEEPERBOX_MAX_ASSET_BYTES` (default 8 MiB) — an oversized file returns a clear error, not a truncated body. Internally proxies `GET /v1/assets/serve?url=…`, so a remote deployment publishing only `:23375` can still read attachments. Raise the cap or hit `serve` directly for larger files. +**Security:** `src_url` is confined to real attachments — `mxc://` / `localmxc://`, or a `file://` URL inside Beeper's media cache (`BEEPERBOX_ASSET_FILE_ROOT`, default `/root/.config/BeeperTexts/media/`); any other path, scheme, URL host, or encoded-`../` traversal is refused before the fetch. This is defense-in-depth — Beeper's own `serve` independently `403`s/`400`s those — so a caller can't turn `download_asset` into an arbitrary-file reader even if the upstream guard regresses. ## Schemas diff --git a/docs/PRD.md b/docs/PRD.md index a9dd0ca..d416a61 100644 --- a/docs/PRD.md +++ b/docs/PRD.md @@ -180,6 +180,10 @@ beeperbox is a single-tenant container that holds a credential (`BEEPER_TOKEN`) - **Privilege-escalation hardening** — `security_opt: [no-new-privileges:true]`, shrinking the blast radius of Beeper running as root with `--no-sandbox`. - **Reproducible/verified builds (opt-in)** — `BEEPER_VERSION` + `BEEPER_SHA256` pin and hash-check the AppImage; default stays rolling auto-update. +**Unreleased hardening** — alongside `download_asset` (see CHANGELOG): + +- **`download_asset` src_url confinement (defense-in-depth)** — the tool proxies Beeper's `serve` endpoint, which accepts `file://` paths. Beeper independently restricts these (`403` outside its media dir, `400` for a non-`mxc`/`localmxc`/`file` scheme — confirmed live), but `download_asset` is the network-reachable MCP surface and does not rely on that undocumented upstream guard: it allows `mxc://` / `localmxc://`, allows `file://` only inside the media cache (`BEEPERBOX_ASSET_FILE_ROOT`), and refuses any other path, scheme, URL host, or encoded-`../` traversal **before** the fetch — for both a caller-supplied and a message-resolved `src_url`. Not a patched live exploit; a second in-repo boundary that survives an upstream regression. + **Load-bearing decisions (do not relitigate without changing this doc):** - **In-container listeners bind `0.0.0.0` on purpose.** A loopback bind inside the container is unreachable through a Docker published port. The defense is auth + Host/Origin + the loopback *publish*, not the bind address. (A "bind loopback" fix was proposed, tested, and rejected because it silently breaks the published port.)