diff --git a/CHANGELOG.md b/CHANGELOG.md index fa81cf9..27d9ffa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -22,12 +22,15 @@ First feature driven by a real consumer ([multis](https://github.com/hamr0/multi - **The cursor is opaque and fully stateless server-side** — persist it to disk and resume across a process/container restart with no missed or duplicated messages. Same-millisecond messages are deduplicated by id (the `{ts, ids@ts}` cursor encoding), which is the specific bug-class hand-rolled pollers get wrong. - **Echo-guard via a new `source` field on every `Message`** (`"api"` | `"external"`), plus an echoed **`client_tag`**. On a single Beeper account both the human owner's own typed messages and the agent's API replies are `is_self: true`, so `is_self` cannot guard against an agent answering its own sends. `send_message` / `note_to_self` now accept an optional `client_tag` and record what they send to a ledger persisted in the config volume; on read-back (`poll_messages`, `read_chat`, `search_messages`) those messages are marked `source: "api"` with the tag echoed. Branch on `source`, not `is_self`: skip `"api"`, process `"external"` (the owner's own Note-to-self commands included). Retires the brittle text-prefix echo-hack. - **Schema additions are additive** (`source`, `client_tag` on `Message`; `client_tag` on the two send tools) — no existing field renamed or removed. New tool + new schema fields = MINOR per the versioning policy. - - **Honest limitation (documented loudly):** the `source` match is best-effort and **unverified against a live Beeper account** (CI has none — the standing gate limitation). Primary match is the `pendingMessageID`; because Beeper may swap it for a real bridge id on ack, a content fallback also matches, but only for the account's own messages in the same chat within 15 minutes — conservative so a human re-typing identical text is never mis-tagged and dropped. + - **Reliable echo via final-id resolution** (second multis ask). The first cut matched read-backs by the `pendingMessageID` returned at send time, but Beeper swaps that for the real bridge id on ack — so the id never matched and every match degraded to the fragile 15-minute text fallback. `send_message` / `note_to_self` now **resolve the pending id to the final bridge id** (GET `.../messages/{pendingMessageID}` until the id swaps, bounded + best-effort) and record **both** ids in the ledger, so a read-back matches by **exact id** whether or not the swap happened. The content fallback survives only as a last-ditch net **and only for sends whose final id could not be resolved** — once a send is resolved, its text fallback is retired, so a human re-typing identical text is no longer mis-tagged as the agent's own and dropped. Lets a consumer drop its own send-echo bookkeeping. Tunable via `BEEPERBOX_RESOLVE_RETRIES` (default 4) / `BEEPERBOX_RESOLVE_DELAY_MS` (default 250) / `BEEPERBOX_RESOLVE_TIMEOUT_MS` (default 3000, per-attempt fetch timeout so a hung API can't stall a send); set retries to 0 to disable resolution. Worst-case added send latency is bounded at `retries × (timeout + delay)`. + - **Additive return fields** on `send_message` / `note_to_self`: `pending_message_id` (the raw pending id) and `resolved` (bool); `message_id` is now the final bridge id when `resolved: true`, else the pending id. No field removed. + - **Honest limitation (documented loudly):** the `source` match is **unverified against a live Beeper account** (CI has none — the standing gate limitation). The exact-id resolution depends on Beeper's live id/ack behavior (the `pendingMessageID` → final-id swap was proven against a real account by multis but is not exercised in CI); validate against your own account before relying on it for a high-stakes auto-responder. - **New env var `BEEPERBOX_SENT_LEDGER`** (default `/root/.config/beeperbox-sent-ledger.json`, inside the persisted config volume) — the echo-guard ledger path. Best-effort: a failed write degrades the guard to in-memory for that run and warns once to stderr; it never fails a send. - **Input bounds on the new caller-supplied fields** (from a `/security` pass on the diff): the opaque `cursor`'s `ids` array is capped at 4096 on decode — rejected as malformed beyond that (a real cursor only holds the ids sharing one millisecond, so this never rejects a server-issued cursor) — closing an O(n)-per-message CPU-amplification path; and `client_tag` is truncated to 256 chars before it enters the ledger, closing a ledger-bloat path. Both are exercised by unit tests that fail without the cap. ### Tests - **`mcp/server.test.js`** — `node --test` unit suite over the pure cursor/dedup/echo-guard logic (encode/decode round-trip, strict-after filtering, same-millisecond id dedup, cursor advance, a full seed→poll→restart-resume walk, the `source` matcher's id/content/window/cross-chat cases, and a real-temp-file ledger persistence round-trip). No live Beeper needed — this is the half of the feature that *is* deterministically testable, and it covers exactly the bug-class. Wired into `mcp-test.yml` as a fast `unit` job; `poll_messages` added to the guard-check + smoke-test tool-contract assertions. + - **Final-id resolution cases** (26 tests now): a read-back tagged by its resolved bridge id; the key acceptance test that **a human re-typing identical text after a resolved send stays `external`** (the bug the resolution fixes); two identical-text sends each matched by their own resolved id; an unresolved entry still caught by the text fallback (safety net preserved); and an `addResolvedId` persist/reload round-trip. The decision logic was POC-validated against the prior matcher to confirm the old behavior mis-tags the human (the test can fail), and the entrypoint supervisor's `API_WAS_UP` gate was POC-simulated across the never-logged-in / half-dead / process-death scenarios. ### Documentation - **Made the raw `/v1/` contract explicit** (`beeperbox.context.md`, `docs/GUIDE.md`) so direct-API consumers match the MCP layer instead of re-discovering its quirks: the `?limit=N` ~25-item *floor* (it's a lower bound — over-fetch then slice), the client-side cursor-enumeration recipe `poll_messages` implements (no server-side `since=`), and the canonical field heuristics (note-to-self = `participants.total === 1 && items[0].isSelf`; network via `accountID` → `/v1/accounts`; group = `type === 'group'`; `pendingMessageID` is not a stable delivered id). @@ -39,6 +42,11 @@ First feature driven by a real consumer ([multis](https://github.com/hamr0/multi ### Fixed — `docker restart` no longer segfaults the display `[PATCH]` - **`docker restart` reliably wedged the container with a stale Xvfb lock.** `docker restart` re-runs the entrypoint but preserves the container's writable layer, so Xvfb's `/tmp/.X99-lock` from the previous boot survived into the new process. Xvfb then saw display `:99` as "already active", half-initialized it, and **segfaulted** (`(EE) Server is already active for display 99`) — the backend never bound `127.0.0.1:23373`, socat looped on connection-refused, and the container sat stuck in `health: starting`. Only a full `docker compose down && up` (which discards `/tmp`) recovered. The entrypoint now removes the stale `/tmp/.X99-lock` and `/tmp/.X11-unix/X99` before starting Xvfb, so `docker restart` — the natural operation after any config change — is survivable. No runtime-contract change (MCP tools, HTTP API, schemas, ports bit-identical); PATCH per the versioning policy. +### Added — beepertexts is now supervised (no more silent half-dead container) `[MINOR]` +- **The entrypoint supervises Beeper Desktop instead of launching it unattended.** Previously beepertexts was started once with `&`: if its API/renderer process crashed while the Electron launcher lingered, the container looked "up" and the MCP layer (`:23375`) kept answering, but every tool call failed with `-32603 fetch failed` because the API (`:23373`) was gone — a half-dead container that never self-healed. The entrypoint now runs a supervision loop that **relaunches beepertexts if the process dies, and recycles it if the API stays down after having been up** (watching both the PID and `:23373/v1/spec`, since the launcher can outlive its crashed children). + - **First-run login is protected by an `API_WAS_UP` gate.** Until the user enables "Start API on launch", the API is down *by design*, so the loop only ever recycles-for-API-down once the API has come up at least once this run — a never-logged-in container just runs Beeper steadily and waits for the human (the prior behavior). Clean `docker stop` shutdown (SIGTERM → flush → exit) is preserved. + - **Tunable / reversible:** `BEEPERBOX_SUPERVISE` (default `1`; set `0` for the old forward-signal-and-wait behavior), `BEEPERBOX_SUPERVISE_INTERVAL` (default `10`s), `BEEPERBOX_SUPERVISE_API_GRACE` (default `6` checks ≈ 60s of API-down before recycle). The Docker `HEALTHCHECK` already probes the API through the socat path, so a genuinely dead backend still marks the container unhealthy; supervision adds the *recovery* the healthcheck alone can't provide. + ## [0.5.1] — 2026-05-25 `[PATCH]` Release-pipeline hardening and documentation. PATCH per the versioning policy — these change how releases are *built and described*, not what the running container does: the MCP tool surface, raw/HTTP API, `Chat`/`Message` schemas, and default ports are bit-identical to v0.5.0, and no client-code edits are required. The headline is that the release path is now **gated on the guard tests** with a `:previous` rollback tag, so a broken upstream Beeper can no longer silently become `:latest`. diff --git a/beeperbox.context.md b/beeperbox.context.md index d41b702..7b00d23 100644 --- a/beeperbox.context.md +++ b/beeperbox.context.md @@ -199,8 +199,8 @@ while (true) { Send a text message to a chat. Markdown supported. **Arguments:** `{chat_id: string, text: string, reply_to_message_id?: string, client_tag?: string}`. -**Returns:** `{chat_id, message_id, client_tag, status: "sent"}`. -**Note:** `message_id` is Beeper's `pendingMessageID` — use it for downstream `react_to_message` on the just-sent message. +**Returns:** `{chat_id, message_id, pending_message_id, resolved, client_tag, status: "sent"}`. +**Note:** `message_id` is the **final bridge id** when beeperbox could resolve it (`resolved: true`), else the `pendingMessageID` (`resolved: false`). `pending_message_id` is always the raw pending id. Use `message_id` for downstream `react_to_message`. The resolution is what lets the echo-guard match read-backs by **exact id**, not text (see the Message schema caveat). **`client_tag` (echo-guard):** optional idempotency/echo key. beeperbox records it against this send and echoes it back on the message — marked `source: "api"` — when it reappears in `poll_messages` / `read_chat`. Lets an agent recognize and skip its own programmatic sends without a brittle text-prefix hack (see `poll_messages` and the Message schema). ### `note_to_self` @@ -208,7 +208,7 @@ Send a text message to a chat. Markdown supported. Send a message to the bot's own Beeper-native Note to self chat. Auto-resolves the correct chat ID, so no `chat_id` parameter needed. **Arguments:** `{text: string, client_tag?: string}`. -**Returns:** `{chat_id, message_id, client_tag, status: "sent"}`. +**Returns:** `{chat_id, message_id, pending_message_id, resolved, client_tag, status: "sent"}` — same shape as `send_message` (`message_id` is the resolved final id when `resolved: true`). **`client_tag`:** same echo-guard semantics as `send_message` — recorded and echoed back as `source: "api"` on read-back, so a `poll_messages` loop can skip the agent's own self-notes. **Use for:** agent self-notes ("processed 5 customer messages"), debug output, scheduled reminders, anything you want recorded but NOT seen by anyone else. The note-to-self chat is excluded from `list_inbox` / `list_unread` / `search_messages`, so messages here will not pollute customer views. @@ -294,7 +294,7 @@ Two normalized shapes. Learn them once and every tool returns the same thing. | `source` | **`"api"`** if *this* beeperbox sent the message via `send_message` / `note_to_self`; **`"external"`** otherwise. This — not `is_self` — is the echo-guard: on one account both the owner's own typed messages and the agent's API replies are `is_self: true`, so only `source` can tell "I sent this programmatically" from "the human owner sent this". Skip `source === "api"` in a poll loop; process `"external"` (the owner's own Note-to-self commands included). | | `client_tag` | The `client_tag` passed to `send_message` / `note_to_self` for this message, echoed back, else `null`. An idempotency key the agent can correlate to a specific send. | -> **`source` matching is best-effort and unverified against a live Beeper account** (CI has none). The primary match is the `pendingMessageID` returned by the send; because Beeper may swap that for a real bridge id on ack, a content-based fallback also matches, but only for the account's own (`is_self`) messages in the same chat within a 15-minute window — deliberately conservative so a human re-typing identical text is never mis-tagged and dropped. The ledger is persisted to the config volume so the guard survives restarts. Validate against your own account before relying on it for a high-stakes auto-responder, and keep a secondary guard if a missed echo would be costly. +> **`source` matching is primarily by exact id; best-effort and unverified against a live Beeper account** (CI has none). On send, beeperbox resolves the `pendingMessageID` to the **final bridge id** Beeper assigns on ack (it GETs the message back until the id swaps) and records both, so a read-back matches by **exact id** whether or not the swap happened. A content-based fallback survives only as a last-ditch net, and **only for sends whose final id could not be resolved** — matching the account's own (`is_self`) messages in the same chat within a 15-minute window. Once a send is resolved, its text fallback is retired, so a human re-typing identical text is **not** mis-tagged as the agent's own and dropped. The ledger is persisted to the config volume so the guard survives restarts. Validate against your own account before relying on it for a high-stakes auto-responder, and keep a secondary guard if a missed echo would be costly. ## Raw `/v1/` contract (for direct-API consumers) diff --git a/docs/GUIDE.md b/docs/GUIDE.md index 4fc43a1..452c759 100644 --- a/docs/GUIDE.md +++ b/docs/GUIDE.md @@ -490,7 +490,7 @@ If you call `/v1/*` directly instead of going through the MCP tools, you inherit - **Group** = `chat.type === 'group'` (and not note-to-self) — there is no `isGroup` field. - **Last activity** = `chat.lastActivity` (camelCase ISO 8601), not `last_activity`. - **Sender-is-self** = `message.isSender === true` — true for anything the account sent from *any* client, so it is *not* an "I sent it via the API" signal (the `source` field the MCP layer adds is a beeperbox-side ledger with no raw-API equivalent). - - **Sent message id** = `POST .../messages` returns `pendingMessageID`, a local pending id Beeper replaces with a real id on bridge ack — don't treat it as a stable delivered id. + - **Sent message id** = `POST .../messages` returns `pendingMessageID`, a local pending id Beeper replaces with a real id on bridge ack — don't treat it as a stable delivered id. To get the final id, GET `.../messages/{pendingMessageID}` until its `id` differs (the swap is async); the MCP layer does this for you and returns `message_id` / `resolved`. ## MCP tools reference @@ -507,7 +507,7 @@ beeperbox exposes 11 semantic tools over Model Context Protocol on two interchan | `get_chat` | `chat_id` | `Chat` | Refresh one chat's state before replying | | `read_chat` | `chat_id` | Array of `Message` (oldest first) | Pull conversation context for the LLM to reason about | | `search_messages` | `query` | Array of `Message` | Follow-up lookups, historical context, "what did X say about Y" | -| `send_message` | `chat_id`, `text` | `{chat_id, message_id, client_tag, status}` | The headline reply/notify tool (optional `client_tag` echo key) | +| `send_message` | `chat_id`, `text` | `{chat_id, message_id, pending_message_id, resolved, client_tag, status}` | The headline reply/notify tool (optional `client_tag` echo key). `message_id` is the resolved final bridge id when `resolved: true`, else the pending id | | `note_to_self` | `text` | same | Agent self-notes, debug output, scheduled reminders — auto-resolves to the Beeper-native Note to self chat (won't leak into per-platform saved-messages chats) | | `react_to_message` | `chat_id`, `message_id`, `emoji` | `{...status: reacted}` | Lightweight ack, no full reply needed | | `archive_chat` | `chat_id` | `{chat_id, archived}` | Clean handled chats out of inbox (closest primitive to mark-as-read that Beeper exposes) | @@ -964,6 +964,8 @@ docker exec beeperbox curl -sf http://127.0.0.1:23373/v1/info > /dev/null && ech - If that prints `API OK`: socat is the problem. Restart the container. - If it prints `API DOWN`: Beeper API isn't running. You probably haven't enabled it yet — go back to [first-run setup](#first-run-setup) step 3, or you forgot to turn on **Start API on launch**. +Once you have logged in and the API has come up at least once, the entrypoint **supervises** beepertexts — if it later crashes (process death, or the launcher lingering with a dead API), it is relaunched/recycled automatically, so the half-dead "MCP answers but every tool call fails" state self-heals within ~60s. Tune or disable it with `BEEPERBOX_SUPERVISE` / `BEEPERBOX_SUPERVISE_INTERVAL` / `BEEPERBOX_SUPERVISE_API_GRACE`. The supervisor never recycles before that first login (the API is down by design until you enable it). + ### `401 Unauthorized` on every call except `/v1/info` Your token is missing or wrong. Confirm: diff --git a/docs/PRD.md b/docs/PRD.md index 2c0c47d..bdb58cb 100644 --- a/docs/PRD.md +++ b/docs/PRD.md @@ -104,6 +104,8 @@ All three publish to `127.0.0.1` by design. Remote access is a deliberate opt-in | `MCP_ALLOWED_HOSTS` | `localhost,127.0.0.1,::1,[::1]` | Host/Origin allowlist (DNS-rebinding defense); set when a reverse proxy fronts a custom hostname. | | `MCP_MAX_BODY` | 1 MiB | Request body cap on the MCP HTTP transport; over-cap ⇒ `413`. | | `BEEPERBOX_SENT_LEDGER` | `/root/.config/beeperbox-sent-ledger.json` | Path to the echo-guard sent-message ledger (inside the persisted config volume). Best-effort; a failed write degrades the `source` guard to in-memory for that run, never fails a send. | +| `BEEPERBOX_RESOLVE_RETRIES` / `_DELAY_MS` / `_TIMEOUT_MS` | `4` / `250` / `3000` | Echo-guard id resolution after a send: attempts to resolve `pendingMessageID` → final bridge id, delay between attempts, and per-attempt fetch timeout. Worst-case added send latency is bounded at `retries × (timeout + delay)`. Set retries `0` to disable resolution (falls back to text matching). | +| `BEEPERBOX_SUPERVISE` / `_SUPERVISE_INTERVAL` / `_SUPERVISE_API_GRACE` | `1` / `10` / `6` | Backend supervision: enable (`0` = old forward-signal-and-wait), seconds between checks, and consecutive API-down checks (after the API has been up once) before recycling beepertexts. Non-numeric values fall back to the default. | | `VNC_PASSWORD` | unset | When set, the noVNC/x11vnc session requires a password (RFB security type *VNC auth*). | | `BEEPER_VERSION` / `BEEPER_SHA256` | unset (build args) | Pin & hash-verify an exact Beeper AppImage for a reproducible build; unset ⇒ rolling auto-update. | @@ -155,6 +157,7 @@ Every chat and message carries both `network` (machine slug: `whatsapp`, `telegr - **One-time login, persistent state.** Named volume holds login + bridge state; survives restart/rebuild/reboot. Bridges configured on an existing Beeper account (e.g. on a phone) inherit automatically — bridge state lives on Beeper's servers. - **Healthcheck.** Docker `HEALTHCHECK` probes the API through the same socat path external clients use, so a crashed API *or* forwarder marks the container unhealthy. Paired with `restart: unless-stopped`. +- **Supervised backend.** The entrypoint supervises beepertexts: it relaunches the process if it dies, and recycles it if the API (`:23373`) stays down after having been up — closing the "half-dead" failure where the launcher lingers after its API process crashes, the MCP layer keeps answering, and every tool call fails. An `API_WAS_UP` gate protects first-run login (the API is down by design until the user enables it). Tunable via `BEEPERBOX_SUPERVISE*` env; the healthcheck surfaces a dead backend, supervision recovers it. - **Clean shutdown.** The entrypoint traps SIGTERM/SIGINT and forwards to Beeper Desktop, waiting for full reap so matrix sync / sqlite checkpoint flush before exit (no partial writes on `docker stop`). - **Restart-survivable display.** `docker restart` preserves the container's writable layer, so the entrypoint clears the stale Xvfb lock (`/tmp/.X99-lock`, `/tmp/.X11-unix/X99`) before starting Xvfb — otherwise the surviving lock makes Xvfb half-initialize `:99` and segfault, wedging the stack until a full recreate. `docker restart` (the natural op after a config change) now recovers cleanly. - **Multi-instance density.** Env-overridable ports + container name let many single-tenant instances share one VPS (density guidance in [`GUIDE.md`](GUIDE.md)). diff --git a/entrypoint.sh b/entrypoint.sh index c19f07d..badc022 100644 --- a/entrypoint.sh +++ b/entrypoint.sh @@ -44,8 +44,11 @@ echo "[..] starting beeper desktop" # do not pass --disable-gpu: recent beeper builds bail in their crash-reporter # init when the gpu process is disabled, leaving a black vnc screen. instead # we ship libgl1-mesa-dri so electron falls back to software gl via mesa. -/opt/beeper/beepertexts --no-sandbox --disable-dev-shm-usage 2>&1 & -BEEPER_PID=$! +launch_beeper() { + /opt/beeper/beepertexts --no-sandbox --disable-dev-shm-usage 2>&1 & + BEEPER_PID=$! +} +launch_beeper sleep 5 # beeper api binds to 127.0.0.1:23373 — forward 0.0.0.0:23380 -> 127.0.0.1:23373 so docker can expose it. @@ -82,9 +85,91 @@ echo " api: http://localhost:23373" # (matrix sync flush, sqlite checkpoint) instead of the 10s SIGKILL fallback. # Bash's `wait` does NOT auto-propagate signals to children, so without this # trap the container exits but Beeper is killed mid-write. -trap 'kill -TERM "$BEEPER_PID" 2>/dev/null' TERM INT -# Loop: bash interrupts `wait` on trap delivery, so a single wait would return -# before Beeper has finished flushing. Re-wait until the PID is actually gone. -while kill -0 "$BEEPER_PID" 2>/dev/null; do - wait "$BEEPER_PID" 2>/dev/null || true +SHUTTING_DOWN=0 +trap 'SHUTTING_DOWN=1; kill -TERM "$BEEPER_PID" 2>/dev/null' TERM INT + +# ─── supervise beeper desktop ───────────────────────────────────── +# beepertexts is the one process whose death silently breaks everything: the +# MCP layer (:23375) and forwarder stay up and keep ANSWERING, but every tool +# call fails because the API (:23373) is gone — a "half-dead" container that +# looks healthy to a human while serving nothing, and never self-heals. Worse, +# the Electron launcher can linger after its API/renderer process crashes, so +# watching the top-level PID alone misses it. We supervise on BOTH signals: +# relaunch if the process dies, AND recycle it if the API stays down after we +# have seen it up. +# +# The API_WAS_UP gate is load-bearing. On first run the user has not enabled +# "Start API on launch" yet, so :23373 is down BY DESIGN until they log in via +# noVNC. Recycling beepertexts every interval in that state would make login +# impossible. So we only ever recycle-for-API-down once the API has come up at +# least once this run; a never-logged-in container just runs Beeper steadily +# and waits for the human — exactly the prior behavior. +SUPERVISE=${BEEPERBOX_SUPERVISE:-1} +SUPERVISE_INTERVAL=${BEEPERBOX_SUPERVISE_INTERVAL:-10} # seconds between checks +SUPERVISE_API_GRACE=${BEEPERBOX_SUPERVISE_API_GRACE:-6} # consecutive API-down checks (after up) before recycle (~60s) +# Sanitize to positive integers. A non-numeric value makes `sleep`/`-ge` fail +# instantly and would spin the loop at 100% CPU (and hammer the API probe); 0 +# is a no-wait spin too. Fall back to the documented default on anything else. +case $SUPERVISE_INTERVAL in ''|*[!0-9]*) SUPERVISE_INTERVAL=10 ;; esac +case $SUPERVISE_API_GRACE in ''|*[!0-9]*) SUPERVISE_API_GRACE=6 ;; esac +[ "$SUPERVISE_INTERVAL" -ge 1 ] 2>/dev/null || SUPERVISE_INTERVAL=10 +[ "$SUPERVISE_API_GRACE" -ge 1 ] 2>/dev/null || SUPERVISE_API_GRACE=6 + +# Block until Beeper has fully exited, re-waiting because a trap interrupts +# `wait` before the child finishes flushing. +wait_beeper() { + while kill -0 "$BEEPER_PID" 2>/dev/null; do + wait "$BEEPER_PID" 2>/dev/null || true + done +} + +if [ "$SUPERVISE" != "1" ]; then + echo "[ok] supervision disabled (BEEPERBOX_SUPERVISE=$SUPERVISE) — waiting on beepertexts" + wait_beeper + exit 0 +fi + +echo "[ok] supervising beepertexts (interval ${SUPERVISE_INTERVAL}s, api-down grace ${SUPERVISE_API_GRACE})" +# NOTE: `set -e` is active. Every failure-prone command in this loop is guarded +# (`curl`/`kill -0` inside `if`, `kill`/`sleep` with `|| true`) so a non-zero +# exit can't terminate supervision. Keep that discipline on any new command. +API_WAS_UP=0 +API_DOWN_STREAK=0 +while true; do + # Clean shutdown requested (docker stop): let Beeper flush, then exit. + if [ "$SHUTTING_DOWN" = "1" ]; then + wait_beeper + break + fi + + # Process gone entirely → relaunch on the existing display (Xvfb still up). + if ! kill -0 "$BEEPER_PID" 2>/dev/null; then + # If a SIGTERM landed in this window, don't spawn a fresh Beeper just to + # immediately stop it — the process is already gone, so exit cleanly. + [ "$SHUTTING_DOWN" = "1" ] && break + echo "[!!] beepertexts exited — relaunching" + launch_beeper + API_DOWN_STREAK=0 + sleep "$SUPERVISE_INTERVAL" || true + continue + fi + + # Process alive: watch the API to catch the half-dead case. + if curl -sf http://127.0.0.1:23373/v1/spec > /dev/null 2>&1; then + [ "$API_WAS_UP" = "0" ] && echo "[ok] beeper api up — half-dead guard armed" + API_WAS_UP=1 + API_DOWN_STREAK=0 + elif [ "$API_WAS_UP" = "1" ]; then + API_DOWN_STREAK=$((API_DOWN_STREAK + 1)) + if [ "$API_DOWN_STREAK" -ge "$SUPERVISE_API_GRACE" ]; then + echo "[!!] beeper api down ${API_DOWN_STREAK} checks but process alive — recycling beepertexts (half-dead guard)" + kill -TERM "$BEEPER_PID" 2>/dev/null || true + for _ in 1 2 3; do kill -0 "$BEEPER_PID" 2>/dev/null || break; sleep 1 || true; done + kill -KILL "$BEEPER_PID" 2>/dev/null || true + launch_beeper + API_DOWN_STREAK=0 + fi + fi + + sleep "$SUPERVISE_INTERVAL" || true done diff --git a/mcp/server.js b/mcp/server.js index 10a49f1..d2990d1 100644 --- a/mcp/server.js +++ b/mcp/server.js @@ -35,6 +35,29 @@ const MCP_ALLOWED_HOSTS = new Set( ); const MCP_MAX_BODY = parseInt(process.env.MCP_MAX_BODY || String(1024 * 1024), 10); +// Echo-guard id resolution. A send returns a `pendingMessageID`, but Beeper +// swaps it for the real bridge id once the message is acked — so the id we'd +// record at send time never matches the id the same message reappears under in +// poll_messages / read_chat. We resolve it: GET the message back until its id +// differs from the pending one, then store the FINAL id so the echo-guard +// matches on exact id instead of falling back to fragile text matching. The +// ack is asynchronous, so this is bounded + best-effort (retry a few times, +// then give up and keep the text fallback as the safety net). Tunable so a +// latency-sensitive deployment can shrink or disable it (RETRIES=0). +// Clamp to a non-negative integer, falling back to the default on a malformed +// value. Without this a typo (e.g. RETRIES="four" → NaN) would silently pass +// the `<= 0` / loop guards and DISABLE resolution — degrading the echo-guard to +// text-only with no error. `0` stays a valid, intentional disable. +function envIntNonNeg(name, def) { + const n = parseInt(process.env[name] || '', 10); + return Number.isFinite(n) && n >= 0 ? n : def; +} +const RESOLVE_RETRIES = envIntNonNeg('BEEPERBOX_RESOLVE_RETRIES', 4); +const RESOLVE_DELAY_MS = envIntNonNeg('BEEPERBOX_RESOLVE_DELAY_MS', 250); +// Per-attempt fetch timeout so a hung Beeper API can't stall a send unbounded: +// worst-case added send latency is RETRIES × (TIMEOUT + DELAY), not infinite. +const RESOLVE_TIMEOUT_MS = envIntNonNeg('BEEPERBOX_RESOLVE_TIMEOUT_MS', 3000); + // Strip the port from a Host header value ("127.0.0.1:23375" -> "127.0.0.1", // "[::1]:23375" -> "[::1]"). function hostFromHeader(value) { @@ -81,12 +104,24 @@ async function beeperFetch(path, opts = {}) { init.headers['Content-Type'] = 'application/json'; init.body = JSON.stringify(opts.body); } - const r = await fetch(`${BEEPER_API}${path}`, init); - if (!r.ok) throw rpcError(-32001, `beeper api ${r.status}: ${(await r.text()).slice(0, 200)}`); - // Some POST/DELETE endpoints return empty body — return null instead of throwing on r.json() - const text = await r.text(); - if (!text) return null; - try { return JSON.parse(text); } catch { return text; } + // Opt-in per-call timeout (opts.timeoutMs). Default callers are unbounded as + // before; the resolve path passes one so a hung API can't stall a send. + let timer = null; + if (opts.timeoutMs) { + const ac = new AbortController(); + init.signal = ac.signal; + timer = setTimeout(() => ac.abort(), opts.timeoutMs); + } + try { + const r = await fetch(`${BEEPER_API}${path}`, init); + if (!r.ok) throw rpcError(-32001, `beeper api ${r.status}: ${(await r.text()).slice(0, 200)}`); + // Some POST/DELETE endpoints return empty body — return null instead of throwing on r.json() + const text = await r.text(); + if (!text) return null; + try { return JSON.parse(text); } catch { return text; } + } finally { + if (timer) clearTimeout(timer); + } } // ─── network normalization ──────────────────────────────────────── @@ -330,13 +365,19 @@ function selectDelivery(fresh, cur, page) { // persisted to the config volume so the guard survives a restart — the // brittle text-prefix guard this replaces was restart-durable too. // -// CAVEAT (unverifiable in CI — no live Beeper account; validate against a -// real account before relying on it): the match key is the pendingMessageID -// returned by the send, but Beeper may swap that for a real bridge id once -// the message is acked. So a content fallback also matches, but ONLY for -// our own (is_self) messages, in the same chat, inside a short window — -// tight enough that a human re-typing identical text is not mis-tagged as -// ours and silently dropped. +// PRIMARY match is exact id. The send only knows the pendingMessageID, which +// Beeper swaps for a real bridge id on ack — so we resolve the final id right +// after sending (resolveSentId) and record BOTH against the entry. A read-back +// then matches by exact id whether or not the swap happened. The content +// fallback survives only as a last-ditch safety net, and ONLY for entries we +// could NOT resolve a final id for: it matches our own (is_self) messages, in +// the same chat, inside a short window. Once an entry IS resolved, exact id is +// authoritative for it and the text fallback is deliberately disabled — that is +// what stops a human re-typing identical text from being mis-tagged as ours. +// +// CAVEAT (unverifiable in CI — no live Beeper account; validate against a real +// account before relying on it): the resolution and content fallback both +// depend on Beeper's live id/ack behavior. const LEDGER_MAX = 500; const LEDGER_TEXT_WINDOW_MS = 15 * 60 * 1000; @@ -348,7 +389,7 @@ function ledgerPath() { return process.env.BEEPERBOX_SENT_LEDGER || '/root/.config/beeperbox-sent-ledger.json'; } -let ledger = null; // lazily loaded array of { chat_id, sent_id, text_hash, client_tag, ts } +let ledger = null; // lazily loaded array of { chat_id, sent_ids[], resolved, text_hash, client_tag, ts } function textHash(text) { return crypto.createHash('sha256') @@ -384,31 +425,87 @@ function persistLedger() { function recordSent({ chat_id, sent_id, text, client_tag }) { loadLedger(); - ledger.push({ + const entry = { chat_id, - sent_id: sent_id ? String(sent_id) : null, + // Carries every id this send is known by — the pendingMessageID now, the + // resolved bridge id once addResolvedId() lands it. `resolved` flips true + // when we have the authoritative final id (retires the text fallback). + sent_ids: sent_id ? [String(sent_id)] : [], + resolved: false, text_hash: textHash(text), client_tag: client_tag ? String(client_tag).slice(0, CLIENT_TAG_MAX) : null, ts: Date.now(), - }); + }; + ledger.push(entry); if (ledger.length > LEDGER_MAX) ledger = ledger.slice(-LEDGER_MAX); persistLedger(); + return entry; // so the caller can attach the resolved id (addResolvedId) +} + +// Attach the resolved final bridge id to a ledger entry and mark it exact-id +// authoritative, then persist so the resolution survives a restart. Idempotent. +function addResolvedId(entry, finalID) { + if (!entry || !finalID) return; + const s = String(finalID); + if (!entry.sent_ids) entry.sent_ids = []; + if (!entry.sent_ids.includes(s)) entry.sent_ids.push(s); + entry.resolved = true; + persistLedger(); +} + +// Best-effort resolve a pendingMessageID to the final bridge id Beeper assigns +// on ack. The ack is asynchronous, so GET the message back a few times until +// its id differs from the pending one. Returns the final id, or null if it +// never swapped within the retry budget (caller keeps the pending id + text +// fallback). Never throws — a degraded echo-guard must never fail a send. +async function resolveSentId(chatID, pendingMessageID) { + const pend = pendingMessageID ? String(pendingMessageID) : ''; + if (!pend || !chatID || RESOLVE_RETRIES <= 0) return null; + for (let attempt = 0; attempt < RESOLVE_RETRIES; attempt++) { + try { + const m = await beeperFetch( + `/v1/chats/${encodeURIComponent(chatID)}/messages/${encodeURIComponent(pend)}`, + { timeoutMs: RESOLVE_TIMEOUT_MS }, + ); + // GET is by id → a single message object. Only trust that shape: a + // list/other response would let us read an arbitrary message's id and + // store the WRONG final id, which would mis-tag an unrelated read-back. + const id = m && m.id != null ? String(m.id) : null; + if (id && id !== pend) return id; // swapped to the real bridge id + } catch { /* not acked yet, timed out, or transient — retry */ } + if (attempt < RESOLVE_RETRIES - 1 && RESOLVE_DELAY_MS > 0) { + await new Promise((r) => setTimeout(r, RESOLVE_DELAY_MS)); + } + } + return null; } // Pure: decide api-vs-external origin + echoed client_tag for one message // given the ledger entries and a `now`. Exact id match first (high // confidence); content fallback only for our own recent same-chat sends. function matchSentMessage(msg, entries, now) { + const mid = String(msg.id); + // 1. Exact id match — high confidence. Checks every id the send is known by: + // the pendingMessageID and (once resolved) the final bridge id, so a + // read-back matches whether or not Beeper swapped the id on ack. Tolerates + // the legacy single-`sent_id` entry shape for ledgers written pre-upgrade. for (const e of entries) { if (e.chat_id !== msg.chat_id) continue; - if (e.sent_id && String(msg.id) === e.sent_id) { - return { source: 'api', client_tag: e.client_tag || null }; + const ids = e.sent_ids || (e.sent_id ? [e.sent_id] : []); + for (const id of ids) { + if (id && String(id) === mid) return { source: 'api', client_tag: e.client_tag || null }; } } + // 2. Last-ditch text fallback — ONLY for our own (is_self) messages, same + // chat, recent window, AND only against entries whose final id we could + // NOT resolve (resolved !== true). Once an entry is resolved, step 1 is + // authoritative for it; skipping its text match is what keeps a human + // re-typing identical text from being mis-tagged as ours and dropped. if (msg.is_self) { const h = textHash(msg.text); for (const e of entries) { if (e.chat_id !== msg.chat_id) continue; + if (e.resolved === true) continue; if (e.text_hash === h && (now - e.ts) <= LEDGER_TEXT_WINDOW_MS) { return { source: 'api', client_tag: e.client_tag || null }; } @@ -668,13 +765,19 @@ async function callTool(name, args) { `/v1/chats/${encodeURIComponent(chatID)}/messages`, { method: 'POST', body: { text: args.text } }, ); - const messageID = String(sent?.pendingMessageID || ''); + const pendingID = String(sent?.pendingMessageID || ''); // Echo-guard: record so poll_messages / read_chat can tag this // self-note source:'api' and the agent won't re-process its own note. - recordSent({ chat_id: chatID, sent_id: messageID, text: args.text, client_tag: args.client_tag }); + const entry = recordSent({ chat_id: chatID, sent_id: pendingID, text: args.text, client_tag: args.client_tag }); + // Resolve the pending id to the final bridge id so the guard matches the + // read-back by exact id, not fragile text. Best-effort — null on timeout. + const finalID = await resolveSentId(chatID, pendingID); + if (finalID) addResolvedId(entry, finalID); return { chat_id: chatID, - message_id: messageID, + message_id: finalID || pendingID, + pending_message_id: pendingID, + resolved: !!finalID, client_tag: args.client_tag || null, status: 'sent', }; @@ -690,13 +793,19 @@ async function callTool(name, args) { { method: 'POST', body }, ); const chatID = sent?.chatID || args.chat_id; - const messageID = String(sent?.pendingMessageID || ''); + const pendingID = String(sent?.pendingMessageID || ''); // Echo-guard: record so a poll_messages loop can skip the bot's own // reply (source:'api') instead of treating it as a new inbound message. - recordSent({ chat_id: chatID, sent_id: messageID, text: args.text, client_tag: args.client_tag }); + const entry = recordSent({ chat_id: chatID, sent_id: pendingID, text: args.text, client_tag: args.client_tag }); + // Resolve the pending id to the final bridge id so the guard matches the + // read-back by exact id, not fragile text. Best-effort — null on timeout. + const finalID = await resolveSentId(chatID, pendingID); + if (finalID) addResolvedId(entry, finalID); return { chat_id: chatID, - message_id: messageID, + message_id: finalID || pendingID, + pending_message_id: pendingID, + resolved: !!finalID, client_tag: args.client_tag || null, status: 'sent', }; @@ -972,6 +1081,7 @@ module.exports = { textHash, matchSentMessage, recordSent, + addResolvedId, loadLedger, // test hook: drop the in-memory ledger so a test can re-load from a fresh path _resetLedger: () => { ledger = null; ledgerPersistWarned = false; }, diff --git a/mcp/server.test.js b/mcp/server.test.js index 7280633..38e1aae 100644 --- a/mcp/server.test.js +++ b/mcp/server.test.js @@ -193,6 +193,83 @@ test('matchSentMessage: no match is external with no tag', () => { assert.deepEqual(got, { source: 'external', client_tag: null }); }); +// ── Ask A: reliable echo via resolved bridge id ─────────────────── +test('matchSentMessage: a read-back is tagged by its RESOLVED final bridge id', () => { + // After send, pendingMessageID 'p1' was resolved to final id 'b908'; the + // ledger holds both. The message reappears under the final id → exact match. + const entries = [{ chat_id: 'c1', sent_ids: ['p1', 'b908'], resolved: true, text_hash: S.textHash('hey'), client_tag: 'job-9', ts: NOW }]; + const got = S.matchSentMessage({ id: 'b908', chat_id: 'c1', text: 'hey', is_self: true }, entries, NOW); + assert.deepEqual(got, { source: 'api', client_tag: 'job-9' }); +}); + +test('matchSentMessage: once resolved, a human re-typing identical text is NOT mis-tagged (Ask A acceptance)', () => { + // The load-bearing acceptance criterion. The bot sent "on my way" (resolved + // to final id b908). The human owner later types the exact same words. Exact + // id is authoritative for the resolved entry, so its text fallback is OFF — + // the human message stays external instead of being swallowed as our echo. + const entries = [{ chat_id: 'c1', sent_ids: ['p1', 'b908'], resolved: true, text_hash: S.textHash('on my way'), client_tag: null, ts: NOW }]; + assert.equal(S.matchSentMessage({ id: 'b908', chat_id: 'c1', text: 'on my way', is_self: true }, entries, NOW).source, 'api'); + assert.equal(S.matchSentMessage({ id: 'human1', chat_id: 'c1', text: 'on my way', is_self: true }, entries, NOW).source, 'external'); +}); + +test('matchSentMessage: two identical-text sends are each matched by their OWN resolved id', () => { + // Identical text, two separate sends — exact-id keeps them distinct so each + // read-back echoes its own client_tag (text matching alone could not). + const entries = [ + { chat_id: 'c1', sent_ids: ['pA', 'bA'], resolved: true, text_hash: S.textHash('ok'), client_tag: 'a', ts: NOW }, + { chat_id: 'c1', sent_ids: ['pB', 'bB'], resolved: true, text_hash: S.textHash('ok'), client_tag: 'b', ts: NOW }, + ]; + assert.deepEqual(S.matchSentMessage({ id: 'bA', chat_id: 'c1', text: 'ok', is_self: true }, entries, NOW), { source: 'api', client_tag: 'a' }); + assert.deepEqual(S.matchSentMessage({ id: 'bB', chat_id: 'c1', text: 'ok', is_self: true }, entries, NOW), { source: 'api', client_tag: 'b' }); +}); + +test('matchSentMessage: a legacy {sent_id} entry (pre-upgrade ledger) still id-matches AND text-falls-back', () => { + // Back-compat: ledgers written before this change have `sent_id` (no + // `sent_ids[]`, no `resolved`). Both echo paths must still work for them. + const entries = [{ chat_id: 'c1', sent_id: 'legacyP', text_hash: S.textHash('hi there'), client_tag: 'old', ts: NOW }]; + // exact id via the legacy single field + assert.deepEqual(S.matchSentMessage({ id: 'legacyP', chat_id: 'c1', text: 'hi there', is_self: true }, entries, NOW), { source: 'api', client_tag: 'old' }); + // id swapped & never resolved (resolved is undefined → not true) → text fallback still fires + assert.equal(S.matchSentMessage({ id: 'swapped', chat_id: 'c1', text: 'hi there', is_self: true }, entries, NOW).source, 'api'); +}); + +test('matchSentMessage: an UNRESOLVED entry keeps the text fallback as a safety net', () => { + // Resolution failed (slow ack) so we only have the pending id and the read- + // back uses a swapped id we never learned. The text fallback (resolved:false) + // still catches our own send — preserving today's behavior when resolution + // is unavailable, which is exactly when the net is needed. + const entries = [{ chat_id: 'c1', sent_ids: ['pending-only'], resolved: false, text_hash: S.textHash('ping'), client_tag: 'j1', ts: NOW }]; + const got = S.matchSentMessage({ id: 'swapped-unknown', chat_id: 'c1', text: 'ping', is_self: true }, entries, NOW); + assert.deepEqual(got, { source: 'api', client_tag: 'j1' }); +}); + +test('addResolvedId attaches the final id, retires the text fallback, and persists', () => { + const file = path.join(os.tmpdir(), `bb-resolve-test-${process.pid}.json`); + try { fs.unlinkSync(file); } catch { /* not there */ } + process.env.BEEPERBOX_SENT_LEDGER = file; + try { + S._resetLedger(); + const entry = S.recordSent({ chat_id: 'c1', sent_id: 'pending1', text: 'hi', client_tag: 't' }); + assert.equal(entry.resolved, false); + S.addResolvedId(entry, 'final1'); + + // Matches by the resolved id AND still by the original pending id (covers a + // platform/read path that never swapped). + assert.equal(S.matchSentMessage({ id: 'final1', chat_id: 'c1', text: 'hi', is_self: true }, S.loadLedger(), Date.now()).source, 'api'); + assert.equal(S.matchSentMessage({ id: 'pending1', chat_id: 'c1', text: 'hi', is_self: true }, S.loadLedger(), Date.now()).source, 'api'); + + // Reload from disk → the resolution survived a "restart". + S._resetLedger(); + const reloaded = S.loadLedger(); + assert.equal(reloaded[0].resolved, true); + assert.deepEqual([...reloaded[0].sent_ids].sort(), ['final1', 'pending1']); + } finally { + try { fs.unlinkSync(file); } catch { /* ignore */ } + delete process.env.BEEPERBOX_SENT_LEDGER; + S._resetLedger(); + } +}); + test('recordSent caps an oversized client_tag (ledger-bloat guard)', () => { const file = path.join(os.tmpdir(), `bb-cap-test-${process.pid}.json`); try { fs.unlinkSync(file); } catch { /* not there */ }