feat(uploads): ticket-gated uploads without base64 through the model (supersedes #34)

.env.example

@@ -27,6 +27,11 @@ MCP_AUTH_TOKEN=
 #
 # This server's public URL — used as the token audience / Resource Indicator.
 # Required in OAuth mode (SERVER_URL is accepted as an alias).
+#
+# It ALSO applies outside OAuth mode: create-upload-ticket builds its browser URL and
+# its curl command from this value in every auth mode. If you run with MCP_AUTH_TOKEN
+# (or unauthenticated) behind a real domain, set it here — otherwise those links fall
+# back to http://127.0.0.1:$PORT, which resolves nowhere but on the server itself.
 # OAUTH_RESOURCE=https://your-server.example.com
 # SERVER_URL=https://your-server.example.com
 #

CHANGELOG.md

@@ -4,6 +4,70 @@ All notable changes to this project are documented here. The format is based on
 [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project
 adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [0.1.11]
+
+Upload a receipt without pushing its bytes through the model context. Based on the
+contribution by [@gutencoder](https://github.com/gutencoder) ([#34]); the server-side
+URL-fetch tool from that PR was intentionally held back (see **Security** below).
+
+### Added
+- **`create-upload-ticket` / `get-upload-result` (drafts tier).** The existing `upload-file` /
+  `upload-voucher-file` tools take the file as base64 inline in the JSON-RPC body, so every byte is
+  billed as tokens, sits in the conversation transcript, and a ~8 MB receipt runs into the 12 MB body
+  limit — the file travels through the model even though the model only needs the resulting file id.
+  `create-upload-ticket` issues a short-lived (15 min), single-use ticket and returns a browser URL for
+  drag-and-drop plus a ready-to-run `curl` command; the bytes go client → server → Lexware and the
+  model only ever sees the file id. `get-upload-result` reads that id back after a browser upload (the
+  `curl` path prints it directly). `filename` / `mimeType` travel as `X-Filename-B64` (base64url of the
+  UTF-8 bytes), so names with an en dash, typographic quotes or an emoji survive a header layer that is
+  Latin-1 on the wire.
+- **`SERVER_URL` (or `OAUTH_RESOURCE`) now applies in every auth mode**, not only OAuth. It is the
+  server's public URL, and `create-upload-ticket` builds its browser link and `curl` command from it;
+  a static-token deployment behind a real domain previously had it ignored and handed out loopback
+  links. Unset, the loopback fallback still applies, on the port actually bound (`__PORT` under
+  `skybridge dev`, else `PORT`).
+
+### Changed
+- The existing base64 upload tools are unchanged and remain available — the ticket route is additive.
+- **Body parsing now also defers `/upload` paths** from the pre-applied global JSON parser, alongside
+  `/mcp`: the upload route reads the raw body itself, and letting the JSON parser run first turned a
+  JSON-content-typed upload into an empty file. The route additionally rejects gzip framing
+  (`inflate: false`), rejects an invalid/expired/used ticket before reading any body, buffers at most
+  one request body per ticket at a time, and holds a synchronous single-use lock across the upload so
+  **concurrent or duplicated requests** cannot file a second voucher while an attempt is in flight.
+  One case is deliberately weaker: after a transport failure whose outcome is unknown (the upload may
+  or may not have reached Lexware), the ticket is released so a retry stays possible, and the error
+  says to check for the file before re-uploading — blind retries after such a failure can still
+  duplicate a receipt, which no client-side lock can prevent without upstream idempotency support.
+
+A post-integration review pass hardened the details: the upload result stays readable via
+`get-upload-result` for a full 15 minutes **after the upload completed** (previously it expired on the
+ticket's creation-time clock, so a minute-14 upload left a sub-minute read window and invited a
+duplicate); the generated `curl` command pins `Content-Type` explicitly (curl's `--data-binary`
+otherwise silently declared `application/x-www-form-urlencoded` and bypassed the documented fallback
+chain); a 401/403 from Lexware — the operator's API key being rejected — is answered as a generic 502
+instead of forwarding the upstream status and wording to the unauthenticated uploader;
+`get-upload-result` is annotated read-only so polling it doesn't trigger write-tool confirmations; the
+loopback link fallback follows `__PORT` under `skybridge dev`; and `OAUTH_RESOURCE` outside OAuth
+mode still takes precedence over `SERVER_URL` for upload links, but now announces itself with a
+startup warning instead of doing so silently.
+
+The ticket store is **in-process**, so this is a single-instance feature: a restart drops open tickets
+(they answer `410`, they do not hang), and behind a load balancer without sticky sessions an upload can
+reach a different instance than the one that issued the ticket. The 15-minute lifetime bounds the
+window. The upload route is mounted **only when the drafts capability is enabled** — a read-only
+deployment never exposes it — and the ticket page is served `Cache-Control: no-store`.
+
+### Security
+- **The server-side URL-fetch tool (`upload-file-from-url`) from #34 was deliberately NOT included.**
+  A server-side fetcher is SSRF surface by construction; the version in #34 is guarded by a host
+  allow-list and per-hop private-address checks but carries a DNS-rebinding TOCTOU — the resolved
+  address is validated, then the connection re-resolves independently — which is moot for the built-in
+  Microsoft defaults but live for any custom allow-list. It will be reconsidered separately, with
+  connection-level IP pinning and disabled by default. The ticket flow above carries no such surface.
+
+[#34]: https://github.com/marselsel/lexware-mcp/pull/34
+
 ## [0.1.10]
 
 ### Fixed

README.md

@@ -28,12 +28,12 @@ Related projects — local (stdio) Lexware MCP servers:
 
 ## Capabilities
 
-60 tools across three tiers you enable via environment variables:
+62 tools across three tiers you enable via environment variables:
 
 | Tier | Default | What it covers |
 |------|---------|----------------|
 | **Read** | always on | Profile; contacts & articles (list/get); the voucherlist (plus `summarize-vouchers` for server-side totals); full documents (invoices, quotations, credit notes, order confirmations, delivery notes, dunnings, down-payment invoices, vouchers); **render any document type to PDF** and **download files/receipts** (returned inline as embedded resources); batch & type-dispatched reads (get-vouchers, get-document, get-voucher-file, get-document-file); payments; reference data (countries, payment conditions, posting categories, print layouts); recurring templates (get & list); event subscriptions; document deeplinks |
-| **Drafts/writes** (`LEXWARE_ENABLE_DRAFTS`) | on | Create **draft** invoices/quotations/credit-notes/order-confirmations/delivery-notes/dunnings (the Lexware API has no update endpoint for these — set every field, including payment terms, at creation); create & update contacts, articles, and **bookkeeping vouchers**; **upload files** and **attach receipts** to vouchers; create documents as **follow-ups** (`precedingSalesVoucherId`) |
+| **Drafts/writes** (`LEXWARE_ENABLE_DRAFTS`) | on | Create **draft** invoices/quotations/credit-notes/order-confirmations/delivery-notes/dunnings (the Lexware API has no update endpoint for these — set every field, including payment terms, at creation); create & update contacts, articles, and **bookkeeping vouchers**; **upload files** and **attach receipts** to vouchers — inline as base64, or **without base64** via a short-lived upload ticket (`create-upload-ticket` → browser drag-and-drop or a `curl` one-liner → `get-upload-result`); create documents as **follow-ups** (`precedingSalesVoucherId`) |
 | **Finalize** (`LEXWARE_ENABLE_FINALIZE`) | off | Issue **legally binding** finalized documents in one step via the dedicated `create-finalized-*` tools (confirmation-gated); irreversible article deletes; **manage webhook event subscriptions** (create + delete — a webhook streams financial events to an external URL, so it's opt-in). Enabling this tier also enables Drafts. |
 
 Set `LEXWARE_READ_ONLY=true` to force read-only (overrides the flags above).
@@ -94,7 +94,7 @@ LEXWARE_API_KEY=... MCP_AUTH_TOKEN=... npm start
 |---|---|---|
 | `LEXWARE_API_KEY` | — (**required**) | Your Lexware API key ([create one](https://app.lexware.de/addons/public-api)) |
 | `OAUTH_ISSUER` | — | OAuth authorization-server issuer URL. Setting it enables OAuth mode¹ |
-| `OAUTH_RESOURCE` / `SERVER_URL` | — | This server's public URL (token audience / Resource Indicator). Required in OAuth mode |
+| `OAUTH_RESOURCE` / `SERVER_URL` | `http://127.0.0.1:$PORT` | This server's public URL. **Required in OAuth mode** (token audience / Resource Indicator), and used in *every* mode to build the upload links `create-upload-ticket` hands out (browser URL and `curl` command). Set it whenever the server is reachable under a real domain — without it those links point at the loopback fallback, which only works on the server itself |
 | `OAUTH_ALLOWED_EMAIL_DOMAINS` | — | Comma-separated allow-list of email domains (e.g. `example.com`) |
 | `OAUTH_VERIFY_AUDIENCE` | `true` | Verify the token `aud` matches `OAUTH_RESOURCE`. **Keep `true`.** Setting `false` accepts *any* valid token from the issuer — including one minted for a different app on the same issuer (a confused-deputy risk). Only disable for a dedicated, single-audience issuer that has no Resource Indicator |
 | `OAUTH_AUDIENCE` | — | Comma-separated **additional** accepted `aud` values, on top of `OAUTH_RESOURCE`. For IdPs that ignore the Resource Indicator: Microsoft Entra always puts the API's client ID (a GUID) in `aud`, never the Application ID URI, so without this every token is rejected. Prefer this over `OAUTH_VERIFY_AUDIENCE=false` — the check stays on, just against a value your IdP actually issues. Values are matched **exactly**: they are opaque identifiers, so no normalisation is applied (unlike `OAUTH_RESOURCE`, which also accepts its trailing-slash form) |

package.json

@@ -1,6 +1,6 @@
 {
   "name": "lexware-mcp",
-  "version": "0.1.10",
+  "version": "0.1.11",
   "private": false,
   "license": "MIT",
   "description": "Open-source, self-hostable MCP server for the Lexware Office API",

src/config.ts

@@ -87,6 +87,17 @@ export interface Config {
   /** Web-app base for building document deeplinks, e.g. `https://app.lexware.de`. */
   lexwareAppBaseUrl: string;
   auth: AuthConfig;
+  /**
+   * This server's public base URL, used to build the `/upload/:ticket` links that
+   * `create-upload-ticket` hands to a browser and bakes into its curl command.
+   *
+   * Deliberately NOT part of {@link AuthConfig}: where this server is reachable is a
+   * deployment fact, not an auth one. Deriving it from the auth mode — OAuth resource
+   * or else loopback — meant a static-token deployment behind a real domain (a
+   * supported mode, see README) handed out `http://127.0.0.1:8080/upload/…`, a link
+   * that resolves nowhere but inside the container.
+   */
+  publicBaseUrl: string;
   port: number;
   debugLogging: boolean;
   capabilities: Capabilities;
@@ -166,6 +177,37 @@ function validateIssuerUrl(raw: string): string {
   return value;
 }
 
+/**
+ * Resolve this server's public base URL — in EVERY auth mode, not just OAuth.
+ *
+ * `OAUTH_RESOURCE` first, so an OAuth deployment can never drift from the value its
+ * token audience is built from; then `SERVER_URL`, which the README already documents
+ * as "this server's public URL" and which was previously read only inside the OAuth
+ * branch (a static-token or unauthenticated deployment set it and it was ignored).
+ * Only with neither set does the loopback fallback apply: correct for a local run,
+ * and honest about being unusable anywhere else.
+ *
+ * Validated through {@link normalizeUrl} like every other configured URL, so a typo
+ * or a plain-http public URL fails at startup rather than being pasted into a curl
+ * command an operator then runs.
+ */
+function resolvePublicBaseUrl(env: NodeJS.ProcessEnv, port: number): string {
+  const resource = env.OAUTH_RESOURCE?.trim();
+  if (resource) return normalizeUrl(resource, resource, "OAUTH_RESOURCE");
+  const serverUrl = env.SERVER_URL?.trim();
+  if (serverUrl) return normalizeUrl(serverUrl, serverUrl, "SERVER_URL");
+  // The loopback fallback must name the port the server actually LISTENS on.
+  // Under `skybridge dev` that is `__PORT` — skybridge picks it itself (~3000)
+  // and plain PORT is never consulted; using `port` (default 8080) there handed
+  // out links refusing connections on the very machine the fallback exists for.
+  // `npm start` is unaffected: server.ts copies config.port into __PORT only
+  // AFTER config is loaded, so __PORT is present here only when something else
+  // (skybridge dev, or an operator) chose the bound port explicitly.
+  const bound = env.__PORT?.trim();
+  const boundPort = bound && /^\d+$/.test(bound) ? Number(bound) : NaN;
+  return `http://127.0.0.1:${boundPort >= 1 && boundPort <= 65535 ? boundPort : port}`;
+}
+
 /** Resolve how `/mcp` is authenticated, failing closed if nothing is configured. */
 function resolveAuth(env: NodeJS.ProcessEnv): AuthConfig {
   const issuerRaw = env.OAUTH_ISSUER?.trim();
@@ -286,6 +328,7 @@ export function loadConfig(env: NodeJS.ProcessEnv = process.env): Config {
   }
 
   const auth = resolveAuth(env);
+  const port = parsePort(env.PORT);
 
   const readOnly = parseBool(env.LEXWARE_READ_ONLY, false);
   // READ_ONLY is a hard override: it wins over the individual enable flags.
@@ -303,6 +346,19 @@ export function loadConfig(env: NodeJS.ProcessEnv = process.env): Config {
         "finalize tier issues binding versions of draft documents and cannot run without the drafts tier.",
     );
   }
+  // Outside OAuth mode, OAUTH_RESOURCE has exactly one remaining effect — it wins
+  // over SERVER_URL as the base for upload links (resolvePublicBaseUrl). That is
+  // easy to hit by accident: migrate from OAuth to a static token, remove
+  // OAUTH_ISSUER, update SERVER_URL — and a stale OAUTH_RESOURCE left in the
+  // environment silently keeps every ticket link pointing at the old host, with
+  // nothing anywhere saying why. Say so at startup.
+  if (auth.mode !== "oauth" && env.OAUTH_RESOURCE?.trim()) {
+    warnings.push(
+      "OAUTH_RESOURCE is set but OAuth mode is not enabled (no OAUTH_ISSUER). It still takes precedence " +
+        "over SERVER_URL when building upload links — if that is stale, links point at the wrong host. " +
+        "Unset OAUTH_RESOURCE or use SERVER_URL alone outside OAuth mode.",
+    );
+  }
 
   return {
     lexwareApiKey,
@@ -313,7 +369,8 @@ export function loadConfig(env: NodeJS.ProcessEnv = process.env): Config {
       "LEXWARE_APP_BASE_URL",
     ),
     auth,
-    port: parsePort(env.PORT),
+    publicBaseUrl: resolvePublicBaseUrl(env, port),
+    port,
     debugLogging: parseBool(env.LEXWARE_DEBUG_LOGGING, false),
     capabilities: { read: true, drafts: enableDrafts, finalize: enableFinalize },
     warnings,

src/server-body-parsing.ts

@@ -0,0 +1,71 @@
+import express from "express";
+
+/**
+ * Case-insensitive prefix match on a URL path segment. Express's own routing
+ * is case-insensitive by default (`app.set("case sensitive routing", ...)` is
+ * off unless explicitly enabled, and this project never enables it) — so
+ * `POST /UPLOAD/<ticket>` and `POST /Mcp` really do reach the same handlers as
+ * their lowercase spellings. An earlier version of `isMcpPath`/`isUploadPath`
+ * compared case-sensitively, which meant an uppercase path was routed to the
+ * real handler but NOT recognized by the body-parsing swap below — for
+ * `/UPLOAD`, that reopened the gzip-amplification path (Critical 2) up to the
+ * ~100 KB global-parser limit, since the pre-check + raw-body defenses in
+ * routes.ts never got a chance to run before the global JSON parser did.
+ */
+function matchesPathCaseInsensitive(p: string, exact: string, prefix: string): boolean {
+  const lower = p.toLowerCase();
+  return lower === exact || lower.startsWith(prefix);
+}
+
+/** MCP protocol traffic. Bodies are parsed by a raised-limit parser mounted AFTER the auth gate (see server.ts). */
+export const isMcpPath = (p: string): boolean => matchesPathCaseInsensitive(p, "/mcp", "/mcp/");
+
+/**
+ * Ticket-gated upload endpoints (`registerUploadRoutes`). These read the request
+ * body themselves via `express.raw()` and must never be pre-parsed by the global
+ * JSON layer — if that layer ran first, `req.body` would already be a parsed
+ * object (not a `Buffer`) for any `Content-Type: application/json` upload, and a
+ * naive length guard would treat that as an empty-but-successful upload while
+ * still consuming the ticket.
+ */
+export const isUploadPath = (p: string): boolean => matchesPathCaseInsensitive(p, "/upload", "/upload/");
+
+/**
+ * Reconfigure body parsing so large uploads (and the raw-body ticket routes) work
+ * WITHOUT widening the pre-auth attack surface. Skybridge pre-applies a single
+ * global `express.json()` (~100 KB default) at router-stack index 0 — before the
+ * `/mcp` auth middleware AND before the `/upload` ticket routes' own
+ * `express.raw()`. We swap that layer's handler, in place, so it keeps the
+ * ~100 KB limit for ordinary routes (e.g. `/status`) but calls `next()`
+ * immediately — without touching `req.body` at all — for any path `skipPath`
+ * accepts.
+ *
+ * In-place handler swap (no stack reordering) so it can't mis-order routes.
+ * Guarded: returns `false` if the internal layer can't be located, and the
+ * caller must treat that as "the swap did not happen" (server.ts warns loudly;
+ * routes.ts's own `Buffer.isBuffer` guard is the defense-in-depth backstop for
+ * exactly this case).
+ *
+ * Exported (rather than kept private in server.ts) so the upload routes' tests
+ * can exercise this exact function against a test app shaped like the real
+ * stack, instead of a parallel reimplementation that could silently drift from
+ * production — which is precisely how the original `/upload` JSON-body bug
+ * stayed invisible: the test app never had a global JSON parser to begin with.
+ */
+export function deferBodyParsingFor(app: express.Express, skipPath: (path: string) => boolean): boolean {
+  try {
+    type Layer = { handle?: express.RequestHandler & { name?: string } };
+    const router =
+      (app as unknown as { router?: { stack: Layer[] }; _router?: { stack: Layer[] } }).router ??
+      (app as unknown as { _router?: { stack: Layer[] } })._router;
+    const stack = router?.stack;
+    if (!Array.isArray(stack)) return false;
+    const layer = stack.find((l) => l?.handle?.name === "jsonParser");
+    if (!layer) return false;
+    const smallJson = express.json(); // ~100 KB default — for /status and other ordinary routes
+    layer.handle = (req, res, next) => (skipPath(req.path) ? next() : smallJson(req, res, next));
+    return true;
+  } catch {
+    return false;
+  }
+}