Per-IP connection limit caps relay at 10 connections behind StartOS proxy

[wksantiago]

Summary

Behind the StartOS reverse proxy, Wisp sees every client as a single source IP (the proxy's internal address). With Wisp's default max_connections_per_ip = 10, the entire relay is effectively capped at 10 concurrent WebSocket connections internet-wide. A public relay saturates this instantly and then rejects every new connection with TooManyConnectionsPerIp, never recovering.

This is a packaging/config problem in wisp-startos, not a bug in Wisp. Wisp's defaults (trust_proxy = false, max_connections_per_ip = 10) are reasonable for a relay on its own IP — they're just wrong when the relay is fronted by the StartOS proxy, and that wrapping is defined here.

Root cause

• startos/interfaces.ts binds port 7777 as a proxied ws interface — all client traffic arrives via the platform proxy and is collapsed to one source IP.
• Wisp only honors X-Forwarded-For when trust_proxy = true (Wisp src/server.zig). The package never sets trust_proxy, so it stays undefined → false.
• The per-IP limiter (src/server.zig, WsConn.init) buckets every connection under the proxy IP and rejects past 10.

The trust_proxy field already exists in the schema (startos/fileModels/wisp.toml.ts:69) but is never written by any action.

Immediate workaround (no rebuild)

StartOS UI → Wisp → Actions → Configure Limits → Max Connections Per IP → set high (>= Max Connections, e.g. 1000). Save + restart. Global Max Connections still provides DoS protection.

Proper fix (in this repo)

Two options:

1. Correct — if the StartOS ws proxy injects X-Forwarded-For: have the package write [security] trust_proxy = true plus trusted_proxies = <proxy address> into wisp.toml. Wisp then reads the real client IP and per-IP limiting works as intended. Wire-up point: startos/init/ (seed defaults) and/or the limits/access action.
2. Safe fallback — if the proxy does not forward XFF: raise the package default max_connections_per_ip to a large value (startos/actions/limits.ts, possibly startos/fileModels/wisp.toml.ts) and rely on global max_connections, since real client IPs aren't recoverable.

TODO

• Confirm whether the StartOS ws binding appends X-Forwarded-For for a raw ws proxy (start-sdk / StartOS proxy behavior).
• If yes → implement option 1. If no → implement option 2.

[wksantiago]

Resolved: StartOS proxy injects X-Forwarded-For → Option 1 implemented

XFF question settled (confirmed in source):

• @start9labs/start-sdk 1.5.3 base/lib/interfaces/Host.js — knownProtocols.ws and .wss both set addXForwardedHeaders: true. ws has withSsl: 'wss', so getSslProto() resolves to wss and the bind's addSsl.addXForwardedHeaders is true.
• base/lib/osBindings/AddSslOptions.d.ts documents that the OS reverse proxy adds X-Forwarded-Proto: https and X-Forwarded-For: <client-ip> to incoming HTTP requests before forwarding upstream. The WebSocket upgrade is an HTTP request, so the upgrade carries the real client IP.
• startos/interfaces.ts binds relayPort with protocol: 'ws' → covered.

Wisp semantics settled (wisp v0.5.0 @ fa14e1f):

• src/config.zig:46-49 — trusted_proxies doc: "Comma-separated IPs/prefixes of reverse proxies whose forwarded headers are honored. Empty with trust_proxy=true honors headers from any peer."
• src/server.zig:144-153 clientIp() — if (!trust_proxy) return socket_ip; then if (!trusted_proxies.isTrustedProxy(socket_ip)) return socket_ip; then reads x-forwarded-for.

So trust_proxy = true with an empty trusted_proxies makes Wisp honor X-Forwarded-For from any peer — correct and safe here because the relay port is only reachable through the StartOS proxy. Setting trusted_proxies to a wrong address would silently disable the fix (isTrustedProxy fails → falls back to the socket IP), so it is intentionally left empty.

Patch (Option 1): seed [security] trust_proxy = true in startos/init/seedFiles.ts (runs on every init, so it applies on install and upgrade; deep-merge preserves user-set limits). tsc --noEmit passes.

export const seedFiles = sdk.setupOnInit(async (effects) => {
  await wispToml.merge(effects, {
    security: { trust_proxy: true },
  })
})

The trust_proxy field already existed in startos/fileModels/wisp.toml.ts:69; it was just never written. PR to follow.

[wksantiago]

Security review resolved — patch stands (trust_proxy=true, trusted_proxies empty)

Ran a security review of the trust_proxy=true seed. The two High findings were both about X-Forwarded-For spoofing under empty trusted_proxies. Verified against upstream source + Start9 docs, both are mitigated:

1. wisp reads the rightmost XFF hop, not the leftmost (rate_limiter.zig:252-283, lastIndexOf): "the trusted proxy appends the address it actually received from, so the rightmost entry is proxy-controlled." A client connecting through the StartOS proxy cannot spoof its IP regardless of the XFF it sends.

2. StartOS isolates each service in its own network namespace (Start9Labs/start-docs start-os/src/architecture.md: "Each service runs in its own LXC container with: Network namespace isolation"). wisp binds 0.0.0.0:7777 inside its own netns; only the StartOS reverse proxy bridges in to reach it. Sibling services cannot reach :7777 directly, so there is no non-proxy peer to inject a forged XFF. This is the only thing the empty-trusted_proxies choice depended on.

Pinning trusted_proxies would defend a path netns isolation already closes, while adding a real silent-regression risk (a wrong intra-netns proxy source IP → isTrustedProxy fails → falls back to socket IP → the original 10-connection bug returns). Net negative, so trusted_proxies stays empty.

Functional confirmation for start-tunnel deployments

For relays fronted by start-tunnel: its port forwarding is iptables DNAT (L3/4) with TLS passthrough (start-docs start-tunnel/src/architecture.md) — no source NAT. DNAT-only preserves the original client source IP, so StartOS sees the real public client IP and the reverse proxy puts it in XFF. trust_proxy=true alone makes per-IP limiting work correctly for public clients; no tunnel-IP collapse, and no need to also bump max_connections_per_ip.

Residual (accepted)

• IP-based controls (max_connections_per_ip, ip_whitelist/ip_blacklist) are only as trustworthy as this proxy chain — inherent to any reverse-proxied relay.
• trust_proxy is force-set on every init with no override action — intended for this proxy-only deployment.

Next: on-device sanity check that wisp logs the real client IP (not a single internal/tunnel IP) for an external connection, then open the PR.