Skip to content

Add the loopback stream server behind the embedded terminal - #24

Merged
cjimti merged 1 commit into
mainfrom
feat/3-loopback-stream-server
Aug 3, 2026
Merged

cjimti merged 1 commit into
mainfrom
feat/3-loopback-stream-server

Conversation

@cjimti

@cjimti cjimti commented Aug 3, 2026

Copy link
Copy Markdown
Member

Closes #3. Depends on #2 (merged). DESIGN.md §3.3.

The token-authenticated loopback WebSocket server that carries m6t's
throughput-sensitive traffic, and the first thing it carries: PTY I/O.

Why this exists

Terminal I/O cannot go over the Wails bridge. A build log arrives as thousands
of small writes and every one of them would cross a JSON marshaling boundary. So
the transport is split by payload profile (DESIGN.md §3.3): bindings for RPC,
loopback WebSocket for streams. The only thing the bridge carries for the
terminal is the endpoint that says where to connect and with what token.

What's in it

internal/stream — 838 LOC across six files, five exported names
(Server, New, Endpoint, Attachment, Terminals).

  • HTTP listener on 127.0.0.1:0. Not configurable: a stream server reachable
    from another host is a shell server. Port 0 so two m6t instances can run side
    by side.
  • Per-launch token from crypto/rand.Text() — 26 base32 chars, ~130 bits. Minted
    at construction, so a port learned from a previous run is useless.
  • GET /pty/{sessionID} — binary frames carry the character stream both
    directions, text frames carry control messages.
  • GET /events — backend-push envelopes. PTY exit today; git, watch and helm
    plug into the same envelope as they land, which is why it carries a type
    rather than the endpoint implying one.
  • internal/stream/PROTOCOL.md is the wire
    contract.
    It is written down rather than left to be inferred from the
    handlers, because the frontend is written against it. Read that first.

Architecture: why there is no stream → pty edge

The stream server carries PTY bytes and does not import internal/pty. It
declares a Terminals seam; internal/pty knows nothing about transports; and
internal/app — the one layer that knows about both — holds the adapter
(internal/app/terminals.go). depguard enforces the rule and
TestImportGraphIsPinned records the resulting shape:

.                  -> internal/app
internal/app       -> internal/buildinfo, internal/pty, internal/stream
internal/buildinfo -> (none)
internal/pty       -> (none)
internal/stream    -> (none)

Two service edges out of the binding layer, none between the services. The cost
is that stream.Attachment restates the shape of pty.Attachment and the
adapter translates. The benefit is that either service can be replaced without
touching the other, and a future stream → pty import fails a test as well as
the linter.

Security

Every refusal lands as an HTTP status before the upgrade. No client ever gets
a socket it is not allowed to use, and no 101 is sent and then walked back.

Condition Status
Missing or wrong token 401
Origin present and not allowed 403
Unknown terminal session 404
  • Order matters. The token is checked first, so an unauthenticated caller
    cannot use /pty/{sessionID} to discover which session IDs exist. The attach
    comes second, so an unknown session is a 404 on a plain response rather than a
    socket that opens and dies. The upgrade is last, and it is where a bad origin
    is refused.
  • Constant-time compare on the token (crypto/subtle).
  • Two credential forms. Authorization: Bearer <token> for clients that can
    set headers, and a m6t.token.<token> subprotocol for the browser WebSocket
    API, which cannot. The server negotiates m6t.v1 in that case because a
    browser fails the connection if it offered subprotocols and the server selected
    none. Same pattern the Kubernetes API server uses. Header wins when both are
    present, so a stale subprotocol cannot override an explicit credential.
  • Origin policy: absent (non-browser — browsers always send one, and the
    token is what guards that case), the Wails webview (wails://…, and
    http://wails.localhost on Windows), or loopback on any port. null — what a
    file:// document and a sandboxed frame report — is refused. This is what
    closes DNS rebinding: a rebound attacker page sends its own origin and gets a
    403 before it can try the token it does not have.
  • Nothing in the package logs. The token crosses the bridge to the webview
    that needs it and goes nowhere else — a token in a log file outlives the launch
    it was minted for.
  • Inbound messages are capped at 1 MiB; the request-header read has a timeout.

Backpressure

This is the property that makes the transport usable at all: a webview that has
stopped reading — mid-repaint, mid-GC, or wedged — must not be able to stall the
user's shell.

Each connection has a fixed 64-frame outbound queue. A full queue discards its
oldest frame
and the producer never waits. Every discarded byte is counted, and
the next frame written is preceded by a resync frame carrying the total. Drops
after the final data frame get a trailing marker before the socket closes, so the
accounting always balances:

for any connection, bytes received + bytes reported dropped = bytes the session
produced

resync means the character stream has a hole in it. Escape sequences do not
survive truncation, so a renderer must clear and redraw from a fresh attach
rather than paint across one. Oldest-first is deliberate: a terminal that kept
the start of a build log and dropped the prompt would be showing the user the
wrong thing.

Also fixes a defect in #2

pty.Attachment had no way to detach. Every reconnect — a webview reload, a
project-tab switch — left a consumer registered on the session holding up to
consumerQueue × 32KB of queue for a reader that was never coming back, for the
rest of the session's life. Attachment.Detach closes that leak. It is a struct
field rather than a new method, so internal/pty's exported-surface pin stays at
7.

The subtlety worth reviewing: both a detach and a real exit close the
attachment's channels, and only an exit publishes a status. A detach reported as
an exit would tell the UI that a live shell had died. session.detach and
session.finish are serialized on the same lock and each closes only what the
other did not, which is what makes "closed with no value" mean detach
unambiguously.

Acceptance criteria

From the issue Where
Dial with token, attach to a PTY session, echo round-trip internal/app/stream_test.go — TestATerminalSessionEchoesOverTheStreamSocket
Resize via control frame takes effect (stty size reflects it) TestResizeControlFrameChangesWhatTheChildSees
Close frame kills cleanly TestCloseControlFrameEndsTheChildAndReportsIt
No token / wrong token rejected before upgrade internal/stream/auth_test.go — TestConnectionsAreRefusedBeforeTheUpgrade
≥50MB streamed without unbounded backend memory internal/stream/backpressure_test.go — TestFiftyMegabytesStreamWithoutUnboundedMemoryGrowth

The end-to-end tests live in internal/app on purpose. internal/stream is
tested against a fake Terminals — correctly, since it must not know what a PTY
is — which leaves the adapter between the two services untested by construction.
That adapter is exactly the kind of code that looks obviously right and gets a
channel's close semantics wrong, so the real composition is what gets exercised
against a real shell.

Gates

make verify green.

Gate Result
make test pass, -race -shuffle=on; 5 repeat runs, no flakes
make coverage-report 96.1% total (floor 80%)
make patch-coverage 96.4% — 401/416 changed lines (floor 85%)
make lint / make frontend-lint 0 issues
make security / make semgrep clean
make licenses clean — gorilla/websocket is BSD-3-Clause
make bindings-check current
make build-check pass
gremlins (not in verify) internal/stream 24 killed / 0 lived / 100% efficacy; internal/app 5 killed / 0 lived / 100%

Zero suppressions. Two findings were fixed by changing code, not config:

  • go.gorilla.security.audit.websocket-missing-origin-check could not see
    CheckOrigin set on a struct field. upgrade() now builds the Upgrader at
    the point of use, which reads better anyway: there is no path to Upgrade that
    does not go past the origin policy, and that is now visible in one function.
  • gosec G101 matches identifiers containing "token", so the subprotocol prefix
    constant is named authSubprotocolPrefix, with a comment saying why.

Ratchets raised

Each number carries its justification in the diff, next to the number.

Pin Change Reason
structuralPins["internal/stream"] new: 900 LOC / 5 exported measured 838 + headroom for follow-ups
structuralPins["internal/app"] 200 → 260 LOC measured 201; room for one more service adapter of the same size
maxAppFields 2 → 3 one *stream.Server handle — port, token, connections and subscribers all live behind it
maxAppMethods 1 → 2 StreamEndpoint. Every terminal operation is a WebSocket frame and adds nothing here — a future PR adding a per-operation binding is not raising a ceiling, it is bypassing the transport

locCeilingNote also updated: internal/pty, internal/stream and
internal/app are now measured rather than policy-seeded.

Adversarial review — what I tried to break

One real bug, found and fixed. close initially returned "stop reading" from
the read loop, whose defer c.close() raced the forwarder's exit write. The
client saw 1006 abnormal closure instead of its exit code. The read loop now
stays up on close and the forwarder closes the socket after writing exit, so a
close never costs the client the status it asked for.

Two limits documented rather than fixed. Both want a design decision, not a
patch, and both are in PROTOCOL.md:

  1. PTY exit is published by the connection that observes it. A session ending
    with no socket attached is therefore not announced, and two sockets attached
    to one session both publish — a consumer must treat exit as idempotent. The
    fix, when something other than the terminal tab needs to know, is a dedicated
    attachment that watches the session.
  2. A connection upgraded concurrently with Shutdown can miss the close sweep.
    Moot in practice: the process is exiting.

Deliberately not covered (15 of 416 changed lines): write-failure paths on a
socket that is already gone, the *net.TCPAddr type assertion, and the
queue-full-after-making-room branch, which is unreachable for /pty (single
producer) and only reachable for /events. Everything else is exercised.

Out of scope

Frontend consumption is #4, so StreamEndpoint is the one bound method with no
caller yet — which is what this issue's scope line asks for. Event types beyond
PTY exit are also #4/#5.

projectID is documented in PROTOCOL.md as reserved in the envelope but is
not a Go field yet: nothing has projects until #5, and a field that is never
set is the vaporware the leash exists to refuse. Decoders must tolerate its
absence, which they will have to do anyway.

Reviewing this

  1. internal/stream/PROTOCOL.md — the contract.
  2. internal/stream/conn.go — the backpressure queue. The interesting question
    is whether any path can block a producer.
  3. internal/app/terminals.go — the adapter, and exitCodes in particular: it
    is where detach-versus-exit is preserved across the seam.
  4. internal/pty/session.go — detach against finish, and which one closes.

Terminal I/O cannot go over the Wails bridge. A build log arrives as
thousands of small writes and every one of them would cross a JSON
marshaling boundary, so DESIGN.md §3.3 splits the transport: bindings for
RPC, a loopback WebSocket for throughput. This is that server, and the
first thing it carries is PTY I/O.

internal/stream serves 127.0.0.1 on an OS-assigned port with a per-launch
crypto/rand token. Two endpoints: /pty/{sessionID} carries the character
stream as binary frames with control messages as text, and /events carries
backend-push envelopes. The wire contract is specified in PROTOCOL.md next
to the package rather than left to be inferred from the handlers, because
the frontend is written against it.

Every refusal lands as an HTTP status before the upgrade — 401 for a
missing or wrong token, 403 for a disallowed origin, 404 for an unknown
session — so no client gets a socket it is not allowed to use, and an
unauthenticated caller cannot use the endpoint to discover which sessions
exist. The token is accepted as an Authorization header and as a
subprotocol, because the browser WebSocket API cannot set headers.

Backpressure is the property that makes the transport usable at all: a
webview that has stopped reading must not be able to stall the user's
shell. Each connection has a bounded queue that discards its oldest frame
when full, and every discarded byte is reported in a resync marker so a
renderer knows not to paint across a hole in an escape-sequence stream.

The two services do not know about each other. stream declares a Terminals
seam, pty knows nothing about transports, and internal/app holds the
adapter that joins them — so either can be replaced without touching the
other, and the import graph pin records that shape.

Also fixes a defect in #2 that this exposes: pty.Attachment had no way to
detach, so every reconnect left a consumer holding its queue on the session
until the session ended. Attachment.Detach closes that leak.

Ratchets raised, with the reason recorded next to each number:
internal/app 200 -> 260 LOC (the first service adapter), maxAppFields
2 -> 3 (one *stream.Server handle), maxAppMethods 1 -> 2 (StreamEndpoint;
terminal operations are frames, not bindings).

Closes #3
@codecov

codecov Bot commented Aug 3, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.98328% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 94.53%. Comparing base (cec1bb1) to head (9afdfa5).

Files with missing lines Patch % Lines
internal/stream/conn.go 85.48% 6 Missing and 3 partials ⚠️
internal/stream/server.go 94.93% 3 Missing and 1 partial ⚠️
internal/stream/events.go 90.47% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #24      +/-   ##
==========================================
+ Coverage   94.04%   94.53%   +0.48%     
==========================================
  Files           8       14       +6     
  Lines         252      549     +297     
==========================================
+ Hits          237      519     +282     
- Misses         11       21      +10     
- Partials        4        9       +5     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@cjimti
cjimti merged commit a40f6c1 into main Aug 3, 2026
11 checks passed
@cjimti
cjimti deleted the feat/3-loopback-stream-server branch August 3, 2026 23:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Loopback stream server

1 participant