Conversation
|
The generated output of Full Type Diff |
Contributor
|
@danlapid Bonk workflow was cancelled. View workflow run · To retry, trigger Bonk again. |
danlapid
force-pushed
the
dlapid/rustHttpLean
branch
2 times, most recently
from
September 20, 2026 19:38
5ab44a0 to
ef5c6ce
Compare
Under --//:io_backend=rust, workerd's HTTP/1.1 client and server, WebSockets,
CONNECT tunnels and TLS run in Rust on hyper, rustls and tungstenite's frame
codec. workerd is still written against kj's HTTP interfaces (kj::HttpClient,
HttpService, HttpServer, WebSocket, SecureNetworkWrapper); only their
implementations change, by symbol override, as the I/O layer does. Product code
is unchanged, and the --//:io_backend=cxx build keeps kj's implementations.
Layout:
- kj-hyper (src/rust/cxx/kj-hyper): HTTP/1.1 clients over hyper's connection
API (a connection pool dialed through kj networks, or one connection over a
kj stream), hyper's server fused into one kj event per connection, the
WebSocket frame loop with permessage-deflate (flate2/zlib-rs), rustls TLS,
and the kj <-> tokio stream bridges, with a thin C++ seam. Upgrades are
taken over from hyper by hand, so transports need not be Send; handles are
single-threaded, as kj's HTTP objects are, and operations own their state.
The lifetime, cancellation and threading rules are in src/rust/cxx/AGENTS.md.
- kj-http-tokio.c++: the kj:: HTTP entry points over kj-hyper.
- tls-network: config::TlsOptions as a kj::SecureNetworkWrapper, one source
per engine (rustls or OpenSSL), so server.c++ has no per-backend #if.
- src/rust/kj: the kj crate links kj-http-types only.
- capnproto (companion change): kj-http-types holds everything that is not
kj's wire implementation -- header codec, permessage-deflate negotiation,
WebSocket pump and pipe, client/service adapters -- so this backend reuses
kj's own code for those and links without kj-http-impl.
Behavior follows kj's contracts where workerd relies on them:
- Wire fidelity: header spellings both ways (hyper HeaderCaseMap patch),
reason phrases, kj's Content-Length and Connection: close rules, drain
finishing a partially received request and stopping the listener.
- Errors: HttpServerErrorHandler/WebSocketErrorHandler run as in kj's server,
for CONNECT too; kj exceptions survive the trip through hyper. Unsupported
HttpServerSettings are documented in hyper-http.h.
- Bodies: a write beyond the declared length fails ("overwrote
Content-Length"); a response to HEAD keeps the application's Content-Length
and discards its body.
- Clients: requests over one stream wait their turn for its connection; an
abandoned response body fails later requests as kj's does; a connection is
reused only once its response body has been read.
- WebSocket: text passes through unvalidated, the application owns Close,
maxSize is checked from each frame header against what is left of the
message, one operation writes at a time and writes what was owed meanwhile
(a pong, a protocol-error Close, disconnect()'s shutdown), writes complete
once flushed, abort() fails operations in flight, a send cancelled mid-frame
ends sending, wire byte counters. The server refuses
bad handshakes with 400, or 426 with Sec-WebSocket-Version (RFC 6455 4.2.2);
the client validates Sec-WebSocket-Accept and takes its key from the
configured EntropySource.
- Disconnects: whenWriteDisconnected() and whenAborted() resolve on peer
hangup (and fail as observing it fails), and a peer that disconnects cancels
its connection's service calls.
- Server timeouts: headerTimeout applies to each request head.
- TLS: handshakes, client authentication included, complete before a stream is
handed out, and a listener runs them concurrently. trustBrowserCas uses the
platform verifier, otherwise only trustedCertificates are trusted; a trusted
certificate is also trusted directly, as OpenSSL does, with its validity
and key usage still checked. minVersion TLS 1.3 is honored; cipherList must
name TLS 1.2 suites, and requireClientCerts with trustBrowserCas is refused.
A peer closing without close_notify ends reads with DISCONNECTED, as kj's TLS
does.
Crate patches: http/httparse lenient header values, hyper's public header case
map, graceful shutdown finishing a partial request, lazily read client response
bodies.
Tests: kj-http-test (53) and tls-network-test (16) check kj's HTTP and TLS
entry points against the wire and the TLS policy, and run unchanged on both
backends. kj-hyper_test covers the deflate codec (RFC 7692 vectors), the TLS
options and direct trust, WebSocket writer sharing, abort and cancellation,
body length accounting, flushing writes, TLS waker sharing and the hangup
watch. Under the rust backend server-test drives
its loop with a TokioEventPort; it gains a WebSocket compression case, and
TestServer::wait() lets events due at its final instant settle.
rust-io-link-check also refuses kj's HTTP implementation and kj-tls in the
linked binary.
Known failure under the rust backend: tls-nodejs-test's testTlsOnEmptySocket.
Its sidecar ends the connection before the client's Finished arrives, and node
then sends no close_notify; the strict end-of-stream above reports that as
DISCONNECTED. The cxx backend writes Finished sooner and wins the race.
Requires the capnproto kj-http split (kj-http-types).
Co-Authored-By: Claude Code <noreply@anthropic.com>
danlapid
force-pushed
the
dlapid/rustHttpLean
branch
from
September 20, 2026 21:30
ef5c6ce to
0008bef
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Under --//:io_backend=rust, workerd's HTTP/1.1 client and server, WebSockets, CONNECT tunnels and TLS run in Rust on hyper, hyper-util, rustls and tungstenite's frame codec. workerd is still written against kj's HTTP interfaces (kj::HttpClient, HttpService, HttpServer, WebSocket, SecureNetworkWrapper); only their implementations change, by symbol override, as the I/O layer does. Product code is unchanged, and the default C++ build is untouched.
Layout:
Behavior follows kj's contracts where workerd relies on them:
Crate patches: http/httparse lenient header values, hyper's public header case map, graceful shutdown finishing a partial request, lazily read client response bodies.
Tests: kj-http-test (48) and tls-network-test (14) check kj's HTTP and TLS entry points against the wire and the TLS policy, and run unchanged on both backends. kj-hyper_test covers the deflate codec (RFC 7692 vectors) and the TLS option mapping. server-test gains a WebSocket compression case, and TestServer::wait() lets events due at its final instant settle. bazel test //src/... passes under the rust backend; the cxx build's server-test, kj-http-test and tls-network-test pass.
Requires the capnproto kj-http split (kj-http-types).