🐛 Bug Report — Response remains open after body is flushed (for ~5s)

[RamIdeas]

As shown in this comment and this comment, workerd seems to hold the response open for roughly 5 seconds after the data is fully flushed.

By default, curl will wait to receive the number of bytes specified in the Content-Length response header and stop reading the response. By setting the --ignore-content-length flag, you can force curl to wait for the response to be closed. This seems to take roughly 5 seconds longer than when the flag is not set.

This was noticed because tools like ab have a different default. I need to check what the default is in browsers.

I'm short for time right now, but I will (likely next week) follow up with:

• Do browsers wait for the response to close or the content-length? (I suspect the former but I will check)
• A workerd-only repro with capnp config

[kentonv]

This is working as intended. HTTP/1.1 supports connection reuse: after a request and response have completed, another request can begin on the same connection. By default, KJ HTTP servers give the client 5 seconds (kj::HttpServerSettings::pipelineTimeout) to start another request. After that it closes the connection in order to avoid having an excessive number of idle connections open.

A client must consider the HTTP response complete upon reaching Content-Length (or the final empty chunk of Transfer-Encoding: chunked). It would in fact be a severe error (possible security vulnerability) for the client to treat any bytes after this point as being part of the response. In any case, it is not expected that the connection be closed at the end of a response -- unless the response had neither Content-Length nor Transfer-Encoding headers, in which case EOF is the only way to indicate the end of the response, but this is historical behavior from HTTP/1.0 which HTTP/1.1 implementations rarely use.