support TLS Encrypted ClientHello (ECH) and HTTPS/SVCB

[eighthave]

The next version of the IETF-standardized TLS protocol is known as Encrypted ClientHello (ECH) [0] formerly known as Encrypted SNI (ESNI). There is a related standard known a SVCB or DNS HTTPS RR Type [8]. This ticket is to collect information for what (if anything) is required for Requests to support ECH and HTTPS/SVCB. The Python ssl module will handle the core bits of ECH. ECH adds a new key to the TLS exchange, and the recommended way to get that key is from HTTPS/SVCB DNS queries. It is not yet clear where the right place to handle that DNS is [1]. ECH works in conjunction with the new DNS RR Types HTTPS and SVCB. This means that DNS needs to be handled a bit differently for connections that use HTTPS/SVCB.

ECH is built on top of TLSv1.3 and completes the unfinished work from the TLSv1.3 effort. It is now in draft-13 and there are many implementations that are interoperating. ECH is working for openssl[2], boringssl[3], nginx, Apache HTTPD, lighttpd, HAProxy, Conscrypt[4], curl, and more. There is work underway in Firefox [5] and Chromium [6]. It has been sketched out for OkHTTP [7]. Draft versions of ESNI and ECH have been deployed in Firefox releases and some production web services.

One approach would be to implement the DNS pieces in Requests, and the ECH pieces in the ssl module. That means potentially implementing:

• HTTPS/SVCB DNS queries for getting ECH Config Lists.
• A way to provide ECH Config Lists as bytes directly to the ssl module.
• A method to ensure encrypted DNS is used so all metadata is encrypted.
• A callback that gets called whenever ECH negotiation fails and the server offers a "Retry Config".

FYI, the TLS WG maintain a page with information about other implementations:
https://github.com/tlswg/draft-ietf-tls-esni/wiki/Implementations

[0] https://www.ietf.org/archive/id/draft-ietf-tls-esni-13.html
[1] https://bugs.python.org/issue45567
[2] openssl/openssl#7482
[3] https://bugs.chromium.org/p/boringssl/issues/detail?id=275
[5] https://bugzilla.mozilla.org/show_bug.cgi?id=1725938
[6] https://bugs.chromium.org/p/chromium/issues/detail?id=1091403
[6] google/conscrypt#730
[7] square/okhttp#6539
[8] https://www.ietf.org/archive/id/draft-ietf-dnsop-svcb-https-07.html
[9] https://github.com/sftcd/openssl

[sethmlarson]

Thanks for opening this issue. Unfortunately this is likely not the correct location to open this issue at this time. Requests uses urllib3 under the hood which uses Python's TLS stack which is built on OpenSSL. For us to take advantage of any of these features we'd need:

• Support in OpenSSL
• Support in Python's ssl module
• Likely explicit support in urllib3
• Then finally can add support in Requests.

You can start advocating for these features' inclusion in the projects listed above in order, but without support in the project above it the project below cannot do anything yet. Hope this makes sense, closing for now.

[eighthave]

Makes sense. openssl has been implemented with some review from openssl maintainers. They want to support ECH, but won't merge until its RFC. Python ssl wants to support ECH and is waiting on openssl. I opened this here because in the Python ssl issue, they said that clients should handle the DNS. The IETF process is at the point where we need feedback on interoperability, that's why I'm bringing this up even though the RFC hasn't been issued and the other pieces are not yet in place. Should I open an issue in urllib3 now?