Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/ci-cd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ on:
- patchback/backports/** # Patchback always creates PRs
- pre-commit-ci-update-config # pre-commit.ci always creates a PR
pull_request:
ignore-paths: # changes to the cron workflow are triggered through it
paths-ignore: # changes to the cron workflow are triggered through it
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I extracted this into main. I have this typo across a bunch of repos, actually. Good catch!

@sirosen any insight into why jsonschema didn't catch this?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to work up a fix I can submit to schemastore for this, but I did... sort of figure it out.

The definition of a pull_request event mixes explicit properties with a $ref usage (the target of the JSON Schema $ref is also named "ref", which is a little quirky but fine). When $ref loading is mixed with an explicit schema, as in this case, I don't have any clear expectation about what should happen. It might be covered by the spec, but it's definitely going to trip up implementations.

The definition of "ref" looks fine at a glance:
https://github.com/SchemaStore/schemastore/blame/71c836f7a50aa0f796c990c433307dc7b87e300e/src/schemas/json/github-workflow.json#L340-L386

But I notice that it doesn't set "additionalProperties": false, and I think that's the issue. If I remove that $ref usage in a copy of the schema, I get the appropriate error.

It looks like this is the only usage site for #/definitions/ref, so I think I'll just inline it and add a test which demonstrates the issue. JSON Schema is always a headtrip! 😵‍💫

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did end up getting a PR posted!
It ate a lot of my free time for FOSS work this evening, but I think it's worth it. There's a pretty large class of mistakes which check-jsonschema can't catch until this is fixed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sirosen nice, thanks!

- .github/workflows/scheduled-runs.yml
workflow_call: # a way to embed the main tests
workflow_dispatch:
Expand Down Expand Up @@ -83,6 +83,7 @@ env:
PYTHONIOENCODING
PYTHONLEGACYWINDOWSSTDIO
PYTHONUTF8
PYTHONWARNINGS: "ignore::ResourceWarning:sqlite3"
TOX_VERSION: tox < 4.12
UPSTREAM_REPOSITORY_ID: >-
16620627
Expand Down
2 changes: 2 additions & 0 deletions cheroot/connections.pyi
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
from typing import Any

IS_WINDOWS: bool

def prevent_socket_inheritance(sock) -> None: ...

class _ThreadsafeSelector:
Expand Down
13 changes: 12 additions & 1 deletion cheroot/errors.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,12 @@
import sys


try:
from OpenSSL.SSL import SysCallError as _OpenSSL_SysCallError
except ImportError:
_OpenSSL_SysCallError = None


class MaxSizeExceeded(Exception):
"""Exception raised when a client sends more data then allowed under limit.

Expand Down Expand Up @@ -66,6 +72,7 @@ def plat_specific_errors(*errnames):


acceptable_sock_shutdown_error_codes = {
errno.EBADF,
errno.ENOTCONN,
errno.EPIPE,
errno.ESHUTDOWN, # corresponds to BrokenPipeError in Python 3
Expand All @@ -87,4 +94,8 @@ def plat_specific_errors(*errnames):
* https://docs.microsoft.com/windows/win32/api/winsock/nf-winsock-shutdown
"""

acceptable_sock_shutdown_exceptions = (BrokenPipeError, ConnectionResetError)
acceptable_sock_shutdown_exceptions = (
BrokenPipeError,
ConnectionResetError,
_OpenSSL_SysCallError,
)
75 changes: 73 additions & 2 deletions cheroot/makefile.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,30 @@

# prefer slower Python-based io module
import _pyio as io
import errno
import socket
import sys

from OpenSSL.SSL import SysCallError


# Define a variable to hold the platform-specific "not a socket" error.
_not_a_socket_err = None

if sys.platform == 'win32':
# On Windows, try to get the named constant from the socket module.
# If that fails, fall back to the known numeric value.
try:
_not_a_socket_err = socket.WSAENOTSOCK
except AttributeError:
_not_a_socket_err = 10038
else:
# On other platforms, the relevant error is EBADF (Bad file descriptor),
# which is already in the list of handled errors.
pass

# Expose the error constant for use in the module's public API if needed.
WSAENOTSOCK = _not_a_socket_err

# Write only 16K at a time to sockets
SOCK_WRITE_BLOCKSIZE = 16384
Expand All @@ -30,10 +52,59 @@ def _flush_unlocked(self):
# ssl sockets only except 'bytes', not bytearrays
# so perhaps we should conditionally wrap this for perf?
n = self.raw.write(bytes(self._write_buf))
except io.BlockingIOError as e:
n = e.characters_written
except (io.BlockingIOError, OSError, SysCallError) as e:
# Check for a different error attribute depending
# on the exception type
if isinstance(e, io.BlockingIOError):
n = e.characters_written
else:
error_code = (
e.errno if isinstance(e, OSError) else e.args[0]
)
if error_code in {
errno.EBADF,
errno.ENOTCONN,
errno.EPIPE,
WSAENOTSOCK, # Windows-specific error
}:
# The socket is gone, so just ignore this error.
return
raise
else:
# The 'try' block completed without an exception
if n is None:
# This could happen with non-blocking write
# when nothing was written
break

del self._write_buf[:n]

def close(self):
"""
Close the stream and its underlying file object.

This method is designed to be idempotent (it can be called multiple
times without side effects). It gracefully handles a race condition
where the underlying socket may have already been closed by the remote
client or another thread.

A SysCallError or OSError with errno.EBADF or errno.ENOTCONN is caught
and ignored, as these indicate a normal, expected connection teardown.
Other exceptions are re-raised.
"""
if self.closed: # pylint: disable=W0125
return

try:
super().close()
except (OSError, SysCallError) as e:
error_code = e.errno if isinstance(e, OSError) else e.args[0]
if error_code in {errno.EBADF, errno.ENOTCONN}:
# The socket is already closed, which is expected during
# a race condition.
return
raise


class StreamReader(io.BufferedReader):
"""Socket stream reader."""
Expand Down
7 changes: 7 additions & 0 deletions cheroot/makefile.pyi
Original file line number Diff line number Diff line change
@@ -1,7 +1,14 @@
import io
import sys
from typing import Optional

WSAENOTSOCK: Optional[int]

SOCK_WRITE_BLOCKSIZE: int

if sys.platform == 'win32':
WIN_SOCKET_NOT_OPEN: Optional[int]

class BufferedWriter(io.BufferedWriter):
def write(self, b): ...

Expand Down
53 changes: 45 additions & 8 deletions cheroot/server.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@

import contextlib
import email.utils
import errno
import io
import logging
import os
Expand All @@ -81,6 +82,8 @@
import urllib.parse
from functools import lru_cache

from OpenSSL.SSL import SysCallError

from . import __version__, connections, errors
from ._compat import IS_PPC, bton
from .makefile import MakeFile, StreamWriter
Expand Down Expand Up @@ -1186,12 +1189,25 @@ def ensure_headers_sent(self):

def write(self, chunk):
"""Write unbuffered data to the client."""
if self.chunked_write and chunk:
chunk_size_hex = hex(len(chunk))[2:].encode('ascii')
buf = [chunk_size_hex, CRLF, chunk, CRLF]
self.conn.wfile.write(EMPTY.join(buf))
else:
self.conn.wfile.write(chunk)
try:
if self.chunked_write and chunk:
chunk_size_hex = hex(len(chunk))[2:].encode('ascii')
buf = [chunk_size_hex, CRLF, chunk, CRLF]
self.conn.wfile.write(EMPTY.join(buf))
else:
self.conn.wfile.write(chunk)
except (SysCallError, ConnectionError, OSError) as e:
error_code = e.errno if isinstance(e, OSError) else e.args[0]
if error_code in {
errno.ECONNRESET,
errno.EPIPE,
errno.ENOTCONN,
errno.EBADF,
}:
# The socket is gone, so just ignore this error.
return

raise

def send_headers(self): # noqa: C901 # FIXME
"""Assert, process, and send the HTTP response message-headers.
Expand Down Expand Up @@ -1285,7 +1301,27 @@ def send_headers(self): # noqa: C901 # FIXME
for k, v in self.outheaders:
buf.append(k + COLON + SPACE + v + CRLF)
buf.append(CRLF)
self.conn.wfile.write(EMPTY.join(buf))
try:
self.conn.wfile.write(EMPTY.join(buf))
except (SysCallError, ConnectionError, OSError) as e:
# We explicitly ignore these errors because they indicate the
# client has already closed the connection, which is a normal
# occurrence during a race condition.

# The .errno attribute is only available on OSError
# The .args[0] attribute is available on SysCallError
# Check for both cases to handle different exception types
error_code = e.errno if isinstance(e, OSError) else e.args[0]
if error_code in {
errno.ECONNRESET,
errno.EPIPE,
errno.ENOTCONN,
errno.EBADF,
}:
self.close_connection = True
self.conn.close()
return
raise


class HTTPConnection:
Expand Down Expand Up @@ -1541,9 +1577,10 @@ def _close_kernel_socket(self):
self.socket.shutdown,
)

acceptable_exceptions = errors.acceptable_sock_shutdown_exceptions
try:
shutdown(socket.SHUT_RDWR) # actually send a TCP FIN
except errors.acceptable_sock_shutdown_exceptions:
except acceptable_exceptions: # pylint: disable=E0712
pass
except socket.error as e:
if e.errno not in errors.acceptable_sock_shutdown_error_codes:
Expand Down
13 changes: 13 additions & 0 deletions cheroot/server.pyi
Original file line number Diff line number Diff line change
@@ -1,5 +1,18 @@
from typing import Any

__all__ = (
'ChunkedRFile',
'DropUnderscoreHeaderReader',
'Gateway',
'HTTPConnection',
'HTTPRequest',
'HTTPServer',
'HeaderReader',
'KnownLengthRFile',
'SizeCheckWrapper',
'get_ssl_adapter_class',
)

class HeaderReader:
def __call__(self, rfile, hdict: Any | None = ...): ...

Expand Down
22 changes: 13 additions & 9 deletions cheroot/ssl/builtin.py
Original file line number Diff line number Diff line change
Expand Up @@ -286,11 +286,19 @@ def wrap(self, sock):
raise errors.FatalSSLAlert(
*tls_connection_drop_error.args,
) from tls_connection_drop_error
except ssl.SSLError as generic_tls_error:
peer_speaks_plain_http_over_https = (
generic_tls_error.errno == ssl.SSL_ERROR_SSL
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What other error codes can this case have?

and _assert_ssl_exc_contains(generic_tls_error, 'http request')
)
except (
ssl.SSLError,
OSError,
) as generic_tls_error:
# When the client speaks plain HTTP into a TLS-only connection,
# Python's builtin ssl raises an SSLError with `http request`
# in its message. Sometimes, due to a race condition, the socket
# is closed by the time we try to handle the error, resulting in an
# OSError: [Errno 9] Bad file descriptor.
peer_speaks_plain_http_over_https = isinstance(
generic_tls_error,
ssl.SSLError,
) and _assert_ssl_exc_contains(generic_tls_error, 'http request')
if peer_speaks_plain_http_over_https:
reraised_connection_drop_exc_cls = errors.NoSSLError
else:
Expand All @@ -299,10 +307,6 @@ def wrap(self, sock):
raise reraised_connection_drop_exc_cls(
*generic_tls_error.args,
) from generic_tls_error
except OSError as tcp_connection_drop_error:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why you felt the need to move this into another except-block. That makes it more complicated to follow. And AFAICS the lotig is about the same just more branchy. I was keeping it separate so that it's simple.

raise errors.FatalSSLAlert(
*tcp_connection_drop_error.args,
) from tcp_connection_drop_error

return s, self.get_environ(s)

Expand Down
2 changes: 1 addition & 1 deletion cheroot/test/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
@pytest.fixture
def http_request_timeout():
"""Return a common HTTP request timeout for tests with queries."""
computed_timeout = 0.1
computed_timeout = 0.5

if IS_MACOS:
computed_timeout *= 2
Expand Down
Loading
Loading