Repository navigation
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Asio's objects not being thread-safe is annoying. The typical advice is to use an
asio::strand(ortmc::ex_braid) to serialize the socket handlers; however, this serialization locks out other handlers during the entire duration of the handler, even for parts that don't touch the socket.This PR adds
tmc::asio_safe_*wrappers that are much more efficient; they only lock the minimum necessary critical section for each operation. For async operations, this is the operation initiation. For sync operations (e.g.cancel()), it's the whole operation. This makes the objects fully thread-safe, so you can read, write, close, etc. the socket from multiple threads without needing a strand/braid.Note that this does not alter the other invariants, e.g. max 1 write + 1 read concurrently on a socket.
Every object exposes most of the same methods as the underlying Asio object. However, they are changed slightly:
cancel()becomes an awaitable insteadtmc::aw_asiocompletion token is impliedAdditionally, the socket doesn't implement the Asio stream concept directly, but you can access the underlying socket if you need to with
.inner().The implementation is quite efficient. I tested asio_http_server (1 I/O thread, 7 CPU threads) which currently does 865k RPS with the unsynchronized implementation. Switching to
asio_safe_socketyielded 820k RPS. If we had to use strand/ex_braid here, it would be much more costly.Implementation Background:
The minimal critical section is the operation initiation (the internal
async_initiate()call). After that, it's safe for another thread to initiate another operation. We could shrink the critical section by entering and leaving a strand/braid; however, this causes the the task to enter the braid twice, both at initiation and at resumption:The "re-enter after resumption" behavior occurs because the braid is an executor, and executor affinity causes the aw_asio completion token to send the continuation back to its its originating executor (the braid). So in order to lock only the necessary part - the initiation - without re-locking after resumption, we need a type that behaves like a braid but isn't an executor.
Implementation Details:
That type is
tmc::detail::tiny_mutex, specially created for use within these safe wrappers. Its behavior is a bit unintuitive and correct operation is coupled to the awaitables, so it's only used internally. Its implemented as a locked run queue that calls.resume()on each task. When the.resume()call returns (the task is suspended or finished), the next item in the run queue may be executed. To resume a suspended task, its continuation is posted back to its native executor, not the run queue. Under this model, suspend == unlock, so if the task doesn't suspend, it doesn't unlock. So, the unlock is done two ways:tmc::aw_asiocompletion token always suspend (its await_ready returns false)cancel()don't suspend, so we manually unlock afterward