src/qamqpexchange: add support for rejected/nacked deliveries #73
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
manner82
force-pushed
the
PAM-5269-rabbitmq-nack-handling
branch
4 times, most recently
from
8de0f0d to
a06fd67
Compare
In case the queue has a message count/size limit (config "x-max-length"), rabbitmq can be configured to nack/reject publishing messages when the queue is full (config "x-overflow":"reject-publish"). In this setup, qamqp exchange only does a log and skips the nack message, which leads to these delivery tags are never removed from unconfirmedDeliveryTags vector, therefor allMessagesDelivered signal will never trigger. To make the library user be able to handle this (but try to maintain compatibility), I have added an extra signal (messageDeliveryFinished) which is also sent when rejected/nacked messages remained, identifying the nacked messages by their delivery tags. Also added direct signals for publish confirms / rejects, because handling nack/reject is often just a retry of the publish after some time. This requires storing the messages until they are confirmed. For some use cases keeping all messages in memory until there is a point when all waiting publishes gets confirmed is a waste of resources or not feasible at all (no time when all is confirmed). See https://www.rabbitmq.com/maxlength.html and https://www.rabbitmq.com/confirms.html#server-sent-nacks
…ublish References: jira PAM-5269
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In case the queue has a message count/size limit (config "x-max-length"),
rabbitmq can be configured to nack/reject publishing messages when the
queue is full (config "x-overflow":"reject-publish").
In this setup, qamqp exchange only does a log and skips the nack message,
which leads to these delivery tags are never removed from
unconfirmedDeliveryTags vector, therefor allMessagesDelivered signal
will never trigger.
To make the library user be able to handle this (but try to maintain
compatibility), I have added an extra signal (messageDeliveryFinished)
which is also sent when rejected/nacked messages remained, identifying
the nacked messages by their delivery tags.
Also added direct signals for publish confirms / rejects, because handling
nack/reject is often just a retry of the publish after some time. This requires
storing the messages until they are confirmed. For some use cases keeping
all messages in memory until there is a point when all waiting publishes
gets confirmed is a waste of resources or not feasible at all (no time when
all is confirmed).
See https://www.rabbitmq.com/maxlength.html and
https://www.rabbitmq.com/confirms.html#server-sent-nacks