[DBZ-PGYB] Restrict retries when retry count reaches limit #166
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
With the current retry model, it was being noticed that the connector ended up retrying infinitely whenever an exception was thrown. This could lead to false positives that the connector is still running while the retries will keep failing.
Solution
This PR addresses the issue by adding a check to the task layer so now if the retry count reaches the maximum value, the connector will exit and the task will reach in a failed state - this will help the end user know the status of the task and act accordingly.
This PR also redefines the following properties and changes their default values:
errors.max.retries
- new default value is 60retriable.restart.connector.wait.ms
- new default value is 30000 (30s)With the above change, the complete retry duration with the above default configuration will now be 30 minutes. This effectively means that if the connector/s task fails after exhausting all the retries then it will go into a
FAILED
state.For example, if the connector is needed to retry for a total of 30 minutes then we can handle it in 2 ways:
30 / 15 = 2 minutes = 120 s = 120000 ms
, so the configuration will now add:30 / 1 = 30 retries
, so the configuration will now be: