Skip to content

Add timeouts and connection health checks to Redis connections #7526

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
May 27, 2025

Conversation

carodewig
Copy link
Contributor

@carodewig carodewig commented May 19, 2025

Pulls in #7354 by @aembke.

Changes included in this PR:

  • Update fred to 10.1.0
  • Update fred configuration to improve client resiliency under various failure modes (TCP failures and timeouts, unresponsive sockets, Redis server failures, etc)
  • Add heartbeats (sending PING every 10 sec) to each client in the pool
  • Refactor client initialization
  • Refactor client error and reconnection stream reading to not stop on RecvError::Lagged (which can happen reader end of the channel is too slow)

Note: this PR removes the timeout on wait_for_connect in RedisCacheStorage::create_client. The internal_command_timeout and unresponsive config fields should handle timeouts on connection creation, so hopefully that's not necessary.

Fixes #6855
Fixes #7354


Checklist

Complete the checklist (and note appropriate exceptions) before the PR is marked ready-for-review.

  • Changes are compatible1
  • Documentation2 completed
  • Performance impact assessed and acceptable
  • Tests added and passing3
    • Unit Tests
    • Integration Tests
    • Manual Tests

Exceptions

Note any exceptions here

Notes

Footnotes

  1. It may be appropriate to bring upcoming changes to the attention of other (impacted) groups. Please endeavour to do this before seeking PR approval. The mechanism for doing this will vary considerably, so use your judgement as to how and when to do this.

  2. Configuration is an important part of many changes. Where applicable please try to document configuration examples.

  3. Tick whichever testing boxes are applicable. If you are adding Manual Tests, please document the manual testing (extensively) in the Exceptions.

@carodewig carodewig requested a review from a team as a code owner May 19, 2025 18:32

This comment has been minimized.

@svc-apollo-docs
Copy link
Collaborator

svc-apollo-docs commented May 19, 2025

✅ Docs preview has no changes

The preview was not built because there were no changes.

Build ID: 1f5dd3aafb87e48fa90c3348

@carodewig carodewig requested a review from a team as a code owner May 19, 2025 18:41
@goto-bus-stop goto-bus-stop self-requested a review May 21, 2025 07:09
Copy link
Member

@goto-bus-stop goto-bus-stop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hope I didn't miss anything 😇, but imo this looks great

@carodewig carodewig enabled auto-merge (squash) May 27, 2025 15:30
@carodewig carodewig merged commit 72b08fa into dev May 27, 2025
15 checks passed
@carodewig carodewig deleted the caroline/redis branch May 27, 2025 15:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Redis connection and connection pool maintainance (TCP keepalive, redis ping)
4 participants