fix: prevent blocking scrapes while cache is cold#62
Merged
Conversation
Three coordinated changes to ensure Prometheus scrapes never block on a synchronous DB collection: - Override describe() to return [] so REGISTRY.register() skips the implicit collect() call that would fire full DB queries at startup. - In collect(), return empty immediately when cache is enabled but not yet populated, instead of falling through to a synchronous collection that would block the scrape and race the background thread. - Start _start_background_collection() before REGISTRY.register() so the cache begins warming as early as possible, shrinking the cold window. Co-authored-by: Cursor <cursoragent@cursor.com>
Odrec
added a commit
that referenced
this pull request
May 8, 2026
Two small follow-ups to the cold-cache fix in #62: - Move the "Cache configuration" logger.info block out of _collect_loop. It sat after the `while not self._stop_collection.is_set()` loop, so it only printed during shutdown — which made the cache config invisible at startup. Moved to __init__ alongside the existing "Metric groups configuration" block so both configs log together when the collector is constructed. - Update the collect() docstring to reflect the three-way behavior introduced by #62: cache-warm serves cached, cache-cold yields nothing, cache-disabled collects fresh. Co-authored-by: Odrec <odrec@Odrecs-MacBook-Pro.local>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
prometheus_client's defaultCollector.describe()callscollect()to discover metric names. This meantREGISTRY.register()triggered a full DB collection synchronously, blocking the process for
the entire query duration before the server could accept any requests.
had finished its first collection cycle,
collect()fell through to asynchronous fresh collection — blocking the scrape and racing the
background thread, doubling DB load on every scrape until the cache
was warm.
Three coordinated fixes:
describe()to return[], bypassing the implicitcollect()call during
REGISTRY.register()and letting the server start immediately.collect(), return empty immediately when the cache is enabled but notyet populated, instead of falling through to a synchronous collection.
_start_background_collection()beforeREGISTRY.register()so thecache begins warming as early as possible, shrinking the cold window.
Test plan