Conversation
WalkthroughThis PR adds release-date-aware caching logic for torrent searches. It introduces three new configurable TTL parameters and implements conditional cache freshness checks based on content release recency and debrid availability, with fallback scraping when needed. Changes
Sequence Diagram(s)sequenceDiagram
participant Client as Client/API
participant Cache as Release<br/>Date Cache
participant Debrid as Debrid<br/>Service
participant Lock as Distributed<br/>Lock
participant Scraper as Live<br/>Scraper
participant TorrentCache as Torrent<br/>Cache
Client->>Cache: Fetch release_date for media_id
Cache-->>Client: Return release_date (if available)
alt Release Date Available
Client->>Client: Compute effective_cache_ttl<br/>based on RECENT_RELEASE_DAYS
else No Release Date
Client->>Client: Use standard LIVE_TORRENT_CACHE_TTL
end
Client->>TorrentCache: Check cache freshness<br/>using effective_cache_ttl
TorrentCache-->>Client: Return cached_count
alt Cache Stale & No Debrid Cached Torrents
Client->>Lock: Acquire distributed lock
Lock-->>Client: Lock acquired/waiting
Client->>Scraper: Initiate live scrape
Scraper->>Debrid: Search for torrents
Debrid-->>Scraper: Return results
Scraper->>TorrentCache: Update cache
TorrentCache-->>Scraper: Confirm
Scraper-->>Client: Scrape complete
Client->>Lock: Release lock
Client->>TorrentCache: Re-check cache<br/>after scrape
TorrentCache-->>Client: Return updated counts
else Cache Valid or Debrid Torrents Present
Client-->>Client: Use existing cache
end
Client-->>Client: Return results with appropriate TTL
Possibly related PRs
🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
comet/api/endpoints/stream.py (1)
540-616: Consider extracting the live scrape + debrid re-check pattern into a helper function.This fallback block duplicates logic from earlier in the function (lock acquisition, scraping, debrid availability check). The complexity is manageable but could benefit from extraction for maintainability.
Additionally, the
cached_count == 0check on line 572 is redundant since it's already ensured by the outer condition on line 544.♻️ Suggested simplification for line 572
- if cached_count == 0 and not needs_scraping and not lock_acquired: + if not needs_scraping and not lock_acquired:
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
.env-samplecomet/api/endpoints/stream.pycomet/core/models.py
🧰 Additional context used
🧬 Code graph analysis (1)
comet/api/endpoints/stream.py (4)
comet/core/db_router.py (1)
fetch_val(84-87)comet/services/lock.py (3)
DistributedLock(9-113)acquire(25-81)release(83-94)comet/services/orchestration.py (2)
scrape_torrents(59-105)get_cached_torrents(107-156)comet/services/debrid.py (1)
check_existing_availability(78-116)
🔇 Additional comments (4)
.env-sample (1)
74-89: LGTM!The new environment variables are well-documented with clear explanations of their purpose and interaction with existing TTL settings. The "disabled by default" approach is appropriate for optional features that change caching behavior.
comet/core/models.py (1)
45-47: LGTM!The new configuration fields are correctly typed as
Optional[int]withNonedefaults, matching the .env-sample documentation. Logical placement alongside related TTL settings.comet/api/endpoints/stream.py (2)
283-300: LGTM!The effective TTL is correctly integrated into the fresh cache count query. The outer condition properly checks if the TTL feature is enabled (>= 0), while the inner query uses the computed
effective_cache_ttlwhich accounts for recent releases.
249-277: No issue found. Therelease_dateis correctly stored as a Unix timestamp (BIGINT in the database). It's populated viadatetime.strptime(release_date_str, "%Y-%m-%d").timestamp()incomet/metadata/filter.py(line 76-77), which converts date strings to Unix timestamps in seconds. The calculation(time.time() - release_date) / 86400in line 268 is correct for converting the difference to days.
1 participant
Improve scraping on recent releases with a more aggressive TTL and fallback to live search if no debrid cached results are returned
#466
Summary by CodeRabbit
Release Notes
✏️ Tip: You can customize this high-level summary in your review settings.