Skip to content

Add drift-comparator pushdown for conditional market fetches#51

Closed
IamJasonBian wants to merge 2 commits intomainfrom
IamJasonBian/drift-comparators
Closed

Add drift-comparator pushdown for conditional market fetches#51
IamJasonBian wants to merge 2 commits intomainfrom
IamJasonBian/drift-comparators

Conversation

@IamJasonBian
Copy link
Copy Markdown
Owner

Summary

Implements a drift-comparator layer that conditionally fetches market data only when needed, reducing API calls from 8+ per cycle to 1 in flat markets.

  • DriftComparator: New module caching per-symbol price/metrics state; triggers full OHLCV fetches on price drift (>0.5%, tighter in high-vol) or staleness (>30min)
  • Pushdown in run_once(): Multi-quote first (1 API call), full-fetch only drifted symbols, use cached metrics for others
  • refresh_open_order_metrics(): On-demand method to refresh metrics for symbols with open orders
  • Tests: 14 new unit tests validating threshold logic, caching, and edge cases

Verification

  • All 14 new tests pass
  • No new test failures (7 pre-existing failures unrelated to these changes)
  • Cold start behavior unchanged: all symbols get full fetch on first cycle
  • Subsequent cycles use cached metrics if price hasn't drifted

Implement DriftComparator class to cache per-symbol price/metrics state and
conditionally fetch full OHLCV data only when price drifts >0.5% (or 0.2% in
high-vol regime) or data exceeds 30-minute staleness. Modifies run_once() to
use multi-quote (1 API call) first, only full-fetching symbols that trigger
drift thresholds. Adds refresh_open_order_metrics() method for on-demand order
pricing. Typical cycle reduces from 8 API calls to 1 when market is flat.
…blobs

- AllocationFeedClient reads from market-quotes and options-chain blob stores
- normalize_feed_quotes handles both live Alpaca format (bid/ask/mid) and
  legacy format (bid_price/ask_price/midpoint_price), skips _meta entries
- Options chain client handles per-symbol keyed blobs (e.g. IWN/timestamp)
- 12 offline tests (mock data) + 4 live tests (real Netlify blobs) all pass
Copy link
Copy Markdown
Owner Author

Code Review — PR #51

Overall: Strong optimization — reducing API calls from 8+ to 1 in flat markets is a meaningful improvement. The DriftComparator is well-designed with clean separation and thorough tests (14 unit + integration tests).

Issues

  1. No fallback if get_multi_quote fails — If the multi-quote call raises or returns empty, quotes will be empty/None and the entire symbol loop will skip full fetches (or crash on quotes.get(symbol)). Add a try/except that falls back to full-fetch-all:

    try:
        quotes = self.data_provider.get_multi_quote(self.symbols)
    except Exception:
        quotes = {}
        needs_fetch = list(self.symbols)  # fallback: fetch everything
  2. Cached path skips state_manager market data — When using cached metrics, self.state_manager.update_metrics(symbol, metrics) is called, but the state manager may also depend on market_data (e.g., for daily bars used in drift metrics). Verify that _compute_drift_metrics() still works correctly when some symbols only have cached metrics.

  3. drift_metrics computed before symbol loop but also used by it_compute_drift_metrics() reads cached daily bars, but those bars only exist after fetch_market_data runs. On the first cycle this is fine (all symbols do full fetch), but on subsequent cycles the drift metrics will be stale for symbols using cached data. Consider if that's acceptable.

  4. refresh_open_order_metrics duplicates fetch logic — The full-fetch-or-cache pattern in refresh_open_order_metrics is identical to the main loop. Consider extracting a _fetch_or_cache_metrics(symbol, price) helper to avoid the duplication.

Strengths

  • Staleness window (30 min) is a sensible safety net
  • High-vol regime using tighter thresholds is smart
  • Rate limiting only on full-fetch symbols saves time
  • Integration test with Netlify blob format is thorough

Generated by Claude Code

@IamJasonBian
Copy link
Copy Markdown
Owner Author

Comparator might not be the best use case here - reviewing later. Let's implement for another use case

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant