Real price data: yfinance re-ingest + Interactive Brokers source - #63
Open
digitalflanker-ux wants to merge 2 commits into
Open
Real price data: yfinance re-ingest + Interactive Brokers source#63digitalflanker-ux wants to merge 2 commits into
digitalflanker-ux wants to merge 2 commits into
Conversation
Adds an IBKRDataSource (ib_insync reqHistoricalData) for broker-grade daily bars,
selectable for ongoing ingestion. yfinance remains the default (best for the
full historical backfill); Robinhood is execution-only and not added.
- hrp/data/sources/ibkr_source.py: daily OHLCV via IB Gateway/TWS, pacing-aware
multi-symbol loop, env-config (IBKR_HOST/PORT/IBKR_DATA_CLIENT_ID/
HRP_IBKR_PACE_SECONDS); ib_insync imported lazily (optional `trading` extra).
- factory: register "ibkr" (IBKR primary, yfinance fallback).
- ingest_prices: add `ibkr` source branch (DataSourceBase typing).
- run_job: `--price-source {yfinance,polygon,ibkr}` threads to the prices job.
- tests: schema mapping, duration, factory, injected-client path (no Gateway
needed); live reqHistoricalData verified manually against a running Gateway.
- .env.example + runbook document IBKR usage.
Note: the synthetic prices were separately replaced with real yfinance history
(prices + features re-ingested) as the immediate fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011BFeBAuxWTFRayNGMD7VDn
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011BFeBAuxWTFRayNGMD7VDn
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Replaces the synthetic price data and adds Interactive Brokers as an ongoing data source. (Robinhood is execution-only and a poor research source, so it was not added.)
Immediate fix (done, not a code change)
Wiped the synthetic prices/features and re-ingested ~3 years of real, adjusted daily history from yfinance for the full universe, then recomputed features:
This PR — IBKR historical source (for ongoing)
hrp/data/sources/ibkr_source.py— daily OHLCV viaib_insyncreqHistoricalDataagainst IB Gateway/TWS; pacing-aware multi-symbol loop; env-config (IBKR_HOST/PORT,IBKR_DATA_CLIENT_ID,HRP_IBKR_PACE_SECONDS).ib_insyncis imported lazily (optionaltradingextra), so importing/registering it needs no install."ibkr"(IBKR primary, yfinance fallback).ingest_pricesgains anibkrbranch;run_job --price-source {yfinance,polygon,ibkr}threads it to the prices job..env.example+ the runbook document usage.Verification
reqHistoricalDataround-trip needs a running IB Gateway + market-data permissions, which aren't available in CI — that path is verified manually, not in tests. IBKRTRADESbars are unadjusted, soadj_closemirrorsclose.🤖 Generated with Claude Code
https://claude.ai/code/session_011BFeBAuxWTFRayNGMD7VDn
Need help on this PR? Tag
/codesmithwith what you need. Autofix is disabled.