IQFeed Data Downloader Exception #13

omidkrad · 2024-07-19T01:49:24Z

Expected Behavior

Able to download options data with IQFeed

Actual Behavior

Runtime exception looking up IQFeed Symbols

20240719 00:41:18.959 ERROR:: IQFeedFileHistoryProvider.ProcessHistoryRequests: Historical data request with
TickType 'Quote' is not supported for resolutions other than Tick. Requested Resolution: Minute

Potential Solution

Review and address

Reproducing the Problem

Related post: https://www.quantconnect.com/forum/discussion/17616/problem-with-download-backtesting-options-data-with-lean-cli-and-iqfeed/p1
Also see below.

System Information

Windows 10

Checklist

I have completely filled out this template
I have confirmed that this issue exists on the current master branch
I have confirmed that this is not a duplicate issue by searching issues
I have provided detailed steps to reproduce the issue

The text was updated successfully, but these errors were encountered:

omidkrad · 2024-07-19T01:51:49Z

---- REPRO STEPS ----
Using IQFeed data downloader fails. Here's the algo to reproduce:

using System;
using QuantConnect;
using QuantConnect.Algorithm;
using QuantConnect.Brokerages;
using QuantConnect.Data;

public class SampleOptionAlgo : QCAlgorithm
{
    public override void Initialize()
    {
        var endDate = DateTime.Today.AddDays(-1);
        var startDate = endDate.AddDays(-3);
        SetStartDate(startDate);
        SetEndDate(endDate);

        SetCash(100000);
        SetBrokerageModel(BrokerageName.InteractiveBrokersBrokerage, AccountType.Margin);

        var equity = AddEquity("SPY", Resolution.Minute).Symbol;
        Securities[equity].SetDataNormalizationMode(DataNormalizationMode.Raw);
        SetBenchmark(equity);

        var option = AddOption(equity, Resolution.Minute);
        option.SetFilter(-5, 5, TimeSpan.FromDays(20), TimeSpan.FromDays(60));
    }

    public override void OnData(Slice data)
    {
        foreach (var (symbol, optionChain) in data.OptionChains)
        {
            foreach (var option in optionChain)
            {
                Log($"Received {option}");
            }
        }
    }

}

(1) I made sure to have correct data subscriptions with IQFeed and that they are active.

DTN IQFeed Core Service
OPRA
RT Equity Options-Investor
NASDAQ Level 1
NYSE

(2) Installed IQFeed Client (iqfeed_client_6_2_1_20.exe)

(3) Ran C:\Program Files\DTN\IQFeed\iqlink.exe and connected the IQFeed Client.

(4) After setting up Lean CLI to use IQFeed, lean.json has the followings set:

    "id": "IQFeed",
    "data-provider": "QuantConnect.Lean.Engine.DataFeeds.DownloaderDataProvider",
    "data-downloader": "QuantConnect.Lean.DataSource.IQFeed.IQFeedDataDownloader",
    "iqfeed-username": "<iqfeed username>",
    "iqfeed-password": "<iqfeed password>",
    "iqfeed-productName": "IQFEED_DIAGNOSTICS",
    "iqfeed-version": "1.0",
    "iqfeed-host": "host.docker.internal",

(3) I run the algo with the following command:

lean backtest SampleOptionAlgo --data-provider-historical "IQFeed" --backtest-name qc-iqfeed-backtest-01

After running the algo, it shows that the stock data for the last 3 days get downloaded but option data fails to download. Also can confirm that from the downloaded files in the data folder.

From the logs:

DATA USAGE:: Total data requests 36
DATA USAGE:: Succeeded data requests 4
DATA USAGE:: Failed data requests 32

There is one error during execution:

20240719 00:41:18.959 ERROR:: IQFeedFileHistoryProvider.ProcessHistoryRequests: Historical data request with
TickType 'Quote' is not supported for resolutions other than Tick. Requested Resolution: Minute

omidkrad · 2024-07-19T02:00:47Z

Full log:

Package source with Name: Package source 1 added successfully.
The template "Solution File" was created successfully.
The template "Class Library" was created successfully.
  Determining projects to restore...
  Writing /tmp/tmpeOrTuo.tmp
info : Adding PackageReference for package 'QuantConnect.Lean.DataSource.IQFeed' into project
'/ModulesProject/ModulesProject.csproj'.
info : Restoring packages for /ModulesProject/ModulesProject.csproj...
info : Installed QuantConnect.Lean.DataSource.IQFeed 2.5.16529 from /Modules with content hash
4wLapfAelIX2H7wZ+k20Rd4ih2MhdLZCuEs8kt+Es1dDCI8ucRcfiysP6uBN726dyOiJZsIa4MrVQwkXGQeHbQ==.
info : Package 'QuantConnect.Lean.DataSource.IQFeed' is compatible with all the specified frameworks in project
'/ModulesProject/ModulesProject.csproj'.
info : PackageReference for package 'QuantConnect.Lean.DataSource.IQFeed' version '2.5.16529' added to file
'/ModulesProject/ModulesProject.csproj'.
info : Committing restore...
info : Generating MSBuild file /Compile/obj/ModulesProject/ModulesProject.csproj.nuget.g.props.
info : Generating MSBuild file /Compile/obj/ModulesProject/ModulesProject.csproj.nuget.g.targets.
info : Writing assets file to disk. Path: /Compile/obj/ModulesProject/project.assets.json
log  : Restored /ModulesProject/ModulesProject.csproj (in 2.58 sec).
Microsoft (R) Build Engine version 17.0.1+b177f8fa7 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  Restored /LeanCLI/SampleOptionAlgo/SampleOptionAlgo.csproj (in 193 ms).
  SampleOptionAlgo -> /Compile/bin/SampleOptionAlgo.dll

Build succeeded.
    0 Warning(s)
    0 Error(s)

Time Elapsed 00:00:06.87
20240719 00:40:58.145 TRACE:: Config.Get(): Configuration key not found. Key: version-id - Using default value:
20240719 00:40:58.158 TRACE:: Config.Get(): Configuration key not found. Key: cache-location - Using default value:
20240719 00:40:58.159 TRACE:: Config.GetValue(): project-id - Using default value: 0
20240719 00:40:58.162 TRACE:: Config.Get(): Configuration key not found. Key: plugin-directory - Using default value:
20240719 00:40:58.179 TRACE:: Composer(): Loading Assemblies from /Lean/Launcher/bin/Debug
20240719 00:40:58.352 TRACE:: Python for .NET Assembly: Python.Runtime, Version=2.0.38.0, Culture=neutral,
PublicKeyToken=5000fea6cba702dd
20240719 00:40:58.432 TRACE:: Engine.Main(): LEAN ALGORITHMIC TRADING ENGINE v2.5.0.0 Mode: DEBUG (64bit)
20240719 00:40:58.585 TRACE:: Engine.Main(): Started 12:40 AM
20240719 00:40:58.602 TRACE:: Config.Get(): Configuration key not found. Key: lean-manager-type - Using default value:
LocalLeanManager
20240719 00:40:58.645 TRACE:: JobQueue.NextJob(): Selected SampleOptionAlgo.dll
20240719 00:40:58.779 TRACE:: Config.GetValue(): scheduled-event-leaky-bucket-capacity - Using default value: 120
20240719 00:40:58.782 TRACE:: Config.GetValue(): scheduled-event-leaky-bucket-time-interval-minutes - Using default
value: 1440
20240719 00:40:58.798 TRACE:: Config.GetValue(): scheduled-event-leaky-bucket-refill-amount - Using default value: 18
20240719 00:40:58.800 TRACE:: Config.GetValue(): storage-limit - Using default value: 10737418240
20240719 00:40:58.804 TRACE:: Config.GetValue(): storage-permissions - Using default value: 3
20240719 00:40:58.805 TRACE:: Config.Get(): Configuration key not found. Key: optimization-id - Using default value:
20240719 00:40:58.807 TRACE:: Config.Get(): Configuration key not found. Key: backtest-name - Using default value: local
20240719 00:40:58.823 TRACE:: Config.Get(): Configuration key not found. Key: data-channel - Using default value:
20240719 00:40:58.824 TRACE:: Config.Get(): Configuration key not found. Key: python-venv - Using default value:
20240719 00:40:58.825 TRACE:: Config.Get(): Configuration key not found. Key: out-of-sample-max-end-date - Using default
value:
20240719 00:40:58.826 TRACE:: Config.GetValue(): out-of-sample-days - Using default value: 0
20240719 00:40:58.837 TRACE:: Config.Get(): Configuration key not found. Key: data-permission-manager - Using default
value: DataPermissionManager
20240719 00:40:58.933 TRACE:: Config.GetValue(): downloader-data-update-period - Using default value: 7
20240719 00:41:00.340 TRACE:: Config.GetValue(): zip-data-cache-provider - Using default value: 10
20240719 00:41:00.349 TRACE:: Config.Get(): Configuration key not found. Key: fundamental-data-provider - Using default
value: CoarseFundamentalDataProvider
20240719 00:41:00.364 TRACE:: AlgorithmManager.CreateTokenBucket(): Initializing LeakyBucket: Capacity: 120
RefillAmount: 18 TimeInterval: 1440
20240719 00:41:00.367 TRACE:: Config.GetValue(): algorithm-manager-time-loop-maximum - Using default value: 20
20240719 00:41:00.435 TRACE:: Engine.Run(): Resource limits '0' CPUs. 2147483647 MB RAM.
20240719 00:41:00.441 TRACE:: TextSubscriptionDataSourceReader.SetCacheSize(): Setting cache size to 71582788 items
20240719 00:41:00.458 TRACE:: Config.GetValue(): algorithm-creation-timeout - Using default value: 90
20240719 00:41:00.465 TRACE:: Loader.TryCreateILAlgorithm(): Loading only the algorithm assembly
20240719 00:41:00.584 TRACE:: Config.Get(): Configuration key not found. Key: databases-refresh-period - Using default
value: 1.00:00:00
20240719 00:41:01.018 TRACE:: Loader.TryCreateILAlgorithm(): Loaded SampleOptionAlgo
20240719 00:41:01.029 TRACE:: LocalObjectStore.Initialize(): Storage Root: /Storage. StorageFileCount 9999999.
StorageLimit 10240MB
20240719 00:41:01.050 TRACE:: Config.GetValue(): zip-data-cache-provider - Using default value: 10
20240719 00:41:01.078 TRACE:: HistoryProviderManager.Initialize(): history providers
[SubscriptionDataReaderHistoryProvider]
20240719 00:41:01.085 TRACE:: BacktestingSetupHandler.Setup(): Setting up job: UID: 25898, PID: 0, Version: 2.5.0.0,
Source: WebIDE
20240719 00:41:01.106 TRACE:: Config.Get(): Configuration key not found. Key: security-data-feeds - Using default value:
20240719 00:41:01.503 TRACE:: Config.GetValue(): data-feed-workers-count - Using default value: 8
20240719 00:41:01.510 TRACE:: Config.GetValue(): data-feed-max-work-weight - Using default value: 400
20240719 00:41:01.511 TRACE:: WeightedWorkScheduler(): will use 8 workers and MaxWorkWeight is 400
20240719 00:41:01.570 TRACE:: BaseSetupHandler.SetupCurrencyConversions():
Account Type: Margin

Symbol      Quantity    Conversion = Value in USD
USD: $      100000.00 @       1.00 = $100000
-------------------------------------------------
CashBook Total Value:                $100000

20240719 00:41:01.584 TRACE:: Total margin information: TotalMarginUsed: 0.00, MarginRemaining: 100000.00
20240719 00:41:01.586 TRACE:: SetUp Backtesting: User: 25898 ProjectId: 0 AlgoId: 1803020173
20240719 00:41:01.588 TRACE:: Dates: Start: 07/15/2024 End: 07/17/2024 Cash: ¤100,000.00 MaximumRuntime: 100.00:00:00
MaxOrders: 2147483647
20240719 00:41:01.598 TRACE:: BacktestingResultHandler(): Sample Period Set: 04.00
20240719 00:41:01.604 TRACE:: Config.GetValue(): forward-console-messages - Using default value: True
20240719 00:41:01.607 TRACE:: JOB HANDLERS:
         DataFeed:             QuantConnect.Lean.Engine.DataFeeds.FileSystemDataFeed
         Setup:                QuantConnect.Lean.Engine.Setup.BacktestingSetupHandler
         RealTime:             QuantConnect.Lean.Engine.RealTime.BacktestingRealTimeHandler
         Results:              QuantConnect.Lean.Engine.Results.BacktestingResultHandler
         Transactions:         QuantConnect.Lean.Engine.TransactionHandlers.BacktestingTransactionHandler
         Object Store:         QuantConnect.Lean.Engine.Storage.LocalObjectStore
         History Provider:     QuantConnect.Lean.Engine.HistoricalData.HistoryProviderManager
         Brokerage:            QuantConnect.Brokerages.Backtesting.BacktestingBrokerage
         Data Provider:        QuantConnect.Lean.Engine.DataFeeds.DownloaderDataProvider

20240719 00:41:01.693 TRACE:: Debug: Launching analysis for 1803020173 with LEAN Engine v2.5.0.0
20240719 00:41:01.701 TRACE:: Event Name "Daily Sampling", scheduled to run.
20240719 00:41:01.708 TRACE:: AlgorithmManager.Run(): Begin DataStream - Start: 7/15/2024 12:00:00 AM Stop: 7/17/2024
11:59:59 PM Time: 7/15/2024 12:00:00 AM Warmup: False
20240719 00:41:02.135 TRACE:: Config.GetValue(): zip-data-cache-provider - Using default value: 10
20240719 00:41:02.433 TRACE:: IQFeedDataQueueUniverseProvider.LoadSymbols: Loading IQFeed futures symbol map file...
20240719 00:41:02.445 TRACE:: IQFeedDataQueueUniverseProvider.LoadSymbols: Loading and unzipping IQFeed symbol universe
file (0)...
20240719 00:41:18.436 TRACE:: IQFeedFileHistoryProvider.ProcessHistoryRequests: Unsupported SecurityType 'Option' for
symbol 'Option'
20240719 00:41:18.959 ERROR:: IQFeedFileHistoryProvider.ProcessHistoryRequests: Historical data request with
TickType 'Quote' is not supported for resolutions other than Tick. Requested Resolution: Minute
20240719 00:41:18.984 TRACE:: UniverseSelection.AddPendingInternalDataFeeds(): Adding internal benchmark data
feed SPY,#0,SPY,Hour,TradeBar,Trade,Adjusted,OpenInterest,Internal
20240719 00:41:19.353 TRACE:: Synchronizer.GetEnumerator(): Exited thread.
20240719 00:41:19.354 TRACE:: AlgorithmManager.Run(): Firing On End Of Algorithm...
20240719 00:41:19.355 TRACE:: Engine.Run(): Exiting Algorithm Manager
20240719 00:41:19.369 TRACE:: StopSafely(): Waiting for 'Isolator Thread' thread to stop...
20240719 00:41:19.371 TRACE:: FileSystemDataFeed.Exit(): Start. Setting cancellation token...
20240719 00:41:19.384 TRACE:: FileSystemDataFeed.Exit(): Exit Finished.
20240719 00:41:19.386 TRACE:: BacktestingResultHandler.Exit(): starting...
20240719 00:41:19.387 TRACE:: BacktestingResultHandler.Exit(): Saving logs...
20240719 00:41:19.404 TRACE:: StopSafely(): Waiting for 'Result Thread' thread to stop...
20240719 00:41:19.541 TRACE:: Debug: Algorithm Id:(1803020173) completed in 17.74 seconds at 0k data points per second.
Processing total of 1,185 data points.
20240719 00:41:19.542 TRACE:: Debug: Your log was successfully created and can be retrieved from:
/Results/1803020173-log.txt
20240719 00:41:19.543 TRACE:: BacktestingResultHandler.Run(): Ending Thread...
20240719 00:41:19.594 TRACE:: DownloaderDataProvider.Get(): custom data is not supported, requested: interest-rate 2S
20240719 00:41:19.933 TRACE::
STATISTICS:: Total Orders 0
STATISTICS:: Average Win 0%
STATISTICS:: Average Loss 0%
STATISTICS:: Compounding Annual Return 0%
STATISTICS:: Drawdown 0%
STATISTICS:: Expectancy 0
STATISTICS:: Start Equity 100000
STATISTICS:: End Equity 100000
STATISTICS:: Net Profit 0%
STATISTICS:: Sharpe Ratio 0
STATISTICS:: Sortino Ratio 0
STATISTICS:: Probabilistic Sharpe Ratio 0%
STATISTICS:: Loss Rate 0%
STATISTICS:: Win Rate 0%
STATISTICS:: Profit-Loss Ratio 0
STATISTICS:: Alpha 0
STATISTICS:: Beta 0
STATISTICS:: Annual Standard Deviation 0
STATISTICS:: Annual Variance 0
STATISTICS:: Information Ratio -16.709
STATISTICS:: Tracking Error 0.067
STATISTICS:: Treynor Ratio 0
STATISTICS:: Total Fees $0.00
STATISTICS:: Estimated Strategy Capacity $0
STATISTICS:: Lowest Capacity Asset
STATISTICS:: Portfolio Turnover 0%
STATISTICS:: OrderListHash d41d8cd98f00b204e9800998ecf8427e
20240719 00:41:19.935 TRACE:: BacktestingResultHandler.SendAnalysisResult(): Processed final packet
20240719 00:41:19.936 TRACE:: Engine.Run(): Disconnecting from brokerage...
20240719 00:41:19.938 TRACE:: Engine.Run(): Disposing of setup handler...
20240719 00:41:19.941 TRACE:: Engine.Main(): Analysis Completed and Results Posted.
20240719 00:41:19.945 TRACE:: StopSafely(): Waiting for '' thread to stop...
20240719 00:41:19.957 TRACE:: DataMonitor.GenerateReport():
DATA USAGE:: Total data requests 36
DATA USAGE:: Succeeded data requests 4
DATA USAGE:: Failed data requests 32
DATA USAGE:: Failed data requests percentage 89%
DATA USAGE:: Total universe data requests 0
DATA USAGE:: Succeeded universe data requests 0
DATA USAGE:: Failed universe data requests 0
DATA USAGE:: Failed universe data requests percentage 0%
Engine.Main(): Analysis Complete.
20240719 00:41:19.984 TRACE:: Engine.Main(): Packet removed from queue: 1803020173
20240719 00:41:19.984 TRACE:: LeanEngineSystemHandlers.Dispose(): start...
20240719 00:41:19.988 TRACE:: LeanEngineSystemHandlers.Dispose(): Disposed of system handlers.
20240719 00:41:19.989 TRACE:: LeanEngineAlgorithmHandlers.Dispose(): start...
20240719 00:41:19.991 TRACE:: LeanEngineAlgorithmHandlers.Dispose(): Disposed of algorithm handlers.
20240719 00:41:20.005 TRACE:: StopSafely(): Waiting for 'CpuPerformance' thread to stop...
20240719 00:41:20.006 TRACE:: Program.Main(): Exiting Lean...
Successfully ran 'SampleOptionAlgo' in the 'backtesting' environment and stored the output in
'SampleOptionAlgo\backtests\2024-07-18_17-40-22'

omidkrad mentioned this issue Jul 20, 2024

IBKR Data Downloader Exception QuantConnect/Lean.Brokerages.InteractiveBrokers#114

Open

4 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

IQFeed Data Downloader Exception #13

IQFeed Data Downloader Exception #13

omidkrad commented Jul 19, 2024 •

edited

Loading

omidkrad commented Jul 19, 2024

omidkrad commented Jul 19, 2024

IQFeed Data Downloader Exception #13

IQFeed Data Downloader Exception #13

Comments

omidkrad commented Jul 19, 2024 • edited Loading

Expected Behavior

Actual Behavior

Potential Solution

Reproducing the Problem

System Information

Checklist

omidkrad commented Jul 19, 2024

omidkrad commented Jul 19, 2024

omidkrad commented Jul 19, 2024 •

edited

Loading