Skip to content

feat: migrate scanner news screener to support Tavily alongside Perplexity#2

Open
tavily-integrations wants to merge 2 commits into
samuelvinay91:mainfrom
Tavily-FDE:feat/tavily-migration/scanner-news-screener
Open

feat: migrate scanner news screener to support Tavily alongside Perplexity#2
tavily-integrations wants to merge 2 commits into
samuelvinay91:mainfrom
Tavily-FDE:feat/tavily-migration/scanner-news-screener

Conversation

@tavily-integrations
Copy link
Copy Markdown

Summary

  • Added Tavily web search as an optional parallel news screener in the scanner engine
  • When SKOPAQ_TAVILY_API_KEY is configured, a TavilyClient is initialized and runs a news-topic search alongside existing Perplexity Sonar and Grok screeners
  • Tavily results are parsed directly from structured JSON into ScannerCandidate objects (no LLM post-processing needed)
  • Candidates are tagged source='news_tavily' and merged/deduplicated by existing logic
  • Perplexity Sonar screener remains fully intact — both can run in parallel

Files changed

  • skopaq/scanner/engine.py — Added _tavily_client init, _screen_news_tavily() method, conditional task in _run_screeners(), updated _screener_info()
  • skopaq/scanner/screen.py — Added build_tavily_news_query() and parse_tavily_results() helpers
  • skopaq/config.py — Added tavily_api_key field to SkopaqConfig
  • skopaq/llm/env_bridge.py — Added tavily_api_key → TAVILY_API_KEY bridge mapping
  • .env.example — Added SKOPAQ_TAVILY_API_KEY= placeholder
  • requirements.txt — Added tavily-python>=0.5.0

Dependency changes

  • Added tavily-python>=0.5.0 to requirements.txt

Environment variable changes

  • Added SKOPAQ_TAVILY_API_KEY (optional; Tavily screener disabled when absent)
  • Bridged to TAVILY_API_KEY via env_bridge.py

Notes for reviewers

  • Tavily screener is additive — it only activates when the API key is present
  • All 33 existing sync scanner tests pass; async test failures are pre-existing (missing pytest-asyncio in CI venv)
  • Tavily search() is called with topic="news" and search_depth="basic" for fast, cost-efficient news retrieval

🤖 Generated with Claude Code

Automated Review

  • Passed after 2 attempt(s)
  • Final review: The attempt-2 implementation correctly addresses all 4 issues raised in the prior review. AsyncTavilyClient is used and awaited properly, comprehensive tests cover both screen.py helpers and engine.py integration, import re is at module level, and the symbol cap is tightened to 10. Config, env_bridge, requirements.txt, and .env.example are all updated consistently. No critical or major issues found. Three minor observations noted below.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant