Skip to content

Repository files navigation

Stock Market RAG Bot

Python Java Gemini License: MIT

A WhatsApp bot that answers stock market questions in plain English, Hindi, or Hinglish. Built around a small RAG pipeline over daily-refreshed market data, with intent detection by an LLM and a fallback to live yfinance lookups.

The user texts the bot on WhatsApp ("TCS price today", "BSE top gainers", "Reliance vs HDFC last month") and gets a structured answer back, optionally with an Excel report attached.

Why this exists

Indian retail investors check stock prices dozens of times a day. The default tools — Moneycontrol, ET Markets, broker apps — bury the answer behind ads, popups, and notifications. WhatsApp is already open on every Indian phone. A chatbot that answers "TCS price today" in two seconds, in the language the user actually thinks in, removes ten taps from the daily loop.

Features

  • Live NSE / BSE / US stock prices via yfinance
  • Excel reports with current prices, volume, high/low, and trend columns — attached directly in the WhatsApp reply
  • LLM intent detection in English, Hindi, and Hinglish (mixed-script messages like "Reliance ka price kya hai?" parse correctly)
  • Top-gainer / top-loser analysis with brief explanations from Google Gemini
  • Daily scheduled ingestion after the IST close (18:00 Asia/Kolkata) via APScheduler so morning queries hit fresh data
  • FAISS vector search for "give me historical context for X" style follow-ups
  • Two implementations in this repo:
    • Python (FastAPI) on main and feature/news-integration
    • Java (Spring Boot) on feature/java-conversion for production-grade deployment

Architecture

+---------------------+          +------------------------+
| User (WhatsApp)     |  text -> | Twilio WhatsApp        |
+---------------------+          | sandbox                |
                                 +-----------+------------+
                                             |
                                             | webhook POST
                                             v
                              +--------------+--------------+
                              | FastAPI / Spring Boot       |
                              | /webhook/whatsapp           |
                              +--------------+--------------+
                                             |
                              +--------------+--------------+
                              | LLM intent classification   |
                              | (Gemini 2.5 flash-lite)     |
                              +--------------+--------------+
                                             |
                  +--------------------------+--------------------------+
                  |                          |                          |
                  v                          v                          v
        +------------------+      +----------------------+    +-------------------+
        | Live price query |      | RAG over daily data  |    | Top-gainer/loser  |
        | yfinance / NSE   |      | FAISS + MiniLM-L6-v2 |    | analysis + Gemini |
        +------------------+      +----------------------+    | explanation       |
                                                              +-------------------+
                  |                          |                          |
                  +--------------------------+--------------------------+
                                             |
                                             v
                              +--------------+--------------+
                              | Reply: text + optional      |
                              | xlsx attachment             |
                              +-----------------------------+

   APScheduler @ 18:00 IST -----> Daily ingest -----> FAISS index rebuild

Tech stack

Layer Tech
Web framework FastAPI (Python 3.10+) / Spring Boot (Java 21)
LLM Google Gemini (gemini-2.5-flash-lite)
Embeddings sentence-transformers/all-MiniLM-L6-v2
Vector store FAISS (FlatL2)
Stock data yfinance, NSE Direct API
Messaging Twilio WhatsApp
Tunnelling (dev) ngrok
Scheduling APScheduler / Spring @Scheduled

Repo layout

.
+-- main.py                 FastAPI entry point
+-- api/                    Route handlers, models, query logging
+-- ingest/                 Daily data refresh jobs
+-- rag/                    Embedding + FAISS index code
+-- news/                   News integration for context-aware answers
+-- scheduler.py            APScheduler wiring
+-- run.sh / run_with_ngrok.py
+-- config.py               Loads env + YAML config
+-- requirements.txt
+-- Dockerfile, docker-compose.yml
+-- java-app/               Spring Boot rewrite (on feature/java-conversion)
+-- data/, logs/            Runtime artefacts (gitignored)

Prerequisites

  • Python 3.10+ (or Java 21 + Maven for the Spring Boot path)
  • A Twilio account (the free sandbox works)
  • A Google AI API key for Gemini
  • An ngrok account (free tier works)

Setup (Python)

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
cp .env.example .env
# fill in TWILIO_*, GOOGLE_API_KEY, NGROK_AUTHTOKEN

Then:

./run.sh                    # uvicorn on :8000
# or
python run_with_ngrok.py    # starts uvicorn + opens an ngrok tunnel

Configure your Twilio WhatsApp sandbox webhook to point at the ngrok URL (POST /webhook/whatsapp).

Setup (Java / Spring Boot)

git checkout feature/java-conversion
cd java-app
./start.sh                  # mvn package + java -jar

The Java app reads the same .env keys via spring-dotenv.

Endpoints

Method Path Purpose
POST /webhook/whatsapp Twilio WhatsApp incoming-message webhook
GET /api/stock/{symbol} Live price + day stats for a symbol
GET /api/historical/{symbol} Historical close prices (date range params)
GET /api/reports/daily.xlsx Generated daily report
GET /healthz Liveness probe

Branches

  • main — original Python FastAPI implementation
  • feature/news-integration — adds news-source enrichment to RAG context
  • feature/java-conversion — Spring Boot rewrite (this is the actively developed branch)

Key design decisions

Why an LLM for intent detection instead of a regex / keyword router? The mix of English, Hindi, and Hinglish makes a regex router brittle — users write "Reliance ka price kya hai?", "रिलायंस price", "rel price plz", and "reliance share value today" interchangeably. A small LLM (gemini-2.5-flash-lite) is cheap enough per call to justify the precision win.

Why FAISS + MiniLM-L6-v2 specifically? MiniLM gives 384-dim embeddings at ~10ms on CPU, which is plenty for a sub-second total response budget. FAISS FlatL2 (exact, no approximate index) is the right choice when the corpus is small (daily ingest, single-day window) — building an HNSW index every evening is more code for no recall gain at this scale. If the corpus ever grows to a multi-year history, swap the FlatL2 for IVFFlat and the rest stays the same.

Why two implementations (FastAPI and Spring Boot)? Python prototypes faster — the main branch was running end-to-end in a weekend. Java/Spring Boot is what the production stack at most Indian fintech employers actually runs on. Re-implementing the same service in both was the exercise: confirm the architecture survives a language port, prove I can ship the JVM stack, and pick up the Spring idioms (@Scheduled, RestTemplate, dependency injection) that are interview-relevant.

Why daily ingest after IST close rather than per-query? The NSE closes at 15:30 IST and prices don't change overnight. Pre-warming the daily report once at 18:00 means morning queries hit a cached file in tens of milliseconds instead of waiting for yfinance to round-trip. The per-symbol live-price path is still on yfinance direct for queries that ask for a single ticker right now.

Why WhatsApp and not Telegram / Slack / a web UI? Indian penetration. WhatsApp is the default messaging app — no install, no sign-up, no app-store discovery. The same bot on Telegram would require the user to find and add it; the same logic in a web app would lose to the user's existing broker app's home screen.

Known limitations

  • NSE Direct API is unofficial. It can rate-limit, change response shape, or go down without warning. yfinance is the more stable backstop.
  • Twilio WhatsApp sandbox caps daily messages and requires recipients to opt in by texting a code. Production deployment needs a Twilio business account with a verified WhatsApp Business profile — non-trivial paperwork.
  • No persistent conversation memory. Each query is independent; the bot doesn't remember "the Reliance you asked about earlier." Adding it requires per-user session state in Redis or similar.
  • Gemini cost. Each message hits the API once for intent. At ~$0.05 per 1K queries on flash-lite, free-tier breakeven is around 2-3K queries per month before going paid.
  • News integration on feature/news-integration is rough. It pulls headlines but doesn't yet rank for relevance to the user's query.

Status

This project runs locally (uvicorn or Spring Boot) and exposes itself via ngrok. There is no live deployment. The Twilio sandbox limits are sufficient for personal use.

License

MIT.

About

WhatsApp stock-market bot in Python (FastAPI) and Java (Spring Boot) with RAG, FAISS, Gemini, and yfinance. Answers price queries in English, Hindi, and Hinglish.

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages