Health Query Chatbot (Adaptive Multi-LLM)

Production-grade AI health query service with adaptive routing, NVIDIA LLM ensemble reasoning, and Gemini final aggregation.

Architecture

1. Classifier (fast NVIDIA model) labels query type, complexity, risk, and routing needs.
2. Router decides between fast response, ensemble reasoning, or emergency bypass.
3. Ensemble runs multiple NVIDIA models in parallel and validates strict JSON outputs.
4. Gemini Judge aggregates only overlapping, safe conclusions and adds a medical disclaimer.
5. Safety enforces strict medical constraints and emergency escalation.

Project Structure

health_ai/
+-- app.py
+-- router/
|   +-- classifier.py
|   +-- decision.py
+-- models/
|   +-- nvidia_client.py
|   +-- nemotron.py
|   +-- mistral.py
|   +-- qwen.py
+-- ensemble/
|   +-- runner.py
|   +-- validator.py
|   +-- normalizer.py
+-- judge/
|   +-- gemini.py
+-- safety/
|   +-- rules.py
+-- schemas/
|   +-- ensemble_response.json
+-- utils/
|   +-- env.py
|   +-- logger.py
+-- .env.example
+-- requirements.txt
+-- README.md

Setup

1. Create and activate a virtual environment.
2. Install dependencies:

pip install -r health_ai/requirements.txt

3. Create health_ai/.env from health_ai/.env.example and set your keys. You can set a single NVIDIA_API_KEY for all models or override per model with NVIDIA_API_KEY_*.

Run

uvicorn health_ai.app:app --host 0.0.0.0 --port 8000 --reload

API

• GET /health
• POST /query

Example:

curl -X POST http://localhost:8000/query \
  -H "Content-Type: application/json" \
  -d '{"query":"I have a headache and mild nausea for two days"}'

Safety Guarantees

• No diagnoses, prescriptions, or medication names.
• Emergency queries bypass models and return immediate escalation.
• All responses include a medical disclaimer.

Environment Variables

NVIDIA_API_KEY=...
NVIDIA_API_KEY_CLASSIFIER=...
NVIDIA_API_KEY_FAST=...
NVIDIA_API_KEY_NEMOTRON=...
NVIDIA_API_KEY_MISTRAL=...
NVIDIA_API_KEY_QWEN=...
NVIDIA_BASE_URL=https://integrate.api.nvidia.com/v1
NVIDIA_TIMEOUT_S=20
GEMINI_API_KEY=...
GEMINI_MODEL=gemini-1.5-pro
LOG_LEVEL=INFO

Notes

• Ensemble outputs are strictly validated against schemas/ensemble_response.json.
• If any model flags high severity, the system escalates immediately.
• Gemini aggregation only uses overlapping, safe content.

Security

• No API keys are stored in code.
• Use environment variables or a secrets manager in production.