Starter Agent

A reference project showing the same tool-using AI agent built three ways — hand-rolled, LangChain, and DSPy — plus real RAG and a FastAPI service. Built for client work: clear, framework-agnostic, and runnable with no API key.

What's inside

agent/
  tools.py        # tool fns + JSON schemas (calculator, time, RAG search)
  rag.py          # REAL vector store: Chroma + local MiniLM embeddings
  llm.py          # LLM client; real OpenAI-compatible API or offline MockLLM
  core.py         # hand-rolled ReAct loop (the "from scratch" version)
  api.py          # FastAPI service: /chat, /rag/search, /sessions, /health
  lc_agent.py     # SAME agent via LangChain + LangGraph (comparison)
  dspy_agent.py   # SAME agent via DSPy ReAct (comparison)
  main.py         # CLI entry point
tests/            # 15 tests across all components

The offline-first design (why this matters for demos)

Every component runs with no API key and no network (after first-run model cache), so you can demo on a plane or in CI:

Layer	Real backend	Offline fallback
LLM	OpenAI-compatible API (OPENAI_API_KEY)	rule-based MockLLM
Embeddings	Chroma MiniLM (onnx, local)	deterministic hash embedder
LangChain	ChatOpenAI	MockChatModel adapter
DSPy	dspy.LM (litellm)	OfflineRoutingLM

Set OPENAI_API_KEY (and optionally OPENAI_BASE_URL, AGENT_MODEL) to switch everything to a real model. Because clients are OpenAI-compatible, it works with vLLM, llama.cpp, Groq, OpenRouter, Together, etc.

Quick start

pip install -r requirements.txt

# Hand-rolled agent (CLI)
python -m agent.main "what is 3 * (4 + 1)?"
python -m agent.main "how long until I get my money back?"   # semantic RAG

# Framework comparisons — same tools, same RAG
python -m agent.lc_agent   "tell me about refunds"
python -m agent.dspy_agent "what is 7 * 8?"

# FastAPI service
uvicorn agent.api:app --port 8000
#   curl localhost:8000/health
#   curl -X POST localhost:8000/chat     -H 'Content-Type: application/json' -d '{"message":"what time is it?"}'
#   curl -X POST localhost:8000/rag/search -H 'Content-Type: application/json' -d '{"query":"my package is late","k":2}'

# Tests (all offline)
python -m pytest -q

RAG: real semantic retrieval

agent/rag.py uses Chroma with local MiniLM embeddings, so queries match by meaning, not keywords:

• "how long until I get my money back?" → refund doc
• "when can I call you?" → hours doc
• "my package hasn't arrived" → shipping doc

Swap SEED_DOCS for the client's real content (chunked docs/PDFs), or point at a hosted vector DB (pgvector, Pinecone) by changing the client in KnowledgeBase.

Three implementations, one behavior — pick per client

• core.py (hand-rolled): zero framework lock-in, total control, easiest to debug and explain. Great default for bespoke client agents.
• lc_agent.py (LangChain/LangGraph): ecosystem of integrations, prebuilt ReAct, graph-based control flow for complex multi-step agents.
• dspy_agent.py (DSPy): declarative signatures; lets you optimize prompts against metrics instead of hand-tuning. Strong for accuracy-critical pipelines.

All three share agent/tools.py and agent/rag.py, so you can compare ergonomics without re-implementing the substrate.

Adapt for a client

1. Add a tool: implement the fn, register in TOOLS + TOOL_SCHEMAS (it's automatically available to all three agent versions).
2. Load real docs into agent/rag.py.
3. Set the persona/policy in SYSTEM_PROMPT (core.py) / the DSPy signature.
4. Deploy the FastAPI service (add Redis/DB-backed sessions for production — the in-memory _SESSIONS dict is the only thing to swap).