Retrieval that decides what it can prove before a model speaks.
Ask a question about a pinned source. IntelliRAG finds the passages, checks whether they actually support an answer, packs them into a fixed token budget, and tells you which parts of your question the context never mentions. If the evidence isn't there, it refuses instead of improvising.
Try the live console · Interactive architecture · Browser walkthrough · Measured evaluation · Code graph
In plain words
- Offline, once per source: IntelliRAG reads your repository or documents, cuts them into passages and builds an index. Nothing here waits on a question.
- Online, per question: it finds the passages that match, ranks them, and lets only the ones that clear an evidence gate be cited. That's about 5 ms of search before any model runs. With no passage strong enough, it says the source doesn't cover it.
- Afterwards: marking an answer useful or wrong, or picking the passage that answered you, makes the next similar question better. A background "sleep" builds on those ratings, and keeps a change only if no answer that already works breaks.
Retrieval-augmented systems rarely fail by answering from nothing. They fail by answering with sources attached. In a 2026 penalty-aware audit, 98% and 100% of two commercial RAG products' wrong answers came with cited sources or grounding chunks (arXiv:2608.26385). Practitioners describe the same failure: the retriever returns something topically close, and the model fills the gap convincingly.
Most of these errors happen when the retrieved context is relevant but insufficient (Sufficient Context, ICLR 2025). So IntelliRAG treats "is this enough to answer?" as a first-class, inspectable decision, not something left to the prompt.
| Stage | Decision | Where |
|---|---|---|
| Retrieve | BM25, plus dense vectors only when compatible stored embeddings exist, fused with RRF | retrieve-core.ts |
| Rerank | A calibrated mix of idf recall, phrase and title overlap. It is not a cross-encoder and not a confidence score. | ranking.ts |
| Gate | A passage needs a support floor and distinctive overlap; a decisive rank-1 margin keeps only that source. If nothing passes, the answer is a refusal with no citations. | ranking.ts, evidence.ts |
| Pack | 1,400-token budget. Passages are verbatim slices of the stored chunk, code fences are never split, and a chunk that doesn't fit is trimmed to its most relevant contiguous paragraphs. | ranking.ts, coverage-pack.ts |
| Measure | Lists the question terms (function words and quantities removed) that appear in no packed passage, shown under every answer | coverage-pack.ts, context-coverage.tsx |
| Answer | With a server or visitor key: temperature 0 and a cite-or-refuse prompt. Without one: cited extracts, labelled as extracts. | query.server.ts |
| Cache | Keyed by question, corpus and a policy hash covering retrieval mode, evidence policy, model and graph edits. Corrections invalidate it. | graphify/persist.server.ts, query.server.ts |
Follow the numbers. 1–2: you ask. 3–7: the evidence is decided before any model runs. 8: you get the answer, its citations, and the question terms nothing in the context covers. A–C happen offline: ingest at a pinned commit, an optional model, and a frozen evaluation that calls the same functions the server does.
Click the pipeline for the interactive version, where every stage links to its source file.
The coverage line reads like this on a question the source can't settle:
5/9 question terms in context · not found:
serverprocessesdurable
That is the honest summary of "does this README establish a shared, durable rate limit across three servers?": it doesn't.
One pinned README (sindresorhus/p-queue at 180ab9e), 15 answerable questions with exact evidence anchors, 5 that must be refused, and the same 1,400-token budget for every arm. No model runs in these numbers; they measure whether the required evidence reached the reader.
| Arm | Clean corpus | + distractor documents | Refused 5/5 |
|---|---|---|---|
| Plain BM25 | 73.3% | 63.3% | — |
| IntelliRAG, rank-order packing (default) | 73.3% | 83.3% | ✓ |
| IntelliRAG, coverage-aware packing (opt-in) | 76.7% | 73.3% | ✓ |
The default ties BM25 on the clean corpus and beats it by 20 points once noise documents are mixed in. The coverage packer wins one extra question on the clean corpus, but the gain's bootstrap interval is [−10, +20] points, and it loses 10 points under distractors. So it stays opt-in. Every run, including the losing ones, is in eval/repo-support/runs/, and the evaluation write-up covers method, history and limits.
Retrieval itself is fast: the median query spends about 0.1 ms on keyword retrieval, 4 ms reranking and 0.4 ms packing on a laptop CPU. Network, database, embeddings and the model are not included. The September before/after chart shows how a deduplication bug cost half the evidence (36.7%) and what fixing it recovered.
Paste a bug report and IntelliRAG ranks the repository's files. I tested it on 170 real issues from django, scikit-learn, sphinx and pytest (SWE-bench Lite), using the reporter's own text. The file the fix changed now comes first 45.3% of the time, up from 35.3%, and is in the top 5 65.9% of the time, up from 60.6%. No model is involved. The gain comes from reading the code an issue names (traceback paths, module.Class, function calls) and linking it to the files that define it. Settings were tuned on different issues (sympy). Method, per-repository numbers and a negative result: eval/github-issues.
The design follows Google's "Language Models Need Sleep" (Behrouz, Hashemi, Mirrokni) and Self-Improvement Memory (Yu et al., arXiv 2611.18856). IntelliRAG has no weights to train, so the sleep acts on its memory:
| Phase | What happens | Guard |
|---|---|---|
| Awake | Every rated answer and every picked source is filed against the passages it came from. | The source text is never edited. |
| Replay | During sleep, the questions people confirmed are asked again. | This is the metric. |
| Consolidate | It learns which words people use for which words the sources use. | Kept in separate memory; the source's term statistics never change. |
| Dream | It writes practice questions for passages nobody has asked about: words hidden in identifiers, section context, people's vocabulary, or a model's questions when one is configured. | Each dream is tried on a copy of memory. It is kept only if its passage is now found, no confirmed question is lost, and no dead end comes back. A final full replay rolls back the whole sleep if anything slipped through. |
| Journal | Each strategy's kept-of-tried count and the reason for every rejection are recorded. | The next sleep budgets toward what worked and pauses a strategy that failed twice running. |
It sleeps by itself after every 8 ratings, or when you press Sleep now in the Memory panel. The panel shows what was kept and why everything else wasn't.
Measured honestly. On the 14-question rewording test, one sleep neither helped nor hurt (54.8% before and after on held-out wordings). On the GitHub issues, sleep's guard cut the damage of a memory that didn't transfer from −6.5 to −1.8 points, but didn't turn it into a gain. The gains so far come from ratings and picks (held-out rewordings 50.0% → 54.8%, original questions 71.4% → 81.0%) and from the code links above. With a model configured, the model-written dream strategy is where the paper expects the bigger effect. That hasn't been measured here yet.
Answers never stop mid-sentence. If the model runs out of output budget, it continues from where it stopped. If it still can't finish, the answer is cut back to its last whole sentence or list item, with a one-line note.
Three loops improve IntelliRAG as people use it. Each is measured on held-out data, and each can only add evidence: none of them can make an unsupported question look answered.
| Loop | What it learns | Measured |
|---|---|---|
Conversation (session-intent.ts) |
A follow-up is searched with the earlier questions on its topic | Follow-ups with complete evidence: 63.6% → 86.4%, refusals 1/3 → 2/3 (eval) |
Experience (experience.ts) |
A rated answer, or a passage picked as the one that answers a question, is filed against that passage. Later questions worded like it find it. | Originals asked again: 71.4% → 81.0%; new wordings +2.4 to +4.8 points; 20% wrong ratings cost nothing (eval) |
Self-improvement memory (sim-optimize.ts, after arXiv 2611.18856) |
The retrieval policy is edited round by round. Each attempt is recorded with its root cause, rationale, predicted fixes and observed result. | Held-out wording 52.9% → 58.8%. Its accepted packing edit also lifted the follow-up eval it never saw by 9 points (write-up, including the null result) |
Experience is the "file good answers back" step of Karpathy's LLM-wiki pattern, done per passage: the source is never edited, and links go stale when the passage they learned about changes. Agents get the same memory through the MCP server's feedback tool. It is kept per repository, so a repo keeps learning across sessions.
Press ⌘K (Ctrl+K) to search every indexed source instantly. It splits identifiers
(parseJSONResponse matches "parse json response"), weights function names and
docstrings, and takes operators: path:src/queue, lang:ts, sym:resume,
kind:prose, "exact words", -word. Results show the matching lines
highlighted, with "did you mean" for typos and completions from your sources.
The same engine answers GET /api/search?q=…&corpus=….
On CoSQA (313 real web queries, 6,267 Python functions) it scores 46.4 nDCG@10 against 42.9 for plain BM25, with no model. Details and an honest comparison with dense retrievers: eval/code-search.
| Paper | What we took | Status here |
|---|---|---|
| Sufficient Context (Joren et al., ICLR 2025) | Errors concentrate where context is relevant but insufficient, so expose sufficiency before trusting fluency | Lexical proxy shipped on every answer; an LLM autorater is not used |
| AdaGReS (2025) | Redundancy-penalised greedy selection under a token budget with an adaptive penalty | Implemented in coverage-pack.ts; opt-in after measurement |
| Lin & Bilmes (ACL 2011) | Cost-benefit greedy plus a best-single-unit check for knapsack-constrained submodular selection | Implemented; the guarantee holds only without the redundancy term |
| Recall Is Not Enough (2026) | Score what the packed context contains, not the candidate list | The eval scores the text actually packed, after trimming |
| Lost in the Middle (TACL 2024) | Position matters; keep the strongest evidence first | The top supported chunk is the whole-chunk anchor |
IntelliRAG also runs as an MCP server with no dependencies, so Claude Code, Codex, Hermes or any MCP client can answer from a repository's docs with citations, or refuse:
claude mcp add intellirag -- npx -y github:charan-rathore/IntelliRAG # Claude Code
codex mcp add intellirag -- npx -y github:charan-rathore/IntelliRAG # Codex
hermes mcp add intellirag --command npx --args -y github:charan-rathore/IntelliRAG # Hermes Agent> index_folder (the repo you're in, or index_github owner/repo at a pinned commit)
> ask "How do I cancel a queued task with an AbortSignal?" session: fix-1234
3 supported passages from sindresorhus/p-queue@180ab9e …
> ask "What if it is already running?" session: fix-1234
Reading: Read as a follow-up: searched together with the previous question (about abortsignal, cancel).
[readme.md#14] readme.md › options › signal
If the operation is already running, the signal will need to be handled by the operation itself.
> ask "What production database password does it specify?" session: fix-1234
REFUSED. Nothing in this corpus passed the support gate for this question.
It exposes seven tools: index_folder, index_github, ask, search, feedback, new_topic and sources. They call the same retrieval, support gate, packing and conversation reading as the console (web/src/lib/rag), bundled into mcp/dist and checked over stdio in CI. The agent skill is skills/intellirag/SKILL.md, written to Hermes Agent's authoring standard. The Hermes catalog entry is integrations/hermes/manifest.yaml.
git clone https://github.com/charan-rathore/IntelliRAG.git
cd IntelliRAG/web
npm ci
npm run dev # http://localhost:8080, pinned source works without a keynpm run typecheck && npm run build
npx tsx --test src/lib/rag/*.test.ts # retrieval, gate, packing, persistence
npx tsx scripts/repo-support-eval.ts # the measured table above, no network
npx tsx scripts/repo-support-eval.ts --distractors # the noise stress testdocs/WEB.md covers the Vercel setup, security posture and release gate. Durable imports, persistent semantic search and provider generation need private configuration; they aren't a frontend toggle.
flowchart LR
subgraph web["web/ · the served app"]
UI["src/components<br/>console, inspector, coverage"] --> Q["lib/rag/query.server.ts<br/>stream: sources → tokens → done"]
Q --> R["retrieve-core.ts<br/>BM25 · dense · RRF"]
R --> K["ranking.ts<br/>rerank · gate · pack"]
K --> C["coverage-pack.ts<br/>budgeted selection · coverage"]
R --> E["evidence.ts<br/>classify support"]
Q --> S["store.server.ts<br/>Postgres / PGlite · cache"]
I["ingest.server.ts<br/>chunking · GitHub · SSRF guard"] --> S
end
EV["web/scripts/repo-support-eval.ts"] -.same functions.-> R
PY["rag-platform/<br/>Python reference + benchmarks"]
| Folder | What's in it |
|---|---|
web/ |
The app you use: TanStack Start UI and the retrieval pipeline in web/src/lib/rag/ |
mcp/ |
The same pipeline as an MCP server for Claude Code, Codex and Hermes |
eval/ |
Frozen benchmarks: repository support, rewordings, conversations, code search, real GitHub issues |
docs/ |
Diagrams, deployment (WEB.md), roadmap, audit trail, code graph |
rag-platform/ |
Python reference implementation and benchmarks (not the served runtime) |
skills/, integrations/ |
Agent skill and editor integrations |
scripts/ |
Code graph and diagram generators |
docs/codegraph/ is a generated dependency graph of the whole repository (files, imports, hubs and import cycles), rebuilt with node scripts/codegraph.mjs. Start there to find the blast radius of a change. The Python rag-platform/ is a separate benchmark and reference implementation; it is not the serving runtime.
- The evaluation is one README with project-authored annotations. Both splits have been inspected, so the results are exploratory, not a blind holdout.
- The public host has no durable vector index or server model key. It shows keyword retrieval and cited extracts honestly; it doesn't demonstrate production semantic RAG.
- The coverage check is lexical. A paraphrase can answer a "missing" term, and a matched term can still be used in an unrelated sense.
- The graph view shows lexical document links, not verified entity extraction.
The target is verified task completion per unit of compute. A change ships when it beats the frozen baseline at the same budget and its failures are published.
MIT

