feat(demo): HyDE Theater — hypothetical document embeddings, before/after

[konjoinfinity]

A sixth flagship demo surface (/hyde) showcasing HyDE (Hypothetical Document Embeddings, Gao et al. 2022): embed a hypothetical answer (document-space) instead of the raw query (query-space) to close the distribution gap.

What's new

• demo/hyde.py — HyDEEngine runs the real before/after comparison. It synthesizes a document-shaped hypothesis with a deterministic, generic template (labelled rule-based / no-LLM, following konjoai.retrieve.hyde's real _HYDE_PROMPT intent), then runs the same real dense cosine retrieval on both the raw query and the hypothesis. Before/after rankings, per-document score deltas, and the closed top-document gap are measured, not staged.
• demo/pipeline.py — public dense() (real cosine retrieval as formatted rows), reused for both runs.
• demo/hyde.html — cinematic "close the gap" view: query vs hypothesis with embedding stats (tokens / non-zero dims), a typewriter hypothesis reveal, an animated top-similarity gap meter (query → HyDE delta), and side-by-side retrieval columns with per-document rank/score deltas. Ships reduced-motion, focus-visible, View Transitions, and responsive layout.
• demo/server.py — GET /hyde + GET /api/hyde/analyze (bounded inputs).
• HyDE linked in the nav across all surfaces — now Playground · Observatory · Pipeline · Agent · Security · HyDE.
• tests/unit/test_demo_hyde.py — 7 contract tests (shape + no-LLM label, hypothesis denser than query, real bounded ranked retrieval, delta arithmetic, winner is a real source).

The effect (real numbers)

For "What are my GDPR rights?" the top-document cosine rises 0.18 → 0.51 (+0.33) — the query→document gap closing, because the hypothesis paragraph lands in document-space.

Honesty notes

• Only the hypothesis text is rule-based (disclosed in hypothesis_source and on the page). The embeddings, cosine scores, and rankings are real konjoai dense retrieval.
• The UI reports whatever actually happens (the caption says "rose"/"fell"), so it never overclaims the gap always closes.

Verification

• 43 demo tests pass, stable across multiple PYTHONHASHSEED values; ruff check . clean repo-wide.
• All six surfaces serve 200; the HyDE endpoint returns the measured delta.
• Rendered & screenshotted headless against the live server.
• Only demo/* + a new test changed — no production konjoai/ code touched.

Run it:

python3 demo/server.py        # http://localhost:8766/hyde

https://claude.ai/code/session_01WWWgV1NwxaaNdPfy4PHN1K

---

Generated by Claude Code