docs: add ADRs, architecture refresh, what-is-what, and doc index

[Rakesh1002]

Summary

Consolidates the docs around three new entry points and collapses seven stale or duplicate root-level markdown files into the canonical docs/ tree. Aimed at the open-source contributor influx and the intern onboarding cohort — they should be able to find what they need without trial and error.

New content

docs/README.md — doc entry point

Task-oriented ("I want to understand / use / contribute / operate / sell UnSearch") replacing the implicit "open the docs folder and guess" UX.

docs/what-is-what.md — component map

• Directory-by-directory tour of the repo
• Plain-language layer diagram (Public → Edge → Container → State)
• Glossary of recurring terms (DO, KV, D1, tier, namespace, drop-in, persona, etc.)
• Naming conventions across Python / TS / URLs / env vars
• "Where does that live?" cheat sheet for common changes
• "How to read a PR" walkthrough
• Explicit note about the app/ vs apps/backend/ dual layout

docs/adr/ — Architecture Decision Records

Eight ADRs covering the past decisions a new contributor would otherwise have to reverse-engineer:

#	Title
0001	Cloudflare-native edge architecture
0002	SearXNG as the meta-search aggregator
0003	Tavily-compatible drop-in API surface
0004	Workers AI with tiered model selection
0005	Apache 2.0 + self-hostable from day one
0006	Monorepo layout with apps/* and workers/
0007	Python SDK ships sync + async clients
0008	Honest feature-status policy (✅ / 🔶 / 📋)

Each ADR follows a MADR-style template — Context / Decision / Consequences / Alternatives considered — so future supersessions link cleanly. The README explains when to write an ADR and the status taxonomy (Accepted / Superseded by ADR-XXXX / Deprecated).

Rewrites

docs/architecture.md

Replaced the stale 58-endpoint / 27,500-LOC snapshot of the v1 FastAPI-only architecture with the current v2.0 Cloudflare-native picture:

• Edge worker (Hono on Workers), FastAPI Container, four Durable Objects (RateLimiter, TopicMonitor, ResearchAgent, SessionManager)
• Data-placement table — what lives in D1 vs KV vs R2 vs Vectorize vs origin Postgres
• End-to-end request lifecycle examples (cache-hit, cache-miss, research-agent, topic-monitor)
• Honest tech-debt callouts from docs/roadmap.md
• Per-component file map so changes are easy to scope

Deletes (7 files)

File	Reason
DEPLOYMENT.md	0 incoming refs; superseded by docs/deployment/
DEPLOYMENT_GUIDE.md	1 ref from IMPLEMENTATION_SUMMARY.md (also deleted); superseded by docs/deployment/
DOCKER-COMPOSE-README.md	0 incoming refs; covered by top-of-file comments in docker-compose*.yml
ENV_VARIABLES.md	Synced (newer version, with the docs-scrub fix) into docs/configuration/env-variables.md before deletion
IMPLEMENTATION_SUMMARY.md	Stale — referenced packages/shared/ which doesn't exist; 0 incoming refs
stripe_webhook_setup.md	Byte-identical to docs/configuration/stripe-webhook.md
webhook_events_explained.md	Byte-identical to docs/configuration/webhook-events.md

Verified zero dangling references to any of the deleted paths before deletion.

Stats

22 files changed: +869 / −2219 lines

Test plan

• Browse the new docs/README.md and click each linked path
• Open one ADR (e.g., 0001) and check the cross-links to other ADRs resolve
• Confirm git log -- DEPLOYMENT.md etc. still work for anyone who needs the deleted history
• If you spot a broken anchor or stale claim, comment inline and I'll fix before merge

Why now

Two interns onboarding this week. The first thing they'll do is read the docs.