Keep the index fresh: incremental reindex on push + freshness indicator

[imkp1]

Summary

Keep the index honest after the one-time initial index by (a) incrementally reindexing on push and (b) surfacing an index freshness indicator so both users and the agent know how current the index is.

This bundles two tightly-related improvements — a freshness signal is most of the value of incremental indexing, and both touch the same indexer surface.

Motivation

Today the indexer indexes the default branch "a single time." The README's core pitch — "no documentation debt… tracks the actual committed code" — only holds if the index follows HEAD. A stale index produces confidently-wrong answers, which is the worst failure mode for this product.

Part A — Incremental reindex on push

• GitHub push webhook (or a poll fallback for repos without webhook access) → indexer reindexes only changed files by diffing the new SHA against the last indexed SHA.
• Re-embed added/modified files, drop vectors for deleted files.
• Debounce rapid pushes; make it idempotent on the target SHA.

Part B — Index freshness indicator

• Indexer tracks/records the last-indexed commit SHA + timestamp per repo (already needed for Part A and for stable citation links — see Answer citations: ground streamed answers in clickable file:line sources #8).
• Web: show "index is N commits / HH:MM behind HEAD" in the chat header and on the indexer page.
• Engine: when the index is behind, the agent can disclaim potential staleness in its answer.

Implementation notes

• New indexer REST/MCP surface is transparent through the backend proxy (per CLAUDE.md), so much of this needs no backend change.
• Reuse the central git token already held server-side by the indexer for fetching.
• Mutating/reindex ops stay admin-only; freshness reads are open to members.

Scope

In-scope: single-workspace, Claude-only. No new services; extends the existing indexer.

Acceptance

• Pushing to an indexed repo's default branch updates only changed files in Qdrant.
• Deleted files' vectors are removed.
• The UI shows how far behind HEAD the index is; the agent can flag staleness.
• Tests added (make test-indexer, web).