Add a Prometheus /metrics endpoint for proxy and system observability

[wankhede04]

Problem

agent-vault is meant to be deployed as its own trust boundary — a separate service sitting in front of every credentialed request an agent makes (per the README: "deploy Agent Vault on a separate machine from your AI agents"). For something running as production infrastructure, there's currently no metrics export of any kind.

Looking through the environment variables reference, there's a lot of operational configuration already (rate-limit tiers, request-log retention, DB pool sizing) but the only runtime visibility into what the proxy is actually doing is:

• AGENT_VAULT_LOG_LEVEL=debug, which adds one text log line per proxied request, or
• querying the internal request-log table directly (capped by AGENT_VAULT_LOGS_MAX_AGE_HOURS / AGENT_VAULT_LOGS_MAX_ROWS_PER_VAULT).

Neither of those wires into Prometheus/Grafana/Datadog, which makes it hard to alert on things operators would actually care about — e.g. a spike in netguard-blocked requests (possible SSRF attempt), rate-limit rejections climbing, or proposal backlog growing.

Proposal

Add a /metrics endpoint (standard promhttp handler) on the management API, behind the same auth/network posture as the rest of the management surface. Initial counters/histograms would mostly just be instrumenting things that already exist internally rather than adding new logic:

• agent_vault_proxy_requests_total{service, status} — proxied request outcomes
• agent_vault_proxy_request_duration_seconds — latency histogram
• agent_vault_netguard_blocked_total{reason} — private-range / metadata-endpoint denials (from the existing network guard)
• agent_vault_ratelimit_rejected_total{tier} — 429s by tier (AUTH, AUTHED, PROXY, GLOBAL)
• agent_vault_proposals_pending / agent_vault_proposals_total{status} — proposal backlog and lifecycle counts

Could gate it behind an env var (e.g. AGENT_VAULT_METRICS_ENABLED=true) to keep it opt-in for anyone who doesn't want the extra endpoint exposed.

Why this is worth doing

This closes a real gap for anyone running Agent Vault as actual infrastructure rather than a local dev tool — which is exactly how the docs recommend deploying it. It's additive instrumentation on top of existing counters/events, not new business logic, so it should be reviewable in relatively contained pieces (e.g. proxy metrics first, netguard/rate-limit metrics as a follow-up).

Happy to take a first pass at this if it's something you'd want — would start with the proxy request counters/histogram and the /metrics endpoint scaffolding, then layer on netguard and rate-limit counters.