Skip to content

Prometheus metrics endpoint for production monitoring #8

@sarvanithin

Description

@sarvanithin

Overview

Production deployments need observability. Add a /metrics endpoint exposing Prometheus-compatible metrics so ops teams can monitor guardrail trigger rates, latency, and blocked request counts.

Metrics to expose

medguard_requests_total{guardrail="phi", result="triggered"} 42
medguard_requests_total{guardrail="drug_safety", result="blocked"} 3
medguard_requests_total{guardrail="hallucination", result="flagged"} 7
medguard_pipeline_duration_seconds{quantile="0.99"} 0.45
medguard_pubmed_lookups_total{result="verified"} 15
medguard_llm_calls_total{provider="anthropic", status="success"} 100

What to build

  • Add prometheus-client to optional deps: pip install medguard[metrics]
  • medguard/api/metrics.py — counter/histogram definitions
  • Instrument GuardrailPipeline.run* methods
  • Add GET /metrics route to FastAPI app
  • Add MEDGUARD_METRICS_ENABLED=true env var toggle

Files to modify

  • medguard/api/metrics.py — new file
  • medguard/guardrails/pipeline.py — add instrumentation
  • medguard/api/routes.py — add /metrics endpoint
  • pyproject.toml — add metrics optional dep

Acceptance criteria

  • GET /metrics returns valid Prometheus text format
  • phi_detection, drug_safety, hallucination, fact_check all have counters
  • Pipeline duration histogram with p50/p95/p99 quantiles
  • Disabled by default (zero overhead when not enabled)
  • Works with standard Prometheus scrape config

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions