feat: optional OpenTelemetry tracing and Sentry error monitoring

[amal66]

Design note

Ported from the fork's ADR 0002 ("Optional-by-default observability", amal66/mike docs/adr/0002-optional-by-default-observability.md). The platform runs in two very different worlds: a hosted deployment that wants distributed tracing and error reporting, and a self-hosted / confidentiality-sensitive deployment that must generate zero external network traffic by default (traces and error payloads can carry document snippets). So both integrations are gated on their configuration variable simply being present, and the gates read process.env directly because init must run before any instrumented module loads — the auto-instrumentations patch http/express at require time, which also makes the import order in index.ts load-bearing: anything imported above initOtel() silently escapes instrumentation.

Summary

See what broke without asking users to reproduce it: optional OpenTelemetry distributed tracing and Sentry error monitoring for the backend. Both are off by default and provably inert until explicitly configured — a firm that never sets OTEL_EXPORTER_OTLP_ENDPOINT or SENTRY_DSN gets no SDK initialization, no patched modules, no background traffic, and no third-party egress, verifiable in the first two boot log lines.

Changes

• backend/src/lib/observability/otel.ts — initOtel() no-ops unless OTEL_EXPORTER_OTLP_ENDPOINT is set; the SDK and auto-instrumentations are required lazily inside the enabled branch so the disabled path never even loads them. shutdownOtel() flushes pending spans on graceful shutdown.
• backend/src/lib/observability/sentry.ts — initSentry() no-ops unless SENTRY_DSN is set; captureException() and setupSentryErrorHandler() are no-ops when disabled.
• backend/src/index.ts — init both at the very top (before express is imported; order is load-bearing for load-time patching), register Sentry's Express error handler after all routes, forward fatal unhandledRejection/uncaughtException to Sentry before exiting, and add a SIGTERM/SIGINT graceful-shutdown path that drains connections and flushes spans.
• backend/.env.example — documents the five optional variables (SENTRY_DSN, SENTRY_TRACES_SAMPLE_RATE, SENTRY_ENVIRONMENT, OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_ENVIRONMENT), all blank/commented by default.
• backend/package.json — adds @opentelemetry/sdk-node, @opentelemetry/auto-instrumentations-node, @opentelemetry/exporter-trace-otlp-http, @opentelemetry/resources, @opentelemetry/semantic-conventions, @sentry/node (fork's versions).

Why

Confidentiality-sensitive firms need "no telemetry unless I turn it on" to be provably true, and the fork ships it that way — this port preserves that gating exactly:

• Default off, zero egress: otel.ts returns before constructing anything when OTEL_EXPORTER_OTLP_ENDPOINT is unset; sentry.ts returns before Sentry.init when SENTRY_DSN is unset. Verified at runtime: a default boot logs OpenTelemetry disabled (OTEL_EXPORTER_OTLP_ENDPOINT not set) / Sentry disabled (SENTRY_DSN not set) and nothing else.
• AIRGAPPED=true is a hard kill switch that wins over both gates — even a misconfigured DSN/endpoint cannot cause egress in air-gapped mode (checked first in both modules; verified at runtime).
• Zero cost when off: the heavy OTel SDK is loaded lazily only in the enabled branch; no middleware, timers, or collectors are registered when disabled.
• Deps: six new runtime dependencies (listed above) that the fork ships for this feature. They are installed but never initialized unless env-configured; turning either integration on is one env var.

Testing

• npm install + npm run build (tsc) green as committed on the branch.
• With the vitest harness (upstream-pr/test-harness) merged locally: 12/12 tests pass (harness suite; this PR adds runtime gating code with no new unit tests — the fork's observability tests cover its queue trace-propagation and Prometheus modules, which are not part of this port, see Provenance).
• Runtime verification of all three gate states: default boot → both "disabled (… not set)" lines, /health OK, graceful SIGTERM shutdown; AIRGAPPED=true with an endpoint set → both "disabled (AIRGAPPED)"; endpoint set → "OpenTelemetry tracing initialized".
• No frontend changes.

Provenance

All added lines are mechanical ports of amal66/mike@origin/main (b3166dd) — apps/api/src/lib/observability/{otel,sentry}.ts and the fork's apps/api/src/index.ts wiring, with path moves and import rewrites. Exceptions, all mechanical adaptations to what exists upstream:

• The fork logs via its pino logger and validates env through a zod env module; upstream has neither, so log calls became console.log/error and sentry.ts reads process.env directly (the fork's otel.ts already reads process.env by design; the sample-rate clamp reproduces the fork's zod min(0).max(1).default(0) rule).
• The fork's graceful-shutdown/crash handlers also stop its BullMQ workers and use its secret-guard; those fork-only pieces were omitted — the ported handlers keep the fork's structure (drain server, flush spans, exit codes) minus the queue calls.
• Not ported (fork observability code outside this PR's scope): metrics.ts (Prometheus/prom-client), traceContext.ts (W3C trace propagation across the fork's BullMQ queues) and requestContext.ts (pino log stamping) — they depend on the fork's queue/logging infrastructure that upstream doesn't have. @opentelemetry/api (a fork dependency used by traceContext.ts) is correspondingly not added as a direct dependency.
• The AIRGAPPED=true kill switch is kept verbatim from the fork even though upstream has no airgapped mode yet — it is inert unless that variable is set and preserves the fork's defense-in-depth ordering.

Credits & prior art

• @CaliLuke (CaliLuke/mike) — independently parallels this work: their fork (a Go/SurrealDB rewrite) built OpenTelemetry observability for Mike. Entirely different implementation, same observation that operators need tracing to run this in production.

🤖 Generated with Claude Code

https://claude.ai/code/session_01CEguyEgXa9JjCciXCcVemC