Skip to content

telemetry.md

Latest commit

 

History

History
130 lines (83 loc) · 3.52 KB

telemetry.md

File metadata and controls

130 lines (83 loc) · 3.52 KB
Raw

Telemetry

Langfuse integration for request tracing, latency, and cost visibility.

Overview

Corvo Cortex records telemetry for both successful and failed LLM requests.

Tracing is fail-open:

  • if telemetry fails, API responses still complete normally
  • tracing never blocks request handling

Configuration

Set Langfuse secrets:

wrangler secret put LANGFUSE_PUBLIC_KEY --env production
wrangler secret put LANGFUSE_SECRET_KEY --env production

Set base URL (US region):

[vars]
LANGFUSE_BASE_URL = "https://us.cloud.langfuse.com"

If LANGFUSE_BASE_URL is missing, Corvo Cortex defaults to https://us.cloud.langfuse.com and logs a warning.

Data Minimization

Per-client telemetry mode

Each client record in CORTEX_CLIENTS may set telemetry:

Mode Input/output payloads Usage, cost, latency, provider, model Trace at all
"full" (default) Yes — redacted + truncated Yes Yes
"metadata" No (input/output omitted) Yes Yes
"off" — — No trace is created

Omitting the field means "full" (backward compatible with existing records). New clients should prefer "metadata".

Redaction

Before ingestion, full-mode payloads pass through a redaction filter with built-in patterns for common secret shapes (sk-… API keys, Bearer … tokens, x-api-key values). Additional patterns can be configured (regex source strings, ≤ 256 chars) via CORTEX_CONFIG key config:telemetry-redactions; invalid entries are skipped. Matches are replaced with [REDACTED].

Truncation

Input and output payloads are capped at 50,000 characters per side. Oversized payloads are replaced with { truncated: true, originalChars, preview }.

Data retention statement

Corvo Cortex does not store request data at rest beyond the request lifetime; telemetry traces are transferred to and retained by Langfuse US cloud under the workspace's Langfuse retention settings (default retention: 30 days in Langfuse cloud free tier — operators should configure retention explicitly in Langfuse). Clients requiring stricter guarantees should set telemetry: "metadata" or "off".

Transport

Corvo Cortex uses direct Langfuse ingestion API calls:

  • Endpoint: POST /api/public/ingestion
  • Reason: production Workers reliability

Upstream issue reference (SDK behavior in Workers):

What Is Traced

Each request emits:

  • 1 trace (name = "llm-request")
  • 1 generation child (name = "provider-call")

Captured fields include:

  • appId
  • provider
  • model
  • statusCode
  • durationMs
  • error (for failed requests)
  • input (full request payload)
  • output (full response payload)
  • usage (prompt/completion/total tokens when available)
  • costUsd (when usage is present and provider/model pricing is available)

Streaming Behavior

For streaming responses:

  • usage is captured from SSE chunks when provided
  • output is captured from stream chunks
  • telemetry finalization waits for stream completion to avoid partial traces

Analytics Endpoints

Current admin analytics routes are pointers to Langfuse dashboard:

  • GET /analytics/costs
  • GET /analytics/metrics
  • GET /analytics/export

Source of truth remains Langfuse dashboard.

Implementation

  • src/middleware/telemetry.ts
  • src/services/telemetry.ts
  • src/routes/chat.ts
  • src/routes/responses.ts
  • src/utils/streaming.ts

Dashboard

Langfuse US: https://us.cloud.langfuse.com