Skip to content

feat(trace): enforce metadata-only privacy and OTLP mapping - #2919

Open
alectimison-maker wants to merge 3 commits into
webbrain-one:mainfrom
alectimison-maker:feat/trace-privacy-otlp
Open

feat(trace): enforce metadata-only privacy and OTLP mapping#2919
alectimison-maker wants to merge 3 commits into
webbrain-one:mainfrom
alectimison-maker:feat/trace-privacy-otlp

Conversation

@alectimison-maker

@alectimison-maker alectimison-maker commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR establishes a privacy-preserving trace pipeline and collector-compatible OTLP mapping. The motivation is to keep default traces useful for diagnosis without persisting message content, model output, tool payloads, or screenshots, while ensuring intentional redaction is not reported as a failure or an empty response.

Privacy behavior

  • Default run records omit user messages and final model responses.
  • Default events omit model response content, tool arguments, tool results, free-form error text, and vision descriptions.
  • Default screenshot recording keeps only the timeline marker and does not persist screenshot bytes.
  • Metadata such as IDs, model/provider labels, statuses, error codes, counts, token usage, costs, and timing remains available.
  • The existing lossless_trace opt-in preserves bounded content recording and screenshot storage.

Redaction semantics

  • Default-tier tool events retain content-free resultStatus (success, error, or unknown) and an optional bounded resultErrorCode.
  • OTLP output, Markdown export, and the Traces UI consume that status instead of treating a missing result as a failed tool.
  • Default-tier model responses retain toolCallCount and empty, so a tool-call-only response is not rendered as EMPTY_RESPONSE after tool-call details are removed.

OTLP behavior

  • Session bundles use one OTLP trace per session and one span per run.
  • Parent runs in the same session use parentSpanId.
  • Parent runs in another session use an OTLP Span Link and never become a cross-trace parent.
  • Step and turn lifecycle records remain span events.
  • The legacy single-run export layout remains unchanged.

Compatibility and risks

  • Legacy events without outcome metadata retain their previous interpretation, including the historical missing-result failure marker.
  • Historical records are not rewritten; consumers remain tolerant of records created before the metadata fields existed.
  • No trace format version bump or provider traceparent propagation is included.
  • The main compatibility risk was consumer disagreement about intentionally redacted fields; the UI, Markdown, and OTLP regression coverage now exercises the same projected data.

Validation

  • node test/run.js — 2046 passed; 2 unrelated repository baseline failures remain:
    • the newest changelog entry does not match the package version
    • the tracked Chrome store archive is absent
  • npm run test:toolbar-guard — 33 passed
  • npm run test:security — 60/60 checks passed
  • JavaScript syntax checks — passed
  • git diff --check — passed

Review context

The redaction and consumer behavior in this PR were refined in response to maintainer review on PR #2919. No separate issue is linked.

@vercel

vercel Bot commented Aug 25, 2026

Copy link
Copy Markdown

@alectimison-maker is attempting to deploy a commit to the esokullu's projects Team on Vercel.

A member of the Team first needs to authorize it.

@webbrain-one

Copy link
Copy Markdown
Owner

Review findings:

  1. [P1] Preserve tool outcome metadata before dropping results.

    src/chrome/src/trace/privacy.js:103 (and the Firefox mirror) removes result, but the current OTLP converter, Markdown exporter, and Traces UI interpret a missing result as failure. A successful projected click_ax event reproduced as error.type=tool_error in the session OTLP output and as ✗ (missing tool result) in Markdown. Please retain a content-free success/status/error-code field, or update every consumer so intentional redaction is treated as unknown rather than failure. Add regression coverage that passes projected default-tier data through these consumers.

  2. [P2] Do not classify redacted tool-call responses as empty.

    src/chrome/src/trace/privacy.js:97-100 (and the Firefox mirror) removes toolCalls while retaining toolCallCount. Both the Traces UI and Markdown exporter currently use the missing toolCalls array to infer an empty response, so a valid tool-call-only response is rendered as EMPTY_RESPONSE even when the same line reports 1 tool calls. Empty-state rendering should use the projected empty flag and/or toolCallCount, not the intentionally removed array.

Validation performed: node test/run.js reported 2045 passed and the same two stated baseline failures; toolbar guard passed 33/33, security corpus passed 60/60, and syntax plus git diff --check passed.

@alectimison-maker

Copy link
Copy Markdown
Contributor Author

Addressed both review findings in commit 8428a174d:

  • Default-tier tool events now retain content-free resultStatus and optional resultErrorCode; OTLP, Markdown, and the Traces UI no longer treat intentional redaction as a failed tool result.
  • Redacted tool-call-only responses now use empty and toolCallCount, so they are not rendered as EMPTY_RESPONSE.

Added regression coverage through the privacy projection, both Markdown serializers, the OTLP converter, and the Traces UI contract. Validation: node test/run.js — 2046 passed with the same two repository baseline failures; toolbar guard 33/33; security 60/60.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants