Add Langfuse tracing for Codex and Claude Code runs#15
Closed
Quantumlyy wants to merge 1 commit into
Closed
Conversation
Opt-in via a shared `agent.langfuse` block ({enabled, public_key,
secret_key, base_url}; all three credentials required when enabled).
Tracing is side-channel only: each agent uses its own native integration,
reads transcripts, fails open, and doesn't change agent behavior or
Teich's output files.
- Codex: the codex-observability plugin is baked into the runtime image at
a staging CODEX_HOME and seeded into each session's CODEX_HOME offline;
the generated config enables plugin_hooks, and `codex exec` runs with
--dangerously-bypass-hook-trust (it silently skips plugin hooks otherwise).
- Claude Code: the Langfuse Stop/SessionEnd hook is registered per session.
The langfuse SDK is installed into /opt/venv and the hook calls that
interpreter by absolute path, since Claude strips /opt/venv from a hook's
PATH. The plugin is pinned to a known-good commit.
A localhost/127.0.0.1 base_url is rewritten to host.docker.internal for the
in-container env value and the --add-host gate.
Collaborator
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds opt-in Langfuse tracing for agent sessions, configured through a single shared
agent.langfuseblock:All three credentials are required when
enabledis true. Tracing is side-channel only: each agent uses its own native integration, reads transcripts, fails open, and does not change agent behavior or Teich's output files. Coverage is partial by design — Codex and Claude Code are wired; other providers are left untouched.Codex
The codex-observability plugin is baked into the runtime image at a staging
CODEX_HOMEand copied into each session'sCODEX_HOMEat run time, so containers seed offline with no per-run network. The generatedconfig.tomlenablesplugin_hooksand the tracing plugin, andcodex execruns with--dangerously-bypass-hook-trust— non-interactive exec silently skips plugin hooks without it.Claude Code
Each session's
settings.jsonregisters the official Langfuse Stop/SessionEnd hook. The langfuse SDK is installed into/opt/venvand the hook invokes that interpreter by absolute path, because Claude sanitizes a hook's PATH and would otherwise run a Python without the SDK. The plugin is pinned to a known-good commit for reproducible image builds.Networking
A
localhost/127.0.0.1Langfusebase_urlis rewritten tohost.docker.internalfor both the in-container env value and the--add-hostgate, so a host-local Langfuse instance is reachable from the sandbox.Tests
Adds
tests/test_codex_langfuse.pyandtests/test_langfuse_agents.pycovering config validation, the Codex and Claude env wiring, theconfig.tomlplugin blocks, the base-URL rewrite, the trust-bypass flag, and the Claude hook registration.