Skip to content

Feature: richer Codex metrics and request inspection commands #6

@riatzukiza

Description

@riatzukiza

Summary

Extend the plugin's diagnostics with Codex-style metrics and request inspection commands (e.g. /codex-metrics, /codex-inspect) that expose structured information similar to ResponsesRequest in openai/codex.

Background

In openai/codex:

  • codex-rs/core/tests/common/responses.rs and core/tests/suite/tool_harness.rs use a ResponsesRequest helper to inspect:
    • body_json, input, function_call_output, custom_tool_call_output, call_output, header, path, query_param.
  • Tests assert fine-grained behaviors around tool calls, request paths, and payload structure.

In this plugin:

  • We already have a /codex-metrics command and logging controlled by ENABLE_PLUGIN_REQUEST_LOGGING.
  • We also track basic cache and instruction metrics.

There is still room to provide more structured, Codex-CLI-like diagnostics that make it easier to debug tool calls, prompt caching, and request transformation.

Proposed Features

  1. Enhanced /codex-metrics output

    • Include per-session counters for:
      • Number of Codex requests made.
      • Cache hits vs misses vs "no cache key" cases.
      • Number of tool calls, broken down by tool type (shell, apply_patch, custom tools, MCP tools).
      • Number of compaction events (once plugin-level compaction is implemented).
    • Surface the current normalized model and reasoning config (effort/summary/verbosity) as seen by the plugin.
  2. New /codex-inspect command

    • Provide a command that prints the last N Codex requests in a structured way (similar to ResponsesRequest):
      • HTTP method, path, and key headers (e.g. chatgpt-account-id, beta headers).
      • The transformed request body:
        • model, prompt_cache_key, store, stream, include.
        • tools and tool_choice / parallel_tool_calls after normalization.
        • input after filtering (stripped IDs, filtered system prompts, injected bridge/tool-remap message).
    • Provide safe redaction for sensitive values (e.g. tokens, account IDs) so inspection is safe to use in logs or bug reports.
  3. Optional JSON output for automation

    • Allow /codex-inspect to return a JSON-encoded structure when a flag is present (e.g. /codex-inspect --json) so that advanced users can pipe it into other tools.
  4. Testing and observability

    • Add tests that:
      • Verify /codex-metrics and /codex-inspect work without making additional network calls.
      • Ensure redaction logic is applied correctly (no secrets leaked in snapshots).
      • Exercise edge cases like empty history, no tools, and multiple requests per session.

Rationale

  • Brings the plugin's debugging experience closer to Codex CLI's ResponsesRequest-driven tests and tooling.
  • Makes it much easier to debug complex tool flows and caching behavior from within opencode.
  • Provides a natural place to hang future diagnostics (e.g. model upgrades, parallel tool-call behavior, compaction metrics).

Acceptance Criteria

  • /codex-metrics exposes richer per-session stats without regressing existing output.
  • /codex-inspect can show at least the last request in a structured, partially redacted way.
  • JSON output option is available and documented.
  • Tests cover the major paths and redaction behavior.

Metadata

Metadata

Assignees

No one assigned

    Labels

    blockedeffort-largeLarge effort issues - more than 12 hours of workmetricsIssues related to request inspection and performance metricspriority-mediumMedium priority issues - important features or significant improvements

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions