Skip to content

Large static instruction/context blocks are re-sent per turn instead of being cached or referenced #402

Description

@baron

Impact

Re-injecting large unchanged instruction/context blocks each turn inflates every provider request. Worst sessions showed prompt/tool-history injections up to about 1.75M characters.

Actual behavior

Agent turn assembly appears to re-materialize large static blocks, such as instructions and workspace context, per turn rather than separating stable/static segments from dynamic turn content.

Expected behavior

  • Static instruction/context segments are byte-stable across turns so provider prompt caching can apply where supported.
  • Large context artifacts are passed by reference where supported, instead of repeatedly inlined.
  • Turn assembly distinguishes static and dynamic segments and logs their sizes for diagnostics.

Publishable evidence

  • Worst sessions showed injections up to about 1.75M characters.
  • AgentSession JSON totaled about 64 MB across about 2,075 files.

Scope / non-goals

In scope:

  • prompt assembly segmentation,
  • static/dynamic byte accounting,
  • provider-cache-friendly stable segments,
  • diagnostics.

Out of scope:

Likely affected areas

  • Sources/RepoPrompt/Features/AgentMode/Runtime/
  • Sources/RepoPrompt/Features/AgentMode/Runtime/Claude/
  • Sources/RepoPrompt/Features/AgentMode/Runtime/Codex/
  • Sources/RepoPrompt/Infrastructure/AI/Prompts/
  • Tests/RepoPromptTests/AgentMode/

Acceptance criteria

  • Static prompt segments are byte-identical across consecutive turns of the same session when inputs do not change.
  • Per-turn diagnostics report static versus dynamic payload bytes.
  • Large artifact reference-vs-inline behavior is documented.
  • Existing provider payload tests continue to pass.

Interlinks

Part of #406. Related: #398.


Metadata

Metadata

Assignees

No one assigned

    Labels

    area:agent-modeAgent Mode runtime and interfaceenhancementNew feature or requestprovider:claudeClaude-compatible provider behaviorprovider:codexCodex-specific integration behavior

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions