Repository navigation
Stream source/evidence capture without reintroducing file caps #217
Description
Activity
Codex review: this still needs some work.
Summary
This accepted capacity follow-up remains necessary: current main still materializes complete inputs and prompt lists, and the merged partition repair does not provide bounded-memory capture.Reproducibility: Current source directly establishes whole-file accumulation, decoding, evidence rendering, and retained prompt construction; no memory experiment was run and no ordinary-review failure is established.
Root-cause cluster
Relationship:canonical
Canonical: #217
Summary: This issue owns the deferred complete-input memory redesign; merged scope and partition changes leave that work open.Members:
adjacent_distinct: fix(autoreview): preserve complete single-checkout review scope #215 - Established uncapped complete capture and explicitly deferred streaming to this issue.partial_overlap: fix: use measured capacity before refusing review partitions #287 - Reduces allocations during failed fallback planning while retaining complete capture buffers and successful prompt lists.
Proposal only: this assessment does not dispatch repair, suppress jobs, mutate sibling items, close, or merge anything.
Maintainer decision needed
- Question: What measurable full-pipeline memory target and private-spool lifecycle should govern this redesign?
- Recommendation: Define the resource contract: Align the obsolete scanning criterion with current policy, specify spool ownership and cleanup, and establish controlled memory measurements before implementation.
- Why: The accepted direction establishes complete uncapped text coverage but leaves resource targets and cleanup ownership undefined across capture, planning, retries, and cancellation.
Next step
Align the scanning criterion with current policy and define memory targets, private spool ownership, and cancellation cleanup before implementing the cross-stage redesign.Review details
Best possible solution:
Use private spools and incremental prompt preparation while preserving complete frozen inputs, attribution, integrity checks, isolation, and current scanner-free policy.
Do we have a high-confidence way to reproduce the issue?
Current source directly establishes whole-file accumulation, decoding, evidence rendering, and retained prompt construction; no memory experiment was run and no ordinary-review failure is established.
Is this the best way to solve the issue?
Whole-pipeline streaming or owned spooling addresses the retained copies; changing only the read loop leaves downstream allocations intact, while restoring secret scanners would contradict current policy.
AGENTS.md: found and applied where relevant.
Remaining risk / open question:
- The requested pre-send scanning criterion conflicts with the current intentional scanner-free policy.
- The full-pipeline memory target and private spool lifecycle across retries, failure, and cancellation remain unspecified.
Codex review notes: model internal, reasoning medium; reviewed against 38b67bc23842.
Label changes
Label changes:
No label changes.
Label justifications:
P3: This is an accepted capacity improvement without a demonstrated ordinary-review outage or host-OOM failure.impact:other: Whole-input capture and retained prompt lists constrain large-review memory capacity.
Evidence reviewed
What I checked:
- Current default branch: GitHub main and the clean checkout both resolve to 38b67bc. (38b67bc23842)
- Whole-file buffering remains: read_file_bytes retains every 64 KiB chunk before joining the complete byte buffer; file_bundle_snapshot then decodes that buffer into a complete string. (
skills/autoreview/scripts/autoreview:2385, 38b67bc23842) - Downstream materialization remains: Evidence capture retains content records and rendered prompt-file text; planning creates a complete prompt list, which execution retains alongside captured sources and evidence. (
skills/autoreview/scripts/autoreview:4433, 38b67bc23842) - Explicitly accepted follow-up: Merged fix(autoreview): preserve complete single-checkout review scope #215 acknowledges finite-memory buffering, declines to restore file caps, and explicitly defers whole-pipeline streaming to this issue without claiming it fixed. (836f2fe390db)
- Partition repair addresses only part: Merged fix: use measured capacity before refusing review partitions #287 sizes failed fallback prompts transiently but retains successful prompt lists; steipete’s landing comment expressly limits its proof to deterministic preparation without memory-exhaustion claims. (
skills/autoreview/scripts/autoreview:4421, af1967eac34e) - Scanning criterion is obsolete: Current policy intentionally leaves approved secret scanning outside Autoreview and requires an explicit maintainer decision before restoring a scanner, following fix(autoreview): remove restored TruffleHog dependency #240. (
skills/autoreview/SKILL.md:315, 38b67bc23842)
How this review workflow works
- ClawSweeper keeps one durable marker-backed review comment per issue or PR.
- Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
- A fresh review can be triggered by eligible
@clawsweeper re-reviewcomments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch. - PR/issue authors and users with repository write access can comment
@clawsweeper re-reviewor@clawsweeper re-runon an open PR or issue to request a fresh review only. - Maintainers can also comment
@clawsweeper reviewto request a fresh review only. - Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge; those execution lanes are retired.
- Maintainers can comment
@clawsweeper explainto ask for more context, or@clawsweeper stopto stop active automation.
Reviewed October 11, 2026, 5:52 PM ET / 21:52 UTC.
- added and removed
on Sep 3, 2026 🦞
⚠️
Automatic implementation stopped before completion.
Reason: The automatic implementation worker stopped before all deterministic gates completed. Open the workflow run for the exact blocker.
Worker: https://github.com/openclaw/clawsweeper/actions/runs/38118732074- added and removed
on Oct 10, 2026
Capacity follow-up
Track a bounded-memory implementation of complete source/evidence capture without reintroducing per-file size caps or silently truncating review input. This follows the explicit uncapped-input contract in #215.
The current
read_file_bytesretains byte chunks and joins them.file_bundle_snapshotthen checks binary content and decodes the complete buffer; evidence and bundle construction retain and serialize content before prompt partitioning. The per-request prompt ceiling therefore does not bound peak capture memory.This is a source-confirmed finite-memory scaling limitation. No host-OOM experiment, arbitrary multi-gigabyte guarantee, bounded-RAM claim, or demonstrated P1 failure of ordinary reviews is asserted. The maintainer disposition for #215 is to retain the requested uncapped behavior and address streaming separately, rather than restoring the old cap or expanding that PR into a pipeline rewrite.
Desired outcome
Use streaming or safely owned temporary spooling through the capture, evidence, bundle, partitioning, and scan owners so large complete inputs need not all be materialized simultaneously. Changing only the file-read loop would leave downstream copies and full-input serialization unaddressed.
Acceptance criteria:
Related: #215 and its late ClawSweeper capacity finding. This issue records the accepted limitation and improvement scope; it does not claim the limitation was fixed by #215.