Repository navigation
Add a loop guard to the Strands runner - #213
Merged
Merged
Conversation
A small model can repeat one tool call hundreds of times. Under --agent strands-harness, minicpm5-2b ran `git diff HEAD --stat` 674 times in one task and a `grep` 542 times in another; the second hit the recursion limit after 1,148 turns with no artifacts. Those two tasks used 125M of the run's 209M tokens. _LoopGuard counts every tool call by tool name and exact input over the whole attempt, since a loop can cycle through a few commands. Past LOOP_WARN_REPEATS (20) it cancels the repeated call through BeforeToolCallEvent.cancel_tool and tells the model to move on. At LOOP_STOP_REPEATS (40) it sets AfterToolsEvent.end_turn, which ends the attempt like a normal finish, so the harness scores the artifacts already written and does not retry. Each action is a loop_guard event in the stream, and the result event records loop_guard_stopped. The thresholds come from 42 task attempts across both Strands runs of minicpm5-2b: no healthy task repeated an identical call more than 16 times; the three loops reached 29, 542 and 674. The guard covers --agent strands and --agent strands-harness.
aarora79
force-pushed
the
feat/strands-loop-guard
branch
from
October 7, 2026 14:05
b56a629 to
70ad507
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds a loop guard to the Strands runner. It covers
--agent strandsand--agent strands-harness. It builds on #212, whose commit is included here; merge #212 first.Why
A small model can repeat one tool call hundreds of times. In the 21-task
--agent strands-harnessrun ofminicpm5-2b, two tasks looped:lifecycle-workflow-webhooksgit diff HEAD --statlogout-id-token-hint-out-of-browser-urlgrepThose two tasks used 125M of the run's 209M tokens.
How it works
_LoopGuardis a Strands hook provider that counts every tool call by tool name and exact input over the whole attempt. It counts totals, not back-to-back repeats, because the loops cycled through several commands.LOOP_WARN_REPEATS(20),BeforeToolCallEvent.cancel_toolcancels the repeated call. The model gets back a message telling it the result will not change and to finish or try something else.LOOP_STOP_REPEATS(40),AfterToolsEvent.end_turnends the attempt like a normal finish. The harness then scores the artifacts already written and does not retry.loop_guardevent instrands-stream.jsonl, and theresultevent recordsloop_guard_stopped.Thresholds
Across 42 task attempts in both Strands runs of
minicpm5-2b, no healthy task repeated an identical call more than 16 times. The three loops reached 29, 542 and 674. Warning at 20 and stopping at 40 leaves every measured healthy task untouched.Testing
stop_reason=end_turnand the text "Stopped by the loop guard: the same tool call was repeated 40 times."hello-worldsmoke run under--agent strands-harnessfinished 6/6 in 20 turns, and the guard did not fire.uv run python -m unittest discover -s testspasses inbenchmarks/, apart from the known intermittent failure intest_run_throughput_harness(Flaky test: test_untracked_file_written_during_the_level_is_quarantined fails about 1 in 15 runs #211). Ruff and Bandit are clean, and thesecurity-checkgate passed.