[lizradway]

Human Overview

Description

ContextManager graduates in #5205, and after that the Offload .when() conditions can only change in 2.0, so their meanings need to be right first. On main, setting utilization silently switches drop, truncate and summarize into whole-message mode. That causes three problems:

• threshold picks messages rather than blocks.
• utilization decides granularity as well as timing.
• Summarize can merge messages that aren't next to each other into one summary, which reorders the content.

The new rule is that the target chooses what a strategy acts on, threshold sets the minimum size to act on, utilization only says when it fires, and preserve_recent only says what's protected. Target "*" always acts on whole messages, and threshold then measures each message. Every other target acts on blocks, and threshold measures each block. When threshold isn't set, there's no size check, so every matching message or block is acted on.

	Target "*" (messages)	Any other target (blocks)
drop	removes eligible messages and keeps tool-use/result pairs together: [Dropped: N messages]	replaces each block with [Dropped]
truncate	with no threshold, keeps the head and tail within preview_tokens and elides the middle. With a threshold, shortens each large message in place and leaves the newest one alone	previews each block
summarize	summarizes each contiguous run	summarizes each block

Values follow one rule: a number counts in the condition's unit, and "N%" is a share of the whole. threshold="5%" means 5% of the context window. Decimals like 0.85 are not accepted, so each value has one meaning in every condition.

Public API Changes

# Before
Offload.drop("tool_results")                      # was usable as a strategy directly
Offload.summarize("*").when(utilization=0.85)

# After
Offload.drop("tool_results").when()               # .when() is required; an empty one runs on every new message
Offload.truncate("tool_results").when(threshold="5%")              # tokens, or a share of the window
Offload.summarize("*").when(utilization="85%", preserve_recent=4)  # 0.85 raises: write "85%"
Offload.drop("*").when(threshold=1000)            # drops old messages larger than 1,000 tokens
Offload.truncate("*").when(threshold=1000)        # shortens each of them in place
Offload.drop("media").when(preserve_recent=1)     # new target: images, documents, video, audio
Offload.drop("*").when()                          # ValueError: set utilization, threshold, or preserve_recent
Offload.drop("media").when()                      # ValueError: would remove media before the model sees it

• Offload.* returns an opaque builder that is not a ContextStrategy. If you pass one in strategies without calling .when(), it raises TypeError.
• The new "media" target matches image, document, video and audio blocks outside tool results. It needs utilization or preserve_recent. Without either, the strategy would run on each message as it arrives and replace the media before the model's first call.
• A tool list can mix included and excluded tools: it acts on the includes minus the excludes, so ["bash", "!python"] acts on bash. A list whose excludes cancel every include (["bash", "!bash"]) matches nothing and raises ValueError when it's built. Before, a mixed list logged a warning and ignored the excludes.
• OffloadStrategyBuilder is exported from strands.experimental.context_manager, since every Offload.* call returns one.
• A whole-number utilization counts conversation tokens and must be at least 1,000, so a missing % like 80 fails with a hint to write "80%". If it's larger than the model's context_window_limit, the strategy raises ValueError on init. If the model has no context_window_limit, it warns and skips that check; the count still fires at the right size, since it doesn't depend on the window.

Behaviour changes for existing configs

• A decimal utilization or preserve_recent (0.85) raises a ValueError whose message gives the "N%" to write. A number now always counts, so utilization=1, which meant 100%, is rejected too. The presets and the "auto" and "agentic" context managers use "N%".
• Drop or summarize with a non-"*" target plus utilization now acts on blocks.
• drop("*") and summarize("*") now act on whole messages for every combination of conditions.
• truncate("*") without threshold is now middle-out.
• Without a threshold, a strategy acts on blocks of any size. Truncate no longer defaults the threshold to preview_tokens, because its preview already leaves blocks that fit untouched. It also skips blocks it has already truncated, so it doesn't re-truncate its own previews.
• A block strategy gated by utilization or preserve_recent leaves message 0 and the last message alone, as message mode already did.
• threshold must be at least 1, or a "N%" below "100%". A threshold of 0 got past the "*" guard and acted on every message, and "100%" could never select anything.
• Empty and whitespace-only text blocks are skipped.
• A dropped media block becomes [Dropped: image, removed from context after it was viewed], not [Dropped]. On Opus 5 the bare marker read like media that never arrived, and the model took back answers it had already given correctly.
• An offloaded media block names its kind the same way: [Offloaded: image, ~N tokens, removed from context after it was viewed].
• [Offloaded] and [Summarized: …] leave out the token count when it's 0, which is what the heuristic counter returns for media.
• Truncate checks threshold > preview_tokens for every target, including "*", whose threshold would otherwise select messages it can't shorten. A token threshold is checked when the strategy is built, and a "N%" threshold on init, against the context window.
• overflow_protection drops the oldest messages instead of truncating them.

Risks / Callouts

• A summary can still land after the message that follows it. This happens when the message right after a summarized run is an assistant message. _insert_marker appends the summary to the end of that message to keep its stash block indices stable. main already does the same for every marker. A pinned assistant message next to the gap also absorbs the marker. I'd fix this in a follow-up so the stash-ref trade-off gets its own review.
• Under the heuristic token counter, media size is unknown. The heuristic counts media as 0 tokens, so a media block always passes threshold instead of never passing it, and its [Offloaded] marker leaves out the size. Estimating media size in the fallback counter is a separate follow-up.
• Experimental API, but breaking. The ! in the commits is deliberate.

Related Issues

Prerequisite for #5205. The TypeScript side is in #5325.

Documentation PR

Docs will follow in a separate PR.

Type of Change

Breaking change

Testing

• Python: hatch fmt --formatter, hatch fmt --linter, and hatch test all pass (7983 passed). hatch run prepare also passes.

• Complexity: npm run complexity. The highest score among the touched functions is 14, for _apply_per_block.

• Real-model runs on Bedrock Claude Haiku 4.5 and Opus 5, using a script that has the agent read two large files and then asks it to recall facts from both:

  • drop("*"), truncate("*", {"preview_tokens": 200}) and summarize("*") with threshold=1000 each reduced the old large messages and left the newest message alone. Both models then recalled the earlier facts correctly.
  • drop("media").when(preserve_recent=1) across three image turns replaced the older image with the new marker. Both models then listed every color they'd seen. With the bare [Dropped] marker, Opus 5 took back its earlier answer.
  • Through an Agent with a mocked model, drop("media").when() now raises, and with preserve_recent=1 the model still receives the image on the turn it was sent.
  • Native Bedrock token counting returned AccessDenied for this account, so the runs used the heuristic counter.

• I ran hatch run prepare

Additional Details

Alternatives considered

• Let utilization keep choosing granularity, and reject the conflicting combinations instead. Rejected because one condition would still mean two things, and drop's message and block forms would still behave inconsistently.
• Act on blocks when "*" has a threshold. This was an earlier version of this PR. Rejected because "*" would then mean blocks or messages depending on another condition, the same coupling this PR removes from utilization.
• Default the threshold to preview_tokens for truncate. Rejected because it silently skipped media and small blocks, and truncate's preview already leaves content that fits unchanged.
• Keep decimals as aliases for "N%". This was an earlier version of this PR. Rejected because a number then meant a share or a count depending on its size: TypeScript can't tell 1 from 1.0, and the two SDKs share these values, so utilization of 1 meant 100% while 2 meant two tokens, and each condition needed its own parsing rules. Breaking experimental configs now is cheaper than graduating two spellings that can only be removed in 2.0.

Checklist

• I have read the CONTRIBUTING document
• I have reviewed and understand every line of code in this PR, including any generated by AI tools, and I can explain why it works
• My change is focused and reasonably small; I have split unrelated work into separate PRs
• I have added any necessary tests that prove my fix is effective or my feature works
• I have updated the documentation accordingly
• I have added an appropriate example to the documentation to outline the feature, or no new docs are needed
• My changes generate no new warnings
• Any dependent changes have been merged and published

---

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

🤖 Generated with Claude Code

[codecov]

Codecov Report

❌ Patch coverage is 96.36364% with 10 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...trands/_context_manager/strategies/offload/base.py	96.47%	3 Missing and 2 partials ⚠️
...y/src/strands/_context_manager/methods/truncate.py	88.88%	1 Missing and 1 partial ⚠️
...s/_context_manager/strategies/offload/summarize.py	94.11%	1 Missing and 1 partial ⚠️
...ds/_context_manager/strategies/offload/truncate.py	98.27%	0 Missing and 1 partial ⚠️

📢 Thoughts on this report? Let us know!

[lizradway]

@strandly-the-agent pls review

[github-actions]

Issue: The Risks section notes a summary can still land after the assistant message that follows a summarized run, reordering content. This is a real correctness nuance that's easy to lose once this PR merges.

Suggestion: File a tracking issue for the follow-up fix (and reference it here) so the stash-ref trade-off gets its own review rather than relying on the PR description to carry it forward.

[github-actions]

Issue (process): This is a breaking change (feat(context)!) to the public Offload.*().when() contract — threshold → block_threshold, utilization no longer selecting granularity, .when() now required. Per API_BAR_RAISING, a change to an existing customer-facing API contract should carry the api/needs-review label and a designated API reviewer before merge.

Suggestion: Add api/needs-review and route to an API reviewer. The PR description already contains the required materials (use cases, full signatures, before/after examples, alternatives considered), so the review should be quick. Since the TS counterpart is in #5325, confirm both land with aligned semantics.

[github-actions]

Assessment: Comment

The new rule — target + block_threshold choose what, utilization chooses when, preserve_recent chooses what's protected — is a clear, well-motivated simplification, and the builder/.when() split reads nicely. Tests are thorough and parametrized, and the PR description is exemplary (use cases, signatures, alternatives, risks). My comments are mostly refinements plus one process gate.

Review themes

• API process: Breaking public-API change should carry api/needs-review + a designated reviewer before merge; the description already has everything a reviewer needs.
• Config coherence: Percent block_threshold bypasses the > preview_tokens convergence guard and can resolve to a silent no-op for truncate.
• Robustness: _is_truncated prefix-matching is a content-dependent heuristic — fine if intentional, worth a note.
• Follow-up tracking: The acknowledged summary-ordering edge case should get a tracking issue so it isn't lost after merge.

Nice work carving out semantics that will hold up through graduation.

[strandly-the-agent]

Changes requested — one regression in message-mode summarize; the redesign itself holds up.

• 🔴 summarize("*") now mutates run-by-run and only repairs alternation after the loop. An aux-model error on run 2+ leaves consecutive assistant messages persisted in agent.messages (repro: 3/18 pinned layouts fail). On main the batched summary was computed before any splice, so this is new. try/finally fixes it (inline).
• 🟡 utilization=80 means 80 tokens, so the likeliest typo for "80%" fires on turn 1 silently (inline).
• 🟡 block_threshold=0 bypasses the "*" message-mode guard that "0%" enforces, re-enabling eager drop-everything (inline).

The public .when() surface changed and there's no needs-api-review label — worth one, since this is explicitly the pre-graduation contract.

Verified / questions / reading order / appendix

✅ Verified at bebc2429

• ruff check + ruff format --check clean; mypy src/strands/_context_manager clean; pytest tests/strands/_context_manager → 327 passed.
• Repros through Agent + ContextManager with a mocked model: toolUse/toolResult pairing and alternation hold for drop("*")/truncate("*") with a pinned message at every index, inside tool loops, and for preserve_recent 0–4; message-mode truncate is idempotent across turns; overflow_protection on 3- and 5-message histories preserves the live turn; the eager hook skips pinned messages and retrieve_context results; percent parsing rejects " 5 %", "5", "105%", "-5%", "abc%", "", bools, nan, inf.
• No .when(threshold= call sites remain outside team/designs/0015-context-manager.md.

Questions (non-blocking)

• Is the absolute-token form of utilization needed by anyone? Dropping it removes the int/float cliff at 1 and makes the hint str | float honest.
• OffloadConditions is exported from strands.experimental.context_manager but no public callable accepts it (.when() is kwargs-only), while OffloadStrategyBuilder — the return type of every Offload.* call — isn't exported. Intentional?

Reading order: base.py:28-135 (parsing) → base.py:385-471 (mode selection, dispatch) → summarize.py:59-89 → truncate.py:74-98 → presets.py.

Appendix — non-blocking (5)

• presets.py:70-72: the raw-builder TypeError says "call .when() with no conditions", which for a "*" target then raises the ValueError at base.py:411. Mention utilization/preserve_recent too.
• truncate.py:65: the block_threshold > preview_tokens check only covers ints; "0.1%" of a 200k window is 200 tokens and passes. Harmless since _is_truncated stops re-truncation, but test_truncate_skips_preview_check_for_percent_threshold pins the asymmetry as intended behaviour.
• methods/truncate.py _is_truncated: a tool result whose genuine content starts with [Truncated: is never truncated (repro'd: 20k chars left in place). Contrived.
• presets.py:24: "drop tool results older than 5 messages" — preserve_recent counts matching messages, not messages.
• Test gaps: utilization int/float boundary (1 vs 1.0 vs 2), and preserve_recent=0 vs "0%" equivalence.

Evidence (pytest log, F2 repro script + output, the Agent/ContextManager test harness, and the three raw pass reports) is in the artifact store under harness-sdk/pr/5324.

[lizradway]

@strandly-the-agent pls review

[strandly-the-agent]

Approve — the 🔴 is fixed and verified; the two 🟡 threads are still open but non-blocking.

Status of my 10-21 findings (bebc2429 → d3b5e992):

• 🔴 summarize run failure → invalid history: fixed (552e578 guards _summarize_content per run; test_a_failed_run_is_left_intact_and_the_others_still_summarize is my repro scenario and asserts alternation). The one call left outside the try, model.count_tokens(safe), can't raise on Bedrock (falls back to the heuristic), so no residual.
• 🟡 utilization=80 = 80 tokens: still open — 9d8f309 caps token counts at the context window instead, which catches 200001 but not 80. Deliberate call; leaving the thread for you to close.
• 🟡 threshold=0 bypasses the "*" guard: still open, unchanged (drop("*").when(threshold=0) is eager while "0%" is rejected). Non-blocking.

New in the delta — block_threshold → threshold rename, gated block strategies skip the newest message, message-mode truncate skips its own markers, raw-builder TypeError wording — all check out. The PR body still documents block_threshold and #5325's title still says blockThreshold (TS source already uses threshold). Still no needs-api-review label.

Verified / appendix

✅ Verified at d3b5e992: ruff check + ruff format --check clean; mypy src/strands/_context_manager clean; pytest tests/strands/_context_manager → 337 passed. Direct probes: utilization=200001 → ValueError at init() naming the window; drop("*").when() and the raw-builder path raise the new messages; _apply_per_block pops the newest message only when not eager.

Appendix — non-blocking (2)

• base.py:443-446 vs :457: with context_window_limit=None, _check_utilization_fits skips the check but _utilization_reached divides by DEFAULT_CONTEXT_WINDOW_LIMIT, so utilization=10**9 passes init() and never fires. Probed.
• truncate.py _has_elision_marker: a message that carries an appended marker (the _insert_marker neighbour case) is excluded from eligibility for good, so its own content is never elided. One message, bounded — probed with a 5k-char carrier.

[github-actions]

Assessment: Comment (re-review at 9401589)

The warn-instead-of-raise fix is correct and well-tested. I verified the premise: estimate_utilization and _window_limit share the same context_window_limit or DEFAULT fallback, so for a token-count utilization the comparison in _utilization_reached cancels to tokens >= value — genuinely window-independent, so hard-failing when no window is set was indeed too strict. stacklevel=1 matches the existing estimate_utilization warning pattern, and coverage (test_token_utilization_without_window_warns_on_init plus the raise/within-window cases) is solid.

State of prior review items

• ✅ block_threshold=0 guard bypass — resolved: renamed to threshold, which now rejects < 1.
• ✅ summarize run-by-run regression — resolved (d3b5e992), with a covering test.
• ✅ truncate percent convergence — resolved: explanatory comment plus the _is_truncated convergence guarantee.
• 🔄 utilization=80 token typo-cliff — still open; this commit makes it more reachable (no window needed). Left a follow-up on that thread rather than duplicating.
• 🔄 _is_truncated prefix heuristic — still open, minor.
• ⚠️ API process — still no api/needs-review label on a breaking pre-graduation contract change.

No new blocking issues in this commit — it's a clean, well-scoped fix.

[github-actions]

Assessment: Comment (re-review at a7cebc6)

Docs-only change since 9401589: _is_truncated now documents that detection is a text-prefix match, making the simplicity-over-robustness tradeoff explicit. This addresses my last remaining open thread — no code or behavior change, nothing new to flag.

Remaining non-blocking items are the still-open utilization=80 token typo-cliff (another reviewer's thread) and the api/needs-review label on this breaking pre-graduation contract.

[lizradway]

@strandly-the-agent pls review

[strandly-the-agent]

Changes requested — both of my open 🟡s are now fixed; one new 🔴 on the new "media" target.

Status of previous findings (d3b5e992 → a7cebc61):

• 🟡 utilization=80 = 80 tokens: fixed — _MIN_UTILIZATION_TOKENS = 1000 (base.py:79) rejects it with a hint to write "80%". Probed.
• 🟡 threshold=0 bypassing the "*" guard: fixed — value < 1 now rejected (base.py:122). Probed.

New:

• 🔴 Offload.drop("media").when() runs eagerly, so the user's image is replaced before the model's first call with [Dropped: image, removed from context after it was viewed] — the model never saw it and is told it did. Repro'd through a real Agent; inline with a suggestion.

The "*"-always-messages rule, threshold as a size filter, and the new gated-path protections all check out (attacks A–D, F–G pass). The PR body is now a good spec. Still no needs-api-review label for a new target + semantics change on the pre-graduation contract.

Verified / appendix

✅ Verified at a7cebc61: ruff check + ruff format --check clean; mypy src/strands/_context_manager clean; pytest tests/strands/_context_manager → 353 passed. Repros: drop("*").when(threshold=N) splices toolUse/toolResult pairs in both directions, never removes messages[-1], refuses a pair anchored on message 0; truncate("*").when(threshold=N) never touches message 0 and converges on the second pass; summarize("*").when(threshold=N) with two runs keeps alternation; drop("media").when(preserve_recent=1) keeps the newest image and excludes message 0; media-only messages become a single text block (never empty content); media inside toolResults is skipped. Evidence (repro-eager-media.py/.log, findings-adv3.md, adv3-bcdefg.py, pytest-a7cebc61.log) is in the artifact store under harness-sdk/pr/5324.

Appendix — non-blocking (3)

• truncate.py:100-111 _shorten_each: a message made of many blocks each under preview_tokens but over threshold in total is never shortened, yet is re-measured (one count_tokens per message) on every model call. Cost only.
• base.py:299-302: an oversized messages[1] paired with message 0 is permanently unremovable and re-measured each pass. Correct, just worth knowing.
• threshold="100%" is accepted and is a permanent no-op (resolves to the whole window). Reject or document.

[lizradway]

@strandly-the-agent pls review

[strandly-the-agent]

Approve — the 🔴 is fixed and verified; nothing new to flag in the a7cebc61 → 51831e42 delta.

• 🔴 eager "media" removing the image before the first model call: fixed (ee2b268, base.py:456-460). Re-ran my Agent repro: drop("media").when() now raises Offload.drop("media") would remove media before the model sees it; set utilization or a preserve_recent above 0; .when(threshold=100) and truncate("media").when() raise too. The gated form drop("media").when(preserve_recent=1) over three image turns shows the model the new image each time and drops only older ones (message 0 kept).
• Also verified: threshold="100%" rejected; percent threshold checked against preview_tokens at init() ("0.1%" of 200k → 200 tokens → clear error); ["bash", "!bash"] raises instead of silently ignoring the exclusion while ["bash", "ls", "!bash"] → {ls}; unmeasured media (0 tokens) always passes a block threshold; OffloadStrategyBuilder exported.

Still no needs-api-review label; this is the one thing I'd want before it leaves draft.

Verified

ruff check + ruff format --check clean; mypy src/strands/_context_manager clean; pytest tests/strands/_context_manager → 372 passed at 51831e42. Repros: repro-eager-media.py (now raises), gated_media.py (3 image turns, block keys per model call: [text,image] → [text,image],[text],[text,image] → [text,image],[text],[text,text],[text],[text,image]). Delta reviewed directly — fixes to filed findings plus small validation tightening, no fan-out.