Skip to content

[ENHANCEMENT] Dynamic Thinking Effort: AI self-decides per-turn reasoning effort (experimental) — design + implementation plan #28

Description

@easonLiangWorldedtech

[ENHANCEMENT] Dynamic Thinking Effort — 畀 AI 自己決定每 turn 嘅 reasoning effort(實驗性)

Design + implementation plan。Research-backed(2026-08 industry + arXiv review)。
相關 upstream gap:Roo Code issue #7048(effort override semantics)。

1. 問題同目標

而家 reasoning effort 只可以喺 Settings 一次過定死(固定 per-profile),無法:

  • 用戶畀 model 自己按任務複雜度決定 effort
  • task 中途按需要升/降 effort(例如:先 cheap 開始,tests 失敗先升 high)
  • 用戶喺 chat 入面即時手動調,而唔使入 Settings

目標(本 issue scope):

# 要求
F1 Fix:Anthropic adaptive models 收唔到 thinking 內容 + 冇 thinking token 數據 + 冇 effort envelope(見 §4)
F2 實驗性設定開關(experimental settings)啟用整套 feature
F3 Native tool set_thinking_effort:AI 自己 mid-task 決定 effort,chat 有 display(跟 switch_mode 同款模式)
F4 Top dashboard(TaskHeader)顯示現行 effort,包括 init state(task 開始時嘅 resolved default)
F5 Bottom 手動按鈕(composer 底欄):temp、current chat only,scope 同 AI 用 tool 完全一樣(同一個 task-local state)
F6 Orchestrator:new_task 支援 thinking effort input(model 可指定),用戶喺 enter subtask 之前(click 按鈕之前)可以切換

**非目標(留 backlog):**per-message quick chips / prompt trigger words("ultrathink")、session-level thinking ledger / overthinking 警號、per-mode effort policy、cost-saver mode。

2. Permission 問題:AI auto switch 需唔需要 approval?

結論:唔需要。 set_thinking_effort 直接執行(冇 approval gate),改用 guardrails:

理由 說明
非破壞性 唔改檔案、唔執行命令、唔影響 permission context。對比:switch_mode 改 mode(改可用工具/permission)所以要 approval;new_task spawn 新 task 所以要 approval。effort 兩者都唔係
可逆 任何一刻都可以由(用戶或 model)改返;bottom 按鈕就係 undo 機制
成本有上界 effort 只能喺 model 支持嘅 levels 入面 clamp,唔會無限升;加 escalation cap(§5)
產品邏輯 要求每次 switch 都人肉 approve 會打斷 dynamic loop 嘅目的(用戶走開時 model 就唔可以自我調節),而且每次多一輪 latency。業界先例(Claude Code /effort + adaptive default、OpenCode adaptive-thinking plugin)都冇 approval

Guardrails(替代 approval):

  1. Always notify:每次 model 或用戶改動都喺 chat stream 出齊一線 display(絕無 silent change)
  2. Escalation cap:單個 task 最多 N 次(建議 3)向上調整;超咗就 clamp + notify
  3. Oscillation 偵測:3 steps 內 low→high→low 就 notify 一次(提示可能 cost spiral)
  4. Hard cap:永遠唔可以超過 model 支持嘅最高 level(跟 supportsReasoningEffort capability array)
  5. (後備,預設 off) 如用戶想保守,可加 setting「effort 改動需 approval」— 唔係首版 scope

3. 高層次 UI/UX Design

3.1 單一事實來源:task-level effective effort

resolution order(由強到弱):
  1. task.runtimeThinkingEffort   ← tool 或 bottom 按鈕寫入(同一個 state,last-write-wins)
  2. apiConfiguration.reasoningEffort   ← Settings 持久化值
  3. model.reasoningEffort default  ← model 定義
  • runtimeThinkingEfforttask-local(current chat only):task 結束即 discard,唔寫入 Settings、唔污染 profile。F5 嘅「scope 同 AI 用 tool 一樣」= 兩邊都 read/write 呢同一個 field。
  • 每個 display 位都顯示 effective value + source(default / Zoo (auto) / you),用戶永遠分得清呢個值係邊嚟 — 呢個係「let AI decide」可信賴嘅關鍵(transparency 先至有信任)。

3.2 四個 surface

(a) Experimental settings 開關

  • 跟現有 ExperimentalSettings.tsx 通用 pattern:EXPERIMENT_IDS.DYNAMIC_THINKING_EFFORT + i18n (settings:experimental.DYNAMIC_THINKING_EFFORT.name/.description)
  • 開關語意:「Let the model decide its thinking effort per step, and let you adjust it in-chat」
  • Off = 完全唔注入 tool、唔顯示 bottom 按鈕、TaskHeader chip 保持原樣(顯示 static 值)

(b) Top dashboard(TaskHeader)chip

  • 位置:cost / tokens 排附近,小 chip:
┌──────────────────────────────────────────────┐
│ Fix the login bug…        🧠 High · Zoo      │  ← thinking chip(新)
│ $0.05 · 2.1k in · 980 out · 42%             │
└──────────────────────────────────────────────┘
  • Init state:task 開始時顯示 resolution order 解出嘅 default(例如 🧠 Medium · default) — 用戶唔使等第一次 switch 先知道而家係咩 level
  • source 標記隨改動更新:· default· Zoo(model tool)→ · you(手動)
  • Tooltip:「Current thinking effort for this task. Changed by the model or you; resets when the task ends.」
  • Provider honesty:對冇 native adaptive 嘅 provider,tooltip 照實講 effort 係 per-request 參數(下一 request 生效),唔吹「adaptive」

(c) Bottom 手動按鈕(ChatTextArea 底欄)

  • 位置:ModeSelector 右邊,同一 row 嘅小 button(跟 AutoApproveDropdown 嘅視覺級別):
[ Code ▾ ]  [ api-profile ▾ ]  [⚡▾]   [ 🧠 High ▾ ]      [ 发送 ]
  • Click → 小 menu:effort levels(只列出 model supportsReasoningEffort 支持嘅值,跟现有 ThinkingBudget clamp 邏輯),頂部可加一個「Reset to default」
  • 選咗立即生效:寫入 task.runtimeThinkingEffort + 重建 handler + post 到 webview(chip 更新),chat 出一線 display(source = you)
  • Temp 語意:純 task-local;換 task / 重開 task 即消失;Settings 嘅持久化值唔會被改
  • Disabled state:experiment off、model 唔支持 per-request effort、或者 task 唔存在時

(d) Orchestrator(new_task)pre-enter effort

  • new_task tool schema 加 optional thinking_effort:model 可以為子任務指定起點 effort(例如「呢個 subtask 係 mechanical,用 low」)
  • ChatRow 嘅 newTask ask block(而家有 mode + message + todos + approve/enter 按鈕)加一個 effort selector:
┌─ New task ─────────────────────────────────────┐
│ Mode: Code                                      │
│ Message: Implement the retry logic…             │
│ Thinking: [ High ▾ ]   ← 預填 = model 指定值    │
│                                    [ Enter ]    │
└─────────────────────────────────────────────────┘
  • 用戶喺 click Enter 之前可以改;enter 後呢個值就係 child task 嘅 initial effective effort(child 嘅 TaskHeader chip init state 直接顯示佢)
  • model 冇指定時預填 = parent 而家嘅 effective effort

(e) Chat stream display(跟 switch_mode 同款)

🧠 Zoo raised thinking to High — "multi-file refactor, 3 modules affected"
  • partial streaming 跟 switchModeTool.handlePartial 同款(task.ask("tool", ...))
  • 用戶手動改動用同一個 say 類型,source 不同(icon/文字微調),方便 task history 審計

3.3 Wireframe 總覽

┌─ VS Code Panel ──────────────────────────────────────────┐
│ ┌ TASK HEADER ────────────────────────────────────────┐  │
│ │ Task title…            $0.05 · tokens …   🧠 H·Zoo │  │ ← (b)
│ └────────────────────────────────────────────────────┘  │
│                                                          │
│  👤 refactor the auth module                             │
│                                                          │
│  🧠 Zoo set thinking to Medium — "scoped, low risk"     │ ← (e)
│                                                          │
│  🤖 …working…                                            │
│                                                          │
│  🧠 New task (Code) · Thinking: [High ▾]     [Enter]    │ ← (d)
│                                                          │
│ ┌ COMPOSER ──────────────────────────────────────────┐   │
│ │ [Code ▾] [profile ▾] [⚡▾]        [🧠 High ▾] [➤]  │   │ ← (c)
│ └────────────────────────────────────────────────────┘   │
└──────────────────────────────────────────────────────────┘
Settings → Experimental → ☑ Dynamic thinking effort        ← (a)

3.4 設計決策摘要(對應研究發現)

決策 理由
Control 放 Settings(持久)+ composer 底欄(temp)+ dashboard(顯示)三層 跟 2026 業界收斂方向:VS Code/Cursor 用 picker(≈我哋底欄),Cline/Roo 用 settings;三層唔衝突:settings = 持久 default,底欄 = 呢個 chat 臨時,tool = AI 臨時
「Auto」= 實驗性 feature 本身(而唔係 selector 入面一個 option) 呢個 repo 嘅 effort 已經 per-model 有 default;「model 決定」要 tool + adaptive pass-through 先完整,做實驗性開關乾淨,唔會同既有 settings 語意打架
無 approval,always notify §2
Source 標記(default/Zoo/you) transparency → trust;業界 pattern(VS Code hover 顯示 model、Cursor picker label)都係做 state visibility
只列 model 支持嘅 levels supportsReasoningEffort capability array(OpenAI gpt-5.2 default none、o1 只有 low/med/high 等),避免 send 咗 400
Anthropic 走 native adaptive + soft envelope,其餘走 per-request 參數 §5 provider 矩陣;UI 文案照實(唔得吹)

4. Fix 部分(Anthropic adaptive models)

代碼證據(SDK 0.109.1 已支援全部所需 types,唔使升依賴):

  1. Thinking 內容完全收唔到:src/api/providers/anthropic.ts 從未 send display 參數;Opus 4.7 / Fable 5 / Mythos class 預設 display: "omitted" → stream handler 有 thinking_delta case 但收唔到嘢。Fix:getAnthropicProviderReasoning(src/api/transform/reasoning.ts)adaptive branch 轉出 { type: "adaptive", display: "summarized" }
  2. 冇 thinking token 數據:usage.thinking_tokens 未 parse。Fix:message_start / message_delta 嘅 usage chunk 加 reasoningTokens(0 ⇒ model 該 turn skip thinking — 正正係「model 實際諗咗幾多」嘅 telemetry)
  3. 冇 effort envelope:output_config.effort(low/medium/high/xhigh/max,soft guidance)係獨立 top-level field(放錯入 thinking 會 ValidationException)。Fix:transform 返出 { thinking, outputConfig? },handler 兩個 requestParams branch merge output_config
  4. Bedrock handler(Converse API)同款 fix(additionalModelRequestFields)

呢三筆本身就算獨立 fix(而家 Opus 4.7 用戶完全睇唔到 thinking 都係 bug),可以先行。

5. Provider 支持矩陣

Provider Native「model 決定」 Tool-driven(F3) 備註
Anthropic 4.7+/Fable 5/Opus 5 ✅ adaptive ✅ adaptive + output_config.effort envelope adaptive 時 API 無 per-turn effort 參數;interleaved thinking 自動喺 tool calls 之間諗;Opus 4.7 必須 explicit send adaptive 否則 thinking off
Gemini 3.x / 2.5 ✅ dynamic(default) thinkingLevel per-request 3.1 Pro 唔可以 fully disable;thoughtSignature 已處理
OpenAI native / OpenRouter / OpenAI-compatible ⭕ 無(omit = model default) reasoning_effort per-request clamp 入 capability array;gpt-5.2 default none
DeepSeek V4 ⭕(thinking on + default high) ✅(粗粒度:medium/xhigh 靜默映射 high) reasoning_content round-trip 已處理(openai-format.ts)

6. 實作計劃(file-level)

Phase 0 — Fix(獨立可 merge,~0.5d)

  • src/api/transform/reasoning.ts:getAnthropicProviderReasoning 新 return shape + ADAPTIVE_THINKING_EFFORT_LEVELS
  • src/api/providers/anthropic.ts / bedrock.ts / anthropic-vertex.ts:output_config merge + thinking_tokens parse
  • unit tests:src/api/transform/__tests__/reasoning.spec.ts(adaptive + display + outputConfig)

Phase 1 — 實驗性開關 + types(~0.5d)

  • packages/types/src/experiment.ts:dynamicThinkingEffort
  • src/shared/experiments.ts:DYNAMIC_THINKING_EFFORT config
  • webview-ui/src/components/settings/ExperimentalSettings.tsx + i18n(en + zh-TW,其餘 locale 跟進)

Phase 2 — Tool + Task state(~2d)

  • packages/types/src/tool.ts:set_thinking_effort ToolName
  • src/shared/tools.ts:ALWAYS_AVAILABLE_TOOLS + NativeToolArgs + toolParamNames(thinking_effort)
  • src/core/prompts/tools/native-tools/set_thinking_effort.ts:schema(effort enum、reason required)+ 指引 copy(「揀最低安全 effort;ambiguity/debugging/risky changes 先升;state your reason」)
  • src/core/prompts/tools/native-tools/index.ts:註冊
  • src/core/prompts/tools/filter-tools-for-mode.ts:gating = experiment on AND model supports per-request effort(照 generate_image / run_slash_command 先例;prompt cache 守則:一旦 task 開始,tool list 穩定唔隨 state 增減)
  • src/core/tools/SetThinkingEffortTool.ts:executor(無 approval);clamp vs capability;escalation cap + oscillation 偵測;write task state;say display
  • src/core/assistant-message/presentAssistantMessage.ts + NativeToolCallParser.ts:dispatch case(照 switch_mode)
  • src/core/task/Task.ts:runtimeThinkingEffort + setRuntimeThinkingEffort(effort, source)(merge 入 apiConfiguration copy + updateApiConfiguration() rebuild handler — 現有 profile 切換先例;task 結束自然 discard)+ post 到 webview

Phase 3 — Webview UI(~2d)

  • webview-ui/src/components/chat/TaskHeader.tsx:chip + source 標記 + init state
  • webview-ui/src/components/chat/ChatTextArea.tsx:ThinkingEffortButton(底欄)
  • webview-ui/src/components/chat/ChatRow.tsx:case "setThinkingEffort" display(ask/say 兩態)
  • ExtensionState / message handler plumbing(extension→webview effort 變更事件)
  • Vitest:ChatRow case、chip、button interaction(webview-ui AGENTS.md 兩層策略)

Phase 4 — Orchestrator(~1d)

  • src/core/prompts/tools/native-tools/new_task.ts:+ optional thinking_effort
  • src/core/tools/NewTaskTool.ts:ask JSON 帶 effort;pass-through
  • webview-ui/src/components/chat/ChatRow.tsx newTask block:effort selector(可喺 Enter 前改)
  • src/core/webview/ClineProvider.ts:delegateParentAndOpenChildthinkingEffort? → child task init 時 set

順序

P0 → P1 → P2 → P3 → P4;P0 可獨立先行。總估 ~6d(單人)。

7. 風險

風險 緩解
Cost spiral(model over-escalate / oscillate) §2 guardrails:cap + 偵測 + always notify + 用戶可即時壓低
Prompt cache 爆(tool 條件注入) tool 只在 experiment on + model 支持時暴露,task 內穩定;description 靜態
Anthropic 粒度粗(adaptive 只係 on/off + soft envelope) UI 文案照實;對佢 tool 改動係 envelope 而唔係 hard level
舊 o1 世代 thinking + tool calls 限制 跟 model capability flags gate;不支持就唔暴露 tool
用戶誤以為「temp」會持久化 文案明確「resets when the task ends」+ Settings 值永不變

8. 測試計劃(跟 AGENTS.md test pyramid)

  • unit(src):transform(adaptive/display/outputConfig/clamp)、filter gating、SetThinkingEffortTool(cap、oscillation、無 approval)、Task state(reset、merge)、new_task effort 參數
  • webview-ui(Vitest+JSDOM):ChatRow 新 case、TaskHeader chip(init + source 更新)、ThinkingEffortButton(clamp、disabled、click → postMessage)
  • e2e:唔需要(唔涉及新 boundary;tool 執行已有 e2e 覆蓋面)

9. 驗收標準

  • Experimental 開 + 支持 model:AI 可以喺 task 中途 call set_thinking_effort,chat 出一線 display,下一個 request 實際帶新 effort(network / 日誌可證)
  • Top dashboard chip:task 開始即顯示 default;model 或用戶改動後即時更新 + source 正確
  • Bottom 按鈕:只改現行 task;換 task 後重置;Settings 值唔變
  • Opus 4.7 / Fable class:thinking 文字可見(display fix)+ 有 thinking token 數據(usage fix)
  • Orchestrator:newTask 可帶 effort;Enter 前可改;child init 生效
  • Experiment off:完全無痕跡(無 tool、無按鈕、無 chip 改動)
  • 無 approval gate;所有改動皆 notify;escalation cap 生效

執行狀態 (synced 2026-08-22)

Base reference: upstream/main = 87077e1b1 (moved 2026-08-23: 11 commits since db52d7f incl. v3.80.0 release prep Zoo-Code-Org#1347, Zoo-Code-Org#1351/Zoo-Code-Org#1340 async fixes, Zoo-Code-Org#1323 task persistence; merge-tree preview dte-3+newmain=4be3fe72, dte-5+newmain=b534d097 = clean). ⚠️ Zoo-Code-Org#1345 (ViX3L feat(ollama): reasoning effort selectors, OPEN non-draft, 15pass/2fail codecov/patch+e2e-mock, REVIEW_REQUIRED, unmerged) adds webview ReasoningEffortSelector into ChatTextArea bottom bar (PR-4's specified toggle slot) + utils/reasoning-effort.ts + SettingsView refactors → PR-4 must base-refresh post-Zoo-Code-Org#1345 and place the toggle adjacent; PR-3/PR-5 zero overlap. (prior: db52d7f = Gemini Flash-Lite Zoo-Code-Org#1334 merged 2026-08-22.) PR-1/PR-2 base = 1ad8f528d (1 commit behind, gemini-only, non-conflicting → no rebase; re-triggering CI/CodeRabbit not worth it). Wave-2 (PR-3/PR-5) will base-refresh via git merge upstream/main at launch so they are truly on the latest main (gemini commit cancels out of the main..head diff as a common ancestor).

F1 fix — delivered

Feature rollout — 5 stacked PRs (one issue each, upstream):

PR Issue Branch Base State
1/5 experimental setting Zoo-Code-Org#1328PR Zoo-Code-Org/Zoo-Code#1336 (a05830c + 1cf4f0d + base-merge 5db5cf4) feat/dte-1-experiment main DONE (final, on latest main): merge 5db5cf4 = 1ad8f52/Zoo-Code-Org#1069 (clean ort, locale coexistence re-verified); new head 15/15 CI green; CodeRabbit real review new range = 'No actionable comments 🎉'; Codecov all-coverable-lines; final comment updated in place (base-update section); awaiting maintainer merge (BLOCKED = review policy)
2/5 task state + metadata + envelope Zoo-Code-Org#1329PR Zoo-Code-Org/Zoo-Code#1338 (6ea45b3 + base-merge 9275aa1 + fix 14d1f35 + docstring 90b47b0, UNDRAFTED) feat/dte-2-task-state main 🔄 MAJOR RESOLVED (noAct 🎉) + Description PASS + 100% patch cov (30/30+10/10). Premature FINAL 09:11 REJECTED (Docstring 33%<80%) → JSDoc instruction → docstring commit 90b47b0 pushed 09:38 (all 3 modified fns documented). CodeRabbit settled 01:50:35Z re-scanned 90b47b0 but docstring still 33.33% = STALE CACHE (verified: all 7 diff fns documented in code = 100%; bot noAct 🎉 + check pass = non-blocking). 09:54 override: stop docstring iteration, finalize. **10:22 agent stuck on final PATCH (~6.5min zero I/O) → interrupt + finish instruction; ALL substantive gates VERIFIED (14/14 + noAct 🎉 + 100% cov) → final comment PATCH landed 03:02:37Z (5345 chars; live-verified: 90b47b0 + 14/14 + stale-cache note + corrected title/body line + signature). FINAL report received 11:0x (all gates self-verified, hard-rule compliant). ✅ COMPLETE (awaiting maintainer merge). Docstring-stale root cause (agent closing report 11:0x): CodeRabbit's incremental review skipped both JSDoc files as "similar to previous changes (2)" so the coverage check never re-measured; the /meow full re-review (posted 02:18:55Z) never produced a fresh walkthrough (~9h later still the 01:50:35Z run) — stale heuristic proven, non-blocking (noAct + check pass).
3/5 set_thinking_effort tool Zoo-Code-Org#1330 feat/dte-3-native-tool PR1+PR2 🚀 LAUNCHED 10:22 (wave 2): wt-dte-3; fcc3cf4 feat commit (33 files +1440; SIZE approved by user — do not grow) + 0ab4a60 base-refresh to main 78c712a; DRAFT PR Zoo-Code-Org/Zoo-Code#1354 OPEN (review fixes 19954d3 landed: 24 files +266/−75 — parser strictness, 'disable'-only capability hidden, baseline-seeded oscillation guard, i18n values 17 locales, 2 out-of-scope replies → PR-1 Zoo-Code-Org#1336; parent-verified 19/19 checks green) → bot tail verified by parent 22:3x: all 6 findings confirmed/withdrawn (11:49-11:50Z) = CLEANuser directive 22:3x: e2e moves to SEPARATE addendum PR-6 (feat/dte-3-e2e from 19954d3) — undraft NOW (gates already met: 19/19 + bot clean + 100% cov) → agent e06e4e01 executing: undraft Zoo-Code-Org#1354 → addendum PR-6 DRAFT → CI → undraft → bot → final
4/5 webview UI Zoo-Code-Org#1331 feat/dte-4-webview-ui PR3 🚀 LAUNCHED (user directive 16:1x): wt-dte-4 @ 0ab4a60 (stack dte-3 head); launch re-verify done: Zoo-Code-Org#1345 still OPEN non-draft (base 78c712a, 17pass/2fail, MERGEABLE) → pending-overlap rule (note in PR body; toggle lands adjacent to their selector post-merge) + per-tab Zoo-Code-Org#977/Zoo-Code-Org#981 NOT merged → extension-state push path + follow-up note; pnpm install done; agent: implement (TaskHeader chip + composer toggle + display + message types) → JSDOM + Playwright CT + e2e Part C round-trip → DRAFT → CI → undraft → bot → 100% cov
5/5 orchestrator new_task Zoo-Code-Org#1332 feat/dte-5-orchestrator PR2 🚀 LAUNCHED 10:22 (wave 2): wt-dte-5; DRAFT PR Zoo-Code-Org/Zoo-Code#1355 OPEN @ f6410bb (feat 146c5c8 = 15 files +1041/−16, SIZE approved by user — do not grow; base-refresh to main 78c712a clean; all post-merge verification green; also fixed ChatView approval pass-through gap found in testing) → e2e files written uncommitted (16KB test + subtasks DTE markers) → user directive 22:3x: e2e moves to SEPARATE addendum PR-7 (feat/dte-5-e2e from f6410bb)parent verified 3 UNRESOLVED MAJOR CodeRabbit comments (NewTaskTool boolean capability / ClineProvider post-mode-switch child revalidation / ChatView prefill normalization — full fix specs extracted + bot's proposed ChatView diff) → agent 4c597817 executing: STEP 1 e2e → addendum branch (DRAFT PR-7) → STEP 2 fix 3 Majors (1 fix commit) → STEP 3-5 push → fresh CI green → reply to bot → THEN undraft → cov → final

Per-PR protocol: draft PR until CI fully green → undraft → CodeRabbit review fully addressed → 100% test coverage on patch lines → only then next PR. Each PR <= 1000 diff lines (standalone, vs stacked base). Max 2 implementation agents concurrent; waves [1∥2] → [3∥5] → [4].

Design updates since original post:

  • Composer button placement: between the API-profile selector and the auto-approve (⚡) control (user-visible row order, plan §3.2(c)/§3.3 wireframe).
  • §3.5 reuse design: effort body fields stay in the transform layer (RequestConfigBuilder = SDK options only); per-request override rides metadata.reasoningEffort (same pattern as abort signal / Bedrock metadata.thinking); webview effort display state goes in the per-tab view state container (sequencing dependency on the view-local-state merge noted).
  • §8/§9 updated for the webview-ui two-layer test strategy (Vitest+JSDOM behavioral + Playwright CT visual snapshots in the same PR) and Codecov gates (80% src patch / 70% webview patch; requirement: 100% on patch lines).

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions