Skip to content

fix(bin): raise grok reasoning-effort ceiling to xhigh - #2429

Open
rqms40 wants to merge 3 commits into
kunchenguid:mainfrom
rqms40:fm/fm-grok-xhigh-effort
Open

fix(bin): raise grok reasoning-effort ceiling to xhigh#2429
rqms40 wants to merge 3 commits into
kunchenguid:mainfrom
rqms40:fm/fm-grok-xhigh-effort

Conversation

@rqms40

@rqms40 rqms40 commented Aug 15, 2026

Copy link
Copy Markdown

Intent

Make xhigh reachable for grok so a crewmate spawn resolved to grok / grok-4.6 / xhigh actually passes --reasoning-effort xhigh to the launch command, and so config/crew-dispatch.json holding {"harness":"grok","model":"grok-4.6","effort":"xhigh"} validates cleanly in bootstrap with no CREW_DISPATCH: invalid.

Firstmate previously recorded grok's reasoning-effort ceiling as high from grok 0.2.99. That is now wrong. Verified 2026-08-15 on grok 1.0.3 (1a29d5bc12): --reasoning-effort accepts exactly xhigh, high, medium, low. grok --model grok-4.6 --reasoning-effort bogus prints "use one of: xhigh, high, medium, low". grok --model grok-4.6 --reasoning-effort xhigh --print succeeds. The same accepted set was reconfirmed on the installed grok 1.0.4 (d846eb93d9). max stays unsupported for grok: grok 1.0.3/1.0.4 reject it, and firstmate must keep omitting max from grok launch flags and keep rejecting grok:max in crew-dispatch validation.

The captain's standing preference is that crewmates run grok-4.6 at xhigh. Previously that value was accepted into task metadata and then silently dropped from the launch command, and config/crew-dispatch.json could not even hold it.

Known sites: bin/fm-spawn.sh grok case in the effort-flag mapping and its surrounding comment; bin/fm-bootstrap.sh jq dispatch-profile validator grok effort list; .agents/skills/harness-adapters/SKILL.md launch-profile axes table row for grok and any prose repeating the old 0.2.99 ceiling; any docs/ prose or tests/ expectation that encodes grok's accepted effort set (tests/fm-bootstrap.test.sh and tests/fm-spawn-dispatch-profile.test.sh at minimum).

Design decision already made: statically raise the ceiling to xhigh and treat grok 1.0.x as the supported floor. Do not invent CLI-probe machinery. Existing adapters are static tables; bootstrap must validate crew-dispatch.json even when grok is not installed.

Accepted review decision: the harness-adapters grok launch-profile row must state that firstmate now passes --reasoning-effort xhigh unconditionally and that a home still on grok 0.2.x therefore fails the launch instead of silently omitting the flag. The PR body must also say plainly that homes still on grok 0.2.x are affected. Do not add a grok-only version floor to README.md; README records no per-harness version floor for any harness, and that migration notice belongs in the PR body plus the harness-adapters row.

Acceptance criteria:

  • A crewmate spawn resolved to grok / grok-4.6 / xhigh actually passes --reasoning-effort xhigh to the launch command on grok 1.0.3.
  • config/crew-dispatch.json holding {"harness":"grok","model":"grok-4.6","effort":"xhigh"} validates cleanly in bootstrap (no CREW_DISPATCH: invalid). That file is gitignored captain-local state - do not commit one; verify with a temporary copy and restore whatever was there.
  • max remains omitted for grok.
  • Recorded harness facts match the verified behavior, with the grok version and date stated the way the surrounding rows do, including the unconditional-xhigh / 0.2.x-fails-launch floor clause.
  • bin/fm-lint.sh passes, and the repo's own tests covering these tables pass.

Do not change Herdr lifecycle behavior. Do not write a project's AGENTS.md unless this produced durable project-intrinsic knowledge (it should not).

What Changed

  • bin/fm-spawn.sh now emits --reasoning-effort xhigh for grok instead of dropping the flag, so a spawn resolved to grok / grok-4.6 / xhigh actually launches at xhigh; max is still omitted.
  • bin/fm-bootstrap.sh's crew-dispatch validator accepts xhigh in grok's effort list, so config/crew-dispatch.json holding {"harness":"grok","model":"grok-4.6","effort":"xhigh"} validates without CREW_DISPATCH: invalid; grok:max still fails validation.
  • .agents/skills/harness-adapters/SKILL.md records the accepted set as xhigh, high, medium, low, verified 2026-08-15 on grok 1.0.3 (1a29d5bc12) and reconfirmed on 1.0.4 (d846eb93d9), replacing the old 0.2.99 high ceiling; the grok tests in tests/fm-bootstrap.test.sh and tests/fm-spawn-dispatch-profile.test.sh flip from asserting xhigh is rejected/omitted to asserting it is accepted and threaded through.

Migration note: firstmate passes a requested xhigh through unconditionally rather than probing the installed CLI, so grok 1.0.x is now the supported floor. A home still on grok 0.2.x that requests xhigh will fail the launch on --reasoning-effort xhigh instead of silently omitting the flag — such homes should upgrade grok or pick a lower effort.

Risk Assessment

✅ Low: The change is a well-bounded static table update at the single shared effort-flag boundary plus its jq validator, with matching behavioral tests and documentation, no stale ceiling references left anywhere in the repo, and max still correctly rejected for grok.

Testing

Ran the two repo test files that encode grok's effort tables (tests/fm-spawn-dispatch-profile.test.sh and tests/fm-bootstrap.test.sh) — both green after working around two pre-existing host-PATH shadowing failures via the suite's supported FM_TEST_BASE_PATH knob, with no test edits. On top of that I produced product-level evidence for each acceptance criterion: a before/after capture of the literal launch command firstmate sends into the crewmate pane (base commit recorded effort=xhigh in meta but emitted no --reasoning-effort; target commit emits grok --always-approve --model 'grok-4.6' --reasoning-effort 'xhigh' ...), a before/after bootstrap run over a temporary captain-local config/crew-dispatch.json holding the grok/grok-4.6/xhigh profile (base printed CREW_DISPATCH: invalid - invalid effort: grok:xhigh, target validates cleanly and reports the active default), and confirmation that max is still omitted from the launch and still rejected by dispatch validation. I also exercised the installed grok 1.0.4 directly, which accepts the exact flag string firstmate now emits and reports its accepted set as xhigh/high/medium/low, matching the updated harness-adapters row. This is a shell/CLI change with no rendered surface, so the end-user evidence is CLI transcripts rather than screenshots. All temp homes, symlink farms and the gitignored config/ dir created during testing were removed; the worktree is clean. Per the phase rules I did not run bin/fm-lint.sh or the full suite.

Evidence: Launch command firstmate sends for grok/grok-4.6/xhigh (before vs after, plus max omission)

=== BEFORE (base commit f1a4af4) === meta: ... harness=grok ... model=grok-4.6 effort=xhigh launch: env -u CURSOR_AGENT -u CURSOR_INVOKED_AS grok --always-approve --model 'grok-4.6' "$('.../bin/fm-operational-input.sh' encode launch-brief < '.../brief.md')" === AFTER (target commit fc1e9b0) === meta: ... harness=grok ... model=grok-4.6 effort=xhigh launch: env -u CURSOR_AGENT -u CURSOR_INVOKED_AS grok --always-approve --model 'grok-4.6' --reasoning-effort 'xhigh' "$('.../bin/fm-operational-input.sh' encode launch-brief < '.../brief.md')" === max stays omitted from the grok launch command (target commit) === meta: ... model=grok-4.6 effort=max launch: env -u CURSOR_AGENT -u CURSOR_INVOKED_AS grok --always-approve --model 'grok-4.6' "$(... encode launch-brief < '.../brief.md')"

captain asks for a crewmate on grok / grok-4.6 / xhigh:
  fm-spawn.sh fm-grok-xhigh-evidence <project> --harness grok --model grok-4.6 --effort xhigh

=== BEFORE (base commit f1a4af4) ===
warning: /tmp/fm-grok-xhigh-evidence.CY2guk/before/home/data/fm-grok-xhigh-evidence/brief.md records no delivery contract line (scaffolded before ship briefs recorded one); launching on the explicit --mode no-mistakes - confirm its definition of done matches
meta:   window=firstmate:fm-fm-grok-xhigh-evidence endpoint_task_id=fm-grok-xhigh-evidence worktree=/tmp/fm-grok-xhigh-evidence.CY2guk/before/wt project=/tmp/fm-grok-xhigh-evidence.CY2guk/before/project harness=grok kind=ship mode=no-mistakes yolo=off tasktmp=/tmp/fm-fm-grok-xhigh-evidence model=grok-4.6 effort=xhigh spawn_gen=s1786811826.4192601.26274 
launch: env -u CURSOR_AGENT -u CURSOR_INVOKED_AS grok --always-approve --model 'grok-4.6' "$('/tmp/fm-base-root/bin/fm-operational-input.sh' encode launch-brief < '/tmp/fm-grok-xhigh-evidence.CY2guk/before/home/data/fm-grok-xhigh-evidence/brief.md')"

=== AFTER (target commit fc1e9b0) ===
warning: /tmp/fm-grok-xhigh-evidence.CY2guk/after/home/data/fm-grok-xhigh-evidence/brief.md records no delivery contract line (scaffolded before ship briefs recorded one); launching on the explicit --mode no-mistakes - confirm its definition of done matches
meta:   window=firstmate:fm-fm-grok-xhigh-evidence endpoint_task_id=fm-grok-xhigh-evidence worktree=/tmp/fm-grok-xhigh-evidence.CY2guk/after/wt project=/tmp/fm-grok-xhigh-evidence.CY2guk/after/project harness=grok kind=ship mode=no-mistakes yolo=off tasktmp=/tmp/fm-fm-grok-xhigh-evidence model=grok-4.6 effort=xhigh spawn_gen=s1786811828.4192864.19095 
launch: env -u CURSOR_AGENT -u CURSOR_INVOKED_AS grok --always-approve --model 'grok-4.6' --reasoning-effort 'xhigh' "$('/home/kali/.no-mistakes/worktrees/557e9e75f1aa/01M0340MTEDS0RA7W00GMGMH0N/bin/fm-operational-input.sh' encode launch-brief < '/tmp/fm-grok-xhigh-evidence.CY2guk/after/home/data/fm-grok-xhigh-evidence/brief.md')"

=== max stays omitted from the grok launch command (target commit) ===
=== AFTER (target commit fc1e9b0) ===
warning: /tmp/fm-grok-xhigh-evidence.3rtlH9/after/home/data/fm-grok-xhigh-evidence/brief.md records no delivery contract line (scaffolded before ship briefs recorded one); launching on the explicit --mode no-mistakes - confirm its definition of done matches
meta:   window=firstmate:fm-fm-grok-xhigh-evidence endpoint_task_id=fm-grok-xhigh-evidence worktree=/tmp/fm-grok-xhigh-evidence.3rtlH9/after/wt project=/tmp/fm-grok-xhigh-evidence.3rtlH9/after/project harness=grok kind=ship mode=no-mistakes yolo=off tasktmp=/tmp/fm-fm-grok-xhigh-evidence model=grok-4.6 effort=max spawn_gen=s1786811939.10250.829 
launch: env -u CURSOR_AGENT -u CURSOR_INVOKED_AS grok --always-approve --model 'grok-4.6' "$('/home/kali/.no-mistakes/worktrees/557e9e75f1aa/01M0340MTEDS0RA7W00GMGMH0N/bin/fm-operational-input.sh' encode launch-brief < '/tmp/fm-grok-xhigh-evidence.3rtlH9/after/home/data/fm-grok-xhigh-evidence/brief.md')"
Evidence: bootstrap validating a captain-local config/crew-dispatch.json (before vs after, plus grok:max)

# config/crew-dispatch.json (temporary copy): { "default": {"harness": "grok", "model": "grok-4.6", "effort": "xhigh"} } === BEFORE (base commit f1a4af4) === CREW_DISPATCH: invalid config/crew-dispatch.json - invalid effort: grok:xhigh === AFTER (target commit fc1e9b0) === BOOTSTRAP_INFO: crew dispatch active config/crew-dispatch.json BOOTSTRAP_INFO: crew dispatch default: grok/grok-4.6/xhigh (no CREW_DISPATCH: invalid line = validates cleanly) === max stays unsupported for grok (target commit) === # {"default": {"harness": "grok", "model": "grok-4.6", "effort": "max"}} CREW_DISPATCH: invalid config/crew-dispatch.json - invalid effort: grok:max

# captain-local config/crew-dispatch.json (gitignored state, temporary copy):
{
  "default": {"harness": "grok", "model": "grok-4.6", "effort": "xhigh"}
}

=== BEFORE (base commit f1a4af4) — bin/fm-bootstrap.sh ===
CREW_DISPATCH: invalid config/crew-dispatch.json - invalid effort: grok:xhigh

=== AFTER (target commit fc1e9b0) — bin/fm-bootstrap.sh ===
BOOTSTRAP_INFO: crew dispatch active config/crew-dispatch.json
BOOTSTRAP_INFO: crew dispatch default: grok/grok-4.6/xhigh
(no CREW_DISPATCH: invalid line above = validates cleanly)

=== max stays unsupported for grok (target commit) ===
# config/crew-dispatch.json: {"default": {"harness": "grok", "model": "grok-4.6", "effort": "max"}}
CREW_DISPATCH: invalid config/crew-dispatch.json - invalid effort: grok:max
Evidence: Installed grok 1.0.4 accepted reasoning-effort set

$ grok --version grok 1.0.4 (d846eb93d9) [stable] $ grok --model grok-4.6 --reasoning-effort bogus --print hi Error: --effort/--reasoning-effort: unknown effort level 'bogus'; use one of: xhigh, high, medium, low $ grok --model grok-4.6 --reasoning-effort max --print hi Error: --effort/--reasoning-effort: unknown effort level 'max'; use one of: xhigh, high, medium, low # the flags firstmate now emits, handed to grok verbatim: $ grok --always-approve --model grok-4.6 --reasoning-effort xhigh --print "reply with the single word: ok" ...ok

$ grok --version
grok 1.0.4 (d846eb93d9) [stable]

$ grok --model grok-4.6 --reasoning-effort bogus --print hi
--effort/--reasoning-effort: unknown effort level 'bogus'; use one of: xhigh, high, medium, low
Error: --effort/--reasoning-effort: unknown effort level 'bogus'; use one of: xhigh, high, medium, low
[exit=0]

$ grok --model grok-4.6 --reasoning-effort max --print hi
--effort/--reasoning-effort: unknown effort level 'max'; use one of: xhigh, high, medium, low
Error: --effort/--reasoning-effort: unknown effort level 'max'; use one of: xhigh, high, medium, low
[exit=0]
$ grok --model grok-4.6 --reasoning-effort xhigh --print "reply with the single word: ok"
I'll start the session, then reply as asked.ok
[pipestatus=]
# The flags firstmate now emits, handed to the installed grok 1.0.4 verbatim:
$ grok --always-approve --model grok-4.6 --reasoning-effort xhigh --print "reply with the single word: ok"
Running the required session start, then I'll answer exactly as you asked.Handling the queued network-check notification and the missing-tool diagnostic, then I'll answer in the one word you asked for.ok
[exit=]
Evidence: Evidence harness used to capture the real launch command (fake tmux, real fm-spawn.sh)
#!/usr/bin/env bash
# Evidence harness: what literal launch command does firstmate send into the
# crewmate pane for a spawn resolved to grok / grok-4.6 / xhigh?
#
# Drives the real bin/fm-spawn.sh with a fake tmux that records the `send-keys -l`
# payload, exactly as tests/fm-spawn-dispatch-profile.test.sh does. Runs twice:
# once against the base-commit fm-spawn.sh (symlink farm) and once against the
# fixed one in the worktree.
set -u

WT_ROOT=$1        # firstmate worktree at the target commit
BASE_ROOT=$2      # symlink farm whose bin/fm-spawn.sh is the base-commit copy

. "$WT_ROOT/tests/lib.sh"

TMP_ROOT=$(fm_test_tmproot fm-grok-xhigh-evidence)

make_case() {
  local name=$1 case_dir home proj wt fakebin
  case_dir="$TMP_ROOT/$name"
  home="$case_dir/home"; proj="$case_dir/project"; wt="$case_dir/wt"
  fakebin=$(fm_fakebin "$case_dir/fake")
  cat > "$fakebin/tmux" <<'SH'
#!/usr/bin/env bash
set -u
case "$*" in
  *"#{pane_current_path}"*) printf '%s\n' "${FM_FAKE_PANE_PATH:-}"; exit 0 ;;
esac
case "${1:-}" in
  display-message) printf 'firstmate\n'; exit 0 ;;
  list-windows) exit 0 ;;
  has-session|new-session|new-window|kill-window) exit 0 ;;
  send-keys)
    if [ -n "${FM_FAKE_LAUNCH_LOG:-}" ]; then
      prev=
      for a in "$@"; do
        [ "$prev" = "-l" ] && printf '%s\n' "$a" >> "$FM_FAKE_LAUNCH_LOG"
        prev=$a
      done
    fi
    exit 0 ;;
esac
exit 0
SH
  chmod +x "$fakebin/tmux"
  fm_fake_exit0 "$fakebin" treehouse
  cat > "$fakebin/timeout" <<'SH'
#!/usr/bin/env bash
shift
exec "$@"
SH
  chmod +x "$fakebin/timeout"
  mkdir -p "$home/data" "$home/projects" "$home/state" "$home/config"
  printf '%s\n' grok > "$home/config/crew-harness"
  fm_git_worktree "$proj" "$wt" "wt-$name"
  touch "$home/state/.last-watcher-beat"
  mkdir -p "$home/data/$CREW_ID"
  printf 'Investigate the current grok launch flags.\n' > "$home/data/$CREW_ID/brief.md"
  printf '%s\n' "$case_dir|$home|$proj|$wt|$fakebin|$case_dir/launch.log"
}

spawn_with() {
  local root=$1 label=$2 rec case_dir home proj wt fakebin launchlog
  rec=$(make_case "$label")
  IFS='|' read -r case_dir home proj wt fakebin launchlog <<EOF
$rec
EOF
  : > "$launchlog"
  FM_ROOT_OVERRIDE='' FM_HOME="$home" \
    FM_STATE_OVERRIDE="$home/state" FM_DATA_OVERRIDE="$home/data" \
    FM_PROJECTS_OVERRIDE="$home/projects" FM_CONFIG_OVERRIDE="$home/config" \
    FM_SPAWN_NO_GUARD=1 FM_FAKE_PANE_PATH="$wt" TMUX="fake,1,0" \
    FM_FAKE_LAUNCH_LOG="$launchlog" CLAUDE_CONFIG_DIR='' \
    PATH="$fakebin:$PATH" \
    "$root/bin/fm-spawn.sh" --mode no-mistakes --yolo off \
      "$CREW_ID" "$proj" --harness grok --model grok-4.6 --effort xhigh >/dev/null
  printf 'meta:   %s\n' "$(tr '\n' ' ' < "$home/state/$CREW_ID.meta" | sed 's/  */ /g')"
  printf 'launch: %s\n' "$(cat "$launchlog")"
}

CREW_ID=fm-grok-xhigh-evidence

echo "captain asks for a crewmate on grok / grok-4.6 / xhigh:"
echo "  fm-spawn.sh $CREW_ID <project> --harness grok --model grok-4.6 --effort xhigh"
echo
echo "=== BEFORE (base commit f1a4af4) ==="
spawn_with "$BASE_ROOT" before
echo
echo "=== AFTER (target commit fc1e9b0) ==="
spawn_with "$WT_ROOT" after
- Outcome: ⚠️ 1 info across 1 run (9m36s)

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

⚠️ **Review** - 1 info
  • ℹ️ bin/fm-spawn.sh:1394 - Passing --reasoning-effort xhigh unconditionally makes grok 1.0.x a hard launch floor: a home still on grok 0.2.x now fails the spawn outright (0.2.x rejects xhigh with use one of: high, medium, low) instead of launching with the flag omitted. This is the explicitly authorized design decision ("statically raise the ceiling to xhigh and treat grok 1.0.x as the supported floor. Do not invent CLI-probe machinery"), and the required migration clause is recorded at .agents/skills/harness-adapters/SKILL.md:130. Noted only so the behavior change is visible; no action needed in source.
⚠️ **Test** - 1 info
  • ℹ️ tests/fm-bootstrap.test.sh:513 - Pre-existing, host-dependent fragility in tests/fm-bootstrap.test.sh (unrelated to this change): test_orca_backend_gates_orca_tool_only_when_selected and test_network_phase_partitions_the_run assume orca and node are absent from the system PATH. On this host /usr/bin/orca (the GNOME screen reader) and /usr/bin/node exist, so both tests fail with the default BASE_PATH. Worked around for this run via the test's own FM_TEST_BASE_PATH knob pointed at a sanitized /usr/bin copy; no test files were modified.
  • bash tests/fm-spawn-dispatch-profile.test.sh (all 31 cases pass, incl. test_grok_threads_xhigh_reasoning_effort and test_grok_omits_invalid_max_reasoning_effort)
  • FM_TEST_BASE_PATH=&lt;sanitized /usr/bin without orca,node&gt; bash tests/fm-bootstrap.test.sh (all 28 cases pass, incl. test_crew_dispatch_validation covering grok:xhigh accepted / grok:max flagged)
  • Manual before/after launch capture: drove real bin/fm-spawn.sh --harness grok --model grok-4.6 --effort xhigh with a fake tmux recording the literal send-keys -l payload, against both the base-commit script and the fixed one
  • Manual: same spawn harness with --effort max, confirming no --reasoning-effort reaches the launch command
  • Manual: FM_HOME=&lt;temp home&gt; FM_BOOTSTRAP_VERBOSE_FACTS=1 bin/fm-bootstrap.sh with a temporary config/crew-dispatch.json = {&#34;default&#34;:{&#34;harness&#34;:&#34;grok&#34;,&#34;model&#34;:&#34;grok-4.6&#34;,&#34;effort&#34;:&#34;xhigh&#34;}}, run against base and target bootstrap; temp home deleted afterwards, worktree left clean
  • Manual: same bootstrap run with &#34;effort&#34;:&#34;max&#34; to confirm CREW_DISPATCH: invalid ... invalid effort: grok:max still fires
  • grok --version -> grok 1.0.4 (d846eb93d9)
  • grok --model grok-4.6 --reasoning-effort bogus --print hi and ... --reasoning-effort max --print hi -> use one of: xhigh, high, medium, low
  • grok --always-approve --model grok-4.6 --reasoning-effort xhigh --print &#34;reply with the single word: ok&#34; -> completes successfully
  • git status --porcelain after cleanup (empty; the test-created gitignored config/ dir and all temp dirs removed)
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

rqms40 added 3 commits August 15, 2026 23:53
Grok 1.0.3+ accepts xhigh, high, medium, and low. The old 0.2.99
ceiling silently dropped xhigh from the launch command and rejected
it in crew-dispatch.json.

Raise the static tables to xhigh and keep max omitted. grok 1.0.x is
the supported floor; homes still on 0.2.x will fail a launch that
requests xhigh rather than degrade.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant