Skip to content

feat(llm-council): add --max-tokens CLI flag - #89

Open
23r2efewvcs wants to merge 2 commits into
rohitg00:mainfrom
23r2efewvcs:feat/llm-council-max-tokens
Open

feat(llm-council): add --max-tokens CLI flag#89
23r2efewvcs wants to merge 2 commits into
rohitg00:mainfrom
23r2efewvcs:feat/llm-council-max-tokens

Conversation

@23r2efewvcs

@23r2efewvcs 23r2efewvcs commented Jul 28, 2026

Copy link
Copy Markdown

Fixes #88.

What

skills/llm-council/scripts/council.js hardcoded max_tokens: 4000 in both callOpenAICompat (line 87) and callAnthropic (line 103). This is too small for modern reasoning models (Nemotron-3-Ultra, GLM-5.2, DeepSeek-V4, Claude with thinking enabled), which routinely need 8K–32K output tokens for their reasoning chain. With the current default they either:

  • return an empty choices[0].message.content (the model exhausted its budget before emitting any final answer), or
  • silently truncate the chain, producing a degraded synthesis.

There's no way to override this from the CLI — users have to monkey-patch the constant.

Change

  • Introduce a tiny RUN_OPTS struct (const RUN_OPTS = { max_tokens: 4000 };) at the top of the file.
  • Both call functions read RUN_OPTS.max_tokens instead of the literal.
  • Add --max-tokens N parsing in cmdRun (default preserves current behavior).
  • Document the flag in usage().

Reproduction (before this PR)

node council.js run "Top 3 criteria Georgian diaspora in EU uses to pick a bank." \
  --provider openrouter --models "nvidia/nemotron-3-ultra-550b-a55b:free" --chairman "..." \
  --timeout 300000
# phase1_responses.json: { "success": true, "content": "", "latency_ms": 18916 }

After this PR

node council.js run "<query>" --models "..." --chairman "..." --max-tokens 16000

Scope

This PR is intentionally minimal — only adds the knob, doesn't touch retry/timeout/sequential logic (separate concerns for separate PRs). Diff: 13 +, 4 -.

Backward compatibility

Default 4000 preserved verbatim. Existing users see no behavior change unless they pass --max-tokens.

Summary by CodeRabbit

  • New Features

    • Added a --max-tokens option to customize the maximum response length when running the council script.
    • The default remains 4,000 tokens.
    • The setting is applied consistently across supported AI providers.
  • Documentation

    • Updated command-line usage information with the new option, default value, and guidance.

Hardcoded max_tokens: 4000 is too small for reasoning models (Nemotron
Ultra, GLM-5.2, DeepSeek-V4, Claude thinking) which need 8K-32K output
tokens for their reasoning chain or they emit empty content.

Introduce RUN_OPTS struct + --max-tokens flag (default 4000, preserves
current behavior). Both callOpenAICompat and callAnthropic read from
RUN_OPTS.max_tokens.

Fixes rohitg00#88.
@coderabbitai

coderabbitai Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Warning

Review limit reached

@23r2efewvcs, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 18 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: f4dc6aa1-1df0-42c3-a249-cbd3cba3c04f

📥 Commits

Reviewing files that changed from the base of the PR and between 22251ab and 2635c07.

📒 Files selected for processing (1)
  • skills/llm-council/scripts/council.js
📝 Walkthrough

Walkthrough

The council script adds a --max-tokens CLI option, defaults it to 4000, applies the value to OpenAI-compatible and Anthropic requests, and documents the option in the usage output.

Changes

Maximum token configuration

Layer / File(s) Summary
Configure and apply runtime token limit
skills/llm-council/scripts/council.js
Adds the default RUN_OPTS.max_tokens value, parses the --max-tokens override, uses it in both provider request payloads, and documents the option in usage().

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: adding a configurable --max-tokens CLI flag.
Linked Issues check ✅ Passed The changes match issue #88 by making max_tokens configurable via --max-tokens, defaulting to 4000, and updating both API paths and usage text.
Out of Scope Changes check ✅ Passed The diff appears scoped to the requested max_tokens configuration and documentation updates, with no unrelated changes evident.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@skills/llm-council/scripts/council.js`:
- Line 171: Validate args['max-tokens'] before assigning RUN_OPTS.max_tokens:
require a fully numeric, integer value greater than zero, rejecting malformed
inputs such as prefixes and NaN. On invalid input, emit a clear CLI error and
stop without mutating the runtime configuration; otherwise assign the parsed
value.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: f02f9fe6-a774-4b65-95b0-a8ef044affb4

📥 Commits

Reviewing files that changed from the base of the PR and between 7f7209d and 22251ab.

📒 Files selected for processing (1)
  • skills/llm-council/scripts/council.js

Comment thread skills/llm-council/scripts/council.js Outdated
23r2efewvcs added a commit to 23r2efewvcs/pro-workflow that referenced this pull request Jul 31, 2026
- Fix runCalls: invoke callables before Promise.allSettled (was passing
  function objects instead of promises, breaking Phase 1/2)
- Add parseIntSafe helper: validate --max-tokens, --timeout, --max-retries
  are positive integers, reject NaN/negative/non-integer
- Validate --reasoning-effort: only low|medium|high accepted
- Fix Anthropic thinking: use adaptive type instead of fixed budget_tokens
- Add 5xx-specific error hint (retry later, check provider status)
- Update SKILL.md with all new CLI flags and runtime options table

Addresses CodeRabbit comments on PRs rohitg00#89-rohitg00#101.
Add parseIntSafe: require positive integer, reject NaN/negative.
Addresses CodeRabbit comment on PR rohitg00#89.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

llm-council: max_tokens hardcoded to 4000 — can't be raised for reasoning models

1 participant