feat: configure model output and Bedrock stream limits - #459
Open
Geonwoo Jeong (geonwoo-jeong) wants to merge 2 commits into
Open
feat: configure model output and Bedrock stream limits#459Geonwoo Jeong (geonwoo-jeong) wants to merge 2 commits into
Geonwoo Jeong (geonwoo-jeong) wants to merge 2 commits into
Conversation
Geonwoo Jeong (geonwoo-jeong)
marked this pull request as ready for review
July 24, 2026 12:47
This was referenced Jul 25, 2026
Geonwoo Jeong (geonwoo-jeong)
force-pushed
the
codex/configurable-max-output-tokens
branch
from
July 27, 2026 05:41
16766a7 to
119ff55
Compare
Expose OPENWIKI_STREAM_IDLE_TIMEOUT in milliseconds, accept 0 to disable the LangChain watchdog, and pass the option only to ChatBedrockConverse. Validate the value against Node's timer range, surface it in diagnostics and debug logs, and preserve the output-limit, truncation guard, and patched @langchain/aws behavior.
Geonwoo Jeong (geonwoo-jeong)
force-pushed
the
codex/configurable-max-output-tokens
branch
from
July 28, 2026 06:19
119ff55 to
41a30d6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #474
Closes #485
Context: #458 and #457 cover truncated tool-call detection. This PR stays separate from that failure guard and only exposes model-client runtime limits.
Summary
OPENWIKI_MAX_OUTPUT_TOKENSsetting across supported model providers.OPENWIKI_STREAM_IDLE_TIMEOUTsetting for the first/inter-chunk watchdog.Why these controls belong together
OpenWiki currently exposes neither the model output budget nor Bedrock's stream idle watchdog. The two limits are independent:
maxTokenscontrols the allowed response length, whilestreamIdleTimeoutcontrols how long the Bedrock client waits between chunks.We observed the 60-second LangChain watchdog after enabling a larger output budget, but this PR does not claim that a larger budget causes first-chunk latency. It exposes both controls so operators can configure the output budget and the watchdog appropriate to their workload without replacing either provider default globally.
Exact behavior
OPENWIKI_MAX_OUTPUT_TOKENSmaxOutputTokensfor Google clients.maxTokensfor Anthropic, OpenAI, OpenAI-compatible, OpenRouter, Bedrock, Gemini Enterprise non-Google surfaces, ChatGPT OAuth, and GitHub Copilot.OPENWIKI_STREAM_IDLE_TIMEOUT0to2147483647milliseconds.ChatBedrockConverse.streamIdleTimeout.0uses LangChain's documented watchdog-disable behavior and produces a diagnostic warning that stalled streams may hang indefinitely.@langchain/awsdefault.Scope
This PR changes model construction, environment management/diagnostics, documentation, and focused tests for these two runtime limits. It does not patch dependencies, alter prompts or workflows, or add truncation detection.
Validation
pnpm install --frozen-lockfilepnpm test— 64 files, 792 testspnpm run buildpnpm run typecheckpnpm run lint:checkpnpm run format:checkpnpm audit --prod— no known vulnerabilities