Summary
Allow Bedrock users to configure the first/inter-chunk idle timeout used by ChatBedrockConverse.
Motivation
@langchain/aws@1.4.2 added a 60,000 ms watchdog around Bedrock Converse streams. It applies while waiting for both the first chunk and each subsequent chunk, and currently raises:
Bedrock Converse stream timed out after 60000 ms without receiving a chunk.
OpenWiki does not pass streamIdleTimeout when it creates ChatBedrockConverse, so Bedrock users cannot tune this provider setting. We observed the timeout on a repository generation after increasing the output-token budget. A larger token budget is not established as the cause of first-chunk latency; the two controls are independent, but users who configure longer model outputs also need a way to set an appropriate stream watchdog.
Proposed Solution
Add an optional OPENWIKI_STREAM_IDLE_TIMEOUT setting that:
- accepts an integer number of milliseconds in Node's timer range;
- passes the value only to
ChatBedrockConverse as streamIdleTimeout;
- accepts
0, matching LangChain's documented way to disable the watchdog;
- preserves the
@langchain/aws default when unset;
- appears in environment diagnostics and debug output as a non-secret value.
Documentation should recommend a sufficiently long finite timeout over 0 so a genuinely stalled stream cannot hang forever.
Alternatives Considered
- Hard-code a longer OpenWiki timeout: this would replace one universal default with another and remove the upstream default's ability to evolve.
- Disable the watchdog unconditionally: this would restore the indefinite-hang behavior the LangChain guard was introduced to prevent.
- Treat this as part of truncation detection: the timeout can occur before OpenWiki receives the protocol events used by truncation detection, so it is a separate configuration concern.
Additional Context
Summary
Allow Bedrock users to configure the first/inter-chunk idle timeout used by
ChatBedrockConverse.Motivation
@langchain/aws@1.4.2added a 60,000 ms watchdog around Bedrock Converse streams. It applies while waiting for both the first chunk and each subsequent chunk, and currently raises:OpenWiki does not pass
streamIdleTimeoutwhen it createsChatBedrockConverse, so Bedrock users cannot tune this provider setting. We observed the timeout on a repository generation after increasing the output-token budget. A larger token budget is not established as the cause of first-chunk latency; the two controls are independent, but users who configure longer model outputs also need a way to set an appropriate stream watchdog.Proposed Solution
Add an optional
OPENWIKI_STREAM_IDLE_TIMEOUTsetting that:ChatBedrockConverseasstreamIdleTimeout;0, matching LangChain's documented way to disable the watchdog;@langchain/awsdefault when unset;Documentation should recommend a sufficiently long finite timeout over
0so a genuinely stalled stream cannot hang forever.Alternatives Considered
Additional Context