pollWebAi can hang past --timeout when assistant DOM evaluation stalls

[dbc-hbin]

Summary

With agbrowse@0.1.18, ChatGPT polling can remain alive without progress after a long conversation makes assistant-message DOM extraction slow or stuck. The outer --timeout does not bound the individual page.evaluate() used to read assistant messages, so the poll loop cannot re-check its deadline while that await is pending.

Environment

• agbrowse: 0.1.18
• macOS
• Node.js: 24.16.0
• Headed Chrome over local CDP
• ChatGPT Thinking conversation continued with query --session

Observed scenario

A single saved ChatGPT conversation accumulated three complete assistant planning responses of approximately:

• 56,000 characters
• 60,000 characters
• 49,000 characters

The next continuation requested another complete response.

Observed behavior:

1. The initial continuation ran with --timeout 900 and returned provider.poll-timeout with zero recovered response characters.

2. Explicit follow-up polls of 600, 240, and 240 seconds also recovered zero characters.

3. web-ai sessions doctor reported:

   • valid target
   • no tab recovery needed
   • healthy command-lock heartbeat
   • no CDP or login failure

4. A later query --session ... --timeout 800 remained alive with zero stdout. Its stderr emitted:

   [poll] 1s — streaming...
   [poll] 32s — streaming...
   [poll] 62s — streaming...
   

   After that, heartbeat output stopped while the Node process and session lock remained alive for several more minutes.

The issue is therefore distinct from a normal provider timeout, expired session, target drift, or failed login.

Relevant code path

web-ai/chatgpt.mjs checks the command deadline only at the while loop boundary, then performs:

const answers = await readAssistantMessages(page);

readAssistantMessages() performs:

await page.evaluate(readTopLevelAssistantTexts, ASSISTANT_SELECTORS)

That await has no call-level timeout or remaining-deadline guard. If the ChatGPT page is very large and evaluation or serialization does not return, control never reaches the next outer deadline check. The same function also reads and cleans all top-level assistant messages on every polling tick even though only messages after baseline.assistantCount are needed.

The stopped stderr heartbeat at the same time the process and lock remain healthy is consistent with the poller being suspended inside one of these unbounded DOM reads. This is an inference from the runtime evidence and current source structure.

Minimal reproduction shape

1. Start a ChatGPT thinking session.

2. Generate several large assistant answers in the same saved conversation, roughly 40–60 KB each.

3. Continue it with another large response:

   agbrowse web-ai query \
     --session <latest-session-id> \
     --inline-only \
     --prompt "Return another complete, detailed document" \
     --timeout 900 \
     --json

4. Observe whether poll heartbeat output stops while the process remains alive past expected progress and produces no JSON until externally terminated or much later.

A deterministic unit test can reproduce the deadline defect without ChatGPT by making page.evaluate() never resolve inside readAssistantMessages().

Expected

• Every assistant-DOM read should be bounded by the remaining polling deadline or a shorter per-read timeout.
• A stalled read should return a recoverable provider.poll-timeout or continue through a bounded fallback; it should not suspend the command indefinitely.
• Long conversations should avoid serializing all historical assistant text every 500 ms when only the latest post-baseline answer is relevant.
• Session lock heartbeat and diagnostics should identify a DOM-read timeout separately from provider generation still streaming.

Suggested direction

• Pass the remaining deadline or an AbortSignal into readAssistantMessages().
• Wrap page.evaluate() and locator fallback reads in a bounded timeout.
• Prefer selecting/counting assistant turns first and extract only the latest turn after baseline.assistantCount.
• Add regression coverage for a never-resolving evaluate call and a large multi-turn conversation fixture.

No private prompt or response content is needed to reproduce the issue.