Skip to content

docs(browser): teach the eval-js output contract and piping - #40

Merged
giordano-lucas merged 1 commit into
mainfrom
t3code/eval-js-output-contract
Aug 25, 2026
Merged

docs(browser): teach the eval-js output contract and piping#40
giordano-lucas merged 1 commit into
mainfrom
t3code/eval-js-output-contract

Conversation

@giordano-lucas

Copy link
Copy Markdown
Member

Why

nottelabs/notte-cli#72 (merged) changed what notte page eval-js prints: the evaluated value now lands alone on stdout (objects/arrays as JSON, a JS null as null) with the status line on stderr, and a failing script exits non-zero with the actual JavaScript error instead of the generic user-facing sentence.

That makes eval-js composable, which matters more for a skill than for docs — the agent can now chain it instead of reading a banner and a Result: prefix.

What

In notte-browser/SKILL.md, where eval-js is taught:

  • State the output contract (value alone on stdout, status on stderr, -o json for the full envelope).
  • Show both shell shapes:
    title=$(notte page eval-js --session-id <session-id> 'document.title')
    
    notte page eval-js --session-id <session-id> \\
      'JSON.stringify([...document.querySelectorAll("a")].map(a => a.href))' | jq length
  • Recommend returning JSON.stringify(...) for structured answers so jq filters locally instead of costing another round trip through the page.
  • Note that failures exit non-zero with the real JS error, so set -e and || fallbacks behave.
  • Correct a stale line: "Don't use logging - stdout is not captured" → what's discarded is console.log; the returned value is exactly what stdout carries now.

In notte-functions-build/references/exploration.md, add the same piping note to the endpoint-verification flow, which drives eval-js.

Verification

python3 scripts/validate-plugins.py → 37 checks, no problems. Both taught commands were run against staging with a CLI built from notte-cli main: title=$(...) captured Example Domain, and the JSON.stringify(...) | jq length pipe returned a count.

Gate

None for the CLI teaching — notte-cli#72 is merged; it applies once a CLI release ships. (The session.evaluate_js mapping added in #39 still waits on the SDK release, unchanged here.)

🤖 Generated with Claude Code

notte-cli#72 prints the evaluated value alone on stdout with the status
line on stderr, so eval-js is now composable: the agent can capture it
with $(...) or pipe it into jq instead of reading a banner and a
"Result:" prefix. Say that where eval-js is taught, show both shapes,
and recommend returning JSON.stringify(...) for structured answers so
jq filters locally rather than costing another round trip through the
page. Also correct the old "stdout is not captured" line - what is
discarded is console.log, not the returned value - and note that a
failing script now exits non-zero with the actual JavaScript error.

The same piping note goes on the functions-build exploration flow,
which drives eval-js to verify discovered endpoints.

Verified against staging: title=$(notte page eval-js 'document.title')
captures "Example Domain" and the JSON.stringify example pipes into jq.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Aug 25, 2026

Copy link
Copy Markdown

Greptile Summary

This PR documents the revised notte page eval-js output contract and teaches direct shell capture and jq piping.

  • Clarifies stdout, stderr, JSON-envelope, and JavaScript failure behavior.
  • Adds command-substitution and structured-output pipeline examples.
  • Extends the same piping guidance to Function endpoint exploration.

Confidence Score: 4/5

The documentation-only PR appears safe to merge, with the non-blocking concern that its exact CLI output contract has no automated regression coverage.

The changed guidance is internally consistent and the only accepted concern is missing test coverage for stream-sensitive shell examples.

Files Needing Attention: plugins/notte/skills/notte-browser/SKILL.md; plugins/notte/skills/notte-functions-build/references/exploration.md

Important Files Changed

Filename Overview
plugins/notte/skills/notte-browser/SKILL.md Documents the composable eval-js output contract and shell examples, but adds no automated regression coverage.
plugins/notte/skills/notte-functions-build/references/exploration.md Adds direct jq-piping and command-substitution guidance without automated coverage.

Fix all with Greploop Fix All in Claude Code

Prompt To Fix All With AI
### Issue 1
plugins/notte/skills/notte-browser/SKILL.md:290-296
**Output contract lacks regression coverage**

These stream-sensitive examples depend on exact stdout, stderr, JSON, and exit-status behavior, but this PR adds no unit or integration test to keep the documentation aligned with the CLI contract as it changes.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Reviews (1): Last reviewed commit: "docs(browser): teach the eval-js output ..." | Re-trigger Greptile

@giordano-lucas
giordano-lucas merged commit fcdf493 into main Aug 25, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant