Repository navigation
Conversation
The tapping rule was one instance of a class: guidance that mandates a call whose inputs already live server-side. #958 retired the pre-tap `describe`; this does the same for three more. - tv-remote takes the auto-screenshot. It is the only navigation primitive on a TV, and was in neither auto-capture set, so every D-pad press cost a separate look at the screen. It stays out of the auto-describe set: on a booted Apple TV 4K (tvOS 18.5) a home-screen `describe` returned `tv-control request timed out` after 10.1s, 11.2s and 39.8s where `screenshot` answered in 0.6s, and a UUID-shaped id cannot be told apart from an iPhone's at the MCP layer. `shake` joins both sets. - launch-app / restart-app assert the Android Metro reverse before starting the process. Six skill files told the agent to run `adb reverse` by hand and nothing in the tool-server ever did; the mapping dies on a device reboot or an adb restart, and `argent-metro-debugger` documented that as the most common cause of a failed Metro connection. `ARGENT_METRO_PORT` overrides the port, `disable-metro-reverse` opts out. - react-profiler-analyze accepts a gesture's own `timestampMs`. The skill asked the agent to save `startedAtEpochMs`, subtract it after every tap and swipe, and carry the offsets across the session; the anchor is already stored with the profile. `offsetMs` still works. An annotation that gives a timestamp with no stored anchor fails loudly rather than landing every action on the first commit. Also corrects a stale claim: the skills told the agent to ToolSearch every gesture tool before first use, but gesture-tap/swipe/scroll/drag are alwaysLoad, so the ceremony was already a no-op for four of the seven.
This comment has been minimized.
This comment has been minimized.
This was referenced Aug 31, 2026
Member
Author
|
Still needed as of 2026-09-28 (main |
Member
Author
|
Closing: the tv-remote change goes the opposite way from open #1262, which gives tv-remote the element tree as well as the screenshot and conflicts with this PR in argent-tv-interact/SKILL.md. The |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A fellow pointed out that we mandate a
describeafter every gesture, when the tool-server could just return it. That was one instance of a class, and #958 fixed exactly that instance. I audited every skill, rule file and tool description for the same shape - guidance that mandates a call whose inputs already live server-side - and found nine more. Three are fixed here; the rest are listed at the bottom with why they are not.What
tv-remotetakes the auto-screenshotA TV is focus-driven and
tv-remoteis its only navigation primitive, but it was in neither auto-capture set.argent-tv-interacttherefore prescribeddescribe->tv-remote-> "describeagain to confirm": two mandated calls per D-pad move, on the one platform that never got #958's benefit.shakewas missing from both sets too, and joins them.tv-remotedeliberately does not get the element tree. I booted an Apple TV 4K (tvOS 18.5) to check, because that was the load-bearing unknown:describe×3tv-control request timed outafter 10.1s, 11.2s, 39.8sscreenshotawait-screen-idle(1500ms cap)settled:falsetv-remoteclassifyDevicekeys off id shape, so an Apple TV UDID and an iPhone UDID are indistinguishable at the MCP layer - the tree cannot be appended for TVs without risking that wait on every press. The screenshot carries focus visually instead, and the skill now records the measurement.launch-app/restart-appassert the Android Metro reverseadb reverse tcp:8081 tcp:8081was mandated in six skill files and automated in zero (grep -rn reverse packages/tool-server/srcreturned nothing). The mapping dies on a device reboot or an adb restart, andargent-metro-debuggerdocumented it as the most common cause of a failed Metro connection - a footgun the code could remove. Both tools now assert it before starting the process (adb reverseis idempotent). Best-effort: a device that refuses still launches, since most launches are of apps that never talk to Metro.ARGENT_METRO_PORToverrides the port;disable-metro-reverseopts out. The skills keep the manual command for the one path that still needs it -npx react-native run-androidand Android Studio start the app outside argent.react-profiler-analyzeaccepts a gesture's owntimestampMsThe skill asked the agent to save
startedAtEpochMs, computeoffsetMs = timestampMs - startedAtEpochMsafter every tap and swipe, and carry the list across the session. Both operands are already server-side:react-profiler-startstoresprofileStartWallMswith the profile. Annotations now take{ timestampMs, label }and the tool does the subtraction.{ offsetMs, label }still works; exactly one of the two per annotation. A timestamp with no stored anchor fails withREACT_PROFILER_ANALYZE_ANNOTATION_UNANCHOREDrather than silently landing every action on the first commit.One stale claim corrected
The skills said to ToolSearch every gesture tool before first use.
gesture-tap,gesture-swipe,gesture-scrollandgesture-dragarealwaysLoad: true, so the mandated ceremony was already a no-op for four of the seven. Now names only the three that are actually deferred.Verification
tv-remotefrom the describe set, adding it back, adding a tool with no settle budget, removing theoffsetMssubtraction, ignoringdisable-metro-reverse, trustingARGENT_METRO_PORTblindly, letting an adb failure escape, and moving the reverse to afteram starteach fail the suite. The tvOS numbers above are from a real booted simulator, not inference.adb reversetest redirects both flag scopes ($HOMEand cwd) into a temp dir - it passes with a hostiledisable-metro-reverse: trueplanted at the repo root, which is the exact condition that broke it while I was writing it.npm run build,lint --max-warnings 0,knip(ceiling 0),format,typecheck:tests -w @argent/tool-server,npx docusaurus build.Docs
reference/configuration.mdxgains thedisable-metro-reverserow;features/interacting-with-apps.mdxrecords that atv-remotepress returns the screenshot only, and why.Audited but deliberately not changed
list-devicesbefore everything (rules, both setup skills,argent-test-ui-flow,argent-lens) -udidis required on 25 tool schemas and optional on none. Worth noting for whoever picks this up:resolveFlowDeviceinflows/flow-device.tsalready implements the exact fallback (explicit id -> the single booted device ofplatform-> the single booted device overall -> throw, enumerating what is available), andflow-runexposes it as auto-detection today. So this is extending an existing mechanism to the interaction tools, not inventing one - but it changes the contract on 25 tools, so it wants its own PR.flow-add-stepcannot take a selector - the recorder reverse-derives one from the pre-tap tree inid->text->roleorder, and two of the three outcomes are wrong (one silently). The agent had the selector before it computed the frame centre. Touches recording semantics, so out of scope here.screenshot-diffbaseline bookkeeping, Lens screenshot hand-off (propose_variantalready describes the device at propose time),stop-all-simulator-serversdefault scope - all real, all better as separate changes.I have not filed these as issues yet; say the word and I will, or I can take any of them next.