Repository navigation
feat(harmony): drive HarmonyOS devices over hdc - #767
Conversation
Adds a `harmony` platform driven by DevEco Studio's `Emulator` manager: `list-devices` reports emulator instances and `boot-device` starts one by name. Device ids are `harmony-<instanceName>`, minted by argent the way `chromium-cdp-<port>` is, so classification never depends on a vendor serial shape - the connect key of a running HarmonyOS emulator could not be observed on a host that cannot obtain an image. The manager's exit code is not a success signal: `-create`, `-stop` and `-install` report failures with exit 0, while `-start` exits 1. So `emulatorFailure` classifies from stdout against diagnostics measured against DevEco Studio 6.1, and `runHarmonyEmulator` deliberately does not reject on a non-zero exit, keeping one verdict path instead of two that disagree. Dispatch rejects the platform explicitly rather than falling through: chromium is the last branch of `dispatchByPlatform`, so a HarmonyOS device would otherwise be handed to the CDP handler and its result reported as a success. Huawei serves HarmonyOS emulator images only within mainland China. Outside it no image downloads, so no instance can be created; boot-device reports what the manager said instead of claiming a boot it did not perform, and distinguishes that restriction from an empty instance list, which is also what a host that has simply not created one yet looks like. Interaction tools are out of scope - screenshot, describe, input and app-lifecycle return 501 on a HarmonyOS device - because with no bootable image their behaviour could not be verified.
`__resetHarmonyBinaryCacheForTests` had no caller, and `HarmonyInstance` was only ever used as an internal return element type, so it becomes a plain type alias. Together they pushed the knip second pass one over its 215-issue ceiling; the run is back at 215 with no harmony entries.
`list-devices` backstops every discovery branch at BRANCH_DEADLINE_MS, an invariant that only holds while each branch's worst case stays below it - otherwise the backstop truncates a branch that would have completed and drops a real device. The HarmonyOS branch inherited runHarmonyEmulator's 30s default, which is above the 25s deadline, so it now passes its own 6s bound; `Emulator -list` reads a local directory in under 0.1s, making that headroom rather than a real cost. The invariant test covered only the Vega and Android branches, so it stayed green through the breach; it now sums the HarmonyOS branch too and fails if the bound is raised past the deadline.
Adds the `hdc` transport the platform was missing, and with it physical HarmonyOS device discovery. `hdc` exits 0 for everything — an unknown target, a missing file, a remote command that exits 3 — so `runHdc` classifies from the `[Fail]` prefix rather than the status, and `runHdcShell` recovers the remote command's own exit code by echoing it back through a sentinel. Without that, a `uitest` that is not installed is indistinguishable from one that ran. `Emulator -list -details` turns out to report `isRunning`, form factor and OS version as JSON, so instances no longer carry state "unknown"; discovery reads the richer form, which costs the same ~30ms. HarmonyOS ids now split by what they address: `harmony-emulator-<instance>` for a DevEco instance that boot-device starts, `harmony-<connectKey>` for a target hdc drives. A running emulator appears under both, exactly as a running AVD appears in `avds` and in `adb devices`. Verified against a physical HarmonyOS 6.0.1 handset (Mate 60) over USB and against DevEco Studio 6.1's Emulator 6.1.1.200.
Eleven tools now accept a HarmonyOS target: describe, screenshot, gesture-tap, gesture-swipe, keyboard, button, launch-app, restart-app, open-url, await-ui-element and await-screen-idle. All of it goes through the device's own `uitest`, which is HarmonyOS' uiautomator, screencap and input in one binary, reached over `hdc`. That is the same shape `button` and `keyboard` already use on Android, where presses go over `adb` rather than the simulator-server HID transport. `describe` parses `uitest dumpLayout` into the shared tree. Each top-level child is a window, labelled with the bundle that owns it, so an app plus the status bar reads as two named windows rather than two anonymous stacks. ArkUI layout scaffolding is walked through unless it is itself clickable — ArkUI builds real buttons out of Stack and Row. A `UIExtensionComponent` renders in another process and the dump carries the node with no children, so it is labelled rather than emitted as an empty container that reads like an empty screen. Two device behaviours drive the error handling. `aa start` exits 0 on failure and puts the useful code on its second line, so success is read off stdout and the coded line is kept. And `aa start -a` does not take `mainAbility`: it takes the matching `abilityInfos` name, which Calculator spells short and Settings spells qualified — passing `mainAbility` verbatim launches one and fails the other with 10104001. `button` exposes home/back/power, the three keys `uitest` names, and `keyboard` exposes the five keycodes whose effect was observed in a real text field. The rest are refused rather than mapped from documentation: `uitest` accepts any keycode and answers `No Error` whatever it does, so an unverified mapping would be indistinguishable from a working one. Verified end to end against a physical HarmonyOS 6.0.1 handset through the tool-server's HTTP API: launch-app, await-screen-idle, describe, four gesture-taps and a second describe assert the calculator reads 15; gesture-swipe moves a Settings list; keyboard types into the search field and backspace removes a character; button home and back change the foreground bundle; open-url's failure path reports the code and its success path carries the caveat that HarmonyOS reports success for a web URL that opens nothing.
`run-sequence` and `screenshot-diff` supported iOS and Android but rejected HarmonyOS, so a HarmonyOS session paid an `hdc` process spawn per interaction and had no visual-regression path at all. `run-sequence` is platform-neutral by construction — every step re-dispatches through its own tool — so it only needed the outer gate opened; a step the platform cannot run (`gesture-pinch`) is still reported per-step after the preceding steps have landed. `screenshot-diff` needed a capture branch: HarmonyOS has no simulator-server controller, so a live side is captured with `uitest screenCap` over `hdc` and the service is not requested at all, which would otherwise throw before the capture ran. `rotation` is now refused on both rather than accepted and dropped. `uitest screenCap` writes the display in its current orientation and takes no orientation argument, so honouring the parameter would return an unrotated image — and a diff of the wrong orientation reads as a whole-screen change. The `udid` descriptions of all eleven HarmonyOS-capable tools said which id shapes they take and omitted this one; they now name it. Verified on a physical HarmonyOS 6.0.1 handset: a `button power` step through `run-sequence` moves `powerStatus` SUSPEND→ON, a two-step button + keyboard sequence completes 2/2, and `screenshot-diff` with `captureCurrent` reports `unchanged` against a static screen and then one changed region of 369 px once the lock-screen clock ticked a minute. Both rotation guards return 400 from the device path.
latekvo
changed the title
The guard ran while the capture strategy was being chosen, so it also rejected a two-path diff that merely carried `rotation` — a parameter that is inert for a static diff on every platform, making HarmonyOS the one platform where an unused argument failed the call. It now runs inside the capture itself, where the parameter actually has no effect to give.
…forms `keyboard`, `launch-app` and `open-url` each enumerate the platforms they drive, and each enumeration was left describing the set from before this branch — an agent reading `keyboard`'s description had no way to learn the tool works on HarmonyOS, or that only four of its named keys land there.
The asleep hint only fired when the parsed tree had no windows at all, on the belief that a suspended display dumps a bare root. It does not: measured on a Mate 60 at `powerStatus=POWER_STATUS_SUSPEND`, `uitest dumpLayout` returns both `com.ohos.sceneboard` windows, visible flags and all — a tree an agent cannot tell apart from a live screen, while every tap injected against it is a silent no-op. So the hint now keys on the power status it is about, and the empty-tree hint keeps the case it was written for.
`boot-device` returned the instance id, which no interaction tool accepts, so booting a HarmonyOS emulator left the caller with nothing to drive and no way to find out what it had become: the emulator manager names instances and never reports a port, `hdc` names connect keys and never mentions an instance, and the two share no field to join on. It now snapshots the connected targets, starts the instance, and waits for the one that appears — arrival being the only signal that does not depend on how an emulator's key is spelled, which is unobservable outside mainland China. `force` restarts an instance that is already running, which is also the only way to resolve the key of one argent did not start. Whenever the key cannot be had, the payload still names the instance and carries a `note` saying which of the three reasons applied. Also stops reading `harmony-<connectKey>` as an instance name, so a phone's id no longer silently starts an instance named after a serial.
`hdc tconn` registers an `ip:port` target and lists it before any handshake succeeds, so a TCP row with state `Offline` is a state a booted emulator can genuinely be found in — which is what the boot path's wait-for-Connected is there for. The fixture is now that output verbatim rather than a plausible spelling of it. Also drops two stale claims: `harmony-cli.ts` said nothing here talks to a booted device, which stopped being true when the interaction layer landed.
|
Knip is red for a reason outside this PR: Measured on this branch with build output removed, the way The other 13 checks pass. |
Both of these only showed up once a real emulator existed to boot. `Emulator -start` is not a launcher that hands off — it is the emulator's supervisor, and it runs for as long as the emulator does. It was being awaited with the boot budget as its subprocess timeout, so a boot spent the whole budget inside the start, gave the arrival wait nothing, and then the expiring timeout killed the emulator it had just started: 905s to return the wrong id and leave a dead instance behind. It is now spawned detached, exactly as the Android emulator is, with the arrival wait polling alongside and an early exit surfacing the manager's own diagnostic instead of waiting out the budget. Measured after: 8s to a drivable connect key. `-stop` likewise returns once it has asked, not once the emulator is gone, and the guest's `hdc` endpoint dies well before the process does. A restart that waited on the target dropping called `-start` into "Failed to start emulator: this emulator instance is already running" — now a recognised diagnostic — so it waits on `isRunning` instead, the only signal that tracks the process itself.
latekvo
left a comment
latekvo
left a comment
There was a problem hiding this comment.
[Diplomat]: Static review of this PR at f41ae423: read the full diff, then each changed file in whole together with its callers and its nearest sibling implementation (the Android path, the other tree sources, the other capability declarations), and checked every added or changed test against the question of whether it would still pass with the production hunk reverted. Pre-PR behaviour was read from the merge base (git show <merge-base>:<path>) rather than inferred.
An earlier pass over 727e385e is superseded: that head has since been reworked, and the two findings it produced (the boot-device description asserting no interaction tool targets HarmonyOS, and the shape of the dispatch test) no longer apply as written. Everything below was derived fresh against the current head by two reviewers on different lenses, and each observation then survived an adversarial refutation check; anything the check could not confirm was dropped.
Nothing here was reproduced on a device — this was a static read.
| chromium: { handler: chromiumHandler }, | ||
| }); | ||
|
|
||
| await expect(execute({}, { udid: "harmony-Phone_1" })).rejects.toThrow( |
There was a problem hiding this comment.
[Diplomat]: The test guarding the new harmony dispatch arm never reaches it — assertSupported rejects first.
The test "rejects a HarmonyOS device instead of falling through to the chromium handler" (packages/tool-server/test/cross-platform-tool.test.ts:190-210) never reaches the new harmony arm, so it does not pin the guard its comment describes. Trigger: resolveDevice("harmony-Phone_1") returns { platform: "harmony", kind: "device" } — src/utils/device-info.ts:149-152 yields kind "emulator" only for a harmony-emulator- prefix — while the test's capability at line 200 declares harmony: { emulator: true } with no device key. assertSupported (called at src/utils/cross-platform-tool.ts:98, check at src/utils/capability.ts:150) therefore throws UnsupportedOperationError: Tool 'test' is not supported on harmony device (kind 'device' not supported). before the device.platform === "harmony" block at cross-platform-tool.ts:142 is evaluated.
There was a problem hiding this comment.
Confirmed exactly as described, then proven by mutation. A probe printed what the test actually caught: UnsupportedOperationError: Tool 'test' is not supported on harmony device (kind 'device' not supported). — the |harmony alternative of the regex matched it, so the test passed on the 400 path. Deleting the entire device.platform === "harmony" arm from cross-platform-tool.ts left test/cross-platform-tool.test.ts at 8/8 green.
The fix declares both kinds (harmony: { device: true, emulator: true }), drives both a harmony-Phone_1 and a harmony-emulator-argent_phone id through the same assertion, and swaps the regex for rejects.toBeInstanceOf(NotImplementedOnPlatformError) — matching the sibling at test/chromium-dispatch.test.ts:67. Re-running the same mutation now fails it with AssertionError: promise resolved "'from-chromium'" instead of rejecting, which is the exact fallthrough the comment describes.
| opts.source === "cdp-dom" || | ||
| opts.source === "vega-automation" | ||
| opts.source === "vega-automation" || | ||
| opts.source === "harmony-uitest" |
There was a problem hiding this comment.
[Diplomat]: The harmony-uitest nested-renderer addition is the one nested source with no test, though the identical defect on a sibling source has a named regression test.
No test pins harmony-uitest in the nested-renderer condition. Every other source in that disjunction has a test asserting Mode: nested for its exact source string — uiautomator and android-devtools in packages/tool-server/test/describe-format-tree.test.ts (the latter at :491, carrying the comment "Regression: nested mode previously keyed on 'uiautomator' only, so 'android-devtools' responses rendered flat and lost all descendants"), cdp-dom in test/chromium-format-tree.test.ts:35, vega-automation in test/vega-source-parser.test.ts:137. formatDescribeTree is never called with { source: "harmony-uitest" } anywhere in the suite, and test/harmony-describe.test.ts only asserts tree.children.length and the hint string — it never renders, so it cannot observe the mode. Deleting || opts.source === "harmony-uitest" from format-tree.ts:188 still typechecks (plain boolean chain, no exhaustiveness guard) and leaves the whole suite green.
There was a problem hiding this comment.
test/harmony-describe.test.ts now renders what describeHarmony actually returns — its tree and its source — through formatDescribeTree, and asserts Source: harmony-uitest, Mode: nested, and that the 04:39 clock text nested one level inside a WindowScene survives. Feeding the real return value rather than a hand-built { source: "harmony-uitest" } ties the two halves together: a change to either the source string or the renderer condition breaks it.
Verified by the mutation you named — deleting || opts.source === "harmony-uitest" from format-tree.ts:188 now fails with expected 'Source: harmony-uitest\nMode: flat\nC…' to contain 'Mode: nested'.
| | **Desktop & web** | Electron and Chromium apps (incl. React Native Web / Expo web) over CDP | Mouse / keyboard | | ||
| | **HarmonyOS** | Phones over `hdc`, and DevEco Studio emulator instances | Touch / gesture | | ||
|
|
||
| > **HarmonyOS drives whatever `hdc` is connected to.** Connect a phone with USB debugging enabled and it appears as `harmony-<connectKey>` (`kind: "device"`), driven by `describe`, `screenshot`, `screenshot-diff`, `gesture-tap`, `gesture-swipe`, `keyboard`, `button`, `launch-app`, `restart-app`, `open-url`, `run-sequence` and the two `await-*` tools - all through the device's own `uitest`. DevEco Studio's emulator instances appear separately as `harmony-emulator-<name>`, which `boot-device` starts; it then waits for the instance to register with `hdc` and returns that connect key, so a booted emulator is driven by the same tools through the same id shape as a phone. Huawei restricts the emulator **image** download to mainland China, so on a host outside it no instance can be created until an image is obtained. Multi-touch and free-form touch (`gesture-pinch`, `gesture-rotate`, `gesture-custom`) return `501 NotImplementedOnPlatform`, on both device kinds: `uitest uiInput` injects only whole gestures (click / longClick / swipe / drag / fling), one contact at a time, and the level below is shut - `hdc shell` is uid 2000 and `/dev/input` is `Permission denied` on a retail phone and on Huawei's own emulator image alike, where `hdc smode` answers `Cannot set root run mode in undebuggable version`. `rotate`, the profilers and the debugger also return 501: the device exposes no orientation CLI, and argent's profiler and debugger surfaces have no HarmonyOS ingest path. |
There was a problem hiding this comment.
[Diplomat]: Docs promise 501 for the unsupported HarmonyOS tools; every one of them actually returns 400.
The README (and two skill docs) state that the HarmonyOS-unsupported tools return 501 NotImplementedOnPlatform; every one of them returns 400 instead. Trigger: call gesture-pinch, gesture-rotate, gesture-custom, rotate, any profiler tool, or any debugger tool with a harmony-<connectKey> id. None of those tools declares a harmony block in its capability (gesture-pinch/index.ts:66, gesture-rotate/index.ts:86, gesture-custom/index.ts:51, rotate/index.ts:20, native-profiler-start.ts:51, and RN_ONLY_TOOL_CAPABILITY / DEBUGGER_TOOL_CAPABILITY in debugger/debugger-service-ref.ts:60 and :20), so the HTTP capability gate at http.ts:791-808 throws UnsupportedOperationError and answers 400 with TOOL_CAPABILITY_UNSUPPORTED_OPERATION and the message "Tool '' is not supported on harmony device." The 501 / NotImplementedOnPlatformError path is only reachable from dispatchByPlatform (utils/cross-platform-tool.ts:147) when a capability does declare harmony but wires no branch — and every tool in this PR that declares harmony: also wires a handler, so no shi …
There was a problem hiding this comment.
Confirmed against a live server, not just a read: stood up createHttpApp from this branch's dist/ with the real tool definitions and POSTed a harmony- id.
gesture-pinch -> HTTP 400 {"error":"Tool 'gesture-pinch' is not supported on harmony device (no harmony support declared)."}
gesture-rotate -> HTTP 400
gesture-custom -> HTTP 400
rotate -> HTTP 400
native-profiler-start -> HTTP 400
debugger-component-tree -> HTTP 400
Re-run with harmony-emulator-argent_phone: identical 400s ("…on harmony emulator"), so the README's explicit "on both device kinds" was wrong on both kinds.
I checked whether this is loose house style before changing anything, and it is not. README.md:32 was the only occurrence of 501 in any markdown in the repo, and the nearest twin describing the identical mechanism names the gate and the literal message instead: argent-metro-debugger/SKILL.md:12 — "are RN-only and reject Chromium at the capability gate with Tool 'X' is not supported on chromium app".
Worse than the wrong number, the class named is the opposite of the reason printed beside it. capability.ts:34-43 reserves NotImplementedOnPlatformError for "capability could say yes once filled in; the work just hasn't happened", and gives pinch on Android — adb has no multi-touch as the UnsupportedOperationError example. One contact and /dev/input denied is that first case verbatim. So the docs now say the tools are refused at the capability gate with the literal message they actually produce; the code is correct as it stands.
| if (params.text) { | ||
| // `uitest uiInput text` types into whatever holds focus, in one shot — there | ||
| // is no per-character injection, so `delayMs` has nothing to pace (the tool | ||
| // description already lists the platforms that ignore it). |
There was a problem hiding this comment.
[Diplomat]: delayMs is silently ignored on HarmonyOS, but the param description omits it and the code comment claims otherwise.
keyboard's delayMs is accepted and silently dropped on HarmonyOS, and both places that describe it say otherwise. typeHarmony (packages/tool-server/src/tools/keyboard/platforms/harmony.ts:49-68) never reads params.delayMs — harmonyTypeText issues one uitest uiInput text (packages/tool-server/src/utils/harmony-uitest.ts:201-203) — while the same parameter IS honoured on iOS (keyboard/simulator-server-keys.ts:22) and Chromium (keyboard/platforms/chromium.ts:11). Trigger: keyboard with text and delayMs against a platform: "harmony" device (cross-platform-tool.ts:142 dispatches straight to harmonyImpl; no flag or schema branch rejects delayMs there). Impact: the schema description at keyboard/index.ts:42 enumerates exactly the platforms that ignore it — "Ignored on Android phones/tablets ..., on Vega ..., and on TV targets ..." — and HarmonyOS is not among them, so an agent pacing input into a debounced HarmonyOS field is told the delay applies, gets one-shot typing, and receives no signal. That line is unchanged by this PR.
There was a problem hiding this comment.
Both halves were true, and the code comment was the load-bearing one: its parenthetical asserted the tool description already listed the ignoring platforms, which is exactly what would stop a reviewer from opening index.ts:42 to check. The enumeration there named Android phones/tablets, Vega and TV, and HarmonyOS was absent — a negative-only enumeration, so its silence reads as an affirmative claim that the delay applies.
index.ts:42 now reads "…on HarmonyOS (uitest uiInput text types the whole string in one shot)…", which makes the comment true without touching it. argent-device-interact/SKILL.md:173 got the same clause — its positive half ("applies to the iOS simulator and Chromium") was already correct, but its negative list had the same hole.
Left as-is deliberately: run-sequence's step-level delayMs is a runner sleep, honoured on harmony, and a different parameter — worth not conflating.
One divergence the finding did not name, and it is wider than per-character pacing: on iOS and Chromium delayMs is also the hold duration for a lone key press (simulator-server-keys.ts:30-32, chromium.ts:73-85 both sleep between keyDown and keyUp), while harmonyKeyEvent is one atomic uiInput keyEvent. The current wording ("between key presses") does not cover that on any platform, so I have not tried to fix it here.
| }, | ||
| }); | ||
| } | ||
|
|
There was a problem hiding this comment.
[Diplomat]: describe and await-ui-element still enumerate only iOS/Android/Chromium/Vega as tree sources after adding HarmonyOS.
describe and await-ui-element both gained a HarmonyOS path in this PR — harmony: { device: true } in each capability, a harmony dispatch branch (describe/index.ts:138) and fetchTree branch (await-ui-element/index.ts:250-252) routing to describeHarmony, and "HarmonyOS id" added to each udid param — but neither tool's description prose was updated, and in both the prose is an enumeration that was exhaustive before this PR and no longer is:
- describe/index.ts:155-162 documents one backend per platform ("On iOS, uses the AXRuntime…", "On Android, runs
uiautomator dump", "On Chromium, walks the renderer's DOM…", "On Vega (Fire TV), reads the on-device automation toolkit…") and stops. HarmonyOS'suitest dumpLayoutbackend, thesource: "harmony-uitest"value it puts in the response header, and its two runtime hints (display suspended; empty layout dump) are undocumented — including the display-off case, which is the HarmonyOS analogue of the Vega "empty tree, relaunch the app" caveat this same paragraph spells out.
There was a problem hiding this comment.
[Diplomat]: Fixed in 1840a2b — and the sweep found seven more of the same class.
Both prose blocks are byte-identical to origin/main: the PR changed only udid, capability, the import and the dispatch branch in each. describe now documents the uitest dumpLayout backend, the source: "harmony-uitest" value, and both runtime hints — including the suspended-display case, which as you note is the analogue of the Vega empty-tree caveat in the same paragraph, and strictly worse: an empty tree is visibly empty, a last-composited one is not. await-ui-element's four-item parenthetical gained the fifth backend.
The wider class, all in the same commit. contract.ts:72-80 is a comment whose stated purpose is that agents branch on source, and harmony-uitest was the one union member it did not explain — its twin in format-tree.ts:180-182 was updated, so this was an isolated miss. gesture-tap:133 picks the discovery tool by platform and stopped at Chromium. launch-app's bundleId and activity, restart-app's activity, and open-url's scheme list each enumerate every platform but this one — and open-url's note line now carries the aa start -U caveat that HARMONY_WEB_URL_CAVEAT only delivers after a call, which is a poor time to learn it. Five tools' "fails if … not reachable" sentences named the sim-server, the emulator backend and Chromium CDP but not hdc. Two more enumerations in argent-device-interact/SKILL.md — the describe row at :86, and :230, which listed exactly the three platforms await-screen-idle supported before this PR added the fourth, so as written it asserted HarmonyOS was unsupported.
keyboard's searchHint also named Vega and TV while omitting the platform it had just gained, and it is a deferred tool, so "harmony type text" would not have surfaced it. Added there and to describe / screenshot-diff / restart-app.
Not touched, having checked them: await-screen-idle's description is correctly generic, and run-sequence / boot-device / list-devices were all updated properly — run-sequence's per-step platform tags are the reference the rest should have matched.
`resolveHarmonyKeycode` indexed HARMONY_KEYCODES bare and tested the result
against undefined, so "constructor" / "__proto__" / "toString" resolved to a
prototype value, passed the guard, and reached `uiInput keyEvent ${key}` -
which `harmonyKeyEvent` interpolates unquoted, against runHdcShell's documented
contract. The four sibling backends all use `Object.hasOwn` plus a case-fold
and each carries a comment naming this exact failure; HarmonyOS had neither,
so `key: "Enter"` also 400'd here alone.
Backfills the harmony cases the sibling test files were each missing: the
per-backend prototype-key bucket in keyboard-error-taxonomy, the text-then-key
ordering pair in keyboard-key-order, and the button loop asserting every
HarmonyOS button reaches its own `uitest` key name over hdc.
Also fixes two tests that could not fail. The dispatch test declared
`harmony: { emulator: true }` while driving a `kind: "device"` id, so
assertSupported rejected before the arm it names; deleting the whole harmony
dispatch arm left it green. And nothing rendered a `harmony-uitest` tree, so
dropping it from format-tree's nested condition was free.
…me HarmonyOS in the enumerations it was left out of The three docs promising `501 NotImplementedOnPlatform` for gesture-pinch / gesture-rotate / gesture-custom / rotate / the profilers / the debugger described a path none of them takes: none declares a `harmony` capability, so the HTTP gate refuses them with UnsupportedOperationError (400) before dispatch runs. 501 is reachable only for a tool that declares harmony and wires no branch, and every tool here that declares it wires one. The class named is also the opposite of the reason given beside it: capability.ts reserves NotImplementedOnPlatform for work not yet done, and the README's own justification - one contact, /dev/input denied - is the platform-impossible case UnsupportedOperation exists for. The rest are enumerations that were exhaustive before HarmonyOS and stopped being so. `describe` and `await-ui-element` list one tree backend per platform and omitted `uitest dumpLayout`, its `harmony-uitest` source value, and the suspended-display hint - the analogue of the Vega empty-tree caveat spelled out in the same paragraph. gesture-tap's discovery-tool advice is picked by platform; launch-app's `bundleId` and `activity`, restart-app's `activity`, open-url's scheme list and the "not reachable" sentence on five tools each enumerate every platform but this one. `keyboard`'s `delayMs` lists exactly the platforms that ignore it, and HarmonyOS - which types in one `uitest uiInput text` shot - was not among them, which is what the comment beside the harmony backend claimed it was.
Each of these ships conditional behaviour, not a delegation, and each sat beside a test file that already held one case per other backend: - gesture-tap: the clickCount===2 native doubleClick fast path, the paced loop above it, and the normalized -> device-pixel conversion, against a non-square display and fractions where a swapped axis reads differently - gesture-swipe: the settle -> swipe/fling verb choice, which reads backwards at a glance, plus the duration->velocity conversion and both ends of its clamp - open-url: the web-URL caveat appended to the shared deep-link note, and the omit case for a custom scheme that already fails loudly on device - await-ui-element / await-screen-idle: that an unreadable screen cannot confirm `hidden`. HarmonyOS reaches that guarantee differently from its siblings - its fetcher throws rather than returning a flagged empty tree - so this pins the shared pollDescribeTree guard from a fifth direction. Every assertion was checked against a mutation of the code it covers: deleting the doubleClick path, swapping the tap axes, inverting the swipe verb, dropping the velocity clamp, reversing the swipe endpoints, dropping and unconditionally attaching the URL caveat, neutering isBlindRead, deleting the harmony fetchTree branch, and passing the prefixed device id where the connect key belongs. All fail. Neutering isBlindRead also reddens the iOS and Android cases, confirming the guard is shared rather than re-implemented.
…w, and enforce the hdc timeout Three defects the sibling implementations already handle. **Frames were clamped per component instead of clipped.** ArkUI reports real off-screen bounds for rows scrolled past the edge of a List - the old comment said so - and clamping x, y, width and height independently turned a row at [0,-400][1216,-260] into y=0 height=0.052: a full-width tap target at the top of the screen, with the same frame as the row genuinely straddling that edge. A row past the bottom reported a tap centre of y=1.026, and one past the right edge x=1.078. Both the Android and Vega adapters clip to the screen rect before normalising, and the Android call site carries a comment describing this exact failure; harmony now reuses the helper the Android one already exports. **The same-rect collapse discarded everything but `clickable`.** ArkUI sets `.id()` and the state flags on the OUTER component, so the wrapper of a same-rect pair is usually the only node that knows it is checked, disabled or scrollable. Collapsing it dropped the identifier agents and flows select on, turned a List holding one full-height row into a plain Cell with no scrollable region to swipe, reduced a Toggle to an unlabelled Image with no on/off state, and reported a disabled Button as an ordinary tappable one. The collapse now fires only when the wrapper genuinely knows nothing its child does not, which is what the comment always claimed it did. **`hdc` and `Emulator` were spawned without a kill signal.** `execFile`'s `timeout` sends `killSignal` once and never escalates, and the default SIGTERM is what a client blocked on a wedged daemon ignores. Measured against a stub that traps TERM: still running 6s after a 1s timeout, versus 1005ms with SIGKILL. That left every per-call HarmonyOS budget advisory - list-devices has a deadline backstop, the interaction tools have none. adb, vega-cli and simctl all set SIGKILL, each with a written rationale. Every assertion is mutation-checked, including per-field over the new collapse guard: seven of its eight fields fail the suite individually when dropped, and `checked` is pinned jointly with `checkable` (a checked switch is always checkable, so isolating it would mean asserting on a dump shape that does not occur). `value` was dropped from the guard as unreachable - it is only ever set alongside a label, which the collapse already refuses.
holdUitestQueue takes the call's signal and refuses with an AbortError before each injection, so a tap/swipe/pinch/button/keyboard call cancelled while queued no longer injects and reports success. Also rewords the boot-device in-flight comment in present tense.
Recommending gesture-scroll/gesture-drag for cdp-dom is a Chromium change unrelated to HarmonyOS; HarmonyOS uses the default list either way.
# Conflicts: # packages/skills/rules/argent.md # packages/skills/skills/argent-device-interact/SKILL.md # packages/skills/skills/argent-screen-recording/SKILL.md # packages/skills/skills/argent-test-ui-flow/SKILL.md # packages/tool-server/src/tools/await-screen-idle/index.ts # packages/tool-server/src/tools/await-ui-element/index.ts # packages/tool-server/src/tools/devices/boot-device.ts # packages/tool-server/src/tools/devices/list-devices.ts # packages/tool-server/src/tools/gesture-drag/index.ts # packages/tool-server/src/tools/gesture-scroll/index.ts # packages/tool-server/src/tools/gesture-tap/index.ts # packages/tool-server/src/tools/keyboard/index.ts # packages/tool-server/src/tools/launch-app/index.ts # packages/tool-server/src/tools/open-url/index.ts # packages/tool-server/src/tools/restart-app/index.ts # packages/tool-server/src/tools/run-sequence/index.ts # packages/tool-server/src/tools/screen-recording/screen-recording-start.ts # packages/tool-server/src/tools/screenshot/index.ts # packages/tool-server/src/utils/vega-screen.ts # packages/tool-server/test/screenshot-default-scale.test.ts
# Conflicts: # packages/docs/docs/reference/tools.mdx # packages/skills/rules/argent.md # packages/skills/skills/argent-device-interact/SKILL.md # packages/skills/skills/argent-screen-recording/SKILL.md # packages/skills/skills/argent-test-ui-flow/SKILL.md
list-devices skips hdc and DevEco when the flag is off; harmony ids are refused at the capability gate and at harmonyConnectKey, the chokepoint every drive path takes; boot-device refuses HarmonyOS instances. GET /tools renders <flag:...> spans and x-argent-flag properties per request.
- wait tools: a zero-budget deadline poll on HarmonyOS settles as unsettled instead of self-refusing (a hidden wait timed out unreadable), and the shared loop keeps main's handling of a real final-poll error - harmonyDisplay prefers an awake panel over the first one listed - cap the in-queue display re-read at HARMONY_DISPLAY_TIMEOUT_MS - boot notes no longer suggest a longer bootTimeoutMs, which a retry ignores - a harmony-emulator- id gets a capability message naming the id to drive - e2e: skip the post-swipe tree settle where the runner has no tree; the recorder says an await:/assert: conversion fails there - tests: flag gate + mutation-checked cases
Always-loaded text (the argent rule, skill descriptions, README headline) no longer mentions HarmonyOS. Every HarmonyOS fact from the skill bodies moves to argent-device-interact/references/harmonyos.md, reached through one pointer line in argent-device-interact and one in the e2e YAML reference. The e2e section there states what the runner actually does on a named harmony device: coordinate tap, selector-less swipe/pinch, launch, snapshot without cropOn, wait and tool steps run; selector steps, idle, long-press and rotate do not. Docs mark HarmonyOS as experimental and off by default, and list the harmonyos flag in the configuration reference.
…rmonyos flag Wrap every HarmonyOS-specific span of the advertised tool text (descriptions, .describe() texts, searchHints) in <flag:harmonyos> tags, gate boot-device's harmonyInstance property with x-argent-flag, and render boot-device's target-selection error through renderFlagGated. With the flag off the text matches origin/main; with it on it matches the untagged text, except for a few lists reordered so the off form keeps its 'or'. extract-tools.mjs emits flag spans as if every flag were on, so SpiderShield scores the full text.
… no HarmonyOS text
The render service's resolution is the panel mode and stays portrait when the display rotates, while uitest coordinates and layout bounds rotate with it. A tap at a describe frame in landscape landed off-screen and was still reported as tapped. Read the display manager's ScreenRotation for the same screen and swap the size on a quarter turn.
…scriptions - redactSecretsFromError also matches a value as a shell-quoted command line carries it, so a typed secret containing ' no longer leaks through a failed HarmonyOS uitest or Android input text call - gesture-swipe: on HarmonyOS durationMs sets speed, clamped to uitest's range - open-url: HarmonyOS fails whenever aa start -U refuses the URI
…ith the tool text
# Conflicts: # packages/tool-server/src/tools/gesture-swipe/index.ts # packages/tool-server/src/tools/gesture-tap/index.ts
Validates the .hap's module.json bundleName before uninstalling, tolerates only 'not installed' on the uninstall, and reads the result from hdc's msg: line since hdc exits 0 either way.
…r load A 40ms budget let the first failing read miss the deadline under a full suite run, so the note said 'Ran out of time' instead of the dump failure.
…tion Keep one statement that HarmonyOS support is experimental and off by default, in supported-platforms. Drop the repeated experimental and flag qualifiers from passing mentions.
#1438 removed every em dash from tool descriptions. This branch brought back nine: four in text served with the harmonyos flag off (button, keyboard `key`) and five inside flag spans (boot-device, describe, gesture-tap). With the flag off, the button and keyboard `key` text is now byte-identical to main again.
|
Done a final verification pass, hidden the entire feature behind a feature flag, i believe this works well as a base for the support of this platform - all the most important features are ported and tested. |
|
The remaining CIs are Mac runners. They were passing on the last completed pass. No mac changes since. Verified and the changes to existing surface are minimal. So taking the risk onto myself and merging without waiting for them - queued for 2h already and likely will have to wait till 1AM before they even start running. |
Adds a
harmonyplatform behind theharmonyosfeature flag (off by default,argent enable harmonyos).list-devicesreports HarmonyOS phones (overhdc) and DevEco Studio emulator instances.boot-devicestarts an instance and returns thehdcconnect key it registers as. The tools below then drive that key, whether it is a phone or a booted emulator.argent enable harmonyoslist-deviceshdc)boot-devicedescribeuitest dumpLayout; the status bar is includedscreenshot/screenshot-diffrotationparam)await-ui-element/await-screen-idlegesture-tapgesture-swipedurationMssets speed, clamped to 200-40000 px/s)gesture-pinchgesture-rotate/gesture-custom(long-press)keyboarddelayMs, no newlinespastebuttonlaunch-app/restart-appactivityignored)reinstall-app.hapviahdc uninstall+hdc installopen-urlaa start -U; always returns a verify-the-screen noterun-sequencerotate/fold/shakesettings-permissionsdeviceonly; coordinate steps,launch, uncroppedsnapshot,wait,script; selectors error per stepTransport:
uitestoverhdc shellfor input, layout and capture. The second pinch finger comes from/system/bin/uinput -T -m. Calls touitestare serialized per device because overlapping invocations block each other and then get SIGKILLed. Every input tool refuses to inject while the panel is suspended, becauseuitestreportsNo Errorfor input that lands nowhere. An input cancelled while it waits in that queue injects nothing.Flag off:
list-devicesnever runshdcor the Emulator manager. Anyharmony-*id, andboot-devicewithharmonyInstance, get a 400 that namesargent enable harmonyos. Tool descriptions and schemas contain no HarmonyOS text; the skills carry it inargent-device-interact/references/harmonyos.mdand two one-line pointers to it. Description spans marked<flag:harmonyos>are rendered per request inGET /tools, so toggling the flag needs no tool-server restart.Other platforms: one behaviour change. Secret redaction also matches a value in its shell-quoted form (
'→'\''), so a secret containing'no longer leaks through a failed Androidinput textcall.Docs:
packages/docsmarks HarmonyOS as experimental and flag-gated, and documents the flag in the configuration reference. The skill guidance moved toargent-device-interact/references/harmonyos.md.Out of scope: the cross-platform secret-redaction rewrite (#797), stale screenshots from a suspended panel (#792), and capability-checking flow
tool:steps (#796; an unsupported tool there fails on a service error instead of the refusal).Verification
Emulator 6.1.1.200(macOS). No device screenshots are included because the handset is a personal phone.boot-devicecan start an instance only if one is already installed, and it says so instead of reporting a boot it did not perform.screenshotuses the same default scale as the other platforms (ARGENT_SCREENSHOT_SCALE, default 0.25). Arotationargument is refused becauseuitest screenCapcannot take one.open-urlattaches a caveat to every URL, becauseaa start -Ureports success for anything the system accepts, including schemes that only open a chooser.await-ui-elementreturnscause: "unreadable"(notunmet) when it times out on a suspended panel's last frame. Both wait tools add anoteto a success only when the read came from a suspended panel.test/harmony-input-cancel.test.tscancels tap, swipe, pinch, button and keyboard while anotheruitestcall holds the queue. All 5 cases fail on the previous head (each reports success and injects). Removing the signal from any one call site fails that site's case.test/harmony-flag-gate.test.tsandtest/harmony-flag-tool-text.test.ts. Removing each gate (capability, connect key, boot, list-devices,GET /toolsrendering,requiredfiltering) fails a case. E2E on the emulator: togglingflags.jsonbetween requests flipsGET /toolsand MCPtools/listbetween 0 and 118 HarmonyOS mentions, and the id between 400 and driving.reinstall-app: on the emulator with a minimal signed.hapover HTTP. A mismatched bundle id and a non-zip.hapare refused with 400 andinstallTimeis unchanged. A matching.hapreinstalls (installTimechanges,launch-app+describeshow the app), including when it was not installed. Each check intest/harmony-reinstall.test.tsfails when its branch is removed.main: prettier, eslint,tsc --build, the tool-server test typecheck, knip, docusaurus build, every workspace's tests, and the full tool-server suite (6 shards). The one local failure is e2e-tool-aliases test fails on macOS while argent-lens is off #1526, which is macOS-only and independent of this PR.