From 0ab3c51f3b5caaaf90d39d55e66a6b6752c0b95a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Mon, 10 Aug 2026 19:27:44 +0200 Subject: [PATCH 001/110] feat(harmony): add HarmonyOS emulator discovery and boot Adds a `harmony` platform driven by DevEco Studio's `Emulator` manager: `list-devices` reports emulator instances and `boot-device` starts one by name. Device ids are `harmony-`, minted by argent the way `chromium-cdp-` is, so classification never depends on a vendor serial shape - the connect key of a running HarmonyOS emulator could not be observed on a host that cannot obtain an image. The manager's exit code is not a success signal: `-create`, `-stop` and `-install` report failures with exit 0, while `-start` exits 1. So `emulatorFailure` classifies from stdout against diagnostics measured against DevEco Studio 6.1, and `runHarmonyEmulator` deliberately does not reject on a non-zero exit, keeping one verdict path instead of two that disagree. Dispatch rejects the platform explicitly rather than falling through: chromium is the last branch of `dispatchByPlatform`, so a HarmonyOS device would otherwise be handed to the CDP handler and its result reported as a success. Huawei serves HarmonyOS emulator images only within mainland China. Outside it no image downloads, so no instance can be created; boot-device reports what the manager said instead of claiming a boot it did not perform, and distinguishes that restriction from an empty instance list, which is also what a host that has simply not created one yet looks like. Interaction tools are out of scope - screenshot, describe, input and app-lifecycle return 501 on a HarmonyOS device - because with no bootable image their behaviour could not be verified. --- README.md | 4 + packages/registry/src/types.ts | 18 +- packages/skills/rules/argent.md | 4 +- .../skills/argent-device-interact/SKILL.md | 2 +- packages/telemetry/src/events.ts | 1 + .../tool-server/src/tools/button/index.ts | 4 + .../src/tools/devices/boot-device.ts | 184 ++++++++++++++++-- .../src/tools/devices/list-devices.ts | 55 ++++-- packages/tool-server/src/utils/capability.ts | 2 + packages/tool-server/src/utils/check-deps.ts | 8 + .../src/utils/cross-platform-tool.ts | 6 + packages/tool-server/src/utils/device-info.ts | 32 ++- packages/tool-server/src/utils/harmony-cli.ts | 174 +++++++++++++++++ .../tool-server/src/utils/harmony-devices.ts | 53 +++++ .../test/cross-platform-tool.test.ts | 27 +++ packages/tool-server/test/harmony-cli.test.ts | 133 +++++++++++++ .../tool-server/test/harmony-devices.test.ts | 84 ++++++++ .../test/variant-proposals.test.ts | 2 +- 18 files changed, 752 insertions(+), 41 deletions(-) create mode 100644 packages/tool-server/src/utils/harmony-cli.ts create mode 100644 packages/tool-server/src/utils/harmony-devices.ts create mode 100644 packages/tool-server/test/harmony-cli.test.ts create mode 100644 packages/tool-server/test/harmony-devices.test.ts diff --git a/README.md b/README.md index dceff443cf..ae048b9fa1 100644 --- a/README.md +++ b/README.md @@ -27,6 +27,9 @@ Argent drives a growing set of targets through a single toolkit, each with the r | **Android** | Emulators (AVDs) and physical devices over adb | Touch / gesture | | **TV** | Apple TV (tvOS), Android TV / Google TV, Amazon Fire TV (Vega) | D-pad / remote | | **Desktop & web** | Electron and Chromium apps (incl. React Native Web / Expo web) over CDP | Mouse / keyboard | +| **HarmonyOS** | Emulator instances managed by DevEco Studio | None - see below | + +> **HarmonyOS is discovery-only.** `list-devices` lists DevEco Studio's emulator instances (id `harmony-`, always `state: "unknown"` - Argent does not determine whether an instance is running) and `boot-device` can start one by name. Every other tool - screenshot, describe, tap and gestures, keyboard, launch-app - returns `501 NotImplementedOnPlatform`. Huawei restricts the emulator image download to mainland China; outside it no instance can be created, so there is nothing to list or boot. ## Capabilities @@ -56,6 +59,7 @@ Argent drives a growing set of targets through a single toolkit, each with the r - For iOS / tvOS: macOS with **Xcode** installed (Apple TV uses tvOS simulators — Xcode downloads the tvOS runtime on demand) - For Android / Android TV: **Android SDK Platform Tools** (`adb`) on `PATH`, and the **Android Emulator** package if you want to boot AVDs from Argent. Create AVDs via Android Studio or `avdmanager`. - For Fire TV (Vega): the **Vega SDK** (`vega` CLI) on `PATH` +- For HarmonyOS: **DevEco Studio**, which ships the `Emulator` manager Argent drives. It is looked up inside DevEco Studio's install root - the macOS app bundle by default, `$DEVECO_STUDIO_HOME` on any other host - and never on `PATH`, since `Emulator` is too generic a name to match safely. Huawei restricts the emulator image download to mainland China, so outside it no emulator instance can be created. - For Electron / Chromium: nothing extra to control an already-running app - just launch it with `--remote-debugging-port`, or let Argent spawn your Electron app for you ##### Linux host: extra prerequisites for Android emulators diff --git a/packages/registry/src/types.ts b/packages/registry/src/types.ts index 4036dc6b6e..75cfbaf5ca 100644 --- a/packages/registry/src/types.ts +++ b/packages/registry/src/types.ts @@ -144,7 +144,7 @@ export interface ToolContext extends InvokeToolOptions { // ── Device + Capability Types ── -export type Platform = "ios" | "android" | "ios-remote" | "chromium" | "vega"; +export type Platform = "ios" | "android" | "ios-remote" | "chromium" | "vega" | "harmony"; export type DeviceKind = "simulator" | "emulator" | "vvd" | "device" | "app" | "unknown"; @@ -193,6 +193,14 @@ export interface ToolCapability { vvd?: boolean; device?: boolean; }; + /** + * HarmonyOS support. Emulator-only: the tool-server neither detects nor + * connects to physical HarmonyOS hardware, so there is no `device` flag to + * declare — adding one is deferred to a version where it can be tested. + */ + harmony?: { + emulator?: boolean; + }; /** Optional refiner. Returns true if this device is supported. */ supports?: (device: DeviceInfo) => boolean; } @@ -217,7 +225,13 @@ export interface ToolCapability { * On a missing binary, the HTTP layer returns 424 Failed Dependency with an * install hint the agent can surface verbatim. */ -export type ToolDependency = "adb" | "xcrun" | "emulator" | "sim-remote" | "vega"; +export type ToolDependency = + | "adb" + | "xcrun" + | "emulator" + | "sim-remote" + | "vega" + | "harmony-emulator"; // ── Tool Types ── diff --git a/packages/skills/rules/argent.md b/packages/skills/rules/argent.md index dbf578b2d1..a9ad223faa 100644 --- a/packages/skills/rules/argent.md +++ b/packages/skills/rules/argent.md @@ -4,7 +4,7 @@ alwaysApply: true --- -If argent is installed and configured in this environment, its MCP tools are the preferred form of interaction with the application for iOS simulator, Android emulator, Chromium (CDP) app, and Vega (Amazon Fire TV) device control; otherwise see `` below before attempting any argent workflow. A "Chromium (CDP) app" is any Chromium runtime exposing a Chrome DevTools Protocol endpoint — an Electron app, or any Chromium-family browser (Chrome/Brave/Edge) launched with `--remote-debugging-port`; all are driven through the same tool surface and tagged `platform: "chromium"`. A "Vega device" is a virtual device (VVD) or physical unit — driven by tv-remote (D-pad) and tagged `platform: "vega"`. +If argent is installed and configured in this environment, its MCP tools are the preferred form of interaction with the application for iOS simulator, Android emulator, Chromium (CDP) app, and Vega (Amazon Fire TV) device control; otherwise see `` below before attempting any argent workflow. A "Chromium (CDP) app" is any Chromium runtime exposing a Chrome DevTools Protocol endpoint — an Electron app, or any Chromium-family browser (Chrome/Brave/Edge) launched with `--remote-debugging-port`; all are driven through the same tool surface and tagged `platform: "chromium"`. A "Vega device" is a virtual device (VVD) or physical unit — driven by tv-remote (D-pad) and tagged `platform: "vega"`. HarmonyOS emulator instances (id `harmony-`, tagged `platform: "harmony"`) support only `list-devices` and `boot-device`; every other tool returns 501, so never pick one as an interaction target. Running MCP server and managing the Argent toolkit utilises `argent` command - if asked use `argent --help` for reference. To check current version of MCP server run `argent --version` command. @@ -98,7 +98,7 @@ Load the matching skill before starting work and executing tools from argent-mcp procedure and edge-case handling for each workflow. PLATFORM DETECTION -If the user did not specify a platform, call `list-devices` first and pick the booted target — do not default to iOS. Vega (Amazon Fire TV) devices appear as `platform:"vega"`, when present load `argent-tv-interact` +If the user did not specify a platform, call `list-devices` first and pick the booted target — do not default to iOS. Vega (Amazon Fire TV) devices appear as `platform:"vega"`, when present load `argent-tv-interact`. HarmonyOS entries appear as `platform:"harmony"` with `state:"unknown"` (Argent never reports their readiness); they are boot targets only, so pick one only when the user asked for HarmonyOS. iOS SIMULATOR SETUP Skill: `argent-ios-simulator-setup` diff --git a/packages/skills/skills/argent-device-interact/SKILL.md b/packages/skills/skills/argent-device-interact/SKILL.md index 871ebe1b60..840e50516d 100644 --- a/packages/skills/skills/argent-device-interact/SKILL.md +++ b/packages/skills/skills/argent-device-interact/SKILL.md @@ -21,7 +21,7 @@ For platform-specific caveats (Metro `adb reverse`, locked-screen describe error If you delegate simulator tasks to sub-agents, make sure they have MCP permissions. -Use `list-devices` to get a target id. Results are tagged with `platform` (`ios`, `android`, or `chromium`); booted/ready devices come first. Pick the first entry that matches the platform you need — if none are ready, call `boot-device` with `udid` (iOS), `avdName` (Android), or `electronAppPath` (boots an Electron app as a `chromium` device). A Chromium browser already running with a CDP port shows up directly — no `boot-device` needed. See `argent-ios-simulator-setup` / `argent-android-emulator-setup` for full setup flow. +Use `list-devices` to get a target id. Results are tagged with `platform` (`ios`, `ios-remote`, `android`, `chromium`, `vega`, or `harmony`); booted/ready devices come first. A `vega` target is a TV; drive it with `argent-tv-interact` (see the note above). A `harmony` target is a HarmonyOS emulator instance: only `list-devices` and `boot-device` support it, every tool in this skill returns 501 for one. Pick the first entry that matches the platform you need — if none are ready, call `boot-device` with `udid` (iOS), `avdName` (Android), or `electronAppPath` (boots an Electron app as a `chromium` device). A Chromium browser already running with a CDP port shows up directly — no `boot-device` needed. See `argent-ios-simulator-setup` / `argent-android-emulator-setup` for full setup flow. **Load tool schemas before first use.** Gesture tools (`gesture-tap`, `gesture-swipe`, `gesture-pinch`, `gesture-rotate`, `gesture-custom`) may be deferred — their parameter schemas are not loaded until fetched. Always use ToolSearch to load the schemas of all gesture tools you plan to use **before** calling any of them. If you skip this step, parameters may be coerced to strings instead of numbers, causing validation errors. diff --git a/packages/telemetry/src/events.ts b/packages/telemetry/src/events.ts index a536bff92e..78dcb765bf 100644 --- a/packages/telemetry/src/events.ts +++ b/packages/telemetry/src/events.ts @@ -21,6 +21,7 @@ export const PLATFORMS = [ "android", "chromium", "vega", + "harmony", "tvos", "android-tv", ] as const; diff --git a/packages/tool-server/src/tools/button/index.ts b/packages/tool-server/src/tools/button/index.ts index 94f27bcd37..bfb603bc98 100644 --- a/packages/tool-server/src/tools/button/index.ts +++ b/packages/tool-server/src/tools/button/index.ts @@ -44,6 +44,10 @@ export const BUTTONS_BY_PLATFORM: Record // `tv-remote` tool, and this tool's capability omits `vega` so a Vega device is // rejected before this map is consulted. Empty set keeps the record total. "vega": new Set([]), + // No tool drives a booted HarmonyOS device yet, and this tool's capability + // omits `harmony`, so a HarmonyOS device is rejected before this map is + // consulted. Empty set keeps the record total. + "harmony": new Set([]), }; const capability: ToolCapability = { diff --git a/packages/tool-server/src/tools/devices/boot-device.ts b/packages/tool-server/src/tools/devices/boot-device.ts index 168452107b..f54ec066b0 100644 --- a/packages/tool-server/src/tools/devices/boot-device.ts +++ b/packages/tool-server/src/tools/devices/boot-device.ts @@ -35,7 +35,12 @@ import { ensureDep } from "../../utils/check-deps"; import { linuxBootDiagnostics } from "../../utils/linux-preflight"; import { listIosSimulators } from "../../utils/ios-devices"; import { deviceSetForUdid, simctlPrefix } from "../../utils/ios-device-sets"; -import { classifyDevice, stripRemotePrefix } from "../../utils/device-info"; +import { + classifyDevice, + harmonyInstanceName, + stripRemotePrefix, + HARMONY_ID_PREFIX, +} from "../../utils/device-info"; import { simctlBoot as simRemoteBoot, simctlBootstatus as simRemoteBootstatus, @@ -45,12 +50,18 @@ import { import { listVvdImages } from "../../utils/vega-sdk"; import { startVvd, stopVvd, isVvdRunning, waitForVvdRunning } from "../../utils/vega-vvd"; import { resolveRunningVvdSerial, listVegaDevices } from "../../utils/vega-devices"; +import { + emulatorFailure, + isChinaOnlyRestriction, + runHarmonyEmulator, +} from "../../utils/harmony-cli"; +import { listHarmonyInstances } from "../../utils/harmony-devices"; import { bootElectronApp, type ElectronBootResult } from "./boot-electron"; const execFileAsync = promisify(execFile); -// NOTE on mutual exclusion: `udid` / `avdName` / `vvdImage` / `electronAppPath` -// are exactly-one — but zod's +// NOTE on mutual exclusion: `udid` / `avdName` / `vvdImage` / `harmonyInstance` +// / `electronAppPath` are exactly-one — but zod's // `.refine()` returns a ZodEffects that our Registry ToolDefinition type does // not accept (it requires a ZodObject so the JSON Schema generator can walk // `.shape`). The exactly-one check therefore lives inside `execute` and @@ -62,19 +73,25 @@ const zodSchema = z.object({ .string() .optional() .describe( - "iOS: simulator UDID to boot (from `list-devices`). Provide exactly one of `udid`, `avdName`, `vvdImage`, or `electronAppPath`." + "iOS: simulator UDID to boot (from `list-devices`). Provide exactly one of `udid`, `avdName`, `vvdImage`, `harmonyInstance`, or `electronAppPath`." ), avdName: z .string() .optional() .describe( - "Android: AVD name to launch a new emulator from (from `list-devices` → `avds[].name`). Provide exactly one of `udid`, `avdName`, `vvdImage`, or `electronAppPath`." + "Android: AVD name to launch a new emulator from (from `list-devices` → `avds[].name`). Provide exactly one of `udid`, `avdName`, `vvdImage`, `harmonyInstance`, or `electronAppPath`." ), vvdImage: z .string() .optional() .describe( - "Vega (Fire TV): VVD image to boot — the `vvdImage` of a Vega device from `list-devices` (e.g. `tv`). Starts the single SDK-managed Vega Virtual Device. Provide exactly one of `udid`, `avdName`, `vvdImage`, or `electronAppPath`." + "Vega (Fire TV): VVD image to boot — the `vvdImage` of a Vega device from `list-devices` (e.g. `tv`). Starts the single SDK-managed Vega Virtual Device. Provide exactly one of `udid`, `avdName`, `vvdImage`, `harmonyInstance`, or `electronAppPath`." + ), + harmonyInstance: z + .string() + .optional() + .describe( + "HarmonyOS: emulator instance to start — the `name` of a harmony device from `list-devices` (e.g. `Phone_1`); the `harmony-` id that entry reports as its `udid` is accepted too. Provide exactly one of `udid`, `avdName`, `vvdImage`, `harmonyInstance`, or `electronAppPath`." ), bootTimeoutMs: z .number() @@ -83,23 +100,25 @@ const zodSchema = z.object({ .max(900_000) .optional() .describe( - "Android/Vega: overall budget for the boot sequence. Default 480000 (8 min) on Android, 120000 (2 min) on Vega. Clamped to [30s, 15min]. Ignored on iOS." + "Android/Vega/HarmonyOS: overall budget for the boot sequence. Default 480000 (8 min) on Android, 120000 (2 min) on Vega and HarmonyOS. Clamped to [30s, 15min]. Ignored on iOS." ), force: z .boolean() .optional() - .describe("Shut down and re-boot the device even if already running."), + .describe( + "Shut down and re-boot the device even if already running. iOS/Android/Vega only: HarmonyOS exposes no running state to act on, and an Electron launch always spawns a fresh process." + ), headless: z .boolean() .optional() .describe( - "iOS only: boot the simulator core WITHOUT opening the Simulator.app GUI window. The device still streams via simulator-server; used by Argent Lens. Set the `ARGENT_SIMULATOR_NO_WINDOW` env var (1/true/yes) to force this host-wide without passing the flag per call (the iOS analog of `ARGENT_EMULATOR_NO_WINDOW`). Ignored on Android/Vega/Electron, which have no equivalent GUI step." + "iOS only: boot the simulator core WITHOUT opening the Simulator.app GUI window. The device still streams via simulator-server; used by Argent Lens. Set the `ARGENT_SIMULATOR_NO_WINDOW` env var (1/true/yes) to force this host-wide without passing the flag per call (the iOS analog of `ARGENT_EMULATOR_NO_WINDOW`). Ignored on Android/Vega/HarmonyOS/Electron, which have no equivalent GUI step." ), electronAppPath: z .string() .optional() .describe( - "Electron: path to the Electron app to launch. Either a packaged .app bundle / executable, or a project directory whose package.json points the Electron binary at the entry script. Mutually exclusive with udid/avdName." + "Electron: path to the Electron app to launch. Either a packaged .app bundle / executable, or a project directory whose package.json points the Electron binary at the entry script. Provide exactly one of `udid`, `avdName`, `vvdImage`, `harmonyInstance`, or `electronAppPath`." ), electronPort: z .number() @@ -125,11 +144,19 @@ type BootDeviceResult = | { platform: "ios-remote"; udid: string; booted: true } | { platform: "android"; serial: string; avdName: string; booted: true } | VegaBootResult + | HarmonyBootResult | ElectronBootResult | NativeDevtoolsInitFailedResult; function bootTarget(params: BootDeviceParams): string { - return params.udid ?? params.avdName ?? params.vvdImage ?? params.electronAppPath ?? "device"; + return ( + params.udid ?? + params.avdName ?? + params.vvdImage ?? + params.harmonyInstance ?? + params.electronAppPath ?? + "device" + ); } // Flags every boot-device launch should always pass. Two purposes: @@ -1418,12 +1445,112 @@ function bootVega(params: { return promise; } +// A HarmonyOS emulator exposes no serial that can be observed, so its id is +// minted from the instance name — see `HARMONY_ID_PREFIX`. +type HarmonyBootResult = { + platform: "harmony"; + udid: string; + instanceName: string; + booted: true; +}; + +// Coalesce concurrent HarmonyOS boots (mirrors `inFlightVegaBoots`): two callers +// must not both shell out `Emulator -start` for the same instance. Keyed on the +// name alone — unlike Vega there is no `force` mode to keep apart. +const inFlightHarmonyBoots = new Map>(); + +const HARMONY_BOOT_TIMEOUT_MS = 120_000; + +/** + * `Emulator -install` outside mainland China prints "…available only in the + * Chinese mainland" and downloads nothing, and no instance can be created + * without an image. The manager only ever reports the symptom of that — a + * missing instance, or a missing image — so name the cause ourselves. + */ +const HARMONY_IMAGE_RESTRICTION = + "Huawei serves HarmonyOS emulator images only within mainland China; outside it no image can " + + "be downloaded, so no instance can be created and none can be started. argent cannot supply " + + "the image; an instance has to be created in DevEco Studio on a host that can download one."; + +/** + * Said when the start failed and the manager lists no instances at all. Zero + * instances is equally what a host that simply has not created one yet looks + * like — including inside mainland China, where the download works — so this + * states what was observed and offers the restriction as a possible cause + * rather than asserting it the way {@link HARMONY_IMAGE_RESTRICTION} does. + */ +const HARMONY_NO_INSTANCES = + "The emulator manager lists no HarmonyOS instances at all; create one in DevEco Studio before " + + "starting it. If creating one fails for want of an image, note that Huawei serves HarmonyOS " + + "emulator images only within mainland China."; + +async function bootHarmonyImpl(params: { + instanceName: string; + bootTimeoutMs: number; +}): Promise { + await ensureDep("harmony-emulator"); + + // `-start` is the manager's launcher, not the emulator process itself (the + // manager has a separate `-stop ` verb). Whether it returns as soon as + // the instance is handed off or blocks until HarmonyOS is up could not be + // observed, so it gets the caller's whole boot budget instead of the 30 s + // default every other `runHarmonyEmulator` call takes. + const result = await runHarmonyEmulator(["-start", params.instanceName], params.bootTimeoutMs); + // The exit code is no verdict: `-start` on a missing instance exits 1 while + // the manager's other failures exit 0, so the diagnostic it printed is the + // only signal (see `harmony-cli.ts`). + const diagnostic = emulatorFailure(result); + if (diagnostic) { + const imageMissing = + isChinaOnlyRestriction(diagnostic) || diagnostic.includes("Cannot find image"); + // A start that failed because no instance exists is the same wall one step + // earlier, so ask whether any exists at all. `-list` is read only here, on + // the failure path: its populated output shape could not be observed, so + // misreading it must never block a start that would otherwise have worked. + // `null` means the listing itself failed — unknown, so claim nothing. + const instances = imageMissing ? null : await listHarmonyInstances().catch(() => null); + // Only the manager's own words justify blaming the region; an empty instance + // list is merely consistent with it. + const cause = imageMissing + ? ` ${HARMONY_IMAGE_RESTRICTION}` + : instances?.length === 0 + ? ` ${HARMONY_NO_INSTANCES}` + : ""; + throw new Error( + cause + ? `Failed to start HarmonyOS emulator "${params.instanceName}".${cause} The manager reported: ${diagnostic}` + : `Failed to start HarmonyOS emulator "${params.instanceName}": ${diagnostic}` + ); + } + + return { + platform: "harmony", + udid: `${HARMONY_ID_PREFIX}${params.instanceName}`, + instanceName: params.instanceName, + booted: true, + }; +} + +function bootHarmony(params: { + instanceName: string; + bootTimeoutMs: number; +}): Promise { + const existing = inFlightHarmonyBoots.get(params.instanceName); + if (existing) return existing; + const promise = bootHarmonyImpl(params).finally(() => { + inFlightHarmonyBoots.delete(params.instanceName); + }); + inFlightHarmonyBoots.set(params.instanceName, promise); + return promise; +} + const capability: ToolCapability = { apple: { simulator: true, device: true }, appleRemote: { simulator: true }, android: { emulator: true, device: true, unknown: true }, chromium: { app: true }, vega: { vvd: true }, + harmony: { emulator: true }, }; export function createBootDeviceTool( @@ -1437,14 +1564,15 @@ export function createBootDeviceTool( failedMsg: ({ params, failureSignal }) => `Failed to start ${bootTarget(params)}: ${failureSignal.error_code}`, }, - description: `Start an iOS simulator, launch an Android emulator, start a Vega (Fire TV) Virtual Device, or spawn an Electron app and wait until it is ready to accept interactions. -Pick the platform by which argument you pass: 'udid' for an iOS simulator from list-devices, 'avdName' for an Android AVD (a serial is assigned automatically), 'vvdImage' for a Vega VVD (the 'vvdImage' of a vega device from list-devices, e.g. 'tv'), or 'electronAppPath' for an Electron app (a CDP remote-debugging port is picked automatically, or pass 'electronPort' to fix one). + description: `Start an iOS simulator, launch an Android emulator, start a Vega (Fire TV) Virtual Device, start a HarmonyOS emulator, or spawn an Electron app and wait until it is ready to accept interactions. +Pick the platform by which argument you pass: 'udid' for an iOS simulator from list-devices, 'avdName' for an Android AVD (a serial is assigned automatically), 'vvdImage' for a Vega VVD (the 'vvdImage' of a vega device from list-devices, e.g. 'tv'), 'harmonyInstance' for a HarmonyOS emulator instance (the 'name' of a harmony device from list-devices), or 'electronAppPath' for an Electron app (a CDP remote-debugging port is picked automatically, or pass 'electronPort' to fix one). Use at the start of a session once you have picked a target. -Returns a tagged payload: { platform: 'ios', udid, booted } or { platform: 'android', serial, avdName, booted } or { platform: 'vega', serial, vvdImage, booted } or { platform: 'chromium', id, port, pid, booted } (an Electron app boots as a Chromium/CDP device). -Android boots take 2–10 minutes depending on machine and cold/warm state; the tool transparently hot-boots from the AVD's default_boot snapshot when usable and falls back to cold boot otherwise. Vega starts the single SDK-managed VVD via the vega CLI (~10s) and returns once it reports running. If an Android/Electron boot stage fails, the tool terminates the device it spawned so the next retry starts clean.`, +Returns a tagged payload: { platform: 'ios', udid, booted } or { platform: 'android', serial, avdName, booted } or { platform: 'vega', serial, vvdImage, booted } or { platform: 'harmony', udid, instanceName, booted } or { platform: 'chromium', id, port, pid, booted } (an Electron app boots as a Chromium/CDP device). +Android boots take 2–10 minutes depending on machine and cold/warm state; the tool transparently hot-boots from the AVD's default_boot snapshot when usable and falls back to cold boot otherwise. Vega starts the single SDK-managed VVD via the vega CLI (~10s) and returns once it reports running. If an Android/Electron boot stage fails, the tool terminates the device it spawned so the next retry starts clean. +HarmonyOS hands the instance to DevEco Studio's Emulator manager, which reports no readiness signal — the call returns once the manager accepts the start, not once the OS is up, and no interaction tool drives a HarmonyOS device yet. Huawei serves the emulator images only within mainland China, so on a host outside it no instance can exist to start.`, alwaysLoad: true, searchHint: - "boot start launch simulator emulator avd device session ios android vega vvd firetv cold hot", + "boot start launch simulator emulator avd device session ios android vega vvd firetv harmony harmonyos deveco cold hot", zodSchema, capability, services: () => ({}), @@ -1452,11 +1580,12 @@ Android boots take 2–10 minutes depending on machine and cold/warm state; the const hasUdid = Boolean(params.udid); const hasAvd = Boolean(params.avdName); const hasVega = Boolean(params.vvdImage); + const hasHarmony = Boolean(params.harmonyInstance); const hasElectron = Boolean(params.electronAppPath); - const provided = [hasUdid, hasAvd, hasVega, hasElectron].filter(Boolean).length; + const provided = [hasUdid, hasAvd, hasVega, hasHarmony, hasElectron].filter(Boolean).length; if (provided !== 1) { throw new FailureError( - "Provide exactly one of `udid` (iOS), `avdName` (Android), `vvdImage` (Vega VVD), or `electronAppPath` (Electron).", + "Provide exactly one of `udid` (iOS), `avdName` (Android), `vvdImage` (Vega VVD), `harmonyInstance` (HarmonyOS), or `electronAppPath` (Electron).", { error_code: FAILURE_CODES.BOOT_DEVICE_TARGET_SELECTION_INVALID, failure_stage: "boot_device_target_selection", @@ -1466,9 +1595,20 @@ Android boots take 2–10 minutes depending on machine and cold/warm state; the ); } if (hasUdid) { - if (classifyDevice(params.udid!) === "ios-remote") { + const platform = classifyDevice(params.udid!); + if (platform === "ios-remote") { return bootIosRemote(params.udid!, registry, params.force); } + // `list-devices` reports a HarmonyOS instance's id as `harmony-` + // and the capability gate lets that id through, so route it to the + // emulator manager rather than to simctl, which would reject it as an + // unknown iOS udid. + if (platform === "harmony") { + return bootHarmony({ + instanceName: harmonyInstanceName(params.udid!), + bootTimeoutMs: params.bootTimeoutMs ?? HARMONY_BOOT_TIMEOUT_MS, + }); + } return bootIos(params.udid!, registry, params.force, params.headless); } if (hasAvd) { @@ -1485,6 +1625,12 @@ Android boots take 2–10 minutes depending on machine and cold/warm state; the force: params.force, }); } + if (hasHarmony) { + return bootHarmony({ + instanceName: harmonyInstanceName(params.harmonyInstance!), + bootTimeoutMs: params.bootTimeoutMs ?? HARMONY_BOOT_TIMEOUT_MS, + }); + } return bootElectronApp({ appPath: params.electronAppPath!, port: params.electronPort, diff --git a/packages/tool-server/src/tools/devices/list-devices.ts b/packages/tool-server/src/tools/devices/list-devices.ts index 166bda09b0..388ff7add3 100644 --- a/packages/tool-server/src/tools/devices/list-devices.ts +++ b/packages/tool-server/src/tools/devices/list-devices.ts @@ -9,7 +9,8 @@ import { import { listRunningVvdConsolePorts } from "../../utils/vega-process"; import { listIosSimulators, type IosSimulator } from "../../utils/ios-devices"; import { simctlListDevices } from "../../utils/sim-remote"; -import { withRemotePrefix } from "../../utils/device-info"; +import { withRemotePrefix, HARMONY_ID_PREFIX } from "../../utils/device-info"; +import { listHarmonyInstances } from "../../utils/harmony-devices"; import { discoverChromiumDevices, type ChromiumDevice } from "../../utils/chromium-discovery"; import { listVegaDevices, @@ -42,8 +43,26 @@ type AndroidDevice = { runtimeKind?: "mobile" | "tv"; }; +/** + * A HarmonyOS emulator instance reported by DevEco Studio's `Emulator` manager. + * + * `state` is always "unknown": nothing here determines whether an instance is + * running. Huawei restricts the emulator image download to mainland China, so no + * instance could be created and the manager's populated output was never + * observed, leaving no verified way to read readiness. Claiming "stopped" would + * be a guess; "unknown" is the fact. + */ +type HarmonyDevice = { + platform: "harmony"; + udid: string; + name: string; + state: "unknown"; +}; + type ListDevicesResult = { - devices: Array; + devices: Array< + IosDevice | IosRemoteDevice | AndroidDevice | ChromiumDevice | VegaDevice | HarmonyDevice + >; avds: Array<{ name: string }>; }; @@ -68,11 +87,12 @@ function sortAndroid(a: AndroidDevice, b: AndroidDevice): number { // Float booted/ready devices to the top of the merged list regardless of // platform — without this, all iOS entries are emitted before any Android. function readinessRank( - d: IosDevice | IosRemoteDevice | AndroidDevice | ChromiumDevice | VegaDevice + d: IosDevice | IosRemoteDevice | AndroidDevice | ChromiumDevice | VegaDevice | HarmonyDevice ): number { if (d.platform === "android") return d.state === "device" ? 0 : 1; if (d.platform === "vega") return d.state === "running" || d.state === "device" ? 0 : 1; if (d.platform === "chromium") return 0; // Chromium entries are only listed when their CDP is responsive + if (d.platform === "harmony") return 1; // readiness is never determined — rank below anything confirmed ready return d.state === "Booted" ? 0 : 1; // ios + ios-remote } @@ -201,17 +221,18 @@ export const listDevicesTool: ToolDefinition, ListDevicesR }, failedMsg: ({ failureSignal }) => `Failed to list devices: ${failureSignal.error_code}`, }, - description: `List iOS simulators, Android emulators, connected physical Android devices, running Chromium apps, and Vega (Fire TV) devices in one place. -Use at the start of a session to pick a target id ('udid' for iOS entries, 'serial' for Android/Vega entries, 'id' for Chromium) to pass to interaction tools, and to see which targets are already running. -Returns { devices, avds } where each device carries a 'platform' discriminator ('ios', 'android', 'chromium', or 'vega'); 'avds' lists Android AVDs bootable via boot-device. A Vega VVD is listed under 'devices' whether running or stopped (state 'running'/'stopped'); start a stopped one with boot-device using its 'vvdImage'. + description: `List iOS simulators, Android emulators, connected physical Android devices, running Chromium apps, Vega (Fire TV) devices, and HarmonyOS emulators in one place. +Use at the start of a session to pick a target id ('udid' for iOS/HarmonyOS entries, 'serial' for Android/Vega entries, 'id' for Chromium) to pass to interaction tools, and to see which targets are already running. +Returns { devices, avds } where each device carries a 'platform' discriminator ('ios', 'android', 'chromium', 'vega', or 'harmony'); 'avds' lists Android AVDs bootable via boot-device. A Vega VVD is listed under 'devices' whether running or stopped (state 'running'/'stopped'); start a stopped one with boot-device using its 'vvdImage'. +HarmonyOS emulator instances come from DevEco Studio's emulator manager and always carry state 'unknown' — Argent does not determine whether an instance is running. No interaction tool drives a HarmonyOS device yet, so they are listed for discovery and boot-device only; anything else returns 501. Android entries also carry a 'kind' ('emulator' for a local AVD, 'device' for a physical phone connected over USB / wireless adb) — physical phones are detected from \`adb devices\` (any serial that is not an \`emulator-*\` one) and are driven through the same interaction tools as emulators; they do not need boot-device (just connect the phone with USB debugging authorised). TV targets are tagged with runtimeKind 'tv' (Apple TV simulators on iOS, Android TV / leanback devices on Android) — these are focus-driven, not touch-driven: use \`describe\` to read focus, \`tv-remote\` for remote presses (up/down/left/right/select/back/menu/home), and \`keyboard\` to type, rather than the coordinate/gesture tools. iOS simulators from an additional CoreSimulator device set (the 'ios.additionalDeviceSets' configuration — e.g. devices created by Radon IDE) are listed alongside default-set ones, tagged with their owning 'deviceSet' path; they are driven through the same tools by udid, but run headless (no Simulator.app window attaches to them). Chromium apps are discovered by probing CDP debugging ports (default 9222; extend via the ARGENT_CHROMIUM_PORTS= env var). They must already be running with --remote-debugging-port= — use boot-device with electronAppPath to launch one. -Booted/ready devices are listed first. Platforms whose CLI is unavailable are silently omitted — an empty result usually means xcode-select, Android platform-tools, or the Vega SDK is not installed.`, +Booted/ready devices are listed first. Platforms whose CLI is unavailable are silently omitted — an empty result usually means xcode-select, Android platform-tools, the Vega SDK, or DevEco Studio is not installed.`, alwaysLoad: true, searchHint: - "list devices simulators emulators avd serial udid ios android chromium vega app fire tv session start", + "list devices simulators emulators avd serial udid ios android chromium vega harmony harmonyos app fire tv session start", zodSchema, services: () => ({}), async execute(_services, _params) { @@ -223,7 +244,7 @@ Booted/ready devices are listed first. Platforms whose CLI is unavailable are si // timer is cleared on the fast happy path). The deadline only substitutes a // fallback on *slowness*; a rejection still propagates exactly as before — so the // `.catch(() => [])` wrappers (and the lack of one on iOS/AVDs) are unchanged. - const [ios, iosRemote, android, avds, chromium, vega] = await Promise.all([ + const [ios, iosRemote, android, avds, chromium, vega, harmony] = await Promise.all([ withDeadline(listIosSimulators(), [], "ios"), withDeadline(listRemoteIosSimulators(), [], "ios-remote"), withDeadline( @@ -249,6 +270,11 @@ Booted/ready devices are listed first. Platforms whose CLI is unavailable are si [], "vega" ), + withDeadline( + listHarmonyInstances().catch(() => []), + [], + "harmony" + ), ]); const iosTagged: IosDevice[] = ios.map((s) => ({ platform: "ios", ...s })); iosTagged.sort(sortIos); @@ -269,9 +295,16 @@ Booted/ready devices are listed first. Platforms whose CLI is unavailable are si const androidDeduped = filterVvdShadowsFromAndroid(androidTagged, vvdShadowSerials); androidDeduped.sort(sortAndroid); + const harmonyTagged: HarmonyDevice[] = harmony.map((h) => ({ + platform: "harmony", + udid: `${HARMONY_ID_PREFIX}${h.name}`, + name: h.name, + state: "unknown", + })); + const devices: Array< - IosDevice | IosRemoteDevice | AndroidDevice | ChromiumDevice | VegaDevice - > = [...iosTagged, ...iosRemote, ...androidDeduped, ...chromium, ...vega]; + IosDevice | IosRemoteDevice | AndroidDevice | ChromiumDevice | VegaDevice | HarmonyDevice + > = [...iosTagged, ...iosRemote, ...androidDeduped, ...chromium, ...vega, ...harmonyTagged]; devices.sort((a, b) => readinessRank(a) - readinessRank(b)); return { devices, avds }; diff --git a/packages/tool-server/src/utils/capability.ts b/packages/tool-server/src/utils/capability.ts index 9afffc4899..bfb79e2172 100644 --- a/packages/tool-server/src/utils/capability.ts +++ b/packages/tool-server/src/utils/capability.ts @@ -127,6 +127,8 @@ function platformMatrix( return capability.chromium; case "vega": return capability.vega; + case "harmony": + return capability.harmony; } } diff --git a/packages/tool-server/src/utils/check-deps.ts b/packages/tool-server/src/utils/check-deps.ts index ba54c98395..4eebcbeafc 100644 --- a/packages/tool-server/src/utils/check-deps.ts +++ b/packages/tool-server/src/utils/check-deps.ts @@ -1,6 +1,7 @@ import { FAILURE_CODES, withFailureSignal, type ToolDependency } from "@argent/registry"; import { resolveAndroidBinary } from "./android-binary"; import { resolveVegaBinary } from "./vega-cli"; +import { resolveHarmonyEmulator } from "./harmony-cli"; import { commandOnPath } from "./command-on-path"; /** @@ -45,6 +46,8 @@ const INSTALL_HINTS: Record = { "`sim-remote` CLI not found on PATH. Install via the radon-cloud project (see its README) and run `sim-remote login` before invoking any ios-remote tool. Only required for remote iOS simulators.", "vega": "Vega SDK CLI not found. Install the Amazon Vega SDK and run `source ~/vega/env` so `vega` (or its `kepler` alias) is on PATH; the resolver also checks `~/vega/bin/vega`. Only required for Vega (Fire TV) devices.", + "harmony-emulator": + "HarmonyOS emulator manager not found. Install DevEco Studio; on a non-macOS host set `$DEVECO_STUDIO_HOME` to its install root. Only required to list or launch HarmonyOS emulators.", }; async function probe(dep: ToolDependency): Promise { @@ -63,6 +66,11 @@ async function probe(dep: ToolDependency): Promise { if (dep === "vega") { return (await resolveVegaBinary()) !== null; } + // The emulator manager lives inside DevEco Studio rather than on PATH, so it + // resolves through the DevEco-aware resolver. + if (dep === "harmony-emulator") { + return (await resolveHarmonyEmulator()) !== null; + } // `commandOnPath` probes existence without invoking the dep itself — a bare // `xcrun` call would fork the tool just to check existence, which is both // slower and (for xcrun) can prompt the license agreement dialog on first diff --git a/packages/tool-server/src/utils/cross-platform-tool.ts b/packages/tool-server/src/utils/cross-platform-tool.ts index fa8acc2019..f339365db6 100644 --- a/packages/tool-server/src/utils/cross-platform-tool.ts +++ b/packages/tool-server/src/utils/cross-platform-tool.ts @@ -131,6 +131,12 @@ export function dispatchByPlatform< } return opts.vega.handler(services as unknown as VegaServices, params, device, invokeOptions); } + if (device.platform === "harmony") { + // No tool drives a HarmonyOS device yet, so there is no branch to call — + // but the check has to be explicit, because chromium is the fallthrough + // below and a harmony device would otherwise be handed to the CDP branch. + throw new NotImplementedOnPlatformError({ toolId: opts.toolId, platform: "harmony" }); + } // chromium if (!opts.chromium) { throw new NotImplementedOnPlatformError({ diff --git a/packages/tool-server/src/utils/device-info.ts b/packages/tool-server/src/utils/device-info.ts index 89994d38a3..07d2fe3577 100644 --- a/packages/tool-server/src/utils/device-info.ts +++ b/packages/tool-server/src/utils/device-info.ts @@ -44,11 +44,31 @@ export const CHROMIUM_ID_PREFIX = "chromium-cdp-"; */ export const VEGA_SERIAL_PREFIX = "amazon-"; +/** + * HarmonyOS device-id prefix, applied by `list-devices` to an emulator instance + * name (`harmony-Phone_1`). + * + * Unlike the other platforms this is not a vendor-assigned serial shape: the + * `hdc` connect key of a running HarmonyOS emulator could not be observed, since + * Huawei restricts the emulator image download to mainland China and no instance + * can exist without one. Minting the id from the instance name — which the + * `Emulator` CLI does report — keeps classification exact and avoids encoding a + * guessed serial format, the same way `chromium-cdp-` is argent's own id + * rather than something CDP hands out. + */ +export const HARMONY_ID_PREFIX = "harmony-"; + +/** Strip the `harmony-` prefix, returning the bare emulator instance name. */ +export function harmonyInstanceName(udid: string): string { + return udid.startsWith(HARMONY_ID_PREFIX) ? udid.slice(HARMONY_ID_PREFIX.length) : udid; +} + /** Returns the platform a `udid` belongs to based on its shape. */ export function classifyDevice(udid: string): Platform { if (udid.startsWith(REMOTE_PREFIX)) return "ios-remote"; if (udid.startsWith(VEGA_SERIAL_PREFIX)) return "vega"; if (udid.startsWith(CHROMIUM_ID_PREFIX)) return "chromium"; + if (udid.startsWith(HARMONY_ID_PREFIX)) return "harmony"; return IOS_UDID_SHAPE.test(udid) ? "ios" : "android"; } @@ -90,11 +110,13 @@ export function resolveDevice(udid: string): DeviceInfo { ? "simulator" : platform === "vega" ? "vvd" - : platform === "android" - ? isAndroidEmulatorSerial(udid) - ? "emulator" - : "device" - : "app"; + : platform === "harmony" + ? "emulator" + : platform === "android" + ? isAndroidEmulatorSerial(udid) + ? "emulator" + : "device" + : "app"; return { id: udid, platform, kind }; } diff --git a/packages/tool-server/src/utils/harmony-cli.ts b/packages/tool-server/src/utils/harmony-cli.ts new file mode 100644 index 0000000000..81d2f67d11 --- /dev/null +++ b/packages/tool-server/src/utils/harmony-cli.ts @@ -0,0 +1,174 @@ +import { execFile } from "node:child_process"; +import { promisify } from "node:util"; +import { access, constants as fsConstants } from "node:fs/promises"; +import { join } from "node:path"; +import { formatSubprocessFailure } from "./subprocess-error"; + +const execFileAsync = promisify(execFile); + +/** + * Wrapper for DevEco Studio's `Emulator` manager — HarmonyOS' `emulator` and + * `avdmanager` rolled into one (`-list` / `-create` / `-start` / `-stop` / + * `-imageList` / `-install`). + * + * It violates the convention every other subprocess wrapper here relies on: the + * exit code does not indicate success. Measured against DevEco Studio 6.1 + * (Emulator 6.1.1.200): + * + * -list (none) exit 0 ok + * -create (no image) exit 0 FAILED + * -stop (missing) exit 0 FAILED + * -install (outside China) exit 0 FAILED + * -start (missing) exit 1 FAILED + * + * So a failure is usually exit 0, but not always — `-start` is the outlier. Since + * the code is unreliable in both directions, `runHarmonyEmulator` deliberately + * does NOT reject on a non-zero exit: it returns the child's output either way + * and leaves the verdict to `emulatorFailure`, which reads stdout. That keeps one + * classification path instead of two that disagree. + * + * `hdc`, the HarmonyOS device connector, is deliberately absent: nothing here + * talks to a booted device yet, so wrapping it would add an unused binary + * dependency. It belongs with the interaction layer. + */ + +/** The manager prints this exact token for an empty list rather than no output. */ +export const HARMONY_EMPTY_SENTINEL = "[Empty]"; + +/** + * DevEco Studio's macOS install root. Non-macOS hosts (DevEco also ships for + * Windows) are supported through `$DEVECO_STUDIO_HOME` rather than a second + * hardcoded root, because only the macOS layout has been verified here. + */ +const MACOS_DEVECO_ROOT = "/Applications/DevEco-Studio.app/Contents"; + +/** Path of the emulator manager relative to a DevEco Studio install root. */ +const EMULATOR_RELATIVE = join("tools", "emulator", "Emulator"); + +// Mirrors android-binary.ts / vega-cli.ts: memoize briefly so a burst of tool +// calls pays one lookup, but a *negative* result expires — a user who installs +// DevEco Studio mid-session recovers without restarting the tool-server. +const BINARY_TTL_MS = 60_000; +let cachedEmulator: { path: string | null; checkedAt: number } | undefined; + +// X_OK rather than F_OK (as in vega-cli.ts): a present-but-non-executable file +// at the DevEco path is a partial install, and returning it would surface as an +// opaque EACCES at spawn instead of the actionable not-found hint. +async function isExecutable(p: string): Promise { + try { + await access(p, fsConstants.X_OK); + return true; + } catch { + return false; + } +} + +/** + * Absolute path to the HarmonyOS emulator manager, or null when DevEco Studio + * isn't installed. Not resolved from PATH: the binary is named `Emulator`, too + * generic a name to match on PATH without risking an unrelated executable. + */ +export async function resolveHarmonyEmulator(): Promise { + const now = Date.now(); + if (cachedEmulator && now - cachedEmulator.checkedAt < BINARY_TTL_MS) { + return cachedEmulator.path; + } + const configured = process.env.DEVECO_STUDIO_HOME?.trim(); + const root = configured || (process.platform === "darwin" ? MACOS_DEVECO_ROOT : null); + let path: string | null = null; + if (root) { + const candidate = join(root, EMULATOR_RELATIVE); + if (await isExecutable(candidate)) path = candidate; + } + cachedEmulator = { path, checkedAt: now }; + return path; +} + +/** Test-only: clear the binary-resolution memo. */ +export function __resetHarmonyBinaryCacheForTests(): void { + cachedEmulator = undefined; +} + +export interface HarmonyRunResult { + stdout: string; + stderr: string; +} + +const DEFAULT_TIMEOUT_MS = 30_000; + +export async function runHarmonyEmulator( + args: string[], + timeoutMs = DEFAULT_TIMEOUT_MS +): Promise { + const bin = await resolveHarmonyEmulator(); + if (!bin) { + throw new Error( + "The HarmonyOS `Emulator` manager was not found. Install DevEco Studio, or set " + + "`$DEVECO_STUDIO_HOME` to its install root, then retry." + ); + } + try { + const { stdout, stderr } = await execFileAsync(bin, args, { + timeout: timeoutMs, + maxBuffer: 8 * 1024 * 1024, + }); + return { stdout, stderr }; + } catch (err) { + const e = err as { killed?: boolean; code?: unknown; stdout?: string; stderr?: string }; + // The child ran and exited non-zero (`-start` on a missing instance does + // this) — its diagnostic is on stdout, so hand the output back and let the + // caller classify it exactly as it would an exit-0 failure. A numeric `code` + // is what distinguishes this from a spawn error, whose `code` is a string + // like ENOENT. + if (!e.killed && typeof e.code === "number" && (e.stdout != null || e.stderr != null)) { + return { stdout: e.stdout ?? "", stderr: e.stderr ?? "" }; + } + // Spawn failure or timeout SIGKILL: no diagnostic to classify, so surface it + // the way every other subprocess wrapper here does. + throw new Error(formatSubprocessFailure("Emulator", args, err), { cause: err }); + } +} + +/** + * Emulator-image downloads are restricted by Huawei to mainland China; outside it + * `Emulator -install` prints exactly this and exits 0. Without an image no + * instance can be created, so this is the wall every non-China host hits — worth + * naming precisely instead of reporting the generic create failure it causes. + */ +const CHINA_ONLY_MARKER = "available only in the Chinese mainland"; + +/** + * Verified `Emulator` failure diagnostics, each observed on a real invocation. + * + * Ordered most-specific first, because a failure often prints two lines and the + * first match wins. `-start` on a missing instance prints both `"" is not + * found. Please create the device(folder): ` and a bare `Unable to start + * the emulator`; the naming line is the one that tells the caller what to do, so + * the generic trailers sit at the bottom. + */ +const EMULATOR_FAILURE_MARKERS = [ + CHINA_ONLY_MARKER, + "Cannot find image", + "is not found. Please create the device", + "failed, emulator is not exists", + "Device create fail", + "Unable to start the emulator", +] as const; + +/** + * The diagnostic the emulator manager printed, or null if the call succeeded. + * Matched against the verified marker list rather than a bare "fail" substring, + * so an instance or image name containing "fail" cannot forge a failure. + */ +export function emulatorFailure(result: HarmonyRunResult): string | null { + const text = `${result.stdout}\n${result.stderr}`; + const marker = EMULATOR_FAILURE_MARKERS.find((m) => text.includes(m)); + if (!marker) return null; + const line = text.split(/\r?\n/).find((l) => l.includes(marker)); + return (line ?? marker).trim(); +} + +/** True when the diagnostic is Huawei's mainland-China image-download restriction. */ +export function isChinaOnlyRestriction(diagnostic: string): boolean { + return diagnostic.includes(CHINA_ONLY_MARKER); +} diff --git a/packages/tool-server/src/utils/harmony-devices.ts b/packages/tool-server/src/utils/harmony-devices.ts new file mode 100644 index 0000000000..01d0425bfe --- /dev/null +++ b/packages/tool-server/src/utils/harmony-devices.ts @@ -0,0 +1,53 @@ +import { + HARMONY_EMPTY_SENTINEL, + emulatorFailure, + resolveHarmonyEmulator, + runHarmonyEmulator, +} from "./harmony-cli"; + +/** + * Discovery for the HarmonyOS platform: the emulator instances DevEco Studio's + * `Emulator` manager knows about. + * + * Readers here treat `[Empty]` as "none" and a recognised diagnostic as a + * failure, because the exit code says neither (see `harmony-cli.ts`). + */ + +/** A HarmonyOS emulator instance as reported by `Emulator -list`. */ +export interface HarmonyInstance { + name: string; +} + +/** + * Lines the emulator manager prints alongside a listing that are not instance + * names. `Config file not found: …` is emitted per malformed instance directory + * *before* the `[Empty]`/listing body, so it would otherwise be read as a name. + */ +function isDiagnosticLine(line: string): boolean { + return line.startsWith("Config file not found:") || line.startsWith("["); +} + +/** + * Instance names from `Emulator -list`. + * + * Only the empty case (`[Empty]`) could be verified on a host outside mainland + * China, because Huawei restricts the image download that creating an instance + * requires — so the populated case is parsed conservatively as one name per + * line, with known diagnostics filtered out, and anything unrecognised simply + * yields no instances rather than a bogus one. + */ +export function parseHarmonyInstances(stdout: string): HarmonyInstance[] { + return stdout + .split(/\r?\n/) + .map((l) => l.trim()) + .filter((l) => l.length > 0 && l !== HARMONY_EMPTY_SENTINEL && !isDiagnosticLine(l)) + .map((name) => ({ name })); +} + +/** Emulator instances, or [] when DevEco Studio isn't installed. */ +export async function listHarmonyInstances(): Promise { + if (!(await resolveHarmonyEmulator())) return []; + const result = await runHarmonyEmulator(["-list"]); + if (emulatorFailure(result)) return []; + return parseHarmonyInstances(result.stdout); +} diff --git a/packages/tool-server/test/cross-platform-tool.test.ts b/packages/tool-server/test/cross-platform-tool.test.ts index 6ef2f85036..56f3cf1dc0 100644 --- a/packages/tool-server/test/cross-platform-tool.test.ts +++ b/packages/tool-server/test/cross-platform-tool.test.ts @@ -181,4 +181,31 @@ describe("dispatchByPlatform", () => { await expect(a({}, { udid: iosUdid })).resolves.toBe("ok"); await expect(a({}, { udid: androidUdid })).resolves.toBe("ok"); }); + + // Chromium is the fallthrough at the end of the dispatch chain, so a platform + // with no branch of its own lands there instead of erroring. Deleting the + // explicit harmony check would hand a HarmonyOS device to the CDP handler and + // report its result as success, which is why this asserts the chromium handler + // is never entered rather than only that the call rejects. + it("rejects a HarmonyOS device instead of falling through to the chromium handler", async () => { + const chromiumHandler = vi.fn().mockResolvedValue("from-chromium"); + + const execute = dispatchByPlatform< + Record, + Record, + { udid: string }, + string + >({ + toolId: "test", + capability: { ...capabilityBoth, harmony: { emulator: true }, chromium: { app: true } }, + ios: { handler: vi.fn() }, + android: { handler: vi.fn() }, + chromium: { handler: chromiumHandler }, + }); + + await expect(execute({}, { udid: "harmony-Phone_1" })).rejects.toThrow( + /not implemented .*harmony|harmony/i + ); + expect(chromiumHandler).not.toHaveBeenCalled(); + }); }); diff --git a/packages/tool-server/test/harmony-cli.test.ts b/packages/tool-server/test/harmony-cli.test.ts new file mode 100644 index 0000000000..1f8133c73c --- /dev/null +++ b/packages/tool-server/test/harmony-cli.test.ts @@ -0,0 +1,133 @@ +import { describe, it, expect } from "vitest"; +import { + emulatorFailure, + isChinaOnlyRestriction, + type HarmonyRunResult, +} from "../src/utils/harmony-cli"; + +/** + * Every fixture here is verbatim stdout captured from DevEco Studio 6.1's + * `Emulator` manager on macOS (Emulator 6.1.1.200), together with the exit code + * measured alongside it. Those codes are the reason this classifier exists: a + * failure is usually exit 0 (`-create`, `-stop`, `-install`) but not always + * (`-start` exits 1), so the exit code is unreliable in both directions and the + * verdict has to come from stdout. + */ + +/** `Emulator -list` with no instances deployed. Exit 0. */ +const EMULATOR_EMPTY = "[Empty]\n"; + +/** `Emulator -create testinst -deviceType phone -osVersion "…"` with no image. Exit 0. */ +const EMULATOR_CREATE_NO_IMAGE = `Cannot find image, please verify your SDK installation. + +Device create fail. +`; + +/** `Emulator -start nosuchinstance`. Exit 1 — the one failure that does not exit 0. */ +const EMULATOR_START_MISSING = `"nosuchinstance" is not found. Please create the device(folder): "/Users/ignacylatka/.Huawei/Emulator/deployed/nosuchinstance" +Unable to start the emulator +`; + +/** `Emulator -stop nosuchinstance`. Exit 0. */ +const EMULATOR_STOP_MISSING = `Stop emulator "nosuchinstance" failed, emulator is not exists. +`; + +/** `Emulator -install -deviceType phone -osVersion "…"` outside mainland China. Exit 0. */ +const EMULATOR_CHINA_ONLY = + "Currently, this capability is available only in the Chinese mainland.\n"; + +/** `Emulator -imageList -downloaded true` with nothing downloaded. Exit 0, and not a failure. */ +const EMULATOR_NO_MATCHING_IMAGES = "No images matching the criteria were found.\n"; + +function result(stdout: string, stderr = ""): HarmonyRunResult { + return { stdout, stderr }; +} + +describe("emulatorFailure", () => { + it("returns null for the empty-instance-list sentinel", () => { + expect(emulatorFailure(result(EMULATOR_EMPTY))).toBeNull(); + }); + + it("does not fire on a listing whose instance name merely contains 'fail'", () => { + // Pins the "verified markers, not a bare `fail` substring" design: an + // instance the user named `failsafe_phone` must still list as an instance. + expect(emulatorFailure(result("failsafe_phone\nPhone_1\n"))).toBeNull(); + }); + + it("does not fire on an image query that simply matched nothing", () => { + expect(emulatorFailure(result(EMULATOR_NO_MATCHING_IMAGES))).toBeNull(); + }); + + it("finds a diagnostic printed on stderr rather than stdout", () => { + expect(emulatorFailure(result("", EMULATOR_CHINA_ONLY))).toBe( + "Currently, this capability is available only in the Chinese mainland." + ); + }); +}); + +describe("failure is read from stdout, whatever the exit code was", () => { + // Branching on the exit status misclassifies these in both directions: the + // exit-0 rows would read as success, and were `-start`'s exit 1 taken as the + // signal, an exit-0 `-create` failure would still slip through. + it.each([ + { + label: "`-create` with no image downloaded (exit 0)", + output: EMULATOR_CREATE_NO_IMAGE, + diagnostic: "Cannot find image, please verify your SDK installation.", + }, + { + // Second line of the `-create` output above, on its own: the marker is + // otherwise shadowed by "Cannot find image" and nothing would pin it. + label: "`-create` reporting only the trailing create-fail line (exit 0)", + output: "Device create fail.\n", + diagnostic: "Device create fail.", + }, + { + // This output carries two diagnostics — the naming line and a bare + // "Unable to start the emulator". Marker order has to pick the naming + // line, since it is the one that says what to do about it. + label: "`-start` on an instance that does not exist (exit 1)", + output: EMULATOR_START_MISSING, + diagnostic: + '"nosuchinstance" is not found. Please create the device(folder): "/Users/ignacylatka/.Huawei/Emulator/deployed/nosuchinstance"', + }, + { + // The bare trailer on its own: otherwise shadowed by the naming line + // above, so nothing would pin this marker. + label: "`-start` reporting only the trailing unable-to-start line (exit 1)", + output: "Unable to start the emulator\n", + diagnostic: "Unable to start the emulator", + }, + { + label: "`-stop` on an instance that does not exist (exit 0)", + output: EMULATOR_STOP_MISSING, + diagnostic: 'Stop emulator "nosuchinstance" failed, emulator is not exists.', + }, + { + label: "`-install` outside mainland China (exit 0)", + output: EMULATOR_CHINA_ONLY, + diagnostic: "Currently, this capability is available only in the Chinese mainland.", + }, + ])("Emulator $label", ({ output, diagnostic }) => { + expect(emulatorFailure(result(output))).toBe(diagnostic); + }); +}); + +describe("isChinaOnlyRestriction", () => { + it("is true for the mainland-China image-download restriction", () => { + const diagnostic = emulatorFailure(result(EMULATOR_CHINA_ONLY)); + expect(diagnostic).not.toBeNull(); + expect(isChinaOnlyRestriction(diagnostic as string)).toBe(true); + }); + + it("is false for every other verified diagnostic", () => { + expect(isChinaOnlyRestriction("Cannot find image, please verify your SDK installation.")).toBe( + false + ); + expect(isChinaOnlyRestriction("Device create fail.")).toBe(false); + expect(isChinaOnlyRestriction("Unable to start the emulator")).toBe(false); + expect( + isChinaOnlyRestriction('Stop emulator "nosuchinstance" failed, emulator is not exists.') + ).toBe(false); + }); +}); diff --git a/packages/tool-server/test/harmony-devices.test.ts b/packages/tool-server/test/harmony-devices.test.ts new file mode 100644 index 0000000000..173f27af18 --- /dev/null +++ b/packages/tool-server/test/harmony-devices.test.ts @@ -0,0 +1,84 @@ +import { describe, it, expect } from "vitest"; +import { parseHarmonyInstances } from "../src/utils/harmony-devices"; +import { + HARMONY_ID_PREFIX, + classifyDevice, + harmonyInstanceName, + resolveDevice, +} from "../src/utils/device-info"; + +/** + * Fixtures are verbatim stdout from DevEco Studio 6.1's `Emulator` manager on + * macOS (Emulator 6.1.1.200). + */ + +/** `Emulator -list` with nothing deployed. */ +const EMULATOR_NO_INSTANCES = "[Empty]\n"; + +/** + * `Emulator -list` prints this once per directory under ~/.Huawei/Emulator/deployed/ + * that lacks a config.ini, ahead of the listing body. + */ +const EMULATOR_CONFIG_NOT_FOUND = + 'Config file not found: "/Users/ignacylatka/.Huawei/Emulator/deployed/probe_instance/config.ini"\n'; + +/** The full `Emulator -list` output observed with one such directory and no instances. */ +const EMULATOR_LIST_MISSING_CONFIG = `${EMULATOR_CONFIG_NOT_FOUND}[Empty]\n`; + +/** `Emulator -imageList`, abridged to two entries. Note `downloaded` is the *string* "false". */ +/** Same shape with `downloaded` flipped to the string "true". */ +/** `Emulator -imageList -downloaded true` with nothing downloaded — plain prose, not `[Empty]`. */ +describe("parseHarmonyInstances", () => { + it("returns no instances for the empty sentinel", () => { + expect(parseHarmonyInstances(EMULATOR_NO_INSTANCES)).toEqual([]); + }); + + it("does not read the `Config file not found:` diagnostic as an instance name", () => { + expect(parseHarmonyInstances(EMULATOR_LIST_MISSING_CONFIG)).toEqual([]); + }); + + it("returns a plain name line as an instance", () => { + expect(parseHarmonyInstances("Phone_1\n")).toEqual([{ name: "Phone_1" }]); + }); + + it("keeps the real instances listed after a config diagnostic", () => { + expect(parseHarmonyInstances(`${EMULATOR_CONFIG_NOT_FOUND}Phone_1\nTv_1\n`)).toEqual([ + { name: "Phone_1" }, + { name: "Tv_1" }, + ]); + }); +}); + +describe("HarmonyOS device ids", () => { + it("classifies a harmony- prefixed id as harmony", () => { + expect(classifyDevice("harmony-Phone_1")).toBe("harmony"); + }); + + it("does not classify an id without the prefix as harmony", () => { + // The bare instance name and a name that merely contains "harmony" are an + // Android serial by shape — only the leading prefix routes to HarmonyOS. + expect(classifyDevice("Phone_1")).toBe("android"); + expect(classifyDevice("my-harmony-Phone_1")).toBe("android"); + expect(classifyDevice("AAAAAAAA-AAAA-AAAA-AAAA-AAAAAAAAAAAA")).toBe("ios"); + }); + + it("resolves a harmony id to an emulator", () => { + const d = resolveDevice("harmony-Phone_1"); + expect(d.platform).toBe("harmony"); + expect(d.kind).toBe("emulator"); + expect(d.id).toBe("harmony-Phone_1"); + }); + + it("round-trips the instance name through the id prefix", () => { + expect(`${HARMONY_ID_PREFIX}Phone_1`).toBe("harmony-Phone_1"); + expect(harmonyInstanceName(`${HARMONY_ID_PREFIX}Phone_1`)).toBe("Phone_1"); + }); + + it("leaves an unprefixed name alone", () => { + expect(harmonyInstanceName("Phone_1")).toBe("Phone_1"); + }); + + it("strips only the leading prefix from an instance named after it", () => { + expect(harmonyInstanceName("harmony-harmony-1")).toBe("harmony-1"); + }); +}); diff --git a/packages/tool-server/test/variant-proposals.test.ts b/packages/tool-server/test/variant-proposals.test.ts index 27a9f48d22..65a8839822 100644 --- a/packages/tool-server/test/variant-proposals.test.ts +++ b/packages/tool-server/test/variant-proposals.test.ts @@ -7,7 +7,7 @@ const variant = (name: string, extra: Record = {}) => ({ ...extra, }); -const KNOWN_PLATFORMS = new Set(["ios", "ios-remote", "android", "chromium", "vega"]); +const KNOWN_PLATFORMS = new Set(["ios", "ios-remote", "android", "chromium", "vega", "harmony"]); /** * Assert a Lens telemetry payload carries only privacy-safe primitives: numbers, From 704f9f645dcdfc604ae5ef611c3ea9941f818049 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Mon, 10 Aug 2026 19:38:39 +0200 Subject: [PATCH 002/110] fix(harmony): drop two exports nothing imports `__resetHarmonyBinaryCacheForTests` had no caller, and `HarmonyInstance` was only ever used as an internal return element type, so it becomes a plain type alias. Together they pushed the knip second pass one over its 215-issue ceiling; the run is back at 215 with no harmony entries. --- packages/tool-server/src/utils/harmony-cli.ts | 5 ----- packages/tool-server/src/utils/harmony-devices.ts | 4 +--- 2 files changed, 1 insertion(+), 8 deletions(-) diff --git a/packages/tool-server/src/utils/harmony-cli.ts b/packages/tool-server/src/utils/harmony-cli.ts index 81d2f67d11..f427198803 100644 --- a/packages/tool-server/src/utils/harmony-cli.ts +++ b/packages/tool-server/src/utils/harmony-cli.ts @@ -84,11 +84,6 @@ export async function resolveHarmonyEmulator(): Promise { return path; } -/** Test-only: clear the binary-resolution memo. */ -export function __resetHarmonyBinaryCacheForTests(): void { - cachedEmulator = undefined; -} - export interface HarmonyRunResult { stdout: string; stderr: string; diff --git a/packages/tool-server/src/utils/harmony-devices.ts b/packages/tool-server/src/utils/harmony-devices.ts index 01d0425bfe..9d5eeaf040 100644 --- a/packages/tool-server/src/utils/harmony-devices.ts +++ b/packages/tool-server/src/utils/harmony-devices.ts @@ -14,9 +14,7 @@ import { */ /** A HarmonyOS emulator instance as reported by `Emulator -list`. */ -export interface HarmonyInstance { - name: string; -} +type HarmonyInstance = { name: string }; /** * Lines the emulator manager prints alongside a listing that are not instance From 364223c6413250504b912c363c89a2edf5d1c421 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Mon, 10 Aug 2026 19:43:40 +0200 Subject: [PATCH 003/110] fix(harmony): bound the -list call under the discovery deadline `list-devices` backstops every discovery branch at BRANCH_DEADLINE_MS, an invariant that only holds while each branch's worst case stays below it - otherwise the backstop truncates a branch that would have completed and drops a real device. The HarmonyOS branch inherited runHarmonyEmulator's 30s default, which is above the 25s deadline, so it now passes its own 6s bound; `Emulator -list` reads a local directory in under 0.1s, making that headroom rather than a real cost. The invariant test covered only the Vega and Android branches, so it stayed green through the breach; it now sums the HarmonyOS branch too and fails if the bound is raised past the deadline. --- .../tool-server/src/tools/devices/list-devices.ts | 2 ++ packages/tool-server/src/utils/harmony-devices.ts | 12 +++++++++++- .../tool-server/test/list-devices-deadline.test.ts | 9 +++++++++ 3 files changed, 22 insertions(+), 1 deletion(-) diff --git a/packages/tool-server/src/tools/devices/list-devices.ts b/packages/tool-server/src/tools/devices/list-devices.ts index 388ff7add3..381e2cc749 100644 --- a/packages/tool-server/src/tools/devices/list-devices.ts +++ b/packages/tool-server/src/tools/devices/list-devices.ts @@ -170,6 +170,8 @@ async function resolveVvdShadowAdbSerials( // entirely — see listVegaDevices — so the wedged-VVD case is just ~6s.) // - Android: one bounded `adb devices` call (6s) + ~5s concurrent getprop // enrichment = ~11s. +// - HarmonyOS: one `Emulator -list` under its own HARMONY_LIST_TIMEOUT_MS (6s); +// it reads a local directory, so the bound is headroom rather than a real cost. // - iOS / AVD-list / Chromium self-bound by their own subprocess/socket timeouts // (iOS `simctl` ~10s, AVD-list ~5s, Chromium <1s) — all comfortably under 25s. // The Vega binary resolution (`resolveVegaBinary`) runs first but is memoized and diff --git a/packages/tool-server/src/utils/harmony-devices.ts b/packages/tool-server/src/utils/harmony-devices.ts index 9d5eeaf040..44066cbc19 100644 --- a/packages/tool-server/src/utils/harmony-devices.ts +++ b/packages/tool-server/src/utils/harmony-devices.ts @@ -42,10 +42,20 @@ export function parseHarmonyInstances(stdout: string): HarmonyInstance[] { .map((name) => ({ name })); } +/** + * Bound for the `-list` call, well under `list-devices`' BRANCH_DEADLINE_MS — + * that backstop must stay above every branch's worst case, or it truncates a + * branch that would have completed and drops a real device from the list. + * `runHarmonyEmulator`'s 30s default is above the deadline, so this call must + * pass its own. `-list` only reads `~/.Huawei/Emulator/deployed` (measured at + * under 0.1s), making 6s pure headroom for a loaded machine. + */ +export const HARMONY_LIST_TIMEOUT_MS = 6_000; + /** Emulator instances, or [] when DevEco Studio isn't installed. */ export async function listHarmonyInstances(): Promise { if (!(await resolveHarmonyEmulator())) return []; - const result = await runHarmonyEmulator(["-list"]); + const result = await runHarmonyEmulator(["-list"], HARMONY_LIST_TIMEOUT_MS); if (emulatorFailure(result)) return []; return parseHarmonyInstances(result.stdout); } diff --git a/packages/tool-server/test/list-devices-deadline.test.ts b/packages/tool-server/test/list-devices-deadline.test.ts index 35d54fe088..f2b179eed2 100644 --- a/packages/tool-server/test/list-devices-deadline.test.ts +++ b/packages/tool-server/test/list-devices-deadline.test.ts @@ -6,6 +6,7 @@ import { } from "../src/utils/vega-devices"; import { VVD_PS_PROBE_TIMEOUT_MS } from "../src/utils/vega-process"; import { ADB_DEVICES_TIMEOUT_MS, ENRICH_TIMEOUT_MS } from "../src/utils/adb"; +import { HARMONY_LIST_TIMEOUT_MS } from "../src/utils/harmony-devices"; // Guards the timeout/backstop ordering: the backstop must sit ABOVE every branch's // FULL per-call worst case, or it stops being a last-resort and starts truncating a @@ -39,6 +40,14 @@ describe("discovery timeout vs backstop invariant", () => { expect(androidWorstCase).toBeLessThan(BRANCH_DEADLINE_MS); expect(BRANCH_DEADLINE_MS - androidWorstCase).toBeGreaterThanOrEqual(MIN_MARGIN_MS); }); + + it("the HarmonyOS branch's full worst case stays comfortably under the branch deadline", () => { + // A single bounded `Emulator -list`. It must carry its own timeout: the + // wrapper's 30s default is itself above the deadline, so relying on it would + // let the backstop cut off a branch that was still working. + expect(HARMONY_LIST_TIMEOUT_MS).toBeLessThan(BRANCH_DEADLINE_MS); + expect(BRANCH_DEADLINE_MS - HARMONY_LIST_TIMEOUT_MS).toBeGreaterThanOrEqual(MIN_MARGIN_MS); + }); }); // Unit cover for the hard backstop that keeps a single wedged discovery branch from From 7dd071716c08586a99845eb93a27f784d507793e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Mon, 10 Aug 2026 21:42:46 +0200 Subject: [PATCH 004/110] feat(harmony): drive real HarmonyOS devices over hdc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Adds the `hdc` transport the platform was missing, and with it physical HarmonyOS device discovery. `hdc` exits 0 for everything — an unknown target, a missing file, a remote command that exits 3 — so `runHdc` classifies from the `[Fail]` prefix rather than the status, and `runHdcShell` recovers the remote command's own exit code by echoing it back through a sentinel. Without that, a `uitest` that is not installed is indistinguishable from one that ran. `Emulator -list -details` turns out to report `isRunning`, form factor and OS version as JSON, so instances no longer carry state "unknown"; discovery reads the richer form, which costs the same ~30ms. HarmonyOS ids now split by what they address: `harmony-emulator-` for a DevEco instance that boot-device starts, `harmony-` for a target hdc drives. A running emulator appears under both, exactly as a running AVD appears in `avds` and in `adb devices`. Verified against a physical HarmonyOS 6.0.1 handset (Mate 60) over USB and against DevEco Studio 6.1's Emulator 6.1.1.200. --- packages/registry/src/failure-codes.ts | 8 + packages/registry/src/types.ts | 11 +- .../src/tools/devices/boot-device.ts | 19 +- .../src/tools/devices/list-devices.ts | 140 ++++++---- packages/tool-server/src/utils/check-deps.ts | 9 + .../src/utils/cross-platform-tool.ts | 27 +- packages/tool-server/src/utils/device-info.ts | 65 ++++- .../tool-server/src/utils/harmony-devices.ts | 151 +++++++++-- packages/tool-server/src/utils/harmony-hdc.ts | 255 ++++++++++++++++++ .../tool-server/src/utils/harmony-uitest.ts | 243 +++++++++++++++++ .../tool-server/test/harmony-devices.test.ts | 162 ++++++++--- .../test/list-devices-deadline.test.ts | 18 +- 12 files changed, 962 insertions(+), 146 deletions(-) create mode 100644 packages/tool-server/src/utils/harmony-hdc.ts create mode 100644 packages/tool-server/src/utils/harmony-uitest.ts diff --git a/packages/registry/src/failure-codes.ts b/packages/registry/src/failure-codes.ts index b5628a3821..86c6feb813 100644 --- a/packages/registry/src/failure-codes.ts +++ b/packages/registry/src/failure-codes.ts @@ -61,6 +61,14 @@ export const FAILURE_CODES = { VEGA_SCREENSHOT_FAILED: "VEGA_SCREENSHOT_FAILED", VEGA_TEXT_INVALID: "VEGA_TEXT_INVALID", + HARMONY_HDC_NOT_FOUND: "HARMONY_HDC_NOT_FOUND", + HARMONY_HDC_COMMAND_FAILED: "HARMONY_HDC_COMMAND_FAILED", + HARMONY_DEVICE_UNREACHABLE: "HARMONY_DEVICE_UNREACHABLE", + HARMONY_SHELL_NO_STATUS: "HARMONY_SHELL_NO_STATUS", + HARMONY_FILE_TRANSFER_FAILED: "HARMONY_FILE_TRANSFER_FAILED", + HARMONY_UITEST_FAILED: "HARMONY_UITEST_FAILED", + HARMONY_ABILITY_START_FAILED: "HARMONY_ABILITY_START_FAILED", + ANDROID_ADB_NOT_FOUND: "ANDROID_ADB_NOT_FOUND", ANDROID_EMULATOR_NOT_FOUND: "ANDROID_EMULATOR_NOT_FOUND", ANDROID_ADB_COMMAND_FAILED: "ANDROID_ADB_COMMAND_FAILED", diff --git a/packages/registry/src/types.ts b/packages/registry/src/types.ts index 8aeb62374d..a24f91883e 100644 --- a/packages/registry/src/types.ts +++ b/packages/registry/src/types.ts @@ -196,12 +196,14 @@ export interface ToolCapability { device?: boolean; }; /** - * HarmonyOS support. Emulator-only: the tool-server neither detects nor - * connects to physical HarmonyOS hardware, so there is no `device` flag to - * declare — adding one is deferred to a version where it can be tested. + * HarmonyOS support. `device` is a phone reached over `hdc`; `emulator` is a + * DevEco Studio instance, which only `boot-device` addresses — every + * interaction tool drives a *connected* target, and a booted emulator presents + * as one of those, so interaction tools declare `device` alone. */ harmony?: { emulator?: boolean; + device?: boolean; }; /** Optional refiner. Returns true if this device is supported. */ supports?: (device: DeviceInfo) => boolean; @@ -233,7 +235,8 @@ export type ToolDependency = | "emulator" | "sim-remote" | "vega" - | "harmony-emulator"; + | "harmony-emulator" + | "hdc"; // ── Tool Types ── diff --git a/packages/tool-server/src/tools/devices/boot-device.ts b/packages/tool-server/src/tools/devices/boot-device.ts index f54ec066b0..b58015d085 100644 --- a/packages/tool-server/src/tools/devices/boot-device.ts +++ b/packages/tool-server/src/tools/devices/boot-device.ts @@ -39,7 +39,7 @@ import { classifyDevice, harmonyInstanceName, stripRemotePrefix, - HARMONY_ID_PREFIX, + harmonyEmulatorId, } from "../../utils/device-info"; import { simctlBoot as simRemoteBoot, @@ -1445,8 +1445,11 @@ function bootVega(params: { return promise; } -// A HarmonyOS emulator exposes no serial that can be observed, so its id is -// minted from the instance name — see `HARMONY_ID_PREFIX`. +// The returned id names the *instance*, not a connect key: `-start` reports +// only that it handed the instance off, and the key a booted emulator registers +// with `hdc` could not be observed here (no image is obtainable — see +// HARMONY_IMAGE_RESTRICTION). Once it has registered, `list-devices` reports it +// as a `kind: "device"` entry, which is the id the interaction tools take. type HarmonyBootResult = { platform: "harmony"; udid: string; @@ -1504,10 +1507,10 @@ async function bootHarmonyImpl(params: { const imageMissing = isChinaOnlyRestriction(diagnostic) || diagnostic.includes("Cannot find image"); // A start that failed because no instance exists is the same wall one step - // earlier, so ask whether any exists at all. `-list` is read only here, on - // the failure path: its populated output shape could not be observed, so - // misreading it must never block a start that would otherwise have worked. - // `null` means the listing itself failed — unknown, so claim nothing. + // earlier, so ask whether any exists at all. Read only here, on the failure + // path — a listing that misfires must never block a start that would + // otherwise have worked. `null` means the listing itself failed — unknown, + // so claim nothing. const instances = imageMissing ? null : await listHarmonyInstances().catch(() => null); // Only the manager's own words justify blaming the region; an empty instance // list is merely consistent with it. @@ -1525,7 +1528,7 @@ async function bootHarmonyImpl(params: { return { platform: "harmony", - udid: `${HARMONY_ID_PREFIX}${params.instanceName}`, + udid: harmonyEmulatorId(params.instanceName), instanceName: params.instanceName, booted: true, }; diff --git a/packages/tool-server/src/tools/devices/list-devices.ts b/packages/tool-server/src/tools/devices/list-devices.ts index 381e2cc749..76fb10dadc 100644 --- a/packages/tool-server/src/tools/devices/list-devices.ts +++ b/packages/tool-server/src/tools/devices/list-devices.ts @@ -9,8 +9,8 @@ import { import { listRunningVvdConsolePorts } from "../../utils/vega-process"; import { listIosSimulators, type IosSimulator } from "../../utils/ios-devices"; import { simctlListDevices } from "../../utils/sim-remote"; -import { withRemotePrefix, HARMONY_ID_PREFIX } from "../../utils/device-info"; -import { listHarmonyInstances } from "../../utils/harmony-devices"; +import { withRemotePrefix, harmonyDeviceId, harmonyEmulatorId } from "../../utils/device-info"; +import { listHarmonyInstances, listHarmonyHdcTargets } from "../../utils/harmony-devices"; import { discoverChromiumDevices, type ChromiumDevice } from "../../utils/chromium-discovery"; import { listVegaDevices, @@ -44,19 +44,29 @@ type AndroidDevice = { }; /** - * A HarmonyOS emulator instance reported by DevEco Studio's `Emulator` manager. + * A HarmonyOS target, from either of the platform's two discovery sources. * - * `state` is always "unknown": nothing here determines whether an instance is - * running. Huawei restricts the emulator image download to mainland China, so no - * instance could be created and the manager's populated output was never - * observed, leaving no verified way to read readiness. Claiming "stopped" would - * be a guess; "unknown" is the fact. + * `kind: "device"` is a target `hdc` is connected to — a phone over USB, or a + * booted emulator that has registered — and is what every interaction tool + * drives. `kind: "emulator"` is a DevEco Studio *instance*, which exists whether + * or not it is running and is what `boot-device` starts. + * + * A running emulator is therefore listed twice, once under each kind, the same + * way a running Android AVD appears in both `avds` and `adb devices`. */ type HarmonyDevice = { platform: "harmony"; udid: string; name: string; - state: "unknown"; + kind: "emulator" | "device"; + /** `Connected`/`Offline` for a connected target; `running`/`stopped` for an instance. */ + state: string; + /** Instance form factor (`Phone`, `Foldable`, …). Null for a connected target. */ + deviceType?: string | null; + /** e.g. `HarmonyOS 6.1.1(24)`. Null for a connected target. */ + osVersion?: string | null; + /** Transport of a connected target (`USB`, `TCP`). Absent for an instance. */ + connection?: string | null; }; type ListDevicesResult = { @@ -92,7 +102,9 @@ function readinessRank( if (d.platform === "android") return d.state === "device" ? 0 : 1; if (d.platform === "vega") return d.state === "running" || d.state === "device" ? 0 : 1; if (d.platform === "chromium") return 0; // Chromium entries are only listed when their CDP is responsive - if (d.platform === "harmony") return 1; // readiness is never determined — rank below anything confirmed ready + // A `Connected` hdc target is drivable now; a `running` instance has booted. + // Anything else (a stopped instance, an `Offline` target) needs work first. + if (d.platform === "harmony") return d.state === "Connected" || d.state === "running" ? 0 : 1; return d.state === "Booted" ? 0 : 1; // ios + ios-remote } @@ -170,8 +182,11 @@ async function resolveVvdShadowAdbSerials( // entirely — see listVegaDevices — so the wedged-VVD case is just ~6s.) // - Android: one bounded `adb devices` call (6s) + ~5s concurrent getprop // enrichment = ~11s. -// - HarmonyOS: one `Emulator -list` under its own HARMONY_LIST_TIMEOUT_MS (6s); -// it reads a local directory, so the bound is headroom rather than a real cost. +// - HarmonyOS: two independent branches, each one bounded call — `Emulator +// -list -details` under HARMONY_LIST_TIMEOUT_MS (6s), which reads a local +// directory, and `hdc list targets` under HDC_LIST_TIMEOUT_MS (8s), which +// talks to (and may start) the hdc daemon. They run concurrently in the +// fan-out, so the branch worst case is the larger of the two, 8s. // - iOS / AVD-list / Chromium self-bound by their own subprocess/socket timeouts // (iOS `simctl` ~10s, AVD-list ~5s, Chromium <1s) — all comfortably under 25s. // The Vega binary resolution (`resolveVegaBinary`) runs first but is memoized and @@ -223,10 +238,10 @@ export const listDevicesTool: ToolDefinition, ListDevicesR }, failedMsg: ({ failureSignal }) => `Failed to list devices: ${failureSignal.error_code}`, }, - description: `List iOS simulators, Android emulators, connected physical Android devices, running Chromium apps, Vega (Fire TV) devices, and HarmonyOS emulators in one place. + description: `List iOS simulators, Android emulators, connected physical Android devices, running Chromium apps, Vega (Fire TV) devices, and HarmonyOS phones and emulators in one place. Use at the start of a session to pick a target id ('udid' for iOS/HarmonyOS entries, 'serial' for Android/Vega entries, 'id' for Chromium) to pass to interaction tools, and to see which targets are already running. Returns { devices, avds } where each device carries a 'platform' discriminator ('ios', 'android', 'chromium', 'vega', or 'harmony'); 'avds' lists Android AVDs bootable via boot-device. A Vega VVD is listed under 'devices' whether running or stopped (state 'running'/'stopped'); start a stopped one with boot-device using its 'vvdImage'. -HarmonyOS emulator instances come from DevEco Studio's emulator manager and always carry state 'unknown' — Argent does not determine whether an instance is running. No interaction tool drives a HarmonyOS device yet, so they are listed for discovery and boot-device only; anything else returns 501. +HarmonyOS entries carry a 'kind': 'device' is a target connected over \`hdc\` (a phone on USB, or a booted emulator) and is what the interaction tools drive; 'emulator' is a DevEco Studio instance, running or not, and is what boot-device starts. A running emulator appears under both, exactly as a running Android AVD appears in both 'avds' and the device list. Instance state is 'running'/'stopped'; connected-target state is 'Connected'/'Offline'. Android entries also carry a 'kind' ('emulator' for a local AVD, 'device' for a physical phone connected over USB / wireless adb) — physical phones are detected from \`adb devices\` (any serial that is not an \`emulator-*\` one) and are driven through the same interaction tools as emulators; they do not need boot-device (just connect the phone with USB debugging authorised). TV targets are tagged with runtimeKind 'tv' (Apple TV simulators on iOS, Android TV / leanback devices on Android) — these are focus-driven, not touch-driven: use \`describe\` to read focus, \`tv-remote\` for remote presses (up/down/left/right/select/back/menu/home), and \`keyboard\` to type, rather than the coordinate/gesture tools. iOS simulators from an additional CoreSimulator device set (the 'ios.additionalDeviceSets' configuration — e.g. devices created by Radon IDE) are listed alongside default-set ones, tagged with their owning 'deviceSet' path; they are driven through the same tools by udid, but run headless (no Simulator.app window attaches to them). @@ -246,38 +261,44 @@ Booted/ready devices are listed first. Platforms whose CLI is unavailable are si // timer is cleared on the fast happy path). The deadline only substitutes a // fallback on *slowness*; a rejection still propagates exactly as before — so the // `.catch(() => [])` wrappers (and the lack of one on iOS/AVDs) are unchanged. - const [ios, iosRemote, android, avds, chromium, vega, harmony] = await Promise.all([ - withDeadline(listIosSimulators(), [], "ios"), - withDeadline(listRemoteIosSimulators(), [], "ios-remote"), - withDeadline( - // Opt into runtimeKind enrichment (list-devices surfaces TV vs mobile to - // the agent, so the extra feature probe per device is warranted here — the - // boot-loop poller deliberately omits it), and pass the tight `adb devices` - // bound (NOT boot-device's 30s default) so the Android branch self-bounds - // under BRANCH_DEADLINE_MS — see ADB_DEVICES_TIMEOUT_MS. - listAndroidDevices({ runtimeKind: true, devicesTimeoutMs: ADB_DEVICES_TIMEOUT_MS }).catch( - () => [] + const [ios, iosRemote, android, avds, chromium, vega, harmony, harmonyTargets] = + await Promise.all([ + withDeadline(listIosSimulators(), [], "ios"), + withDeadline(listRemoteIosSimulators(), [], "ios-remote"), + withDeadline( + // Opt into runtimeKind enrichment (list-devices surfaces TV vs mobile to + // the agent, so the extra feature probe per device is warranted here — the + // boot-loop poller deliberately omits it), and pass the tight `adb devices` + // bound (NOT boot-device's 30s default) so the Android branch self-bounds + // under BRANCH_DEADLINE_MS — see ADB_DEVICES_TIMEOUT_MS. + listAndroidDevices({ runtimeKind: true, devicesTimeoutMs: ADB_DEVICES_TIMEOUT_MS }).catch( + () => [] + ), + [], + "android" ), - [], - "android" - ), - withDeadline(listAvds(), [], "avds"), - withDeadline( - discoverChromiumDevices().catch(() => []), - [], - "chromium" - ), - withDeadline( - listVegaDevices().catch(() => []), - [], - "vega" - ), - withDeadline( - listHarmonyInstances().catch(() => []), - [], - "harmony" - ), - ]); + withDeadline(listAvds(), [], "avds"), + withDeadline( + discoverChromiumDevices().catch(() => []), + [], + "chromium" + ), + withDeadline( + listVegaDevices().catch(() => []), + [], + "vega" + ), + withDeadline( + listHarmonyInstances().catch(() => []), + [], + "harmony" + ), + withDeadline( + listHarmonyHdcTargets().catch(() => []), + [], + "harmony-hdc" + ), + ]); const iosTagged: IosDevice[] = ios.map((s) => ({ platform: "ios", ...s })); iosTagged.sort(sortIos); iosRemote.sort(sortIosRemote); @@ -297,12 +318,29 @@ Booted/ready devices are listed first. Platforms whose CLI is unavailable are si const androidDeduped = filterVvdShadowsFromAndroid(androidTagged, vvdShadowSerials); androidDeduped.sort(sortAndroid); - const harmonyTagged: HarmonyDevice[] = harmony.map((h) => ({ - platform: "harmony", - udid: `${HARMONY_ID_PREFIX}${h.name}`, - name: h.name, - state: "unknown", - })); + const harmonyTagged: HarmonyDevice[] = [ + ...harmonyTargets.map( + (t): HarmonyDevice => ({ + platform: "harmony", + udid: harmonyDeviceId(t.connectKey), + name: t.connectKey, + kind: "device", + state: t.state, + connection: t.connection, + }) + ), + ...harmony.map( + (h): HarmonyDevice => ({ + platform: "harmony", + udid: harmonyEmulatorId(h.name), + name: h.name, + kind: "emulator", + state: h.running ? "running" : "stopped", + deviceType: h.deviceType, + osVersion: h.osVersion, + }) + ), + ]; const devices: Array< IosDevice | IosRemoteDevice | AndroidDevice | ChromiumDevice | VegaDevice | HarmonyDevice diff --git a/packages/tool-server/src/utils/check-deps.ts b/packages/tool-server/src/utils/check-deps.ts index 4eebcbeafc..93c882949b 100644 --- a/packages/tool-server/src/utils/check-deps.ts +++ b/packages/tool-server/src/utils/check-deps.ts @@ -2,6 +2,7 @@ import { FAILURE_CODES, withFailureSignal, type ToolDependency } from "@argent/r import { resolveAndroidBinary } from "./android-binary"; import { resolveVegaBinary } from "./vega-cli"; import { resolveHarmonyEmulator } from "./harmony-cli"; +import { resolveHdc } from "./harmony-hdc"; import { commandOnPath } from "./command-on-path"; /** @@ -48,6 +49,8 @@ const INSTALL_HINTS: Record = { "Vega SDK CLI not found. Install the Amazon Vega SDK and run `source ~/vega/env` so `vega` (or its `kepler` alias) is on PATH; the resolver also checks `~/vega/bin/vega`. Only required for Vega (Fire TV) devices.", "harmony-emulator": "HarmonyOS emulator manager not found. Install DevEco Studio; on a non-macOS host set `$DEVECO_STUDIO_HOME` to its install root. Only required to list or launch HarmonyOS emulators.", + "hdc": + "`hdc`, the HarmonyOS device connector, was not found. Install DevEco Studio (it ships under `sdk/default/openharmony/toolchains`), or put `hdc` from the OpenHarmony command-line tools on PATH, or set `$DEVECO_STUDIO_HOME` to the DevEco install root. Only required to drive HarmonyOS devices.", }; async function probe(dep: ToolDependency): Promise { @@ -71,6 +74,12 @@ async function probe(dep: ToolDependency): Promise { if (dep === "harmony-emulator") { return (await resolveHarmonyEmulator()) !== null; } + // `hdc` ships inside DevEco Studio's SDK rather than on PATH for IDE installs, + // so it resolves through the DevEco-aware resolver (which falls back to PATH + // for standalone command-line-tools installs). + if (dep === "hdc") { + return (await resolveHdc()) !== null; + } // `commandOnPath` probes existence without invoking the dep itself — a bare // `xcrun` call would fork the tool just to check existence, which is both // slower and (for xcrun) can prompt the license agreement dialog on first diff --git a/packages/tool-server/src/utils/cross-platform-tool.ts b/packages/tool-server/src/utils/cross-platform-tool.ts index f339365db6..28eba6f117 100644 --- a/packages/tool-server/src/utils/cross-platform-tool.ts +++ b/packages/tool-server/src/utils/cross-platform-tool.ts @@ -60,6 +60,7 @@ export function dispatchByPlatform< ChromiumServices = Record, VegaServices = unknown, IosRemoteServices = IosServices, + HarmonyServices = unknown, >(opts: { toolId: string; capability: ToolCapability; @@ -80,6 +81,13 @@ export function dispatchByPlatform< * `NotImplementedOnPlatformError` (501) rather than silently falling through. */ vega?: PlatformImpl; + /** + * HarmonyOS branch, driven over `hdc`. Optional on the same terms as `vega`: + * a tool whose capability declares `harmony` but wires no branch here throws + * `NotImplementedOnPlatformError` (501) rather than falling through to + * chromium. + */ + harmony?: PlatformImpl; }): ( services: Record, params: Params, @@ -132,10 +140,21 @@ export function dispatchByPlatform< return opts.vega.handler(services as unknown as VegaServices, params, device, invokeOptions); } if (device.platform === "harmony") { - // No tool drives a HarmonyOS device yet, so there is no branch to call — - // but the check has to be explicit, because chromium is the fallthrough - // below and a harmony device would otherwise be handed to the CDP branch. - throw new NotImplementedOnPlatformError({ toolId: opts.toolId, platform: "harmony" }); + // Explicit, not merely for symmetry: chromium is the fallthrough below, so + // without this branch a HarmonyOS device would be handed to the CDP handler + // and its result reported as a success. + if (!opts.harmony) { + throw new NotImplementedOnPlatformError({ toolId: opts.toolId, platform: "harmony" }); + } + if (opts.harmony.requires?.length) { + await ensureDeps(opts.harmony.requires); + } + return opts.harmony.handler( + services as unknown as HarmonyServices, + params, + device, + invokeOptions + ); } // chromium if (!opts.chromium) { diff --git a/packages/tool-server/src/utils/device-info.ts b/packages/tool-server/src/utils/device-info.ts index 07d2fe3577..6eaf607230 100644 --- a/packages/tool-server/src/utils/device-info.ts +++ b/packages/tool-server/src/utils/device-info.ts @@ -45,21 +45,55 @@ export const CHROMIUM_ID_PREFIX = "chromium-cdp-"; export const VEGA_SERIAL_PREFIX = "amazon-"; /** - * HarmonyOS device-id prefix, applied by `list-devices` to an emulator instance - * name (`harmony-Phone_1`). - * - * Unlike the other platforms this is not a vendor-assigned serial shape: the - * `hdc` connect key of a running HarmonyOS emulator could not be observed, since - * Huawei restricts the emulator image download to mainland China and no instance - * can exist without one. Minting the id from the instance name — which the - * `Emulator` CLI does report — keeps classification exact and avoids encoding a - * guessed serial format, the same way `chromium-cdp-` is argent's own id - * rather than something CDP hands out. + * HarmonyOS device-id prefix. Every HarmonyOS id carries it, so the platform is + * decided by shape like the others — argent mints these ids rather than + * receiving them, the same way `chromium-cdp-` is argent's own. */ export const HARMONY_ID_PREFIX = "harmony-"; -/** Strip the `harmony-` prefix, returning the bare emulator instance name. */ +/** + * Marks the ids that name an emulator *instance* rather than a connected target. + * + * The two are different things addressed by different CLIs: an instance is a + * config directory that `Emulator -start` boots, a connected target is an `hdc` + * connect key that `uitest` drives. They cannot be told apart by shape — an + * instance name is user-chosen and could be spelled exactly like a hardware + * serial — so the distinction is carried in the id itself, mirroring how a local + * Android AVD is identifiable from its `emulator-` serial. + * + * A running emulator therefore appears twice in `list-devices`: once as its + * instance (bootable, stoppable) and once as whatever connect key it registered + * with `hdc` (drivable). This is exactly what Android does with `avds` vs + * `adb devices`, and for the same reason. + */ +export const HARMONY_EMULATOR_ID_PREFIX = "harmony-emulator-"; + +/** Build the `list-devices` id for an emulator instance. */ +export function harmonyEmulatorId(instanceName: string): string { + return `${HARMONY_EMULATOR_ID_PREFIX}${instanceName}`; +} + +/** Build the `list-devices` id for a target `hdc` is connected to. */ +export function harmonyDeviceId(connectKey: string): string { + return `${HARMONY_ID_PREFIX}${connectKey}`; +} + +/** + * The emulator instance name behind a `harmony-emulator-` id. + * + * Exactly one prefix is stripped, so an instance genuinely named `emulator-x` + * round-trips through `harmonyEmulatorId` unharmed. + */ export function harmonyInstanceName(udid: string): string { + return udid.startsWith(HARMONY_EMULATOR_ID_PREFIX) + ? udid.slice(HARMONY_EMULATOR_ID_PREFIX.length) + : udid.startsWith(HARMONY_ID_PREFIX) + ? udid.slice(HARMONY_ID_PREFIX.length) + : udid; +} + +/** The `hdc` connect key behind a `harmony-` id. */ +export function harmonyConnectKey(udid: string): string { return udid.startsWith(HARMONY_ID_PREFIX) ? udid.slice(HARMONY_ID_PREFIX.length) : udid; } @@ -93,8 +127,9 @@ export function isAndroidEmulatorSerial(serial: string): boolean { /** * Build a `DeviceInfo` from a raw udid, by shape. Kind defaults per platform: * 'simulator' for iOS / ios-remote, 'vvd' for Vega, 'emulator'/'device' for - * Android by serial shape, 'app' for Chromium — platform impls can enrich with - * name/state/sdkLevel via simctl/adb/sim-remote if needed. + * Android by serial shape and for HarmonyOS by id prefix, 'app' for Chromium — + * platform impls can enrich with name/state/sdkLevel via simctl/adb/sim-remote + * if needed. * * Vega is VVD-only in v1: the tool-server does not connect to or detect physical * Fire TV hardware, so every `amazon-` serial resolves to kind `vvd` by shape. A @@ -111,7 +146,9 @@ export function resolveDevice(udid: string): DeviceInfo { : platform === "vega" ? "vvd" : platform === "harmony" - ? "emulator" + ? udid.startsWith(HARMONY_EMULATOR_ID_PREFIX) + ? "emulator" + : "device" : platform === "android" ? isAndroidEmulatorSerial(udid) ? "emulator" diff --git a/packages/tool-server/src/utils/harmony-devices.ts b/packages/tool-server/src/utils/harmony-devices.ts index 44066cbc19..e6984c488e 100644 --- a/packages/tool-server/src/utils/harmony-devices.ts +++ b/packages/tool-server/src/utils/harmony-devices.ts @@ -4,42 +4,92 @@ import { resolveHarmonyEmulator, runHarmonyEmulator, } from "./harmony-cli"; +import { HDC_EMPTY_SENTINEL, hdcFailure, resolveHdc, runHdc } from "./harmony-hdc"; /** - * Discovery for the HarmonyOS platform: the emulator instances DevEco Studio's - * `Emulator` manager knows about. + * Discovery for the HarmonyOS platform, from the two sources that know about a + * HarmonyOS target: + * + * - DevEco Studio's `Emulator` manager knows about emulator *instances*, + * running or not. An instance is what `boot-device` starts. + * - `hdc` knows about *connected* targets — a physical phone over USB, and a + * running emulator once it has registered. A connect key is what every + * interaction tool drives. + * + * This is the same split Android has between `avdmanager` and `adb devices`, + * and it has the same consequence: a running emulator legitimately appears in + * both, under an instance name in one and a connect key in the other. * * Readers here treat `[Empty]` as "none" and a recognised diagnostic as a - * failure, because the exit code says neither (see `harmony-cli.ts`). + * failure, because neither CLI's exit code says either (see `harmony-cli.ts` + * and `harmony-hdc.ts`). */ -/** A HarmonyOS emulator instance as reported by `Emulator -list`. */ -type HarmonyInstance = { name: string }; +/** A HarmonyOS emulator instance as reported by `Emulator -list -details`. */ +export interface HarmonyInstance { + name: string; + /** `Phone`, `Foldable`, `Tablet`, `TV`, … as configured at creation. */ + deviceType: string | null; + /** e.g. `HarmonyOS 6.1.1(24)`. */ + osVersion: string | null; + running: boolean; +} /** - * Lines the emulator manager prints alongside a listing that are not instance - * names. `Config file not found: …` is emitted per malformed instance directory - * *before* the `[Empty]`/listing body, so it would otherwise be read as a name. + * One `Emulator -list -details` record. Every value is a string, including the + * booleans — `isRunning` is `"true"`/`"false"`, never a JSON boolean, the same + * quirk `-imageList` has with `downloaded`. */ -function isDiagnosticLine(line: string): boolean { - return line.startsWith("Config file not found:") || line.startsWith("["); +interface RawInstance { + "name"?: unknown; + "deviceName"?: unknown; + "deviceType"?: unknown; + "isRunning"?: unknown; + "os.osVersion"?: unknown; } +const str = (v: unknown): string | null => (typeof v === "string" && v.length > 0 ? v : null); + /** - * Instance names from `Emulator -list`. + * Instances from `Emulator -list -details`. + * + * `-details` rather than the bare `-list` (one name per line) because it is the + * strict superset and costs the same — both were measured at ~30ms, since each + * only reads `~/.Huawei/Emulator/deployed`. It is also the only form that + * reports `isRunning`, without which every instance would have to be listed + * with an unknown state. * - * Only the empty case (`[Empty]`) could be verified on a host outside mainland - * China, because Huawei restricts the image download that creating an instance - * requires — so the populated case is parsed conservatively as one name per - * line, with known diagnostics filtered out, and anything unrecognised simply - * yields no instances rather than a bogus one. + * The name is taken from the config's `name` key, not the directory: the manager + * itself keys off the config, so two directories carrying the same configured + * name are reported as one instance. */ export function parseHarmonyInstances(stdout: string): HarmonyInstance[] { - return stdout - .split(/\r?\n/) - .map((l) => l.trim()) - .filter((l) => l.length > 0 && l !== HARMONY_EMPTY_SENTINEL && !isDiagnosticLine(l)) - .map((name) => ({ name })); + const text = stdout.trim(); + if (text.length === 0 || text.startsWith(HARMONY_EMPTY_SENTINEL)) return []; + // A malformed instance directory makes the manager print `Config file not + // found: …` *before* the JSON body, so parse from the first bracket rather + // than the first byte. + const start = text.indexOf("["); + if (start === -1) return []; + let raw: unknown; + try { + raw = JSON.parse(text.slice(start)); + } catch { + return []; + } + if (!Array.isArray(raw)) return []; + const out: HarmonyInstance[] = []; + for (const entry of raw as RawInstance[]) { + const name = str(entry?.name) ?? str(entry?.deviceName); + if (!name) continue; + out.push({ + name, + deviceType: str(entry?.deviceType), + osVersion: str(entry?.["os.osVersion"]), + running: entry?.isRunning === "true", + }); + } + return out; } /** @@ -47,15 +97,68 @@ export function parseHarmonyInstances(stdout: string): HarmonyInstance[] { * that backstop must stay above every branch's worst case, or it truncates a * branch that would have completed and drops a real device from the list. * `runHarmonyEmulator`'s 30s default is above the deadline, so this call must - * pass its own. `-list` only reads `~/.Huawei/Emulator/deployed` (measured at - * under 0.1s), making 6s pure headroom for a loaded machine. + * pass its own. `-list -details` only reads `~/.Huawei/Emulator/deployed` + * (measured at under 0.1s), making 6s pure headroom for a loaded machine. */ export const HARMONY_LIST_TIMEOUT_MS = 6_000; /** Emulator instances, or [] when DevEco Studio isn't installed. */ export async function listHarmonyInstances(): Promise { if (!(await resolveHarmonyEmulator())) return []; - const result = await runHarmonyEmulator(["-list"], HARMONY_LIST_TIMEOUT_MS); + const result = await runHarmonyEmulator(["-list", "-details"], HARMONY_LIST_TIMEOUT_MS); if (emulatorFailure(result)) return []; return parseHarmonyInstances(result.stdout); } + +/** A target `hdc` is connected to — a physical device, or a running emulator. */ +export interface HarmonyHdcTarget { + /** The key every `hdc -t ` call takes. A hardware serial, or `ip:port`. */ + connectKey: string; + /** `USB`, `TCP`, … as reported in column 2. */ + connection: string | null; + /** `Connected`, `Offline`, … — only a Connected target can be driven. */ + state: string; +} + +/** + * Bound for `hdc list targets`, chosen on the same basis as the instance list + * above: it must stay under `BRANCH_DEADLINE_MS`. Unlike `-list` this one does + * talk to a daemon (and starts it if absent), so it gets more room than a + * directory read while staying far below the deadline. + */ +export const HDC_LIST_TIMEOUT_MS = 8_000; + +/** + * Parse `hdc list targets -v`, whose rows are tab-separated: + * + * 025DEK236V035771\t\tUSB\tConnected\tlocalhost + * + * Note the empty second column. Splitting on runs of whitespace rather than on + * single tabs keeps that hole from shifting `connection` and `state` one column + * left — which would report every attached phone's state as `USB`, a string no + * readiness check matches, hiding a perfectly healthy device. + */ +export function parseHdcTargets(stdout: string): HarmonyHdcTarget[] { + const out: HarmonyHdcTarget[] = []; + for (const line of stdout.split(/\r?\n/)) { + const trimmed = line.trim(); + if (trimmed.length === 0 || trimmed.startsWith(HDC_EMPTY_SENTINEL)) continue; + if (trimmed.startsWith("[")) continue; + const cols = trimmed.split(/\s+/); + if (cols.length === 1) { + // `hdc list targets` without `-v` prints the bare connect key. + out.push({ connectKey: cols[0], connection: null, state: "Connected" }); + continue; + } + out.push({ connectKey: cols[0], connection: cols[1] ?? null, state: cols[2] ?? "Unknown" }); + } + return out; +} + +/** Connected HarmonyOS targets, or [] when `hdc` isn't installed. */ +export async function listHarmonyHdcTargets(): Promise { + if (!(await resolveHdc())) return []; + const result = await runHdc(["list", "targets", "-v"], HDC_LIST_TIMEOUT_MS); + if (hdcFailure(result)) return []; + return parseHdcTargets(result.stdout); +} diff --git a/packages/tool-server/src/utils/harmony-hdc.ts b/packages/tool-server/src/utils/harmony-hdc.ts new file mode 100644 index 0000000000..b433c6fd3a --- /dev/null +++ b/packages/tool-server/src/utils/harmony-hdc.ts @@ -0,0 +1,255 @@ +import { execFile } from "node:child_process"; +import { promisify } from "node:util"; +import { access, constants as fsConstants } from "node:fs/promises"; +import { join } from "node:path"; +import { FAILURE_CODES, FailureError } from "@argent/registry"; +import { formatSubprocessFailure } from "./subprocess-error"; +import { commandOnPath } from "./command-on-path"; + +const execFileAsync = promisify(execFile); + +/** + * Wrapper for `hdc`, the HarmonyOS device connector — the platform's `adb`. + * + * It shares the `Emulator` manager's defining hazard (see `harmony-cli.ts`) and + * makes it worse: `hdc` exits 0 for *everything*. Measured against hdc 3.2.0d: + * + * list targets (none) exit 0 ok, prints `[Empty]` + * shell echo hi exit 0 ok + * shell (unknown target) exit 0 FAILED, `[Fail]Not match target...` + * file recv (missing file) exit 0 FAILED, `[Fail]Error opening file...` + * shell 'exit 3' exit 0 remote status DISCARDED + * + * Two consequences drive this module: + * + * 1. Transport failures are classified from the `[Fail]` prefix, never the exit + * code, so `runHdc` returns output rather than rejecting on a non-zero exit. + * 2. The remote command's own exit status never reaches the host — `hdc shell` + * reports the status of the *connection*, not of what ran. `runHdcShell` + * therefore appends an `echo` of `$?` and parses it back off stdout, which is + * the only way to tell `uitest` succeeding from `uitest` not being installed. + */ + +/** Prefix `hdc` puts on its own transport-level diagnostics. */ +const HDC_FAILURE_PREFIX = "[Fail]"; + +/** `hdc list targets` prints this token rather than nothing when no device is attached. */ +export const HDC_EMPTY_SENTINEL = "[Empty]"; + +/** + * Sentinel used to smuggle the remote exit status back over a transport that + * drops it. Prefixed with `__argent` so it cannot collide with a line the + * command under test legitimately prints. + */ +const RC_SENTINEL = "__argent_hdc_rc"; + +const MACOS_DEVECO_ROOT = "/Applications/DevEco-Studio.app/Contents"; + +/** Path of `hdc` relative to a DevEco Studio install root. */ +const HDC_RELATIVE = join("sdk", "default", "openharmony", "toolchains", "hdc"); + +const BINARY_TTL_MS = 60_000; +let cachedHdc: { path: string | null; checkedAt: number } | undefined; + +async function isExecutable(p: string): Promise { + try { + await access(p, fsConstants.X_OK); + return true; + } catch { + return false; + } +} + +/** + * Absolute path to `hdc`, or null when neither DevEco Studio nor a standalone + * OpenHarmony command-line-tools install provides it. + * + * Unlike the emulator manager — whose binary is named `Emulator`, too generic to + * match on PATH — `hdc` is a distinctive name owned by the HarmonyOS toolchain, + * so PATH is consulted as a fallback for hosts that installed the command-line + * tools without the IDE. + */ +export async function resolveHdc(): Promise { + const now = Date.now(); + if (cachedHdc && now - cachedHdc.checkedAt < BINARY_TTL_MS) { + return cachedHdc.path; + } + const configured = process.env.DEVECO_STUDIO_HOME?.trim(); + const root = configured || (process.platform === "darwin" ? MACOS_DEVECO_ROOT : null); + let path: string | null = null; + if (root) { + const candidate = join(root, HDC_RELATIVE); + if (await isExecutable(candidate)) path = candidate; + } + path ??= await commandOnPath("hdc"); + cachedHdc = { path, checkedAt: now }; + return path; +} + +export async function resolveHdcOrThrow(): Promise { + const path = await resolveHdc(); + if (!path) { + throw new FailureError( + "`hdc` (the HarmonyOS device connector) was not found. Install DevEco Studio, or set " + + "`$DEVECO_STUDIO_HOME` to its install root, or put `hdc` from the OpenHarmony " + + "command-line tools on PATH.", + { + error_code: FAILURE_CODES.HARMONY_HDC_NOT_FOUND, + failure_stage: "harmony_hdc_resolve_binary", + failure_area: "tool_server", + error_kind: "dependency_missing", + } + ); + } + return path; +} + +export interface HdcRunResult { + stdout: string; + stderr: string; +} + +const DEFAULT_TIMEOUT_MS = 30_000; + +/** + * Run `hdc` with the given argv. Returns the child's output whether it exited 0 + * or not — see the header: the exit code carries no signal, so classification is + * left to `hdcFailure`. Only a spawn failure or a timeout kill rejects. + */ +export async function runHdc( + args: string[], + timeoutMs = DEFAULT_TIMEOUT_MS +): Promise { + const bin = await resolveHdcOrThrow(); + try { + const { stdout, stderr } = await execFileAsync(bin, args, { + timeout: timeoutMs, + maxBuffer: 64 * 1024 * 1024, + }); + return { stdout, stderr }; + } catch (err) { + const e = err as { killed?: boolean; code?: unknown; stdout?: string; stderr?: string }; + if (!e.killed && typeof e.code === "number" && (e.stdout != null || e.stderr != null)) { + return { stdout: e.stdout ?? "", stderr: e.stderr ?? "" }; + } + throw new FailureError(formatSubprocessFailure("hdc", args, err), { + error_code: FAILURE_CODES.HARMONY_HDC_COMMAND_FAILED, + failure_stage: "harmony_hdc_run", + failure_area: "tool_server", + error_kind: "subprocess", + }); + } +} + +/** + * The `[Fail]…` line `hdc` printed, or null when it reported no transport error. + * + * Matched on the prefix rather than a substring so a remote command that merely + * prints the token — a log line, a test name — cannot forge a transport failure. + */ +export function hdcFailure(result: HdcRunResult): string | null { + const text = `${result.stdout}\n${result.stderr}`; + const line = text.split(/\r?\n/).find((l) => l.trimStart().startsWith(HDC_FAILURE_PREFIX)); + return line ? line.trim() : null; +} + +/** + * Escape a string for interpolation into the single remote command line that + * `hdc shell` hands to the device's `/bin/sh`. + * + * `hdc shell` takes a *command line*, not an argv — so every value that reaches + * it (text to type, a bundle name, a file path) is shell metacharacter-bearing + * input on a shell running as the `shell` user. POSIX single-quoting with the + * `'\''` break is the only form that needs no escape table: verified through a + * real device against `;`, backticks, `$`, both quote characters and non-ASCII. + */ +export function shellQuote(value: string): string { + return `'${value.replaceAll("'", `'\\''`)}'`; +} + +export interface HdcShellResult { + /** Everything the remote command wrote, with the status sentinel removed. */ + stdout: string; + /** The remote command's own exit status, recovered via the sentinel. */ + exitCode: number; +} + +/** + * Run a command on the device and return its output *and its real exit status*. + * + * `command` is interpolated into a remote `/bin/sh` line verbatim — build it with + * `shellQuote` around every caller-supplied value. + * + * Throws on a transport failure (unknown target, dead connection) so callers can + * treat the result as "the command ran"; a non-zero `exitCode` then means the + * command ran and failed, which is a different thing and theirs to interpret. + */ +export async function runHdcShell( + connectKey: string, + command: string, + timeoutMs = DEFAULT_TIMEOUT_MS +): Promise { + const result = await runHdc( + ["-t", connectKey, "shell", `${command}; echo ${RC_SENTINEL}=$?`], + timeoutMs + ); + const failure = hdcFailure(result); + if (failure) { + throw new FailureError( + `hdc could not reach HarmonyOS device '${connectKey}': ${failure}. ` + + `Check that it is still listed by \`hdc list targets\` and that USB debugging is authorised.`, + { + error_code: FAILURE_CODES.HARMONY_DEVICE_UNREACHABLE, + failure_stage: "harmony_hdc_shell", + failure_area: "tool_server", + error_kind: "not_found", + } + ); + } + const lines = `${result.stdout}`.split(/\r?\n/); + // Scan from the end: the sentinel is the last thing echoed, and a command that + // printed something resembling it earlier must not win over the real one. + const idx = lines.findLastIndex((l) => l.trim().startsWith(`${RC_SENTINEL}=`)); + if (idx === -1) { + // The device dropped the trailing echo — the command was killed on-device, or + // the transport truncated. Either way the status is unknown, and reporting a + // fabricated 0 here would turn a dead command into a silent success. + throw new FailureError( + `HarmonyOS device '${connectKey}' returned no exit status for \`${command}\`. ` + + `The command was terminated on the device or the hdc connection dropped mid-call.`, + { + error_code: FAILURE_CODES.HARMONY_SHELL_NO_STATUS, + failure_stage: "harmony_hdc_shell", + failure_area: "tool_server", + error_kind: "subprocess", + } + ); + } + const exitCode = Number.parseInt(lines[idx].trim().slice(RC_SENTINEL.length + 1), 10); + return { + stdout: lines.slice(0, idx).join("\n").replaceAll("\r", "").trimEnd(), + exitCode: Number.isFinite(exitCode) ? exitCode : -1, + }; +} + +/** Copy a file off the device. Throws with hdc's own diagnostic if it did not arrive. */ +export async function hdcFileRecv( + connectKey: string, + remotePath: string, + localPath: string, + timeoutMs = DEFAULT_TIMEOUT_MS +): Promise { + const result = await runHdc(["-t", connectKey, "file", "recv", remotePath, localPath], timeoutMs); + const failure = hdcFailure(result); + if (failure) { + throw new FailureError( + `Failed to copy '${remotePath}' off HarmonyOS device '${connectKey}': ${failure}`, + { + error_code: FAILURE_CODES.HARMONY_FILE_TRANSFER_FAILED, + failure_stage: "harmony_hdc_file_recv", + failure_area: "tool_server", + error_kind: "subprocess", + } + ); + } +} diff --git a/packages/tool-server/src/utils/harmony-uitest.ts b/packages/tool-server/src/utils/harmony-uitest.ts new file mode 100644 index 0000000000..6dbd7f2567 --- /dev/null +++ b/packages/tool-server/src/utils/harmony-uitest.ts @@ -0,0 +1,243 @@ +import { readFile } from "node:fs/promises"; +import { FAILURE_CODES, FailureError } from "@argent/registry"; +import { hdcFileRecv, runHdcShell, shellQuote } from "./harmony-hdc"; + +/** + * Driver for `uitest`, the on-device binary that is HarmonyOS' entire UI + * automation surface — `uiautomator`, `screencap` and `input` in one: + * + * uitest dumpLayout -p the accessibility tree, as JSON + * uitest screenCap -p a PNG of the display + * uitest uiInput click|doubleClick|longClick touch + * uitest uiInput swipe|drag|fling|dircFling gestures + * uitest uiInput keyEvent hardware keys + * uitest uiInput text|inputText typing + * + * Two measured properties of the tool shape everything here (hdc 3.2.0d / + * HarmonyOS 6.0.1, on a physical Mate 60): + * + * - **Its exit status is trustworthy, unlike its transport's.** `hdc` exits 0 + * for everything (see `harmony-hdc.ts`), but `uitest` itself exits 1 and names + * the problem — `Invalid parameters.`, `Please confirm that the coordinate + * values are correct.`, `The number of parameters is incorrect.` — so the + * status recovered by `runHdcShell` is the success signal, and `No Error` on + * stdout is not needed to confirm it. + * + * - **It validates almost nothing.** A click at `99999 99999`, far outside a + * 1216x2688 display, returns `No Error` and exit 0 having done nothing + * observable. Only *negative* and non-numeric coordinates are rejected. So + * out-of-range coordinates are argent's to catch: `toDevicePoint` clamps into + * the display, which is what keeps a normalized 0-1 rounding error at the + * right edge from silently becoming a no-op tap. + */ + +/** Where on-device artifacts are staged before being copied to the host. */ +const REMOTE_TMP = "/data/local/tmp"; + +const UITEST_TIMEOUT_MS = 20_000; + +/** + * `uitest` prints its multi-line usage block after every failure. Only the + * leading line names the actual problem, so surface that and drop the rest — + * pasting 12 lines of usage into an agent's context buries the diagnostic. + */ +function uitestDiagnostic(stdout: string): string { + const first = stdout.split(/\r?\n/).find((l) => l.trim().length > 0); + return first?.trim() ?? "uitest failed without a diagnostic"; +} + +/** Run a `uitest` subcommand, throwing with its own diagnostic if it exits non-zero. */ +async function runUitest( + connectKey: string, + args: string, + timeoutMs = UITEST_TIMEOUT_MS +): Promise { + const { stdout, exitCode } = await runHdcShell(connectKey, `uitest ${args}`, timeoutMs); + if (exitCode !== 0) { + throw new FailureError(`uitest ${args} failed on ${connectKey}: ${uitestDiagnostic(stdout)}`, { + error_code: FAILURE_CODES.HARMONY_UITEST_FAILED, + failure_stage: "harmony_uitest", + failure_area: "tool_server", + error_kind: "subprocess", + }); + } + return stdout; +} + +export interface HarmonyDisplay { + width: number; + height: number; + /** False when the display is suspended — injected touches land nowhere. */ + screenOn: boolean; +} + +/** + * Display geometry and power state, read from the render service. + * + * Not cached. The obvious optimisation is wrong on this platform: HarmonyOS' + * flagship form factors are foldables, whose resolution changes when the user + * unfolds the device, and `powerStatus` changes on any screen timeout. A cached + * width would silently misplace every subsequent tap on the other half of a + * fold. The call is a local service dump measured at 50-190ms, cheap enough to + * pay per gesture rather than risk that. + */ +export async function harmonyDisplay(connectKey: string): Promise { + const { stdout } = await runHdcShell( + connectKey, + "hidumper -s RenderService -a screen", + UITEST_TIMEOUT_MS + ); + const res = /render resolution=(\d+)x(\d+)/.exec(stdout); + if (!res) { + throw new FailureError( + `Could not read the display size of HarmonyOS device '${connectKey}' from the render service.`, + { + error_code: FAILURE_CODES.HARMONY_UITEST_FAILED, + failure_stage: "harmony_display_size", + failure_area: "tool_server", + error_kind: "subprocess", + } + ); + } + return { + width: Number.parseInt(res[1], 10), + height: Number.parseInt(res[2], 10), + screenOn: !/powerStatus=POWER_STATUS_(OFF|SUSPEND)/.test(stdout), + }; +} + +/** + * Convert argent's normalized 0-1 coordinates to device pixels. + * + * Clamped to the last addressable pixel rather than the bound itself: `uitest` + * accepts an out-of-range coordinate silently (see the header), so `y = 1.0` + * would otherwise inject at `height`, one row past the display, and report + * success for a touch that never happened. + */ +export function toDevicePoint( + x: number, + y: number, + display: { width: number; height: number } +): { x: number; y: number } { + const clamp = (v: number, max: number) => Math.max(0, Math.min(max - 1, Math.round(v * max))); + return { x: clamp(x, display.width), y: clamp(y, display.height) }; +} + +export type HarmonyTouchCommand = "click" | "doubleClick" | "longClick"; + +export async function harmonyTouch( + connectKey: string, + command: HarmonyTouchCommand, + point: { x: number; y: number } +): Promise { + await runUitest(connectKey, `uiInput ${command} ${point.x} ${point.y}`); +} + +export type HarmonySwipeCommand = "swipe" | "drag" | "fling"; + +/** + * `uitest` rejects a velocity outside this range with `Invalid parameters.`, so + * callers translating a duration into a velocity must land inside it. + */ +export const HARMONY_VELOCITY_MIN = 200; +export const HARMONY_VELOCITY_MAX = 40_000; + +export async function harmonySwipe( + connectKey: string, + command: HarmonySwipeCommand, + from: { x: number; y: number }, + to: { x: number; y: number }, + velocity: number +): Promise { + const v = Math.max(HARMONY_VELOCITY_MIN, Math.min(HARMONY_VELOCITY_MAX, Math.round(velocity))); + await runUitest(connectKey, `uiInput ${command} ${from.x} ${from.y} ${to.x} ${to.y} ${v}`); +} + +export async function harmonyKeyEvent(connectKey: string, key: string): Promise { + await runUitest(connectKey, `uiInput keyEvent ${key}`); +} + +/** Type into whatever currently holds focus. */ +export async function harmonyTypeText(connectKey: string, text: string): Promise { + await runUitest(connectKey, `uiInput text ${shellQuote(text)}`); +} + +/** Focus the field at a point and type into it in one call. */ +export async function harmonyInputTextAt( + connectKey: string, + point: { x: number; y: number }, + text: string +): Promise { + await runUitest(connectKey, `uiInput inputText ${point.x} ${point.y} ${shellQuote(text)}`); +} + +/** + * Run `producer` against a freshly-named path under the device's tmp directory, + * copy the result to `localPath`, then delete the on-device copy. + * + * The unique name matters for more than tidiness: two tool calls racing a fixed + * path would have one overwrite the other's capture between write and fetch, and + * the loser would silently receive the winner's screen. The delete runs even + * when the fetch throws — otherwise a device accumulates a multi-hundred-KB PNG + * per failed screenshot, on a partition nothing else prunes. + */ +async function viaDeviceTmp( + connectKey: string, + suffix: string, + localPath: string, + producer: (remotePath: string) => Promise +): Promise { + const remotePath = `${REMOTE_TMP}/argent-${process.pid}-${process.hrtime.bigint()}${suffix}`; + try { + await producer(remotePath); + await hdcFileRecv(connectKey, remotePath, localPath); + } finally { + await runHdcShell(connectKey, `rm -f ${shellQuote(remotePath)}`).catch(() => {}); + } +} + +/** Capture the display to `localPath` as a PNG. */ +export async function harmonyScreenCap(connectKey: string, localPath: string): Promise { + await viaDeviceTmp(connectKey, ".png", localPath, async (remotePath) => { + await runUitest(connectKey, `screenCap -p ${shellQuote(remotePath)}`); + }); +} + +/** A node of the tree `uitest dumpLayout` writes. */ +export interface HarmonyLayoutNode { + attributes: Record; + children?: HarmonyLayoutNode[]; +} + +/** + * The current UI tree. + * + * `-i` is deliberately not passed: without it `uitest` merges the window stack + * into one tree and filters invisible nodes, which is the view a caller asking + * "what is on screen" wants. The unmerged form exposes every background window + * of every app, including ones the user cannot see. + */ +export async function harmonyDumpLayout( + connectKey: string, + localPath: string +): Promise { + await viaDeviceTmp(connectKey, ".json", localPath, async (remotePath) => { + await runUitest(connectKey, `dumpLayout -p ${shellQuote(remotePath)}`); + }); + const raw = await readFile(localPath, "utf8"); + try { + return JSON.parse(raw) as HarmonyLayoutNode; + } catch (err) { + throw new FailureError( + `HarmonyOS device '${connectKey}' returned a layout dump that is not valid JSON ` + + `(${raw.length} bytes).`, + { + error_code: FAILURE_CODES.HARMONY_UITEST_FAILED, + failure_stage: "harmony_dump_layout", + failure_area: "tool_server", + error_kind: "subprocess", + }, + { cause: err as Error } + ); + } +} diff --git a/packages/tool-server/test/harmony-devices.test.ts b/packages/tool-server/test/harmony-devices.test.ts index 173f27af18..22e14836e9 100644 --- a/packages/tool-server/test/harmony-devices.test.ts +++ b/packages/tool-server/test/harmony-devices.test.ts @@ -1,57 +1,145 @@ import { describe, it, expect } from "vitest"; -import { parseHarmonyInstances } from "../src/utils/harmony-devices"; +import { parseHarmonyInstances, parseHdcTargets } from "../src/utils/harmony-devices"; import { + HARMONY_EMULATOR_ID_PREFIX, HARMONY_ID_PREFIX, classifyDevice, + harmonyConnectKey, + harmonyDeviceId, + harmonyEmulatorId, harmonyInstanceName, resolveDevice, } from "../src/utils/device-info"; /** - * Fixtures are verbatim stdout from DevEco Studio 6.1's `Emulator` manager on - * macOS (Emulator 6.1.1.200). + * Fixtures are verbatim stdout from DevEco Studio 6.1's `Emulator` manager + * (6.1.1.200) and from `hdc` 3.2.0d talking to a physical HarmonyOS 6.0.1 + * handset, both on macOS. */ -/** `Emulator -list` with nothing deployed. */ +/** `Emulator -list -details` with nothing deployed. */ const EMULATOR_NO_INSTANCES = "[Empty]\n"; /** - * `Emulator -list` prints this once per directory under ~/.Huawei/Emulator/deployed/ - * that lacks a config.ini, ahead of the listing body. + * Printed once per directory under ~/.Huawei/Emulator/deployed/ that lacks a + * config.ini — ahead of the JSON body, which is why the parser scans forward to + * the first bracket instead of parsing from byte zero. */ const EMULATOR_CONFIG_NOT_FOUND = - 'Config file not found: "/Users/ignacylatka/.Huawei/Emulator/deployed/probe_instance/config.ini"\n'; + 'Config file not found: "/Users/u/.Huawei/Emulator/deployed/zz_broken/config.ini"\n'; -/** The full `Emulator -list` output observed with one such directory and no instances. */ -const EMULATOR_LIST_MISSING_CONFIG = `${EMULATOR_CONFIG_NOT_FOUND}[Empty]\n`; +/** One instance, abridged to the keys the parser reads plus a few it ignores. */ +const EMULATOR_ONE_INSTANCE = `[ + { + "deviceName": "argent_probe", + "deviceType": "Phone", + "hw.hdc.port": "10000", + "isRunning": "false", + "name": "argent_probe", + "os.apiVersion": "24", + "os.osVersion": "HarmonyOS 6.1.1(24)" + } +] +`; + +const EMULATOR_TWO_INSTANCES = `[ + { + "deviceName": "argent_probe", + "deviceType": "Phone", + "isRunning": "false", + "name": "argent_probe", + "os.osVersion": "HarmonyOS 6.1.1(24)" + }, + { + "deviceName": "argent_probe2", + "deviceType": "TV", + "isRunning": "true", + "name": "argent_probe2", + "os.osVersion": "HarmonyOS 6.1.1(24)" + } +] +`; + +/** `hdc list targets -v` with one phone attached. Note the empty second column. */ +const HDC_ONE_DEVICE = "025DEK236V035771\t\tUSB\tConnected\tlocalhost\n"; + +/** `hdc list targets` (and `-v`) with nothing attached. */ +const HDC_NO_DEVICES = "[Empty]\n"; -/** `Emulator -imageList`, abridged to two entries. Note `downloaded` is the *string* "false". */ -/** Same shape with `downloaded` flipped to the string "true". */ -/** `Emulator -imageList -downloaded true` with nothing downloaded — plain prose, not `[Empty]`. */ describe("parseHarmonyInstances", () => { it("returns no instances for the empty sentinel", () => { expect(parseHarmonyInstances(EMULATOR_NO_INSTANCES)).toEqual([]); }); - it("does not read the `Config file not found:` diagnostic as an instance name", () => { - expect(parseHarmonyInstances(EMULATOR_LIST_MISSING_CONFIG)).toEqual([]); + it("reads name, form factor, OS version and running state", () => { + expect(parseHarmonyInstances(EMULATOR_ONE_INSTANCE)).toEqual([ + { + name: "argent_probe", + deviceType: "Phone", + osVersion: "HarmonyOS 6.1.1(24)", + running: false, + }, + ]); + }); + + it("reads isRunning as the string it actually is, not a JSON boolean", () => { + // The manager emits every value as a string, `isRunning` included. Comparing + // it as a boolean would make every instance read as stopped, so a booted + // emulator would be reported as needing a boot. + const [, tv] = parseHarmonyInstances(EMULATOR_TWO_INSTANCES); + expect(tv.running).toBe(true); }); - it("returns a plain name line as an instance", () => { - expect(parseHarmonyInstances("Phone_1\n")).toEqual([{ name: "Phone_1" }]); + it("keeps the instances listed after a config diagnostic", () => { + expect( + parseHarmonyInstances(`${EMULATOR_CONFIG_NOT_FOUND}${EMULATOR_ONE_INSTANCE}`).map( + (i) => i.name + ) + ).toEqual(["argent_probe"]); }); - it("keeps the real instances listed after a config diagnostic", () => { - expect(parseHarmonyInstances(`${EMULATOR_CONFIG_NOT_FOUND}Phone_1\nTv_1\n`)).toEqual([ - { name: "Phone_1" }, - { name: "Tv_1" }, + it("returns both instances when two are deployed", () => { + expect(parseHarmonyInstances(EMULATOR_TWO_INSTANCES).map((i) => i.name)).toEqual([ + "argent_probe", + "argent_probe2", ]); }); + + it("returns no instances rather than throwing on unparseable output", () => { + expect(parseHarmonyInstances("[ this is not json")).toEqual([]); + expect(parseHarmonyInstances("some future banner line\n")).toEqual([]); + }); +}); + +describe("parseHdcTargets", () => { + it("returns no targets for the empty sentinel", () => { + expect(parseHdcTargets(HDC_NO_DEVICES)).toEqual([]); + }); + + it("reads the connect key, transport and state past the empty second column", () => { + // `-v` leaves column 2 blank, so splitting on single tabs would shift every + // field one left and report this connected phone's state as "USB" — a value + // no readiness check matches, hiding a healthy device from the device list. + expect(parseHdcTargets(HDC_ONE_DEVICE)).toEqual([ + { connectKey: "025DEK236V035771", connection: "USB", state: "Connected" }, + ]); + }); + + it("reads the bare non-verbose form as a connected target", () => { + expect(parseHdcTargets("025DEK236V035771\n")).toEqual([ + { connectKey: "025DEK236V035771", connection: null, state: "Connected" }, + ]); + }); + + it("reports a non-connected target's real state", () => { + expect(parseHdcTargets("127.0.0.1:5555\t\tTCP\tOffline\tlocalhost\n")[0].state).toBe("Offline"); + }); }); describe("HarmonyOS device ids", () => { - it("classifies a harmony- prefixed id as harmony", () => { - expect(classifyDevice("harmony-Phone_1")).toBe("harmony"); + it("classifies both harmony id forms as harmony", () => { + expect(classifyDevice("harmony-025DEK236V035771")).toBe("harmony"); + expect(classifyDevice("harmony-emulator-Phone_1")).toBe("harmony"); }); it("does not classify an id without the prefix as harmony", () => { @@ -62,23 +150,29 @@ describe("HarmonyOS device ids", () => { expect(classifyDevice("AAAAAAAA-AAAA-AAAA-AAAA-AAAAAAAAAAAA")).toBe("ios"); }); - it("resolves a harmony id to an emulator", () => { - const d = resolveDevice("harmony-Phone_1"); - expect(d.platform).toBe("harmony"); - expect(d.kind).toBe("emulator"); - expect(d.id).toBe("harmony-Phone_1"); + it("resolves an emulator id to kind emulator and a target id to kind device", () => { + // The two are driven by different CLIs — `Emulator -start` boots an instance, + // `uitest` over hdc drives a connected target — so the kind decides which + // tools accept the id at all. + expect(resolveDevice("harmony-emulator-Phone_1").kind).toBe("emulator"); + expect(resolveDevice("harmony-025DEK236V035771").kind).toBe("device"); }); - it("round-trips the instance name through the id prefix", () => { - expect(`${HARMONY_ID_PREFIX}Phone_1`).toBe("harmony-Phone_1"); - expect(harmonyInstanceName(`${HARMONY_ID_PREFIX}Phone_1`)).toBe("Phone_1"); + it("round-trips an instance name and a connect key through their ids", () => { + expect(harmonyEmulatorId("Phone_1")).toBe(`${HARMONY_EMULATOR_ID_PREFIX}Phone_1`); + expect(harmonyInstanceName(harmonyEmulatorId("Phone_1"))).toBe("Phone_1"); + expect(harmonyDeviceId("025DEK236V035771")).toBe(`${HARMONY_ID_PREFIX}025DEK236V035771`); + expect(harmonyConnectKey(harmonyDeviceId("025DEK236V035771"))).toBe("025DEK236V035771"); }); - it("leaves an unprefixed name alone", () => { - expect(harmonyInstanceName("Phone_1")).toBe("Phone_1"); + it("round-trips an instance whose own name looks like the emulator marker", () => { + // One prefix stripped, not a greedy match: an instance a user named + // `emulator-1` must not come back as `1` and boot the wrong instance. + expect(harmonyInstanceName(harmonyEmulatorId("emulator-1"))).toBe("emulator-1"); }); - it("strips only the leading prefix from an instance named after it", () => { - expect(harmonyInstanceName("harmony-harmony-1")).toBe("harmony-1"); + it("leaves an unprefixed name alone", () => { + expect(harmonyInstanceName("Phone_1")).toBe("Phone_1"); + expect(harmonyConnectKey("025DEK236V035771")).toBe("025DEK236V035771"); }); }); diff --git a/packages/tool-server/test/list-devices-deadline.test.ts b/packages/tool-server/test/list-devices-deadline.test.ts index f2b179eed2..74abec1979 100644 --- a/packages/tool-server/test/list-devices-deadline.test.ts +++ b/packages/tool-server/test/list-devices-deadline.test.ts @@ -6,7 +6,7 @@ import { } from "../src/utils/vega-devices"; import { VVD_PS_PROBE_TIMEOUT_MS } from "../src/utils/vega-process"; import { ADB_DEVICES_TIMEOUT_MS, ENRICH_TIMEOUT_MS } from "../src/utils/adb"; -import { HARMONY_LIST_TIMEOUT_MS } from "../src/utils/harmony-devices"; +import { HARMONY_LIST_TIMEOUT_MS, HDC_LIST_TIMEOUT_MS } from "../src/utils/harmony-devices"; // Guards the timeout/backstop ordering: the backstop must sit ABOVE every branch's // FULL per-call worst case, or it stops being a last-resort and starts truncating a @@ -41,12 +41,16 @@ describe("discovery timeout vs backstop invariant", () => { expect(BRANCH_DEADLINE_MS - androidWorstCase).toBeGreaterThanOrEqual(MIN_MARGIN_MS); }); - it("the HarmonyOS branch's full worst case stays comfortably under the branch deadline", () => { - // A single bounded `Emulator -list`. It must carry its own timeout: the - // wrapper's 30s default is itself above the deadline, so relying on it would - // let the backstop cut off a branch that was still working. - expect(HARMONY_LIST_TIMEOUT_MS).toBeLessThan(BRANCH_DEADLINE_MS); - expect(BRANCH_DEADLINE_MS - HARMONY_LIST_TIMEOUT_MS).toBeGreaterThanOrEqual(MIN_MARGIN_MS); + it("both HarmonyOS branches' full worst case stays comfortably under the branch deadline", () => { + // Two branches, each a single bounded call: `Emulator -list -details` and + // `hdc list targets`. They are separate entries in the same `Promise.all` + // fan-out, so the platform's worst case is the slower of the two, not their + // sum. Each must carry its own timeout: both wrappers default to 30s, itself + // above the deadline, so relying on the default would let the backstop cut + // off a branch that was still working. + const harmonyWorstCase = Math.max(HARMONY_LIST_TIMEOUT_MS, HDC_LIST_TIMEOUT_MS); + expect(harmonyWorstCase).toBeLessThan(BRANCH_DEADLINE_MS); + expect(BRANCH_DEADLINE_MS - harmonyWorstCase).toBeGreaterThanOrEqual(MIN_MARGIN_MS); }); }); From e2cc0639fb1f4aedb9e7c44475afe15ea4ad6052 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Mon, 10 Aug 2026 22:20:47 +0200 Subject: [PATCH 005/110] feat(harmony): drive a HarmonyOS device with the interaction tools MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Eleven tools now accept a HarmonyOS target: describe, screenshot, gesture-tap, gesture-swipe, keyboard, button, launch-app, restart-app, open-url, await-ui-element and await-screen-idle. All of it goes through the device's own `uitest`, which is HarmonyOS' uiautomator, screencap and input in one binary, reached over `hdc`. That is the same shape `button` and `keyboard` already use on Android, where presses go over `adb` rather than the simulator-server HID transport. `describe` parses `uitest dumpLayout` into the shared tree. Each top-level child is a window, labelled with the bundle that owns it, so an app plus the status bar reads as two named windows rather than two anonymous stacks. ArkUI layout scaffolding is walked through unless it is itself clickable — ArkUI builds real buttons out of Stack and Row. A `UIExtensionComponent` renders in another process and the dump carries the node with no children, so it is labelled rather than emitted as an empty container that reads like an empty screen. Two device behaviours drive the error handling. `aa start` exits 0 on failure and puts the useful code on its second line, so success is read off stdout and the coded line is kept. And `aa start -a` does not take `mainAbility`: it takes the matching `abilityInfos` name, which Calculator spells short and Settings spells qualified — passing `mainAbility` verbatim launches one and fails the other with 10104001. `button` exposes home/back/power, the three keys `uitest` names, and `keyboard` exposes the five keycodes whose effect was observed in a real text field. The rest are refused rather than mapped from documentation: `uitest` accepts any keycode and answers `No Error` whatever it does, so an unverified mapping would be indistinguishable from a working one. Verified end to end against a physical HarmonyOS 6.0.1 handset through the tool-server's HTTP API: launch-app, await-screen-idle, describe, four gesture-taps and a second describe assert the calculator reads 15; gesture-swipe moves a Settings list; keyboard types into the search field and backspace removes a character; button home and back change the foreground bundle; open-url's failure path reports the code and its success path carries the caveat that HarmonyOS reports success for a web URL that opens nothing. --- .../src/tools/await-screen-idle/index.ts | 8 +- .../src/tools/await-ui-element/index.ts | 8 +- .../tool-server/src/tools/button/index.ts | 41 +- .../src/tools/describe/contract.ts | 1 + .../src/tools/describe/format-tree.ts | 7 +- .../tool-server/src/tools/describe/index.ts | 9 +- .../tools/describe/platforms/harmony/index.ts | 53 +++ .../platforms/harmony/layout-parser.ts | 361 ++++++++++++++++++ .../src/tools/gesture-swipe/index.ts | 34 +- .../src/tools/gesture-tap/index.ts | 46 ++- .../tool-server/src/tools/keyboard/index.ts | 5 + .../src/tools/keyboard/platforms/harmony.ts | 78 ++++ .../tool-server/src/tools/launch-app/index.ts | 13 +- .../src/tools/launch-app/platforms/harmony.ts | 24 ++ .../tool-server/src/tools/launch-app/types.ts | 1 + .../tool-server/src/tools/open-url/index.ts | 8 +- .../src/tools/open-url/platforms/harmony.ts | 40 ++ .../src/tools/restart-app/index.ts | 13 +- .../tools/restart-app/platforms/harmony.ts | 26 ++ .../src/tools/restart-app/types.ts | 1 + .../tool-server/src/tools/screenshot/index.ts | 19 +- .../tool-server/src/utils/harmony-apps.ts | 229 +++++++++++ .../tool-server/src/utils/harmony-devices.ts | 4 +- packages/tool-server/src/utils/harmony-hdc.ts | 6 +- .../tool-server/src/utils/harmony-screen.ts | 32 ++ .../tool-server/src/utils/harmony-uitest.ts | 52 ++- packages/tool-server/src/utils/png-scale.ts | 31 ++ packages/tool-server/src/utils/vega-screen.ts | 28 +- .../tool-server/test/harmony-apps.test.ts | 186 +++++++++ packages/tool-server/test/harmony-hdc.test.ts | 80 ++++ .../test/harmony-layout-parser.test.ts | 248 ++++++++++++ 31 files changed, 1617 insertions(+), 75 deletions(-) create mode 100644 packages/tool-server/src/tools/describe/platforms/harmony/index.ts create mode 100644 packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts create mode 100644 packages/tool-server/src/tools/keyboard/platforms/harmony.ts create mode 100644 packages/tool-server/src/tools/launch-app/platforms/harmony.ts create mode 100644 packages/tool-server/src/tools/open-url/platforms/harmony.ts create mode 100644 packages/tool-server/src/tools/restart-app/platforms/harmony.ts create mode 100644 packages/tool-server/src/utils/harmony-apps.ts create mode 100644 packages/tool-server/src/utils/harmony-screen.ts create mode 100644 packages/tool-server/src/utils/png-scale.ts create mode 100644 packages/tool-server/test/harmony-apps.test.ts create mode 100644 packages/tool-server/test/harmony-hdc.test.ts create mode 100644 packages/tool-server/test/harmony-layout-parser.test.ts diff --git a/packages/tool-server/src/tools/await-screen-idle/index.ts b/packages/tool-server/src/tools/await-screen-idle/index.ts index fb3bf5dbfd..1d06541054 100644 --- a/packages/tool-server/src/tools/await-screen-idle/index.ts +++ b/packages/tool-server/src/tools/await-screen-idle/index.ts @@ -8,7 +8,7 @@ import type { ToolDefinition, } from "@argent/registry"; import { chromiumCdpRef, type ChromiumCdpApi } from "../../blueprints/chromium-cdp"; -import { resolveDevice } from "../../utils/device-info"; +import { resolveDevice, harmonyConnectKey } from "../../utils/device-info"; import { isTvOsSimulator } from "../../utils/ios-devices"; import { isAndroidTv } from "../../utils/adb"; import { assertSupported } from "../../utils/capability"; @@ -18,6 +18,7 @@ import type { DescribeNode, DescribeTreeData } from "../describe/contract"; import { describeIos, iosRequires } from "../describe/platforms/ios"; import { describeAndroid, androidRequires } from "../describe/platforms/android"; import { describeChromium } from "../describe/platforms/chromium"; +import { describeHarmony, harmonyRequires } from "../describe/platforms/harmony"; export const AWAIT_SCREEN_IDLE_TOOL_ID = "await-screen-idle"; @@ -72,6 +73,7 @@ const capability: ToolCapability = { apple: { simulator: true, device: true }, android: { emulator: true, device: true, unknown: true }, chromium: { app: true }, + harmony: { device: true }, }; // A cheap fingerprint of the screen: role + label + value + frame (rounded to @@ -109,6 +111,9 @@ export function createAwaitScreenIdleTool(registry: Registry): ToolDefinition // `tv-remote` tool, and this tool's capability omits `vega` so a Vega device is // rejected before this map is consulted. Empty set keeps the record total. "vega": new Set([]), - // No tool drives a booted HarmonyOS device yet, and this tool's capability - // omits `harmony`, so a HarmonyOS device is rejected before this map is - // consulted. Empty set keeps the record total. - "harmony": new Set([]), + // `uitest uiInput keyEvent` names exactly these three (`Back`/`Home`/`Power`) + // and otherwise takes a raw numeric keyID. The named three are listed because + // each was confirmed on a device: Power toggles `powerStatus` ON↔SUSPEND, Home + // and Back both move the foreground bundle out of an app and back to the + // launcher. The rest are omitted rather than mapped from documented keycodes — + // `uitest` accepts any number and reports `No Error` whatever it does, so an + // unverified mapping would be indistinguishable from a working one. + "harmony": new Set(["home", "back", "power"]), }; const capability: ToolCapability = { apple: { simulator: true, device: true }, appleRemote: { simulator: true }, android: { emulator: true, device: true, unknown: true }, + harmony: { device: true }, +}; + +/** The names `uitest uiInput keyEvent` accepts, for the buttons it can press. */ +const HARMONY_BUTTON_KEYS: Partial> = { + home: "Home", + back: "Back", + power: "Power", }; export const buttonTool: ToolDefinition = { @@ -64,8 +77,8 @@ export const buttonTool: ToolDefinition = { failedMsg: ({ params, failureSignal }) => `Failed to press ${params.button} button: ${failureSignal.error_code}`, }, - description: `Press a device hardware button (iOS simulator, Android emulator or device). iOS sends a Down then Up event automatically; Android injects a single \`adb\` key event. -Supported buttons depend on the platform: home, back, power, volumeUp, volumeDown, appSwitch, actionButton — buttons not present on the target platform (e.g. 'back' on iOS, 'actionButton' on Android) are rejected with a clear error. + description: `Press a device hardware button (iOS simulator, Android emulator or device, HarmonyOS device). iOS sends a Down then Up event automatically; Android injects a single \`adb\` key event; HarmonyOS injects one \`uitest uiInput keyEvent\`. +Supported buttons depend on the platform: home, back, power, volumeUp, volumeDown, appSwitch, actionButton — buttons not present on the target platform (e.g. 'back' on iOS, 'actionButton' on Android, anything beyond home/back/power on HarmonyOS) are rejected with a clear error. Use when you need to trigger hardware button events. Returns { pressed: buttonName }. Fails if the device backend is not reachable — the simulator-server for iOS, or \`adb\` for Android (Android presses are injected with \`adb shell input keyevent\`).`, @@ -79,7 +92,9 @@ Fails if the device backend is not reachable — the simulator-server for iOS, o // actually consumes it (mirrors the sibling `keyboard` tool's lazy services). services: (params): Record => { const device = resolveDevice(params.udid); - return device.platform === "android" ? {} : { simulatorServer: simulatorServerRef(device) }; + return device.platform === "android" || device.platform === "harmony" + ? {} + : { simulatorServer: simulatorServerRef(device) }; }, async execute(services, params) { const device = resolveDevice(params.udid); @@ -105,6 +120,16 @@ Fails if the device backend is not reachable — the simulator-server for iOS, o await injectAndroidKeycode(params.udid, ANDROID_BUTTON_KEYCODES[params.button]!); return { pressed: params.button }; } + if (device.platform === "harmony") { + // Same reasoning as the Android branch above: presses go over the + // platform's own injection path rather than the simulator-server HID + // transport, so preflight the connector here (services() skips the + // sim-server for HarmonyOS). The BUTTONS_BY_PLATFORM guard above + // guarantees a key name exists for every accepted button. + await ensureDep("hdc"); + await harmonyKeyEvent(harmonyConnectKey(device.id), HARMONY_BUTTON_KEYS[params.button]!); + return { pressed: params.button }; + } const api = services.simulatorServer as SimulatorServerApi; sendCommand(api, { cmd: "button", diff --git a/packages/tool-server/src/tools/describe/contract.ts b/packages/tool-server/src/tools/describe/contract.ts index 004c54fb64..d2de36f046 100644 --- a/packages/tool-server/src/tools/describe/contract.ts +++ b/packages/tool-server/src/tools/describe/contract.ts @@ -85,6 +85,7 @@ export type DescribeSource = | "android-devtools" | "cdp-dom" | "vega-automation" + | "harmony-uitest" | "tv-focus"; // Internal shape produced by the per-platform adapters. The `tree` is consumed diff --git a/packages/tool-server/src/tools/describe/format-tree.ts b/packages/tool-server/src/tools/describe/format-tree.ts index 82d3ef156d..7d91663e32 100644 --- a/packages/tool-server/src/tools/describe/format-tree.ts +++ b/packages/tool-server/src/tools/describe/format-tree.ts @@ -178,13 +178,14 @@ export function formatDescribeTree(root: DescribeNode, opts: FormatDescribeOptio // iOS providers (ax-service, native-devtools) emit a flat list under a // synthetic root, so the flat renderer is correct. Sources that produce // real parent/child trees (uiautomator / android-devtools on Android, - // cdp-dom on Chromium, vega-automation on Vega) use the nested renderer so - // descendants beyond depth 1 are visible. + // cdp-dom on Chromium, vega-automation on Vega, harmony-uitest on HarmonyOS) + // use the nested renderer so descendants beyond depth 1 are visible. const mode: "flat" | "nested" = opts.source === "uiautomator" || opts.source === "android-devtools" || opts.source === "cdp-dom" || - opts.source === "vega-automation" + opts.source === "vega-automation" || + opts.source === "harmony-uitest" ? "nested" : "flat"; const isVega = opts.source === "vega-automation"; diff --git a/packages/tool-server/src/tools/describe/index.ts b/packages/tool-server/src/tools/describe/index.ts index 48cebf2f89..06d602521f 100644 --- a/packages/tool-server/src/tools/describe/index.ts +++ b/packages/tool-server/src/tools/describe/index.ts @@ -13,8 +13,9 @@ import { iosRequires, describeIos } from "./platforms/ios"; import { describeChromium } from "./platforms/chromium"; import { describeTv } from "./platforms/tv"; import { describeVega, vegaRequires } from "./platforms/vega"; +import { describeHarmony, harmonyRequires } from "./platforms/harmony"; import { chromiumCdpRef, type ChromiumCdpApi } from "../../blueprints/chromium-cdp"; -import { resolveDevice } from "../../utils/device-info"; +import { resolveDevice, harmonyConnectKey } from "../../utils/device-info"; import { isTvOsSimulator } from "../../utils/ios-devices"; import { isAndroidTv } from "../../utils/adb"; import { formatDescribeTree } from "./format-tree"; @@ -59,6 +60,7 @@ const capability: ToolCapability = { android: { emulator: true, device: true, unknown: true }, chromium: { app: true }, vega: { vvd: true }, + harmony: { device: true }, }; interface ChromiumServices { @@ -133,6 +135,11 @@ function makeDescribeExecute( requires: vegaRequires, handler: async (_services, params) => withDescription(await describeVega(params.udid)), }, + harmony: { + requires: harmonyRequires, + handler: async (_services, _params, device) => + withDescription(await describeHarmony(harmonyConnectKey(device.id))), + }, }); } diff --git a/packages/tool-server/src/tools/describe/platforms/harmony/index.ts b/packages/tool-server/src/tools/describe/platforms/harmony/index.ts new file mode 100644 index 0000000000..37a4527d90 --- /dev/null +++ b/packages/tool-server/src/tools/describe/platforms/harmony/index.ts @@ -0,0 +1,53 @@ +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { rm } from "node:fs/promises"; +import type { ToolDependency } from "@argent/registry"; +import type { DescribeTreeData } from "../../contract"; +import { harmonyDisplay, harmonyDumpLayout } from "../../../../utils/harmony-uitest"; +import { parseHarmonyLayout } from "./layout-parser"; + +/** `uitest` runs on the device; reaching it needs only the connector. */ +export const harmonyRequires: ToolDependency[] = ["hdc"]; + +/** + * Said when the dump has no window at all. `uitest` returns a tree with a bare + * root while the display is asleep, and an agent that reads that as "the app + * rendered nothing" will start debugging the app instead of waking the screen. + */ +const ASLEEP_HINT = + "The display is off, so there is nothing to describe and injected taps would land nowhere. " + + "Wake it with `button` (power) and describe again."; + +const EMPTY_HINT = + "The layout dump contains no windows. The foreground app may still be starting — " + + "call describe again, or screenshot to see what is on screen."; + +/** + * Describe the current HarmonyOS screen from `uitest dumpLayout`. + * + * The dump is written on the device and copied back, so it needs somewhere to + * land on the host; the file is temporary and removed once parsed, since + * describe's contract is the rendered text and nothing downstream reads it. + */ +export async function describeHarmony(connectKey: string): Promise { + const localPath = join(tmpdir(), `argent-harmony-dump-${process.hrtime.bigint()}.json`); + const display = await harmonyDisplay(connectKey); + try { + const raw = await harmonyDumpLayout(connectKey, localPath); + const { tree, screen } = parseHarmonyLayout(raw, { + width: display.width, + height: display.height, + }); + if (tree.children.length === 0) { + return { + tree, + source: "harmony-uitest", + screen, + hint: display.screenOn ? EMPTY_HINT : ASLEEP_HINT, + }; + } + return { tree, source: "harmony-uitest", screen }; + } finally { + await rm(localPath, { force: true }).catch(() => {}); + } +} diff --git a/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts b/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts new file mode 100644 index 0000000000..d5fe6c691d --- /dev/null +++ b/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts @@ -0,0 +1,361 @@ +import type { DescribeNode } from "../../contract"; +import type { HarmonyLayoutNode } from "../../../../utils/harmony-uitest"; + +/** + * Turn a `uitest dumpLayout` tree into the shared describe tree. + * + * The dump is ArkUI's component hierarchy, and it is shaped much like a + * uiautomator dump — `[x1,y1][x2,y2]` bounds, `clickable`/`checkable` flags, a + * `type` that plays the part of Android's `class` — so this mirrors the Android + * trimmer's structure. It is markedly shorter for one measured reason: `uitest` + * already merges the window stack and drops invisible nodes before writing, so + * the scroll-clip pruning the Android side needs has nothing to prune (176 + * nodes for a full Settings screen, none of them `visible: false`). + * + * What it must handle that Android's does not: + * + * - **Every value is a string**, `"true"`/`"false"` included. + * - **`description` is often a single space** rather than empty — the + * calculator's keypad labels every Button `" "`. Trimming is what stops each + * one becoming a blank-labelled node that reads as a real label. + * - **Windows are top-level children**, each carrying `bundleName` / + * `abilityName`, so which app owns which subtree is knowable and worth + * surfacing. + * - **`UIExtensionComponent` subtrees are opaque.** System extension UI — the + * "open with" app selector, share sheets — renders in another process, and + * the dump contains the node with no children. Silently emitting an empty + * container would tell an agent the screen is empty when it is covered by a + * dialog it cannot see, so these are labelled. + */ + +/** Bounds arrive as `[left,top][right,bottom]`, in device pixels. */ +export function parseHarmonyBounds(bounds: string): PixelRect | null { + const m = /\[(-?\d+),(-?\d+)]\[(-?\d+),(-?\d+)]/.exec(bounds); + if (!m) return null; + const [x1, y1, x2, y2] = m.slice(1, 5).map((v) => Number.parseInt(v, 10)); + return { x: x1, y: y1, w: x2 - x1, h: y2 - y1 }; +} + +interface PixelRect { + x: number; + y: number; + w: number; + h: number; +} + +const isTrue = (attrs: Record, key: string): boolean => attrs[key] === "true"; + +/** + * ArkUI layout primitives and framework scaffolding: real nodes in the tree, + * but never a tap target and never worth a line of an agent's context. Walked + * through rather than emitted, exactly as the Android trimmer treats + * `FrameLayout`/`LinearLayout`. + */ +const LAYOUT_CONTAINERS = new Set([ + "", + "Column", + "Row", + "Stack", + "Flex", + "RelativeContainer", + "__Common__", + "EffectComponent", + "Navigation", + "NavBar", + "NavBarContent", + "WindowScene", + "root", + "JsView", +]); + +/** + * Pure decoration with no label and no behaviour. Dropped outright rather than + * walked through — unlike the containers above, these never carry a subtree + * worth keeping. + */ +const DECORATIONS = new Set(["Divider", "ScrollBar", "metaballNode"]); + +/** + * Renders in another process, so the dump carries the node and none of its + * content. Kept, and labelled, so the emptiness is visible as a fact rather + * than as an absence. + */ +const OPAQUE_EXTENSION = "UIExtensionComponent"; + +/** ArkUI containers that scroll their content. */ +const SCROLL_TYPES = new Set(["List", "Grid", "Scroll", "Swiper", "WaterFlow"]); + +function isScrollable(attrs: Record): boolean { + return SCROLL_TYPES.has(attrs.type ?? "") || isTrue(attrs, "scrollable"); +} + +/** + * The screen-reader-meaningful label. + * + * `description` (ArkUI's `accessibilityText`) wins over `text` for the same + * reason `content-desc` does on Android: when both are set they describe + * different things, and the contract carries the typed content separately as + * `value`. `hint` is the placeholder of an empty field, which is the only label + * such a field has. + */ +export function harmonyLabel(attrs: Record): string { + const description = (attrs.description ?? "").trim(); + if (description) return description; + const text = (attrs.text ?? "").trim(); + if (text) return text; + return (attrs.hint ?? "").trim(); +} + +function isInteractive(attrs: Record): boolean { + return ( + isTrue(attrs, "clickable") || + isTrue(attrs, "longClickable") || + isTrue(attrs, "checkable") || + isScrollable(attrs) + ); +} + +interface HarmonyNode { + role: string; + pixelBounds: PixelRect | null; + label?: string; + identifier?: string; + value?: string; + clickable: boolean; + longClickable: boolean; + scrollable: boolean; + checkable: boolean; + checked: boolean; + disabled: boolean; + focused: boolean; + selected: boolean; + children: HarmonyNode[]; +} + +/** A node with no size cannot be tapped and cannot be seen. */ +function hasArea(b: PixelRect | null): boolean { + return b !== null && b.w > 0 && b.h > 0; +} + +function rectsEqual(a: PixelRect, b: PixelRect): boolean { + return a.x === b.x && a.y === b.y && a.w === b.w && a.h === b.h; +} + +function build(node: HarmonyLayoutNode): HarmonyNode[] { + const attrs = node.attributes ?? {}; + const type = attrs.type ?? ""; + const bounds = parseHarmonyBounds(attrs.bounds ?? ""); + const children = (node.children ?? []).flatMap(build); + + if (DECORATIONS.has(type)) return []; + + if (type === OPAQUE_EXTENSION) { + return [ + { + ...blank(attrs, bounds), + role: "SystemOverlay", + label: + "[system UI rendered in another process — its contents are not in the layout dump; " + + "screenshot to see it]", + children: [], + }, + ]; + } + + const label = harmonyLabel(attrs); + const interactive = isInteractive(attrs); + + // Scaffolding contributes nothing itself; hoist whatever survived beneath it. + // A container that is *itself* clickable or labelled is a real target wearing + // a layout type (ArkUI builds buttons out of `Stack`/`Row`), so it stays. + if (LAYOUT_CONTAINERS.has(type) && !interactive && !label) { + return children; + } + + if (!hasArea(bounds) && children.length === 0) return []; + + const own: HarmonyNode = { + ...blank(attrs, bounds), + role: deriveRole(type, attrs), + children, + }; + if (label) own.label = label; + // Only surface `text` separately when `description` already claimed the label + // slot and the two genuinely differ — otherwise `value` just repeats `label`. + const text = (attrs.text ?? "").trim(); + if (text && text !== own.label) own.value = text; + const identifier = (attrs.id ?? "").trim() || (attrs.key ?? "").trim(); + if (identifier) own.identifier = identifier; + + // A wrapper whose only child covers exactly the same rect is a duplicate + // layer: collapse to whichever of the two carries the label. + if (own.children.length === 1 && own.pixelBounds) { + const only = own.children[0]; + if (only.pixelBounds && rectsEqual(only.pixelBounds, own.pixelBounds) && !own.label) { + return [{ ...only, clickable: only.clickable || own.clickable }]; + } + } + + return [own]; +} + +function blank(attrs: Record, bounds: PixelRect | null): HarmonyNode { + return { + role: "Group", + pixelBounds: bounds, + clickable: isTrue(attrs, "clickable"), + longClickable: isTrue(attrs, "longClickable"), + scrollable: isScrollable(attrs), + checkable: isTrue(attrs, "checkable"), + checked: isTrue(attrs, "checked"), + // `enabled` is present on every node, so a missing value means the dump + // changed shape — treat that as enabled rather than reporting the whole + // screen as greyed out. + disabled: attrs.enabled === "false", + focused: isTrue(attrs, "focused"), + selected: isTrue(attrs, "selected"), + children: [], + }; +} + +/** + * Map an ArkUI component type onto the role vocabulary the describe formatter + * and the agent-facing guidance already use, so a HarmonyOS tree reads like an + * iOS or Android one. + */ +function deriveRole(type: string, attrs: Record): string { + switch (type) { + case "Text": + case "Span": + case "TextClock": + return "StaticText"; + case "TextInput": + case "TextArea": + case "SearchField": + case "Search": + return "TextField"; + case "Image": + case "SymbolGlyph": + return "Image"; + case "Toggle": + return attrs.checkable === "true" ? "Switch" : "Button"; + case "Checkbox": + return "Checkbox"; + case "Slider": + return "Slider"; + case "List": + case "Grid": + case "Scroll": + case "WaterFlow": + return "ScrollView"; + case "ListItem": + case "GridItem": + return "Cell"; + case "Swiper": + return "Pager"; + case "Dialog": + return "Dialog"; + case "Button": + return "Button"; + default: + // An unmapped type is still better information than a generic label: it + // is the component the app author actually wrote. + return type || "Group"; + } +} + +function toDescribeNode(n: HarmonyNode, screenW: number, screenH: number): DescribeNode { + const children = n.children.map((c) => toDescribeNode(c, screenW, screenH)); + const b = n.pixelBounds; + const frame = b + ? { + // Clamp into [0,1]: ArkUI reports off-screen bounds for items scrolled + // past the edge of a List, and the describe frame contract is closed. + x: clamp01(b.x / screenW), + y: clamp01(b.y / screenH), + width: clamp01(b.w / screenW), + height: clamp01(b.h / screenH), + } + : unionFrame(children); + const out: DescribeNode = { role: n.role, frame, children }; + if (n.label) out.label = n.label; + if (n.identifier) out.identifier = n.identifier; + if (n.value) out.value = n.value; + if (n.clickable) out.clickable = true; + if (n.longClickable) out.longClickable = true; + if (n.scrollable) out.scrollable = true; + if (n.checkable) out.checkable = true; + if (n.checked) out.checked = true; + if (n.disabled) out.disabled = true; + if (n.focused) out.focused = true; + if (n.selected) out.selected = true; + return out; +} + +function clamp01(v: number): number { + if (!Number.isFinite(v)) return 0; + return Math.max(0, Math.min(1, v)); +} + +function unionFrame(children: DescribeNode[]): DescribeNode["frame"] { + if (children.length === 0) return { x: 0, y: 0, width: 0, height: 0 }; + const x1 = Math.min(...children.map((c) => c.frame.x)); + const y1 = Math.min(...children.map((c) => c.frame.y)); + const x2 = Math.max(...children.map((c) => c.frame.x + c.frame.width)); + const y2 = Math.max(...children.map((c) => c.frame.y + c.frame.height)); + return { x: x1, y: y1, width: Math.max(0, x2 - x1), height: Math.max(0, y2 - y1) }; +} + +interface HarmonyTreeResult { + tree: DescribeNode; + screen: { width: number; height: number }; +} + +/** + * Build the describe tree. + * + * The screen size comes from the dump's own root bounds rather than a separate + * device query: the two are read at different instants, and on a foldable that + * is enough for a rotation between them to normalise every frame against the + * wrong axis. `fallback` covers a root that reports no bounds. + */ +export function parseHarmonyLayout( + root: HarmonyLayoutNode, + fallback: { width: number; height: number } +): HarmonyTreeResult { + const rootBounds = parseHarmonyBounds(root.attributes?.bounds ?? ""); + const screen = + rootBounds && rootBounds.w > 0 && rootBounds.h > 0 + ? { width: rootBounds.w, height: rootBounds.h } + : fallback; + + // Each top-level child is a window. Label it with the app that owns it so a + // tree spanning an app plus the status bar (always a separate `sceneboard` + // window) is readable rather than two anonymous stacks. + const windows: DescribeNode[] = []; + for (const child of root.children ?? []) { + const attrs = child.attributes ?? {}; + const bundle = (attrs.bundleName ?? "").trim(); + const built = build(child).map((n) => toDescribeNode(n, screen.width, screen.height)); + if (built.length === 0) continue; + if (!bundle) { + windows.push(...built); + continue; + } + windows.push({ + role: "Window", + label: bundle, + frame: unionFrame(built), + children: built, + }); + } + + return { + tree: { + role: "Screen", + frame: { x: 0, y: 0, width: 1, height: 1 }, + children: windows, + }, + screen, + }; +} diff --git a/packages/tool-server/src/tools/gesture-swipe/index.ts b/packages/tool-server/src/tools/gesture-swipe/index.ts index a7724dadc2..5b3885f049 100644 --- a/packages/tool-server/src/tools/gesture-swipe/index.ts +++ b/packages/tool-server/src/tools/gesture-swipe/index.ts @@ -1,7 +1,9 @@ import { z } from "zod"; -import type { ToolCapability, ToolDefinition } from "@argent/registry"; +import type { ServiceRef, ToolCapability, ToolDefinition } from "@argent/registry"; import { simulatorServerRef, type SimulatorServerApi } from "../../blueprints/simulator-server"; -import { resolveDevice } from "../../utils/device-info"; +import { resolveDevice, harmonyConnectKey } from "../../utils/device-info"; +import { harmonySwipeNormalized } from "../../utils/harmony-uitest"; +import { ensureDep } from "../../utils/check-deps"; import { sendCommand } from "../../utils/simulator-client"; const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)); @@ -45,6 +47,7 @@ const capability: ToolCapability = { apple: { simulator: true, device: true }, appleRemote: { simulator: true }, android: { emulator: true, device: true, unknown: true }, + harmony: { device: true }, }; export const gestureSwipeTool: ToolDefinition = { @@ -56,7 +59,7 @@ export const gestureSwipeTool: ToolDefinition = { `Swiped from (${Math.round(params.fromX * 100)}%, ${Math.round(params.fromY * 100)}%) to (${Math.round(params.toX * 100)}%, ${Math.round(params.toY * 100)}%)`, failedMsg: ({ failureSignal }) => `Failed to swipe: ${failureSignal.error_code}`, }, - description: `Execute a smooth swipe / drag touch gesture between two points on the device (iOS simulator or Android emulator). All from/to positions are normalized 0.0–1.0 (fractions of screen width/height, not pixels), same as gesture-tap. + description: `Execute a smooth swipe / drag touch gesture between two points on the device (iOS simulator, Android emulator, or HarmonyOS device). All from/to positions are normalized 0.0–1.0 (fractions of screen width/height, not pixels), same as gesture-tap. Generates interpolated Move events for a natural feel (~60fps). Swipe up (fromY > toY) to scroll content down. Use when you need to scroll a list, dismiss a modal, drag an element, or navigate between pages. Not supported on Chromium — use gesture-scroll there instead. @@ -65,13 +68,32 @@ Pass settle:true for a momentum-free swipe that lands exactly where the finger l searchHint: "swipe scroll drag pan gesture device simulator emulator touch move", zodSchema, capability, - services: (params) => ({ - simulatorServer: simulatorServerRef(resolveDevice(params.udid)), - }), + services: (params): Record => { + const device = resolveDevice(params.udid); + // HarmonyOS swipes go over hdc; resolving the iOS/Android-only blueprint + // would spawn a backend this path never uses and wait on it to come up. + if (device.platform === "harmony") return {}; + return { simulatorServer: simulatorServerRef(device) }; + }, async execute(services, params) { const duration = params.durationMs ?? 300; const settle = params.settle ?? false; const timestampMs = Date.now(); + const device = resolveDevice(params.udid); + // HarmonyOS has no simulator-server controller: the whole gesture is one + // `uitest uiInput` call, which owns its own interpolation on-device, so + // there is no per-frame Move train to emit here. + if (device.platform === "harmony") { + await ensureDep("hdc"); + await harmonySwipeNormalized( + harmonyConnectKey(device.id), + { x: params.fromX, y: params.fromY }, + { x: params.toX, y: params.toY }, + duration, + settle + ); + return { swiped: true, timestampMs }; + } const api = services.simulatorServer as SimulatorServerApi; const steps = Math.max(1, Math.round(duration / 16)); diff --git a/packages/tool-server/src/tools/gesture-tap/index.ts b/packages/tool-server/src/tools/gesture-tap/index.ts index 7a55e9eb26..2c1883f00a 100644 --- a/packages/tool-server/src/tools/gesture-tap/index.ts +++ b/packages/tool-server/src/tools/gesture-tap/index.ts @@ -3,7 +3,9 @@ import type { ServiceRef, ToolCapability, ToolDefinition } from "@argent/registr import { simulatorServerRef, type SimulatorServerApi } from "../../blueprints/simulator-server"; import { chromiumCdpRef, type ChromiumCdpApi } from "../../blueprints/chromium-cdp"; import { assertChromiumWindowVisible } from "../../utils/chromium-visibility"; -import { resolveDevice } from "../../utils/device-info"; +import { resolveDevice, harmonyConnectKey } from "../../utils/device-info"; +import { harmonyDisplay, harmonyTouch, toDevicePoint } from "../../utils/harmony-uitest"; +import { ensureDep } from "../../utils/check-deps"; import { sendCommand } from "../../utils/simulator-client"; const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)); @@ -54,6 +56,7 @@ const capability: ToolCapability = { appleRemote: { simulator: true }, android: { emulator: true, device: true, unknown: true }, chromium: { app: true }, + harmony: { device: true }, }; // One press-release is 50ms; taps in a multi-tap gesture are 100ms apart — @@ -82,6 +85,36 @@ async function tapChromium( } } +/** + * HarmonyOS taps go through the device's own `uitest uiInput`, not + * simulator-server — the platform has no simulator-server controller, and + * `uitest` is the vendor's supported injection path. This mirrors how `button` + * and `keyboard` already reach Android through `adb` rather than the HID + * transport. + * + * `uitest` has a native double-click, so a 2-tap request uses it rather than two + * timed clicks: two separate injections are not guaranteed to land inside the + * OS double-tap window, which is the whole point of `clickCount`. Counts above 2 + * have no native form and fall back to repeated clicks. + */ +async function tapHarmony( + connectKey: string, + x: number, + y: number, + clickCount: number +): Promise { + const display = await harmonyDisplay(connectKey); + const point = toDevicePoint(x, y, display); + if (clickCount === 2) { + await harmonyTouch(connectKey, "doubleClick", point); + return; + } + for (let i = 0; i < clickCount; i++) { + if (i > 0) await sleep(MULTI_TAP_GAP_MS); + await harmonyTouch(connectKey, "click", point); + } +} + export const gestureTapTool: ToolDefinition = { id: "gesture-tap", interaction: { @@ -90,7 +123,7 @@ export const gestureTapTool: ToolDefinition = { failedMsg: ({ params, failureSignal }) => `Failed to tap at (${Math.round(params.x * 100)}%, ${Math.round(params.y * 100)}%): ${failureSignal.error_code}`, }, - description: `Press the device screen (iOS simulator, Android emulator, or Chromium app) at normalized coordinates: x and y are fractions of screen width and height in 0.0–1.0 (not pixels). + description: `Press the device screen (iOS simulator, Android emulator, HarmonyOS device, or Chromium app) at normalized coordinates: x and y are fractions of screen width and height in 0.0–1.0 (not pixels). Sends a Down event followed by an Up event at the same point. For Chromium, this dispatches a CDP mouse-press/release on the renderer. Set clickCount: 2 for a double-tap / double-click — the taps are dispatched as one gesture with proper click counting, which two separate tap calls cannot guarantee. Use when you need to tap a button, link, or any tappable element on the screen. @@ -105,6 +138,10 @@ Before tapping, determine the correct coordinates by using discovery tools — p if (device.platform === "chromium") { return { chromium: chromiumCdpRef(device) }; } + // HarmonyOS drives `uitest` over hdc, so resolving the iOS/Android-only + // simulator-server blueprint here would spawn a backend the tap never uses + // and block on its ready-wait. + if (device.platform === "harmony") return {}; return { simulatorServer: simulatorServerRef(device) }; }, async execute(services, params) { @@ -119,6 +156,11 @@ Before tapping, determine the correct coordinates by using discovery tools — p await tapChromium(chromium, params.x, params.y, clickCount); return { tapped: true, timestampMs }; } + if (device.platform === "harmony") { + await ensureDep("hdc"); + await tapHarmony(harmonyConnectKey(device.id), params.x, params.y, clickCount); + return { tapped: true, timestampMs }; + } const api = services.simulatorServer as SimulatorServerApi; for (let i = 1; i <= clickCount; i++) { if (i > 1) await sleep(MULTI_TAP_GAP_MS); diff --git a/packages/tool-server/src/tools/keyboard/index.ts b/packages/tool-server/src/tools/keyboard/index.ts index 3677c95915..4d985235cc 100644 --- a/packages/tool-server/src/tools/keyboard/index.ts +++ b/packages/tool-server/src/tools/keyboard/index.ts @@ -7,6 +7,7 @@ import { makeIosImpl, makeIosRemoteImpl } from "./platforms/ios"; import { makeAndroidImpl } from "./platforms/android"; import { makeChromiumImpl } from "./platforms/chromium"; import { vegaImpl } from "./platforms/vega"; +import { harmonyImpl } from "./platforms/harmony"; const zodSchema = z.object({ udid: z @@ -50,6 +51,7 @@ const capability: ToolCapability = { android: { emulator: true, device: true, unknown: true }, chromium: { app: true }, vega: { vvd: true }, + harmony: { device: true }, }; // `keyboard` goes through `dispatchByPlatform`. The chromium branch resolves the @@ -69,6 +71,8 @@ export function createKeyboardTool(registry: Registry): ToolDefinition, + Record, + Record, Record >({ toolId: "keyboard", @@ -78,6 +82,7 @@ export function createKeyboardTool(registry: Registry): ToolDefinition = { + "enter": 2054, + "backspace": 2055, + "space": 2050, + "arrow-left": 2014, + "arrow-right": 2015, +}; + +function resolveHarmonyKeycode(key: string): number { + const code = HARMONY_KEYCODES[key]; + if (code === undefined) { + throw new InvalidToolInputError( + `Key '${key}' is not available on HarmonyOS. Supported: ` + + `${Object.keys(HARMONY_KEYCODES).join(", ")}.`, + { + error_code: FAILURE_CODES.KEYBOARD_KEY_UNSUPPORTED, + failure_stage: "harmony_keyboard_resolve_key", + } + ); + } + return code; +} + +async function typeHarmony(connectKey: string, params: KeyboardParams): Promise { + // Resolve before injecting anything: a combined key+text call with an unknown + // key name must reject having typed nothing, not type the text and then 400. + const keycode = params.key ? resolveHarmonyKeycode(params.key) : null; + let keysPressed = 0; + if (params.text) { + // `uitest uiInput text` types into whatever holds focus, in one shot — there + // is no per-character injection, so `delayMs` has nothing to pace (the tool + // description already lists the platforms that ignore it). + await harmonyTypeText(connectKey, params.text); + keysPressed += [...params.text].length; + } + // Key after text, so a combined call means "type, then submit". Pressing first + // would submit an empty field. + if (keycode !== null) { + await harmonyKeyEvent(connectKey, String(keycode)); + keysPressed++; + } + return { typed: params.text ?? params.key ?? "", keys: keysPressed }; +} + +/** + * HarmonyOS has no simulator-server controller; typing goes to the device's own + * `uitest`, reached over `hdc`. Declaring the dependency here lets + * `dispatchByPlatform` preflight it and return a clean 424 install hint. + */ +export const harmonyImpl: PlatformImpl, KeyboardParams, KeyboardResult> = { + requires: ["hdc"], + handler: (_services, params, device) => typeHarmony(harmonyConnectKey(device.id), params), +}; diff --git a/packages/tool-server/src/tools/launch-app/index.ts b/packages/tool-server/src/tools/launch-app/index.ts index 4dc12a9d1e..08e29a5880 100644 --- a/packages/tool-server/src/tools/launch-app/index.ts +++ b/packages/tool-server/src/tools/launch-app/index.ts @@ -4,12 +4,18 @@ import { chromiumCdpRef } from "../../blueprints/chromium-cdp"; import { nativeDevtoolsRef } from "../../blueprints/native-devtools"; import { resolveDevice } from "../../utils/device-info"; import { dispatchByPlatform } from "../../utils/cross-platform-tool"; -import type { LaunchAppResult, LaunchAppVegaServices, LaunchAppIosServices } from "./types"; +import type { + LaunchAppResult, + LaunchAppVegaServices, + LaunchAppIosServices, + LaunchAppHarmonyServices, +} from "./types"; import { makeIosImpl } from "./platforms/ios"; import { iosRemoteImpl } from "./platforms/ios-remote"; import { androidImpl } from "./platforms/android"; import { chromiumImpl, type LaunchAppChromiumServices } from "./platforms/chromium"; import { vegaImpl } from "./platforms/vega"; +import { harmonyImpl } from "./platforms/harmony"; // Android package grammar is `[A-Za-z_][A-Za-z0-9_]*(\.[A-Za-z_][A-Za-z0-9_]*)+`; // iOS bundle ids use the same reverse-DNS shape with dashes allowed. The union @@ -51,6 +57,7 @@ const capability: ToolCapability = { android: { emulator: true, device: true, unknown: true }, chromium: { app: true }, vega: { vvd: true }, + harmony: { device: true }, }; // `launch-app` resolves native-devtools through `registry` inside the iOS @@ -102,7 +109,8 @@ Common Android packages: com.android.settings, com.android.chrome, com.google.an LaunchAppResult, LaunchAppChromiumServices, LaunchAppVegaServices, - LaunchAppIosServices + LaunchAppIosServices, + LaunchAppHarmonyServices >({ toolId: "launch-app", capability, @@ -111,6 +119,7 @@ Common Android packages: com.android.settings, com.android.chrome, com.google.an android: androidImpl, chromium: chromiumImpl, vega: vegaImpl, + harmony: harmonyImpl, }), }; } diff --git a/packages/tool-server/src/tools/launch-app/platforms/harmony.ts b/packages/tool-server/src/tools/launch-app/platforms/harmony.ts new file mode 100644 index 0000000000..dcca126ce0 --- /dev/null +++ b/packages/tool-server/src/tools/launch-app/platforms/harmony.ts @@ -0,0 +1,24 @@ +import type { PlatformImpl } from "../../../utils/cross-platform-tool"; +import { harmonyConnectKey } from "../../../utils/device-info"; +import { launchHarmonyApp } from "../../../utils/harmony-apps"; +import type { LaunchAppParams, LaunchAppResult, LaunchAppHarmonyServices } from "../types"; + +/** + * HarmonyOS launches an app by bundle name, but not by bundle name alone: + * `aa start -b ` is an *implicit* start and fails without a matching + * action, so `launchHarmonyApp` looks the entry ability up first. `activity` is + * Android-only and ignored — the HarmonyOS equivalent is the ability, which is + * resolved rather than accepted from the caller, because a bundle's launcher + * entry is a property of the bundle and not something a caller should have to + * know. + * + * No native-devtools injection exists for the platform, so no service is needed. + */ +export const harmonyImpl: PlatformImpl = + { + requires: ["hdc"], + handler: async (_services, params, device) => { + await launchHarmonyApp(harmonyConnectKey(device.id), params.bundleId); + return { launched: true, bundleId: params.bundleId }; + }, + }; diff --git a/packages/tool-server/src/tools/launch-app/types.ts b/packages/tool-server/src/tools/launch-app/types.ts index d3938673f0..1a4015c505 100644 --- a/packages/tool-server/src/tools/launch-app/types.ts +++ b/packages/tool-server/src/tools/launch-app/types.ts @@ -22,3 +22,4 @@ export interface LaunchAppIosServices { } export type LaunchAppAndroidServices = Record; export type LaunchAppVegaServices = Record; +export type LaunchAppHarmonyServices = Record; diff --git a/packages/tool-server/src/tools/open-url/index.ts b/packages/tool-server/src/tools/open-url/index.ts index 439a3913bc..6473af52bb 100644 --- a/packages/tool-server/src/tools/open-url/index.ts +++ b/packages/tool-server/src/tools/open-url/index.ts @@ -6,6 +6,7 @@ import { chromiumCdpRef } from "../../blueprints/chromium-cdp"; import type { OpenUrlResult, OpenUrlServices } from "./types"; import { iosImpl } from "./platforms/ios"; import { androidImpl } from "./platforms/android"; +import { harmonyImpl } from "./platforms/harmony"; import { iosRemoteImpl } from "./platforms/ios-remote"; import { chromiumImpl, type OpenUrlChromiumServices } from "./platforms/chromium"; @@ -41,6 +42,7 @@ const capability: ToolCapability = { appleRemote: { simulator: true }, android: { emulator: true, device: true, unknown: true }, chromium: { app: true }, + harmony: { device: true }, }; export const openUrlTool: ToolDefinition = { @@ -70,12 +72,16 @@ Returns { opened, url, note? }. note carries the deep-linking caveat when a web OpenUrlServices, Params, OpenUrlResult, - OpenUrlChromiumServices + OpenUrlChromiumServices, + OpenUrlServices, + OpenUrlServices, + OpenUrlServices >({ toolId: "open-url", capability, ios: iosImpl, android: androidImpl, + harmony: harmonyImpl, iosRemote: iosRemoteImpl, chromium: chromiumImpl, }), diff --git a/packages/tool-server/src/tools/open-url/platforms/harmony.ts b/packages/tool-server/src/tools/open-url/platforms/harmony.ts new file mode 100644 index 0000000000..11f93e1aaf --- /dev/null +++ b/packages/tool-server/src/tools/open-url/platforms/harmony.ts @@ -0,0 +1,40 @@ +import type { PlatformImpl } from "../../../utils/cross-platform-tool"; +import { harmonyConnectKey } from "../../../utils/device-info"; +import { openHarmonyUrl } from "../../../utils/harmony-apps"; +import type { OpenUrlParams, OpenUrlResult, OpenUrlServices } from "../types"; +import { httpDeepLinkNote } from "../deep-link-note"; + +/** + * `aa start -U ` is HarmonyOS' implicit open — the system picks the + * handler, which is exactly what this tool wants (and the one place implicit + * start is the right verb; see `harmony-apps`). + * + * A **custom scheme** with no registered handler is caught: `aa` prints + * `10103101 Failed to find a matching application for implicit launch`, which + * `openHarmonyUrl` turns into a throw. (It also leaves a "No options to open + * with" chooser on the device, which the thrown message says.) + * + * A **web URL** is not, and cannot be. Measured on HarmonyOS 6.0.1: `aa start + * -U https://example.com` prints `start ability successfully.` and the + * foreground app does not change — no browser opens, no chooser appears, and + * nothing on the device distinguishes that from a handled link. So the caveat is + * stated rather than papered over; `opened: true` here means "the system + * accepted the URI", which on this platform is weaker than it sounds. + */ +const HARMONY_WEB_URL_CAVEAT = + "On HarmonyOS specifically, `aa start -U` reports success for a web URL even when nothing " + + "opens it — a device with no browser registered for https shows no chooser and stays on the " + + "current screen. Confirm with describe or screenshot before treating the link as followed."; + +export const harmonyImpl: PlatformImpl = { + requires: ["hdc"], + handler: async (_services, params, device) => { + await openHarmonyUrl(harmonyConnectKey(device.id), params.url); + const shared = httpDeepLinkNote(params.url); + return { + opened: true, + url: params.url, + note: shared ? `${shared} ${HARMONY_WEB_URL_CAVEAT}` : undefined, + }; + }, +}; diff --git a/packages/tool-server/src/tools/restart-app/index.ts b/packages/tool-server/src/tools/restart-app/index.ts index 49fd37f759..37b254e9d9 100644 --- a/packages/tool-server/src/tools/restart-app/index.ts +++ b/packages/tool-server/src/tools/restart-app/index.ts @@ -3,11 +3,17 @@ import type { Registry, ServiceRef, ToolCapability, ToolDefinition } from "@arge import { nativeDevtoolsRef } from "../../blueprints/native-devtools"; import { resolveDevice } from "../../utils/device-info"; import { dispatchByPlatform } from "../../utils/cross-platform-tool"; -import type { RestartAppResult, RestartAppVegaServices, RestartAppIosServices } from "./types"; +import type { + RestartAppResult, + RestartAppVegaServices, + RestartAppIosServices, + RestartAppHarmonyServices, +} from "./types"; import { makeIosImpl } from "./platforms/ios"; import { iosRemoteImpl } from "./platforms/ios-remote"; import { androidImpl } from "./platforms/android"; import { vegaImpl } from "./platforms/vega"; +import { harmonyImpl } from "./platforms/harmony"; // Bundle id / package name. Head must be letter or underscore so a bundleId // like `--user` can't masquerade as a flag inside `am force-stop …`. @@ -43,6 +49,7 @@ const capability: ToolCapability = { appleRemote: { simulator: true }, android: { emulator: true, device: true, unknown: true }, vega: { vvd: true }, + harmony: { device: true }, }; // `restart-app` resolves native-devtools through `registry` inside the iOS @@ -89,7 +96,8 @@ Returns { restarted, bundleId }. Fails if the app is not installed.`, // No chromium branch — falls back to the ChromiumServices default. Record, RestartAppVegaServices, - RestartAppIosServices + RestartAppIosServices, + RestartAppHarmonyServices >({ toolId: "restart-app", capability, @@ -97,6 +105,7 @@ Returns { restarted, bundleId }. Fails if the app is not installed.`, iosRemote: iosRemoteImpl, android: androidImpl, vega: vegaImpl, + harmony: harmonyImpl, }), }; } diff --git a/packages/tool-server/src/tools/restart-app/platforms/harmony.ts b/packages/tool-server/src/tools/restart-app/platforms/harmony.ts new file mode 100644 index 0000000000..fcaa1d9cb5 --- /dev/null +++ b/packages/tool-server/src/tools/restart-app/platforms/harmony.ts @@ -0,0 +1,26 @@ +import type { PlatformImpl } from "../../../utils/cross-platform-tool"; +import { harmonyConnectKey } from "../../../utils/device-info"; +import { launchHarmonyApp, terminateHarmonyApp } from "../../../utils/harmony-apps"; +import type { RestartAppParams, RestartAppResult, RestartAppHarmonyServices } from "../types"; + +/** + * Restart on HarmonyOS is `aa force-stop` then a resolved `aa start`. + * + * The stop is tolerated rather than required: force-stopping an app that is not + * running is not an error worth failing a restart over, and the launch that + * follows is what the caller actually asked for. `activity` is Android-only and + * ignored — the HarmonyOS entry ability is resolved from the bundle. + */ +export const harmonyImpl: PlatformImpl< + RestartAppHarmonyServices, + RestartAppParams, + RestartAppResult +> = { + requires: ["hdc"], + handler: async (_services, params, device) => { + const connectKey = harmonyConnectKey(device.id); + await terminateHarmonyApp(connectKey, params.bundleId).catch(() => {}); + await launchHarmonyApp(connectKey, params.bundleId); + return { restarted: true, bundleId: params.bundleId }; + }, +}; diff --git a/packages/tool-server/src/tools/restart-app/types.ts b/packages/tool-server/src/tools/restart-app/types.ts index 4fa13959b7..4c602edfc6 100644 --- a/packages/tool-server/src/tools/restart-app/types.ts +++ b/packages/tool-server/src/tools/restart-app/types.ts @@ -21,3 +21,4 @@ export interface RestartAppIosServices { } export type RestartAppAndroidServices = Record; export type RestartAppVegaServices = Record; +export type RestartAppHarmonyServices = Record; diff --git a/packages/tool-server/src/tools/screenshot/index.ts b/packages/tool-server/src/tools/screenshot/index.ts index 38af5ae17a..fc54a31b5f 100644 --- a/packages/tool-server/src/tools/screenshot/index.ts +++ b/packages/tool-server/src/tools/screenshot/index.ts @@ -6,11 +6,13 @@ import { z } from "zod"; import type { Registry, ToolCapability, ToolDefinition } from "@argent/registry"; import { simulatorServerRef, type SimulatorServerApi } from "../../blueprints/simulator-server"; import { chromiumCdpRef, type ChromiumCdpApi } from "../../blueprints/chromium-cdp"; -import { resolveDevice } from "../../utils/device-info"; +import { resolveDevice, harmonyConnectKey } from "../../utils/device-info"; import { getScreenshotScale, httpScreenshot } from "../../utils/simulator-client"; import { isTvOsSimulator } from "../../utils/ios-devices"; import { simctlArgsForUdid } from "../../utils/ios-device-sets"; import { captureVegaScreenshotPng } from "../../utils/vega-screen"; +import { captureHarmonyScreenshotPng } from "../../utils/harmony-screen"; +import { ensureDep } from "../../utils/check-deps"; import { requireArtifacts, type ArtifactHandle } from "../../artifacts"; const execFileAsync = promisify(execFile); @@ -69,6 +71,7 @@ const capability: ToolCapability = { android: { emulator: true, device: true, unknown: true }, chromium: { app: true }, vega: { vvd: true }, + harmony: { device: true }, }; /** @@ -131,7 +134,7 @@ export function createScreenshotTool(registry: Registry): ToolDefinition `Captured screenshot ${result.image.filename}`, failedMsg: ({ failureSignal }) => `Failed to capture screenshot: ${failureSignal.error_code}`, }, - description: `Capture a screenshot of the device screen (iOS simulator, Android emulator, Apple TV simulator, Vega, or Chromium app). Returns { image }; the MCP adapter renders it as a visible image unless the caller passed includeImageInContext: false. + description: `Capture a screenshot of the device screen (iOS simulator, Android emulator, Apple TV simulator, Vega, HarmonyOS device, or Chromium app). Returns { image }; the MCP adapter renders it as a visible image unless the caller passed includeImageInContext: false. Use when you need a baseline image before an interaction or to inspect the current screen state after a delay. Fails if the simulator-server / emulator backend / Chromium CDP is not reachable for the given device.`, alwaysLoad: true, @@ -178,6 +181,18 @@ Fails if the simulator-server / emulator backend / Chromium CDP is not reachable return { image }; } + // HarmonyOS captures on-device with `uitest screenCap` and copies the PNG + // back over hdc; there is no simulator-server controller for the platform. + if (device.platform === "harmony") { + await ensureDep("hdc"); + const pngPath = await captureHarmonyScreenshotPng({ + connectKey: harmonyConnectKey(device.id), + scale: params.scale, + }); + const image = await requireArtifacts(ctx).register(pngPath, { mimeType: "image/png" }); + return { image }; + } + const ref = simulatorServerRef(device); const api = (await registry.resolveService(ref.urn, ref.options)) as SimulatorServerApi; const { path: capturedPath } = await httpScreenshot( diff --git a/packages/tool-server/src/utils/harmony-apps.ts b/packages/tool-server/src/utils/harmony-apps.ts new file mode 100644 index 0000000000..26d4450d51 --- /dev/null +++ b/packages/tool-server/src/utils/harmony-apps.ts @@ -0,0 +1,229 @@ +import { FAILURE_CODES, FailureError } from "@argent/registry"; +import { runHdcShell, shellQuote } from "./harmony-hdc"; + +/** + * App lifecycle on HarmonyOS, over the device's `aa` (ability assistant) and + * `bm` (bundle manager). + * + * `aa` continues the platform's pattern of unreliable exit codes and takes it + * one step further than `hdc` or `Emulator`: measured on HarmonyOS 6.0.1, a + * failed `aa start` prints `error: failed to start ability.` with a numbered + * error code and still exits **0**. Success prints `start ability successfully.` + * So the verdict is read off stdout, and a missing success line is a failure + * even when nothing recognisable was printed — the alternative is reporting a + * launch that did not happen. + */ + +/** Printed verbatim by `aa start` on success. */ +const AA_SUCCESS = "start ability successfully."; + +interface HarmonyBundleEntry { + /** The ability to launch, as `bm` reports it — always fully qualified. */ + mainAbility: string; + /** The HAP module the ability lives in; `aa start -m` needs it. */ + module: string; +} + +/** + * Resolve a bundle's launchable entry point. + * + * Necessary rather than convenient: `aa start -b ` alone does **not** + * launch an app. It is an *implicit* start, and with no matching action it + * fails with `10103101 Failed to find a matching application for implicit + * launch` — and, worse, leaves a system "No options to open with" chooser on the + * user's screen, which then has to be dismissed. So the ability and its module + * are looked up first and passed explicitly, which is also how DevEco Studio + * launches an app. + * + * The subtlety, and the reason this is not a one-line read of `mainAbility`: + * **`aa start -a` does not accept `mainAbility`.** It accepts the `name` of the + * matching entry in `abilityInfos`, and the two are spelled differently from + * bundle to bundle. Measured on HarmonyOS 6.0.1: + * + * bundle mainAbility abilityInfos name + * calculator com.huawei.hmos.calculator.CalculatorAbility CalculatorAbility + * settings com.huawei.hmos.settings.MainAbility (identical) + * notepad MainAbility (identical) + * + * Passing `mainAbility` verbatim launches Settings and fails on Calculator with + * `10104001 The specified ability does not exist`; passing the bare final + * segment does the reverse. So `mainAbility` is used to *identify* the entry and + * the entry's own `name` is what gets passed. + * + * `mainAbility` is what identifies it, rather than the first entry of + * `abilityInfos`: a bundle can declare a dozen abilities (Settings declares + * background, OOBE and external-intent ones) and only `mainAbility` picks the + * one the launcher icon opens. + */ +export async function resolveHarmonyEntry( + connectKey: string, + bundleId: string +): Promise { + const { stdout } = await runHdcShell(connectKey, `bm dump -n ${shellQuote(bundleId)}`); + const start = stdout.indexOf("{"); + if (start === -1) { + // `bm dump` prints a prose line for an unknown bundle rather than JSON. + throw new FailureError( + `HarmonyOS device '${connectKey}' has no app with bundle name '${bundleId}'. ` + + `List what is installed with \`bm dump -a\`.`, + { + error_code: FAILURE_CODES.HARMONY_ABILITY_START_FAILED, + failure_stage: "harmony_resolve_bundle", + failure_area: "tool_server", + error_kind: "not_found", + } + ); + } + let parsed: { + mainEntry?: unknown; + hapModuleInfos?: Array<{ + name?: unknown; + mainAbility?: unknown; + abilityInfos?: Array<{ name?: unknown }>; + }>; + }; + try { + parsed = JSON.parse(stdout.slice(start)); + } catch (err) { + throw new FailureError( + `Could not read the bundle description of '${bundleId}' from HarmonyOS device '${connectKey}'.`, + { + error_code: FAILURE_CODES.HARMONY_ABILITY_START_FAILED, + failure_stage: "harmony_resolve_bundle", + failure_area: "tool_server", + error_kind: "subprocess", + }, + { cause: err as Error } + ); + } + const modules = Array.isArray(parsed.hapModuleInfos) ? parsed.hapModuleInfos : []; + const mainEntry = typeof parsed.mainEntry === "string" ? parsed.mainEntry : null; + // `mainEntry` names the module that owns the launcher entry; fall back to the + // first module that declares a mainAbility for bundles that omit it. + const chosen = + modules.find((m) => m.name === mainEntry && typeof m.mainAbility === "string") ?? + modules.find((m) => typeof m.mainAbility === "string"); + if (!chosen || typeof chosen.mainAbility !== "string" || typeof chosen.name !== "string") { + throw new FailureError( + `App '${bundleId}' on HarmonyOS device '${connectKey}' declares no launchable main ability, ` + + `so there is nothing to start. It may be a service or extension rather than an app.`, + { + error_code: FAILURE_CODES.HARMONY_ABILITY_START_FAILED, + failure_stage: "harmony_resolve_bundle", + failure_area: "tool_server", + error_kind: "not_found", + } + ); + } + return { + mainAbility: startableAbilityName(chosen.abilityInfos, chosen.mainAbility), + module: chosen.name, + }; +} + +/** + * The spelling of the main ability that `aa start -a` accepts — see the note on + * `resolveHarmonyEntry`. + * + * Falls back to `mainAbility` itself when no declared ability matches: two of + * the three bundles measured spell them identically, so it is the right guess, + * and `aa`'s own `10104001` names the problem precisely if it turns out wrong. + */ +function startableAbilityName( + abilityInfos: Array<{ name?: unknown }> | undefined, + mainAbility: string +): string { + const names = (Array.isArray(abilityInfos) ? abilityInfos : []) + .map((a) => a?.name) + .filter((n): n is string => typeof n === "string" && n.length > 0); + if (names.includes(mainAbility)) return mainAbility; + return names.find((n) => mainAbility.endsWith(`.${n}`)) ?? mainAbility; +} + +/** Bring an app to the foreground, resolving its entry ability first. */ +export async function launchHarmonyApp(connectKey: string, bundleId: string): Promise { + const entry = await resolveHarmonyEntry(connectKey, bundleId); + const { stdout } = await runHdcShell( + connectKey, + `aa start -b ${shellQuote(bundleId)} -a ${shellQuote(entry.mainAbility)} -m ${shellQuote(entry.module)}` + ); + assertAbilityStarted(stdout, connectKey, bundleId); +} + +/** + * Stop every process of an app. + * + * `aa force-stop` takes the bundle name and, unlike `aa start`, needs no ability + * — there is nothing to resolve. + */ +export async function terminateHarmonyApp(connectKey: string, bundleId: string): Promise { + const { stdout } = await runHdcShell(connectKey, `aa force-stop ${shellQuote(bundleId)}`); + if (stdout.includes("error:")) { + throw new FailureError( + `Failed to stop '${bundleId}' on HarmonyOS device '${connectKey}': ${firstLine(stdout)}`, + { + error_code: FAILURE_CODES.HARMONY_ABILITY_START_FAILED, + failure_stage: "harmony_force_stop", + failure_area: "tool_server", + error_kind: "subprocess", + } + ); + } +} + +/** + * Open a URI through whichever app claims it. + * + * This *is* an implicit start — the one case where it is the right verb, since + * the point is to let the system choose the handler. It carries the failure mode + * documented on `resolveHarmonyEntry`: with no handler installed, HarmonyOS + * shows a chooser on the device. The caller is told so rather than left to + * discover a dialog sitting on the screen. + */ +export async function openHarmonyUrl(connectKey: string, url: string): Promise { + const { stdout } = await runHdcShell(connectKey, `aa start -U ${shellQuote(url)}`); + if (!stdout.includes(AA_SUCCESS)) { + throw new FailureError( + `HarmonyOS device '${connectKey}' could not open '${url}': ${firstLine(stdout)}. ` + + `No installed app claims this URI scheme; the device may now be showing a chooser dialog.`, + { + error_code: FAILURE_CODES.HARMONY_ABILITY_START_FAILED, + failure_stage: "harmony_open_url", + failure_area: "tool_server", + error_kind: "not_found", + } + ); + } +} + +function assertAbilityStarted(stdout: string, connectKey: string, bundleId: string): void { + if (stdout.includes(AA_SUCCESS)) return; + throw new FailureError( + `Failed to launch '${bundleId}' on HarmonyOS device '${connectKey}': ${firstLine(stdout)}`, + { + error_code: FAILURE_CODES.HARMONY_ABILITY_START_FAILED, + failure_stage: "harmony_ability_start", + failure_area: "tool_server", + error_kind: "subprocess", + } + ); +} + +/** + * The part of an `aa` failure worth surfacing. + * + * `aa` prints a useless headline (`error: failed to start ability.`) and puts + * the actual cause on the next line (`Error Code:10104001 Error Message:The + * specified ability does not exist`), then several lines of generic advice. So + * the coded line is kept alongside the headline — reporting only the first line + * tells the agent that something failed and nothing about what. + */ +function firstLine(stdout: string): string { + const lines = stdout + .split(/\r?\n/) + .map((l) => l.trim()) + .filter((l) => l.length > 0); + if (lines.length === 0) return "the ability assistant printed nothing"; + const coded = lines.find((l) => l.startsWith("Error Code:")); + return coded ? `${lines[0]} ${coded}` : lines[0]; +} diff --git a/packages/tool-server/src/utils/harmony-devices.ts b/packages/tool-server/src/utils/harmony-devices.ts index e6984c488e..030a23798d 100644 --- a/packages/tool-server/src/utils/harmony-devices.ts +++ b/packages/tool-server/src/utils/harmony-devices.ts @@ -26,7 +26,7 @@ import { HDC_EMPTY_SENTINEL, hdcFailure, resolveHdc, runHdc } from "./harmony-hd */ /** A HarmonyOS emulator instance as reported by `Emulator -list -details`. */ -export interface HarmonyInstance { +interface HarmonyInstance { name: string; /** `Phone`, `Foldable`, `Tablet`, `TV`, … as configured at creation. */ deviceType: string | null; @@ -111,7 +111,7 @@ export async function listHarmonyInstances(): Promise { } /** A target `hdc` is connected to — a physical device, or a running emulator. */ -export interface HarmonyHdcTarget { +interface HarmonyHdcTarget { /** The key every `hdc -t ` call takes. A hardware serial, or `ip:port`. */ connectKey: string; /** `USB`, `TCP`, … as reported in column 2. */ diff --git a/packages/tool-server/src/utils/harmony-hdc.ts b/packages/tool-server/src/utils/harmony-hdc.ts index b433c6fd3a..c8bdb75be6 100644 --- a/packages/tool-server/src/utils/harmony-hdc.ts +++ b/packages/tool-server/src/utils/harmony-hdc.ts @@ -86,7 +86,7 @@ export async function resolveHdc(): Promise { return path; } -export async function resolveHdcOrThrow(): Promise { +async function resolveHdcOrThrow(): Promise { const path = await resolveHdc(); if (!path) { throw new FailureError( @@ -104,7 +104,7 @@ export async function resolveHdcOrThrow(): Promise { return path; } -export interface HdcRunResult { +interface HdcRunResult { stdout: string; stderr: string; } @@ -167,7 +167,7 @@ export function shellQuote(value: string): string { return `'${value.replaceAll("'", `'\\''`)}'`; } -export interface HdcShellResult { +interface HdcShellResult { /** Everything the remote command wrote, with the status sentinel removed. */ stdout: string; /** The remote command's own exit status, recovered via the sentinel. */ diff --git a/packages/tool-server/src/utils/harmony-screen.ts b/packages/tool-server/src/utils/harmony-screen.ts new file mode 100644 index 0000000000..1b05559f66 --- /dev/null +++ b/packages/tool-server/src/utils/harmony-screen.ts @@ -0,0 +1,32 @@ +import { readFile, writeFile, rm } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { PNG } from "pngjs"; +import { harmonyScreenCap } from "./harmony-uitest"; +import { scaleDecodedPng } from "./png-scale"; + +/** + * Capture the HarmonyOS display and return the path to a host-side PNG. + * + * `uitest screenCap` always writes the panel at full resolution — there is no + * scale flag — so a Mate 60 frame arrives as a ~250KB 1216x2688 RGBA PNG. The + * downscale therefore happens here, through the same shared helper Vega uses, + * so `ARGENT_SCREENSHOT_SCALE` and the 0.3 default mean the same thing on + * HarmonyOS as everywhere else. + */ +export async function captureHarmonyScreenshotPng(opts: { + connectKey: string; + scale?: number; +}): Promise { + const rawPath = join(tmpdir(), `argent-harmony-raw-${process.hrtime.bigint()}.png`); + try { + await harmonyScreenCap(opts.connectKey, rawPath); + const decoded = PNG.sync.read(await readFile(rawPath)); + const scaled = scaleDecodedPng(decoded, opts.scale); + const outPath = join(tmpdir(), `harmony-screenshot-${process.hrtime.bigint()}.png`); + await writeFile(outPath, PNG.sync.write(scaled)); + return outPath; + } finally { + await rm(rawPath, { force: true }).catch(() => {}); + } +} diff --git a/packages/tool-server/src/utils/harmony-uitest.ts b/packages/tool-server/src/utils/harmony-uitest.ts index 6dbd7f2567..1f2e8acfcc 100644 --- a/packages/tool-server/src/utils/harmony-uitest.ts +++ b/packages/tool-server/src/utils/harmony-uitest.ts @@ -64,7 +64,7 @@ async function runUitest( return stdout; } -export interface HarmonyDisplay { +interface HarmonyDisplay { width: number; height: number; /** False when the display is suspended — injected touches land nowhere. */ @@ -123,7 +123,7 @@ export function toDevicePoint( return { x: clamp(x, display.width), y: clamp(y, display.height) }; } -export type HarmonyTouchCommand = "click" | "doubleClick" | "longClick"; +type HarmonyTouchCommand = "click" | "doubleClick" | "longClick"; export async function harmonyTouch( connectKey: string, @@ -133,16 +133,16 @@ export async function harmonyTouch( await runUitest(connectKey, `uiInput ${command} ${point.x} ${point.y}`); } -export type HarmonySwipeCommand = "swipe" | "drag" | "fling"; +type HarmonySwipeCommand = "swipe" | "drag" | "fling"; /** * `uitest` rejects a velocity outside this range with `Invalid parameters.`, so * callers translating a duration into a velocity must land inside it. */ -export const HARMONY_VELOCITY_MIN = 200; -export const HARMONY_VELOCITY_MAX = 40_000; +const HARMONY_VELOCITY_MIN = 200; +const HARMONY_VELOCITY_MAX = 40_000; -export async function harmonySwipe( +async function harmonySwipe( connectKey: string, command: HarmonySwipeCommand, from: { x: number; y: number }, @@ -153,6 +153,37 @@ export async function harmonySwipe( await runUitest(connectKey, `uiInput ${command} ${from.x} ${from.y} ${to.x} ${to.y} ${v}`); } +/** + * Swipe between two normalized points over `durationMs`. + * + * `uitest` takes a **velocity**, not a duration, so the duration argent's tools + * speak is converted here: velocity = pixels travelled / seconds. Doing it the + * other way — passing a fixed velocity — would make a short swipe and a + * screen-length one take wildly different times, and the callers that pace a + * scroll loop against `durationMs` would be pacing against nothing. + * + * `settle` picks the verb rather than reshaping the path. `uitest` exposes both + * `swipe` (a drag that ends where it ends) and `fling` (which hands the scroller + * a release velocity to coast on), so the momentum-free request maps onto the + * one the platform already means by it. That is the vendor's own distinction + * between the two commands; the resulting difference in coast distance was not + * measured here. + */ +export async function harmonySwipeNormalized( + connectKey: string, + from: { x: number; y: number }, + to: { x: number; y: number }, + durationMs: number, + settle: boolean +): Promise { + const display = await harmonyDisplay(connectKey); + const fromPx = toDevicePoint(from.x, from.y, display); + const toPx = toDevicePoint(to.x, to.y, display); + const distance = Math.hypot(toPx.x - fromPx.x, toPx.y - fromPx.y); + const seconds = Math.max(durationMs, 1) / 1000; + await harmonySwipe(connectKey, settle ? "swipe" : "fling", fromPx, toPx, distance / seconds); +} + export async function harmonyKeyEvent(connectKey: string, key: string): Promise { await runUitest(connectKey, `uiInput keyEvent ${key}`); } @@ -162,15 +193,6 @@ export async function harmonyTypeText(connectKey: string, text: string): Promise await runUitest(connectKey, `uiInput text ${shellQuote(text)}`); } -/** Focus the field at a point and type into it in one call. */ -export async function harmonyInputTextAt( - connectKey: string, - point: { x: number; y: number }, - text: string -): Promise { - await runUitest(connectKey, `uiInput inputText ${point.x} ${point.y} ${shellQuote(text)}`); -} - /** * Run `producer` against a freshly-named path under the device's tmp directory, * copy the result to `localPath`, then delete the on-device copy. diff --git a/packages/tool-server/src/utils/png-scale.ts b/packages/tool-server/src/utils/png-scale.ts new file mode 100644 index 0000000000..4329514dc6 --- /dev/null +++ b/packages/tool-server/src/utils/png-scale.ts @@ -0,0 +1,31 @@ +import { PNG } from "pngjs"; +import { getScreenshotScale } from "./simulator-client"; +import { resizeDecodedPng } from "../tools/screenshot-diff/resize"; + +/** + * Downscale a decoded RGBA PNG by `scale`. + * + * Used by the platforms that capture a full-resolution PNG host-side and have + * to shrink it themselves — Vega over the emulator console, HarmonyOS over + * `uitest screenCap` — where iOS and Android get the same reduction from + * simulator-server. Sharing one implementation is what keeps + * `ARGENT_SCREENSHOT_SCALE` meaning the same thing on every platform: the + * default and range handling come from `getScreenshotScale()` (which rejects + * out-of-(0,1] values and falls back to 0.3), and the resample is the lanczos3 + * `resizeDecodedPng()` that screenshot-diff uses, so a HarmonyOS baseline and an + * Android one are comparable at the same quality. + */ +export function scaleDecodedPng(src: PNG, scale?: number): PNG { + const s = scale ?? getScreenshotScale(); + if (s >= 1) return src; + const outW = Math.max(1, Math.round(src.width * s)); + const outH = Math.max(1, Math.round(src.height * s)); + const resized = resizeDecodedPng( + { width: src.width, height: src.height, data: src.data }, + outW, + outH + ); + const out = new PNG({ width: resized.width, height: resized.height }); + resized.data.copy(out.data); + return out; +} diff --git a/packages/tool-server/src/utils/vega-screen.ts b/packages/tool-server/src/utils/vega-screen.ts index 260dc9167e..adeb0a1967 100644 --- a/packages/tool-server/src/utils/vega-screen.ts +++ b/packages/tool-server/src/utils/vega-screen.ts @@ -5,8 +5,7 @@ import { PNG } from "pngjs"; import { FAILURE_CODES, FailureError } from "@argent/registry"; import { runAdb } from "./adb"; import { discoverVegaConsolePort } from "./vega-vvd"; -import { getScreenshotScale } from "./simulator-client"; -import { resizeDecodedPng } from "../tools/screenshot-diff/resize"; +import { scaleDecodedPng } from "./png-scale"; /** * Capture the VVD screen as a PNG. @@ -48,7 +47,7 @@ async function captureViaEmulatorConsole(opts: { scale?: number }): Promise {}); } } - -/** - * Downscale a decoded RGBA PNG by `scale`. Defaults and resampling are shared - * with the other platforms rather than re-derived here: the default + range - * handling comes from `getScreenshotScale()` (the iOS/Android env parser, which - * rejects out-of-(0,1] values and falls back to 0.3), and the resample is the - * lanczos3 `resizeDecodedPng()` used by screenshot-diff — so Vega screenshots - * honour `ARGENT_SCREENSHOT_SCALE` identically and at the same quality. - */ -function scalePng(src: PNG, scale?: number): PNG { - const s = scale ?? getScreenshotScale(); - if (s >= 1) return src; - const outW = Math.max(1, Math.round(src.width * s)); - const outH = Math.max(1, Math.round(src.height * s)); - const resized = resizeDecodedPng( - { width: src.width, height: src.height, data: src.data }, - outW, - outH - ); - const out = new PNG({ width: resized.width, height: resized.height }); - resized.data.copy(out.data); - return out; -} diff --git a/packages/tool-server/test/harmony-apps.test.ts b/packages/tool-server/test/harmony-apps.test.ts new file mode 100644 index 0000000000..7dd0b0fb0a --- /dev/null +++ b/packages/tool-server/test/harmony-apps.test.ts @@ -0,0 +1,186 @@ +import { describe, it, expect, vi, beforeEach } from "vitest"; +import { runHdcShell as realRunHdcShell } from "../src/utils/harmony-hdc"; +import { resolveHarmonyEntry, launchHarmonyApp, openHarmonyUrl } from "../src/utils/harmony-apps"; + +// Only the transport is faked: these tests are about what the module makes of +// `bm`/`aa` output, and both CLIs report failure on stdout while exiting 0. +vi.mock("../src/utils/harmony-hdc", async (importOriginal) => { + const actual = await importOriginal(); + return { ...actual, runHdcShell: vi.fn() }; +}); + +const runHdcShell = vi.mocked(realRunHdcShell); + +/** + * `bm dump -n` output, trimmed to the keys the resolver reads. The three + * fixtures are the three real bundles measured on a HarmonyOS 6.0.1 handset, + * because the spelling of the launchable ability differs between them and that + * difference is the whole reason the resolver is not a one-line read. + */ +const CALCULATOR = JSON.stringify({ + // mainAbility is fully qualified, the ability declares itself by short name. + mainEntry: "phone", + hapModuleInfos: [ + { + name: "phone", + mainAbility: "com.huawei.hmos.calculator.CalculatorAbility", + abilityInfos: [{ name: "CalculatorAbility" }], + }, + ], +}); + +const SETTINGS = JSON.stringify({ + // mainAbility is fully qualified and the ability declares the same string. + mainEntry: "phone_settings", + hapModuleInfos: [ + { + name: "phone_settings", + mainAbility: "com.huawei.hmos.settings.MainAbility", + abilityInfos: [ + { name: "DefaultIntentBackgroundUiAbility" }, + { name: "com.huawei.hmos.settings.MainAbility" }, + ], + }, + ], +}); + +const NOTEPAD = JSON.stringify({ + // mainAbility is already a short name. + mainEntry: "phone", + hapModuleInfos: [ + { name: "phone", mainAbility: "MainAbility", abilityInfos: [{ name: "MainAbility" }] }, + ], +}); + +const ok = (stdout: string) => ({ stdout, exitCode: 0 }); + +beforeEach(() => runHdcShell.mockReset()); + +describe("resolveHarmonyEntry", () => { + it("passes the short ability name when that is how the bundle declares it", async () => { + // Regression: sending `mainAbility` verbatim here fails on a real device + // with `10104001 The specified ability does not exist`, and launch-app + // reported a launch that never happened. + runHdcShell.mockResolvedValueOnce(ok(CALCULATOR)); + expect(await resolveHarmonyEntry("dev", "com.huawei.hmos.calculator")).toEqual({ + mainAbility: "CalculatorAbility", + module: "phone", + }); + }); + + it("passes the qualified ability name when that is how the bundle declares it", async () => { + // The mirror case: shortening this one to `MainAbility` fails the same way. + runHdcShell.mockResolvedValueOnce(ok(SETTINGS)); + expect(await resolveHarmonyEntry("dev", "com.huawei.hmos.settings")).toEqual({ + mainAbility: "com.huawei.hmos.settings.MainAbility", + module: "phone_settings", + }); + }); + + it("passes an already-short mainAbility through unchanged", async () => { + runHdcShell.mockResolvedValueOnce(ok(NOTEPAD)); + expect((await resolveHarmonyEntry("dev", "com.huawei.hmos.notepad")).mainAbility).toBe( + "MainAbility" + ); + }); + + it("picks the module named by mainEntry, not the first one listed", async () => { + runHdcShell.mockResolvedValueOnce( + ok( + JSON.stringify({ + mainEntry: "phone", + hapModuleInfos: [ + { + name: "feature", + mainAbility: "FeatureAbility", + abilityInfos: [{ name: "FeatureAbility" }], + }, + { name: "phone", mainAbility: "MainAbility", abilityInfos: [{ name: "MainAbility" }] }, + ], + }) + ) + ); + expect(await resolveHarmonyEntry("dev", "x")).toEqual({ + mainAbility: "MainAbility", + module: "phone", + }); + }); + + it("reports an unknown bundle rather than parsing prose as JSON", async () => { + runHdcShell.mockResolvedValueOnce(ok("error: bundle is not existed.")); + await expect(resolveHarmonyEntry("dev", "com.nope")).rejects.toThrow(/no app with bundle name/); + }); + + it("reports a bundle that declares no launchable ability", async () => { + runHdcShell.mockResolvedValueOnce( + ok(JSON.stringify({ mainEntry: "svc", hapModuleInfos: [{ name: "svc" }] })) + ); + await expect(resolveHarmonyEntry("dev", "com.svc")).rejects.toThrow( + /no launchable main ability/ + ); + }); +}); + +describe("launchHarmonyApp", () => { + it("fails when `aa` prints an error, even though it exits 0", async () => { + // `aa start` exits 0 on failure. Trusting the status would report a launch + // that did not happen — the failure mode this whole module is shaped around. + runHdcShell + .mockResolvedValueOnce(ok(CALCULATOR)) + .mockResolvedValueOnce( + ok( + "error: failed to start ability.\nError Code:10104001 Error Message:The specified ability does not exist" + ) + ); + await expect(launchHarmonyApp("dev", "com.huawei.hmos.calculator")).rejects.toThrow(/10104001/); + }); + + it("keeps the coded line, not just `aa`'s useless headline", async () => { + runHdcShell + .mockResolvedValueOnce(ok(CALCULATOR)) + .mockResolvedValueOnce( + ok( + "error: failed to start ability.\nError Code:10104001 Error Message:The specified ability does not exist\nError cause: not installed" + ) + ); + await expect(launchHarmonyApp("dev", "c")).rejects.toThrow( + /error: failed to start ability\. Error Code:10104001/ + ); + }); + + it("succeeds only on the success line", async () => { + runHdcShell + .mockResolvedValueOnce(ok(CALCULATOR)) + .mockResolvedValueOnce(ok("start ability successfully.")); + await expect(launchHarmonyApp("dev", "c")).resolves.toBeUndefined(); + }); + + it("passes the resolved ability and module to `aa start`", async () => { + runHdcShell + .mockResolvedValueOnce(ok(CALCULATOR)) + .mockResolvedValueOnce(ok("start ability successfully.")); + await launchHarmonyApp("dev", "com.huawei.hmos.calculator"); + // Explicit, never a bare `-b`: implicit start fails and leaves a chooser on + // the user's screen. + expect(runHdcShell.mock.calls[1][1]).toBe( + "aa start -b 'com.huawei.hmos.calculator' -a 'CalculatorAbility' -m 'phone'" + ); + }); +}); + +describe("openHarmonyUrl", () => { + it("rejects a URI no app claims", async () => { + runHdcShell.mockResolvedValueOnce( + ok( + "error: failed to start ability.\nError Code:10103101 Error Message:Failed to find a matching application for implicit launch." + ) + ); + await expect(openHarmonyUrl("dev", "nope://x")).rejects.toThrow(/10103101/); + }); + + it("shell-quotes the URI so its query string cannot reach the device shell", async () => { + runHdcShell.mockResolvedValueOnce(ok("start ability successfully.")); + await openHarmonyUrl("dev", "app://x?a=1&b=2;id"); + expect(runHdcShell.mock.calls[0][1]).toBe("aa start -U 'app://x?a=1&b=2;id'"); + }); +}); diff --git a/packages/tool-server/test/harmony-hdc.test.ts b/packages/tool-server/test/harmony-hdc.test.ts new file mode 100644 index 0000000000..e932ccc608 --- /dev/null +++ b/packages/tool-server/test/harmony-hdc.test.ts @@ -0,0 +1,80 @@ +import { describe, it, expect } from "vitest"; +import { hdcFailure, shellQuote } from "../src/utils/harmony-hdc"; +import { toDevicePoint } from "../src/utils/harmony-uitest"; + +describe("hdcFailure", () => { + it("reports the `[Fail]` line hdc printed", () => { + // hdc exits 0 for this, so the prefix is the only failure signal there is. + expect( + hdcFailure({ + stdout: "[Fail]Not match target founded, check connect-key please\n", + stderr: "", + }) + ).toBe("[Fail]Not match target founded, check connect-key please"); + }); + + it("reads a failure written to stderr", () => { + expect(hdcFailure({ stdout: "", stderr: "[Fail]Error opening file: no such file\n" })).toBe( + "[Fail]Error opening file: no such file" + ); + }); + + it("returns null when nothing failed", () => { + expect(hdcFailure({ stdout: "hi\n", stderr: "" })).toBeNull(); + expect(hdcFailure({ stdout: "[Empty]\n", stderr: "" })).toBeNull(); + }); + + it("does not let a remote command's own output forge a transport failure", () => { + // Matched at the start of a line, not as a substring: a device log or a test + // name containing the token must not read as hdc losing the connection. + expect(hdcFailure({ stdout: "test case [Fail]ing on purpose\n", stderr: "" })).toBeNull(); + }); +}); + +describe("shellQuote", () => { + // `hdc shell` takes a command LINE, not an argv, so every caller-supplied + // value lands in a device-side /bin/sh. Each of these was round-tripped + // through a real device via `echo` and came back byte-identical. + it.each([ + ["hello world", `'hello world'`], + ["a'b", `'a'\\''b'`], + ['a"b', `'a"b'`], + ["a$b", `'a$b'`], + ["a;echo PWNED", `'a;echo PWNED'`], + ["a`echo X`b", "'a`echo X`b'"], + ["ünïcode 中文", `'ünïcode 中文'`], + ])("quotes %j", (input, expected) => { + expect(shellQuote(input)).toBe(expected); + }); + + it("neutralises a command substitution rather than letting it run", () => { + // The single quote is the whole defence; if it were double quotes the + // backticks below would execute on the device as the `shell` user. + expect(shellQuote("`id`")).toBe("'`id`'"); + }); +}); + +describe("toDevicePoint", () => { + const display = { width: 1216, height: 2688 }; + + it("scales a normalized point into device pixels", () => { + expect(toDevicePoint(0.5, 0.5, display)).toEqual({ x: 608, y: 1344 }); + }); + + it("keeps the far edge inside the display", () => { + // `uitest` accepts an off-screen coordinate, returns `No Error` and does + // nothing — so 1.0 must land on the last addressable pixel, not one past it, + // or a tap on a right-edge element silently misses while reporting success. + expect(toDevicePoint(1, 1, display)).toEqual({ x: 1215, y: 2687 }); + }); + + it("clamps a point outside the unit square instead of going negative", () => { + // `uitest` *does* reject negative coordinates, so an un-clamped caller would + // turn an out-of-range frame into a hard error rather than an edge tap. + expect(toDevicePoint(-0.2, 1.4, display)).toEqual({ x: 0, y: 2687 }); + }); + + it("puts the origin at the first pixel", () => { + expect(toDevicePoint(0, 0, display)).toEqual({ x: 0, y: 0 }); + }); +}); diff --git a/packages/tool-server/test/harmony-layout-parser.test.ts b/packages/tool-server/test/harmony-layout-parser.test.ts new file mode 100644 index 0000000000..fa78d659f3 --- /dev/null +++ b/packages/tool-server/test/harmony-layout-parser.test.ts @@ -0,0 +1,248 @@ +import { describe, it, expect } from "vitest"; +import { + harmonyLabel, + parseHarmonyBounds, + parseHarmonyLayout, +} from "../src/tools/describe/platforms/harmony/layout-parser"; +import type { HarmonyLayoutNode } from "../src/utils/harmony-uitest"; + +const SCREEN = { width: 1216, height: 2688 }; + +/** Build a dump node with `uitest`'s string-valued attributes. */ +function node( + attrs: Record, + children: HarmonyLayoutNode[] = [] +): HarmonyLayoutNode { + return { attributes: attrs, children }; +} + +/** The synthetic root `uitest` emits: no attributes but the display bounds. */ +function root(windows: HarmonyLayoutNode[]): HarmonyLayoutNode { + return node({ bounds: "[0,0][1216,2688]" }, windows); +} + +function flatten(n: { role: string; children: { role: string }[] }): string[] { + return [n.role, ...n.children.flatMap((c) => flatten(c as never))]; +} + +describe("parseHarmonyBounds", () => { + it("reads the [l,t][r,b] pixel form", () => { + expect(parseHarmonyBounds("[55,1387][292,1624]")).toEqual({ x: 55, y: 1387, w: 237, h: 237 }); + }); + + it("reads negative coordinates, which a list emits for items scrolled off", () => { + expect(parseHarmonyBounds("[-40,-10][60,90]")).toEqual({ x: -40, y: -10, w: 100, h: 100 }); + }); + + it("returns null for anything else", () => { + expect(parseHarmonyBounds("")).toBeNull(); + expect(parseHarmonyBounds("55,1387")).toBeNull(); + }); +}); + +describe("harmonyLabel", () => { + it("prefers the accessibility description over the visible text", () => { + expect(harmonyLabel({ description: "Delete", text: "x" })).toBe("Delete"); + }); + + it("treats a whitespace-only description as absent", () => { + // The calculator labels every keypad Button `" "`. Without the trim each one + // becomes a blank-labelled node that reads like a real, empty label. + expect(harmonyLabel({ description: " ", text: "7" })).toBe("7"); + }); + + it("falls back to the placeholder of an empty field", () => { + expect(harmonyLabel({ description: "", text: "", hint: "Search" })).toBe("Search"); + }); + + it("returns empty when the node carries nothing", () => { + expect(harmonyLabel({})).toBe(""); + }); +}); + +describe("parseHarmonyLayout", () => { + it("labels each window with the app that owns it", () => { + const tree = parseHarmonyLayout( + root([ + node({ type: "root", bundleName: "com.app", bounds: "[0,107][1216,2688]" }, [ + node({ type: "Text", text: "Hi", bounds: "[0,107][100,207]" }), + ]), + ]), + SCREEN + ).tree; + expect(tree.children).toHaveLength(1); + expect(tree.children[0].role).toBe("Window"); + // A tree spanning an app and the status bar (always a separate sceneboard + // window) is unreadable if both are anonymous stacks. + expect(tree.children[0].label).toBe("com.app"); + }); + + it("walks through ArkUI layout scaffolding instead of emitting it", () => { + const tree = parseHarmonyLayout( + root([ + node({ type: "root", bundleName: "com.app", bounds: "[0,0][1216,2688]" }, [ + node({ type: "Column", bounds: "[0,0][1216,2688]" }, [ + node({ type: "Row", bounds: "[0,0][1216,2688]" }, [ + node({ type: "Stack", bounds: "[0,0][1216,2688]" }, [ + node({ type: "Text", text: "Deep", bounds: "[10,10][110,110]" }), + ]), + ]), + ]), + ]), + ]), + SCREEN + ).tree; + expect(flatten(tree)).toEqual(["Screen", "Window", "StaticText"]); + }); + + it("keeps a layout type that is itself clickable", () => { + // ArkUI builds real buttons out of Stack/Row, so walking through every + // container by type would delete tap targets. + const tree = parseHarmonyLayout( + root([ + node({ type: "root", bundleName: "com.app", bounds: "[0,0][1216,2688]" }, [ + node({ type: "Stack", clickable: "true", id: "tap-me", bounds: "[0,0][100,100]" }), + ]), + ]), + SCREEN + ).tree; + const target = tree.children[0].children[0]; + expect(target.identifier).toBe("tap-me"); + expect(target.clickable).toBe(true); + }); + + it("drops decoration that can never be a target", () => { + const tree = parseHarmonyLayout( + root([ + node({ type: "root", bundleName: "com.app", bounds: "[0,0][1216,2688]" }, [ + node({ type: "Divider", bounds: "[0,0][1216,2]" }), + node({ type: "ScrollBar", bounds: "[1164,0][1216,2688]" }), + node({ type: "Text", text: "Real", bounds: "[0,10][100,110]" }), + ]), + ]), + SCREEN + ).tree; + expect(flatten(tree)).toEqual(["Screen", "Window", "StaticText"]); + }); + + it("normalizes bounds into the [0,1] frame contract", () => { + const tree = parseHarmonyLayout( + root([ + node({ type: "root", bundleName: "com.app", bounds: "[0,0][1216,2688]" }, [ + node({ type: "Button", id: "b", bounds: "[608,1344][1216,2688]" }), + ]), + ]), + SCREEN + ).tree; + expect(tree.children[0].children[0].frame).toEqual({ + x: 0.5, + y: 0.5, + width: 0.5, + height: 0.5, + }); + }); + + it("clamps a frame for content scrolled off the top", () => { + // A List reports negative bounds for rows above the viewport, and the + // describe frame contract is closed over [0,1]. + const tree = parseHarmonyLayout( + root([ + node({ type: "root", bundleName: "com.app", bounds: "[0,0][1216,2688]" }, [ + node({ type: "Button", id: "b", bounds: "[-100,-200][100,200]" }), + ]), + ]), + SCREEN + ).tree; + const f = tree.children[0].children[0].frame; + expect(f.x).toBe(0); + expect(f.y).toBe(0); + }); + + it("surfaces interactivity and state flags", () => { + const tree = parseHarmonyLayout( + root([ + node({ type: "root", bundleName: "com.app", bounds: "[0,0][1216,2688]" }, [ + node({ + type: "Button", + id: "b", + bounds: "[0,0][100,100]", + clickable: "true", + longClickable: "true", + checkable: "true", + checked: "true", + enabled: "false", + focused: "true", + selected: "true", + }), + ]), + ]), + SCREEN + ).tree; + expect(tree.children[0].children[0]).toMatchObject({ + clickable: true, + longClickable: true, + checkable: true, + checked: true, + disabled: true, + focused: true, + selected: true, + }); + }); + + it("treats a node as enabled when `enabled` is absent", () => { + // Every node carries `enabled`; a missing one means the dump shape changed, + // and defaulting the other way would grey out the whole screen. + const tree = parseHarmonyLayout( + root([ + node({ type: "root", bundleName: "com.app", bounds: "[0,0][1216,2688]" }, [ + node({ type: "Button", id: "b", bounds: "[0,0][100,100]" }), + ]), + ]), + SCREEN + ).tree; + expect(tree.children[0].children[0].disabled).toBeUndefined(); + }); + + it("marks a List as scrollable by type even without the flag", () => { + const tree = parseHarmonyLayout( + root([ + node({ type: "root", bundleName: "com.app", bounds: "[0,0][1216,2688]" }, [ + node({ type: "List", bounds: "[0,0][1216,2688]" }, [ + node({ type: "Text", text: "row", bounds: "[0,0][100,100]" }), + ]), + ]), + ]), + SCREEN + ).tree; + expect(tree.children[0].children[0]).toMatchObject({ role: "ScrollView", scrollable: true }); + }); + + it("says so when a system overlay hides its own contents", () => { + // The app-selector / share sheet renders in another process and the dump + // carries the node with no children. Emitting an empty container would tell + // an agent the screen is empty when a dialog is covering it. + const tree = parseHarmonyLayout( + root([ + node({ type: "root", bundleName: "com.ohos.sceneboard", bounds: "[0,0][1216,2688]" }, [ + node({ type: "UIExtensionComponent", id: "AppSelector", bounds: "[0,0][1216,2688]" }), + ]), + ]), + SCREEN + ).tree; + const overlay = tree.children[0].children[0]; + expect(overlay.role).toBe("SystemOverlay"); + expect(overlay.label).toMatch(/another process/); + }); + + it("takes the screen size from the dump's own root, not the caller's", () => { + // The two are read at different instants; on a foldable that gap is enough + // for an unfold to normalize every frame against the wrong axis. + const { screen } = parseHarmonyLayout(node({ bounds: "[0,0][2200,2480]" }, []), SCREEN); + expect(screen).toEqual({ width: 2200, height: 2480 }); + }); + + it("falls back to the queried size when the root reports no bounds", () => { + const { screen } = parseHarmonyLayout(node({}, []), SCREEN); + expect(screen).toEqual(SCREEN); + }); +}); From 0a5dce5056e5d0316ad927a63428577cf8fcbc64 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Mon, 10 Aug 2026 22:46:02 +0200 Subject: [PATCH 006/110] feat(harmony): batch and diff on a HarmonyOS device MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `run-sequence` and `screenshot-diff` supported iOS and Android but rejected HarmonyOS, so a HarmonyOS session paid an `hdc` process spawn per interaction and had no visual-regression path at all. `run-sequence` is platform-neutral by construction — every step re-dispatches through its own tool — so it only needed the outer gate opened; a step the platform cannot run (`gesture-pinch`) is still reported per-step after the preceding steps have landed. `screenshot-diff` needed a capture branch: HarmonyOS has no simulator-server controller, so a live side is captured with `uitest screenCap` over `hdc` and the service is not requested at all, which would otherwise throw before the capture ran. `rotation` is now refused on both rather than accepted and dropped. `uitest screenCap` writes the display in its current orientation and takes no orientation argument, so honouring the parameter would return an unrotated image — and a diff of the wrong orientation reads as a whole-screen change. The `udid` descriptions of all eleven HarmonyOS-capable tools said which id shapes they take and omitted this one; they now name it. Verified on a physical HarmonyOS 6.0.1 handset: a `button power` step through `run-sequence` moves `powerStatus` SUSPEND→ON, a two-step button + keyboard sequence completes 2/2, and `screenshot-diff` with `captureCurrent` reports `unchanged` against a static screen and then one changed region of 369 px once the lock-screen clock ticked a minute. Both rotation guards return 400 from the device path. --- README.md | 6 +- packages/skills/rules/argent.md | 4 +- .../skills/argent-device-interact/SKILL.md | 4 +- .../src/tools/await-screen-idle/index.ts | 4 +- .../src/tools/await-ui-element/index.ts | 4 +- .../tool-server/src/tools/button/index.ts | 4 +- .../tool-server/src/tools/describe/index.ts | 2 +- .../src/tools/gesture-swipe/index.ts | 4 +- .../src/tools/gesture-tap/index.ts | 4 +- .../tool-server/src/tools/keyboard/index.ts | 2 +- .../tool-server/src/tools/launch-app/index.ts | 4 +- .../tool-server/src/tools/open-url/index.ts | 4 +- .../src/tools/restart-app/index.ts | 2 +- .../src/tools/run-sequence/index.ts | 18 ++- .../src/tools/screenshot-diff/index.ts | 123 +++++++++++------- .../tool-server/src/tools/screenshot/index.ts | 15 ++- .../tool-server/test/run-sequence.test.ts | 45 +++++++ .../test/screenshot-diff-tool.test.ts | 76 +++++++++++ .../tool-server/test/screenshot-tool.test.ts | 27 ++++ 19 files changed, 282 insertions(+), 70 deletions(-) diff --git a/README.md b/README.md index ae048b9fa1..6227f176c4 100644 --- a/README.md +++ b/README.md @@ -27,9 +27,9 @@ Argent drives a growing set of targets through a single toolkit, each with the r | **Android** | Emulators (AVDs) and physical devices over adb | Touch / gesture | | **TV** | Apple TV (tvOS), Android TV / Google TV, Amazon Fire TV (Vega) | D-pad / remote | | **Desktop & web** | Electron and Chromium apps (incl. React Native Web / Expo web) over CDP | Mouse / keyboard | -| **HarmonyOS** | Emulator instances managed by DevEco Studio | None - see below | +| **HarmonyOS** | Phones over `hdc`, and DevEco Studio emulator instances | Touch / gesture | -> **HarmonyOS is discovery-only.** `list-devices` lists DevEco Studio's emulator instances (id `harmony-`, always `state: "unknown"` - Argent does not determine whether an instance is running) and `boot-device` can start one by name. Every other tool - screenshot, describe, tap and gestures, keyboard, launch-app - returns `501 NotImplementedOnPlatform`. Huawei restricts the emulator image download to mainland China; outside it no instance can be created, so there is nothing to list or boot. +> **HarmonyOS drives physical devices; the emulator is discovery and boot only.** Connect a phone with USB debugging enabled and it appears as `harmony-` (`kind: "device"`), driven by `describe`, `screenshot`, `screenshot-diff`, `gesture-tap`, `gesture-swipe`, `keyboard`, `button`, `launch-app`, `restart-app`, `open-url`, `run-sequence` and the two `await-*` tools - all through the device's own `uitest`. DevEco Studio's emulator instances appear separately as `harmony-emulator-`, which `boot-device` starts; Huawei restricts the emulator **image** download to mainland China, so outside it no instance can be created in the first place. Multi-touch and free-form touch (`gesture-pinch`, `gesture-rotate`, `gesture-custom`) return `501 NotImplementedOnPlatform`: `uitest uiInput` injects only whole gestures (click / longClick / swipe / drag / fling), one contact at a time, and `hdc shell` runs as uid 2000 with no access to `/dev/input`. The profilers, the debugger and `rotate` return 501 as unimplemented. ## Capabilities @@ -59,7 +59,7 @@ Argent drives a growing set of targets through a single toolkit, each with the r - For iOS / tvOS: macOS with **Xcode** installed (Apple TV uses tvOS simulators — Xcode downloads the tvOS runtime on demand) - For Android / Android TV: **Android SDK Platform Tools** (`adb`) on `PATH`, and the **Android Emulator** package if you want to boot AVDs from Argent. Create AVDs via Android Studio or `avdmanager`. - For Fire TV (Vega): the **Vega SDK** (`vega` CLI) on `PATH` -- For HarmonyOS: **DevEco Studio**, which ships the `Emulator` manager Argent drives. It is looked up inside DevEco Studio's install root - the macOS app bundle by default, `$DEVECO_STUDIO_HOME` on any other host - and never on `PATH`, since `Emulator` is too generic a name to match safely. Huawei restricts the emulator image download to mainland China, so outside it no emulator instance can be created. +- For HarmonyOS: **DevEco Studio**, which ships both binaries Argent drives - `hdc` (the device connector, for phones) and the `Emulator` manager. Both are looked up inside DevEco Studio's install root - the macOS app bundle by default, `$DEVECO_STUDIO_HOME` on any other host. `hdc` also falls back to `PATH` for hosts that installed the OpenHarmony command-line tools without the IDE; `Emulator` never does, since it is too generic a name to match safely. Driving a phone needs only `hdc`. Huawei restricts the emulator image download to mainland China, so outside it no emulator instance can be created. - For Electron / Chromium: nothing extra to control an already-running app - just launch it with `--remote-debugging-port`, or let Argent spawn your Electron app for you ##### Linux host: extra prerequisites for Android emulators diff --git a/packages/skills/rules/argent.md b/packages/skills/rules/argent.md index 89e3dacb9e..659955bb32 100644 --- a/packages/skills/rules/argent.md +++ b/packages/skills/rules/argent.md @@ -4,7 +4,7 @@ alwaysApply: true --- -If argent is installed and configured in this environment, its MCP tools are the preferred form of interaction with the application for iOS simulator, Android emulator, Chromium (CDP) app, and Vega (Amazon Fire TV) device control; otherwise see `` below before attempting any argent workflow. A "Chromium (CDP) app" is any Chromium runtime exposing a Chrome DevTools Protocol endpoint — an Electron app, or any Chromium-family browser (Chrome/Brave/Edge) launched with `--remote-debugging-port`; all are driven through the same tool surface and tagged `platform: "chromium"`. A "Vega device" is a virtual device (VVD) or physical unit — driven by tv-remote (D-pad) and tagged `platform: "vega"`. HarmonyOS emulator instances (id `harmony-`, tagged `platform: "harmony"`) support only `list-devices` and `boot-device`; every other tool returns 501, so never pick one as an interaction target. +If argent is installed and configured in this environment, its MCP tools are the preferred form of interaction with the application for iOS simulator, Android emulator, Chromium (CDP) app, Vega (Amazon Fire TV), and HarmonyOS device control; otherwise see `` below before attempting any argent workflow. A "Chromium (CDP) app" is any Chromium runtime exposing a Chrome DevTools Protocol endpoint — an Electron app, or any Chromium-family browser (Chrome/Brave/Edge) launched with `--remote-debugging-port`; all are driven through the same tool surface and tagged `platform: "chromium"`. A "Vega device" is a virtual device (VVD) or physical unit — driven by tv-remote (D-pad) and tagged `platform: "vega"`. A "HarmonyOS device" is a phone connected over `hdc` (id `harmony-`, `kind: "device"`, tagged `platform: "harmony"`) — driven by the normal touch tools, with multi-touch, the profilers and the debugger returning 501. HarmonyOS entries with `kind: "emulator"` (id `harmony-emulator-`) are DevEco Studio instances that only `boot-device` accepts, so never pick one of those as an interaction target. Running MCP server and managing the Argent toolkit utilises `argent` command - if asked use `argent --help` for reference. To check current version of MCP server run `argent --version` command. @@ -102,7 +102,7 @@ Load the matching skill before starting work and executing tools from argent-mcp procedure and edge-case handling for each workflow. PLATFORM DETECTION -If the user did not specify a platform, call `list-devices` first and pick the booted target — do not default to iOS. Vega (Amazon Fire TV) devices appear as `platform:"vega"`, when present load `argent-tv-interact`. HarmonyOS entries appear as `platform:"harmony"` with `state:"unknown"` (Argent never reports their readiness); they are boot targets only, so pick one only when the user asked for HarmonyOS. +If the user did not specify a platform, call `list-devices` first and pick the booted target — do not default to iOS. Vega (Amazon Fire TV) devices appear as `platform:"vega"`, when present load `argent-tv-interact`. HarmonyOS entries appear as `platform:"harmony"`; pick one only when the user asked for HarmonyOS, and pick the `kind:"device"` entry (`state:"Connected"`) — a `kind:"emulator"` entry is a boot target, not an interaction target. iOS SIMULATOR SETUP Skill: `argent-ios-simulator-setup` diff --git a/packages/skills/skills/argent-device-interact/SKILL.md b/packages/skills/skills/argent-device-interact/SKILL.md index 234ec90fd8..bc18ab7ebf 100644 --- a/packages/skills/skills/argent-device-interact/SKILL.md +++ b/packages/skills/skills/argent-device-interact/SKILL.md @@ -1,6 +1,6 @@ --- name: argent-device-interact -description: Interact with an iOS simulator, Android emulator, or Chromium (CDP) app using argent MCP tools. Use when tapping UI elements, performing gestures, scrolling/swiping, typing text, pressing hardware buttons, launching apps, opening URLs, taking screenshots, waiting for an element to appear or disappear, or checking visible app state after interactions. +description: Interact with an iOS simulator, Android emulator, HarmonyOS device, or Chromium (CDP) app using argent MCP tools. Use when tapping UI elements, performing gestures, scrolling/swiping, typing text, pressing hardware buttons, launching apps, opening URLs, taking screenshots, waiting for an element to appear or disappear, or checking visible app state after interactions. --- ## Unified tool surface @@ -21,7 +21,7 @@ For platform-specific caveats (Metro `adb reverse`, locked-screen describe error If you delegate simulator tasks to sub-agents, make sure they have MCP permissions. -Use `list-devices` to get a target id. Results are tagged with `platform` (`ios`, `ios-remote`, `android`, `chromium`, `vega`, or `harmony`); booted/ready devices come first. A `vega` target is a TV; drive it with `argent-tv-interact` (see the note above). A `harmony` target is a HarmonyOS emulator instance: only `list-devices` and `boot-device` support it, every tool in this skill returns 501 for one. Pick the first entry that matches the platform you need — if none are ready, call `boot-device` with `udid` (iOS), `avdName` (Android), or `electronAppPath` (boots an Electron app as a `chromium` device). A Chromium browser already running with a CDP port shows up directly — no `boot-device` needed. See `argent-ios-simulator-setup` / `argent-android-emulator-setup` for full setup flow. +Use `list-devices` to get a target id. Results are tagged with `platform` (`ios`, `ios-remote`, `android`, `chromium`, `vega`, or `harmony`); booted/ready devices come first. A `vega` target is a TV; drive it with `argent-tv-interact` (see the note above). A `harmony` target with `kind: "device"` is a HarmonyOS phone on `hdc`: the tools in this skill drive it, except `gesture-pinch`, `gesture-rotate`, `gesture-custom` and `rotate`, which return 501 — the device injects whole gestures one contact at a time, so there is no long-press or multi-finger path. Use `button` for home/back/power only. A `harmony` target with `kind: "emulator"` is a DevEco Studio instance that only `boot-device` accepts. Pick the first entry that matches the platform you need — if none are ready, call `boot-device` with `udid` (iOS), `avdName` (Android), or `electronAppPath` (boots an Electron app as a `chromium` device). A Chromium browser already running with a CDP port shows up directly — no `boot-device` needed. See `argent-ios-simulator-setup` / `argent-android-emulator-setup` for full setup flow. **Load tool schemas before first use.** Gesture tools (`gesture-tap`, `gesture-swipe`, `gesture-pinch`, `gesture-rotate`, `gesture-custom`) may be deferred — their parameter schemas are not loaded until fetched. Always use ToolSearch to load the schemas of all gesture tools you plan to use **before** calling any of them. If you skip this step, parameters may be coerced to strings instead of numbers, causing validation errors. diff --git a/packages/tool-server/src/tools/await-screen-idle/index.ts b/packages/tool-server/src/tools/await-screen-idle/index.ts index 1d06541054..3b6c2c80dd 100644 --- a/packages/tool-server/src/tools/await-screen-idle/index.ts +++ b/packages/tool-server/src/tools/await-screen-idle/index.ts @@ -30,7 +30,9 @@ const zodSchema = z.object({ udid: z .string() .min(1) - .describe("Target device id from `list-devices` (iOS UDID, Android serial, or Chromium id)."), + .describe( + "Target device id from `list-devices` (iOS UDID, Android serial, HarmonyOS id, or Chromium id)." + ), timeoutMs: z .number() .int() diff --git a/packages/tool-server/src/tools/await-ui-element/index.ts b/packages/tool-server/src/tools/await-ui-element/index.ts index f7acd34359..7689637825 100644 --- a/packages/tool-server/src/tools/await-ui-element/index.ts +++ b/packages/tool-server/src/tools/await-ui-element/index.ts @@ -58,7 +58,9 @@ const zodSchema = z udid: z .string() .min(1) - .describe("Target device id from `list-devices` (iOS UDID, Android serial, or Chromium id)."), + .describe( + "Target device id from `list-devices` (iOS UDID, Android serial, HarmonyOS id, or Chromium id)." + ), condition: z .enum(["exists", "visible", "hidden", "text"]) .describe( diff --git a/packages/tool-server/src/tools/button/index.ts b/packages/tool-server/src/tools/button/index.ts index 4a5bbcd811..dae0eb7b39 100644 --- a/packages/tool-server/src/tools/button/index.ts +++ b/packages/tool-server/src/tools/button/index.ts @@ -11,7 +11,9 @@ import { ensureDep } from "../../utils/check-deps"; const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)); const zodSchema = z.object({ - udid: z.string().describe("Target device id from `list-devices` (iOS UDID or Android serial)."), + udid: z + .string() + .describe("Target device id from `list-devices` (iOS UDID, Android serial, or HarmonyOS id)."), button: z .enum(["home", "back", "power", "volumeUp", "volumeDown", "appSwitch", "actionButton"]) .describe("Hardware button to press"), diff --git a/packages/tool-server/src/tools/describe/index.ts b/packages/tool-server/src/tools/describe/index.ts index 06d602521f..ade246f022 100644 --- a/packages/tool-server/src/tools/describe/index.ts +++ b/packages/tool-server/src/tools/describe/index.ts @@ -40,7 +40,7 @@ const zodSchema = z.object({ .string() .min(1) .describe( - "Target device id from `list-devices` (iOS UDID, Android serial, Vega serial, or Chromium id)." + "Target device id from `list-devices` (iOS UDID, Android serial, Vega serial, HarmonyOS id, or Chromium id)." ), bundleId: z .string() diff --git a/packages/tool-server/src/tools/gesture-swipe/index.ts b/packages/tool-server/src/tools/gesture-swipe/index.ts index 5b3885f049..5e4800bf3a 100644 --- a/packages/tool-server/src/tools/gesture-swipe/index.ts +++ b/packages/tool-server/src/tools/gesture-swipe/index.ts @@ -15,7 +15,9 @@ const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)); const SETTLE_EASE_EXPONENT = 3; const zodSchema = z.object({ - udid: z.string().describe("Target device id from `list-devices` (iOS UDID or Android serial)."), + udid: z + .string() + .describe("Target device id from `list-devices` (iOS UDID, Android serial, or HarmonyOS id)."), fromX: z.number().describe("Start x: normalized 0.0–1.0 (not pixels; same as tap)"), fromY: z.number().describe("Start y: normalized 0.0–1.0 (not pixels; same as tap)"), toX: z.number().describe("End x: normalized 0.0–1.0 (not pixels; same as tap)"), diff --git a/packages/tool-server/src/tools/gesture-tap/index.ts b/packages/tool-server/src/tools/gesture-tap/index.ts index 2c1883f00a..75b4a5dcbb 100644 --- a/packages/tool-server/src/tools/gesture-tap/index.ts +++ b/packages/tool-server/src/tools/gesture-tap/index.ts @@ -13,7 +13,9 @@ const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)); const zodSchema = z.object({ udid: z .string() - .describe("Target device id from `list-devices` (iOS UDID, Android serial, or Chromium id)."), + .describe( + "Target device id from `list-devices` (iOS UDID, Android serial, HarmonyOS id, or Chromium id)." + ), x: z.number().describe("Normalized horizontal position 0.0–1.0 (left=0, right=1), not pixels"), y: z.number().describe("Normalized vertical position 0.0–1.0 (top=0, bottom=1), not pixels"), clickCount: z diff --git a/packages/tool-server/src/tools/keyboard/index.ts b/packages/tool-server/src/tools/keyboard/index.ts index 4d985235cc..bc46e460ef 100644 --- a/packages/tool-server/src/tools/keyboard/index.ts +++ b/packages/tool-server/src/tools/keyboard/index.ts @@ -13,7 +13,7 @@ const zodSchema = z.object({ udid: z .string() .describe( - "Target device id from `list-devices` (iOS UDID, Android serial, Vega serial, or Chromium id)." + "Target device id from `list-devices` (iOS UDID, Android serial, Vega serial, HarmonyOS id, or Chromium id)." ), text: z .string() diff --git a/packages/tool-server/src/tools/launch-app/index.ts b/packages/tool-server/src/tools/launch-app/index.ts index 08e29a5880..5a10ad6151 100644 --- a/packages/tool-server/src/tools/launch-app/index.ts +++ b/packages/tool-server/src/tools/launch-app/index.ts @@ -33,7 +33,9 @@ const zodSchema = z.object({ udid: z .string() .min(1) - .describe("Target device id from `list-devices` (iOS UDID, Android serial, or Chromium id)."), + .describe( + "Target device id from `list-devices` (iOS UDID, Android serial, HarmonyOS id, or Chromium id)." + ), bundleId: z .string() .regex(BUNDLE_ID_PATTERN, "bundleId may only contain letters, digits, '.', '_' and '-'") diff --git a/packages/tool-server/src/tools/open-url/index.ts b/packages/tool-server/src/tools/open-url/index.ts index 6473af52bb..efc4cbe915 100644 --- a/packages/tool-server/src/tools/open-url/index.ts +++ b/packages/tool-server/src/tools/open-url/index.ts @@ -14,7 +14,9 @@ const zodSchema = z.object({ udid: z .string() .min(1) - .describe("Target device id from `list-devices` (iOS UDID, Android serial, or Chromium id)."), + .describe( + "Target device id from `list-devices` (iOS UDID, Android serial, HarmonyOS id, or Chromium id)." + ), url: z .string() .describe( diff --git a/packages/tool-server/src/tools/restart-app/index.ts b/packages/tool-server/src/tools/restart-app/index.ts index 37b254e9d9..cd5958f7d8 100644 --- a/packages/tool-server/src/tools/restart-app/index.ts +++ b/packages/tool-server/src/tools/restart-app/index.ts @@ -27,7 +27,7 @@ const zodSchema = z.object({ udid: z .string() .min(1) - .describe("Target device id from `list-devices` (iOS UDID or Android serial)."), + .describe("Target device id from `list-devices` (iOS UDID, Android serial, or HarmonyOS id)."), bundleId: z .string() .min(1) diff --git a/packages/tool-server/src/tools/run-sequence/index.ts b/packages/tool-server/src/tools/run-sequence/index.ts index 65e3e7222d..a1b5753705 100644 --- a/packages/tool-server/src/tools/run-sequence/index.ts +++ b/packages/tool-server/src/tools/run-sequence/index.ts @@ -27,7 +27,7 @@ const zodSchema = z.object({ udid: z .string() .describe( - "Target device id from `list-devices` (iOS UDID, Android serial, Vega serial, or Chromium id) — shared across all steps." + "Target device id from `list-devices` (iOS UDID, Android serial, Vega serial, HarmonyOS id, or Chromium id) — shared across all steps." ), steps: z .array( @@ -77,6 +77,10 @@ const capability: ToolCapability = { // capability gate (HTTP layer's assertSupported) would reject a Vega udid // before any step runs — each step's own capability is still enforced below. vega: { vvd: true }, + // Same for HarmonyOS: `gesture-tap`, `gesture-swipe`, `button`, `keyboard` + // and `await-ui-element` all support it, and batching matters more there + // than anywhere else — every HarmonyOS step is an `hdc` process spawn. + harmony: { device: true }, }; export function createRunSequenceTool( @@ -90,7 +94,7 @@ export function createRunSequenceTool( failedMsg: ({ failureSignal }) => `Failed to run interaction sequence: ${failureSignal.error_code}`, }, - description: `Execute multiple device interaction steps in a single call (iOS simulator, Android emulator, Apple TV / Android TV, or Chromium app). + description: `Execute multiple device interaction steps in a single call (iOS simulator, Android emulator, Apple TV / Android TV, HarmonyOS device, or Chromium app). Use when you need sequential actions and do NOT need to observe the screen between them (e.g. scrolling multiple times, typing then pressing enter, rotating back and forth). Returns { completed, total, steps } with per-step results. Fails if an unrecognised tool name is used in a step (error returned at that step, execution stops). @@ -102,20 +106,20 @@ a prior tap), use individual tool calls instead. Allowed tools and their args (udid is auto-injected, do NOT include it in args): - gesture-tap: { x: number, y: number, clickCount?: number } [ios/android/chromium] - gesture-swipe: { fromX: number, fromY: number, toX: number, toY: number, durationMs?: number } [ios/android] + gesture-tap: { x: number, y: number, clickCount?: number } [ios/android/chromium/harmony] + gesture-swipe: { fromX: number, fromY: number, toX: number, toY: number, durationMs?: number } [ios/android/harmony] gesture-scroll: { x: number, y: number, deltaX?: number, deltaY?: number, durationMs?: number } [chromium only] gesture-drag: { fromX: number, fromY: number, toX: number, toY: number, durationMs?: number } [chromium only] gesture-custom: { events: [{ type: "Down"|"Move"|"Up", x: number, y: number, x2?: number, y2?: number, delayMs?: number }], interpolate?: number } [ios/android] gesture-pinch: { centerX: number, centerY: number, startDistance: number, endDistance: number, endCenterX?: number, endCenterY?: number, angle?: number, durationMs?: number } [ios/android] gesture-rotate: { centerX: number, centerY: number, radius?: number, radiusX?: number, radiusY?: number, startAngle: number, endAngle: number, durationMs?: number } [ios/android] - button: { button: "home"|"back"|"power"|"volumeUp"|"volumeDown"|"appSwitch"|"actionButton" } [ios/android] - keyboard: { text?: string, key?: string, delayMs?: number } (key pressed after text; TV: text only) [ios/android/chromium/vega/tv] + button: { button: "home"|"back"|"power"|"volumeUp"|"volumeDown"|"appSwitch"|"actionButton" } [ios/android/harmony — harmony: home/back/power only] + keyboard: { text?: string, key?: string, delayMs?: number } (key pressed after text; TV: text only) [ios/android/chromium/vega/tv/harmony] text supports {{secret:}} placeholders, resolved server-side from ARGENT_SECRET_ env vars or an argent secrets file — credentials never enter agent context rotate: { orientation: "Portrait"|"LandscapeLeft"|"LandscapeRight"|"PortraitUpsideDown" } [ios/android] tv-remote: { button: , repeat?: number } [apple tv/android tv/vega] buttons: up/down/left/right/select/back/home/menu/playPause (+ rewind/fastForward/next/previous/volumeUp/volumeDown/mute — work on Android TV and Vega; rejected on the Apple TV simulator) - await-ui-element: { condition: "exists"|"visible"|"hidden"|"text", selector: {text?,identifier?,role?}, expectedText?, timeoutMs?, pollIntervalMs? } [ios/android/chromium] + await-ui-element: { condition: "exists"|"visible"|"hidden"|"text", selector: {text?,identifier?,role?}, expectedText?, timeoutMs?, pollIntervalMs? } [ios/android/chromium/harmony] Example — scroll down three times (use gesture-scroll with positive deltaY on Chromium): { "udid": "", "steps": [ diff --git a/packages/tool-server/src/tools/screenshot-diff/index.ts b/packages/tool-server/src/tools/screenshot-diff/index.ts index eada184943..89843980c1 100644 --- a/packages/tool-server/src/tools/screenshot-diff/index.ts +++ b/packages/tool-server/src/tools/screenshot-diff/index.ts @@ -12,7 +12,10 @@ import type { ToolDefinition, } from "@argent/registry"; import { simulatorServerRef, type SimulatorServerApi } from "../../blueprints/simulator-server"; -import { resolveDevice } from "../../utils/device-info"; +import { resolveDevice, harmonyConnectKey } from "../../utils/device-info"; +import { UnsupportedOperationError } from "../../utils/capability"; +import { captureHarmonyScreenshotPng } from "../../utils/harmony-screen"; +import { ensureDep } from "../../utils/check-deps"; import { httpScreenshot } from "../../utils/simulator-client"; import { requireArtifacts, type ArtifactHandle } from "../../artifacts"; import { diffPngFiles } from "./screenshot-diff"; @@ -32,7 +35,9 @@ const zodSchema = z udid: z .string() .min(1) - .describe("Target device id from `list-devices` (iOS UDID or Android serial)."), + .describe( + "Target device id from `list-devices` (iOS UDID, Android serial, or HarmonyOS id)." + ), captureBaseline: z.coerce .boolean() .optional() @@ -48,7 +53,9 @@ const zodSchema = z rotation: z .enum(["Portrait", "LandscapeLeft", "LandscapeRight", "PortraitUpsideDown"]) .optional() - .describe("Orientation override for live baseline/current captures."), + .describe( + "Orientation override for live baseline/current captures. Rejected on HarmonyOS, which captures the display in its current orientation and has no override." + ), outputDir: z .string() .min(1) @@ -77,6 +84,7 @@ type CaptureScreenshot = typeof httpScreenshot; const capability: ToolCapability = { apple: { simulator: true, device: true }, android: { emulator: true, device: true, unknown: true }, + harmony: { device: true }, }; /** @@ -100,7 +108,7 @@ export const screenshotDiffTool: ToolDefinition = failedMsg: ({ failureSignal }) => `Failed to compare screenshots: ${failureSignal.error_code}`, }, description: `Compare two PNG screenshots and return a compact visual-diff summary. -Accepts saved baseline/current PNG paths, or one saved PNG plus one live full-resolution capture from a device. Always provide udid so the simulator-server dependency can be resolved. +Accepts saved baseline/current PNG paths, or one saved PNG plus one live full-resolution capture from a device (iOS, Android or HarmonyOS). Always provide udid so the capture backend can be resolved. Use when stable before/after screenshots exist and the expected result is pixel-visible: layout, spacing, color, typography, image/icon rendering, clipping, overflow, or text rendering. For live captures, set exactly one of captureBaseline or captureCurrent; use baselinePath + captureCurrent for the common visual-regression flow. Returns { summary, diffPath, contextDiffPath }. The summary uses normalized [0,1] screen locations matching describe coordinates; diffPath is the full-size diff image and contextDiffPath is a downscaled image for MCP/agent display. @@ -116,10 +124,13 @@ Fails if the input sources are invalid, PNG files cannot be read, outputDir cann // Requesting it unconditionally causes it to be resolved (and started) even // for pure static-PNG diffs, which fails on tvOS simulators that have no // SimulatorServer backend. - if (params.captureBaseline || params.captureCurrent) { - return { simulatorServer: simulatorServerRef(resolveDevice(params.udid)) }; - } - return {}; + if (!params.captureBaseline && !params.captureCurrent) return {}; + const device = resolveDevice(params.udid); + // HarmonyOS captures on-device over `hdc`; it has no simulator-server + // controller at all, so resolving the blueprint for one would throw before + // the capture path runs. + if (device.platform === "harmony") return {}; + return { simulatorServer: simulatorServerRef(device) }; }, async execute(services, params, options) { return executeScreenshotDiffTool(services, params, options); @@ -194,31 +205,72 @@ async function resolveInputPaths( ): Promise<{ baselinePath: string; currentPath: string }> { validateInputSources(params); + const capture = liveCapture(services, params, options, captureScreenshot); + const baselinePath = params.captureBaseline - ? await captureLiveInput({ - api: requireSimulatorServer(services), - outputDir, - name: "baseline", - rotation: params.rotation, - signal: options?.signal, - captureScreenshot, - }) + ? await captureLiveInput({ capture, outputDir, name: "baseline" }) : params.baselinePath!; const currentPath = params.captureCurrent - ? await captureLiveInput({ - api: requireSimulatorServer(services), - outputDir, - name: "current", - rotation: params.rotation, - signal: options?.signal, - captureScreenshot, - }) + ? await captureLiveInput({ capture, outputDir, name: "current" }) : params.currentPath!; return { baselinePath, currentPath }; } +/** + * How this device produces a live full-resolution PNG, as a path on this host. + * + * iOS and Android go through the simulator-server; HarmonyOS has no controller + * there and captures on-device with `uitest screenCap` instead, the same split + * the `screenshot` tool makes. + */ +function liveCapture( + services: Record, + params: Params, + options: Partial | undefined, + captureScreenshot: CaptureScreenshot +): () => Promise { + const device = resolveDevice(params.udid); + + if (device.platform === "harmony") { + if (params.rotation) { + // `uitest screenCap` captures the display as the device is currently + // oriented and takes no orientation argument, so honouring this would + // mean silently returning an unrotated image — and a diff of the wrong + // orientation reads as a huge legitimate visual change. + throw new UnsupportedOperationError( + "screenshot-diff", + device, + "rotation is not supported for a live HarmonyOS capture: `uitest screenCap` captures the display in its current orientation and has no override. Rotate the device itself, or drop the rotation parameter." + ); + } + return async () => { + await ensureDep("hdc"); + return captureHarmonyScreenshotPng({ + connectKey: harmonyConnectKey(device.id), + scale: 1.0, + }); + }; + } + + return async () => { + const api = requireSimulatorServer(services); + // Prefer a full-resolution capture for maximum diff fidelity. Some Android + // emulator configurations cannot stream a full-res frame — the simulator-server + // rejects it with a "wrong data size" framebuffer mismatch — which previously + // made the entire baselinePath + captureCurrent flow unusable on Android. Fall + // back to the server's default scale, which captures reliably; same-aspect + // normalization in diffPngFiles keeps a scaled capture diff-compatible with a + // baseline saved at any scale. Full-res is preserved wherever it works (iOS). + try { + return (await captureScreenshot(api, params.rotation, options?.signal, 1.0)).path; + } catch { + return (await captureScreenshot(api, params.rotation, options?.signal)).path; + } + }; +} + function validateInputSources(params: Params): void { const invalid = (message: string, stage: string): FailureError => new FailureError(message, { @@ -276,31 +328,14 @@ function requireSimulatorServer(services: Record): SimulatorSer } async function captureLiveInput(params: { - // Resolved and validated by requireSimulatorServer at the call site, so it is - // never undefined here. - api: SimulatorServerApi; + capture: () => Promise; outputDir: string; name: "baseline" | "current"; - rotation?: Params["rotation"]; - signal?: AbortSignal; - captureScreenshot: CaptureScreenshot; }): Promise { - // Prefer a full-resolution capture for maximum diff fidelity. Some Android - // emulator configurations cannot stream a full-res frame — the simulator-server - // rejects it with a "wrong data size" framebuffer mismatch — which previously - // made the entire baselinePath + captureCurrent flow unusable on Android. Fall - // back to the server's default scale, which captures reliably; same-aspect - // normalization in diffPngFiles keeps a scaled capture diff-compatible with a - // baseline saved at any scale. Full-res is preserved wherever it works (iOS). - let capture: Awaited>; - try { - capture = await params.captureScreenshot(params.api, params.rotation, params.signal, 1.0); - } catch { - capture = await params.captureScreenshot(params.api, params.rotation, params.signal); - } + const capturedPath = await params.capture(); const suffix = crypto.randomBytes(4).toString("hex"); const destination = path.join(params.outputDir, `${params.name}-${suffix}.live.png`); await fs.mkdir(params.outputDir, { recursive: true }); - await fs.copyFile(capture.path, destination); + await fs.copyFile(capturedPath, destination); return destination; } diff --git a/packages/tool-server/src/tools/screenshot/index.ts b/packages/tool-server/src/tools/screenshot/index.ts index fc54a31b5f..42a504f61b 100644 --- a/packages/tool-server/src/tools/screenshot/index.ts +++ b/packages/tool-server/src/tools/screenshot/index.ts @@ -13,6 +13,7 @@ import { simctlArgsForUdid } from "../../utils/ios-device-sets"; import { captureVegaScreenshotPng } from "../../utils/vega-screen"; import { captureHarmonyScreenshotPng } from "../../utils/harmony-screen"; import { ensureDep } from "../../utils/check-deps"; +import { UnsupportedOperationError } from "../../utils/capability"; import { requireArtifacts, type ArtifactHandle } from "../../artifacts"; const execFileAsync = promisify(execFile); @@ -21,13 +22,13 @@ const zodSchema = z.object({ udid: z .string() .describe( - "Target device id from `list-devices` (iOS UDID, Android serial, Apple TV UDID, Vega serial, or Chromium id)." + "Target device id from `list-devices` (iOS UDID, Android serial, Apple TV UDID, Vega serial, HarmonyOS id, or Chromium id)." ), rotation: z .enum(["Portrait", "LandscapeLeft", "LandscapeRight", "PortraitUpsideDown"]) .optional() .describe( - "Orientation override for the screenshot (rotates the captured image after Page.captureScreenshot on Chromium)." + "Orientation override for the screenshot (rotates the captured image after Page.captureScreenshot on Chromium). Rejected on HarmonyOS, which captures the display in its current orientation and has no override." ), scale: z .number() @@ -184,6 +185,16 @@ Fails if the simulator-server / emulator backend / Chromium CDP is not reachable // HarmonyOS captures on-device with `uitest screenCap` and copies the PNG // back over hdc; there is no simulator-server controller for the platform. if (device.platform === "harmony") { + if (params.rotation) { + // `uitest screenCap` writes the display as it is currently oriented + // and takes no orientation argument, so accepting this would return + // an unrotated image under a name that says otherwise. + throw new UnsupportedOperationError( + "screenshot", + device, + "rotation is not supported on HarmonyOS: `uitest screenCap` captures the display in its current orientation and has no override. Rotate the device itself, or drop the rotation parameter." + ); + } await ensureDep("hdc"); const pngPath = await captureHarmonyScreenshotPng({ connectKey: harmonyConnectKey(device.id), diff --git a/packages/tool-server/test/run-sequence.test.ts b/packages/tool-server/test/run-sequence.test.ts index 1635f75121..351c02c28f 100644 --- a/packages/tool-server/test/run-sequence.test.ts +++ b/packages/tool-server/test/run-sequence.test.ts @@ -1,6 +1,10 @@ import { describe, it, expect, vi } from "vitest"; import type { Registry, ToolContext } from "@argent/registry"; import { createRunSequenceTool } from "../src/tools/run-sequence"; +import { gestureTapTool } from "../src/tools/gesture-tap"; +import { gesturePinchTool } from "../src/tools/gesture-pinch"; +import { assertSupported } from "../src/utils/capability"; +import { resolveDevice } from "../src/utils/device-info"; // A minimal registry stub: records every invokeTool call and returns a marker. function makeMockRegistry() { @@ -30,6 +34,7 @@ const TVOS_UDID = "DDDDDDDD-DDDD-DDDD-DDDD-DDDDDDDDDDDD"; // iOS-shaped udid so `resolveDevice` classifies it as an iOS simulator without // touching a real device (classification is purely shape-based). const IOS = "11111111-1111-1111-1111-111111111111"; +const HARMONY = "harmony-025DEK236V035771"; describe("run-sequence", () => { it("allows TV steps and dispatches them in order with the shared udid injected", async () => { @@ -58,6 +63,46 @@ describe("run-sequence", () => { expect(calls[0]!.args).toMatchObject({ button: "right", udid: TVOS_UDID }); }); + it("accepts a HarmonyOS device, and stops at the first step that platform cannot run", async () => { + const { registry, calls } = makeMockRegistry(); + const tool = createRunSequenceTool(registry); + // The outer gate runs in the HTTP layer, not in execute(), so assert it + // directly: without `harmony` on run-sequence's own capability every step + // below is unreachable no matter what the step tools support. + expect(() => + assertSupported("run-sequence", tool.capability, resolveDevice(HARMONY)) + ).not.toThrow(); + + // Real capabilities: gesture-tap supports HarmonyOS, gesture-pinch does not + // (`uitest uiInput` has no multi-touch verb). + registry.getTool = vi.fn((id: string) => + id === "gesture-tap" ? gestureTapTool : gesturePinchTool + ); + + const result = await tool.execute!( + {}, + { + udid: HARMONY, + steps: [ + { tool: "gesture-tap", args: { x: 0.5, y: 0.5 } }, + { + tool: "gesture-pinch", + args: { centerX: 0.5, centerY: 0.5, startDistance: 0.1, endDistance: 0.3 }, + }, + { tool: "gesture-tap", args: { x: 0.5, y: 0.6 } }, + ], + } + ); + + expect(result.completed).toBe(1); + expect(calls.map((c) => c.tool)).toEqual(["gesture-tap"]); + expect(calls[0]!.args.udid).toBe(HARMONY); + const blocked = result.steps[1]; + expect(blocked && "error" in blocked && blocked.error).toContain( + "is not supported on harmony device" + ); + }); + it("rejects a tool that isn't in the allow-list and stops the sequence", async () => { const { registry, calls } = makeMockRegistry(); const tool = createRunSequenceTool(registry); diff --git a/packages/tool-server/test/screenshot-diff-tool.test.ts b/packages/tool-server/test/screenshot-diff-tool.test.ts index 3d0a8ea820..812d1a20b8 100644 --- a/packages/tool-server/test/screenshot-diff-tool.test.ts +++ b/packages/tool-server/test/screenshot-diff-tool.test.ts @@ -5,6 +5,12 @@ import { PNG } from "pngjs"; import { describe, expect, it, vi } from "vitest"; import { ArtifactStore } from "@argent/registry"; import { executeScreenshotDiffTool, screenshotDiffTool } from "../src/tools/screenshot-diff"; +import { captureHarmonyScreenshotPng } from "../src/utils/harmony-screen"; + +// HarmonyOS captures over `hdc`; both are stubbed so the live-capture branch is +// exercised without a device. +vi.mock("../src/utils/harmony-screen", () => ({ captureHarmonyScreenshotPng: vi.fn() })); +vi.mock("../src/utils/check-deps", () => ({ ensureDep: vi.fn(async () => {}) })); describe("screenshotDiffTool", () => { it("rejects public tuning options so defaults stay internal", () => { @@ -244,6 +250,76 @@ describe("screenshotDiffTool", () => { ) ).rejects.toThrow("Provide either currentPath or captureCurrent, not both."); }); + + it("captures a live HarmonyOS side over hdc instead of the simulator-server", async () => { + const dir = await fs.mkdtemp(path.join(os.tmpdir(), "argent-screenshot-diff-harmony-")); + const baselinePath = path.join(dir, "baseline.png"); + const capturedPath = path.join(dir, "captured.png"); + await writePng(baselinePath, 2, 2, { r: 0, g: 0, b: 0 }); + await writePng(capturedPath, 2, 2, { r: 0, g: 0, b: 0 }); + vi.mocked(captureHarmonyScreenshotPng).mockResolvedValue(capturedPath); + // Passed to prove the sim-server path is not taken: it would be called with + // this stub, and HarmonyOS has no simulator-server controller to call. + const captureScreenshot = vi.fn(); + + const result = await executeScreenshotDiffTool( + {}, + { baselinePath, captureCurrent: true, udid: "harmony-025DEK236V035771", outputDir: dir }, + { artifacts: new ArtifactStore() }, + captureScreenshot as never + ); + + expect(captureScreenshot).not.toHaveBeenCalled(); + // Full resolution, not ARGENT_SCREENSHOT_SCALE's 0.3 default: a diff against + // a full-res baseline is only as precise as the coarser of the two images. + expect(captureHarmonyScreenshotPng).toHaveBeenCalledWith({ + connectKey: "025DEK236V035771", + scale: 1.0, + }); + const liveCaptures = (await fs.readdir(dir)).filter((name) => + /^current-[a-f0-9]{8}\.live\.png$/.test(name) + ); + expect(liveCaptures).toHaveLength(1); + expect(result.diffPath).toBeTruthy(); + }); + + it("rejects a rotation override on a live HarmonyOS capture", async () => { + const dir = await fs.mkdtemp(path.join(os.tmpdir(), "argent-screenshot-diff-harmony-rot-")); + const baselinePath = path.join(dir, "baseline.png"); + await writePng(baselinePath, 2, 2, { r: 0, g: 0, b: 0 }); + vi.mocked(captureHarmonyScreenshotPng).mockClear().mockResolvedValue(baselinePath); + + // `uitest screenCap` has no orientation argument, so accepting this would + // diff an unrotated capture against a rotated baseline and report the whole + // screen as changed. + await expect( + executeScreenshotDiffTool( + {}, + { + baselinePath, + captureCurrent: true, + udid: "harmony-025DEK236V035771", + rotation: "LandscapeLeft", + outputDir: dir, + }, + { artifacts: new ArtifactStore() } + ) + ).rejects.toThrow(/rotation is not supported/); + expect(captureHarmonyScreenshotPng).not.toHaveBeenCalled(); + }); + + it("declares no simulator-server service for a HarmonyOS live capture", () => { + // Resolving the iOS/Android-only blueprint for a HarmonyOS device throws + // before the capture path runs. + expect( + screenshotDiffTool.services({ + baselinePath: "/tmp/baseline.png", + captureCurrent: true, + udid: "harmony-025DEK236V035771", + outputDir: "/tmp", + }) + ).toEqual({}); + }); }); async function writePng( diff --git a/packages/tool-server/test/screenshot-tool.test.ts b/packages/tool-server/test/screenshot-tool.test.ts index be2b900540..c13dbdcb35 100644 --- a/packages/tool-server/test/screenshot-tool.test.ts +++ b/packages/tool-server/test/screenshot-tool.test.ts @@ -1,6 +1,10 @@ import { afterEach, describe, expect, it, vi } from "vitest"; import { ArtifactStore } from "@argent/registry"; import { createScreenshotTool } from "../src/tools/screenshot"; +import { captureHarmonyScreenshotPng } from "../src/utils/harmony-screen"; + +vi.mock("../src/utils/harmony-screen", () => ({ captureHarmonyScreenshotPng: vi.fn() })); +vi.mock("../src/utils/check-deps", () => ({ ensureDep: vi.fn(async () => {}) })); describe("screenshot tool", () => { afterEach(() => { @@ -48,4 +52,27 @@ describe("screenshot tool", () => { expect(result).not.toHaveProperty("includeImageInContext"); expect(result).not.toHaveProperty("url"); }); + + it("refuses a rotation override on HarmonyOS rather than returning an unrotated capture", async () => { + const registry = { + resolveService: vi.fn(), + } as unknown as import("@argent/registry").Registry; + const screenshotTool = createScreenshotTool(registry); + + // `uitest screenCap` writes the display in its current orientation and has + // no override, so the only alternative to this error is a capture whose + // orientation silently contradicts the parameter that was accepted. + await expect( + screenshotTool.execute( + {}, + { + udid: "harmony-025DEK236V035771", + rotation: "LandscapeLeft", + includeImageInContext: true, + }, + { artifacts: new ArtifactStore() } + ) + ).rejects.toThrow(/rotation is not supported on HarmonyOS/); + expect(captureHarmonyScreenshotPng).not.toHaveBeenCalled(); + }); }); From 26b892c89db6cb8c0d9dc27259de46f1c6cc128e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Mon, 10 Aug 2026 22:48:56 +0200 Subject: [PATCH 007/110] fix(harmony): scope the screenshot-diff rotation guard to a live capture MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The guard ran while the capture strategy was being chosen, so it also rejected a two-path diff that merely carried `rotation` — a parameter that is inert for a static diff on every platform, making HarmonyOS the one platform where an unused argument failed the call. It now runs inside the capture itself, where the parameter actually has no effect to give. --- .../src/tools/screenshot-diff/index.ts | 24 +++++++++-------- .../test/screenshot-diff-tool.test.ts | 27 +++++++++++++++++++ 2 files changed, 40 insertions(+), 11 deletions(-) diff --git a/packages/tool-server/src/tools/screenshot-diff/index.ts b/packages/tool-server/src/tools/screenshot-diff/index.ts index 89843980c1..040d92df2f 100644 --- a/packages/tool-server/src/tools/screenshot-diff/index.ts +++ b/packages/tool-server/src/tools/screenshot-diff/index.ts @@ -234,18 +234,20 @@ function liveCapture( const device = resolveDevice(params.udid); if (device.platform === "harmony") { - if (params.rotation) { - // `uitest screenCap` captures the display as the device is currently - // oriented and takes no orientation argument, so honouring this would - // mean silently returning an unrotated image — and a diff of the wrong - // orientation reads as a huge legitimate visual change. - throw new UnsupportedOperationError( - "screenshot-diff", - device, - "rotation is not supported for a live HarmonyOS capture: `uitest screenCap` captures the display in its current orientation and has no override. Rotate the device itself, or drop the rotation parameter." - ); - } return async () => { + if (params.rotation) { + // `uitest screenCap` captures the display as the device is currently + // oriented and takes no orientation argument, so honouring this would + // mean silently returning an unrotated image — and a diff of the wrong + // orientation reads as a huge legitimate visual change. Checked here + // rather than up front so a static two-path diff, where rotation is + // inert on every platform, is not rejected for carrying it. + throw new UnsupportedOperationError( + "screenshot-diff", + device, + "rotation is not supported for a live HarmonyOS capture: `uitest screenCap` captures the display in its current orientation and has no override. Rotate the device itself, or drop the rotation parameter." + ); + } await ensureDep("hdc"); return captureHarmonyScreenshotPng({ connectKey: harmonyConnectKey(device.id), diff --git a/packages/tool-server/test/screenshot-diff-tool.test.ts b/packages/tool-server/test/screenshot-diff-tool.test.ts index 812d1a20b8..8c33c4d397 100644 --- a/packages/tool-server/test/screenshot-diff-tool.test.ts +++ b/packages/tool-server/test/screenshot-diff-tool.test.ts @@ -308,6 +308,33 @@ describe("screenshotDiffTool", () => { expect(captureHarmonyScreenshotPng).not.toHaveBeenCalled(); }); + it("still diffs two saved PNGs on HarmonyOS when a rotation is passed", async () => { + // rotation only ever applies to a live capture, and is inert on every + // platform for a two-path diff — rejecting it here would make HarmonyOS + // the one platform where an unused parameter fails the call. + const dir = await fs.mkdtemp(path.join(os.tmpdir(), "argent-screenshot-diff-harmony-static-")); + const baselinePath = path.join(dir, "baseline.png"); + const currentPath = path.join(dir, "current.png"); + await writePng(baselinePath, 2, 2, { r: 1, g: 2, b: 3 }); + await writePng(currentPath, 2, 2, { r: 1, g: 2, b: 3 }); + vi.mocked(captureHarmonyScreenshotPng).mockClear(); + + const result = await executeScreenshotDiffTool( + {}, + { + baselinePath, + currentPath, + udid: "harmony-025DEK236V035771", + rotation: "LandscapeLeft", + outputDir: dir, + }, + { artifacts: new ArtifactStore() } + ); + + expect(result.summary).toContain("Screenshot diff summary"); + expect(captureHarmonyScreenshotPng).not.toHaveBeenCalled(); + }); + it("declares no simulator-server service for a HarmonyOS live capture", () => { // Resolving the iOS/Android-only blueprint for a HarmonyOS device throws // before the capture path runs. From 63bba282f7f2a2f29362ee47d4ac388ff0bc4fb8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Mon, 10 Aug 2026 22:52:28 +0200 Subject: [PATCH 008/110] docs(harmony): name HarmonyOS where a tool description lists its platforms MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `keyboard`, `launch-app` and `open-url` each enumerate the platforms they drive, and each enumeration was left describing the set from before this branch — an agent reading `keyboard`'s description had no way to learn the tool works on HarmonyOS, or that only four of its named keys land there. --- packages/tool-server/src/tools/keyboard/index.ts | 4 ++-- packages/tool-server/src/tools/launch-app/index.ts | 4 ++-- packages/tool-server/src/tools/open-url/index.ts | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-) diff --git a/packages/tool-server/src/tools/keyboard/index.ts b/packages/tool-server/src/tools/keyboard/index.ts index bc46e460ef..16d2b33082 100644 --- a/packages/tool-server/src/tools/keyboard/index.ts +++ b/packages/tool-server/src/tools/keyboard/index.ts @@ -102,11 +102,11 @@ export function createKeyboardTool(registry: Registry): ToolDefinition `Failed to use keyboard: ${failureSignal.error_code}`, }, - description: `Type text or press special keys on the device (iOS simulator, Android emulator or device, Chromium app, Vega Virtual Device, or Apple TV / Android TV) using keyboard events. + description: `Type text or press special keys on the device (iOS simulator, Android emulator or device, HarmonyOS device, Chromium app, Vega Virtual Device, or Apple TV / Android TV) using keyboard events. Use when you need to enter text or trigger a named key such as enter, escape, or arrow keys. On Vega and Apple TV / Android TV, prefer the remote tools for D-pad navigation; use keyboard to type into a focused text field (e.g. a search or login box). Returns { typed: string, keys: number }. Fails if an unsupported key name is provided or the device's input backend is not reachable. - text: types a string (supports uppercase, digits, common punctuation). To type a credential, use \`{{secret:}}\` — resolved server-side from the \`ARGENT_SECRET_\` env var or an argent secrets file (\`.argent/secrets.env\` in the project, \`~/.argent/secrets.env\`, or an \`ARGENT_SECRET_\`-prefixed key in the project's \`.env\`/\`.env.local\`), so the plaintext never enters agent context; the result echoes the placeholder, not the value, and the after-typing auto-screenshot is skipped. -- key: presses a single named key (enter, escape, backspace, tab, arrow-up/down/left/right, f1–f12) — NOT supported on TV targets; move focus with \`tv-remote\` instead. +- key: presses a single named key (enter, escape, backspace, tab, arrow-up/down/left/right, f1–f12) — NOT supported on TV targets; move focus with \`tv-remote\` instead. HarmonyOS presses enter, backspace, space and the horizontal arrows; any other key name is rejected. On a TV target (runtimeKind 'tv') only \`text\` applies — focus a text field first (with \`tv-remote\`), then type into it (injected HID keyboard on Apple TV, \`adb input text\` on Android TV). Provide text, key, or both — when both are given, the text is typed first and the key is pressed after it (text + key:"enter" types and submits).`, zodSchema, diff --git a/packages/tool-server/src/tools/launch-app/index.ts b/packages/tool-server/src/tools/launch-app/index.ts index 5a10ad6151..c00b2de929 100644 --- a/packages/tool-server/src/tools/launch-app/index.ts +++ b/packages/tool-server/src/tools/launch-app/index.ts @@ -80,9 +80,9 @@ export function createLaunchAppTool(registry: Registry): ToolDefinition `Failed to launch ${params.bundleId}: ${failureSignal.error_code}`, }, - description: `Open an app by its bundle id (iOS) or package name (Android), or confirm the running renderer (Chromium). + description: `Open an app by its bundle id (iOS, HarmonyOS) or package name (Android), or confirm the running renderer (Chromium). Use when starting any app — prefer this over tapping home-screen / launcher icons. Also prepares the native-devtools injection before the app starts (the iOS slice on iOS, the tvOS slice on Apple TV); on tvOS, interaction is focus-driven — use the tv-* tools rather than coordinate taps. -Returns { launched, bundleId }. Fails if the app is not installed on the target device (iOS / Android). +Returns { launched, bundleId }. Fails if the app is not installed on the target device (iOS / Android / HarmonyOS). For Chromium, the app is already running behind a CDP port; this call simply refreshes the cached viewport and acknowledges the bundleId tag. To change the visible route, use \`open-url\`. On Vega (Fire TV), pass the interactive component app id from manifest.toml (e.g. com.example.app.main) as bundleId. diff --git a/packages/tool-server/src/tools/open-url/index.ts b/packages/tool-server/src/tools/open-url/index.ts index efc4cbe915..dec30a284d 100644 --- a/packages/tool-server/src/tools/open-url/index.ts +++ b/packages/tool-server/src/tools/open-url/index.ts @@ -59,7 +59,7 @@ export const openUrlTool: ToolDefinition = { Use to navigate to a web page or deep-link into an app. On Chromium, this navigates the primary renderer to the given URL. Cross-platform schemes: https://, tel:, mailto:. iOS also: messages://, settings://, maps://. Android also: geo:, plus any app-specific deep link. Deep-linking caveat: an https:// link opens the native app only when an installed app is verified for the link's domain (iOS Universal Links / Android App Links) — otherwise it opens in the browser, and on iOS simulators it may open in Safari even when the owning app is installed. To reliably open an installed app, use its custom scheme (scheme://path) or launch-app with its bundle id. -Returns { opened, url, note? }. note carries the deep-linking caveat when a web URL was opened on a native device. Fails if no app is registered to handle the URI (iOS/Android) or the renderer rejects the navigation (Chromium).`, +Returns { opened, url, note? }. note carries the deep-linking caveat when a web URL was opened on a native device. Fails if no app is registered to handle the URI (iOS/Android/HarmonyOS) or the renderer rejects the navigation (Chromium).`, zodSchema, capability, services: (params): Record => { From afca62f75396242a0a2d160748e966d1f2a0f3a8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Mon, 10 Aug 2026 22:54:11 +0200 Subject: [PATCH 009/110] chore(harmony): trim comments that restate the error beside them --- .../tool-server/src/tools/run-sequence/index.ts | 5 ++--- .../tool-server/src/tools/screenshot-diff/index.ts | 14 +++++--------- packages/tool-server/src/tools/screenshot/index.ts | 3 --- 3 files changed, 7 insertions(+), 15 deletions(-) diff --git a/packages/tool-server/src/tools/run-sequence/index.ts b/packages/tool-server/src/tools/run-sequence/index.ts index a1b5753705..df7674cb48 100644 --- a/packages/tool-server/src/tools/run-sequence/index.ts +++ b/packages/tool-server/src/tools/run-sequence/index.ts @@ -77,9 +77,8 @@ const capability: ToolCapability = { // capability gate (HTTP layer's assertSupported) would reject a Vega udid // before any step runs — each step's own capability is still enforced below. vega: { vvd: true }, - // Same for HarmonyOS: `gesture-tap`, `gesture-swipe`, `button`, `keyboard` - // and `await-ui-element` all support it, and batching matters more there - // than anywhere else — every HarmonyOS step is an `hdc` process spawn. + // Same for HarmonyOS, where batching pays most: every step there is an + // `hdc` process spawn. harmony: { device: true }, }; diff --git a/packages/tool-server/src/tools/screenshot-diff/index.ts b/packages/tool-server/src/tools/screenshot-diff/index.ts index 040d92df2f..afc7d8e6eb 100644 --- a/packages/tool-server/src/tools/screenshot-diff/index.ts +++ b/packages/tool-server/src/tools/screenshot-diff/index.ts @@ -126,9 +126,8 @@ Fails if the input sources are invalid, PNG files cannot be read, outputDir cann // SimulatorServer backend. if (!params.captureBaseline && !params.captureCurrent) return {}; const device = resolveDevice(params.udid); - // HarmonyOS captures on-device over `hdc`; it has no simulator-server - // controller at all, so resolving the blueprint for one would throw before - // the capture path runs. + // HarmonyOS has no simulator-server controller, so resolving the blueprint + // for one would throw before the capture path runs. if (device.platform === "harmony") return {}; return { simulatorServer: simulatorServerRef(device) }; }, @@ -236,12 +235,9 @@ function liveCapture( if (device.platform === "harmony") { return async () => { if (params.rotation) { - // `uitest screenCap` captures the display as the device is currently - // oriented and takes no orientation argument, so honouring this would - // mean silently returning an unrotated image — and a diff of the wrong - // orientation reads as a huge legitimate visual change. Checked here - // rather than up front so a static two-path diff, where rotation is - // inert on every platform, is not rejected for carrying it. + // Inside the capture rather than up front: for a static two-path diff + // rotation is inert on every platform, and rejecting it there would + // make HarmonyOS the one platform an unused argument fails. throw new UnsupportedOperationError( "screenshot-diff", device, diff --git a/packages/tool-server/src/tools/screenshot/index.ts b/packages/tool-server/src/tools/screenshot/index.ts index 42a504f61b..4e9e9fa4cb 100644 --- a/packages/tool-server/src/tools/screenshot/index.ts +++ b/packages/tool-server/src/tools/screenshot/index.ts @@ -186,9 +186,6 @@ Fails if the simulator-server / emulator backend / Chromium CDP is not reachable // back over hdc; there is no simulator-server controller for the platform. if (device.platform === "harmony") { if (params.rotation) { - // `uitest screenCap` writes the display as it is currently oriented - // and takes no orientation argument, so accepting this would return - // an unrotated image under a name that says otherwise. throw new UnsupportedOperationError( "screenshot", device, From 44e56d25d0c56330900f9ac11680dce0f9bc0cbb Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Mon, 10 Aug 2026 23:03:28 +0200 Subject: [PATCH 010/110] fix(harmony): say the panel is off even when the dump has windows MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The asleep hint only fired when the parsed tree had no windows at all, on the belief that a suspended display dumps a bare root. It does not: measured on a Mate 60 at `powerStatus=POWER_STATUS_SUSPEND`, `uitest dumpLayout` returns both `com.ohos.sceneboard` windows, visible flags and all — a tree an agent cannot tell apart from a live screen, while every tap injected against it is a silent no-op. So the hint now keys on the power status it is about, and the empty-tree hint keeps the case it was written for. --- .../tools/describe/platforms/harmony/index.ts | 28 ++++---- .../tool-server/test/harmony-describe.test.ts | 72 +++++++++++++++++++ 2 files changed, 86 insertions(+), 14 deletions(-) create mode 100644 packages/tool-server/test/harmony-describe.test.ts diff --git a/packages/tool-server/src/tools/describe/platforms/harmony/index.ts b/packages/tool-server/src/tools/describe/platforms/harmony/index.ts index 37a4527d90..f7600e102a 100644 --- a/packages/tool-server/src/tools/describe/platforms/harmony/index.ts +++ b/packages/tool-server/src/tools/describe/platforms/harmony/index.ts @@ -10,13 +10,16 @@ import { parseHarmonyLayout } from "./layout-parser"; export const harmonyRequires: ToolDependency[] = ["hdc"]; /** - * Said when the dump has no window at all. `uitest` returns a tree with a bare - * root while the display is asleep, and an agent that reads that as "the app - * rendered nothing" will start debugging the app instead of waking the screen. + * A suspended display still dumps its windows — measured on a Mate 60 with + * `powerStatus=POWER_STATUS_SUSPEND`: two visible `com.ohos.sceneboard` windows, + * indistinguishable in the tree from a live screen. Nothing else in the result + * says the panel is off, and injected touches land nowhere while it is, so the + * hint is the only thing standing between an agent and a silent run of + * successful-looking taps. */ const ASLEEP_HINT = - "The display is off, so there is nothing to describe and injected taps would land nowhere. " + - "Wake it with `button` (power) and describe again."; + "The display is off. This tree is what was last composited, and injected taps land nowhere " + + "until the panel is on. Wake it with `button` (power), then describe again."; const EMPTY_HINT = "The layout dump contains no windows. The foreground app may still be starting — " + @@ -38,15 +41,12 @@ export async function describeHarmony(connectKey: string): Promise {}); } diff --git a/packages/tool-server/test/harmony-describe.test.ts b/packages/tool-server/test/harmony-describe.test.ts new file mode 100644 index 0000000000..acb7dca3d4 --- /dev/null +++ b/packages/tool-server/test/harmony-describe.test.ts @@ -0,0 +1,72 @@ +import { describe, expect, it, vi, beforeEach } from "vitest"; +import { harmonyDisplay, harmonyDumpLayout } from "../src/utils/harmony-uitest"; +import { describeHarmony } from "../src/tools/describe/platforms/harmony"; + +vi.mock("../src/utils/harmony-uitest", () => ({ + harmonyDisplay: vi.fn(), + harmonyDumpLayout: vi.fn(), +})); + +const CONNECT_KEY = "025DEK236V035771"; + +/** The two `com.ohos.sceneboard` windows a Mate 60 dumps on its lock screen. */ +function lockScreenDump() { + return { + attributes: { bounds: "[0,0][1216,2688]" }, + children: [ + { + attributes: { + type: "WindowScene", + bundleName: "com.ohos.sceneboard", + bounds: "[0,107][1216,2688]", + }, + children: [{ attributes: { type: "Text", text: "04:39", bounds: "[300,900][900,1200]" } }], + }, + { + attributes: { + type: "WindowScene", + bundleName: "com.ohos.sceneboard", + bounds: "[0,0][1216,188]", + }, + children: [], + }, + ], + }; +} + +beforeEach(() => { + vi.mocked(harmonyDumpLayout).mockReset(); + vi.mocked(harmonyDisplay).mockReset(); +}); + +describe("describeHarmony", () => { + it("warns that the panel is off even though the dump still lists windows", async () => { + // Measured on a suspended Mate 60: `powerStatus=POWER_STATUS_SUSPEND` still + // dumps both sceneboard windows, so the tree alone cannot be told apart + // from a live screen — and every tap injected against it is a silent no-op. + vi.mocked(harmonyDisplay).mockResolvedValue({ width: 1216, height: 2688, screenOn: false }); + vi.mocked(harmonyDumpLayout).mockResolvedValue(lockScreenDump()); + + const result = await describeHarmony(CONNECT_KEY); + + expect(result.tree.children.length).toBeGreaterThan(0); + expect(result.hint).toMatch(/display is off/); + }); + + it("says nothing extra when the screen is on and the dump has windows", async () => { + vi.mocked(harmonyDisplay).mockResolvedValue({ width: 1216, height: 2688, screenOn: true }); + vi.mocked(harmonyDumpLayout).mockResolvedValue(lockScreenDump()); + + expect((await describeHarmony(CONNECT_KEY)).hint).toBeUndefined(); + }); + + it("points at a still-starting app when the screen is on but no window dumped", async () => { + vi.mocked(harmonyDisplay).mockResolvedValue({ width: 1216, height: 2688, screenOn: true }); + vi.mocked(harmonyDumpLayout).mockResolvedValue({ + attributes: { bounds: "[0,0][1216,2688]" }, + children: [], + }); + + expect((await describeHarmony(CONNECT_KEY)).hint).toMatch(/no windows/); + }); +}); From 727e385e1ea4b2d5b2bb8d12d6e631821d44fde8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Mon, 10 Aug 2026 23:05:17 +0200 Subject: [PATCH 011/110] docs(harmony): name every tool that returns 501, and why each does --- README.md | 2 +- packages/skills/rules/argent.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 6227f176c4..5c4c556b72 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ Argent drives a growing set of targets through a single toolkit, each with the r | **Desktop & web** | Electron and Chromium apps (incl. React Native Web / Expo web) over CDP | Mouse / keyboard | | **HarmonyOS** | Phones over `hdc`, and DevEco Studio emulator instances | Touch / gesture | -> **HarmonyOS drives physical devices; the emulator is discovery and boot only.** Connect a phone with USB debugging enabled and it appears as `harmony-` (`kind: "device"`), driven by `describe`, `screenshot`, `screenshot-diff`, `gesture-tap`, `gesture-swipe`, `keyboard`, `button`, `launch-app`, `restart-app`, `open-url`, `run-sequence` and the two `await-*` tools - all through the device's own `uitest`. DevEco Studio's emulator instances appear separately as `harmony-emulator-`, which `boot-device` starts; Huawei restricts the emulator **image** download to mainland China, so outside it no instance can be created in the first place. Multi-touch and free-form touch (`gesture-pinch`, `gesture-rotate`, `gesture-custom`) return `501 NotImplementedOnPlatform`: `uitest uiInput` injects only whole gestures (click / longClick / swipe / drag / fling), one contact at a time, and `hdc shell` runs as uid 2000 with no access to `/dev/input`. The profilers, the debugger and `rotate` return 501 as unimplemented. +> **HarmonyOS drives physical devices; the emulator is discovery and boot only.** Connect a phone with USB debugging enabled and it appears as `harmony-` (`kind: "device"`), driven by `describe`, `screenshot`, `screenshot-diff`, `gesture-tap`, `gesture-swipe`, `keyboard`, `button`, `launch-app`, `restart-app`, `open-url`, `run-sequence` and the two `await-*` tools - all through the device's own `uitest`. DevEco Studio's emulator instances appear separately as `harmony-emulator-`, which `boot-device` starts; Huawei restricts the emulator **image** download to mainland China, so outside it no instance can be created in the first place. Multi-touch and free-form touch (`gesture-pinch`, `gesture-rotate`, `gesture-custom`) return `501 NotImplementedOnPlatform`: `uitest uiInput` injects only whole gestures (click / longClick / swipe / drag / fling), one contact at a time, and `hdc shell` runs as uid 2000 with no access to `/dev/input`. `rotate`, the profilers and the debugger also return 501: the device exposes no orientation CLI, and argent's profiler and debugger surfaces have no HarmonyOS ingest path. ## Capabilities diff --git a/packages/skills/rules/argent.md b/packages/skills/rules/argent.md index 659955bb32..f92ec795ae 100644 --- a/packages/skills/rules/argent.md +++ b/packages/skills/rules/argent.md @@ -4,7 +4,7 @@ alwaysApply: true --- -If argent is installed and configured in this environment, its MCP tools are the preferred form of interaction with the application for iOS simulator, Android emulator, Chromium (CDP) app, Vega (Amazon Fire TV), and HarmonyOS device control; otherwise see `` below before attempting any argent workflow. A "Chromium (CDP) app" is any Chromium runtime exposing a Chrome DevTools Protocol endpoint — an Electron app, or any Chromium-family browser (Chrome/Brave/Edge) launched with `--remote-debugging-port`; all are driven through the same tool surface and tagged `platform: "chromium"`. A "Vega device" is a virtual device (VVD) or physical unit — driven by tv-remote (D-pad) and tagged `platform: "vega"`. A "HarmonyOS device" is a phone connected over `hdc` (id `harmony-`, `kind: "device"`, tagged `platform: "harmony"`) — driven by the normal touch tools, with multi-touch, the profilers and the debugger returning 501. HarmonyOS entries with `kind: "emulator"` (id `harmony-emulator-`) are DevEco Studio instances that only `boot-device` accepts, so never pick one of those as an interaction target. +If argent is installed and configured in this environment, its MCP tools are the preferred form of interaction with the application for iOS simulator, Android emulator, Chromium (CDP) app, Vega (Amazon Fire TV), and HarmonyOS device control; otherwise see `` below before attempting any argent workflow. A "Chromium (CDP) app" is any Chromium runtime exposing a Chrome DevTools Protocol endpoint — an Electron app, or any Chromium-family browser (Chrome/Brave/Edge) launched with `--remote-debugging-port`; all are driven through the same tool surface and tagged `platform: "chromium"`. A "Vega device" is a virtual device (VVD) or physical unit — driven by tv-remote (D-pad) and tagged `platform: "vega"`. A "HarmonyOS device" is a phone connected over `hdc` (id `harmony-`, `kind: "device"`, tagged `platform: "harmony"`) — driven by the normal touch tools, with multi-touch and free-form touch (`gesture-pinch`, `gesture-rotate`, `gesture-custom`), `rotate`, the profilers and the debugger returning 501. HarmonyOS entries with `kind: "emulator"` (id `harmony-emulator-`) are DevEco Studio instances that only `boot-device` accepts, so never pick one of those as an interaction target. Running MCP server and managing the Argent toolkit utilises `argent` command - if asked use `argent --help` for reference. To check current version of MCP server run `argent --version` command. From 78f260c7264d2eb4ca4ae9fc7a8ef673439e062b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Tue, 11 Aug 2026 14:32:30 +0200 Subject: [PATCH 012/110] feat(harmony): hand back the connect key a booted emulator registers as MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `boot-device` returned the instance id, which no interaction tool accepts, so booting a HarmonyOS emulator left the caller with nothing to drive and no way to find out what it had become: the emulator manager names instances and never reports a port, `hdc` names connect keys and never mentions an instance, and the two share no field to join on. It now snapshots the connected targets, starts the instance, and waits for the one that appears — arrival being the only signal that does not depend on how an emulator's key is spelled, which is unobservable outside mainland China. `force` restarts an instance that is already running, which is also the only way to resolve the key of one argent did not start. Whenever the key cannot be had, the payload still names the instance and carries a `note` saying which of the three reasons applied. Also stops reading `harmony-` as an instance name, so a phone's id no longer silently starts an instance named after a serial. --- README.md | 2 +- packages/skills/rules/argent.md | 2 +- .../skills/argent-device-interact/SKILL.md | 2 +- .../src/tools/devices/boot-device.ts | 209 +++++++++++++-- packages/tool-server/src/utils/device-info.ts | 9 +- .../test/boot-device-harmony.test.ts | 251 ++++++++++++++++++ 6 files changed, 441 insertions(+), 34 deletions(-) create mode 100644 packages/tool-server/test/boot-device-harmony.test.ts diff --git a/README.md b/README.md index 5c4c556b72..b046a0a002 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ Argent drives a growing set of targets through a single toolkit, each with the r | **Desktop & web** | Electron and Chromium apps (incl. React Native Web / Expo web) over CDP | Mouse / keyboard | | **HarmonyOS** | Phones over `hdc`, and DevEco Studio emulator instances | Touch / gesture | -> **HarmonyOS drives physical devices; the emulator is discovery and boot only.** Connect a phone with USB debugging enabled and it appears as `harmony-` (`kind: "device"`), driven by `describe`, `screenshot`, `screenshot-diff`, `gesture-tap`, `gesture-swipe`, `keyboard`, `button`, `launch-app`, `restart-app`, `open-url`, `run-sequence` and the two `await-*` tools - all through the device's own `uitest`. DevEco Studio's emulator instances appear separately as `harmony-emulator-`, which `boot-device` starts; Huawei restricts the emulator **image** download to mainland China, so outside it no instance can be created in the first place. Multi-touch and free-form touch (`gesture-pinch`, `gesture-rotate`, `gesture-custom`) return `501 NotImplementedOnPlatform`: `uitest uiInput` injects only whole gestures (click / longClick / swipe / drag / fling), one contact at a time, and `hdc shell` runs as uid 2000 with no access to `/dev/input`. `rotate`, the profilers and the debugger also return 501: the device exposes no orientation CLI, and argent's profiler and debugger surfaces have no HarmonyOS ingest path. +> **HarmonyOS drives whatever `hdc` is connected to.** Connect a phone with USB debugging enabled and it appears as `harmony-` (`kind: "device"`), driven by `describe`, `screenshot`, `screenshot-diff`, `gesture-tap`, `gesture-swipe`, `keyboard`, `button`, `launch-app`, `restart-app`, `open-url`, `run-sequence` and the two `await-*` tools - all through the device's own `uitest`. DevEco Studio's emulator instances appear separately as `harmony-emulator-`, which `boot-device` starts; it then waits for the instance to register with `hdc` and returns that connect key, so a booted emulator is driven by the same tools through the same id shape as a phone. Huawei restricts the emulator **image** download to mainland China, so outside it no instance can be created in the first place - the emulator half is therefore built against `hdc` and the manager CLI but has never been exercised against a running instance here. Multi-touch and free-form touch (`gesture-pinch`, `gesture-rotate`, `gesture-custom`) return `501 NotImplementedOnPlatform`: `uitest uiInput` injects only whole gestures (click / longClick / swipe / drag / fling), one contact at a time, and `hdc shell` runs as uid 2000 with no access to `/dev/input`. `rotate`, the profilers and the debugger also return 501: the device exposes no orientation CLI, and argent's profiler and debugger surfaces have no HarmonyOS ingest path. ## Capabilities diff --git a/packages/skills/rules/argent.md b/packages/skills/rules/argent.md index f92ec795ae..014dfc6280 100644 --- a/packages/skills/rules/argent.md +++ b/packages/skills/rules/argent.md @@ -4,7 +4,7 @@ alwaysApply: true --- -If argent is installed and configured in this environment, its MCP tools are the preferred form of interaction with the application for iOS simulator, Android emulator, Chromium (CDP) app, Vega (Amazon Fire TV), and HarmonyOS device control; otherwise see `` below before attempting any argent workflow. A "Chromium (CDP) app" is any Chromium runtime exposing a Chrome DevTools Protocol endpoint — an Electron app, or any Chromium-family browser (Chrome/Brave/Edge) launched with `--remote-debugging-port`; all are driven through the same tool surface and tagged `platform: "chromium"`. A "Vega device" is a virtual device (VVD) or physical unit — driven by tv-remote (D-pad) and tagged `platform: "vega"`. A "HarmonyOS device" is a phone connected over `hdc` (id `harmony-`, `kind: "device"`, tagged `platform: "harmony"`) — driven by the normal touch tools, with multi-touch and free-form touch (`gesture-pinch`, `gesture-rotate`, `gesture-custom`), `rotate`, the profilers and the debugger returning 501. HarmonyOS entries with `kind: "emulator"` (id `harmony-emulator-`) are DevEco Studio instances that only `boot-device` accepts, so never pick one of those as an interaction target. +If argent is installed and configured in this environment, its MCP tools are the preferred form of interaction with the application for iOS simulator, Android emulator, Chromium (CDP) app, Vega (Amazon Fire TV), and HarmonyOS device control; otherwise see `` below before attempting any argent workflow. A "Chromium (CDP) app" is any Chromium runtime exposing a Chrome DevTools Protocol endpoint — an Electron app, or any Chromium-family browser (Chrome/Brave/Edge) launched with `--remote-debugging-port`; all are driven through the same tool surface and tagged `platform: "chromium"`. A "Vega device" is a virtual device (VVD) or physical unit — driven by tv-remote (D-pad) and tagged `platform: "vega"`. A "HarmonyOS device" is a phone connected over `hdc` (id `harmony-`, `kind: "device"`, tagged `platform: "harmony"`) — driven by the normal touch tools, with multi-touch and free-form touch (`gesture-pinch`, `gesture-rotate`, `gesture-custom`), `rotate`, the profilers and the debugger returning 501. HarmonyOS entries with `kind: "emulator"` (id `harmony-emulator-`) are DevEco Studio instances that only `boot-device` accepts, so never pick one of those as an interaction target - boot it and drive the `harmony-` id that `boot-device` returns. Running MCP server and managing the Argent toolkit utilises `argent` command - if asked use `argent --help` for reference. To check current version of MCP server run `argent --version` command. diff --git a/packages/skills/skills/argent-device-interact/SKILL.md b/packages/skills/skills/argent-device-interact/SKILL.md index bc18ab7ebf..280183e34b 100644 --- a/packages/skills/skills/argent-device-interact/SKILL.md +++ b/packages/skills/skills/argent-device-interact/SKILL.md @@ -21,7 +21,7 @@ For platform-specific caveats (Metro `adb reverse`, locked-screen describe error If you delegate simulator tasks to sub-agents, make sure they have MCP permissions. -Use `list-devices` to get a target id. Results are tagged with `platform` (`ios`, `ios-remote`, `android`, `chromium`, `vega`, or `harmony`); booted/ready devices come first. A `vega` target is a TV; drive it with `argent-tv-interact` (see the note above). A `harmony` target with `kind: "device"` is a HarmonyOS phone on `hdc`: the tools in this skill drive it, except `gesture-pinch`, `gesture-rotate`, `gesture-custom` and `rotate`, which return 501 — the device injects whole gestures one contact at a time, so there is no long-press or multi-finger path. Use `button` for home/back/power only. A `harmony` target with `kind: "emulator"` is a DevEco Studio instance that only `boot-device` accepts. Pick the first entry that matches the platform you need — if none are ready, call `boot-device` with `udid` (iOS), `avdName` (Android), or `electronAppPath` (boots an Electron app as a `chromium` device). A Chromium browser already running with a CDP port shows up directly — no `boot-device` needed. See `argent-ios-simulator-setup` / `argent-android-emulator-setup` for full setup flow. +Use `list-devices` to get a target id. Results are tagged with `platform` (`ios`, `ios-remote`, `android`, `chromium`, `vega`, or `harmony`); booted/ready devices come first. A `vega` target is a TV; drive it with `argent-tv-interact` (see the note above). A `harmony` target with `kind: "device"` is a HarmonyOS phone on `hdc`: the tools in this skill drive it, except `gesture-pinch`, `gesture-rotate`, `gesture-custom` and `rotate`, which return 501 — the device injects whole gestures one contact at a time, so there is no long-press or multi-finger path. Use `button` for home/back/power only. A `harmony` target with `kind: "emulator"` is a DevEco Studio instance that only `boot-device` accepts; boot it and drive the `harmony-` id it returns. Pick the first entry that matches the platform you need — if none are ready, call `boot-device` with `udid` (iOS), `avdName` (Android), or `electronAppPath` (boots an Electron app as a `chromium` device). A Chromium browser already running with a CDP port shows up directly — no `boot-device` needed. See `argent-ios-simulator-setup` / `argent-android-emulator-setup` for full setup flow. **Load tool schemas before first use.** Gesture tools (`gesture-tap`, `gesture-swipe`, `gesture-pinch`, `gesture-rotate`, `gesture-custom`) may be deferred — their parameter schemas are not loaded until fetched. Always use ToolSearch to load the schemas of all gesture tools you plan to use **before** calling any of them. If you skip this step, parameters may be coerced to strings instead of numbers, causing validation errors. diff --git a/packages/tool-server/src/tools/devices/boot-device.ts b/packages/tool-server/src/tools/devices/boot-device.ts index b58015d085..45af9ef6fe 100644 --- a/packages/tool-server/src/tools/devices/boot-device.ts +++ b/packages/tool-server/src/tools/devices/boot-device.ts @@ -39,6 +39,7 @@ import { classifyDevice, harmonyInstanceName, stripRemotePrefix, + harmonyDeviceId, harmonyEmulatorId, } from "../../utils/device-info"; import { @@ -55,7 +56,8 @@ import { isChinaOnlyRestriction, runHarmonyEmulator, } from "../../utils/harmony-cli"; -import { listHarmonyInstances } from "../../utils/harmony-devices"; +import { listHarmonyHdcTargets, listHarmonyInstances } from "../../utils/harmony-devices"; +import { resolveHdc } from "../../utils/harmony-hdc"; import { bootElectronApp, type ElectronBootResult } from "./boot-electron"; const execFileAsync = promisify(execFile); @@ -91,7 +93,7 @@ const zodSchema = z.object({ .string() .optional() .describe( - "HarmonyOS: emulator instance to start — the `name` of a harmony device from `list-devices` (e.g. `Phone_1`); the `harmony-` id that entry reports as its `udid` is accepted too. Provide exactly one of `udid`, `avdName`, `vvdImage`, `harmonyInstance`, or `electronAppPath`." + 'HarmonyOS: emulator instance to start — the `name` of a `kind: "emulator"` device from `list-devices` (e.g. `Phone_1`); the `harmony-emulator-` id that entry reports as its `udid` is accepted too. Provide exactly one of `udid`, `avdName`, `vvdImage`, `harmonyInstance`, or `electronAppPath`.' ), bootTimeoutMs: z .number() @@ -106,7 +108,7 @@ const zodSchema = z.object({ .boolean() .optional() .describe( - "Shut down and re-boot the device even if already running. iOS/Android/Vega only: HarmonyOS exposes no running state to act on, and an Electron launch always spawns a fresh process." + "Shut down and re-boot the device even if already running. On HarmonyOS this is also the only way to learn the `hdc` connect key of an instance that is already up, since the emulator manager never reports one: without it an already-running instance is left alone and the payload names the instance. Ignored for Electron, which always spawns a fresh process." ), headless: z .boolean() @@ -1445,25 +1447,46 @@ function bootVega(params: { return promise; } -// The returned id names the *instance*, not a connect key: `-start` reports -// only that it handed the instance off, and the key a booted emulator registers -// with `hdc` could not be observed here (no image is obtainable — see -// HARMONY_IMAGE_RESTRICTION). Once it has registered, `list-devices` reports it -// as a `kind: "device"` entry, which is the id the interaction tools take. +/** + * `udid` is the id to drive next, and which of the platform's two id shapes it + * carries depends on whether the instance reached `hdc`: the connect-key id + * once it registered, the instance id (plus a `note`) when it did not. The + * distinction matters because only the first is an id the interaction tools + * accept — `-start` reports no port, and an instance name is nothing `hdc` + * knows, so the key is learned by watching for the target that appears. + */ type HarmonyBootResult = { platform: "harmony"; udid: string; instanceName: string; booted: true; + note?: string; }; // Coalesce concurrent HarmonyOS boots (mirrors `inFlightVegaBoots`): two callers -// must not both shell out `Emulator -start` for the same instance. Keyed on the -// name alone — unlike Vega there is no `force` mode to keep apart. +// must not both shell out `Emulator -start` for the same instance. const inFlightHarmonyBoots = new Map>(); const HARMONY_BOOT_TIMEOUT_MS = 120_000; +/** + * Interval between `hdc list targets` polls while waiting for a started + * instance to register. Each poll spawns `hdc` and round-trips its daemon, so + * it sits an order of magnitude above the adb intervals in + * {@link BOOT_POLL_INTERVALS_MS} — a HarmonyOS boot is minutes, and nothing is + * gained by asking twice a second. + */ +const HARMONY_TARGET_POLL_MS = 2_000; + +/** + * How long a `-stop` gets to take the instance's target off `hdc` before the + * restart proceeds anyway. It bounds only the pessimistic case: a restart that + * lands on the port it had before is indistinguishable from the target that + * never left, so waiting for the old one to drop is what keeps the arrival + * detectable. Proceeding after the grace costs the connect key, not the boot. + */ +const HARMONY_STOP_GRACE_MS = 15_000; + /** * `Emulator -install` outside mainland China prints "…available only in the * Chinese mainland" and downloads nothing, and no instance can be created @@ -1487,18 +1510,142 @@ const HARMONY_NO_INSTANCES = "starting it. If creating one fails for want of an image, note that Huawei serves HarmonyOS " + "emulator images only within mainland China."; +/** + * Said when the instance was already running, since argent then has no way to + * name the target it registered as: the manager reports no port, and the two + * discovery sources share no key to join on. + */ +const HARMONY_ALREADY_RUNNING = + "The instance was already running, so argent did not restart it — and an instance argent did " + + "not start cannot be matched to the `hdc` connect key it registered under. Take the connect " + + 'key from `list-devices` (the `kind: "device"` entry), or re-run with `force: true` to ' + + "restart the instance and have its key resolved here."; + +/** + * Said when the instance started but never showed up on `hdc` within the boot + * budget. Not a failure: the manager did start it, and a HarmonyOS cold boot + * can outlast a budget the caller chose — but the returned id is the instance, + * which no interaction tool accepts, so say what to do instead of leaving the + * caller to notice the id was the wrong shape. + */ +const HARMONY_NO_TARGET = + "The instance started but had not registered with `hdc` before the boot budget ran out, so " + + "this payload names the instance rather than a connect key. Give it longer with " + + "`bootTimeoutMs`, or call `list-devices` once it finishes booting and drive the `kind: " + + '"device"` entry that appears.'; + +/** Said when the instance started but the connector that would reach it is missing. */ +const HARMONY_NO_HDC = + "The instance started, but `hdc` was not found, so argent cannot tell which target it " + + "registered as and no interaction tool can reach it. Install DevEco Studio's device " + + "connector (or put `hdc` on PATH), then call `list-devices`."; + +/** The connect keys `hdc` can drive right now. */ +async function connectedHarmonyKeys(): Promise> { + const targets = await listHarmonyHdcTargets().catch(() => []); + return new Set(targets.filter((t) => t.state === "Connected").map((t) => t.connectKey)); +} + +/** + * The connect key of the target that appears after a start, or null if none + * does before `deadline`. + * + * Which target belongs to the instance is decided by arrival rather than by + * anything either CLI reports: `Emulator` names instances and never mentions a + * port, `hdc` names connect keys and never mentions an instance, and the two + * have no field in common. Arrival is also the one signal that does not depend + * on the key's shape, which — no emulator image being obtainable outside + * mainland China (see {@link HARMONY_IMAGE_RESTRICTION}) — has never been + * observed here. + */ +async function waitForHarmonyTarget(before: Set, deadline: number): Promise { + for (;;) { + for (const key of await connectedHarmonyKeys()) { + if (!before.has(key)) return key; + } + const remaining = deadline - Date.now(); + if (remaining <= 0) return null; + await new Promise((r) => setTimeout(r, Math.min(HARMONY_TARGET_POLL_MS, remaining))); + } +} + +/** Wait until one of `previous` is gone, or until the grace period expires. */ +async function waitForHarmonyTargetLoss(previous: Set, deadline: number): Promise { + // An instance that never registered has no target to lose, and waiting out + // the whole grace for a departure that cannot happen would just delay the + // restart. + if (previous.size === 0) return; + for (;;) { + const now = await connectedHarmonyKeys(); + if ([...previous].some((key) => !now.has(key))) return; + const remaining = deadline - Date.now(); + if (remaining <= 0) return; + await new Promise((r) => setTimeout(r, Math.min(HARMONY_TARGET_POLL_MS, remaining))); + } +} + async function bootHarmonyImpl(params: { instanceName: string; bootTimeoutMs: number; + force?: boolean; }): Promise { await ensureDep("harmony-emulator"); + const bootDeadline = Date.now() + params.bootTimeoutMs; + + // Read once, for two questions: whether this instance is already up, and — if + // the start then fails — whether the host has any instance at all. `null` is + // a listing that itself failed, and answers neither: an unreadable list must + // not block a start that would have worked, nor let one be blamed on a host + // with no instances when that was never established. + const instances = await listHarmonyInstances().catch(() => null); + const alreadyRunning = + instances?.some((i) => i.name === params.instanceName && i.running) ?? false; + + if (alreadyRunning && !params.force) { + return { + platform: "harmony", + udid: harmonyEmulatorId(params.instanceName), + instanceName: params.instanceName, + booted: true, + note: HARMONY_ALREADY_RUNNING, + }; + } + if (alreadyRunning) { + // Snapshot before the stop, not after: what the restart has to be told + // apart from is the target the *running* instance holds. + const beforeStop = await connectedHarmonyKeys(); + const stopped = await runHarmonyEmulator(["-stop", params.instanceName]); + const stopDiagnostic = emulatorFailure(stopped); + if (stopDiagnostic) { + throw new Error( + `Failed to stop HarmonyOS emulator "${params.instanceName}" before restarting it: ${stopDiagnostic}` + ); + } + await waitForHarmonyTargetLoss( + beforeStop, + Math.min(Date.now() + HARMONY_STOP_GRACE_MS, bootDeadline) + ); + } + + // Snapshot immediately before the start, so every target that was already + // being driven — another emulator, a phone on USB — is excluded by identity + // rather than by any assumption about how an emulator's key is spelled. + const before = await connectedHarmonyKeys(); // `-start` is the manager's launcher, not the emulator process itself (the // manager has a separate `-stop ` verb). Whether it returns as soon as // the instance is handed off or blocks until HarmonyOS is up could not be - // observed, so it gets the caller's whole boot budget instead of the 30 s - // default every other `runHarmonyEmulator` call takes. - const result = await runHarmonyEmulator(["-start", params.instanceName], params.bootTimeoutMs); + // observed, so it gets what is left of the caller's boot budget instead of + // the 30 s default every other `runHarmonyEmulator` call takes. One budget + // spans both stages, as on Vega: were the start and the wait each given the + // full value, the worst case before a boot gave up would be twice what the + // caller asked for. + const result = await runHarmonyEmulator( + ["-start", params.instanceName], + // Floored at 1ms, not 0: `execFile`'s `timeout: 0` means *no* timeout, so an + // exhausted budget would turn into an unbounded call. + Math.max(1, bootDeadline - Date.now()) + ); // The exit code is no verdict: `-start` on a missing instance exits 1 while // the manager's other failures exit 0, so the diagnostic it printed is the // only signal (see `harmony-cli.ts`). @@ -1506,12 +1653,6 @@ async function bootHarmonyImpl(params: { if (diagnostic) { const imageMissing = isChinaOnlyRestriction(diagnostic) || diagnostic.includes("Cannot find image"); - // A start that failed because no instance exists is the same wall one step - // earlier, so ask whether any exists at all. Read only here, on the failure - // path — a listing that misfires must never block a start that would - // otherwise have worked. `null` means the listing itself failed — unknown, - // so claim nothing. - const instances = imageMissing ? null : await listHarmonyInstances().catch(() => null); // Only the manager's own words justify blaming the region; an empty instance // list is merely consistent with it. const cause = imageMissing @@ -1526,24 +1667,36 @@ async function bootHarmonyImpl(params: { ); } + // Asked rather than waited out: with no connector there is no target list to + // watch, so the wait would spend the whole budget concluding that nothing + // registered — blaming the emulator for the connector's absence. + const hdcAvailable = Boolean(await resolveHdc()); + const connectKey = hdcAvailable ? await waitForHarmonyTarget(before, bootDeadline) : null; + return { platform: "harmony", - udid: harmonyEmulatorId(params.instanceName), + udid: connectKey ? harmonyDeviceId(connectKey) : harmonyEmulatorId(params.instanceName), instanceName: params.instanceName, booted: true, + ...(connectKey ? {} : { note: hdcAvailable ? HARMONY_NO_TARGET : HARMONY_NO_HDC }), }; } function bootHarmony(params: { instanceName: string; bootTimeoutMs: number; + force?: boolean; }): Promise { - const existing = inFlightHarmonyBoots.get(params.instanceName); + // Key the coalescing on `force` too, for the reason Vega does: a forced boot + // restarts the instance, so it must not join a plain one that would skip the + // restart and hand back the instance still running. + const key = `${params.instanceName}${params.force ? "force" : "normal"}`; + const existing = inFlightHarmonyBoots.get(key); if (existing) return existing; const promise = bootHarmonyImpl(params).finally(() => { - inFlightHarmonyBoots.delete(params.instanceName); + inFlightHarmonyBoots.delete(key); }); - inFlightHarmonyBoots.set(params.instanceName, promise); + inFlightHarmonyBoots.set(key, promise); return promise; } @@ -1570,9 +1723,9 @@ export function createBootDeviceTool( description: `Start an iOS simulator, launch an Android emulator, start a Vega (Fire TV) Virtual Device, start a HarmonyOS emulator, or spawn an Electron app and wait until it is ready to accept interactions. Pick the platform by which argument you pass: 'udid' for an iOS simulator from list-devices, 'avdName' for an Android AVD (a serial is assigned automatically), 'vvdImage' for a Vega VVD (the 'vvdImage' of a vega device from list-devices, e.g. 'tv'), 'harmonyInstance' for a HarmonyOS emulator instance (the 'name' of a harmony device from list-devices), or 'electronAppPath' for an Electron app (a CDP remote-debugging port is picked automatically, or pass 'electronPort' to fix one). Use at the start of a session once you have picked a target. -Returns a tagged payload: { platform: 'ios', udid, booted } or { platform: 'android', serial, avdName, booted } or { platform: 'vega', serial, vvdImage, booted } or { platform: 'harmony', udid, instanceName, booted } or { platform: 'chromium', id, port, pid, booted } (an Electron app boots as a Chromium/CDP device). +Returns a tagged payload: { platform: 'ios', udid, booted } or { platform: 'android', serial, avdName, booted } or { platform: 'vega', serial, vvdImage, booted } or { platform: 'harmony', udid, instanceName, booted, note? } or { platform: 'chromium', id, port, pid, booted } (an Electron app boots as a Chromium/CDP device). Android boots take 2–10 minutes depending on machine and cold/warm state; the tool transparently hot-boots from the AVD's default_boot snapshot when usable and falls back to cold boot otherwise. Vega starts the single SDK-managed VVD via the vega CLI (~10s) and returns once it reports running. If an Android/Electron boot stage fails, the tool terminates the device it spawned so the next retry starts clean. -HarmonyOS hands the instance to DevEco Studio's Emulator manager, which reports no readiness signal — the call returns once the manager accepts the start, not once the OS is up, and no interaction tool drives a HarmonyOS device yet. Huawei serves the emulator images only within mainland China, so on a host outside it no instance can exist to start.`, +HarmonyOS hands the instance to DevEco Studio's Emulator manager, then waits for it to register with 'hdc' and returns its connect key as 'udid' — the id the interaction tools drive. The manager itself reports no readiness signal and never names a port, so an instance that is already running is left alone unless you pass 'force: true'; whenever the key could not be resolved the payload names the instance instead and carries a 'note' saying why. Huawei serves the emulator images only within mainland China, so on a host outside it no instance can exist to start.`, alwaysLoad: true, searchHint: "boot start launch simulator emulator avd device session ios android vega vvd firetv harmony harmonyos deveco cold hot", @@ -1602,14 +1755,15 @@ HarmonyOS hands the instance to DevEco Studio's Emulator manager, which reports if (platform === "ios-remote") { return bootIosRemote(params.udid!, registry, params.force); } - // `list-devices` reports a HarmonyOS instance's id as `harmony-` - // and the capability gate lets that id through, so route it to the + // A HarmonyOS id reaching here is an emulator instance — the + // capability gate admits only `kind: "emulator"` — so route it to the // emulator manager rather than to simctl, which would reject it as an // unknown iOS udid. if (platform === "harmony") { return bootHarmony({ instanceName: harmonyInstanceName(params.udid!), bootTimeoutMs: params.bootTimeoutMs ?? HARMONY_BOOT_TIMEOUT_MS, + force: params.force, }); } return bootIos(params.udid!, registry, params.force, params.headless); @@ -1632,6 +1786,7 @@ HarmonyOS hands the instance to DevEco Studio's Emulator manager, which reports return bootHarmony({ instanceName: harmonyInstanceName(params.harmonyInstance!), bootTimeoutMs: params.bootTimeoutMs ?? HARMONY_BOOT_TIMEOUT_MS, + force: params.force, }); } return bootElectronApp({ diff --git a/packages/tool-server/src/utils/device-info.ts b/packages/tool-server/src/utils/device-info.ts index 6eaf607230..14f06a973e 100644 --- a/packages/tool-server/src/utils/device-info.ts +++ b/packages/tool-server/src/utils/device-info.ts @@ -82,14 +82,15 @@ export function harmonyDeviceId(connectKey: string): string { * The emulator instance name behind a `harmony-emulator-` id. * * Exactly one prefix is stripped, so an instance genuinely named `emulator-x` - * round-trips through `harmonyEmulatorId` unharmed. + * round-trips through `harmonyEmulatorId` unharmed. A connect-key id keeps its + * `harmony-` prefix rather than being read as an instance name, so handing a + * phone's id to `boot-device` fails on an instance that does not exist instead + * of silently starting whichever one happens to be named after a serial. */ export function harmonyInstanceName(udid: string): string { return udid.startsWith(HARMONY_EMULATOR_ID_PREFIX) ? udid.slice(HARMONY_EMULATOR_ID_PREFIX.length) - : udid.startsWith(HARMONY_ID_PREFIX) - ? udid.slice(HARMONY_ID_PREFIX.length) - : udid; + : udid; } /** The `hdc` connect key behind a `harmony-` id. */ diff --git a/packages/tool-server/test/boot-device-harmony.test.ts b/packages/tool-server/test/boot-device-harmony.test.ts new file mode 100644 index 0000000000..7f58dc2502 --- /dev/null +++ b/packages/tool-server/test/boot-device-harmony.test.ts @@ -0,0 +1,251 @@ +import { describe, it, expect, vi, beforeEach, afterEach } from "vitest"; +import type { Registry } from "@argent/registry"; + +const runHarmonyEmulator = vi.fn(); +const listHarmonyInstances = vi.fn(); +const listHarmonyHdcTargets = vi.fn(); +const resolveHdc = vi.fn(); +const ensureDep = vi.fn(); + +vi.mock("../src/utils/harmony-cli", async () => { + const actual = await vi.importActual( + "../src/utils/harmony-cli" + ); + return { ...actual, runHarmonyEmulator: (...a: unknown[]) => runHarmonyEmulator(...a) }; +}); +vi.mock("../src/utils/harmony-devices", () => ({ + listHarmonyInstances: (...a: unknown[]) => listHarmonyInstances(...a), + listHarmonyHdcTargets: (...a: unknown[]) => listHarmonyHdcTargets(...a), +})); +vi.mock("../src/utils/harmony-hdc", async () => { + const actual = await vi.importActual( + "../src/utils/harmony-hdc" + ); + return { ...actual, resolveHdc: (...a: unknown[]) => resolveHdc(...a) }; +}); +vi.mock("../src/utils/check-deps", async () => { + const actual = + await vi.importActual("../src/utils/check-deps"); + return { ...actual, ensureDep: (...a: unknown[]) => ensureDep(...a) }; +}); + +import { createBootDeviceTool } from "../src/tools/devices/boot-device"; + +const registry = {} as Registry; +const INSTANCE = "Phone_1"; +/** The USB phone from `hdc list targets -v`, present throughout. */ +const PHONE = { connectKey: "025DEK236V035771", connection: "USB", state: "Connected" }; +const EMULATOR_KEY = "127.0.0.1:10001"; +const emulatorTarget = { connectKey: EMULATOR_KEY, connection: "TCP", state: "Connected" }; + +function boot(params: Record) { + return createBootDeviceTool(registry).execute!({}, { harmonyInstance: INSTANCE, ...params }); +} + +/** Successive `hdc list targets` results, the last one repeating forever. */ +function targets(...rounds: (typeof PHONE)[][]) { + let call = 0; + listHarmonyHdcTargets.mockImplementation(() => + Promise.resolve(rounds[Math.min(call++, rounds.length - 1)]) + ); +} + +beforeEach(() => { + vi.clearAllMocks(); + ensureDep.mockResolvedValue(undefined); + resolveHdc.mockResolvedValue("/Applications/DevEco-Studio.app/Contents/sdk/.../hdc"); + runHarmonyEmulator.mockResolvedValue({ stdout: "", stderr: "" }); + listHarmonyInstances.mockResolvedValue([ + { name: INSTANCE, deviceType: "Phone", osVersion: "HarmonyOS 6.1.1(24)", running: false }, + ]); + targets([PHONE]); +}); + +afterEach(() => { + vi.useRealTimers(); +}); + +describe("boot-device — HarmonyOS emulator path", () => { + it("returns the connect key of the target that appeared, not the instance id", async () => { + targets([PHONE], [PHONE, emulatorTarget]); + + const result = await boot({}); + + expect(ensureDep).toHaveBeenCalledWith("harmony-emulator"); + expect(runHarmonyEmulator).toHaveBeenCalledWith(["-start", INSTANCE], expect.any(Number)); + expect(result).toEqual({ + platform: "harmony", + udid: `harmony-${EMULATOR_KEY}`, + instanceName: INSTANCE, + booted: true, + }); + }); + + it("ignores a target that was already connected before the start", async () => { + // The phone is connected the whole time and must never be mistaken for the + // instance just started — arrival is the only thing that identifies it. + targets([PHONE], [PHONE], [PHONE, emulatorTarget]); + vi.useFakeTimers(); + + const pending = boot({}); + await vi.advanceTimersByTimeAsync(4_000); + + expect(((await pending) as { udid: string }).udid).toBe(`harmony-${EMULATOR_KEY}`); + }); + + it("only counts a target once it is Connected", async () => { + // A target that has registered but is still Offline cannot be driven, so + // handing its key back would give the caller an id every later tool fails on. + const offline = { connectKey: "127.0.0.1:10002", connection: "TCP", state: "Offline" }; + targets([PHONE], [PHONE, offline], [PHONE, offline, emulatorTarget]); + vi.useFakeTimers(); + + const pending = boot({}); + await vi.advanceTimersByTimeAsync(4_000); + + expect(((await pending) as { udid: string }).udid).toBe(`harmony-${EMULATOR_KEY}`); + }); + + it("names the instance and says why when nothing registers within the budget", async () => { + targets([PHONE]); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 30_000 }); + await vi.advanceTimersByTimeAsync(31_000); + const result = (await pending) as { udid: string; note?: string }; + + expect(runHarmonyEmulator).toHaveBeenCalledWith(["-start", INSTANCE], expect.any(Number)); + expect(result.udid).toBe(`harmony-emulator-${INSTANCE}`); + expect(result.note).toMatch(/bootTimeoutMs/); + }); + + it("blames the missing connector instead of waiting out the budget for it", async () => { + resolveHdc.mockResolvedValue(null); + + const result = (await boot({})) as { udid: string; note?: string }; + + // No fake timers and no advance: without `hdc` the wait is skipped + // outright, so a partial DevEco install fails fast and says which half. + expect(result.udid).toBe(`harmony-emulator-${INSTANCE}`); + expect(result.note).toMatch(/`hdc` was not found/); + }); + + it("leaves a running instance alone and says how to reach it", async () => { + listHarmonyInstances.mockResolvedValue([ + { name: INSTANCE, deviceType: "Phone", osVersion: null, running: true }, + ]); + + const result = (await boot({})) as { udid: string; note?: string }; + + expect(runHarmonyEmulator).not.toHaveBeenCalled(); + expect(result.udid).toBe(`harmony-emulator-${INSTANCE}`); + expect(result.note).toMatch(/already running/); + }); + + it("restarts a running instance on force and resolves the key it comes back on", async () => { + listHarmonyInstances.mockResolvedValue([ + { name: INSTANCE, deviceType: "Phone", osVersion: null, running: true }, + ]); + // The restart lands on the port it had before, and the target lingers one + // poll past the stop. That is what makes waiting for it to drop + // load-bearing: snapshot too early and the key is already known, so its + // return never reads as an arrival. + targets( + [PHONE, emulatorTarget], + [PHONE, emulatorTarget], + [PHONE], + [PHONE], + [PHONE], + [PHONE, emulatorTarget] + ); + vi.useFakeTimers(); + + const pending = boot({ force: true, bootTimeoutMs: 120_000 }); + await vi.advanceTimersByTimeAsync(6_000); + const result = (await pending) as { udid: string; note?: string }; + + expect(runHarmonyEmulator.mock.calls.map((c) => (c[0] as string[])[0])).toEqual([ + "-stop", + "-start", + ]); + expect(result.udid).toBe(`harmony-${EMULATOR_KEY}`); + expect(result.note).toBeUndefined(); + // The stop and the wait for it come out of the same budget the start then + // spends, rather than each stage getting the caller's full value. + const startCall = runHarmonyEmulator.mock.calls.find((c) => (c[0] as string[])[0] === "-start"); + expect(startCall![1]).toBeLessThan(120_000); + }); + + it("does not start the instance when stopping it for a restart failed", async () => { + listHarmonyInstances.mockResolvedValue([ + { name: INSTANCE, deviceType: "Phone", osVersion: null, running: true }, + ]); + runHarmonyEmulator.mockResolvedValue({ + stdout: `"${INSTANCE}" failed, emulator is not exists`, + stderr: "", + }); + + await expect(boot({ force: true })).rejects.toThrow(/Failed to stop HarmonyOS emulator/); + expect(runHarmonyEmulator).toHaveBeenCalledTimes(1); + }); + + it("blames the region when the manager reports the image restriction", async () => { + runHarmonyEmulator.mockResolvedValue({ + stdout: "Currently, this capability is available only in the Chinese mainland.", + stderr: "", + }); + + await expect(boot({})).rejects.toThrow(/only within mainland China/); + }); + + it("points at creating an instance when the start failed and none exist", async () => { + listHarmonyInstances.mockResolvedValue([]); + runHarmonyEmulator.mockResolvedValue({ + stdout: `"${INSTANCE}" is not found. Please create the device(folder): /x`, + stderr: "", + }); + + await expect(boot({})).rejects.toThrow(/create one in DevEco Studio/); + }); + + it("does not claim the host has no instances when the listing itself failed", async () => { + // An unreadable list answers nothing, so it must not be read as "none" — + // that would send the caller off creating an instance they already have. + listHarmonyInstances.mockRejectedValue(new Error("Emulator: spawn EACCES")); + runHarmonyEmulator.mockResolvedValue({ + stdout: `"${INSTANCE}" is not found. Please create the device(folder): /x`, + stderr: "", + }); + + await expect(boot({})).rejects.toThrow(/is not found\. Please create the device/); + await expect(boot({})).rejects.not.toThrow(/create one in DevEco Studio/); + }); + + it("starts the instance behind a harmony-emulator- udid", async () => { + targets([PHONE], [PHONE, emulatorTarget]); + + const result = await createBootDeviceTool(registry).execute!( + {}, + { udid: `harmony-emulator-${INSTANCE}` } + ); + + expect(runHarmonyEmulator).toHaveBeenCalledWith(["-start", INSTANCE], expect.any(Number)); + expect(result).toMatchObject({ instanceName: INSTANCE, udid: `harmony-${EMULATOR_KEY}` }); + }); + + it("does not read a connect-key id as an instance name", async () => { + vi.useFakeTimers(); + + const pending = boot({ + harmonyInstance: `harmony-${PHONE.connectKey}`, + bootTimeoutMs: 30_000, + }); + await vi.advanceTimersByTimeAsync(31_000); + await pending; + + expect(runHarmonyEmulator).toHaveBeenCalledWith( + ["-start", `harmony-${PHONE.connectKey}`], + expect.any(Number) + ); + }); +}); From af1e94e2aa167bb7bdb7de4845c1794483e054f6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Tue, 11 Aug 2026 14:33:39 +0200 Subject: [PATCH 013/110] test(harmony): pin that boot-device hands back a drivable id --- .../test/boot-device-harmony.test.ts | 24 +++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/packages/tool-server/test/boot-device-harmony.test.ts b/packages/tool-server/test/boot-device-harmony.test.ts index 7f58dc2502..8e52882777 100644 --- a/packages/tool-server/test/boot-device-harmony.test.ts +++ b/packages/tool-server/test/boot-device-harmony.test.ts @@ -30,6 +30,9 @@ vi.mock("../src/utils/check-deps", async () => { }); import { createBootDeviceTool } from "../src/tools/devices/boot-device"; +import { createDescribeTool } from "../src/tools/describe"; +import { assertSupported } from "../src/utils/capability"; +import { resolveDevice } from "../src/utils/device-info"; const registry = {} as Registry; const INSTANCE = "Phone_1"; @@ -221,6 +224,27 @@ describe("boot-device — HarmonyOS emulator path", () => { await expect(boot({})).rejects.not.toThrow(/create one in DevEco Studio/); }); + it("returns an id the interaction tools accept, which the instance id is not", async () => { + targets([PHONE], [PHONE, emulatorTarget]); + + const { udid } = (await boot({})) as { udid: string }; + + // The point of resolving the key at all: what boot-device hands back has to + // be drivable. Both halves matter — the instance id is rejected by the same + // gate, so a payload carrying it strands the caller. + const describe = createDescribeTool({} as Registry); + expect(() => + assertSupported("describe", describe.capability, resolveDevice(udid)) + ).not.toThrow(); + expect(() => + assertSupported( + "describe", + describe.capability, + resolveDevice(`harmony-emulator-${INSTANCE}`) + ) + ).toThrow(/not supported on harmony emulator/); + }); + it("starts the instance behind a harmony-emulator- udid", async () => { targets([PHONE], [PHONE, emulatorTarget]); From fbbc3946b71a7a371301e9e572a4e87dda197157 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Tue, 11 Aug 2026 14:36:23 +0200 Subject: [PATCH 014/110] chore(harmony): cut the emulator-boot comments to what the code cannot say --- .../src/tools/devices/boot-device.ts | 78 +++++++------------ .../tool-server/src/utils/harmony-uitest.ts | 9 +++ 2 files changed, 39 insertions(+), 48 deletions(-) diff --git a/packages/tool-server/src/tools/devices/boot-device.ts b/packages/tool-server/src/tools/devices/boot-device.ts index 45af9ef6fe..aa75447527 100644 --- a/packages/tool-server/src/tools/devices/boot-device.ts +++ b/packages/tool-server/src/tools/devices/boot-device.ts @@ -1448,12 +1448,9 @@ function bootVega(params: { } /** - * `udid` is the id to drive next, and which of the platform's two id shapes it - * carries depends on whether the instance reached `hdc`: the connect-key id - * once it registered, the instance id (plus a `note`) when it did not. The - * distinction matters because only the first is an id the interaction tools - * accept — `-start` reports no port, and an instance name is nothing `hdc` - * knows, so the key is learned by watching for the target that appears. + * `udid` is the id to drive next: the connect-key id once the instance reached + * `hdc`, the instance id plus a `note` when it did not. Only the first is an id + * the interaction tools accept. */ type HarmonyBootResult = { platform: "harmony"; @@ -1470,20 +1467,17 @@ const inFlightHarmonyBoots = new Map>(); const HARMONY_BOOT_TIMEOUT_MS = 120_000; /** - * Interval between `hdc list targets` polls while waiting for a started - * instance to register. Each poll spawns `hdc` and round-trips its daemon, so - * it sits an order of magnitude above the adb intervals in - * {@link BOOT_POLL_INTERVALS_MS} — a HarmonyOS boot is minutes, and nothing is - * gained by asking twice a second. + * Each poll spawns `hdc` and round-trips its daemon, an order of magnitude + * dearer than the adb polls in {@link BOOT_POLL_INTERVALS_MS} — and a boot this + * waits on is minutes long. */ const HARMONY_TARGET_POLL_MS = 2_000; /** - * How long a `-stop` gets to take the instance's target off `hdc` before the - * restart proceeds anyway. It bounds only the pessimistic case: a restart that - * lands on the port it had before is indistinguishable from the target that + * How long a `-stop` gets to take the instance's target off `hdc`. A restart + * that lands on the port it had before is indistinguishable from a target that * never left, so waiting for the old one to drop is what keeps the arrival - * detectable. Proceeding after the grace costs the connect key, not the boot. + * detectable; giving up early costs the connect key, not the boot. */ const HARMONY_STOP_GRACE_MS = 15_000; @@ -1510,11 +1504,7 @@ const HARMONY_NO_INSTANCES = "starting it. If creating one fails for want of an image, note that Huawei serves HarmonyOS " + "emulator images only within mainland China."; -/** - * Said when the instance was already running, since argent then has no way to - * name the target it registered as: the manager reports no port, and the two - * discovery sources share no key to join on. - */ +/** Said when the instance was already running, its target therefore unidentifiable. */ const HARMONY_ALREADY_RUNNING = "The instance was already running, so argent did not restart it — and an instance argent did " + "not start cannot be matched to the `hdc` connect key it registered under. Take the connect " + @@ -1522,11 +1512,9 @@ const HARMONY_ALREADY_RUNNING = "restart the instance and have its key resolved here."; /** - * Said when the instance started but never showed up on `hdc` within the boot - * budget. Not a failure: the manager did start it, and a HarmonyOS cold boot - * can outlast a budget the caller chose — but the returned id is the instance, - * which no interaction tool accepts, so say what to do instead of leaving the - * caller to notice the id was the wrong shape. + * Said when the instance started but never reached `hdc` in time. Not a + * failure — the manager did start it, and a cold boot can outlast a budget the + * caller chose. */ const HARMONY_NO_TARGET = "The instance started but had not registered with `hdc` before the boot budget ran out, so " + @@ -1550,13 +1538,11 @@ async function connectedHarmonyKeys(): Promise> { * The connect key of the target that appears after a start, or null if none * does before `deadline`. * - * Which target belongs to the instance is decided by arrival rather than by - * anything either CLI reports: `Emulator` names instances and never mentions a - * port, `hdc` names connect keys and never mentions an instance, and the two - * have no field in common. Arrival is also the one signal that does not depend - * on the key's shape, which — no emulator image being obtainable outside - * mainland China (see {@link HARMONY_IMAGE_RESTRICTION}) — has never been - * observed here. + * Arrival, because nothing else joins the two: `Emulator` names instances and + * never reports a port, `hdc` names connect keys and never mentions an + * instance. It is also the one signal independent of how an emulator's key is + * spelled, which no image being obtainable here (see + * {@link HARMONY_IMAGE_RESTRICTION}) has left unobserved. */ async function waitForHarmonyTarget(before: Set, deadline: number): Promise { for (;;) { @@ -1571,9 +1557,8 @@ async function waitForHarmonyTarget(before: Set, deadline: number): Prom /** Wait until one of `previous` is gone, or until the grace period expires. */ async function waitForHarmonyTargetLoss(previous: Set, deadline: number): Promise { - // An instance that never registered has no target to lose, and waiting out - // the whole grace for a departure that cannot happen would just delay the - // restart. + // Nothing to lose: an instance that never registered would just cost the + // restart a full grace period waiting for a departure that cannot happen. if (previous.size === 0) return; for (;;) { const now = await connectedHarmonyKeys(); @@ -1592,11 +1577,10 @@ async function bootHarmonyImpl(params: { await ensureDep("harmony-emulator"); const bootDeadline = Date.now() + params.bootTimeoutMs; - // Read once, for two questions: whether this instance is already up, and — if - // the start then fails — whether the host has any instance at all. `null` is - // a listing that itself failed, and answers neither: an unreadable list must - // not block a start that would have worked, nor let one be blamed on a host - // with no instances when that was never established. + // Answers two questions: whether this instance is already up, and — if the + // start fails — whether the host has any at all. `null` is a listing that + // itself failed and answers neither, so it neither blocks a start nor lets + // one be blamed on a host with no instances. const instances = await listHarmonyInstances().catch(() => null); const alreadyRunning = instances?.some((i) => i.name === params.instanceName && i.running) ?? false; @@ -1632,14 +1616,12 @@ async function bootHarmonyImpl(params: { // rather than by any assumption about how an emulator's key is spelled. const before = await connectedHarmonyKeys(); - // `-start` is the manager's launcher, not the emulator process itself (the - // manager has a separate `-stop ` verb). Whether it returns as soon as - // the instance is handed off or blocks until HarmonyOS is up could not be - // observed, so it gets what is left of the caller's boot budget instead of - // the 30 s default every other `runHarmonyEmulator` call takes. One budget - // spans both stages, as on Vega: were the start and the wait each given the - // full value, the worst case before a boot gave up would be twice what the - // caller asked for. + // `-start` is the manager's launcher, not the emulator process itself (there + // is a separate `-stop ` verb), and whether it returns at hand-off or + // blocks until HarmonyOS is up could not be observed — so it gets what is + // left of the boot budget rather than the 30 s default. One budget spans both + // stages, as on Vega: a full value each would make the worst case twice what + // the caller asked for. const result = await runHarmonyEmulator( ["-start", params.instanceName], // Floored at 1ms, not 0: `execFile`'s `timeout: 0` means *no* timeout, so an diff --git a/packages/tool-server/src/utils/harmony-uitest.ts b/packages/tool-server/src/utils/harmony-uitest.ts index 1f2e8acfcc..4beae24bbc 100644 --- a/packages/tool-server/src/utils/harmony-uitest.ts +++ b/packages/tool-server/src/utils/harmony-uitest.ts @@ -29,6 +29,15 @@ import { hdcFileRecv, runHdcShell, shellQuote } from "./harmony-hdc"; * out-of-range coordinates are argent's to catch: `toDevicePoint` clamps into * the display, which is what keeps a normalized 0-1 rounding error at the * right edge from silently becoming a no-op tap. + * + * **Why not the simulator-server**, which is how iOS and Android reach a + * device: neither of its controllers has a counterpart here. An Android + * emulator is driven over the emulator's own gRPC console, and DevEco's + * `Emulator` 6.1.1.200 is stock `ohos-qemu` behind a Qt UI — no gRPC anywhere + * in the binary, no QEMU passthrough in its CLI, `hdc` the only host-side + * channel it ships. An Android phone is driven by a screen-sharing agent pushed + * over adb, which HarmonyOS has no equivalent of. What is left either way is + * `hdc`, reaching this same `uitest` one contact at a time, from Rust instead. */ /** Where on-device artifacts are staged before being copied to the host. */ From de209d7b263ea7d6588402ee19cbc3ec72f5f853 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Tue, 11 Aug 2026 14:44:52 +0200 Subject: [PATCH 015/110] test(harmony): pin the hdc row shape against real tconn output MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `hdc tconn` registers an `ip:port` target and lists it before any handshake succeeds, so a TCP row with state `Offline` is a state a booted emulator can genuinely be found in — which is what the boot path's wait-for-Connected is there for. The fixture is now that output verbatim rather than a plausible spelling of it. Also drops two stale claims: `harmony-cli.ts` said nothing here talks to a booted device, which stopped being true when the interaction layer landed. --- packages/tool-server/src/tools/devices/boot-device.ts | 4 ++-- packages/tool-server/src/utils/harmony-cli.ts | 6 +++--- packages/tool-server/test/harmony-devices.test.ts | 8 +++++++- 3 files changed, 12 insertions(+), 6 deletions(-) diff --git a/packages/tool-server/src/tools/devices/boot-device.ts b/packages/tool-server/src/tools/devices/boot-device.ts index aa75447527..458ff38866 100644 --- a/packages/tool-server/src/tools/devices/boot-device.ts +++ b/packages/tool-server/src/tools/devices/boot-device.ts @@ -1541,8 +1541,8 @@ async function connectedHarmonyKeys(): Promise> { * Arrival, because nothing else joins the two: `Emulator` names instances and * never reports a port, `hdc` names connect keys and never mentions an * instance. It is also the one signal independent of how an emulator's key is - * spelled, which no image being obtainable here (see - * {@link HARMONY_IMAGE_RESTRICTION}) has left unobserved. + * spelled — which, no image being obtainable here (see + * {@link HARMONY_IMAGE_RESTRICTION}), has never been seen. */ async function waitForHarmonyTarget(before: Set, deadline: number): Promise { for (;;) { diff --git a/packages/tool-server/src/utils/harmony-cli.ts b/packages/tool-server/src/utils/harmony-cli.ts index f427198803..19579af430 100644 --- a/packages/tool-server/src/utils/harmony-cli.ts +++ b/packages/tool-server/src/utils/harmony-cli.ts @@ -27,9 +27,9 @@ const execFileAsync = promisify(execFile); * and leaves the verdict to `emulatorFailure`, which reads stdout. That keeps one * classification path instead of two that disagree. * - * `hdc`, the HarmonyOS device connector, is deliberately absent: nothing here - * talks to a booted device yet, so wrapping it would add an unused binary - * dependency. It belongs with the interaction layer. + * `hdc`, the HarmonyOS device connector, lives in `harmony-hdc.ts`: this + * manager knows about instances, `hdc` knows about targets, and the two are + * separate binaries with separate failure vocabularies. */ /** The manager prints this exact token for an empty list rather than no output. */ diff --git a/packages/tool-server/test/harmony-devices.test.ts b/packages/tool-server/test/harmony-devices.test.ts index 22e14836e9..5cf2ca905f 100644 --- a/packages/tool-server/test/harmony-devices.test.ts +++ b/packages/tool-server/test/harmony-devices.test.ts @@ -132,7 +132,13 @@ describe("parseHdcTargets", () => { }); it("reports a non-connected target's real state", () => { - expect(parseHdcTargets("127.0.0.1:5555\t\tTCP\tOffline\tlocalhost\n")[0].state).toBe("Offline"); + // Verbatim from `hdc tconn 127.0.0.1:12399` against nothing listening: the + // target is registered and listed before any handshake succeeds. So a TCP + // row is the shape a booted emulator takes, and `Offline` is a state it can + // genuinely be found in — which is why the boot path waits for `Connected` + // rather than for the row to exist. + const row = parseHdcTargets("127.0.0.1:12399\t\tTCP\tOffline\tunknown...\n")[0]; + expect(row).toEqual({ connectKey: "127.0.0.1:12399", connection: "TCP", state: "Offline" }); }); }); From 261a830c56ff3d7b1d64b01f3d1c63db0a3fef03 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Tue, 11 Aug 2026 17:09:30 +0200 Subject: [PATCH 016/110] fix(harmony): stop the boot budget from running out inside the start MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both of these only showed up once a real emulator existed to boot. `Emulator -start` is not a launcher that hands off — it is the emulator's supervisor, and it runs for as long as the emulator does. It was being awaited with the boot budget as its subprocess timeout, so a boot spent the whole budget inside the start, gave the arrival wait nothing, and then the expiring timeout killed the emulator it had just started: 905s to return the wrong id and leave a dead instance behind. It is now spawned detached, exactly as the Android emulator is, with the arrival wait polling alongside and an early exit surfacing the manager's own diagnostic instead of waiting out the budget. Measured after: 8s to a drivable connect key. `-stop` likewise returns once it has asked, not once the emulator is gone, and the guest's `hdc` endpoint dies well before the process does. A restart that waited on the target dropping called `-start` into "Failed to start emulator: this emulator instance is already running" — now a recognised diagnostic — so it waits on `isRunning` instead, the only signal that tracks the process itself. --- .../src/tools/devices/boot-device.ts | 178 ++++++++++----- packages/tool-server/src/utils/harmony-cli.ts | 1 + .../test/boot-device-harmony.test.ts | 207 ++++++++++++------ 3 files changed, 264 insertions(+), 122 deletions(-) diff --git a/packages/tool-server/src/tools/devices/boot-device.ts b/packages/tool-server/src/tools/devices/boot-device.ts index 458ff38866..efca7e1226 100644 --- a/packages/tool-server/src/tools/devices/boot-device.ts +++ b/packages/tool-server/src/tools/devices/boot-device.ts @@ -1,5 +1,5 @@ import { execFile, spawn, type StdioOptions } from "node:child_process"; -import { openSync, closeSync } from "node:fs"; +import { openSync, closeSync, readFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { promisify } from "node:util"; @@ -54,6 +54,7 @@ import { resolveRunningVvdSerial, listVegaDevices } from "../../utils/vega-devic import { emulatorFailure, isChinaOnlyRestriction, + resolveHarmonyEmulator, runHarmonyEmulator, } from "../../utils/harmony-cli"; import { listHarmonyHdcTargets, listHarmonyInstances } from "../../utils/harmony-devices"; @@ -1474,12 +1475,11 @@ const HARMONY_BOOT_TIMEOUT_MS = 120_000; const HARMONY_TARGET_POLL_MS = 2_000; /** - * How long a `-stop` gets to take the instance's target off `hdc`. A restart - * that lands on the port it had before is indistinguishable from a target that - * never left, so waiting for the old one to drop is what keeps the arrival - * detectable; giving up early costs the connect key, not the boot. + * How long a `-stop` gets to actually bring the instance down before the + * restart proceeds anyway. Giving up early costs a start that reports the + * instance is still running, not a corrupted one. */ -const HARMONY_STOP_GRACE_MS = 15_000; +const HARMONY_STOP_GRACE_MS = 30_000; /** * `Emulator -install` outside mainland China prints "…available only in the @@ -1534,6 +1534,64 @@ async function connectedHarmonyKeys(): Promise> { return new Set(targets.filter((t) => t.state === "Connected").map((t) => t.connectKey)); } +/** + * Start an instance, without waiting for it to finish. + * + * `Emulator -start` is not a launcher that hands off — it is the emulator's + * supervisor, and it runs for as long as the emulator does (measured: a + * `-start` awaited under a 15-minute budget returned only when that budget + * killed it, taking the running emulator with it). So it is spawned detached + * exactly as the Android emulator is, which also leaves the emulator up across + * a tool-server restart. Its output goes to a per-instance log so a start that + * dies early can still be classified by what the manager printed. + */ +async function startHarmonyEmulator( + instanceName: string +): Promise<{ exited: () => { reason: string; output: string } | null }> { + const bin = await resolveHarmonyEmulator(); + if (!bin) { + throw new Error( + "The HarmonyOS `Emulator` manager was not found. Install DevEco Studio, or set " + + "`$DEVECO_STUDIO_HOME` to its install root, then retry." + ); + } + const logPath = join(tmpdir(), `argent-harmony-${instanceName.replace(/[^\w.-]/g, "_")}.log`); + const logFd = openSync(logPath, "w"); + const child = spawn(bin, ["-start", instanceName], { + detached: true, + stdio: ["ignore", logFd, logFd], + }); + child.unref(); + // The child holds its own handle; close the parent's copy so a descriptor + // does not leak per boot. + try { + closeSync(logFd); + } catch { + // best-effort — the child keeps writing regardless + } + + let exit: { reason: string; output: string } | null = null; + const record = (reason: string) => { + if (exit) return; + let output = ""; + try { + output = readFileSync(logPath, "utf8"); + } catch { + // no log to read — the reason alone has to carry it + } + exit = { reason, output }; + }; + child.on("exit", (code, signal) => + record(signal ? `signal ${signal}` : `exit code ${code ?? "unknown"}`) + ); + // `spawn` can fail asynchronously (ENOENT, EACCES). An unhandled `error` + // event would escape as an uncaught exception and take the tool-server with + // it, so it goes through the same latch. + child.on("error", (err) => record(`could not be spawned: ${err.message}`)); + + return { exited: () => exit }; +} + /** * The connect key of the target that appears after a start, or null if none * does before `deadline`. @@ -1541,28 +1599,43 @@ async function connectedHarmonyKeys(): Promise> { * Arrival, because nothing else joins the two: `Emulator` names instances and * never reports a port, `hdc` names connect keys and never mentions an * instance. It is also the one signal independent of how an emulator's key is - * spelled — which, no image being obtainable here (see - * {@link HARMONY_IMAGE_RESTRICTION}), has never been seen. + * spelled — measured as `127.0.0.1:5555`, which is neither the hardware-serial + * shape a phone has nor inside the range `-hdcPort` accepts. + * + * `checkAlive` is polled alongside, and throws if the emulator died: without it + * a start that failed after the spawn returned would be indistinguishable from + * one still booting, and would burn the whole budget before saying so. */ -async function waitForHarmonyTarget(before: Set, deadline: number): Promise { +async function waitForHarmonyTarget( + before: Set, + deadline: number, + checkAlive: () => void +): Promise { for (;;) { for (const key of await connectedHarmonyKeys()) { if (!before.has(key)) return key; } + checkAlive(); const remaining = deadline - Date.now(); if (remaining <= 0) return null; await new Promise((r) => setTimeout(r, Math.min(HARMONY_TARGET_POLL_MS, remaining))); } } -/** Wait until one of `previous` is gone, or until the grace period expires. */ -async function waitForHarmonyTargetLoss(previous: Set, deadline: number): Promise { - // Nothing to lose: an instance that never registered would just cost the - // restart a full grace period waiting for a departure that cannot happen. - if (previous.size === 0) return; +/** + * Wait until the manager stops reporting the instance as running. + * + * `-stop` returns as soon as it has asked, not once the emulator is gone, and + * the guest's `hdc` endpoint dies well before the process does — so a restart + * that waited on the target dropping would call `-start` into "this emulator + * instance is already running" (measured). `isRunning` is the one signal that + * tracks the process itself. An unreadable listing answers nothing, so it keeps + * waiting rather than assuming the instance is gone. + */ +async function waitForHarmonyInstanceStopped(name: string, deadline: number): Promise { for (;;) { - const now = await connectedHarmonyKeys(); - if ([...previous].some((key) => !now.has(key))) return; + const instances = await listHarmonyInstances().catch(() => null); + if (instances && !instances.some((i) => i.name === name && i.running)) return; const remaining = deadline - Date.now(); if (remaining <= 0) return; await new Promise((r) => setTimeout(r, Math.min(HARMONY_TARGET_POLL_MS, remaining))); @@ -1595,9 +1668,6 @@ async function bootHarmonyImpl(params: { }; } if (alreadyRunning) { - // Snapshot before the stop, not after: what the restart has to be told - // apart from is the target the *running* instance holds. - const beforeStop = await connectedHarmonyKeys(); const stopped = await runHarmonyEmulator(["-stop", params.instanceName]); const stopDiagnostic = emulatorFailure(stopped); if (stopDiagnostic) { @@ -1605,8 +1675,8 @@ async function bootHarmonyImpl(params: { `Failed to stop HarmonyOS emulator "${params.instanceName}" before restarting it: ${stopDiagnostic}` ); } - await waitForHarmonyTargetLoss( - beforeStop, + await waitForHarmonyInstanceStopped( + params.instanceName, Math.min(Date.now() + HARMONY_STOP_GRACE_MS, bootDeadline) ); } @@ -1616,44 +1686,44 @@ async function bootHarmonyImpl(params: { // rather than by any assumption about how an emulator's key is spelled. const before = await connectedHarmonyKeys(); - // `-start` is the manager's launcher, not the emulator process itself (there - // is a separate `-stop ` verb), and whether it returns at hand-off or - // blocks until HarmonyOS is up could not be observed — so it gets what is - // left of the boot budget rather than the 30 s default. One budget spans both - // stages, as on Vega: a full value each would make the worst case twice what - // the caller asked for. - const result = await runHarmonyEmulator( - ["-start", params.instanceName], - // Floored at 1ms, not 0: `execFile`'s `timeout: 0` means *no* timeout, so an - // exhausted budget would turn into an unbounded call. - Math.max(1, bootDeadline - Date.now()) - ); - // The exit code is no verdict: `-start` on a missing instance exits 1 while - // the manager's other failures exit 0, so the diagnostic it printed is the - // only signal (see `harmony-cli.ts`). - const diagnostic = emulatorFailure(result); - if (diagnostic) { - const imageMissing = - isChinaOnlyRestriction(diagnostic) || diagnostic.includes("Cannot find image"); - // Only the manager's own words justify blaming the region; an empty instance - // list is merely consistent with it. - const cause = imageMissing - ? ` ${HARMONY_IMAGE_RESTRICTION}` - : instances?.length === 0 - ? ` ${HARMONY_NO_INSTANCES}` - : ""; - throw new Error( - cause - ? `Failed to start HarmonyOS emulator "${params.instanceName}".${cause} The manager reported: ${diagnostic}` - : `Failed to start HarmonyOS emulator "${params.instanceName}": ${diagnostic}` - ); - } + const emulator = await startHarmonyEmulator(params.instanceName); // Asked rather than waited out: with no connector there is no target list to // watch, so the wait would spend the whole budget concluding that nothing // registered — blaming the emulator for the connector's absence. const hdcAvailable = Boolean(await resolveHdc()); - const connectKey = hdcAvailable ? await waitForHarmonyTarget(before, bootDeadline) : null; + const connectKey = hdcAvailable + ? await waitForHarmonyTarget(before, bootDeadline, () => { + const exit = emulator.exited(); + if (!exit) return; + // The exit code is no verdict: `-start` on a missing instance exits 1 + // while the manager's other failures exit 0, so the diagnostic it + // printed is the only signal (see `harmony-cli.ts`). + const diagnostic = emulatorFailure({ stdout: exit.output, stderr: "" }); + if (!diagnostic) { + throw new Error( + `The HarmonyOS emulator manager exited (${exit.reason}) before "${params.instanceName}" ` + + `registered with \`hdc\`, so nothing is running to drive. It printed: ${ + exit.output.trim() || "(nothing)" + }` + ); + } + const imageMissing = + isChinaOnlyRestriction(diagnostic) || diagnostic.includes("Cannot find image"); + // Only the manager's own words justify blaming the region; an empty + // instance list is merely consistent with it. + const cause = imageMissing + ? ` ${HARMONY_IMAGE_RESTRICTION}` + : instances?.length === 0 + ? ` ${HARMONY_NO_INSTANCES}` + : ""; + throw new Error( + cause + ? `Failed to start HarmonyOS emulator "${params.instanceName}".${cause} The manager reported: ${diagnostic}` + : `Failed to start HarmonyOS emulator "${params.instanceName}": ${diagnostic}` + ); + }) + : null; return { platform: "harmony", diff --git a/packages/tool-server/src/utils/harmony-cli.ts b/packages/tool-server/src/utils/harmony-cli.ts index 19579af430..dc39d1dc09 100644 --- a/packages/tool-server/src/utils/harmony-cli.ts +++ b/packages/tool-server/src/utils/harmony-cli.ts @@ -146,6 +146,7 @@ const EMULATOR_FAILURE_MARKERS = [ "Cannot find image", "is not found. Please create the device", "failed, emulator is not exists", + "this emulator instance is already running", "Device create fail", "Unable to start the emulator", ] as const; diff --git a/packages/tool-server/test/boot-device-harmony.test.ts b/packages/tool-server/test/boot-device-harmony.test.ts index 8e52882777..86021fbc1f 100644 --- a/packages/tool-server/test/boot-device-harmony.test.ts +++ b/packages/tool-server/test/boot-device-harmony.test.ts @@ -1,17 +1,31 @@ +import { EventEmitter } from "node:events"; +import { writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; import { describe, it, expect, vi, beforeEach, afterEach } from "vitest"; import type { Registry } from "@argent/registry"; const runHarmonyEmulator = vi.fn(); +const resolveHarmonyEmulator = vi.fn(); const listHarmonyInstances = vi.fn(); const listHarmonyHdcTargets = vi.fn(); const resolveHdc = vi.fn(); const ensureDep = vi.fn(); +const spawnMock = vi.fn(); +vi.mock("node:child_process", async () => { + const actual = await vi.importActual("node:child_process"); + return { ...actual, spawn: (...a: unknown[]) => spawnMock(...a) }; +}); vi.mock("../src/utils/harmony-cli", async () => { const actual = await vi.importActual( "../src/utils/harmony-cli" ); - return { ...actual, runHarmonyEmulator: (...a: unknown[]) => runHarmonyEmulator(...a) }; + return { + ...actual, + runHarmonyEmulator: (...a: unknown[]) => runHarmonyEmulator(...a), + resolveHarmonyEmulator: (...a: unknown[]) => resolveHarmonyEmulator(...a), + }; }); vi.mock("../src/utils/harmony-devices", () => ({ listHarmonyInstances: (...a: unknown[]) => listHarmonyInstances(...a), @@ -38,8 +52,21 @@ const registry = {} as Registry; const INSTANCE = "Phone_1"; /** The USB phone from `hdc list targets -v`, present throughout. */ const PHONE = { connectKey: "025DEK236V035771", connection: "USB", state: "Connected" }; -const EMULATOR_KEY = "127.0.0.1:10001"; +/** What a booted emulator registers as, measured on a HarmonyOS 6.1.1 phone image. */ +const EMULATOR_KEY = "127.0.0.1:5555"; const emulatorTarget = { connectKey: EMULATOR_KEY, connection: "TCP", state: "Connected" }; +const logPath = join(tmpdir(), `argent-harmony-${INSTANCE}.log`); + +/** Stands in for the detached `Emulator -start`, which normally never exits. */ +class FakeEmulator extends EventEmitter { + unref = vi.fn(); + /** The manager dying early, having printed `output` to its log. */ + die(output: string, code = 0) { + writeFileSync(logPath, output); + this.emit("exit", code, null); + } +} +let child: FakeEmulator; function boot(params: Record) { return createBootDeviceTool(registry).execute!({}, { harmonyInstance: INSTANCE, ...params }); @@ -55,8 +82,12 @@ function targets(...rounds: (typeof PHONE)[][]) { beforeEach(() => { vi.clearAllMocks(); + writeFileSync(logPath, ""); + child = new FakeEmulator(); + spawnMock.mockReturnValue(child); ensureDep.mockResolvedValue(undefined); - resolveHdc.mockResolvedValue("/Applications/DevEco-Studio.app/Contents/sdk/.../hdc"); + resolveHarmonyEmulator.mockResolvedValue("/Applications/DevEco-Studio.app/.../Emulator"); + resolveHdc.mockResolvedValue("/Applications/DevEco-Studio.app/.../hdc"); runHarmonyEmulator.mockResolvedValue({ stdout: "", stderr: "" }); listHarmonyInstances.mockResolvedValue([ { name: INSTANCE, deviceType: "Phone", osVersion: "HarmonyOS 6.1.1(24)", running: false }, @@ -75,7 +106,11 @@ describe("boot-device — HarmonyOS emulator path", () => { const result = await boot({}); expect(ensureDep).toHaveBeenCalledWith("harmony-emulator"); - expect(runHarmonyEmulator).toHaveBeenCalledWith(["-start", INSTANCE], expect.any(Number)); + expect(spawnMock).toHaveBeenCalledWith( + expect.stringContaining("Emulator"), + ["-start", INSTANCE], + expect.objectContaining({ detached: true }) + ); expect(result).toEqual({ platform: "harmony", udid: `harmony-${EMULATOR_KEY}`, @@ -84,6 +119,20 @@ describe("boot-device — HarmonyOS emulator path", () => { }); }); + it("returns while the emulator is still running, rather than awaiting it", async () => { + // `Emulator -start` is the emulator's supervisor and runs as long as it + // does. Awaiting it spends the whole boot budget inside the start and then + // kills the emulator when that budget expires, so the boot has to resolve + // with the child still alive — which is what this test is, since `child` + // never emits `exit`. + targets([PHONE], [PHONE, emulatorTarget]); + + const result = (await boot({})) as { udid: string }; + + expect(result.udid).toBe(`harmony-${EMULATOR_KEY}`); + expect(child.unref).toHaveBeenCalled(); + }); + it("ignores a target that was already connected before the start", async () => { // The phone is connected the whole time and must never be mistaken for the // instance just started — arrival is the only thing that identifies it. @@ -97,10 +146,10 @@ describe("boot-device — HarmonyOS emulator path", () => { }); it("only counts a target once it is Connected", async () => { - // A target that has registered but is still Offline cannot be driven, so - // handing its key back would give the caller an id every later tool fails on. - const offline = { connectKey: "127.0.0.1:10002", connection: "TCP", state: "Offline" }; - targets([PHONE], [PHONE, offline], [PHONE, offline, emulatorTarget]); + // A killed emulator leaves its row behind as `Offline` — measured — so a + // row existing is not the same as a device there is any point driving. + const offline = { ...emulatorTarget, state: "Offline" }; + targets([PHONE], [PHONE, offline], [PHONE, emulatorTarget]); vi.useFakeTimers(); const pending = boot({}); @@ -117,7 +166,7 @@ describe("boot-device — HarmonyOS emulator path", () => { await vi.advanceTimersByTimeAsync(31_000); const result = (await pending) as { udid: string; note?: string }; - expect(runHarmonyEmulator).toHaveBeenCalledWith(["-start", INSTANCE], expect.any(Number)); + expect(spawnMock).toHaveBeenCalled(); expect(result.udid).toBe(`harmony-emulator-${INSTANCE}`); expect(result.note).toMatch(/bootTimeoutMs/); }); @@ -133,6 +182,60 @@ describe("boot-device — HarmonyOS emulator path", () => { expect(result.note).toMatch(/`hdc` was not found/); }); + it("fails fast when the manager dies before the instance registers", async () => { + targets([PHONE]); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 900_000 }); + const settled = expect(pending).rejects.toThrow(/Unable to start the emulator/); + await vi.advanceTimersByTimeAsync(1_000); + child.die("Unable to start the emulator", 1); + await vi.advanceTimersByTimeAsync(3_000); + await settled; + }); + + it("blames the region when a dead manager reports the image restriction", async () => { + targets([PHONE]); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 900_000 }); + const settled = expect(pending).rejects.toThrow(/only within mainland China/); + await vi.advanceTimersByTimeAsync(1_000); + child.die("Currently, this capability is available only in the Chinese mainland."); + await vi.advanceTimersByTimeAsync(3_000); + await settled; + }); + + it("points at creating an instance when the start failed and none exist", async () => { + listHarmonyInstances.mockResolvedValue([]); + targets([PHONE]); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 900_000 }); + const settled = expect(pending).rejects.toThrow(/create one in DevEco Studio/); + await vi.advanceTimersByTimeAsync(1_000); + child.die(`"${INSTANCE}" is not found. Please create the device(folder): /x`, 1); + await vi.advanceTimersByTimeAsync(3_000); + await settled; + }); + + it("does not claim the host has no instances when the listing itself failed", async () => { + // An unreadable list answers nothing, so it must not be read as "none" — + // that would send the caller off creating an instance they already have. + listHarmonyInstances.mockRejectedValue(new Error("Emulator: spawn EACCES")); + targets([PHONE]); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 900_000 }); + const settled = expect(pending).rejects.toThrow( + /is not found\. Please create the device(?!.*create one in DevEco Studio)/s + ); + await vi.advanceTimersByTimeAsync(1_000); + child.die(`"${INSTANCE}" is not found. Please create the device(folder): /x`, 1); + await vi.advanceTimersByTimeAsync(3_000); + await settled; + }); + it("leaves a running instance alone and says how to reach it", async () => { listHarmonyInstances.mockResolvedValue([ { name: INSTANCE, deviceType: "Phone", osVersion: null, running: true }, @@ -140,43 +243,38 @@ describe("boot-device — HarmonyOS emulator path", () => { const result = (await boot({})) as { udid: string; note?: string }; + expect(spawnMock).not.toHaveBeenCalled(); expect(runHarmonyEmulator).not.toHaveBeenCalled(); expect(result.udid).toBe(`harmony-emulator-${INSTANCE}`); expect(result.note).toMatch(/already running/); }); it("restarts a running instance on force and resolves the key it comes back on", async () => { - listHarmonyInstances.mockResolvedValue([ - { name: INSTANCE, deviceType: "Phone", osVersion: null, running: true }, - ]); - // The restart lands on the port it had before, and the target lingers one - // poll past the stop. That is what makes waiting for it to drop - // load-bearing: snapshot too early and the key is already known, so its - // return never reads as an arrival. - targets( - [PHONE, emulatorTarget], - [PHONE, emulatorTarget], - [PHONE], - [PHONE], - [PHONE], - [PHONE, emulatorTarget] + // `-stop` returns before the emulator is gone, so the restart waits on + // `isRunning`: reported running for one poll after the stop, then down. The + // snapshot is taken only once it is, which is what lets a restart that + // lands on the port it had before still read as an arrival. + let listed = 0; + listHarmonyInstances.mockImplementation(() => + Promise.resolve([ + { name: INSTANCE, deviceType: "Phone", osVersion: null, running: listed++ < 2 }, + ]) ); + targets([PHONE], [PHONE], [PHONE, emulatorTarget]); vi.useFakeTimers(); const pending = boot({ force: true, bootTimeoutMs: 120_000 }); - await vi.advanceTimersByTimeAsync(6_000); + await vi.advanceTimersByTimeAsync(10_000); const result = (await pending) as { udid: string; note?: string }; - expect(runHarmonyEmulator.mock.calls.map((c) => (c[0] as string[])[0])).toEqual([ - "-stop", - "-start", - ]); + expect(runHarmonyEmulator).toHaveBeenCalledWith(["-stop", INSTANCE]); + expect(spawnMock).toHaveBeenCalledWith( + expect.any(String), + ["-start", INSTANCE], + expect.any(Object) + ); expect(result.udid).toBe(`harmony-${EMULATOR_KEY}`); expect(result.note).toBeUndefined(); - // The stop and the wait for it come out of the same budget the start then - // spends, rather than each stage getting the caller's full value. - const startCall = runHarmonyEmulator.mock.calls.find((c) => (c[0] as string[])[0] === "-start"); - expect(startCall![1]).toBeLessThan(120_000); }); it("does not start the instance when stopping it for a restart failed", async () => { @@ -189,39 +287,7 @@ describe("boot-device — HarmonyOS emulator path", () => { }); await expect(boot({ force: true })).rejects.toThrow(/Failed to stop HarmonyOS emulator/); - expect(runHarmonyEmulator).toHaveBeenCalledTimes(1); - }); - - it("blames the region when the manager reports the image restriction", async () => { - runHarmonyEmulator.mockResolvedValue({ - stdout: "Currently, this capability is available only in the Chinese mainland.", - stderr: "", - }); - - await expect(boot({})).rejects.toThrow(/only within mainland China/); - }); - - it("points at creating an instance when the start failed and none exist", async () => { - listHarmonyInstances.mockResolvedValue([]); - runHarmonyEmulator.mockResolvedValue({ - stdout: `"${INSTANCE}" is not found. Please create the device(folder): /x`, - stderr: "", - }); - - await expect(boot({})).rejects.toThrow(/create one in DevEco Studio/); - }); - - it("does not claim the host has no instances when the listing itself failed", async () => { - // An unreadable list answers nothing, so it must not be read as "none" — - // that would send the caller off creating an instance they already have. - listHarmonyInstances.mockRejectedValue(new Error("Emulator: spawn EACCES")); - runHarmonyEmulator.mockResolvedValue({ - stdout: `"${INSTANCE}" is not found. Please create the device(folder): /x`, - stderr: "", - }); - - await expect(boot({})).rejects.toThrow(/is not found\. Please create the device/); - await expect(boot({})).rejects.not.toThrow(/create one in DevEco Studio/); + expect(spawnMock).not.toHaveBeenCalled(); }); it("returns an id the interaction tools accept, which the instance id is not", async () => { @@ -253,7 +319,11 @@ describe("boot-device — HarmonyOS emulator path", () => { { udid: `harmony-emulator-${INSTANCE}` } ); - expect(runHarmonyEmulator).toHaveBeenCalledWith(["-start", INSTANCE], expect.any(Number)); + expect(spawnMock).toHaveBeenCalledWith( + expect.any(String), + ["-start", INSTANCE], + expect.any(Object) + ); expect(result).toMatchObject({ instanceName: INSTANCE, udid: `harmony-${EMULATOR_KEY}` }); }); @@ -267,9 +337,10 @@ describe("boot-device — HarmonyOS emulator path", () => { await vi.advanceTimersByTimeAsync(31_000); await pending; - expect(runHarmonyEmulator).toHaveBeenCalledWith( + expect(spawnMock).toHaveBeenCalledWith( + expect.any(String), ["-start", `harmony-${PHONE.connectKey}`], - expect.any(Number) + expect.any(Object) ); }); }); From f41ae423bd4aa8ed9b8ba1033405f24b979657a1 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Tue, 11 Aug 2026 17:10:12 +0200 Subject: [PATCH 017/110] docs(harmony): say what the emulator was verified against --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index b046a0a002..1c33359d54 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ Argent drives a growing set of targets through a single toolkit, each with the r | **Desktop & web** | Electron and Chromium apps (incl. React Native Web / Expo web) over CDP | Mouse / keyboard | | **HarmonyOS** | Phones over `hdc`, and DevEco Studio emulator instances | Touch / gesture | -> **HarmonyOS drives whatever `hdc` is connected to.** Connect a phone with USB debugging enabled and it appears as `harmony-` (`kind: "device"`), driven by `describe`, `screenshot`, `screenshot-diff`, `gesture-tap`, `gesture-swipe`, `keyboard`, `button`, `launch-app`, `restart-app`, `open-url`, `run-sequence` and the two `await-*` tools - all through the device's own `uitest`. DevEco Studio's emulator instances appear separately as `harmony-emulator-`, which `boot-device` starts; it then waits for the instance to register with `hdc` and returns that connect key, so a booted emulator is driven by the same tools through the same id shape as a phone. Huawei restricts the emulator **image** download to mainland China, so outside it no instance can be created in the first place - the emulator half is therefore built against `hdc` and the manager CLI but has never been exercised against a running instance here. Multi-touch and free-form touch (`gesture-pinch`, `gesture-rotate`, `gesture-custom`) return `501 NotImplementedOnPlatform`: `uitest uiInput` injects only whole gestures (click / longClick / swipe / drag / fling), one contact at a time, and `hdc shell` runs as uid 2000 with no access to `/dev/input`. `rotate`, the profilers and the debugger also return 501: the device exposes no orientation CLI, and argent's profiler and debugger surfaces have no HarmonyOS ingest path. +> **HarmonyOS drives whatever `hdc` is connected to.** Connect a phone with USB debugging enabled and it appears as `harmony-` (`kind: "device"`), driven by `describe`, `screenshot`, `screenshot-diff`, `gesture-tap`, `gesture-swipe`, `keyboard`, `button`, `launch-app`, `restart-app`, `open-url`, `run-sequence` and the two `await-*` tools - all through the device's own `uitest`. DevEco Studio's emulator instances appear separately as `harmony-emulator-`, which `boot-device` starts; it then waits for the instance to register with `hdc` and returns that connect key, so a booted emulator is driven by the same tools through the same id shape as a phone. Huawei restricts the emulator **image** download to mainland China, so on a host outside it no instance can be created until an image is obtained. Multi-touch and free-form touch (`gesture-pinch`, `gesture-rotate`, `gesture-custom`) return `501 NotImplementedOnPlatform`, on both device kinds: `uitest uiInput` injects only whole gestures (click / longClick / swipe / drag / fling), one contact at a time, and the level below is shut - `hdc shell` is uid 2000 and `/dev/input` is `Permission denied` on a retail phone and on Huawei's own emulator image alike, where `hdc smode` answers `Cannot set root run mode in undebuggable version`. `rotate`, the profilers and the debugger also return 501: the device exposes no orientation CLI, and argent's profiler and debugger surfaces have no HarmonyOS ingest path. ## Capabilities From d677d3be4b7e4bab41d546e0cc2f4a817d524b3d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Tue, 11 Aug 2026 17:35:26 +0200 Subject: [PATCH 018/110] fix(harmony): reject a prototype key name instead of shelling it out `resolveHarmonyKeycode` indexed HARMONY_KEYCODES bare and tested the result against undefined, so "constructor" / "__proto__" / "toString" resolved to a prototype value, passed the guard, and reached `uiInput keyEvent ${key}` - which `harmonyKeyEvent` interpolates unquoted, against runHdcShell's documented contract. The four sibling backends all use `Object.hasOwn` plus a case-fold and each carries a comment naming this exact failure; HarmonyOS had neither, so `key: "Enter"` also 400'd here alone. Backfills the harmony cases the sibling test files were each missing: the per-backend prototype-key bucket in keyboard-error-taxonomy, the text-then-key ordering pair in keyboard-key-order, and the button loop asserting every HarmonyOS button reaches its own `uitest` key name over hdc. Also fixes two tests that could not fail. The dispatch test declared `harmony: { emulator: true }` while driving a `kind: "device"` id, so assertSupported rejected before the arm it names; deleting the whole harmony dispatch arm left it green. And nothing rendered a `harmony-uitest` tree, so dropping it from format-tree's nested condition was free. --- .../src/tools/keyboard/platforms/harmony.ts | 8 +- packages/tool-server/test/button.test.ts | 81 +++++++++++++++++++ .../test/cross-platform-tool.test.ts | 19 +++-- .../tool-server/test/harmony-describe.test.ts | 17 ++++ .../test/keyboard-error-taxonomy.test.ts | 30 +++++++ .../test/keyboard-key-order.test.ts | 64 +++++++++++++++ 6 files changed, 212 insertions(+), 7 deletions(-) diff --git a/packages/tool-server/src/tools/keyboard/platforms/harmony.ts b/packages/tool-server/src/tools/keyboard/platforms/harmony.ts index 077126f58a..9dab4fb10f 100644 --- a/packages/tool-server/src/tools/keyboard/platforms/harmony.ts +++ b/packages/tool-server/src/tools/keyboard/platforms/harmony.ts @@ -32,7 +32,13 @@ const HARMONY_KEYCODES: Record = { }; function resolveHarmonyKeycode(key: string): number { - const code = HARMONY_KEYCODES[key]; + const lower = key.toLowerCase(); + // Own-property check, and case-folded like every other backend: `key` is a + // free string, so a prototype key like "constructor" would otherwise pass the + // nullish guard with a garbage value and be interpolated into the remote shell + // line (`harmonyKeyEvent` builds `uiInput keyEvent ${key}`) instead of + // rejecting as an unknown key. + const code = Object.hasOwn(HARMONY_KEYCODES, lower) ? HARMONY_KEYCODES[lower] : undefined; if (code === undefined) { throw new InvalidToolInputError( `Key '${key}' is not available on HarmonyOS. Supported: ` + diff --git a/packages/tool-server/test/button.test.ts b/packages/tool-server/test/button.test.ts index d4ad998320..7813255ef6 100644 --- a/packages/tool-server/test/button.test.ts +++ b/packages/tool-server/test/button.test.ts @@ -14,6 +14,13 @@ vi.mock("../src/utils/android-input", async (importOriginal) => ({ injectAndroidKeycode: vi.fn(), })); +// HarmonyOS presses go over `uitest uiInput keyEvent` on the device; neutralise +// it for the same reason as the adb call above. +vi.mock("../src/utils/harmony-uitest", async (importOriginal) => ({ + ...(await importOriginal()), + harmonyKeyEvent: vi.fn(), +})); + // The Android branch preflights adb via `ensureDep("adb")` before injecting. // Stub it (default: adb present, a no-op) so the happy-path tests don't depend // on adb being installed on the test host (CI runs on a plain ubuntu image); @@ -27,12 +34,29 @@ import { buttonTool, BUTTONS_BY_PLATFORM } from "../src/tools/button"; import { UnsupportedOperationError } from "../src/utils/capability"; import { ANDROID_BUTTON_KEYCODES, injectAndroidKeycode } from "../src/utils/android-input"; import { DependencyMissingError, ensureDep } from "../src/utils/check-deps"; +import { harmonyKeyEvent } from "../src/utils/harmony-uitest"; import { sendCommand } from "../src/utils/simulator-client"; const iosUdid = "AAAAAAAA-AAAA-AAAA-AAAA-AAAAAAAAAAAA"; const androidUdid = "emulator-5554"; +const harmonyConnectKey = "025DEK236V035771"; +const harmonyUdid = `harmony-${harmonyConnectKey}`; const services = { simulatorServer: {} } as never; +/** + * The `uitest uiInput keyEvent` name each HarmonyOS button must reach the device + * as, spelled out rather than derived, so a swapped pair (home ↔ back) fails. + * The loop below indexes this by every entry of `BUTTONS_BY_PLATFORM.harmony`, + * which is what keeps the tool's `HARMONY_BUTTON_KEYS[...]!` assertion honest: + * a button added to the accepted set without a key name to press turns this red + * instead of injecting `undefined`. + */ +const HARMONY_KEY_NAMES: Record = { + home: "Home", + back: "Back", + power: "Power", +}; + describe("button tool — per-platform validation", () => { it("rejects `back` on iOS (no hardware back button) instead of a silent no-op", async () => { await expect( @@ -130,6 +154,57 @@ describe("button tool — per-platform validation", () => { expect(injectAndroidKeycode).not.toHaveBeenCalled(); } }); + + it("rejects `volumeUp` on HarmonyOS — `uitest` names only Home/Back/Power", async () => { + // `uitest uiInput keyEvent` answers `No Error` to any keyID it is handed, so + // an unnamed button would inject something unverified and still resolve + // `{ pressed }`. It has to be refused here or not at all. + await expect( + buttonTool.execute(services, { udid: harmonyUdid, button: "volumeUp" }) + ).rejects.toBeInstanceOf(UnsupportedOperationError); + expect(harmonyKeyEvent).not.toHaveBeenCalled(); + }); + + it("injects the matching key name for EVERY HarmonyOS button, over hdc only", async () => { + for (const button of BUTTONS_BY_PLATFORM.harmony) { + vi.mocked(harmonyKeyEvent).mockClear(); + vi.mocked(sendCommand).mockClear(); + vi.mocked(injectAndroidKeycode).mockClear(); + vi.mocked(ensureDep).mockClear(); + await expect(buttonTool.execute(services, { udid: harmonyUdid, button })).resolves.toEqual({ + pressed: button, + }); + // The connect key, not the `harmony-` prefixed device id — `uitest` is + // addressed by what `hdc list targets` reports. + expect(harmonyKeyEvent).toHaveBeenCalledWith(harmonyConnectKey, HARMONY_KEY_NAMES[button]); + // hdc is preflighted so a missing connector fails with a 424 install hint. + expect(ensureDep).toHaveBeenCalledWith("hdc"); + // Neither of the other backends is touched — in particular the press must + // not fall through to the sim-server HID path, which no HarmonyOS device + // is behind. + expect(sendCommand).not.toHaveBeenCalled(); + expect(injectAndroidKeycode).not.toHaveBeenCalled(); + } + }); + + it("preflights hdc before injecting so a missing connector surfaces as 424, not 500", async () => { + vi.mocked(harmonyKeyEvent).mockClear(); + vi.mocked(ensureDep).mockRejectedValueOnce( + new DependencyMissingError(["hdc"], "install the HarmonyOS command line tools") + ); + await expect( + buttonTool.execute(services, { udid: harmonyUdid, button: "home" }) + ).rejects.toBeInstanceOf(DependencyMissingError); + expect(harmonyKeyEvent).not.toHaveBeenCalled(); + }); + + it("surfaces an hdc transport failure as a throw, not a silent `{ pressed }`", async () => { + vi.mocked(ensureDep).mockResolvedValueOnce(undefined); + vi.mocked(harmonyKeyEvent).mockRejectedValueOnce(new Error("[Fail]Not match target found")); + await expect( + buttonTool.execute(services, { udid: harmonyUdid, button: "home" }) + ).rejects.toThrow(/Not match target found/); + }); }); describe("button tool — service declaration", () => { @@ -139,6 +214,12 @@ describe("button tool — service declaration", () => { expect(buttonTool.services({ udid: androidUdid, button: "back" })).toEqual({}); }); + it("does not declare the simulator-server service for a HarmonyOS target", () => { + // Same reason as Android: the press goes over hdc, and no simulator-server + // controller exists for a HarmonyOS device to spawn one against. + expect(buttonTool.services({ udid: harmonyUdid, button: "home" })).toEqual({}); + }); + it("still declares the simulator-server service eagerly for an iOS target", () => { expect(buttonTool.services({ udid: iosUdid, button: "home" })).toHaveProperty( "simulatorServer" diff --git a/packages/tool-server/test/cross-platform-tool.test.ts b/packages/tool-server/test/cross-platform-tool.test.ts index 56f3cf1dc0..cf9176acdc 100644 --- a/packages/tool-server/test/cross-platform-tool.test.ts +++ b/packages/tool-server/test/cross-platform-tool.test.ts @@ -1,7 +1,7 @@ import { describe, it, expect, beforeEach, vi } from "vitest"; import { dispatchByPlatform, type PlatformImpl } from "../src/utils/cross-platform-tool"; import { __resetDepCacheForTests, __primeDepCacheForTests } from "../src/utils/check-deps"; -import { UnsupportedOperationError } from "../src/utils/capability"; +import { NotImplementedOnPlatformError, UnsupportedOperationError } from "../src/utils/capability"; import type { ToolCapability } from "@argent/registry"; const capabilityBoth: ToolCapability = { @@ -197,15 +197,22 @@ describe("dispatchByPlatform", () => { string >({ toolId: "test", - capability: { ...capabilityBoth, harmony: { emulator: true }, chromium: { app: true } }, + // Both kinds declared: a bare `harmony-` resolves to kind 'device' + // and an `harmony-emulator-` id to kind 'emulator', so declaring + // only one lets assertSupported reject the other before dispatch runs. + capability: { + ...capabilityBoth, + harmony: { device: true, emulator: true }, + chromium: { app: true }, + }, ios: { handler: vi.fn() }, android: { handler: vi.fn() }, chromium: { handler: chromiumHandler }, }); - await expect(execute({}, { udid: "harmony-Phone_1" })).rejects.toThrow( - /not implemented .*harmony|harmony/i - ); - expect(chromiumHandler).not.toHaveBeenCalled(); + for (const udid of ["harmony-Phone_1", "harmony-emulator-argent_phone"]) { + await expect(execute({}, { udid })).rejects.toBeInstanceOf(NotImplementedOnPlatformError); + expect(chromiumHandler).not.toHaveBeenCalled(); + } }); }); diff --git a/packages/tool-server/test/harmony-describe.test.ts b/packages/tool-server/test/harmony-describe.test.ts index acb7dca3d4..9f694552a1 100644 --- a/packages/tool-server/test/harmony-describe.test.ts +++ b/packages/tool-server/test/harmony-describe.test.ts @@ -1,6 +1,7 @@ import { describe, expect, it, vi, beforeEach } from "vitest"; import { harmonyDisplay, harmonyDumpLayout } from "../src/utils/harmony-uitest"; import { describeHarmony } from "../src/tools/describe/platforms/harmony"; +import { formatDescribeTree } from "../src/tools/describe/format-tree"; vi.mock("../src/utils/harmony-uitest", () => ({ harmonyDisplay: vi.fn(), @@ -69,4 +70,20 @@ describe("describeHarmony", () => { expect((await describeHarmony(CONNECT_KEY)).hint).toMatch(/no windows/); }); + + // `uitest dumpLayout` is a real parent/child tree, so the flat renderer would + // keep only the windows and drop every element inside them. Rendering what + // describeHarmony actually returns pins the source string and the renderer + // condition together: changing either loses the clock line below. + it("renders through the nested formatter, so elements inside a window survive", async () => { + vi.mocked(harmonyDisplay).mockResolvedValue({ width: 1216, height: 2688, screenOn: true }); + vi.mocked(harmonyDumpLayout).mockResolvedValue(lockScreenDump()); + + const { tree, source } = await describeHarmony(CONNECT_KEY); + const out = formatDescribeTree(tree, { source }); + + expect(out).toContain("Source: harmony-uitest"); + expect(out).toContain("Mode: nested"); + expect(out).toContain("04:39"); + }); }); diff --git a/packages/tool-server/test/keyboard-error-taxonomy.test.ts b/packages/tool-server/test/keyboard-error-taxonomy.test.ts index 28e0c7bf52..a3cb9f17eb 100644 --- a/packages/tool-server/test/keyboard-error-taxonomy.test.ts +++ b/packages/tool-server/test/keyboard-error-taxonomy.test.ts @@ -1,8 +1,20 @@ import { describe, it, expect, vi } from "vitest"; import { Registry, FAILURE_CODES, getFailureSignal, type DeviceInfo } from "@argent/registry"; + +// The harmony backend rejects an unknown key before injecting, so these tests +// never reach the device — stub the transport anyway, so a regression that +// dropped the guard fails on the assertion rather than shelling out to `hdc`. +vi.mock("../src/utils/harmony-uitest", async (importOriginal) => ({ + ...(await importOriginal()), + harmonyTypeText: vi.fn(async () => {}), + harmonyKeyEvent: vi.fn(async () => {}), +})); + import { InvalidToolInputError } from "../src/utils/capability"; import { typeSimulatorServer } from "../src/tools/keyboard/simulator-server-keys"; import { makeChromiumImpl } from "../src/tools/keyboard/platforms/chromium"; +import { harmonyImpl } from "../src/tools/keyboard/platforms/harmony"; +import { harmonyKeyEvent, harmonyTypeText } from "../src/utils/harmony-uitest"; import { injectVegaNamedKey, injectVegaText } from "../src/utils/vega-input"; import { injectAndroidNamedKey, injectAndroidText } from "../src/utils/android-input"; @@ -147,4 +159,22 @@ describe("keyboard backends — input rejection is a 400 with a uniform telemetr FAILURE_CODES.KEYBOARD_KEY_UNSUPPORTED ); }); + + it("harmony: prototype-chain key name → 400 + KEYBOARD_KEY_UNSUPPORTED", async () => { + const harmonyDevice: DeviceInfo = { id: "harmony-KEY", platform: "harmony", kind: "device" }; + // `String(keycode)` reaches `uiInput keyEvent ${key}` unquoted, so a + // prototype value would be interpolated into the remote shell line rather + // than pressing anything — and with `text` alongside it, the reject must + // still land before a single character is typed. + for (const key of ["constructor", "__proto__", "toString"]) { + vi.mocked(harmonyKeyEvent).mockClear(); + vi.mocked(harmonyTypeText).mockClear(); + await expectInvalidInput( + harmonyImpl.handler({}, { udid: harmonyDevice.id, text: "hi", key }, harmonyDevice), + FAILURE_CODES.KEYBOARD_KEY_UNSUPPORTED + ); + expect(harmonyKeyEvent).not.toHaveBeenCalled(); + expect(harmonyTypeText).not.toHaveBeenCalled(); + } + }); }); diff --git a/packages/tool-server/test/keyboard-key-order.test.ts b/packages/tool-server/test/keyboard-key-order.test.ts index 634f2c7f44..9abea4750a 100644 --- a/packages/tool-server/test/keyboard-key-order.test.ts +++ b/packages/tool-server/test/keyboard-key-order.test.ts @@ -3,6 +3,7 @@ import type { DeviceInfo } from "@argent/registry"; import { typeSimulatorServer } from "../src/tools/keyboard/simulator-server-keys"; import { makeChromiumImpl } from "../src/tools/keyboard/platforms/chromium"; import { vegaImpl } from "../src/tools/keyboard/platforms/vega"; +import { harmonyImpl } from "../src/tools/keyboard/platforms/harmony"; vi.mock("../src/utils/vega-input", async (importOriginal) => { const actual = await importOriginal(); @@ -13,11 +14,27 @@ vi.mock("../src/utils/vega-input", async (importOriginal) => { }; }); +vi.mock("../src/utils/harmony-uitest", async (importOriginal) => ({ + ...(await importOriginal()), + harmonyTypeText: vi.fn(async () => {}), + harmonyKeyEvent: vi.fn(async () => {}), +})); + import { injectVegaNamedKey, injectVegaText } from "../src/utils/vega-input"; +import { harmonyKeyEvent, harmonyTypeText } from "../src/utils/harmony-uitest"; const IOS_SIM: DeviceInfo = { id: "TEST-UDID", platform: "ios", kind: "simulator" }; const CHROMIUM: DeviceInfo = { id: "chromium-cdp-9222", platform: "chromium", kind: "app" }; const VEGA: DeviceInfo = { id: "vega-serial", platform: "vega", kind: "vvd" }; +const HARMONY_CONNECT_KEY = "025DEK236V035771"; +const HARMONY: DeviceInfo = { + id: `harmony-${HARMONY_CONNECT_KEY}`, + platform: "harmony", + kind: "device", +}; + +/** `uitest uiInput keyEvent` keyID for `enter`, watched submitting a real field. */ +const HARMONY_ENTER_KEYID = "2054"; const ENTER_HID_KEYCODE = 40; @@ -119,4 +136,51 @@ describe("keyboard text+key ordering", () => { expect(injectVegaText).not.toHaveBeenCalled(); expect(injectVegaNamedKey).not.toHaveBeenCalled(); }); + + it("harmony: injects the named key after the text, addressed by connect key", async () => { + const order: string[] = []; + vi.mocked(harmonyTypeText) + .mockClear() + .mockImplementationOnce(async () => { + order.push("text"); + }); + vi.mocked(harmonyKeyEvent) + .mockClear() + .mockImplementationOnce(async () => { + order.push("key"); + }); + + const result = await harmonyImpl.handler( + {}, + { udid: HARMONY.id, text: "hi", key: "enter" }, + HARMONY + ); + + expect(order).toEqual(["text", "key"]); + expect(harmonyTypeText).toHaveBeenCalledWith(HARMONY_CONNECT_KEY, "hi"); + // The keyID, not the key name — `uitest` names only Home/Back/Power and + // takes a raw number for everything else. + expect(harmonyKeyEvent).toHaveBeenCalledWith(HARMONY_CONNECT_KEY, HARMONY_ENTER_KEYID); + // Two characters plus the key press. + expect(result.keys).toBe(3); + }); + + it("harmony: rejects an unknown key before typing any text", async () => { + vi.mocked(harmonyTypeText).mockClear(); + vi.mocked(harmonyKeyEvent).mockClear(); + + await expect( + harmonyImpl.handler({}, { udid: HARMONY.id, text: "hi", key: "bogus" }, HARMONY) + ).rejects.toThrow(/'bogus' is not available on HarmonyOS/); + expect(harmonyTypeText).not.toHaveBeenCalled(); + expect(harmonyKeyEvent).not.toHaveBeenCalled(); + }); + + it("harmony: case-folds the named key, like every other backend", async () => { + vi.mocked(harmonyKeyEvent).mockClear(); + + await harmonyImpl.handler({}, { udid: HARMONY.id, key: "Enter" }, HARMONY); + + expect(harmonyKeyEvent).toHaveBeenCalledWith(HARMONY_CONNECT_KEY, HARMONY_ENTER_KEYID); + }); }); From 1840a2bd9332cadc48b80ba136dac9fb68b387c0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Tue, 11 Aug 2026 17:35:40 +0200 Subject: [PATCH 019/110] docs(harmony): say what the unsupported tools actually return, and name HarmonyOS in the enumerations it was left out of The three docs promising `501 NotImplementedOnPlatform` for gesture-pinch / gesture-rotate / gesture-custom / rotate / the profilers / the debugger described a path none of them takes: none declares a `harmony` capability, so the HTTP gate refuses them with UnsupportedOperationError (400) before dispatch runs. 501 is reachable only for a tool that declares harmony and wires no branch, and every tool here that declares it wires one. The class named is also the opposite of the reason given beside it: capability.ts reserves NotImplementedOnPlatform for work not yet done, and the README's own justification - one contact, /dev/input denied - is the platform-impossible case UnsupportedOperation exists for. The rest are enumerations that were exhaustive before HarmonyOS and stopped being so. `describe` and `await-ui-element` list one tree backend per platform and omitted `uitest dumpLayout`, its `harmony-uitest` source value, and the suspended-display hint - the analogue of the Vega empty-tree caveat spelled out in the same paragraph. gesture-tap's discovery-tool advice is picked by platform; launch-app's `bundleId` and `activity`, restart-app's `activity`, open-url's scheme list and the "not reachable" sentence on five tools each enumerate every platform but this one. `keyboard`'s `delayMs` lists exactly the platforms that ignore it, and HarmonyOS - which types in one `uitest uiInput text` shot - was not among them, which is what the comment beside the harmony backend claimed it was. --- README.md | 2 +- packages/skills/rules/argent.md | 2 +- .../skills/argent-device-interact/SKILL.md | 20 +++++++++---------- .../src/tools/await-ui-element/index.ts | 3 ++- .../tool-server/src/tools/button/index.ts | 2 +- .../src/tools/describe/contract.ts | 3 ++- .../tool-server/src/tools/describe/index.ts | 8 +++++++- .../src/tools/gesture-swipe/index.ts | 2 +- .../src/tools/gesture-tap/index.ts | 4 ++-- .../tool-server/src/tools/keyboard/index.ts | 4 ++-- .../tool-server/src/tools/launch-app/index.ts | 4 ++-- .../tool-server/src/tools/open-url/index.ts | 4 ++-- .../src/tools/restart-app/index.ts | 4 ++-- .../src/tools/screenshot-diff/index.ts | 4 ++-- .../tool-server/src/tools/screenshot/index.ts | 2 +- 15 files changed, 38 insertions(+), 30 deletions(-) diff --git a/README.md b/README.md index 1c33359d54..03724d8427 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ Argent drives a growing set of targets through a single toolkit, each with the r | **Desktop & web** | Electron and Chromium apps (incl. React Native Web / Expo web) over CDP | Mouse / keyboard | | **HarmonyOS** | Phones over `hdc`, and DevEco Studio emulator instances | Touch / gesture | -> **HarmonyOS drives whatever `hdc` is connected to.** Connect a phone with USB debugging enabled and it appears as `harmony-` (`kind: "device"`), driven by `describe`, `screenshot`, `screenshot-diff`, `gesture-tap`, `gesture-swipe`, `keyboard`, `button`, `launch-app`, `restart-app`, `open-url`, `run-sequence` and the two `await-*` tools - all through the device's own `uitest`. DevEco Studio's emulator instances appear separately as `harmony-emulator-`, which `boot-device` starts; it then waits for the instance to register with `hdc` and returns that connect key, so a booted emulator is driven by the same tools through the same id shape as a phone. Huawei restricts the emulator **image** download to mainland China, so on a host outside it no instance can be created until an image is obtained. Multi-touch and free-form touch (`gesture-pinch`, `gesture-rotate`, `gesture-custom`) return `501 NotImplementedOnPlatform`, on both device kinds: `uitest uiInput` injects only whole gestures (click / longClick / swipe / drag / fling), one contact at a time, and the level below is shut - `hdc shell` is uid 2000 and `/dev/input` is `Permission denied` on a retail phone and on Huawei's own emulator image alike, where `hdc smode` answers `Cannot set root run mode in undebuggable version`. `rotate`, the profilers and the debugger also return 501: the device exposes no orientation CLI, and argent's profiler and debugger surfaces have no HarmonyOS ingest path. +> **HarmonyOS drives whatever `hdc` is connected to.** Connect a phone with USB debugging enabled and it appears as `harmony-` (`kind: "device"`), driven by `describe`, `screenshot`, `screenshot-diff`, `gesture-tap`, `gesture-swipe`, `keyboard`, `button`, `launch-app`, `restart-app`, `open-url`, `run-sequence` and the two `await-*` tools - all through the device's own `uitest`. DevEco Studio's emulator instances appear separately as `harmony-emulator-`, which `boot-device` starts; it then waits for the instance to register with `hdc` and returns that connect key, so a booted emulator is driven by the same tools through the same id shape as a phone. Huawei restricts the emulator **image** download to mainland China, so on a host outside it no instance can be created until an image is obtained. Multi-touch and free-form touch (`gesture-pinch`, `gesture-rotate`, `gesture-custom`) are refused at the capability gate with `Tool '' is not supported on harmony device`, on both device kinds: `uitest uiInput` injects only whole gestures (click / longClick / swipe / drag / fling), one contact at a time, and the level below is shut - `hdc shell` is uid 2000 and `/dev/input` is `Permission denied` on a retail phone and on Huawei's own emulator image alike, where `hdc smode` answers `Cannot set root run mode in undebuggable version`. `rotate`, the profilers and the debugger are refused the same way: the device exposes no orientation CLI, and argent's profiler and debugger surfaces have no HarmonyOS ingest path. ## Capabilities diff --git a/packages/skills/rules/argent.md b/packages/skills/rules/argent.md index 014dfc6280..143bfd8b3d 100644 --- a/packages/skills/rules/argent.md +++ b/packages/skills/rules/argent.md @@ -4,7 +4,7 @@ alwaysApply: true --- -If argent is installed and configured in this environment, its MCP tools are the preferred form of interaction with the application for iOS simulator, Android emulator, Chromium (CDP) app, Vega (Amazon Fire TV), and HarmonyOS device control; otherwise see `` below before attempting any argent workflow. A "Chromium (CDP) app" is any Chromium runtime exposing a Chrome DevTools Protocol endpoint — an Electron app, or any Chromium-family browser (Chrome/Brave/Edge) launched with `--remote-debugging-port`; all are driven through the same tool surface and tagged `platform: "chromium"`. A "Vega device" is a virtual device (VVD) or physical unit — driven by tv-remote (D-pad) and tagged `platform: "vega"`. A "HarmonyOS device" is a phone connected over `hdc` (id `harmony-`, `kind: "device"`, tagged `platform: "harmony"`) — driven by the normal touch tools, with multi-touch and free-form touch (`gesture-pinch`, `gesture-rotate`, `gesture-custom`), `rotate`, the profilers and the debugger returning 501. HarmonyOS entries with `kind: "emulator"` (id `harmony-emulator-`) are DevEco Studio instances that only `boot-device` accepts, so never pick one of those as an interaction target - boot it and drive the `harmony-` id that `boot-device` returns. +If argent is installed and configured in this environment, its MCP tools are the preferred form of interaction with the application for iOS simulator, Android emulator, Chromium (CDP) app, Vega (Amazon Fire TV), and HarmonyOS device control; otherwise see `` below before attempting any argent workflow. A "Chromium (CDP) app" is any Chromium runtime exposing a Chrome DevTools Protocol endpoint — an Electron app, or any Chromium-family browser (Chrome/Brave/Edge) launched with `--remote-debugging-port`; all are driven through the same tool surface and tagged `platform: "chromium"`. A "Vega device" is a virtual device (VVD) or physical unit — driven by tv-remote (D-pad) and tagged `platform: "vega"`. A "HarmonyOS device" is a phone connected over `hdc` (id `harmony-`, `kind: "device"`, tagged `platform: "harmony"`) — driven by the normal touch tools, with multi-touch and free-form touch (`gesture-pinch`, `gesture-rotate`, `gesture-custom`), `rotate`, the profilers and the debugger refused as unsupported on harmony. HarmonyOS entries with `kind: "emulator"` (id `harmony-emulator-`) are DevEco Studio instances that only `boot-device` accepts, so never pick one of those as an interaction target - boot it and drive the `harmony-` id that `boot-device` returns. Running MCP server and managing the Argent toolkit utilises `argent` command - if asked use `argent --help` for reference. To check current version of MCP server run `argent --version` command. diff --git a/packages/skills/skills/argent-device-interact/SKILL.md b/packages/skills/skills/argent-device-interact/SKILL.md index 280183e34b..5de7928dcc 100644 --- a/packages/skills/skills/argent-device-interact/SKILL.md +++ b/packages/skills/skills/argent-device-interact/SKILL.md @@ -21,7 +21,7 @@ For platform-specific caveats (Metro `adb reverse`, locked-screen describe error If you delegate simulator tasks to sub-agents, make sure they have MCP permissions. -Use `list-devices` to get a target id. Results are tagged with `platform` (`ios`, `ios-remote`, `android`, `chromium`, `vega`, or `harmony`); booted/ready devices come first. A `vega` target is a TV; drive it with `argent-tv-interact` (see the note above). A `harmony` target with `kind: "device"` is a HarmonyOS phone on `hdc`: the tools in this skill drive it, except `gesture-pinch`, `gesture-rotate`, `gesture-custom` and `rotate`, which return 501 — the device injects whole gestures one contact at a time, so there is no long-press or multi-finger path. Use `button` for home/back/power only. A `harmony` target with `kind: "emulator"` is a DevEco Studio instance that only `boot-device` accepts; boot it and drive the `harmony-` id it returns. Pick the first entry that matches the platform you need — if none are ready, call `boot-device` with `udid` (iOS), `avdName` (Android), or `electronAppPath` (boots an Electron app as a `chromium` device). A Chromium browser already running with a CDP port shows up directly — no `boot-device` needed. See `argent-ios-simulator-setup` / `argent-android-emulator-setup` for full setup flow. +Use `list-devices` to get a target id. Results are tagged with `platform` (`ios`, `ios-remote`, `android`, `chromium`, `vega`, or `harmony`); booted/ready devices come first. A `vega` target is a TV; drive it with `argent-tv-interact` (see the note above). A `harmony` target with `kind: "device"` is a HarmonyOS phone on `hdc`: the tools in this skill drive it, except `gesture-pinch`, `gesture-rotate`, `gesture-custom` and `rotate`, which are refused as unsupported on harmony — the device injects whole gestures one contact at a time, so there is no long-press or multi-finger path. Use `button` for home/back/power only. A `harmony` target with `kind: "emulator"` is a DevEco Studio instance that only `boot-device` accepts; boot it and drive the `harmony-` id it returns. Pick the first entry that matches the platform you need — if none are ready, call `boot-device` with `udid` (iOS), `avdName` (Android), or `electronAppPath` (boots an Electron app as a `chromium` device). A Chromium browser already running with a CDP port shows up directly — no `boot-device` needed. See `argent-ios-simulator-setup` / `argent-android-emulator-setup` for full setup flow. **Load tool schemas before first use.** Gesture tools (`gesture-tap`, `gesture-swipe`, `gesture-pinch`, `gesture-rotate`, `gesture-custom`) may be deferred — their parameter schemas are not loaded until fetched. Always use ToolSearch to load the schemas of all gesture tools you plan to use **before** calling any of them. If you skip this step, parameters may be coerced to strings instead of numbers, causing validation errors. @@ -81,13 +81,13 @@ Common schemes: `messages://`, `settings://`, `maps://?q=`, `tel://", "text": "search query", "key": "enter" } ``` -Special keys: `enter`, `escape`, `backspace`, `tab`, `space`, `arrow-up`, `arrow-down`, `arrow-left`, `arrow-right`, `f1`–`f12`. Optional: `"delayMs": 100` between keystrokes (default 50ms) — applies to the iOS simulator and Chromium; it is ignored on Android phones/tablets (typed via `adb input text`, no per-key cadence), on Vega, and on TV targets. +Special keys: `enter`, `escape`, `backspace`, `tab`, `space`, `arrow-up`, `arrow-down`, `arrow-left`, `arrow-right`, `f1`–`f12`. Optional: `"delayMs": 100` between keystrokes (default 50ms) — applies to the iOS simulator and Chromium; it is ignored on Android phones/tablets (typed via `adb input text`, no per-key cadence), on HarmonyOS (`uitest uiInput text` types in one shot), on Vega, and on TV targets. **Typing secrets.** To enter a credential without its plaintext ever entering your context, transcript, or logs, use a secret placeholder in `text` (works in `keyboard`, `paste`, `run-sequence` keyboard steps, and flow `type` steps): @@ -227,7 +227,7 @@ Use after launch/navigation and before a raw tap, when an early-painted element { "udid": "", "timeoutMs": 3000, "minStableMs": 250 } ``` -On local iOS, Android, and Chromium, the tool waits for a non-empty `describe` tree to stop changing. Continue only when `settled: true`. Pair it with a destination-specific `await-ui-element`; stillness does not identify a screen. +On local iOS, Android, Chromium, and HarmonyOS, the tool waits for a non-empty `describe` tree to stop changing. Continue only when `settled: true`. Pair it with a destination-specific `await-ui-element`; stillness does not identify a screen. Use it only for live diagnosis. Do not record it or put it in `run-sequence`. Flows use `await: { idle: true }`, which also compares pixels. This live tool can return during a presentation-layer animation. diff --git a/packages/tool-server/src/tools/await-ui-element/index.ts b/packages/tool-server/src/tools/await-ui-element/index.ts index 7689637825..1af083c577 100644 --- a/packages/tool-server/src/tools/await-ui-element/index.ts +++ b/packages/tool-server/src/tools/await-ui-element/index.ts @@ -278,7 +278,8 @@ The selector is { text?, identifier?, role? }; every provided field must match. case-insensitive substrings of the element's label/value and role; identifier matches exactly (case-insensitive), also accepting the unqualified Android resource-id name ('submit' matches 'com.example.app:id/submit'). It polls the same accessibility / DOM tree as \`describe\` -(iOS AXRuntime, Android uiautomator, Chromium CDP, Vega automation toolkit) every pollIntervalMs +(iOS AXRuntime, Android uiautomator, Chromium CDP, Vega automation toolkit, HarmonyOS \`uitest +dumpLayout\`) every pollIntervalMs (default ${DEFAULT_POLL_INTERVAL_MS}ms) until timeoutMs (default ${DEFAULT_TIMEOUT_MS}ms). Returns { success: boolean, elapsed: number } — success=false means the condition never held before the diff --git a/packages/tool-server/src/tools/button/index.ts b/packages/tool-server/src/tools/button/index.ts index dae0eb7b39..d32734c289 100644 --- a/packages/tool-server/src/tools/button/index.ts +++ b/packages/tool-server/src/tools/button/index.ts @@ -83,7 +83,7 @@ export const buttonTool: ToolDefinition = { Supported buttons depend on the platform: home, back, power, volumeUp, volumeDown, appSwitch, actionButton — buttons not present on the target platform (e.g. 'back' on iOS, 'actionButton' on Android, anything beyond home/back/power on HarmonyOS) are rejected with a clear error. Use when you need to trigger hardware button events. Returns { pressed: buttonName }. -Fails if the device backend is not reachable — the simulator-server for iOS, or \`adb\` for Android (Android presses are injected with \`adb shell input keyevent\`).`, +Fails if the device backend is not reachable — the simulator-server for iOS, \`adb\` for Android (presses are injected with \`adb shell input keyevent\`), or \`hdc\` for HarmonyOS.`, zodSchema, capability, // Android presses go over `adb shell input keyevent` (see execute), not the diff --git a/packages/tool-server/src/tools/describe/contract.ts b/packages/tool-server/src/tools/describe/contract.ts index d2de36f046..f82b38b615 100644 --- a/packages/tool-server/src/tools/describe/contract.ts +++ b/packages/tool-server/src/tools/describe/contract.ts @@ -72,7 +72,8 @@ export const describeNodeSchema: z.ZodType = z.lazy(() => // Where the tree came from. "ax-service" / "native-devtools" come from iOS; // "uiautomator" / "android-devtools" come from Android; "cdp-dom" is the // Chromium branch's DOM walk over Chrome DevTools Protocol; "vega-automation" -// is the Vega on-device automation toolkit; "tv-focus" is the focus-driven view +// is the Vega on-device automation toolkit; "harmony-uitest" is the HarmonyOS +// device's own `uitest dumpLayout`; "tv-focus" is the focus-driven view // returned for a TV target (Apple TV / Android TV), which reports focused / // focusable elements rather than a tap-oriented tree. Agents that branch on // `source` (e.g. to decide whether to also call `native-find-views` for a diff --git a/packages/tool-server/src/tools/describe/index.ts b/packages/tool-server/src/tools/describe/index.ts index ade246f022..f50395f593 100644 --- a/packages/tool-server/src/tools/describe/index.ts +++ b/packages/tool-server/src/tools/describe/index.ts @@ -160,6 +160,12 @@ On Vega (Fire TV), reads the on-device automation toolkit (\`getPageSource\`); e \`[focused]\`/\`[selected]\` so you can see where the D-pad cursor is, then move it with the \`tv-remote\` tool (Vega is remote-driven, not touch). If describe returns an empty tree on Vega, relaunch the foreground app (the toolkit attaches at launch) and try again. +On HarmonyOS, runs the device's own \`uitest dumpLayout\` (\`source: "harmony-uitest"\`); each top-level +child is a window labelled with the bundle that owns it, so an app plus the status bar reads as two +named windows. A suspended display still dumps its last-composited windows, indistinguishable in the +tree from a live screen, so the result carries a \`hint\` saying the panel is off — taps land nowhere +until you wake it with \`button\` (power). An empty dump is hinted the same way, as an app that may +still be starting. When a system dialog is visible, describe returns the dialog's interactive elements (buttons, text) with tap coordinates. When no dialog is present, it returns the foreground app's accessible elements. @@ -183,7 +189,7 @@ since a TV UI has no tap coordinates. Move the highlight with \`tv-remote\` (up/ back/menu/home), then call describe again to confirm where focus landed.`, alwaysLoad: true, searchHint: - "accessibility element tree ui hierarchy tap coordinates ios android chromium vega dom tv tvos apple tv android tv fire tv focus focusable remote dpad", + "accessibility element tree ui hierarchy tap coordinates ios android chromium vega harmony harmonyos dom tv tvos apple tv android tv fire tv focus focusable remote dpad", zodSchema, capability, services: (params): Record => { diff --git a/packages/tool-server/src/tools/gesture-swipe/index.ts b/packages/tool-server/src/tools/gesture-swipe/index.ts index 5e4800bf3a..5c450f7c6f 100644 --- a/packages/tool-server/src/tools/gesture-swipe/index.ts +++ b/packages/tool-server/src/tools/gesture-swipe/index.ts @@ -65,7 +65,7 @@ export const gestureSwipeTool: ToolDefinition = { Generates interpolated Move events for a natural feel (~60fps). Swipe up (fromY > toY) to scroll content down. Use when you need to scroll a list, dismiss a modal, drag an element, or navigate between pages. Not supported on Chromium — use gesture-scroll there instead. -Pass settle:true for a momentum-free swipe that lands exactly where the finger lifts (no fling), when you need a deterministic scroll distance. Returns { swiped: true, timestampMs }. Fails if the simulator-server / emulator backend is not reachable for the given device.`, +Pass settle:true for a momentum-free swipe that lands exactly where the finger lifts (no fling), when you need a deterministic scroll distance. Returns { swiped: true, timestampMs }. Fails if the simulator-server / emulator backend, or \`hdc\` on HarmonyOS, is not reachable for the given device.`, alwaysLoad: true, searchHint: "swipe scroll drag pan gesture device simulator emulator touch move", zodSchema, diff --git a/packages/tool-server/src/tools/gesture-tap/index.ts b/packages/tool-server/src/tools/gesture-tap/index.ts index 75b4a5dcbb..03f7ead83c 100644 --- a/packages/tool-server/src/tools/gesture-tap/index.ts +++ b/packages/tool-server/src/tools/gesture-tap/index.ts @@ -129,8 +129,8 @@ export const gestureTapTool: ToolDefinition = { Sends a Down event followed by an Up event at the same point. For Chromium, this dispatches a CDP mouse-press/release on the renderer. Set clickCount: 2 for a double-tap / double-click — the taps are dispatched as one gesture with proper click counting, which two separate tap calls cannot guarantee. Use when you need to tap a button, link, or any tappable element on the screen. -Returns { tapped: true, timestampMs }. Fails if the simulator-server / emulator backend / Chromium CDP is not reachable for the given device. -Before tapping, determine the correct coordinates by using discovery tools — pick by platform: iOS / Android use \`describe\`, \`native-describe-screen\`, or \`debugger-component-tree\`; Chromium uses \`describe\` (the DOM walker), since the native and RN-specific discovery tools don't apply. More information in \`argent-device-interact\` skill`, +Returns { tapped: true, timestampMs }. Fails if the simulator-server / emulator backend / Chromium CDP / \`hdc\` is not reachable for the given device. +Before tapping, determine the correct coordinates by using discovery tools — pick by platform: iOS / Android use \`describe\`, \`native-describe-screen\`, or \`debugger-component-tree\`; Chromium uses \`describe\` (the DOM walker) and HarmonyOS uses \`describe\` (the \`uitest\` layout dump), since the native and RN-specific discovery tools don't apply to either. More information in \`argent-device-interact\` skill`, alwaysLoad: true, searchHint: "tap press button element device simulator emulator chromium touch down up click", zodSchema, diff --git a/packages/tool-server/src/tools/keyboard/index.ts b/packages/tool-server/src/tools/keyboard/index.ts index 16d2b33082..d6a57f326f 100644 --- a/packages/tool-server/src/tools/keyboard/index.ts +++ b/packages/tool-server/src/tools/keyboard/index.ts @@ -39,7 +39,7 @@ const zodSchema = z.object({ .number() .optional() .describe( - "Delay in ms between key presses (default 50). Ignored on Android phones/tablets (typed via `adb input text`, which has no per-key cadence), on Vega (text/keys injected in a single shot), and on TV targets (Apple TV / Android TV type the whole string at the daemon's own cadence)." + "Delay in ms between key presses (default 50). Ignored on Android phones/tablets (typed via `adb input text`, which has no per-key cadence), on HarmonyOS (`uitest uiInput text` types the whole string in one shot), on Vega (text/keys injected in a single shot), and on TV targets (Apple TV / Android TV type the whole string at the daemon's own cadence)." ), }); @@ -112,7 +112,7 @@ Provide text, key, or both — when both are given, the text is typed first and zodSchema, capability, searchHint: - "type text keyboard input named key enter escape arrow tv vega fire tv search field hid leanback", + "type text keyboard input named key enter escape arrow tv vega fire tv harmony harmonyos search field hid leanback", // No eager service: each branch resolves its backend lazily (TV control, // simulator-server, CDP, or Vega adb), since distinguishing a TV target is // async and a tvOS udid must never resolve simulator-server. diff --git a/packages/tool-server/src/tools/launch-app/index.ts b/packages/tool-server/src/tools/launch-app/index.ts index c00b2de929..05e29f5196 100644 --- a/packages/tool-server/src/tools/launch-app/index.ts +++ b/packages/tool-server/src/tools/launch-app/index.ts @@ -40,14 +40,14 @@ const zodSchema = z.object({ .string() .regex(BUNDLE_ID_PATTERN, "bundleId may only contain letters, digits, '.', '_' and '-'") .describe( - "App identifier. iOS: bundle id (e.g. com.apple.MobileSMS). Android: package name from build.gradle `applicationId` (e.g. com.android.settings). Chromium: arbitrary tag; the call is a no-op since the renderer is already running." + "App identifier. iOS: bundle id (e.g. com.apple.MobileSMS). Android: package name from build.gradle `applicationId` (e.g. com.android.settings). HarmonyOS: bundle name as listed by `bm dump -a` (e.g. com.huawei.hmos.calculator). Chromium: arbitrary tag; the call is a no-op since the renderer is already running." ), activity: z .string() .regex(ACTIVITY_PATTERN, "activity may only contain letters, digits, '.', '_', '-' and '/'") .optional() .describe( - "Android-only: fully-qualified Activity name (e.g. `.MainActivity` or `com.example/com.example.MainActivity`). If omitted on Android, the app's default launcher activity is used. Ignored on iOS / Chromium." + "Android-only: fully-qualified Activity name (e.g. `.MainActivity` or `com.example/com.example.MainActivity`). If omitted on Android, the app's default launcher activity is used. Ignored on iOS / Chromium / HarmonyOS (a HarmonyOS ability is resolved from the bundle's own `abilityInfos`)." ), }); diff --git a/packages/tool-server/src/tools/open-url/index.ts b/packages/tool-server/src/tools/open-url/index.ts index dec30a284d..233c7f0407 100644 --- a/packages/tool-server/src/tools/open-url/index.ts +++ b/packages/tool-server/src/tools/open-url/index.ts @@ -57,9 +57,9 @@ export const openUrlTool: ToolDefinition = { }, description: `Open a URL or URL scheme on the device. Use to navigate to a web page or deep-link into an app. On Chromium, this navigates the primary renderer to the given URL. -Cross-platform schemes: https://, tel:, mailto:. iOS also: messages://, settings://, maps://. Android also: geo:, plus any app-specific deep link. +Cross-platform schemes: https://, tel:, mailto:. iOS also: messages://, settings://, maps://. Android also: geo:, plus any app-specific deep link. HarmonyOS resolves any scheme through \`aa start -U\`. Deep-linking caveat: an https:// link opens the native app only when an installed app is verified for the link's domain (iOS Universal Links / Android App Links) — otherwise it opens in the browser, and on iOS simulators it may open in Safari even when the owning app is installed. To reliably open an installed app, use its custom scheme (scheme://path) or launch-app with its bundle id. -Returns { opened, url, note? }. note carries the deep-linking caveat when a web URL was opened on a native device. Fails if no app is registered to handle the URI (iOS/Android/HarmonyOS) or the renderer rejects the navigation (Chromium).`, +Returns { opened, url, note? }. note carries the deep-linking caveat when a web URL was opened on a native device, and on HarmonyOS additionally warns that \`aa start -U\` reports success for a web URL even when nothing opens it - confirm with describe or screenshot before treating the link as followed. Fails if no app is registered to handle the URI (iOS/Android/HarmonyOS) or the renderer rejects the navigation (Chromium).`, zodSchema, capability, services: (params): Record => { diff --git a/packages/tool-server/src/tools/restart-app/index.ts b/packages/tool-server/src/tools/restart-app/index.ts index cd5958f7d8..a66a561cbf 100644 --- a/packages/tool-server/src/tools/restart-app/index.ts +++ b/packages/tool-server/src/tools/restart-app/index.ts @@ -38,7 +38,7 @@ const zodSchema = z.object({ .regex(ACTIVITY_PATTERN, "activity may only contain letters, digits, '.', '_', '-' and '/'") .optional() .describe( - "Android-only: relaunch a non-launcher Activity (e.g. `.SettingsActivity` or `com.example/com.example.SettingsActivity`). If omitted, the app's default launcher activity is used. Ignored on iOS." + "Android-only: relaunch a non-launcher Activity (e.g. `.SettingsActivity` or `com.example/com.example.SettingsActivity`). If omitted, the app's default launcher activity is used. Ignored on iOS / HarmonyOS." ), }); @@ -75,7 +75,7 @@ Use when you need a clean in-memory state without a full reinstall. Also refresh Returns { restarted, bundleId }. Fails if the app is not installed.`, alwaysLoad: true, searchHint: - "terminate relaunch restart reset app bundle id package simulator emulator vega tvos fire tv", + "terminate relaunch restart reset app bundle id package simulator emulator vega tvos fire tv harmony harmonyos", zodSchema, capability, // ios-remote declares an eager native-devtools service (its handler shares diff --git a/packages/tool-server/src/tools/screenshot-diff/index.ts b/packages/tool-server/src/tools/screenshot-diff/index.ts index afc7d8e6eb..69ca159acb 100644 --- a/packages/tool-server/src/tools/screenshot-diff/index.ts +++ b/packages/tool-server/src/tools/screenshot-diff/index.ts @@ -113,9 +113,9 @@ Use when stable before/after screenshots exist and the expected result is pixel- For live captures, set exactly one of captureBaseline or captureCurrent; use baselinePath + captureCurrent for the common visual-regression flow. Returns { summary, diffPath, contextDiffPath }. The summary uses normalized [0,1] screen locations matching describe coordinates; diffPath is the full-size diff image and contextDiffPath is a downscaled image for MCP/agent display. Ignores the fixed top status-bar band for both pixel and OCR text comparisons. -Fails if the input sources are invalid, PNG files cannot be read, outputDir cannot be written, or the simulator-server / emulator backend is not reachable.`, +Fails if the input sources are invalid, PNG files cannot be read, outputDir cannot be written, or the simulator-server / emulator backend, or \`hdc\` on HarmonyOS, is not reachable.`, searchHint: - "compare screenshots png diff visual UI changes UI regression visual regression screenshot diff changed regions text ocr live capture", + "compare screenshots png diff visual UI changes UI regression visual regression screenshot diff changed regions text ocr live capture harmony harmonyos", zodSchema, capability, fileInputs, diff --git a/packages/tool-server/src/tools/screenshot/index.ts b/packages/tool-server/src/tools/screenshot/index.ts index 4e9e9fa4cb..895c6ec0e9 100644 --- a/packages/tool-server/src/tools/screenshot/index.ts +++ b/packages/tool-server/src/tools/screenshot/index.ts @@ -137,7 +137,7 @@ export function createScreenshotTool(registry: Registry): ToolDefinition Date: Tue, 11 Aug 2026 21:40:31 +0200 Subject: [PATCH 020/110] test(harmony): cover the four tool branches that carry real logic Each of these ships conditional behaviour, not a delegation, and each sat beside a test file that already held one case per other backend: - gesture-tap: the clickCount===2 native doubleClick fast path, the paced loop above it, and the normalized -> device-pixel conversion, against a non-square display and fractions where a swapped axis reads differently - gesture-swipe: the settle -> swipe/fling verb choice, which reads backwards at a glance, plus the duration->velocity conversion and both ends of its clamp - open-url: the web-URL caveat appended to the shared deep-link note, and the omit case for a custom scheme that already fails loudly on device - await-ui-element / await-screen-idle: that an unreadable screen cannot confirm `hidden`. HarmonyOS reaches that guarantee differently from its siblings - its fetcher throws rather than returning a flagged empty tree - so this pins the shared pollDescribeTree guard from a fifth direction. Every assertion was checked against a mutation of the code it covers: deleting the doubleClick path, swapping the tap axes, inverting the swipe verb, dropping the velocity clamp, reversing the swipe endpoints, dropping and unconditionally attaching the URL caveat, neutering isBlindRead, deleting the harmony fetchTree branch, and passing the prefixed device id where the connect key belongs. All fail. Neutering isBlindRead also reddens the iOS and Android cases, confirming the guard is shared rather than re-implemented. --- .../tool-server/test/await-harmony.test.ts | 245 ++++++++++++++++++ .../test/gesture-swipe-harmony.test.ts | 138 ++++++++++ .../test/gesture-tap-harmony.test.ts | 120 +++++++++ .../tool-server/test/open-url-harmony.test.ts | 80 ++++++ 4 files changed, 583 insertions(+) create mode 100644 packages/tool-server/test/await-harmony.test.ts create mode 100644 packages/tool-server/test/gesture-swipe-harmony.test.ts create mode 100644 packages/tool-server/test/gesture-tap-harmony.test.ts create mode 100644 packages/tool-server/test/open-url-harmony.test.ts diff --git a/packages/tool-server/test/await-harmony.test.ts b/packages/tool-server/test/await-harmony.test.ts new file mode 100644 index 0000000000..38b98ac984 --- /dev/null +++ b/packages/tool-server/test/await-harmony.test.ts @@ -0,0 +1,245 @@ +import { beforeEach, describe, expect, it, vi } from "vitest"; +import { + harmonyDisplay, + harmonyDumpLayout, + type HarmonyLayoutNode, +} from "../src/utils/harmony-uitest"; +import { createAwaitUiElementTool } from "../src/tools/await-ui-element"; +import { createAwaitScreenIdleTool } from "../src/tools/await-screen-idle"; +import { __primeDepCacheForTests, __resetDepCacheForTests } from "../src/utils/check-deps"; + +// Stub the `uitest`-over-`hdc` transport rather than `describeHarmony` itself, +// so the wait tools poll through the real dump parser and the real hint rules — +// those are what decide whether an unreadable screen may confirm `hidden`. +vi.mock("../src/utils/harmony-uitest", () => ({ + harmonyDisplay: vi.fn(), + harmonyDumpLayout: vi.fn(), +})); + +const CONNECT_KEY = "025DEK236V035771"; +const HARMONY_ID = `harmony-${CONNECT_KEY}`; +const DISPLAY = { width: 1216, height: 2688, screenOn: true }; + +// Neither wait tool resolves a service on harmony — the tree comes straight off +// `hdc` — so any lookup here is a wiring mistake, not a missing fixture. +function harmonyRegistry() { + return { + resolveService: vi.fn(async (urn: string) => { + throw new Error(`unexpected service: ${urn}`); + }), + } as any; +} + +/** + * A `uitest dumpLayout` result shaped like a Mate 60's: one bounded root, one + * app window, and a `Text` node per label. No labels means a dump with no + * windows at all — the "layout came back blank" case. + */ +function dumpWith(...labels: string[]): HarmonyLayoutNode { + return { + attributes: { bounds: "[0,0][1216,2688]" }, + children: [ + { + attributes: { + type: "WindowScene", + bundleName: "com.demo.app", + bounds: "[0,0][1216,2688]", + }, + children: labels.map((text, i) => ({ + attributes: { + type: "Text", + text, + bounds: `[100,${200 + i * 200}][900,${340 + i * 200}]`, + }, + })), + }, + ], + }; +} + +/** Serve `dumps` one per poll, repeating the last — a screen that changes mid-wait. */ +function sequenceDumps(dumps: HarmonyLayoutNode[]): { calls: () => number } { + let i = 0; + vi.mocked(harmonyDumpLayout).mockImplementation( + async () => dumps[Math.min(i++, dumps.length - 1)] + ); + return { calls: () => i }; +} + +beforeEach(() => { + __resetDepCacheForTests(); + __primeDepCacheForTests(["hdc"]); + vi.mocked(harmonyDisplay).mockReset().mockResolvedValue(DISPLAY); + vi.mocked(harmonyDumpLayout).mockReset(); +}); + +describe("await-ui-element on HarmonyOS", () => { + it("`visible` succeeds once the element appears in the dump", async () => { + const { calls } = sequenceDumps([dumpWith("Loading"), dumpWith("Submit")]); + const tool = createAwaitUiElementTool(harmonyRegistry()); + + const result = await tool.execute( + {}, + { + udid: HARMONY_ID, + condition: "visible", + selector: { text: "Submit" }, + timeoutMs: 2000, + pollIntervalMs: 10, + } + ); + + expect(result.success).toBe(true); + expect(calls()).toBeGreaterThan(1); + }); + + it("passes the bare connect key to `uitest`, not the `harmony-` prefixed device id", async () => { + sequenceDumps([dumpWith("Submit")]); + const tool = createAwaitUiElementTool(harmonyRegistry()); + + await tool.execute( + {}, + { + udid: HARMONY_ID, + condition: "exists", + selector: { text: "Submit" }, + timeoutMs: 2000, + pollIntervalMs: 10, + } + ); + + expect(harmonyDisplay).toHaveBeenCalledWith(CONNECT_KEY); + expect(harmonyDumpLayout).toHaveBeenCalledWith(CONNECT_KEY, expect.any(String)); + }); + + it("`hidden` succeeds once the element leaves a still-populated screen", async () => { + // The rest of the screen survives, which is what a genuine disappearance + // looks like; a dump with nothing left in it is the ambiguous case below. + sequenceDumps([dumpWith("Spinner"), dumpWith("Content")]); + const tool = createAwaitUiElementTool(harmonyRegistry()); + + const result = await tool.execute( + {}, + { + udid: HARMONY_ID, + condition: "hidden", + selector: { text: "Spinner" }, + timeoutMs: 2000, + pollIntervalMs: 10, + } + ); + + expect(result.success).toBe(true); + expect(result.note ?? "").not.toMatch(/never matched/i); + }); + + // ── Unreadable screen must never confirm `hidden` ───────────────────────── + // Same guarantee as the iOS "AX backend is down" and Android "empty dump" + // cases, reached differently: harmony's fetcher throws instead of returning a + // flagged empty tree, so what protects it is `pollDescribeTree` never handing + // a failed fetch to the sample predicate. + + it.each([ + { + when: "hdc cannot reach the device", + fail: () => + vi + .mocked(harmonyDisplay) + .mockRejectedValue(new Error("[Fail]Not match target founded, check connect-key please")), + noteFragment: /Not match target founded/, + }, + { + when: "uitest cannot dump the layout", + fail: () => + vi + .mocked(harmonyDumpLayout) + .mockRejectedValue( + new Error(`uitest dumpLayout failed on ${CONNECT_KEY}: Invalid parameters.`) + ), + noteFragment: /Invalid parameters/, + }, + ])("does NOT report `hidden` success when $when", async ({ fail, noteFragment }) => { + fail(); + const tool = createAwaitUiElementTool(harmonyRegistry()); + + const result = await tool.execute( + {}, + { + udid: HARMONY_ID, + condition: "hidden", + selector: { text: "Spinner" }, + timeoutMs: 40, + pollIntervalMs: 10, + } + ); + + expect(result.success).toBe(false); + expect(result.note).toMatch(/last tree fetch failed/i); + expect(result.note).toMatch(noteFragment); + }); + + it("does NOT report `hidden` success when the dump lists no windows", async () => { + // A dump that fetched cleanly but holds nothing is still not evidence the + // element is gone. Unlike Android — where only a prior match makes an empty + // tree suspicious — describeHarmony flags every windowless dump, so the + // blind-read guard fires from the very first poll and the hint reaches the note. + sequenceDumps([dumpWith()]); + const tool = createAwaitUiElementTool(harmonyRegistry()); + + const result = await tool.execute( + {}, + { + udid: HARMONY_ID, + condition: "hidden", + selector: { text: "Spinner" }, + timeoutMs: 40, + pollIntervalMs: 10, + } + ); + + expect(result.success).toBe(false); + expect(result.note).toMatch(/empty or unreadable/i); + expect(result.note).toMatch(/no windows/i); + }); +}); + +describe("await-screen-idle on HarmonyOS", () => { + it("settles once the dump stops changing", async () => { + sequenceDumps([dumpWith(), dumpWith("Settings")]); + const tool = createAwaitScreenIdleTool(harmonyRegistry()); + + const result = await tool.execute( + {}, + { udid: HARMONY_ID, timeoutMs: 2000, pollIntervalMs: 10, minStableMs: 30 } + ); + + expect(result.settled).toBe(true); + expect(result.waitedMs).toBeGreaterThanOrEqual(30); + expect(result.polls).toBeGreaterThan(1); + }); + + it("does not settle while the dump keeps changing (times out)", async () => { + sequenceDumps(Array.from({ length: 30 }, (_, i) => dumpWith(`item-${i}`))); + const tool = createAwaitScreenIdleTool(harmonyRegistry()); + + const result = await tool.execute( + {}, + { udid: HARMONY_ID, timeoutMs: 80, pollIntervalMs: 5, minStableMs: 40 } + ); + + expect(result.settled).toBe(false); + }); + + it("passes the bare connect key to `uitest`, not the `harmony-` prefixed device id", async () => { + sequenceDumps([dumpWith("Settings")]); + const tool = createAwaitScreenIdleTool(harmonyRegistry()); + + await tool.execute( + {}, + { udid: HARMONY_ID, timeoutMs: 2000, pollIntervalMs: 10, minStableMs: 0 } + ); + + expect(harmonyDisplay).toHaveBeenCalledWith(CONNECT_KEY); + expect(harmonyDumpLayout).toHaveBeenCalledWith(CONNECT_KEY, expect.any(String)); + }); +}); diff --git a/packages/tool-server/test/gesture-swipe-harmony.test.ts b/packages/tool-server/test/gesture-swipe-harmony.test.ts new file mode 100644 index 0000000000..6a9f527093 --- /dev/null +++ b/packages/tool-server/test/gesture-swipe-harmony.test.ts @@ -0,0 +1,138 @@ +import { beforeEach, describe, expect, it, vi } from "vitest"; + +// Only the hdc transport is faked. Everything between the tool and the wire - +// the render-service parse, the normalized-to-pixel conversion, the duration-to- +// velocity conversion and the velocity clamp - runs for real and is read back +// off the `uitest` command line these tests capture. +vi.mock("../src/utils/harmony-hdc", async (importOriginal) => ({ + ...(await importOriginal()), + runHdcShell: vi.fn(), +})); + +// Keep the real module (blueprints import from it too) but neutralise the +// fire-and-forget WebSocket send, so a HarmonyOS swipe that wrongly reached this +// transport shows up as an unexpected call rather than a socket error. +vi.mock("../src/utils/simulator-client", async (importOriginal) => ({ + ...(await importOriginal()), + sendCommand: vi.fn(), +})); + +// The HarmonyOS branch preflights `hdc`; stub it so these tests don't depend on +// a HarmonyOS toolchain being installed on the test host. +vi.mock("../src/utils/check-deps", async (importOriginal) => ({ + ...(await importOriginal()), + ensureDep: vi.fn(async () => {}), +})); + +import { gestureSwipeTool } from "../src/tools/gesture-swipe"; +import { ensureDep } from "../src/utils/check-deps"; +import { runHdcShell as realRunHdcShell } from "../src/utils/harmony-hdc"; +import { sendCommand } from "../src/utils/simulator-client"; + +const runHdcShell = vi.mocked(realRunHdcShell); + +const CONNECT_KEY = "025DEK236V035771"; +const HARMONY_UDID = `harmony-${CONNECT_KEY}`; +const services = {} as never; + +// Deliberately non-square: a transposed width/height moves all four coordinates +// below, so a `toDevicePoint` fed the wrong axis cannot pass. +const DISPLAY_WIDTH = 1200; +const DISPLAY_HEIGHT = 2000; + +// 0.25/0.9 lands on (300, 1800) and 0.75/0.5 on (900, 1000) - four distinct +// values, so a swapped x/y is visible too. The 600x800 leg makes the travel +// exactly 1000px, which keeps every expected velocity below a whole number. +const base = { udid: HARMONY_UDID, fromX: 0.25, fromY: 0.9, toX: 0.75, toY: 0.5 }; + +beforeEach(() => { + vi.mocked(sendCommand).mockClear(); + vi.mocked(ensureDep).mockClear(); + runHdcShell.mockReset(); + runHdcShell.mockImplementation(async (_connectKey, command) => + command.startsWith("hidumper") + ? { + stdout: `render resolution=${DISPLAY_WIDTH}x${DISPLAY_HEIGHT}\npowerStatus=POWER_STATUS_ON`, + exitCode: 0, + } + : { stdout: "No Error", exitCode: 0 } + ); +}); + +/** The single `uitest uiInput …` line the swipe put on the wire, and who it went to. */ +function uiInput(): { connectKey: string; command: string } { + const calls = runHdcShell.mock.calls.filter(([, cmd]) => cmd.startsWith("uitest uiInput")); + expect(calls).toHaveLength(1); + return { connectKey: calls[0][0], command: calls[0][1] }; +} + +/** The trailing velocity argument, which `uitest` takes in place of a duration. */ +function velocity(): number { + return Number(uiInput().command.split(" ").at(-1)); +} + +describe("gesture-swipe on HarmonyOS", () => { + it("sends the `fling` verb when not settling, at the converted pixel endpoints", async () => { + await expect( + gestureSwipeTool.execute(services, { ...base, durationMs: 500 }) + ).resolves.toMatchObject({ swiped: true }); + + // 1000px of travel in 0.5s is 2000px/s. The whole line is asserted because + // `uitest` validates almost nothing: a swapped from/to, a transposed display + // axis or a reordered velocity argument all still exit 0 on-device. + expect(uiInput().command).toBe("uitest uiInput fling 300 1800 900 1000 2000"); + + runHdcShell.mockClear(); + await gestureSwipeTool.execute(services, { ...base, durationMs: 500, settle: false }); + expect(uiInput().command).toBe("uitest uiInput fling 300 1800 900 1000 2000"); + }); + + it("sends the `swipe` verb when settling - the momentum-free one, despite the name", async () => { + await gestureSwipeTool.execute(services, { ...base, durationMs: 500, settle: true }); + + // The mapping reads backwards at a glance, so it is pinned rather than left + // to review: `settle` asks for no fling, so it takes `swipe` (a drag ending + // where it ends) and leaves `fling` (which hands the scroller a release + // velocity to coast on) as the default. Inverting the ternary is silent - + // both verbs exist, both exit 0, and only the coast distance differs. + expect(uiInput().command).toBe("uitest uiInput swipe 300 1800 900 1000 2000"); + }); + + it("clamps a too-fast swipe to uitest's velocity ceiling", async () => { + // 1000px in 10ms is 100_000px/s. Unclamped, `uitest` answers `Invalid + // parameters.` and the gesture does not happen at all - so this is the + // difference between a swipe that is merely capped and one that fails. + await gestureSwipeTool.execute(services, { ...base, durationMs: 10 }); + + expect(velocity()).toBe(40_000); + }); + + it("clamps a too-slow swipe to uitest's velocity floor", async () => { + // The other rejected end: 1000px over 20s is 50px/s. A clamp written with + // only one bound leaves this one out of range. + await gestureSwipeTool.execute(services, { ...base, durationMs: 20_000 }); + + expect(velocity()).toBe(200); + }); + + it("addresses the device by its hdc connect key, and never over the sim-server", async () => { + await gestureSwipeTool.execute(services, { ...base, durationMs: 500 }); + + // `hdc -t` takes what `hdc list targets` reports, not argent's `harmony-` + // prefixed id - passing the id through reaches no target at all. + expect(uiInput().connectKey).toBe(CONNECT_KEY); + // Preflighted, so a missing connector is a 424 install hint rather than a + // 500 from deeper in the hdc path. + expect(ensureDep).toHaveBeenCalledWith("hdc"); + // Dropping the branch's `return` would fall through to the interpolation + // loop and push a Down/Move/Up train at a `services.simulatorServer` no + // HarmonyOS device is behind, while still resolving `{ swiped: true }`. + expect(sendCommand).not.toHaveBeenCalled(); + }); + + it("does not declare the simulator-server service for a HarmonyOS target", () => { + // Resolving the blueprint would spawn a backend this path never uses and + // block on its ready-wait before the hdc call could start. + expect(gestureSwipeTool.services(base)).toEqual({}); + }); +}); diff --git a/packages/tool-server/test/gesture-tap-harmony.test.ts b/packages/tool-server/test/gesture-tap-harmony.test.ts new file mode 100644 index 0000000000..3f74e75492 --- /dev/null +++ b/packages/tool-server/test/gesture-tap-harmony.test.ts @@ -0,0 +1,120 @@ +import { beforeEach, describe, expect, it, vi } from "vitest"; + +// Only the two transport calls are stubbed; `toDevicePoint` stays real so the +// asserted pixel values are the ones a device would actually be handed. +vi.mock("../src/utils/harmony-uitest", async (importOriginal) => ({ + ...(await importOriginal()), + harmonyDisplay: vi.fn(), + harmonyTouch: vi.fn(async () => {}), +})); + +// The harmony branch preflights hdc; stub it so the tests don't need the +// HarmonyOS toolchain on the test host. +vi.mock("../src/utils/check-deps", async (importOriginal) => ({ + ...(await importOriginal()), + ensureDep: vi.fn(async () => {}), +})); + +import { gestureTapTool } from "../src/tools/gesture-tap"; +import { harmonyDisplay, harmonyTouch } from "../src/utils/harmony-uitest"; +import { ensureDep } from "../src/utils/check-deps"; + +const CONNECT_KEY = "025DEK236V035771"; +const HARMONY_UDID = `harmony-${CONNECT_KEY}`; + +/** A Mate 60's render resolution — portrait, so a swapped axis is visible. */ +const DISPLAY = { width: 1216, height: 2688, screenOn: true }; + +/** No service is resolved for a HarmonyOS tap; see the `services` block below. */ +const noServices = {} as never; + +const touchCalls = () => vi.mocked(harmonyTouch).mock.calls.map((c) => [c[1], c[2]] as const); + +beforeEach(() => { + vi.mocked(harmonyTouch).mockClear(); + vi.mocked(ensureDep).mockClear(); + vi.mocked(harmonyDisplay).mockReset(); + vi.mocked(harmonyDisplay).mockResolvedValue(DISPLAY); +}); + +describe("gesture-tap on HarmonyOS", () => { + it("taps once with the plain `click` verb by default", async () => { + await expect( + gestureTapTool.execute(noServices, { udid: HARMONY_UDID, x: 0.5, y: 0.5 }) + ).resolves.toMatchObject({ tapped: true }); + expect(touchCalls()).toEqual([["click", { x: 608, y: 1344 }]]); + // Preflighted so a missing connector fails with a 424 install hint rather + // than a generic 500 from deeper in the hdc path. + expect(ensureDep).toHaveBeenCalledWith("hdc"); + }); + + it("sends clickCount 2 as ONE native doubleClick, not two clicks", async () => { + await gestureTapTool.execute(noServices, { + udid: HARMONY_UDID, + x: 0.5, + y: 0.5, + clickCount: 2, + }); + // Two timed `click` injections are not guaranteed to land inside the OS + // double-tap window, which is the whole reason `clickCount` exists — a + // degradation to the generic loop would still resolve `{ tapped: true }` + // while never producing a double-tap on-device. + expect(touchCalls()).toEqual([["doubleClick", { x: 608, y: 1344 }]]); + }); + + it("falls back to that many `click` injections above 2, paced apart", async () => { + const startedAt = Date.now(); + await gestureTapTool.execute(noServices, { + udid: HARMONY_UDID, + x: 0.5, + y: 0.5, + clickCount: 3, + }); + // `uitest` has no native triple-click; three clicks on the same point are + // the only available form. A `doubleClick` slipping in here (or a dropped + // iteration) would send the app a different gesture than was asked for. + expect(touchCalls()).toEqual([ + ["click", { x: 608, y: 1344 }], + ["click", { x: 608, y: 1344 }], + ["click", { x: 608, y: 1344 }], + ]); + // Lower bound on the two inter-tap gaps: back-to-back injections are not a + // multi-tap the OS can count. Load only widens this, never narrows it. + expect(Date.now() - startedAt).toBeGreaterThanOrEqual(150); + // One display read for the whole gesture. Reading per click would insert a + // 50-190ms hidumper round trip between taps, on top of the gap the OS + // multi-tap window is being paced against. + expect(harmonyDisplay).toHaveBeenCalledTimes(1); + }); + + it("converts normalized coordinates to pixels of the display it just read", async () => { + await gestureTapTool.execute(noServices, { udid: HARMONY_UDID, x: 0.6, y: 0.35 }); + // 0.6*1216 and 0.35*2688, each rounded. The fractions are chosen so a + // swapped axis (941, 730), a width/height mix-up (1613, 426) and a scale + // off by one pixel (729, 940) all differ from the expected pair. + expect(touchCalls()).toEqual([["click", { x: 730, y: 941 }]]); + }); + + it("addresses the device by its hdc connect key, not the `harmony-` prefixed id", async () => { + await gestureTapTool.execute(noServices, { udid: HARMONY_UDID, x: 0.5, y: 0.5 }); + // `hdc -t` only knows the key it reported in `list targets`; the prefix is + // argent's own, and passing it through reaches no device at all. + expect(harmonyDisplay).toHaveBeenCalledWith(CONNECT_KEY); + expect(vi.mocked(harmonyTouch).mock.calls[0][0]).toBe(CONNECT_KEY); + }); +}); + +describe("gesture-tap service declaration", () => { + it("declares no simulator-server for a HarmonyOS target", () => { + // There is no simulator-server controller behind a HarmonyOS device, so + // declaring the blueprint would spawn a backend the tap never uses and + // block on its ready-wait before the hdc path ever runs. + expect(gestureTapTool.services({ udid: HARMONY_UDID, x: 0.5, y: 0.5 })).toEqual({}); + }); + + it("still declares the simulator-server for an Android target", () => { + expect(gestureTapTool.services({ udid: "emulator-5554", x: 0.5, y: 0.5 })).toHaveProperty( + "simulatorServer" + ); + }); +}); diff --git a/packages/tool-server/test/open-url-harmony.test.ts b/packages/tool-server/test/open-url-harmony.test.ts new file mode 100644 index 0000000000..735eae05c8 --- /dev/null +++ b/packages/tool-server/test/open-url-harmony.test.ts @@ -0,0 +1,80 @@ +import { describe, it, expect, vi, beforeEach } from "vitest"; +import { FAILURE_CODES, FailureError, type DeviceInfo } from "@argent/registry"; + +// harmonyImpl's only device work is `openHarmonyUrl`, whose own reading of `aa` +// stdout is covered in harmony-apps.test.ts. Stub it at the module boundary so +// these tests are about what the platform impl builds on top: the note, and the +// arguments it hands down. +const openHarmonyUrlMock = vi.fn(); +vi.mock("../src/utils/harmony-apps", async () => { + const actual = await vi.importActual("../src/utils/harmony-apps"); + return { ...actual, openHarmonyUrl: (...args: unknown[]) => openHarmonyUrlMock(...args) }; +}); + +import { httpDeepLinkNote } from "../src/tools/open-url/deep-link-note"; +import { harmonyImpl } from "../src/tools/open-url/platforms/harmony"; + +// A real handset id: the `harmony-` prefix is the registry's, the serial behind +// it is what `hdc` accepts as a connect key. +const CONNECT_KEY = "025DEK236V035771"; +const device = { + id: `harmony-${CONNECT_KEY}`, + platform: "harmony", + kind: "device", +} as unknown as DeviceInfo; + +beforeEach(() => { + openHarmonyUrlMock.mockReset(); + openHarmonyUrlMock.mockResolvedValue(undefined); +}); + +describe("open-url HarmonyOS handler surfaces the caveat only for web URLs", () => { + it("attaches note for an https URL, appended to the shared web-URL note", async () => { + const url = "https://bsky.app/profile/tvpworld.bsky.social"; + const res = await harmonyImpl.handler({}, { udid: device.id, url }, device); + + expect(res.opened).toBe(true); + expect(res.url).toBe(url); + // The HarmonyOS caveat extends the shared note rather than replacing it — + // both halves are load-bearing, and on this platform the caveat is the only + // signal that `opened: true` may mean nothing happened on screen. + expect(res.note).toContain(httpDeepLinkNote(url)); + expect(res.note).toMatch(/aa start -U. reports success for a web URL even when nothing opens/); + expect(res.note?.startsWith(httpDeepLinkNote(url)!)).toBe(true); + // No stray separator or stringified `undefined` from the composition. + expect(res.note).not.toMatch(/undefined|^\s|\s\s|\s$/); + // hdc gets the connect key, not the prefixed registry id, and the URL is + // handed down unchanged. + expect(openHarmonyUrlMock).toHaveBeenCalledWith(CONNECT_KEY, url); + }); + + it("omits note for a custom-scheme deep link", async () => { + const res = await harmonyImpl.handler( + {}, + { udid: device.id, url: "bluesky://profile/x" }, + device + ); + expect(res.opened).toBe(true); + // A custom scheme no app claims fails loudly on HarmonyOS (10103101), so + // there is nothing here for a caveat to hedge about. + expect(res.note).toBeUndefined(); + }); + + it("propagates a device-side failure instead of resolving with a note", async () => { + openHarmonyUrlMock.mockRejectedValue( + new FailureError( + `HarmonyOS device '${CONNECT_KEY}' could not open 'nope://x': Error Code:10103101 ` + + `Error Message:Failed to find a matching application for implicit launch.`, + { + error_code: FAILURE_CODES.HARMONY_ABILITY_START_FAILED, + failure_stage: "harmony_open_url", + failure_area: "tool_server", + error_kind: "not_found", + } + ) + ); + await expect( + harmonyImpl.handler({}, { udid: device.id, url: "nope://x" }, device) + ).rejects.toThrow(/10103101/); + }); +}); From 2ec6000b6a1dbdf1a6e80d2cb401fcfbbe6eed5e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Tue, 11 Aug 2026 22:15:04 +0200 Subject: [PATCH 021/110] fix(harmony): clip describe frames, keep what a collapsed wrapper knew, and enforce the hdc timeout Three defects the sibling implementations already handle. **Frames were clamped per component instead of clipped.** ArkUI reports real off-screen bounds for rows scrolled past the edge of a List - the old comment said so - and clamping x, y, width and height independently turned a row at [0,-400][1216,-260] into y=0 height=0.052: a full-width tap target at the top of the screen, with the same frame as the row genuinely straddling that edge. A row past the bottom reported a tap centre of y=1.026, and one past the right edge x=1.078. Both the Android and Vega adapters clip to the screen rect before normalising, and the Android call site carries a comment describing this exact failure; harmony now reuses the helper the Android one already exports. **The same-rect collapse discarded everything but `clickable`.** ArkUI sets `.id()` and the state flags on the OUTER component, so the wrapper of a same-rect pair is usually the only node that knows it is checked, disabled or scrollable. Collapsing it dropped the identifier agents and flows select on, turned a List holding one full-height row into a plain Cell with no scrollable region to swipe, reduced a Toggle to an unlabelled Image with no on/off state, and reported a disabled Button as an ordinary tappable one. The collapse now fires only when the wrapper genuinely knows nothing its child does not, which is what the comment always claimed it did. **`hdc` and `Emulator` were spawned without a kill signal.** `execFile`'s `timeout` sends `killSignal` once and never escalates, and the default SIGTERM is what a client blocked on a wedged daemon ignores. Measured against a stub that traps TERM: still running 6s after a 1s timeout, versus 1005ms with SIGKILL. That left every per-call HarmonyOS budget advisory - list-devices has a deadline backstop, the interaction tools have none. adb, vega-cli and simctl all set SIGKILL, each with a written rationale. Every assertion is mutation-checked, including per-field over the new collapse guard: seven of its eight fields fail the suite individually when dropped, and `checked` is pinned jointly with `checkable` (a checked switch is always checkable, so isolating it would mean asserting on a dump shape that does not occur). `value` was dropped from the guard as unreachable - it is only ever set alongside a label, which the collapse already refuses. --- .../platforms/harmony/layout-parser.ts | 70 +++++-- .../src/tools/keyboard/platforms/harmony.ts | 10 +- packages/tool-server/src/utils/harmony-cli.ts | 4 + packages/tool-server/src/utils/harmony-hdc.ts | 9 + .../test/cross-platform-tool.test.ts | 2 +- .../test/harmony-hdc-timeout.test.ts | 37 ++++ .../test/harmony-layout-parser.test.ts | 187 ++++++++++++++++++ 7 files changed, 299 insertions(+), 20 deletions(-) create mode 100644 packages/tool-server/test/harmony-hdc-timeout.test.ts diff --git a/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts b/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts index d5fe6c691d..f4ab6efd7d 100644 --- a/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts +++ b/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts @@ -1,5 +1,6 @@ import type { DescribeNode } from "../../contract"; import type { HarmonyLayoutNode } from "../../../../utils/harmony-uitest"; +import { clipBoundsToScreen } from "../android/uiautomator-parser"; /** * Turn a `uitest dumpLayout` tree into the shared describe tree. @@ -189,7 +190,15 @@ function build(node: HarmonyLayoutNode): HarmonyNode[] { // A wrapper whose only child covers exactly the same rect is a duplicate // layer: collapse to whichever of the two carries the label. - if (own.children.length === 1 && own.pixelBounds) { + // + // Only when the wrapper is genuinely a duplicate, though. ArkUI sets `.id()` + // and the state flags on the OUTER component, so a `Toggle` wrapping one + // full-bleed `Image` — or a `List` holding a single full-height row — is the + // node that knows it is checked, disabled or scrollable, and the child that + // fills its rect knows none of it. Collapsing those dropped the identifier an + // agent selects on and every flag but `clickable`, which reported a disabled + // button as a plain tappable one. + if (own.children.length === 1 && own.pixelBounds && !ownCarriesState(own)) { const only = own.children[0]; if (only.pixelBounds && rectsEqual(only.pixelBounds, own.pixelBounds) && !own.label) { return [{ ...only, clickable: only.clickable || own.clickable }]; @@ -199,6 +208,27 @@ function build(node: HarmonyLayoutNode): HarmonyNode[] { return [own]; } +/** + * Whether a node knows something its same-rect child cannot: an identifier to + * select on, or any state flag other than `clickable` (which the collapse + * merges). `clickable` alone still marks a plain duplicate layer. + * + * `value` is deliberately absent: it is only ever set alongside a label, which + * the caller's `!own.label` already keeps out of the collapse. + */ +function ownCarriesState(n: HarmonyNode): boolean { + return Boolean( + n.identifier || + n.longClickable || + n.scrollable || + n.checkable || + n.checked || + n.disabled || + n.focused || + n.selected + ); +} + function blank(attrs: Record, bounds: PixelRect | null): HarmonyNode { return { role: "Group", @@ -267,16 +297,7 @@ function deriveRole(type: string, attrs: Record): string { function toDescribeNode(n: HarmonyNode, screenW: number, screenH: number): DescribeNode { const children = n.children.map((c) => toDescribeNode(c, screenW, screenH)); const b = n.pixelBounds; - const frame = b - ? { - // Clamp into [0,1]: ArkUI reports off-screen bounds for items scrolled - // past the edge of a List, and the describe frame contract is closed. - x: clamp01(b.x / screenW), - y: clamp01(b.y / screenH), - width: clamp01(b.w / screenW), - height: clamp01(b.h / screenH), - } - : unionFrame(children); + const frame = b ? normalizeFrame(b, screenW, screenH) : unionFrame(children); const out: DescribeNode = { role: n.role, frame, children }; if (n.label) out.label = n.label; if (n.identifier) out.identifier = n.identifier; @@ -292,9 +313,30 @@ function toDescribeNode(n: HarmonyNode, screenW: number, screenH: number): Descr return out; } -function clamp01(v: number): number { - if (!Number.isFinite(v)) return 0; - return Math.max(0, Math.min(1, v)); +/** + * Clip a pixel rect to the screen, then normalise — the same discipline as the + * Android and Vega adapters. + * + * ArkUI reports off-screen bounds for rows scrolled past the edge of a List, so + * the order matters: normalising each component independently and clamping it + * into [0,1] turns a row at `[0,-400][1216,-260]` into `y=0, height=0.052` — a + * full-width tap target at the top of the screen, indistinguishable from the + * genuinely visible row straddling that edge. Clipping first leaves an entirely + * off-screen row with zero area, and keeps `x + width` inside the frame + * contract for one only partly off-screen. + */ +function normalizeFrame( + b: { x: number; y: number; w: number; h: number }, + screenW: number, + screenH: number +): DescribeNode["frame"] { + const clipped = clipBoundsToScreen(b, screenW, screenH); + return { + x: screenW > 0 ? clipped.x / screenW : 0, + y: screenH > 0 ? clipped.y / screenH : 0, + width: screenW > 0 ? clipped.w / screenW : 0, + height: screenH > 0 ? clipped.h / screenH : 0, + }; } function unionFrame(children: DescribeNode[]): DescribeNode["frame"] { diff --git a/packages/tool-server/src/tools/keyboard/platforms/harmony.ts b/packages/tool-server/src/tools/keyboard/platforms/harmony.ts index 9dab4fb10f..7fd6bef117 100644 --- a/packages/tool-server/src/tools/keyboard/platforms/harmony.ts +++ b/packages/tool-server/src/tools/keyboard/platforms/harmony.ts @@ -33,11 +33,11 @@ const HARMONY_KEYCODES: Record = { function resolveHarmonyKeycode(key: string): number { const lower = key.toLowerCase(); - // Own-property check, and case-folded like every other backend: `key` is a - // free string, so a prototype key like "constructor" would otherwise pass the - // nullish guard with a garbage value and be interpolated into the remote shell - // line (`harmonyKeyEvent` builds `uiInput keyEvent ${key}`) instead of - // rejecting as an unknown key. + // Own-property check, and case-folded for parity with the other named-key + // backends: `key` is a free string, so a prototype key like "constructor" + // would otherwise pass the nullish guard with a garbage value and be + // interpolated into the remote shell line (`harmonyKeyEvent` builds + // `uiInput keyEvent ${key}`) instead of rejecting as an unknown key. const code = Object.hasOwn(HARMONY_KEYCODES, lower) ? HARMONY_KEYCODES[lower] : undefined; if (code === undefined) { throw new InvalidToolInputError( diff --git a/packages/tool-server/src/utils/harmony-cli.ts b/packages/tool-server/src/utils/harmony-cli.ts index dc39d1dc09..5bf140531b 100644 --- a/packages/tool-server/src/utils/harmony-cli.ts +++ b/packages/tool-server/src/utils/harmony-cli.ts @@ -91,6 +91,9 @@ export interface HarmonyRunResult { const DEFAULT_TIMEOUT_MS = 30_000; +/** See `HDC_KILL_SIGNAL`: a SIGTERM the child may ignore leaves `timeout` unenforced. */ +const EMULATOR_KILL_SIGNAL = "SIGKILL" as const; + export async function runHarmonyEmulator( args: string[], timeoutMs = DEFAULT_TIMEOUT_MS @@ -105,6 +108,7 @@ export async function runHarmonyEmulator( try { const { stdout, stderr } = await execFileAsync(bin, args, { timeout: timeoutMs, + killSignal: EMULATOR_KILL_SIGNAL, maxBuffer: 8 * 1024 * 1024, }); return { stdout, stderr }; diff --git a/packages/tool-server/src/utils/harmony-hdc.ts b/packages/tool-server/src/utils/harmony-hdc.ts index c8bdb75be6..31b29fc21a 100644 --- a/packages/tool-server/src/utils/harmony-hdc.ts +++ b/packages/tool-server/src/utils/harmony-hdc.ts @@ -111,6 +111,14 @@ interface HdcRunResult { const DEFAULT_TIMEOUT_MS = 30_000; +// `execFile`'s `timeout` sends `killSignal` once and never escalates, and the +// default SIGTERM is exactly what an `hdc` client blocked on a wedged daemon can +// ignore — leaving the parent waiting past the deadline, so every timeout below +// would be advisory. Same reasoning and same shape as `ADB_KILL_SIGNAL`: `hdc` is +// a single-process client of a persistent shared daemon, so reaping the direct +// child is complete, and a group kill would wrongly take the daemon with it. +const HDC_KILL_SIGNAL = "SIGKILL" as const; + /** * Run `hdc` with the given argv. Returns the child's output whether it exited 0 * or not — see the header: the exit code carries no signal, so classification is @@ -124,6 +132,7 @@ export async function runHdc( try { const { stdout, stderr } = await execFileAsync(bin, args, { timeout: timeoutMs, + killSignal: HDC_KILL_SIGNAL, maxBuffer: 64 * 1024 * 1024, }); return { stdout, stderr }; diff --git a/packages/tool-server/test/cross-platform-tool.test.ts b/packages/tool-server/test/cross-platform-tool.test.ts index cf9176acdc..2232cfeb63 100644 --- a/packages/tool-server/test/cross-platform-tool.test.ts +++ b/packages/tool-server/test/cross-platform-tool.test.ts @@ -198,7 +198,7 @@ describe("dispatchByPlatform", () => { >({ toolId: "test", // Both kinds declared: a bare `harmony-` resolves to kind 'device' - // and an `harmony-emulator-` id to kind 'emulator', so declaring + // and a `harmony-emulator-` id to kind 'emulator', so declaring // only one lets assertSupported reject the other before dispatch runs. capability: { ...capabilityBoth, diff --git a/packages/tool-server/test/harmony-hdc-timeout.test.ts b/packages/tool-server/test/harmony-hdc-timeout.test.ts new file mode 100644 index 0000000000..474882612a --- /dev/null +++ b/packages/tool-server/test/harmony-hdc-timeout.test.ts @@ -0,0 +1,37 @@ +import { afterAll, describe, expect, it, vi } from "vitest"; +import { mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; + +// Resolve `hdc` to a stub instead of a real toolchain. Its own module-level +// binary cache starts empty per test file, so this is the only seam needed. +vi.mock("../src/utils/command-on-path", () => ({ + commandOnPath: vi.fn(async () => stub), +})); + +import { runHdc } from "../src/utils/harmony-hdc"; + +const stubDir = mkdtempSync(join(tmpdir(), "argent-hdc-stub-")); +const stub = join(stubDir, "hdc"); +// A wedged `hdc`: blocked on an unresponsive daemon and deaf to SIGTERM. +writeFileSync(stub, "#!/usr/bin/env bash\ntrap '' TERM\nsleep 30\n", { mode: 0o755 }); + +afterAll(() => rmSync(stubDir, { recursive: true, force: true })); + +describe("runHdc timeout enforcement", () => { + it("reaps a child that ignores SIGTERM, so the timeout it was given actually fires", async () => { + // `execFile`'s `timeout` sends `killSignal` once and never escalates, so with + // the default SIGTERM this call never settles and every per-call budget on + // the HarmonyOS path is advisory - only `list-devices` has a deadline behind + // it, and the interaction tools have nothing. Measured against this stub + // without `killSignal`: still running 6s after a 1s timeout. + vi.stubEnv("DEVECO_STUDIO_HOME", ""); + const started = Date.now(); + + await expect(runHdc(["list", "targets"], 800)).rejects.toThrow(); + + // Far below the stub's own 30s sleep: the assertion is that the deadline is + // enforced at all, not its precise latency on a loaded CI box. + expect(Date.now() - started).toBeLessThan(6_000); + }, 20_000); +}); diff --git a/packages/tool-server/test/harmony-layout-parser.test.ts b/packages/tool-server/test/harmony-layout-parser.test.ts index fa78d659f3..7523aa35c0 100644 --- a/packages/tool-server/test/harmony-layout-parser.test.ts +++ b/packages/tool-server/test/harmony-layout-parser.test.ts @@ -111,6 +111,127 @@ describe("parseHarmonyLayout", () => { expect(target.clickable).toBe(true); }); + // ArkUI sets `.id()` and the state flags on the OUTER component, so the + // wrapper of a same-rect pair is often the only node that knows it. Collapsing + // it away kept `clickable` and dropped the rest. + describe("same-rect duplicate layers", () => { + const wrap = (attrs: Record, child: Record) => + parseHarmonyLayout( + root([ + node({ type: "root", bundleName: "com.app", bounds: "[0,0][1216,2688]" }, [ + node(attrs, [node(child)]), + ]), + ]), + SCREEN + ).tree.children[0].children[0]; + + it("collapses a wrapper that knows nothing its child does not", () => { + const B = "[0,0][200,100]"; + const target = wrap( + { type: "Stack", clickable: "true", bounds: B }, + { + type: "Text", + text: "Submit", + bounds: B, + } + ); + expect(target.role).toBe("StaticText"); + expect(target.label).toBe("Submit"); + expect(target.clickable).toBe(true); + expect(target.children).toHaveLength(0); + }); + + it("keeps a wrapper that carries the identifier an agent selects on", () => { + const B = "[0,0][200,100]"; + const target = wrap( + { type: "Stack", clickable: "true", id: "tap-me", bounds: B }, + { + type: "Text", + text: "Submit", + bounds: B, + } + ); + expect(target.identifier).toBe("tap-me"); + // The child's label stays reachable rather than being traded for the id. + expect(target.children[0].label).toBe("Submit"); + }); + + // Each case below carries ONLY the state under test - no `id` - so it pins + // that flag specifically. With an identifier present too, any one of these + // could be dropped from the guard and the case would still pass on the + // identifier alone. + it("does not report a disabled control as a plain tappable one", () => { + const B = "[0,0][200,100]"; + const target = wrap( + { type: "Button", clickable: "true", enabled: "false", bounds: B }, + { type: "Text", text: "Submit", bounds: B } + ); + expect(target.disabled).toBe(true); + expect(target.role).toBe("Button"); + }); + + it("keeps a scroll container that holds a single full-height row", () => { + const B = "[0,0][1216,1000]"; + const target = wrap( + { type: "List", bounds: B }, + { + type: "ListItem", + text: "only row", + bounds: B, + } + ); + // Without this the agent sees no scrollable region and never swipes. + expect(target.role).toBe("ScrollView"); + expect(target.scrollable).toBe(true); + }); + + it("keeps a switch's checked state when its image fills the same rect", () => { + const B = "[0,0][200,100]"; + const target = wrap( + { type: "Toggle", clickable: "true", checkable: "true", checked: "true", bounds: B }, + { type: "Image", bounds: B } + ); + expect(target.role).toBe("Switch"); + expect(target.checkable).toBe(true); + expect(target.checked).toBe(true); + }); + + it("keeps a long-pressable wrapper", () => { + const B = "[0,0][200,100]"; + const target = wrap( + { type: "Stack", longClickable: "true", bounds: B }, + { type: "Image", bounds: B } + ); + expect(target.longClickable).toBe(true); + }); + + it("keeps a switch that is off, not only one that is on", () => { + // `checked` and `checkable` cover for each other on an on-switch, so the + // off case is what pins `checkable` by itself. + const B = "[0,0][200,100]"; + const target = wrap( + { type: "Toggle", clickable: "true", checkable: "true", bounds: B }, + { type: "Image", bounds: B } + ); + expect(target.checkable).toBe(true); + expect(target.checked).toBeUndefined(); + }); + + it("keeps a selected wrapper, and a focused one", () => { + // Not a `Stack`: a layout container carrying neither a label nor one of + // the interactive flags is hoisted away as scaffolding before the collapse + // is ever reached, so it cannot exercise this guard. + const B = "[0,0][200,100]"; + expect( + wrap({ type: "ListItem", selected: "true", bounds: B }, { type: "Image", bounds: B }) + .selected + ).toBe(true); + expect( + wrap({ type: "ListItem", focused: "true", bounds: B }, { type: "Image", bounds: B }).focused + ).toBe(true); + }); + }); + it("drops decoration that can never be a target", () => { const tree = parseHarmonyLayout( root([ @@ -245,4 +366,70 @@ describe("parseHarmonyLayout", () => { const { screen } = parseHarmonyLayout(node({}, []), SCREEN); expect(screen).toEqual(SCREEN); }); + + // A List reports real off-screen bounds for its scrolled rows, so the frame + // has to be clipped to the screen before it is normalized. Normalizing each + // component on its own and clamping into [0,1] gave a row at [0,-400][1216,-260] + // the frame y=0 height=0.052 — a full-width tap target at the top of the + // screen, with the same frame as the row genuinely straddling that edge. + describe("off-screen rows", () => { + const rows = (items: Array<[string, string]>) => + parseHarmonyLayout( + root([ + node({ type: "WindowScene", bundleName: "com.demo.app", bounds: "[0,0][1216,2688]" }, [ + node( + { type: "List", bounds: "[0,0][1216,2688]", scrollable: "true" }, + items.map(([text, bounds]) => + node({ type: "ListItem", text, bounds, clickable: "true" }) + ) + ), + ]), + ]), + SCREEN + ).tree.children[0].children[0].children; + + it("gives a row scrolled fully off the screen no area, on every edge", () => { + const [above, below, right] = rows([ + ["ABOVE", "[0,-400][1216,-260]"], + ["BELOW", "[0,2888][1216,3028]"], + ["RIGHT", "[1266,400][1456,540]"], + ]); + expect(above.frame.height).toBe(0); + expect(below.frame.height).toBe(0); + expect(right.frame.width).toBe(0); + }); + + it("keeps only the visible slice of a row straddling an edge", () => { + // 140px tall, 80px of it on screen — and it must NOT share a frame with + // the fully-off-screen row above it, which is what the old clamp produced. + const [above, straddle] = rows([ + ["ABOVE", "[0,-400][1216,-260]"], + ["STRADDLE", "[0,-60][1216,80]"], + ]); + expect(straddle.frame.y).toBe(0); + expect(straddle.frame.height).toBeCloseTo(80 / 2688, 6); + expect(straddle.frame).not.toEqual(above.frame); + }); + + it("never lets a frame run past the screen, so a tap centre stays on it", () => { + for (const row of rows([ + ["ABOVE", "[0,-400][1216,-260]"], + ["BELOW", "[0,2888][1216,3028]"], + ["RIGHT", "[1266,400][1456,540]"], + ["PART-RIGHT", "[1100,400][1456,540]"], + ["VISIBLE", "[0,1000][1216,1140]"], + ])) { + const { x, y, width, height } = row.frame; + expect(x + width, `${row.label}: x+width`).toBeLessThanOrEqual(1); + expect(y + height, `${row.label}: y+height`).toBeLessThanOrEqual(1); + } + }); + + it("leaves a fully on-screen row exactly as measured", () => { + const [visible] = rows([["VISIBLE", "[0,1000][1216,1140]"]]); + expect(visible.frame.y).toBeCloseTo(1000 / 2688, 6); + expect(visible.frame.height).toBeCloseTo(140 / 2688, 6); + expect(visible.frame.width).toBe(1); + }); + }); }); From 90be9c5d52b9709f9e0bfe4c8fb687e2025df7df Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Tue, 11 Aug 2026 22:24:32 +0200 Subject: [PATCH 022/110] fix(harmony): stop boot-device adopting a device it did not start `waitForHarmonyTarget` returned the first `hdc` target that was not in the pre-start snapshot, in print order, with no cardinality check and nothing else tying the key to the instance. Anything that reached `Connected` inside the boot window was therefore adopted: a handset plugged in or authorised mid-boot, a row settling out of `Offline`, or a concurrent `boot-device` for another instance (`inFlightHarmonyBoots` keys on the instance name, so those are not coalesced). The result is not a failed boot, which is what makes it worth guarding. It is a valid, drivable id returned with `booted: true`, `instanceName` naming the instance it did not reach, and no note - so every later gesture-tap, keyboard and launch-app in the session lands on that device. Measured before the fix: a second handset authorising while the emulator never registers returns `harmony-BQR0223A14001199`. Two signals now bound it, both already in the data the parser returns. A `USB` target is a cable-attached handset and cannot be a host-local emulator, which `hdc` reaches over TCP loopback (`127.0.0.1:5555`, measured), so USB arrivals are excluded; and two eligible arrivals are refused rather than guessed between, exactly as the Android path refuses when more than one serial appears and it cannot disambiguate by AVD name. A null `connection` stays eligible so an image that registers in some shape not seen here still boots. Both degradations reach the caller as the existing named no-target note. Mutation-checked: dropping the USB exclusion, dropping the cardinality guard, inverting the exclusion, and narrowing it to `=== "TCP"` (which would refuse a boot that worked) each fail the suite. --- .../src/tools/devices/boot-device.ts | 43 ++++++++++++-- .../test/boot-device-harmony.test.ts | 59 ++++++++++++++++++- 2 files changed, 96 insertions(+), 6 deletions(-) diff --git a/packages/tool-server/src/tools/devices/boot-device.ts b/packages/tool-server/src/tools/devices/boot-device.ts index efca7e1226..8f4f33e4a4 100644 --- a/packages/tool-server/src/tools/devices/boot-device.ts +++ b/packages/tool-server/src/tools/devices/boot-device.ts @@ -1528,10 +1528,29 @@ const HARMONY_NO_HDC = "registered as and no interaction tool can reach it. Install DevEco Studio's device " + "connector (or put `hdc` on PATH), then call `list-devices`."; +/** The targets `hdc` can drive right now. */ +async function connectedHarmonyTargets() { + const targets = await listHarmonyHdcTargets().catch(() => []); + return targets.filter((t) => t.state === "Connected"); +} + /** The connect keys `hdc` can drive right now. */ async function connectedHarmonyKeys(): Promise> { - const targets = await listHarmonyHdcTargets().catch(() => []); - return new Set(targets.filter((t) => t.state === "Connected").map((t) => t.connectKey)); + return new Set((await connectedHarmonyTargets()).map((t) => t.connectKey)); +} + +/** + * Whether a target could be the emulator this host just started. + * + * An emulator runs on the host and `hdc` reaches it over TCP loopback + * (`127.0.0.1:5555`, measured), so a target the connector reports as `USB` is a + * cable-attached handset and cannot be it. Anything else — including a null + * connection, which is what a `hdc list targets` without `-v` yields — is left + * eligible rather than filtered out, so an image that registers in some shape + * not seen here still boots. + */ +function couldBeHarmonyEmulator(target: { connection: string | null }): boolean { + return target.connection !== "USB"; } /** @@ -1605,6 +1624,19 @@ async function startHarmonyEmulator( * `checkAlive` is polled alongside, and throws if the emulator died: without it * a start that failed after the spawn returned would be indistinguishable from * one still booting, and would burn the whole budget before saying so. + * + * Exactly one *eligible* arrival, or none. Arrival alone does not identify + * anything once a second device can produce one: a handset plugged in or + * authorised inside the window, a row settling out of `Offline`, or a + * concurrent `boot-device` for another instance (`inFlightHarmonyBoots` keys on + * the instance, so those are not coalesced). Returning the wrong key is not a + * failed boot — it is a drivable id, so every later tap and keystroke lands on + * that device instead, which is the case for being strict here. USB arrivals + * are excluded outright (a host-local emulator is not on a cable) and two + * eligible arrivals are refused rather than guessed between, exactly as the + * Android path refuses when more than one serial appears. There is no + * `-avdName` equivalent to disambiguate with, so the caller gets the named + * no-target degradation and `list-devices`. */ async function waitForHarmonyTarget( before: Set, @@ -1612,9 +1644,10 @@ async function waitForHarmonyTarget( checkAlive: () => void ): Promise { for (;;) { - for (const key of await connectedHarmonyKeys()) { - if (!before.has(key)) return key; - } + const arrived = (await connectedHarmonyTargets()).filter( + (t) => !before.has(t.connectKey) && couldBeHarmonyEmulator(t) + ); + if (arrived.length === 1) return arrived[0]!.connectKey; checkAlive(); const remaining = deadline - Date.now(); if (remaining <= 0) return null; diff --git a/packages/tool-server/test/boot-device-harmony.test.ts b/packages/tool-server/test/boot-device-harmony.test.ts index 86021fbc1f..793ef2090e 100644 --- a/packages/tool-server/test/boot-device-harmony.test.ts +++ b/packages/tool-server/test/boot-device-harmony.test.ts @@ -72,8 +72,15 @@ function boot(params: Record) { return createBootDeviceTool(registry).execute!({}, { harmonyInstance: INSTANCE, ...params }); } +/** A row of `hdc list targets -v`; `connection` is null without that flag. */ +interface HdcTargetRow { + connectKey: string; + connection: string | null; + state: string; +} + /** Successive `hdc list targets` results, the last one repeating forever. */ -function targets(...rounds: (typeof PHONE)[][]) { +function targets(...rounds: HdcTargetRow[][]) { let call = 0; listHarmonyHdcTargets.mockImplementation(() => Promise.resolve(rounds[Math.min(call++, rounds.length - 1)]) @@ -158,6 +165,56 @@ describe("boot-device — HarmonyOS emulator path", () => { expect(((await pending) as { udid: string }).udid).toBe(`harmony-${EMULATOR_KEY}`); }); + it("does not adopt a cable-attached handset that arrives during the boot", async () => { + // A second HarmonyOS target reaches Connected inside the boot window - a + // phone plugged in or authorised mid-boot, or a still-settling row flipping + // from Offline - while the emulator itself never registers. Arrival alone + // cannot tell the two apart, so adopting the first new key hands back a + // device this call did not boot. + const OTHER = { connectKey: "BQR0223A14001199", connection: "USB", state: "Connected" }; + targets([PHONE], [PHONE, OTHER]); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 30_000 }); + await vi.advanceTimersByTimeAsync(31_000); + const result = (await pending) as { udid: string; note?: string }; + + expect(result.udid).not.toBe(`harmony-${OTHER.connectKey}`); + expect(result.udid).toBe(`harmony-emulator-${INSTANCE}`); + expect(result.note).toBeTruthy(); + }); + + it("still adopts an arrival whose connection column is absent", async () => { + // `hdc list targets` without `-v` prints the bare key, so `connection` is + // null. Treating that as ineligible would refuse a boot that worked, on any + // image or connector shape not seen here - so the USB exclusion has to fail + // open rather than allow-list a single spelling. + const UNTYPED = { connectKey: "127.0.0.1:5555", connection: null, state: "Connected" }; + targets([PHONE], [PHONE, UNTYPED]); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 30_000 }); + await vi.advanceTimersByTimeAsync(4_000); + + expect(((await pending) as { udid: string }).udid).toBe(`harmony-${EMULATOR_KEY}`); + }); + + it("refuses to guess when two targets it could have started both arrive", async () => { + // A concurrent `boot-device` for a different instance is not coalesced - + // `inFlightHarmonyBoots` keys on the instance name - so both emulators + // register inside this call's window and arrival no longer picks one out. + const OTHER_EMULATOR = { connectKey: "127.0.0.1:5557", connection: "TCP", state: "Connected" }; + targets([PHONE], [PHONE, emulatorTarget, OTHER_EMULATOR]); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 30_000 }); + await vi.advanceTimersByTimeAsync(31_000); + const result = (await pending) as { udid: string; note?: string }; + + expect(result.udid).toBe(`harmony-emulator-${INSTANCE}`); + expect(result.note).toBeTruthy(); + }); + it("names the instance and says why when nothing registers within the budget", async () => { targets([PHONE]); vi.useFakeTimers(); From 4807a3b92fc5815af8c230619ba2cf179f8b6b44 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Tue, 11 Aug 2026 22:30:32 +0200 Subject: [PATCH 023/110] fix(harmony): ask the manager whether it died when there is no connector to wait on Without `hdc` the arrival wait is skipped entirely, so nothing consulted `emulator.exited()` - and the result returned `booted: true` under a note whose first clause, "The instance started", nothing had checked. A manager that had already failed (a missing image, an instance that was still running) was reported as a successful boot with a connector problem. The liveness check is now a named function called on both paths, so the no-hdc case surfaces the manager's own diagnostic instead. Also pins the four keycodes nothing held. `keyboard-key-order` asserted only `enter`; `uitest` answers `No Error` to any keyID, so a swapped `arrow-left` / `arrow-right` pair would have inverted the arrows on a real device with the suite green. Each of the five is now asserted against its literal measured value, the way the Android backend pins android.view.KeyEvent codes. Both mutation-checked: removing the no-connector liveness check, and swapping or altering any keycode, each fail the suite. --- .../src/tools/devices/boot-device.ts | 68 +++++++++++-------- .../test/boot-device-harmony.test.ts | 15 ++++ .../test/keyboard-key-order.test.ts | 20 ++++++ 3 files changed, 74 insertions(+), 29 deletions(-) diff --git a/packages/tool-server/src/tools/devices/boot-device.ts b/packages/tool-server/src/tools/devices/boot-device.ts index 8f4f33e4a4..d99bf5709f 100644 --- a/packages/tool-server/src/tools/devices/boot-device.ts +++ b/packages/tool-server/src/tools/devices/boot-device.ts @@ -1725,37 +1725,47 @@ async function bootHarmonyImpl(params: { // watch, so the wait would spend the whole budget concluding that nothing // registered — blaming the emulator for the connector's absence. const hdcAvailable = Boolean(await resolveHdc()); - const connectKey = hdcAvailable - ? await waitForHarmonyTarget(before, bootDeadline, () => { - const exit = emulator.exited(); - if (!exit) return; - // The exit code is no verdict: `-start` on a missing instance exits 1 - // while the manager's other failures exit 0, so the diagnostic it - // printed is the only signal (see `harmony-cli.ts`). - const diagnostic = emulatorFailure({ stdout: exit.output, stderr: "" }); - if (!diagnostic) { - throw new Error( - `The HarmonyOS emulator manager exited (${exit.reason}) before "${params.instanceName}" ` + - `registered with \`hdc\`, so nothing is running to drive. It printed: ${ - exit.output.trim() || "(nothing)" - }` - ); - } - const imageMissing = - isChinaOnlyRestriction(diagnostic) || diagnostic.includes("Cannot find image"); - // Only the manager's own words justify blaming the region; an empty - // instance list is merely consistent with it. - const cause = imageMissing - ? ` ${HARMONY_IMAGE_RESTRICTION}` - : instances?.length === 0 - ? ` ${HARMONY_NO_INSTANCES}` - : ""; + const assertEmulatorAlive = () => { + { + const exit = emulator.exited(); + if (!exit) return; + // The exit code is no verdict: `-start` on a missing instance exits 1 + // while the manager's other failures exit 0, so the diagnostic it + // printed is the only signal (see `harmony-cli.ts`). + const diagnostic = emulatorFailure({ stdout: exit.output, stderr: "" }); + if (!diagnostic) { throw new Error( - cause - ? `Failed to start HarmonyOS emulator "${params.instanceName}".${cause} The manager reported: ${diagnostic}` - : `Failed to start HarmonyOS emulator "${params.instanceName}": ${diagnostic}` + `The HarmonyOS emulator manager exited (${exit.reason}) before "${params.instanceName}" ` + + `registered with \`hdc\`, so nothing is running to drive. It printed: ${ + exit.output.trim() || "(nothing)" + }` ); - }) + } + const imageMissing = + isChinaOnlyRestriction(diagnostic) || diagnostic.includes("Cannot find image"); + // Only the manager's own words justify blaming the region; an empty + // instance list is merely consistent with it. + const cause = imageMissing + ? ` ${HARMONY_IMAGE_RESTRICTION}` + : instances?.length === 0 + ? ` ${HARMONY_NO_INSTANCES}` + : ""; + throw new Error( + cause + ? `Failed to start HarmonyOS emulator "${params.instanceName}".${cause} The manager reported: ${diagnostic}` + : `Failed to start HarmonyOS emulator "${params.instanceName}": ${diagnostic}` + ); + } + }; + + // Without a connector there is nothing to watch, so the manager is asked once + // instead: otherwise a start that had already failed would be reported as + // `booted: true` under a note whose first clause — "the instance started" — + // nothing had checked. + if (!hdcAvailable) assertEmulatorAlive(); + + const connectKey = hdcAvailable + ? await waitForHarmonyTarget(before, bootDeadline, assertEmulatorAlive) : null; return { diff --git a/packages/tool-server/test/boot-device-harmony.test.ts b/packages/tool-server/test/boot-device-harmony.test.ts index 793ef2090e..2224e5922a 100644 --- a/packages/tool-server/test/boot-device-harmony.test.ts +++ b/packages/tool-server/test/boot-device-harmony.test.ts @@ -239,6 +239,21 @@ describe("boot-device — HarmonyOS emulator path", () => { expect(result.note).toMatch(/`hdc` was not found/); }); + it("does not claim the instance started when the manager already died and hdc is missing", async () => { + // With no connector the arrival wait is skipped, so nothing else ever + // consults the manager - and the note it would otherwise return opens with + // "The instance started", which at that point nothing had checked. + // The manager dies while the connector is being looked for, which is the + // real ordering: `Emulator -start` fails in milliseconds, the `hdc` lookup + // is a filesystem probe. + resolveHdc.mockImplementation(async () => { + child.die("Failed to start emulator: this emulator instance is already running"); + return null; + }); + + await expect(boot({})).rejects.toThrow(/already running/); + }); + it("fails fast when the manager dies before the instance registers", async () => { targets([PHONE]); vi.useFakeTimers(); diff --git a/packages/tool-server/test/keyboard-key-order.test.ts b/packages/tool-server/test/keyboard-key-order.test.ts index 9abea4750a..21314cb7e9 100644 --- a/packages/tool-server/test/keyboard-key-order.test.ts +++ b/packages/tool-server/test/keyboard-key-order.test.ts @@ -183,4 +183,24 @@ describe("keyboard text+key ordering", () => { expect(harmonyKeyEvent).toHaveBeenCalledWith(HARMONY_CONNECT_KEY, HARMONY_ENTER_KEYID); }); + + it("harmony: sends each supported key its own measured keyID", async () => { + // Literal keyIDs, not the source table: `uitest` answers `No Error` to any + // number it is handed, so a swapped pair would invert the arrows on a real + // device and nothing else in the suite would notice. Each was watched + // against a text field on a HarmonyOS 6.0.1 handset. + const expected: Record = { + "enter": "2054", + "backspace": "2055", + "space": "2050", + "arrow-left": "2014", + "arrow-right": "2015", + }; + + for (const [key, keyId] of Object.entries(expected)) { + vi.mocked(harmonyKeyEvent).mockClear(); + await harmonyImpl.handler({}, { udid: HARMONY.id, key }, HARMONY); + expect(harmonyKeyEvent, key).toHaveBeenCalledWith(HARMONY_CONNECT_KEY, keyId); + } + }); }); From e41cd44ee4ced2ab722ee502da8f364a0b1955b6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Tue, 11 Aug 2026 23:05:45 +0200 Subject: [PATCH 024/110] fix(harmony): make the no-connector liveness check actually fire, and drop off-screen rows from describe Both are defects in the previous two commits, found by reviewing them as new code. **The no-hdc liveness check was dead.** 4807a3b9 asked the manager once before returning, to stop `booted: true` being reported under a note whose first clause nothing had checked. But nothing between the spawn and that check does any I/O - `resolveHdc` is a warm memo by then, primed by the pre-start snapshot - so it ran microtasks after `spawn()`, before any child could have exited. Measured with the manager dying 5ms later, as a failing `-start` really does: the boot still resolved `booted: true`. The test only passed because the mock killed the child inside the `resolveHdc` lookup, which production never does. It now watches the manager for a short bounded grace instead; the test uses the real ordering. **Clipped frames were still tap targets.** 2ec6000b stopped an off-screen row claiming a visible slice of screen, but left it in the tree as a zero-area `[clickable]` node - and the tap centre the describe description documents puts that at the status bar for a row above the fold, the nav bar for one below. The Android trimmer prunes the same rows through its scroll-clip pass; harmony has no clip window to inherit, so the clipped rect is the signal. A node with surviving children is kept regardless, since it is the child that would be lost. Two errors in the first attempt at that, both caught by mutation: the predicate required zero in BOTH axes, which never fires (a row above the fold clips to full width and no height); and windows are assembled on a separate path, so the filter applied only below depth 1. Also corrects a claim in `waitForHarmonyTarget`'s own docstring. It said a concurrent `boot-device` for another instance is answered by refusing two arrivals - true only when both register inside one 2s poll window. Two emulators land seconds apart, so each boot sees a single arrival. Serializing every harmony boot would close it, at the cost of head-of-line blocking them all on one slow start, and it still would not cover an emulator started from DevEco Studio; the limit is now stated rather than overclaimed. --- .../platforms/harmony/layout-parser.ts | 26 ++++++- .../src/tools/devices/boot-device.ts | 71 ++++++++++++++----- .../test/boot-device-harmony.test.ts | 9 +-- .../test/harmony-layout-parser.test.ts | 68 +++++++++++++++--- 4 files changed, 141 insertions(+), 33 deletions(-) diff --git a/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts b/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts index f4ab6efd7d..9d787e0c66 100644 --- a/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts +++ b/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts @@ -294,8 +294,28 @@ function deriveRole(type: string, attrs: Record): string { } } +/** + * Whether a node lies entirely outside the screen and takes nothing with it. + * + * A `List` keeps its scrolled-off rows in the dump with real off-screen bounds, + * and clipping those to the screen leaves a zero-area rect. Emitting it anyway + * puts a `[clickable]` line in front of the agent whose documented tap centre + * (`x + width/2`, `y + height/2`) is the status bar for a row above the fold and + * the nav bar for one below it. The Android trimmer prunes the same rows, by its + * own scroll-clip pass; harmony has no clip window to inherit, so the clipped + * rect is the signal. A node with surviving children is kept regardless: it is + * the child that would be lost, and a bounds-less parent's frame is their union. + */ +function isFullyOffScreen(frame: DescribeNode["frame"], children: DescribeNode[]): boolean { + // Zero AREA, not zero in both axes: a row scrolled above the fold clips to the + // full screen width and no height at all. + return children.length === 0 && frame.width * frame.height === 0; +} + function toDescribeNode(n: HarmonyNode, screenW: number, screenH: number): DescribeNode { - const children = n.children.map((c) => toDescribeNode(c, screenW, screenH)); + const children = n.children + .map((c) => toDescribeNode(c, screenW, screenH)) + .filter((c) => !isFullyOffScreen(c.frame, c.children)); const b = n.pixelBounds; const frame = b ? normalizeFrame(b, screenW, screenH) : unionFrame(children); const out: DescribeNode = { role: n.role, frame, children }; @@ -378,7 +398,9 @@ export function parseHarmonyLayout( for (const child of root.children ?? []) { const attrs = child.attributes ?? {}; const bundle = (attrs.bundleName ?? "").trim(); - const built = build(child).map((n) => toDescribeNode(n, screen.width, screen.height)); + const built = build(child) + .map((n) => toDescribeNode(n, screen.width, screen.height)) + .filter((n) => !isFullyOffScreen(n.frame, n.children)); if (built.length === 0) continue; if (!bundle) { windows.push(...built); diff --git a/packages/tool-server/src/tools/devices/boot-device.ts b/packages/tool-server/src/tools/devices/boot-device.ts index d99bf5709f..8049521e06 100644 --- a/packages/tool-server/src/tools/devices/boot-device.ts +++ b/packages/tool-server/src/tools/devices/boot-device.ts @@ -1481,6 +1481,17 @@ const HARMONY_TARGET_POLL_MS = 2_000; */ const HARMONY_STOP_GRACE_MS = 30_000; +/** + * How long the manager is watched for an immediate failure when there is no + * `hdc` to wait on a target with. A failing `-start` prints its diagnostic and + * exits at once, so this only has to outlast process startup — it is not a boot + * budget, and the connector-absent path must still fail fast. + */ +const HARMONY_NO_HDC_GRACE_MS = 3_000; + +/** Cadence for the grace above; short, since it is only outlasting a process start. */ +const HARMONY_EXIT_POLL_MS = 50; + /** * `Emulator -install` outside mainland China prints "…available only in the * Chinese mainland" and downloads nothing, and no instance can be created @@ -1626,17 +1637,21 @@ async function startHarmonyEmulator( * one still booting, and would burn the whole budget before saying so. * * Exactly one *eligible* arrival, or none. Arrival alone does not identify - * anything once a second device can produce one: a handset plugged in or - * authorised inside the window, a row settling out of `Offline`, or a - * concurrent `boot-device` for another instance (`inFlightHarmonyBoots` keys on - * the instance, so those are not coalesced). Returning the wrong key is not a - * failed boot — it is a drivable id, so every later tap and keystroke lands on - * that device instead, which is the case for being strict here. USB arrivals - * are excluded outright (a host-local emulator is not on a cable) and two - * eligible arrivals are refused rather than guessed between, exactly as the - * Android path refuses when more than one serial appears. There is no - * `-avdName` equivalent to disambiguate with, so the caller gets the named - * no-target degradation and `list-devices`. + * anything once a second device can produce one, and returning the wrong key is + * not a failed boot — it is a drivable id, so every later tap and keystroke + * lands on that device instead. So USB arrivals are excluded outright (a + * host-local emulator is not on a cable) and two eligible arrivals are refused + * rather than guessed between, as the Android path refuses when more than one + * serial appears. There is no `-avdName` equivalent to disambiguate with, so the + * caller gets the named no-target degradation and `list-devices`. + * + * What this does NOT cover: a second emulator registering during the window but + * outside it — started from DevEco Studio, or by a concurrent `boot-device` for + * another instance, which `inFlightHarmonyBoots` does not coalesce. Both are + * loopback TCP and land seconds apart, so each boot sees a single arrival and + * the cardinality check never fires. Nothing in `hdc` or the manager ties a key + * to an instance, so closing that needs a lock across every harmony boot on the + * host, which would head-of-line block them all on one slow start. */ async function waitForHarmonyTarget( before: Set, @@ -1655,6 +1670,20 @@ async function waitForHarmonyTarget( } } +/** Resolve when the manager has exited, or when `graceMs` has passed. */ +async function waitForHarmonyExit( + emulator: { exited: () => unknown }, + graceMs: number +): Promise { + const deadline = Date.now() + graceMs; + for (;;) { + if (emulator.exited()) return; + const remaining = deadline - Date.now(); + if (remaining <= 0) return; + await new Promise((r) => setTimeout(r, Math.min(HARMONY_EXIT_POLL_MS, remaining))); + } +} + /** * Wait until the manager stops reporting the instance as running. * @@ -1758,11 +1787,21 @@ async function bootHarmonyImpl(params: { } }; - // Without a connector there is nothing to watch, so the manager is asked once - // instead: otherwise a start that had already failed would be reported as - // `booted: true` under a note whose first clause — "the instance started" — - // nothing had checked. - if (!hdcAvailable) assertEmulatorAlive(); + // Without a connector there is no target list to watch, but the manager can + // still be watched for a short grace: otherwise a start that fails — a missing + // image, an instance already running — is reported as `booted: true` under a + // note whose first clause, "the instance started", nothing has checked. + // + // Asking once is not enough. Nothing between the spawn and here does any I/O + // (`resolveHdc` is a warm memo by this point, primed by the pre-start + // snapshot), so a single check runs microtasks after the spawn, before the + // child could have exited. The grace is small and bounded because a failing + // `-start` prints and exits at once — this must not become the whole-budget + // wait the connector-absent path exists to avoid. + if (!hdcAvailable) { + await waitForHarmonyExit(emulator, HARMONY_NO_HDC_GRACE_MS); + assertEmulatorAlive(); + } const connectKey = hdcAvailable ? await waitForHarmonyTarget(before, bootDeadline, assertEmulatorAlive) diff --git a/packages/tool-server/test/boot-device-harmony.test.ts b/packages/tool-server/test/boot-device-harmony.test.ts index 2224e5922a..25269ea98c 100644 --- a/packages/tool-server/test/boot-device-harmony.test.ts +++ b/packages/tool-server/test/boot-device-harmony.test.ts @@ -246,10 +246,11 @@ describe("boot-device — HarmonyOS emulator path", () => { // The manager dies while the connector is being looked for, which is the // real ordering: `Emulator -start` fails in milliseconds, the `hdc` lookup // is a filesystem probe. - resolveHdc.mockImplementation(async () => { - child.die("Failed to start emulator: this emulator instance is already running"); - return null; - }); + resolveHdc.mockResolvedValue(null); + setTimeout( + () => child.die("Failed to start emulator: this emulator instance is already running"), + 5 + ); await expect(boot({})).rejects.toThrow(/already running/); }); diff --git a/packages/tool-server/test/harmony-layout-parser.test.ts b/packages/tool-server/test/harmony-layout-parser.test.ts index 7523aa35c0..b90a322190 100644 --- a/packages/tool-server/test/harmony-layout-parser.test.ts +++ b/packages/tool-server/test/harmony-layout-parser.test.ts @@ -5,6 +5,7 @@ import { parseHarmonyLayout, } from "../src/tools/describe/platforms/harmony/layout-parser"; import type { HarmonyLayoutNode } from "../src/utils/harmony-uitest"; +import type { DescribeNode } from "../src/tools/describe/contract"; const SCREEN = { width: 1216, height: 2688 }; @@ -388,27 +389,28 @@ describe("parseHarmonyLayout", () => { SCREEN ).tree.children[0].children[0].children; - it("gives a row scrolled fully off the screen no area, on every edge", () => { - const [above, below, right] = rows([ + it("drops a row scrolled fully off the screen, on every edge", () => { + // Clipping alone left these as zero-area `[clickable]` lines whose + // documented tap centre is the status bar or the nav bar. + const kept = rows([ ["ABOVE", "[0,-400][1216,-260]"], ["BELOW", "[0,2888][1216,3028]"], ["RIGHT", "[1266,400][1456,540]"], + ["VISIBLE", "[0,1000][1216,1140]"], ]); - expect(above.frame.height).toBe(0); - expect(below.frame.height).toBe(0); - expect(right.frame.width).toBe(0); + expect(kept.map((r) => r.label)).toEqual(["VISIBLE"]); }); it("keeps only the visible slice of a row straddling an edge", () => { - // 140px tall, 80px of it on screen — and it must NOT share a frame with - // the fully-off-screen row above it, which is what the old clamp produced. - const [above, straddle] = rows([ + // 140px tall, 80px of it on screen. It must survive - it IS reachable - + // and must not share the fully-off-screen row's fate or its old frame. + const kept = rows([ ["ABOVE", "[0,-400][1216,-260]"], ["STRADDLE", "[0,-60][1216,80]"], ]); - expect(straddle.frame.y).toBe(0); - expect(straddle.frame.height).toBeCloseTo(80 / 2688, 6); - expect(straddle.frame).not.toEqual(above.frame); + expect(kept.map((r) => r.label)).toEqual(["STRADDLE"]); + expect(kept[0].frame.y).toBe(0); + expect(kept[0].frame.height).toBeCloseTo(80 / 2688, 6); }); it("never lets a frame run past the screen, so a tap centre stays on it", () => { @@ -425,6 +427,50 @@ describe("parseHarmonyLayout", () => { } }); + it("drops an off-screen node sitting directly under a window", () => { + // Windows are assembled separately from the recursive walk, so the same + // pruning has to be applied there or it holds only below depth 1. + const tree = parseHarmonyLayout( + root([ + node({ type: "root", bundleName: "com.app", bounds: "[0,0][1216,2688]" }, [ + node({ + type: "Button", + text: "GONE", + bounds: "[0,-400][1216,-260]", + clickable: "true", + }), + node({ type: "Button", text: "HERE", bounds: "[0,100][1216,240]", clickable: "true" }), + ]), + ]), + SCREEN + ).tree; + expect(tree.children[0].children.map((c) => c.label)).toEqual(["HERE"]); + }); + + it("does not drop a zero-size container that still holds a visible child", () => { + // A degenerate wrapper survives `build` because it has children; pruning + // it on its own frame would take the reachable child with it. + const tree = parseHarmonyLayout( + root([ + node({ type: "root", bundleName: "com.app", bounds: "[0,0][1216,2688]" }, [ + node({ type: "Dialog", bounds: "[0,0][0,0]" }, [ + node({ type: "Text", text: "Inside", bounds: "[0,1000][1216,1140]" }), + ]), + ]), + ]), + SCREEN + ).tree; + // Assert the child survives ANYWHERE in the tree: pruning a zero-area + // ancestor takes its whole subtree with it, wherever it sits. + const labels: string[] = []; + const walk = (n: DescribeNode) => { + if (n.label) labels.push(n.label); + n.children.forEach(walk); + }; + walk(tree); + expect(labels).toContain("Inside"); + }); + it("leaves a fully on-screen row exactly as measured", () => { const [visible] = rows([["VISIBLE", "[0,1000][1216,1140]"]]); expect(visible.frame.y).toBeCloseTo(1000 / 2688, 6); From 66a26b1ac24f1087ac89b1e476ba2d3b853e7819 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Tue, 11 Aug 2026 23:33:21 +0200 Subject: [PATCH 025/110] test(harmony): make the new tests hermetic, leak-free and able to fail Four defects in this review round's own commits. `harmony-hdc-timeout.test.ts` stubbed only the PATH lookup, so on a macOS host with DevEco Studio installed `resolveHdc` finds the real `hdc` first and the test runs it - red against a warm daemon, vacuously green against a cold one. It now resolves through the documented `$DEVECO_STUDIO_HOME` layout to a stub it owns, which drops the command-on-path mock entirely. The stub also `exec`s its sleep: without that the trapped shell dies and orphans a `sleep 30` for its full duration, once per run. `button.test.ts` looked up each harmony button's key name in a table and compared - but a button added to the accepted set with no name yields `undefined` on both sides and matches. Adding `appSwitch` to `BUTTONS_BY_PLATFORM.harmony` alone was a green mutation, which is exactly the drift the loop claimed to catch. It now asserts the lookup hit first, and its docstring says what it actually does. The no-connector boot test spent 3s of an un-overridden 5s per-test budget in the grace wait, and the comment explaining why it needed no timer advance no longer described the path. Fake-timed instead. Also drops an absolute from `waitForHarmonyTarget`'s docstring that this PR's own `Emulator` fixture falsifies: the manager does report `hw.hdc.port`. It is not the port a started instance registers under, which is the point the sentence was making, so the reasoning stands - the claim as written did not. --- .../src/tools/devices/boot-device.ts | 7 ++-- .../test/boot-device-harmony.test.ts | 10 ++++-- packages/tool-server/test/button.test.ts | 12 ++++--- .../test/harmony-hdc-timeout.test.ts | 35 +++++++++++-------- 4 files changed, 38 insertions(+), 26 deletions(-) diff --git a/packages/tool-server/src/tools/devices/boot-device.ts b/packages/tool-server/src/tools/devices/boot-device.ts index 8049521e06..a925050391 100644 --- a/packages/tool-server/src/tools/devices/boot-device.ts +++ b/packages/tool-server/src/tools/devices/boot-device.ts @@ -1626,9 +1626,10 @@ async function startHarmonyEmulator( * The connect key of the target that appears after a start, or null if none * does before `deadline`. * - * Arrival, because nothing else joins the two: `Emulator` names instances and - * never reports a port, `hdc` names connect keys and never mentions an - * instance. It is also the one signal independent of how an emulator's key is + * Arrival, because nothing else joins the two: the `Emulator` manager reports a + * configured `hw.hdc.port` but not the one a started instance registers under + * (measured as `127.0.0.1:5555`, outside the range `-hdcPort` accepts), and + * `hdc` names connect keys and never mentions an instance. It is also the one signal independent of how an emulator's key is * spelled — measured as `127.0.0.1:5555`, which is neither the hardware-serial * shape a phone has nor inside the range `-hdcPort` accepts. * diff --git a/packages/tool-server/test/boot-device-harmony.test.ts b/packages/tool-server/test/boot-device-harmony.test.ts index 25269ea98c..453f7d078d 100644 --- a/packages/tool-server/test/boot-device-harmony.test.ts +++ b/packages/tool-server/test/boot-device-harmony.test.ts @@ -230,11 +230,15 @@ describe("boot-device — HarmonyOS emulator path", () => { it("blames the missing connector instead of waiting out the budget for it", async () => { resolveHdc.mockResolvedValue(null); + vi.useFakeTimers(); - const result = (await boot({})) as { udid: string; note?: string }; + const pending = boot({}); + // The arrival wait is skipped outright without `hdc`; what remains is the + // short grace that watches the manager for an immediate failure, so this + // advances past that rather than the boot budget. + await vi.advanceTimersByTimeAsync(4_000); + const result = (await pending) as { udid: string; note?: string }; - // No fake timers and no advance: without `hdc` the wait is skipped - // outright, so a partial DevEco install fails fast and says which half. expect(result.udid).toBe(`harmony-emulator-${INSTANCE}`); expect(result.note).toMatch(/`hdc` was not found/); }); diff --git a/packages/tool-server/test/button.test.ts b/packages/tool-server/test/button.test.ts index 7813255ef6..b5a2e1e872 100644 --- a/packages/tool-server/test/button.test.ts +++ b/packages/tool-server/test/button.test.ts @@ -46,10 +46,10 @@ const services = { simulatorServer: {} } as never; /** * The `uitest uiInput keyEvent` name each HarmonyOS button must reach the device * as, spelled out rather than derived, so a swapped pair (home ↔ back) fails. - * The loop below indexes this by every entry of `BUTTONS_BY_PLATFORM.harmony`, - * which is what keeps the tool's `HARMONY_BUTTON_KEYS[...]!` assertion honest: - * a button added to the accepted set without a key name to press turns this red - * instead of injecting `undefined`. + * The loop below indexes this by every entry of `BUTTONS_BY_PLATFORM.harmony` + * and asserts the lookup HIT before comparing - without that, a button added to + * the accepted set with no key name would inject `undefined` and still match, + * since the expectation would be `undefined` too. */ const HARMONY_KEY_NAMES: Record = { home: "Home", @@ -174,9 +174,11 @@ describe("button tool — per-platform validation", () => { await expect(buttonTool.execute(services, { udid: harmonyUdid, button })).resolves.toEqual({ pressed: button, }); + const expectedKey = HARMONY_KEY_NAMES[button]; + expect(expectedKey, `no uitest key name for "${button}"`).toBeTypeOf("string"); // The connect key, not the `harmony-` prefixed device id — `uitest` is // addressed by what `hdc list targets` reports. - expect(harmonyKeyEvent).toHaveBeenCalledWith(harmonyConnectKey, HARMONY_KEY_NAMES[button]); + expect(harmonyKeyEvent).toHaveBeenCalledWith(harmonyConnectKey, expectedKey); // hdc is preflighted so a missing connector fails with a 424 install hint. expect(ensureDep).toHaveBeenCalledWith("hdc"); // Neither of the other backends is touched — in particular the press must diff --git a/packages/tool-server/test/harmony-hdc-timeout.test.ts b/packages/tool-server/test/harmony-hdc-timeout.test.ts index 474882612a..8ff28179d8 100644 --- a/packages/tool-server/test/harmony-hdc-timeout.test.ts +++ b/packages/tool-server/test/harmony-hdc-timeout.test.ts @@ -1,22 +1,28 @@ -import { afterAll, describe, expect, it, vi } from "vitest"; -import { mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { afterAll, beforeAll, describe, expect, it, vi } from "vitest"; +import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; - -// Resolve `hdc` to a stub instead of a real toolchain. Its own module-level -// binary cache starts empty per test file, so this is the only seam needed. -vi.mock("../src/utils/command-on-path", () => ({ - commandOnPath: vi.fn(async () => stub), -})); - import { runHdc } from "../src/utils/harmony-hdc"; -const stubDir = mkdtempSync(join(tmpdir(), "argent-hdc-stub-")); -const stub = join(stubDir, "hdc"); -// A wedged `hdc`: blocked on an unresponsive daemon and deaf to SIGTERM. -writeFileSync(stub, "#!/usr/bin/env bash\ntrap '' TERM\nsleep 30\n", { mode: 0o755 }); +// Resolve `hdc` through the documented `$DEVECO_STUDIO_HOME` layout rather than +// PATH: pointing the resolver at a real stub is hermetic, where stubbing the +// PATH lookup alone still lets a host with DevEco Studio installed find and run +// the real `hdc`. +const root = mkdtempSync(join(tmpdir(), "argent-deveco-")); +const binDir = join(root, "sdk", "default", "openharmony", "toolchains"); +mkdirSync(binDir, { recursive: true }); +// A wedged `hdc`: blocked on an unresponsive daemon and deaf to SIGTERM. `exec` +// so the process that ignores the signal IS the one the timeout kills - without +// it the shell dies and leaves the `sleep` orphaned for its full duration. +writeFileSync(join(binDir, "hdc"), "#!/usr/bin/env bash\ntrap '' TERM\nexec sleep 30\n", { + mode: 0o755, +}); -afterAll(() => rmSync(stubDir, { recursive: true, force: true })); +beforeAll(() => vi.stubEnv("DEVECO_STUDIO_HOME", root)); +afterAll(() => { + vi.unstubAllEnvs(); + rmSync(root, { recursive: true, force: true }); +}); describe("runHdc timeout enforcement", () => { it("reaps a child that ignores SIGTERM, so the timeout it was given actually fires", async () => { @@ -25,7 +31,6 @@ describe("runHdc timeout enforcement", () => { // the HarmonyOS path is advisory - only `list-devices` has a deadline behind // it, and the interaction tools have nothing. Measured against this stub // without `killSignal`: still running 6s after a 1s timeout. - vi.stubEnv("DEVECO_STUDIO_HOME", ""); const started = Date.now(); await expect(runHdc(["list", "targets"], 800)).rejects.toThrow(); From a3a5d78ffd976de86fe6e21ff498970d28a9e25d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Wed, 12 Aug 2026 14:07:01 +0200 Subject: [PATCH 026/110] fix(harmony): let a force restart outlast the shutdown it asked for `-stop` returns as soon as it has asked, and how long the instance then takes to actually go down is unpredictable: 9s, 11s, 22s and ~70s across measured runs, plus one that had still not gone down three minutes later. The 30s grace sat inside that spread, so a restart of a long-lived instance gave up early and called `-start` into "this emulator instance is already running". The wait now spans the boot budget, since an instance still up leaves nothing worth spending the rest on, and a stop that never took effect is reported rather than walked into. The HarmonyOS default rises to 3 min to cover a shutdown and a boot in the one budget. --- .../src/tools/devices/boot-device.ts | 42 +++++++++++-------- .../test/boot-device-harmony.test.ts | 30 ++++++++++--- 2 files changed, 50 insertions(+), 22 deletions(-) diff --git a/packages/tool-server/src/tools/devices/boot-device.ts b/packages/tool-server/src/tools/devices/boot-device.ts index a925050391..bcef8e16ae 100644 --- a/packages/tool-server/src/tools/devices/boot-device.ts +++ b/packages/tool-server/src/tools/devices/boot-device.ts @@ -103,7 +103,7 @@ const zodSchema = z.object({ .max(900_000) .optional() .describe( - "Android/Vega/HarmonyOS: overall budget for the boot sequence. Default 480000 (8 min) on Android, 120000 (2 min) on Vega and HarmonyOS. Clamped to [30s, 15min]. Ignored on iOS." + "Android/Vega/HarmonyOS: overall budget for the boot sequence. Default 480000 (8 min) on Android, 120000 (2 min) on Vega, 180000 (3 min) on HarmonyOS, where with `force` it spans the shutdown as well as the boot. Clamped to [30s, 15min]. Ignored on iOS." ), force: z .boolean() @@ -1465,7 +1465,14 @@ type HarmonyBootResult = { // must not both shell out `Emulator -start` for the same instance. const inFlightHarmonyBoots = new Map>(); -const HARMONY_BOOT_TIMEOUT_MS = 120_000; +/** + * Sized for a `force` restart, which spends one budget on a shutdown and a boot. + * The start reaches a connected target in 8-20s; the shutdown is the + * unpredictable half, since `-stop` returns as soon as it has asked and the + * instance then took anywhere from 9s to ~70s to actually go down across + * measured runs — and once had not gone down three minutes later. + */ +const HARMONY_BOOT_TIMEOUT_MS = 180_000; /** * Each poll spawns `hdc` and round-trips its daemon, an order of magnitude @@ -1474,13 +1481,6 @@ const HARMONY_BOOT_TIMEOUT_MS = 120_000; */ const HARMONY_TARGET_POLL_MS = 2_000; -/** - * How long a `-stop` gets to actually bring the instance down before the - * restart proceeds anyway. Giving up early costs a start that reports the - * instance is still running, not a corrupted one. - */ -const HARMONY_STOP_GRACE_MS = 30_000; - /** * How long the manager is watched for an immediate failure when there is no * `hdc` to wait on a target with. A failing `-start` prints its diagnostic and @@ -1686,7 +1686,8 @@ async function waitForHarmonyExit( } /** - * Wait until the manager stops reporting the instance as running. + * Wait until the manager stops reporting the instance as running, answering + * whether it did. * * `-stop` returns as soon as it has asked, not once the emulator is gone, and * the guest's `hdc` endpoint dies well before the process does — so a restart @@ -1695,12 +1696,12 @@ async function waitForHarmonyExit( * tracks the process itself. An unreadable listing answers nothing, so it keeps * waiting rather than assuming the instance is gone. */ -async function waitForHarmonyInstanceStopped(name: string, deadline: number): Promise { +async function waitForHarmonyInstanceStopped(name: string, deadline: number): Promise { for (;;) { const instances = await listHarmonyInstances().catch(() => null); - if (instances && !instances.some((i) => i.name === name && i.running)) return; + if (instances && !instances.some((i) => i.name === name && i.running)) return true; const remaining = deadline - Date.now(); - if (remaining <= 0) return; + if (remaining <= 0) return false; await new Promise((r) => setTimeout(r, Math.min(HARMONY_TARGET_POLL_MS, remaining))); } } @@ -1738,10 +1739,17 @@ async function bootHarmonyImpl(params: { `Failed to stop HarmonyOS emulator "${params.instanceName}" before restarting it: ${stopDiagnostic}` ); } - await waitForHarmonyInstanceStopped( - params.instanceName, - Math.min(Date.now() + HARMONY_STOP_GRACE_MS, bootDeadline) - ); + // The whole remaining budget rather than a slice of it: an instance still up + // leaves nothing worth spending the rest on, since starting it can only + // report that it is already running. + if (!(await waitForHarmonyInstanceStopped(params.instanceName, bootDeadline))) { + throw new Error( + `HarmonyOS emulator "${params.instanceName}" was still running when the ` + + `${Math.round(params.bootTimeoutMs / 1_000)}s budget ran out after \`-stop\`, so ` + + "starting it now would only report that it is already running. Re-run with a larger " + + "bootTimeoutMs, or stop the instance from DevEco Studio." + ); + } } // Snapshot immediately before the start, so every target that was already diff --git a/packages/tool-server/test/boot-device-harmony.test.ts b/packages/tool-server/test/boot-device-harmony.test.ts index 453f7d078d..aa1e64534d 100644 --- a/packages/tool-server/test/boot-device-harmony.test.ts +++ b/packages/tool-server/test/boot-device-harmony.test.ts @@ -328,20 +328,22 @@ describe("boot-device — HarmonyOS emulator path", () => { it("restarts a running instance on force and resolves the key it comes back on", async () => { // `-stop` returns before the emulator is gone, so the restart waits on - // `isRunning`: reported running for one poll after the stop, then down. The - // snapshot is taken only once it is, which is what lets a restart that - // lands on the port it had before still read as an arrival. + // `isRunning` — and for the whole budget, since how long the instance then + // takes to go down is unpredictable (9s to ~70s measured). Reported running + // for 40s here, past any fixed grace. The snapshot is taken only once it is + // down, which is what lets a restart that lands on the port it had before + // still read as an arrival. let listed = 0; listHarmonyInstances.mockImplementation(() => Promise.resolve([ - { name: INSTANCE, deviceType: "Phone", osVersion: null, running: listed++ < 2 }, + { name: INSTANCE, deviceType: "Phone", osVersion: null, running: listed++ < 21 }, ]) ); targets([PHONE], [PHONE], [PHONE, emulatorTarget]); vi.useFakeTimers(); const pending = boot({ force: true, bootTimeoutMs: 120_000 }); - await vi.advanceTimersByTimeAsync(10_000); + await vi.advanceTimersByTimeAsync(60_000); const result = (await pending) as { udid: string; note?: string }; expect(runHarmonyEmulator).toHaveBeenCalledWith(["-stop", INSTANCE]); @@ -367,6 +369,24 @@ describe("boot-device — HarmonyOS emulator path", () => { expect(spawnMock).not.toHaveBeenCalled(); }); + it("does not start an instance the stop never brought down", async () => { + // One measured `-stop` had still not taken effect three minutes later, so + // the instance being down is never assumable — and one still up when the + // budget ends would make `-start` report only that it is already running. + listHarmonyInstances.mockResolvedValue([ + { name: INSTANCE, deviceType: "Phone", osVersion: null, running: true }, + ]); + vi.useFakeTimers(); + + const pending = boot({ force: true, bootTimeoutMs: 60_000 }); + const settled = expect(pending).rejects.toThrow(/still running when the 60s budget ran out/); + await vi.advanceTimersByTimeAsync(61_000); + await settled; + + expect(runHarmonyEmulator).toHaveBeenCalledWith(["-stop", INSTANCE]); + expect(spawnMock).not.toHaveBeenCalled(); + }); + it("returns an id the interaction tools accept, which the instance id is not", async () => { targets([PHONE], [PHONE, emulatorTarget]); From 508bafa9d5cd1e49befbff113f4106ad1e214a95 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Wed, 12 Aug 2026 14:11:30 +0200 Subject: [PATCH 027/110] docs(harmony): state the measured connect key once, not twice --- packages/tool-server/src/tools/devices/boot-device.ts | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/packages/tool-server/src/tools/devices/boot-device.ts b/packages/tool-server/src/tools/devices/boot-device.ts index bcef8e16ae..69706b4bc0 100644 --- a/packages/tool-server/src/tools/devices/boot-device.ts +++ b/packages/tool-server/src/tools/devices/boot-device.ts @@ -1627,11 +1627,11 @@ async function startHarmonyEmulator( * does before `deadline`. * * Arrival, because nothing else joins the two: the `Emulator` manager reports a - * configured `hw.hdc.port` but not the one a started instance registers under - * (measured as `127.0.0.1:5555`, outside the range `-hdcPort` accepts), and - * `hdc` names connect keys and never mentions an instance. It is also the one signal independent of how an emulator's key is - * spelled — measured as `127.0.0.1:5555`, which is neither the hardware-serial - * shape a phone has nor inside the range `-hdcPort` accepts. + * configured `hw.hdc.port` but not the one a started instance registers under, + * and `hdc` names connect keys and never mentions an instance. It is also the + * one signal independent of how an emulator's key is spelled — measured as + * `127.0.0.1:5555`, which is neither the hardware-serial shape a phone has nor + * inside the range `-hdcPort` accepts. * * `checkAlive` is polled alongside, and throws if the emulator died: without it * a start that failed after the spawn returned would be indistinguishable from From 4ade1c2ce6053d59295f3e87522164e7d25c2cda Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Thu, 13 Aug 2026 14:47:59 +0200 Subject: [PATCH 028/110] fix(harmony): close the device-found gaps the review round proved MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Eight defects the hubgan device pass reproduced, each confirmed against the branch, fixed, and re-run after: - describe's window label carried the bundle id, which the shared matcher reads as visible text: exists:{text:"sceneboard"} was true on every screen and a text wait for an app's own name tied with its title in reading order and lost. The bundle now rides as the window's identifier, not its label. - describe replaced every UIExtensionComponent with an opaque placeholder. On 6.1.1 the same node arrives WITH its subtree (the HUAWEI ID login sheet's TextInput and both Buttons, 45 nodes, all live) and screenCap does not capture the surface — so the placeholder discarded the only controls that dismiss it. A childless one is still labelled; a subtree is parsed. - boot-device snapshot only Connected rows, so a target listed Offline at snapshot read as an arrival when it reconnected mid-boot — adopting a device the call did not start (measured: handed back a wearable). The snapshot is now every listed row, and only a FRESH registration counts. A two-arrival refusal now carries its own note rather than blaming the boot budget. - gesture-tap / gesture-swipe / keyboard reported success against a suspended panel (uitest answers No Error to every touch, which lands nowhere). All three now refuse with an actionable error instead. - a successful await-ui-element against the last-composited frame carried no hint, so a wait-then-tap loop ran to completion against a dead screen. - uitest does not tolerate overlapping invocations; the harmony path now serialises them per connect key instead of racing into a 20s SIGKILL. - screenshot / screenshot-diff rejected a rotation argument as an unsupported-TOOL error; both now throw the per-argument InvalidToolInputError the keyboard key guard uses. - restart-app swallowed terminateHarmonyApp outright, reporting restarted:true for an app it never stopped; it now tolerates only the app-not-running diagnostic and lets every other refusal propagate. - keyboard typed unchecked text and counted keys whatever landed; it now rejects newlines and control characters up front, like every sibling. - the ability resolver matched only the Calculator direction; Photos spells it the other way (short mainAbility, fully-qualified entry) and empty-string mainAbility (14/73 bundles) went out as aa start -a '' — an implicit start plus a stuck modal. Match is now bidirectional on a dot boundary, and "" is read as no launchable ability. - await-screen-idle reported a device that went away mid-wait as a screen that never settled; the result now names the last fetch failure. Plus the four cannot-fail tests the same pass named (the Connected filter, the two-arrival note, the listing-failure lookahead, and the hdc timeout matcher), each hardened and re-verified against its mutation; the list-devices harmony branch, viaDeviceTmp cleanup/unique-path/host-rm, and the resolver branches now have their pins; and the docs enumerations that still read as though HarmonyOS did not exist (SKILL.md key list and id shapes, argent.md's router, the README uitest claim and refusal quote, Telemetry.md's platform disclosure, keyboard's key schema, swipe's durationMs clamp). --- README.md | 2 +- Telemetry.md | 2 +- packages/skills/rules/argent.md | 2 +- .../skills/argent-device-interact/SKILL.md | 4 +- .../src/tools/await-screen-idle/index.ts | 16 +- .../src/tools/await-ui-element/index.ts | 6 + .../platforms/harmony/layout-parser.ts | 38 ++-- .../src/tools/devices/boot-device.ts | 64 +++++-- .../src/tools/gesture-swipe/index.ts | 27 ++- .../src/tools/gesture-tap/index.ts | 10 +- .../tool-server/src/tools/keyboard/index.ts | 2 +- .../src/tools/keyboard/platforms/harmony.ts | 47 ++++- .../tools/restart-app/platforms/harmony.ts | 22 ++- .../src/tools/screenshot-diff/index.ts | 16 +- .../tool-server/src/tools/screenshot/index.ts | 22 ++- .../tool-server/src/utils/harmony-apps.ts | 108 +++++++---- .../tool-server/src/utils/harmony-uitest.ts | 92 +++++++--- .../test/boot-device-harmony.test.ts | 25 ++- .../tool-server/test/harmony-apps.test.ts | 65 +++++++ .../test/harmony-hdc-timeout.test.ts | 17 +- .../test/harmony-layout-parser.test.ts | 89 ++++++++- .../tool-server/test/harmony-screen.test.ts | 109 +++++++++++ .../test/keyboard-error-taxonomy.test.ts | 26 +++ .../test/keyboard-key-order.test.ts | 3 + .../test/list-devices-harmony.test.ts | 170 ++++++++++++++++++ 25 files changed, 851 insertions(+), 133 deletions(-) create mode 100644 packages/tool-server/test/harmony-screen.test.ts create mode 100644 packages/tool-server/test/list-devices-harmony.test.ts diff --git a/README.md b/README.md index 03724d8427..7c9724bdfb 100644 --- a/README.md +++ b/README.md @@ -29,7 +29,7 @@ Argent drives a growing set of targets through a single toolkit, each with the r | **Desktop & web** | Electron and Chromium apps (incl. React Native Web / Expo web) over CDP | Mouse / keyboard | | **HarmonyOS** | Phones over `hdc`, and DevEco Studio emulator instances | Touch / gesture | -> **HarmonyOS drives whatever `hdc` is connected to.** Connect a phone with USB debugging enabled and it appears as `harmony-` (`kind: "device"`), driven by `describe`, `screenshot`, `screenshot-diff`, `gesture-tap`, `gesture-swipe`, `keyboard`, `button`, `launch-app`, `restart-app`, `open-url`, `run-sequence` and the two `await-*` tools - all through the device's own `uitest`. DevEco Studio's emulator instances appear separately as `harmony-emulator-`, which `boot-device` starts; it then waits for the instance to register with `hdc` and returns that connect key, so a booted emulator is driven by the same tools through the same id shape as a phone. Huawei restricts the emulator **image** download to mainland China, so on a host outside it no instance can be created until an image is obtained. Multi-touch and free-form touch (`gesture-pinch`, `gesture-rotate`, `gesture-custom`) are refused at the capability gate with `Tool '' is not supported on harmony device`, on both device kinds: `uitest uiInput` injects only whole gestures (click / longClick / swipe / drag / fling), one contact at a time, and the level below is shut - `hdc shell` is uid 2000 and `/dev/input` is `Permission denied` on a retail phone and on Huawei's own emulator image alike, where `hdc smode` answers `Cannot set root run mode in undebuggable version`. `rotate`, the profilers and the debugger are refused the same way: the device exposes no orientation CLI, and argent's profiler and debugger surfaces have no HarmonyOS ingest path. +> **HarmonyOS drives whatever `hdc` is connected to.** Connect a phone with USB debugging enabled and it appears as `harmony-` (`kind: "device"`), driven by `describe`, `screenshot`, `screenshot-diff`, `gesture-tap`, `gesture-swipe`, `keyboard`, `button`, `launch-app`, `restart-app`, `open-url`, `run-sequence` and the two `await-*` tools - all through the device's own `uitest`, except `launch-app`, `restart-app` and `open-url`, which go through `bm dump` + `aa start` / `aa force-stop`. DevEco Studio's emulator instances appear separately as `harmony-emulator-`, which `boot-device` starts; it then waits for the instance to register with `hdc` and returns that connect key, so a booted emulator is driven by the same tools through the same id shape as a phone. Huawei restricts the emulator **image** download to mainland China, so on a host outside it no instance can be created until an image is obtained. Multi-touch and free-form touch (`gesture-pinch`, `gesture-rotate`, `gesture-custom`) are refused at the capability gate on both device kinds - `Tool '' is not supported on harmony device (no harmony support declared).` on a phone, and the same message with `harmony emulator` in place of `harmony device` on an emulator id: `uitest uiInput` injects only whole gestures (click / longClick / swipe / drag / fling), one contact at a time, and the level below is shut - `hdc shell` is uid 2000 and `/dev/input` is `Permission denied` on a retail phone and on Huawei's own emulator image alike, where `hdc smode` answers `Cannot set root run mode in undebuggable version`. `rotate`, the profilers and the debugger are refused the same way: the device exposes no orientation CLI, and argent's profiler and debugger surfaces have no HarmonyOS ingest path. ## Capabilities diff --git a/Telemetry.md b/Telemetry.md index fd5145e300..0ee6f808e9 100644 --- a/Telemetry.md +++ b/Telemetry.md @@ -63,7 +63,7 @@ When you use the Argent Lens design-review flow (previewing and choosing between - Argent version, Node.js version, operating system, processor architecture; - whether the process runs in an interactive terminal and whether it runs in a CI environment; -- whether Argent is used in connection with Android, iOS, tvOS (Apple TV), Android TV, VegaOS, or a Chromium-based target. +- whether Argent is used in connection with Android, iOS, tvOS (Apple TV), Android TV, VegaOS, HarmonyOS, or a Chromium-based target. ### Diagnostics diff --git a/packages/skills/rules/argent.md b/packages/skills/rules/argent.md index 143bfd8b3d..60b52779cf 100644 --- a/packages/skills/rules/argent.md +++ b/packages/skills/rules/argent.md @@ -114,7 +114,7 @@ When: Beginning a task that involves the Android emulator, no emulator running y TAPPING, SWIPING, TYPING, GESTURES, SCREENSHOTS, SCROLLING Skill: `argent-device-interact` -When: Performing touch interactions, typing, pressing hardware buttons, launching/restarting apps, opening URLs, rotating device, taking standalone screenshots, or verifying a visible UI code change. Phone/tablet iOS and Android only — for any TV target use the TV skill below. +When: Performing touch interactions, typing, pressing hardware buttons, launching/restarting apps, opening URLs, rotating device, taking standalone screenshots, or verifying a visible UI code change. Phone/tablet iOS, Android and HarmonyOS only — for any TV target use the TV skill below. APP PERMISSIONS (GRANT / DENY / RESET WITHOUT THE SETTINGS UI) Skill: `argent-settings-permissions` diff --git a/packages/skills/skills/argent-device-interact/SKILL.md b/packages/skills/skills/argent-device-interact/SKILL.md index 5de7928dcc..59de58ffca 100644 --- a/packages/skills/skills/argent-device-interact/SKILL.md +++ b/packages/skills/skills/argent-device-interact/SKILL.md @@ -5,7 +5,7 @@ description: Interact with an iOS simulator, Android emulator, HarmonyOS device, ## Unified tool surface -All interaction tools below accept a `udid` parameter and auto-dispatch iOS vs Android based on its shape (UUID → iOS simulator, `chromium-cdp-` → Chromium (CDP) app, anything else → Android adb serial). You use the same tool names on every platform. +All interaction tools below accept a `udid` parameter and auto-dispatch iOS vs Android based on its shape (UUID → iOS simulator, `chromium-cdp-` → Chromium (CDP) app, `harmony-` → HarmonyOS device, `harmony-emulator-` → a boot target, anything else → Android adb serial). You use the same tool names on every platform. **Chromium (CDP) app** = any Chromium runtime exposing a Chrome DevTools Protocol endpoint: an Electron app (boot it with `boot-device` + `electronAppPath`), or any Chromium-family browser (Chrome/Brave/Edge) launched with `--remote-debugging-port`. The latter is auto-discovered by `list-devices` on port `9222` plus anything in `ARGENT_CHROMIUM_PORTS`. The same describe/tap/swipe/keyboard/screenshot surface drives all of them. @@ -170,7 +170,7 @@ Values: `home`, `back`, `power`, `volumeUp`, `volumeDown`, `appSwitch`, `actionB { "udid": "", "text": "search query", "key": "enter" } ``` -Special keys: `enter`, `escape`, `backspace`, `tab`, `space`, `arrow-up`, `arrow-down`, `arrow-left`, `arrow-right`, `f1`–`f12`. Optional: `"delayMs": 100` between keystrokes (default 50ms) — applies to the iOS simulator and Chromium; it is ignored on Android phones/tablets (typed via `adb input text`, no per-key cadence), on HarmonyOS (`uitest uiInput text` types in one shot), on Vega, and on TV targets. +Special keys: `enter`, `escape`, `backspace`, `tab`, `space`, `arrow-up`, `arrow-down`, `arrow-left`, `arrow-right`, `f1`–`f12` — on HarmonyOS only `enter`, `backspace`, `space`, `arrow-left` and `arrow-right` exist; any other key is refused with 400 KEYBOARD_KEY_UNSUPPORTED. Optional: `"delayMs": 100` between keystrokes (default 50ms) — applies to the iOS simulator and Chromium; it is ignored on Android phones/tablets (typed via `adb input text`, no per-key cadence), on HarmonyOS (`uitest uiInput text` types in one shot), on Vega, and on TV targets. **Typing secrets.** To enter a credential without its plaintext ever entering your context, transcript, or logs, use a secret placeholder in `text` (works in `keyboard`, `paste`, `run-sequence` keyboard steps, and flow `type` steps): diff --git a/packages/tool-server/src/tools/await-screen-idle/index.ts b/packages/tool-server/src/tools/await-screen-idle/index.ts index 3b6c2c80dd..73ff9f56b9 100644 --- a/packages/tool-server/src/tools/await-screen-idle/index.ts +++ b/packages/tool-server/src/tools/await-screen-idle/index.ts @@ -69,6 +69,8 @@ interface IdleResult { waitedMs: number; /** Number of tree reads taken. */ polls: number; + /** Why a false `settled` is not a screen that stayed busy — e.g. the device went away. */ + note?: string; } const capability: ToolCapability = { @@ -187,7 +189,19 @@ still before the timeout. Use after a launch/navigation to wait for the UI to re }, }); - return { settled: poll.result === true, waitedMs: poll.elapsedMs, polls: poll.polls }; + return { + settled: poll.result === true, + waitedMs: poll.elapsedMs, + polls: poll.polls, + // A device that goes away mid-wait must not read as a screen that never + // settled: the two are opposite diagnoses (one is the target's fault, + // the other the app's), and on HarmonyOS each read is several `hdc` + // round trips to a device that can drop out. `await-ui-element` folds + // the same `lastError` into its note. + ...(poll.result !== true && poll.lastError + ? { note: `last tree fetch failed: ${poll.lastError}` } + : {}), + }; }, }; } diff --git a/packages/tool-server/src/tools/await-ui-element/index.ts b/packages/tool-server/src/tools/await-ui-element/index.ts index 1af083c577..322233e165 100644 --- a/packages/tool-server/src/tools/await-ui-element/index.ts +++ b/packages/tool-server/src/tools/await-ui-element/index.ts @@ -350,6 +350,12 @@ or before tapping an element that appears asynchronously.`, result.note = "condition met immediately — the selector never matched any element, " + "so it may have already been hidden before the wait, or the selector is wrong"; + } else if (data.hint) { + // A successful wait against a degraded read still owes the agent the + // caveat: on HarmonyOS the tree is the last-composited frame while + // the panel is off, so a `visible` that resolved here is followed by + // a tap that lands nowhere unless the screen is woken first. + result.note = data.hint; } return { done: true, result }; } diff --git a/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts b/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts index 9d787e0c66..4edb0b1618 100644 --- a/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts +++ b/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts @@ -22,11 +22,14 @@ import { clipBoundsToScreen } from "../android/uiautomator-parser"; * - **Windows are top-level children**, each carrying `bundleName` / * `abilityName`, so which app owns which subtree is knowable and worth * surfacing. - * - **`UIExtensionComponent` subtrees are opaque.** System extension UI — the - * "open with" app selector, share sheets — renders in another process, and - * the dump contains the node with no children. Silently emitting an empty - * container would tell an agent the screen is empty when it is covered by a - * dialog it cannot see, so these are labelled. + * - **`UIExtensionComponent` subtrees are USUALLY opaque — but not always.** + * System extension UI (the "open with" app selector, share sheets) renders in + * another process and the dump carries the node with no children; those are + * labelled so the emptiness reads as a fact. But on HarmonyOS 6.1.1 the same + * node can arrive with the FULL subtree — the HUAWEI ID login sheet carries + * its TextInput and both Buttons (45 nodes measured), and those targets are + * live (their geometry taps). Children win: a subtree that exists is parsed + * normally, and only a genuinely childless one is labelled opaque. */ /** Bounds arrive as `[left,top][right,bottom]`, in device pixels. */ @@ -77,9 +80,10 @@ const LAYOUT_CONTAINERS = new Set([ const DECORATIONS = new Set(["Divider", "ScrollBar", "metaballNode"]); /** - * Renders in another process, so the dump carries the node and none of its - * content. Kept, and labelled, so the emptiness is visible as a fact rather - * than as an absence. + * Renders in another process, so the dump USUALLY carries the node and none of + * its content. A childless one is kept and labelled, so the emptiness is + * visible as a fact rather than as an absence. One that arrives WITH children + * (measured on HarmonyOS 6.1.1) is parsed normally — see the header. */ const OPAQUE_EXTENSION = "UIExtensionComponent"; @@ -150,7 +154,12 @@ function build(node: HarmonyLayoutNode): HarmonyNode[] { if (DECORATIONS.has(type)) return []; - if (type === OPAQUE_EXTENSION) { + // Only a genuinely childless extension is opaque. One carrying a subtree is + // real, live UI (its Buttons tap), so it falls through and is parsed like + // anything else — replacing it with the placeholder would discard the only + // controls that dismiss the sheet, and `uitest screenCap` does not capture + // the surface either, so the agent would have nothing. + if (type === OPAQUE_EXTENSION && children.length === 0) { return [ { ...blank(attrs, bounds), @@ -391,9 +400,14 @@ export function parseHarmonyLayout( ? { width: rootBounds.w, height: rootBounds.h } : fallback; - // Each top-level child is a window. Label it with the app that owns it so a + // Each top-level child is a window. Tag it with the app that owns it so a // tree spanning an app plus the status bar (always a separate `sceneboard` - // window) is readable rather than two anonymous stacks. + // window) is readable rather than two anonymous stacks. The bundle goes in + // `identifier`, NOT `label`: the shared matcher treats `label` as the + // element's visible text, and a full-screen node "named" `com.huawei.hmos.*` + // is a phantom text node — it ties with the app's real title in reading + // order (poisoning a `text` wait for the app's own name), and it makes + // `exists: {text: "sceneboard"}` true on every screen. const windows: DescribeNode[] = []; for (const child of root.children ?? []) { const attrs = child.attributes ?? {}; @@ -408,7 +422,7 @@ export function parseHarmonyLayout( } windows.push({ role: "Window", - label: bundle, + identifier: bundle, frame: unionFrame(built), children: built, }); diff --git a/packages/tool-server/src/tools/devices/boot-device.ts b/packages/tool-server/src/tools/devices/boot-device.ts index 69706b4bc0..cf4b020133 100644 --- a/packages/tool-server/src/tools/devices/boot-device.ts +++ b/packages/tool-server/src/tools/devices/boot-device.ts @@ -1533,6 +1533,17 @@ const HARMONY_NO_TARGET = "`bootTimeoutMs`, or call `list-devices` once it finishes booting and drive the `kind: " + '"device"` entry that appears.'; +/** + * Said when more than one eligible target registered inside the boot window, so + * arrival cannot pick the instance out. Distinct from {@link HARMONY_NO_TARGET}: + * the budget is not the problem — both targets DID register — and pointing the + * caller at `bootTimeoutMs` would misdiagnose a refusal to guess. + */ +const HARMONY_AMBIGUOUS_TARGET = + "More than one `hdc` target registered while the instance was booting, so argent could not " + + "tell which is the instance it started and did not guess. Call `list-devices` and drive the " + + '`kind: "device"` entry for the instance, or stop the other emulator and re-run.'; + /** Said when the instance started but the connector that would reach it is missing. */ const HARMONY_NO_HDC = "The instance started, but `hdc` was not found, so argent cannot tell which target it " + @@ -1545,9 +1556,20 @@ async function connectedHarmonyTargets() { return targets.filter((t) => t.state === "Connected"); } -/** The connect keys `hdc` can drive right now. */ -async function connectedHarmonyKeys(): Promise> { - return new Set((await connectedHarmonyTargets()).map((t) => t.connectKey)); +/** + * Every connect key `hdc` lists, in ANY state. + * + * The pre-start snapshot must be taken against all rows, not only `Connected` + * ones: a target that is listed but `Offline` when the snapshot is taken reads + * as an arrival the moment it reconnects — and `Offline` is what a still- + * booting instance shows, what a stopped one leaves behind, and what any + * connection blip produces (a row was observed flipping back to `Connected` + * unattended when its guest rebooted mid-session). Adopting that reconnection + * hands back a device this call did not start. + */ +async function knownHarmonyKeys(): Promise> { + const targets = await listHarmonyHdcTargets().catch(() => []); + return new Set(targets.map((t) => t.connectKey)); } /** @@ -1658,15 +1680,19 @@ async function waitForHarmonyTarget( before: Set, deadline: number, checkAlive: () => void -): Promise { +): Promise<{ key: string | null; ambiguous: boolean }> { + let ambiguous = false; for (;;) { const arrived = (await connectedHarmonyTargets()).filter( (t) => !before.has(t.connectKey) && couldBeHarmonyEmulator(t) ); - if (arrived.length === 1) return arrived[0]!.connectKey; + if (arrived.length === 1) return { key: arrived[0]!.connectKey, ambiguous: false }; + // Two eligible fresh arrivals at once is a refusal to guess, not a slow + // boot — remember it so the caller can say so instead of blaming the budget. + if (arrived.length > 1) ambiguous = true; checkAlive(); const remaining = deadline - Date.now(); - if (remaining <= 0) return null; + if (remaining <= 0) return { key: null, ambiguous }; await new Promise((r) => setTimeout(r, Math.min(HARMONY_TARGET_POLL_MS, remaining))); } } @@ -1752,10 +1778,13 @@ async function bootHarmonyImpl(params: { } } - // Snapshot immediately before the start, so every target that was already - // being driven — another emulator, a phone on USB — is excluded by identity - // rather than by any assumption about how an emulator's key is spelled. - const before = await connectedHarmonyKeys(); + // Snapshot immediately before the start, against EVERY row `hdc` lists — + // Connected or not — so a target that was already known (another emulator, a + // phone on USB, a stale Offline row) is excluded by identity rather than by + // any assumption about how an emulator's key is spelled. Only a target that + // registers FRESH after the start can be the instance just booted; a row that + // was already listed and merely reconnects is not an arrival. + const before = await knownHarmonyKeys(); const emulator = await startHarmonyEmulator(params.instanceName); @@ -1812,16 +1841,25 @@ async function bootHarmonyImpl(params: { assertEmulatorAlive(); } - const connectKey = hdcAvailable + const outcome = hdcAvailable ? await waitForHarmonyTarget(before, bootDeadline, assertEmulatorAlive) - : null; + : { key: null, ambiguous: false }; + const connectKey = outcome.key; return { platform: "harmony", udid: connectKey ? harmonyDeviceId(connectKey) : harmonyEmulatorId(params.instanceName), instanceName: params.instanceName, booted: true, - ...(connectKey ? {} : { note: hdcAvailable ? HARMONY_NO_TARGET : HARMONY_NO_HDC }), + ...(connectKey + ? {} + : { + note: !hdcAvailable + ? HARMONY_NO_HDC + : outcome.ambiguous + ? HARMONY_AMBIGUOUS_TARGET + : HARMONY_NO_TARGET, + }), }; } diff --git a/packages/tool-server/src/tools/gesture-swipe/index.ts b/packages/tool-server/src/tools/gesture-swipe/index.ts index 5c450f7c6f..929be3638d 100644 --- a/packages/tool-server/src/tools/gesture-swipe/index.ts +++ b/packages/tool-server/src/tools/gesture-swipe/index.ts @@ -2,7 +2,12 @@ import { z } from "zod"; import type { ServiceRef, ToolCapability, ToolDefinition } from "@argent/registry"; import { simulatorServerRef, type SimulatorServerApi } from "../../blueprints/simulator-server"; import { resolveDevice, harmonyConnectKey } from "../../utils/device-info"; -import { harmonySwipeNormalized } from "../../utils/harmony-uitest"; +import { + assertHarmonyScreenAwake, + harmonyDisplay, + harmonySwipe, + toDevicePoint, +} from "../../utils/harmony-uitest"; import { ensureDep } from "../../utils/check-deps"; import { sendCommand } from "../../utils/simulator-client"; @@ -25,7 +30,9 @@ const zodSchema = z.object({ durationMs: z .number() .optional() - .describe("Total gesture duration in milliseconds (default 300)"), + .describe( + "Total gesture duration in milliseconds (default 300). On HarmonyOS this is converted to a `uitest` velocity and clamped to the 200–40000 range the binary accepts, so a very slow or very fast request lands at the nearest velocity `uitest` will take rather than the exact duration asked for." + ), settle: z .boolean() .optional() @@ -87,13 +94,15 @@ Pass settle:true for a momentum-free swipe that lands exactly where the finger l // there is no per-frame Move train to emit here. if (device.platform === "harmony") { await ensureDep("hdc"); - await harmonySwipeNormalized( - harmonyConnectKey(device.id), - { x: params.fromX, y: params.fromY }, - { x: params.toX, y: params.toY }, - duration, - settle - ); + const connectKey = harmonyConnectKey(device.id); + const display = await harmonyDisplay(connectKey); + // A swipe against a suspended panel reports `No Error` and lands nowhere. + assertHarmonyScreenAwake(display, "swipe"); + const fromPx = toDevicePoint(params.fromX, params.fromY, display); + const toPx = toDevicePoint(params.toX, params.toY, display); + const distance = Math.hypot(toPx.x - fromPx.x, toPx.y - fromPx.y); + const seconds = Math.max(duration, 1) / 1000; + await harmonySwipe(connectKey, settle ? "swipe" : "fling", fromPx, toPx, distance / seconds); return { swiped: true, timestampMs }; } const api = services.simulatorServer as SimulatorServerApi; diff --git a/packages/tool-server/src/tools/gesture-tap/index.ts b/packages/tool-server/src/tools/gesture-tap/index.ts index 03f7ead83c..de665022c6 100644 --- a/packages/tool-server/src/tools/gesture-tap/index.ts +++ b/packages/tool-server/src/tools/gesture-tap/index.ts @@ -4,7 +4,12 @@ import { simulatorServerRef, type SimulatorServerApi } from "../../blueprints/si import { chromiumCdpRef, type ChromiumCdpApi } from "../../blueprints/chromium-cdp"; import { assertChromiumWindowVisible } from "../../utils/chromium-visibility"; import { resolveDevice, harmonyConnectKey } from "../../utils/device-info"; -import { harmonyDisplay, harmonyTouch, toDevicePoint } from "../../utils/harmony-uitest"; +import { + assertHarmonyScreenAwake, + harmonyDisplay, + harmonyTouch, + toDevicePoint, +} from "../../utils/harmony-uitest"; import { ensureDep } from "../../utils/check-deps"; import { sendCommand } from "../../utils/simulator-client"; @@ -106,6 +111,9 @@ async function tapHarmony( clickCount: number ): Promise { const display = await harmonyDisplay(connectKey); + // A tap against a suspended panel reports `No Error` and lands nowhere — + // refuse it rather than report { tapped: true } for a touch that did nothing. + assertHarmonyScreenAwake(display, "tap"); const point = toDevicePoint(x, y, display); if (clickCount === 2) { await harmonyTouch(connectKey, "doubleClick", point); diff --git a/packages/tool-server/src/tools/keyboard/index.ts b/packages/tool-server/src/tools/keyboard/index.ts index d6a57f326f..fe77fac74c 100644 --- a/packages/tool-server/src/tools/keyboard/index.ts +++ b/packages/tool-server/src/tools/keyboard/index.ts @@ -33,7 +33,7 @@ const zodSchema = z.object({ .string() .optional() .describe( - "Named key to press: enter, escape, backspace, tab, space, arrow-up, arrow-down, arrow-left, arrow-right, f1–f12. When combined with `text`, the key is pressed AFTER the text is typed (so text + enter types and submits). Not supported on TV targets — move focus with `tv-remote` (up/down/left/right) instead." + "Named key to press: enter, escape, backspace, tab, space, arrow-up, arrow-down, arrow-left, arrow-right, f1–f12. On HarmonyOS only enter, backspace, space, arrow-left and arrow-right exist — any other key fails with 400 KEYBOARD_KEY_UNSUPPORTED. When combined with `text`, the key is pressed AFTER the text is typed (so text + enter types and submits). Not supported on TV targets — move focus with `tv-remote` (up/down/left/right) instead." ), delayMs: z .number() diff --git a/packages/tool-server/src/tools/keyboard/platforms/harmony.ts b/packages/tool-server/src/tools/keyboard/platforms/harmony.ts index 7fd6bef117..d797be640b 100644 --- a/packages/tool-server/src/tools/keyboard/platforms/harmony.ts +++ b/packages/tool-server/src/tools/keyboard/platforms/harmony.ts @@ -2,9 +2,48 @@ import { FAILURE_CODES } from "@argent/registry"; import type { PlatformImpl } from "../../../utils/cross-platform-tool"; import { InvalidToolInputError } from "../../../utils/capability"; import { harmonyConnectKey } from "../../../utils/device-info"; -import { harmonyKeyEvent, harmonyTypeText } from "../../../utils/harmony-uitest"; +import { + assertHarmonyScreenAwake, + harmonyDisplay, + harmonyKeyEvent, + harmonyTypeText, +} from "../../../utils/harmony-uitest"; import type { KeyboardParams, KeyboardResult } from "../types"; +/** + * `uitest uiInput text` validates almost nothing (see `harmony-uitest.ts`) and + * answers `No Error` whether or not anything landed, so an un-typeable + * character is a silent wrong field while `keys` reports the count asked for. + * Every sibling backend rejects such input up front with a 400 + * (`assertTypeableAndroidText`, `injectVegaText`, the iOS/chromium keycode + * tables) — this is the harmony half of that contract. A newline gets its own + * message pointing at `key: "enter"`, the same advice Android gives. + */ +function assertTypeableHarmonyText(text: string): void { + if (/[\n\r]/.test(text)) { + throw new InvalidToolInputError( + 'HarmonyOS `uitest uiInput text` cannot type a newline. Submit with `key: "enter"` after typing instead.', + { + error_code: FAILURE_CODES.KEYBOARD_CHARACTER_UNSUPPORTED, + failure_stage: "harmony_keyboard_validate_text", + } + ); + } + const bad = [...text].find((c) => { + const cp = c.codePointAt(0)!; + return cp < 0x20 || cp === 0x7f; + }); + if (bad !== undefined) { + throw new InvalidToolInputError( + `HarmonyOS \`uitest uiInput text\` cannot type U+${bad.codePointAt(0)!.toString(16).toUpperCase().padStart(4, "0")} (a control character).`, + { + error_code: FAILURE_CODES.KEYBOARD_CHARACTER_UNSUPPORTED, + failure_stage: "harmony_keyboard_validate_text", + } + ); + } +} + /** * Named keys this backend will press, and the `uitest uiInput keyEvent` keyID * each maps to. @@ -53,11 +92,17 @@ function resolveHarmonyKeycode(key: string): number { } async function typeHarmony(connectKey: string, params: KeyboardParams): Promise { + // Typing into a suspended panel reports `No Error` and lands nowhere — check + // before resolving anything, so a dead screen fails the same way a tap does. + assertHarmonyScreenAwake(await harmonyDisplay(connectKey), "type"); // Resolve before injecting anything: a combined key+text call with an unknown // key name must reject having typed nothing, not type the text and then 400. const keycode = params.key ? resolveHarmonyKeycode(params.key) : null; let keysPressed = 0; if (params.text) { + // Validate before injecting: `uitest` reports `No Error` whether or not the + // text landed, so an un-typeable character must be caught here (see above). + assertTypeableHarmonyText(params.text); // `uitest uiInput text` types into whatever holds focus, in one shot — there // is no per-character injection, so `delayMs` has nothing to pace (the tool // description already lists the platforms that ignore it). diff --git a/packages/tool-server/src/tools/restart-app/platforms/harmony.ts b/packages/tool-server/src/tools/restart-app/platforms/harmony.ts index fcaa1d9cb5..e8f8fb7fb9 100644 --- a/packages/tool-server/src/tools/restart-app/platforms/harmony.ts +++ b/packages/tool-server/src/tools/restart-app/platforms/harmony.ts @@ -1,15 +1,22 @@ import type { PlatformImpl } from "../../../utils/cross-platform-tool"; import { harmonyConnectKey } from "../../../utils/device-info"; -import { launchHarmonyApp, terminateHarmonyApp } from "../../../utils/harmony-apps"; +import { + isHarmonyAppNotRunning, + launchHarmonyApp, + terminateHarmonyApp, +} from "../../../utils/harmony-apps"; import type { RestartAppParams, RestartAppResult, RestartAppHarmonyServices } from "../types"; /** * Restart on HarmonyOS is `aa force-stop` then a resolved `aa start`. * - * The stop is tolerated rather than required: force-stopping an app that is not - * running is not an error worth failing a restart over, and the launch that - * follows is what the caller actually asked for. `activity` is Android-only and - * ignored — the HarmonyOS entry ability is resolved from the bundle. + * The stop is tolerated only when the app was not running — a restart of an + * app that was never launched is a start, and that is what the caller asked + * for. Every other stop failure propagates: `restart-app` exists to guarantee + * a fresh process, so a refused stop followed by a launch that no-ops on the + * still-running app must not read as `restarted: true`. `activity` is + * Android-only and ignored — the HarmonyOS entry ability is resolved from the + * bundle. */ export const harmonyImpl: PlatformImpl< RestartAppHarmonyServices, @@ -19,7 +26,10 @@ export const harmonyImpl: PlatformImpl< requires: ["hdc"], handler: async (_services, params, device) => { const connectKey = harmonyConnectKey(device.id); - await terminateHarmonyApp(connectKey, params.bundleId).catch(() => {}); + await terminateHarmonyApp(connectKey, params.bundleId).catch((err: unknown) => { + if (isHarmonyAppNotRunning(err)) return; + throw err; + }); await launchHarmonyApp(connectKey, params.bundleId); return { restarted: true, bundleId: params.bundleId }; }, diff --git a/packages/tool-server/src/tools/screenshot-diff/index.ts b/packages/tool-server/src/tools/screenshot-diff/index.ts index 69ca159acb..1d553f65dd 100644 --- a/packages/tool-server/src/tools/screenshot-diff/index.ts +++ b/packages/tool-server/src/tools/screenshot-diff/index.ts @@ -13,7 +13,7 @@ import type { } from "@argent/registry"; import { simulatorServerRef, type SimulatorServerApi } from "../../blueprints/simulator-server"; import { resolveDevice, harmonyConnectKey } from "../../utils/device-info"; -import { UnsupportedOperationError } from "../../utils/capability"; +import { InvalidToolInputError } from "../../utils/capability"; import { captureHarmonyScreenshotPng } from "../../utils/harmony-screen"; import { ensureDep } from "../../utils/check-deps"; import { httpScreenshot } from "../../utils/simulator-client"; @@ -237,11 +237,15 @@ function liveCapture( if (params.rotation) { // Inside the capture rather than up front: for a static two-path diff // rotation is inert on every platform, and rejecting it there would - // make HarmonyOS the one platform an unused argument fails. - throw new UnsupportedOperationError( - "screenshot-diff", - device, - "rotation is not supported for a live HarmonyOS capture: `uitest screenCap` captures the display in its current orientation and has no override. Rotate the device itself, or drop the rotation parameter." + // make HarmonyOS the one platform an unused argument fails. A + // per-ARGUMENT refusal (same class as the keyboard key guard), not a + // per-tool one — screenshot-diff works on every harmony device. + throw new InvalidToolInputError( + "rotation is not supported for a live HarmonyOS capture: `uitest screenCap` captures the display in its current orientation and has no override. Rotate the device itself, or drop the rotation parameter.", + { + error_code: FAILURE_CODES.TOOL_INPUT_INVALID, + failure_stage: "harmony_screenshot_diff_rotation", + } ); } await ensureDep("hdc"); diff --git a/packages/tool-server/src/tools/screenshot/index.ts b/packages/tool-server/src/tools/screenshot/index.ts index 895c6ec0e9..db94e4c502 100644 --- a/packages/tool-server/src/tools/screenshot/index.ts +++ b/packages/tool-server/src/tools/screenshot/index.ts @@ -3,7 +3,12 @@ import { promisify } from "node:util"; import * as os from "node:os"; import * as path from "node:path"; import { z } from "zod"; -import type { Registry, ToolCapability, ToolDefinition } from "@argent/registry"; +import { + FAILURE_CODES, + type Registry, + type ToolCapability, + type ToolDefinition, +} from "@argent/registry"; import { simulatorServerRef, type SimulatorServerApi } from "../../blueprints/simulator-server"; import { chromiumCdpRef, type ChromiumCdpApi } from "../../blueprints/chromium-cdp"; import { resolveDevice, harmonyConnectKey } from "../../utils/device-info"; @@ -13,7 +18,7 @@ import { simctlArgsForUdid } from "../../utils/ios-device-sets"; import { captureVegaScreenshotPng } from "../../utils/vega-screen"; import { captureHarmonyScreenshotPng } from "../../utils/harmony-screen"; import { ensureDep } from "../../utils/check-deps"; -import { UnsupportedOperationError } from "../../utils/capability"; +import { InvalidToolInputError } from "../../utils/capability"; import { requireArtifacts, type ArtifactHandle } from "../../artifacts"; const execFileAsync = promisify(execFile); @@ -186,10 +191,15 @@ Fails if the simulator-server / emulator backend / Chromium CDP / \`hdc\` is not // back over hdc; there is no simulator-server controller for the platform. if (device.platform === "harmony") { if (params.rotation) { - throw new UnsupportedOperationError( - "screenshot", - device, - "rotation is not supported on HarmonyOS: `uitest screenCap` captures the display in its current orientation and has no override. Rotate the device itself, or drop the rotation parameter." + // A per-ARGUMENT refusal, not a per-tool one: screenshot works on + // every harmony device — only this parameter has no effect. Same + // class the keyboard backend uses for an unsupported key. + throw new InvalidToolInputError( + "rotation is not supported on HarmonyOS: `uitest screenCap` captures the display in its current orientation and has no override. Rotate the device itself, or drop the rotation parameter.", + { + error_code: FAILURE_CODES.TOOL_INPUT_INVALID, + failure_stage: "harmony_screenshot_rotation", + } ); } await ensureDep("hdc"); diff --git a/packages/tool-server/src/utils/harmony-apps.ts b/packages/tool-server/src/utils/harmony-apps.ts index 26d4450d51..da2f53c45e 100644 --- a/packages/tool-server/src/utils/harmony-apps.ts +++ b/packages/tool-server/src/utils/harmony-apps.ts @@ -1,4 +1,4 @@ -import { FAILURE_CODES, FailureError } from "@argent/registry"; +import { FAILURE_CODES, FailureError, getFailureSignal } from "@argent/registry"; import { runHdcShell, shellQuote } from "./harmony-hdc"; /** @@ -38,17 +38,20 @@ interface HarmonyBundleEntry { * The subtlety, and the reason this is not a one-line read of `mainAbility`: * **`aa start -a` does not accept `mainAbility`.** It accepts the `name` of the * matching entry in `abilityInfos`, and the two are spelled differently from - * bundle to bundle. Measured on HarmonyOS 6.0.1: + * bundle to bundle. Measured on HarmonyOS 6.0.1 and a 6.1.1 emulator: * * bundle mainAbility abilityInfos name * calculator com.huawei.hmos.calculator.CalculatorAbility CalculatorAbility * settings com.huawei.hmos.settings.MainAbility (identical) * notepad MainAbility (identical) + * photos MainAbility com.huawei.hmos.photos.MainAbility * * Passing `mainAbility` verbatim launches Settings and fails on Calculator with * `10104001 The specified ability does not exist`; passing the bare final - * segment does the reverse. So `mainAbility` is used to *identify* the entry and - * the entry's own `name` is what gets passed. + * segment does the reverse. Photos spells them the other way around — a short + * `mainAbility` against a fully-qualified entry — so the match has to run in + * BOTH directions, not just the Calculator one. `mainAbility` is used to + * *identify* the entry and the entry's own `name` is what gets passed. * * `mainAbility` is what identifies it, rather than the first entry of * `abilityInfos`: a bundle can declare a dozen abilities (Settings declares @@ -98,12 +101,20 @@ export async function resolveHarmonyEntry( } const modules = Array.isArray(parsed.hapModuleInfos) ? parsed.hapModuleInfos : []; const mainEntry = typeof parsed.mainEntry === "string" ? parsed.mainEntry : null; + // `bm` serialises the `mainAbility` key on EVERY bundle, so an empty string — + // not an absent key — is how a bundle with no launcher entry reports itself + // (14 of the 73 bundles installed on a 6.1.1 emulator, every service bundle + // among them). Treat "" as missing on BOTH selection arms: accepting it sends + // `aa start -a ''`, which `aa` reads as an implicit start, answers with + // `10103101 Failed to find a matching application for implicit launch`, and + // leaves a modal "No options to open with" dialog on the device — the exact + // failure this module's lookup exists to prevent, with a worse message. + const hasMain = (m: (typeof modules)[number]): boolean => + typeof m.mainAbility === "string" && m.mainAbility.length > 0; // `mainEntry` names the module that owns the launcher entry; fall back to the // first module that declares a mainAbility for bundles that omit it. - const chosen = - modules.find((m) => m.name === mainEntry && typeof m.mainAbility === "string") ?? - modules.find((m) => typeof m.mainAbility === "string"); - if (!chosen || typeof chosen.mainAbility !== "string" || typeof chosen.name !== "string") { + const chosen = modules.find((m) => m.name === mainEntry && hasMain(m)) ?? modules.find(hasMain); + if (!chosen || typeof chosen.name !== "string" || typeof chosen.mainAbility !== "string") { throw new FailureError( `App '${bundleId}' on HarmonyOS device '${connectKey}' declares no launchable main ability, ` + `so there is nothing to start. It may be a service or extension rather than an app.`, @@ -121,13 +132,63 @@ export async function resolveHarmonyEntry( }; } +/** Stop a running app. See {@link terminateHarmonyApp} for why this is separate. */ +const AA_NOT_RUNNING_MARKER = "not running"; + +/** + * Stop every process of an app. + * + * `aa force-stop` takes the bundle name and, unlike `aa start`, needs no ability + * — there is nothing to resolve. + * + * Throws when the stop was refused for any reason other than the app not + * running: `restart-app` exists to guarantee a fresh process, and reporting + * `restarted: true` for an app that is still running with its old state is the + * one outcome it must not produce. + */ +export async function terminateHarmonyApp(connectKey: string, bundleId: string): Promise { + const { stdout } = await runHdcShell(connectKey, `aa force-stop ${shellQuote(bundleId)}`); + if (stdout.includes("error:")) { + throw new FailureError( + `Failed to stop '${bundleId}' on HarmonyOS device '${connectKey}': ${firstLine(stdout)}`, + { + error_code: FAILURE_CODES.HARMONY_ABILITY_START_FAILED, + failure_stage: "harmony_force_stop", + failure_area: "tool_server", + error_kind: "subprocess", + } + ); + } +} + +/** + * The one `aa force-stop` failure that is not a failure: stopping an app that + * is not running. A restart of an app that was never launched must still + * launch it — that is a start, not an error — so `restart-app` narrows its + * tolerance to this exact diagnostic rather than swallowing every refusal. + */ +export function isHarmonyAppNotRunning(error: unknown): boolean { + return ( + error instanceof FailureError && + getFailureSignal(error)?.failure_stage === "harmony_force_stop" && + error.message.includes(AA_NOT_RUNNING_MARKER) + ); +} + /** * The spelling of the main ability that `aa start -a` accepts — see the note on * `resolveHarmonyEntry`. * - * Falls back to `mainAbility` itself when no declared ability matches: two of - * the three bundles measured spell them identically, so it is the right guess, - * and `aa`'s own `10104001` names the problem precisely if it turns out wrong. + * The match runs in BOTH directions on a dot boundary, because the two + * spellings occur both ways on real bundles: Calculator's `mainAbility` is + * fully qualified where its `abilityInfos` entry is short, and Photos' is the + * reverse (short `mainAbility`, fully-qualified entry). Exact match first, then + * either side being the other's final dot-segment. The boundary stops `Ability` + * matching `MainAbility`. + * + * Falls back to `mainAbility` itself when no declared ability matches: the + * common case spells them identically, so it is the right guess, and `aa`'s own + * `10104001` names the problem precisely if it turns out wrong. */ function startableAbilityName( abilityInfos: Array<{ name?: unknown }> | undefined, @@ -137,7 +198,9 @@ function startableAbilityName( .map((a) => a?.name) .filter((n): n is string => typeof n === "string" && n.length > 0); if (names.includes(mainAbility)) return mainAbility; - return names.find((n) => mainAbility.endsWith(`.${n}`)) ?? mainAbility; + return ( + names.find((n) => mainAbility.endsWith(`.${n}`) || n.endsWith(`.${mainAbility}`)) ?? mainAbility + ); } /** Bring an app to the foreground, resolving its entry ability first. */ @@ -150,27 +213,6 @@ export async function launchHarmonyApp(connectKey: string, bundleId: string): Pr assertAbilityStarted(stdout, connectKey, bundleId); } -/** - * Stop every process of an app. - * - * `aa force-stop` takes the bundle name and, unlike `aa start`, needs no ability - * — there is nothing to resolve. - */ -export async function terminateHarmonyApp(connectKey: string, bundleId: string): Promise { - const { stdout } = await runHdcShell(connectKey, `aa force-stop ${shellQuote(bundleId)}`); - if (stdout.includes("error:")) { - throw new FailureError( - `Failed to stop '${bundleId}' on HarmonyOS device '${connectKey}': ${firstLine(stdout)}`, - { - error_code: FAILURE_CODES.HARMONY_ABILITY_START_FAILED, - failure_stage: "harmony_force_stop", - failure_area: "tool_server", - error_kind: "subprocess", - } - ); - } -} - /** * Open a URI through whichever app claims it. * diff --git a/packages/tool-server/src/utils/harmony-uitest.ts b/packages/tool-server/src/utils/harmony-uitest.ts index 4beae24bbc..169f266549 100644 --- a/packages/tool-server/src/utils/harmony-uitest.ts +++ b/packages/tool-server/src/utils/harmony-uitest.ts @@ -45,6 +45,35 @@ const REMOTE_TMP = "/data/local/tmp"; const UITEST_TIMEOUT_MS = 20_000; +/** + * `uitest` does not tolerate overlapping invocations: a second call on the + * same device blocks until the first finishes, and if that takes past the + * timeout the loser is SIGKILLed with the internal status sentinel in its + * error. Measured: two concurrent `dumpLayout`s → one 20s failure; three → + * two. So calls on one connect key are serialised here — queued behind each + * other rather than raced — which turns an opaque 20s kill into the second + * call simply starting once the first is done. Keyed per device so a phone + * and an emulator, or two devices, still run in parallel; the map entry is + * dropped once the queue drains so it cannot grow unboundedly. + */ +const uitestQueues = new Map>(); + +function enqueueUitest(connectKey: string, run: () => Promise): Promise { + const prior = uitestQueues.get(connectKey) ?? Promise.resolve(); + const result = prior.then(run); + // The queue tracks only the settlement signal, not the value, so a rejection + // never propagates into the next caller's chain. + const drained = result.then( + () => undefined, + () => undefined + ); + uitestQueues.set(connectKey, drained); + void drained.finally(() => { + if (uitestQueues.get(connectKey) === drained) uitestQueues.delete(connectKey); + }); + return result; +} + /** * `uitest` prints its multi-line usage block after every failure. Only the * leading line names the actual problem, so surface that and drop the rest — @@ -61,7 +90,9 @@ async function runUitest( args: string, timeoutMs = UITEST_TIMEOUT_MS ): Promise { - const { stdout, exitCode } = await runHdcShell(connectKey, `uitest ${args}`, timeoutMs); + const { stdout, exitCode } = await enqueueUitest(connectKey, () => + runHdcShell(connectKey, `uitest ${args}`, timeoutMs) + ); if (exitCode !== 0) { throw new FailureError(`uitest ${args} failed on ${connectKey}: ${uitestDiagnostic(stdout)}`, { error_code: FAILURE_CODES.HARMONY_UITEST_FAILED, @@ -132,6 +163,30 @@ export function toDevicePoint( return { x: clamp(x, display.width), y: clamp(y, display.height) }; } +/** + * Refuse an input injection against a suspended panel. + * + * `uitest uiInput` reports `No Error` whether or not the touch landed, and + * against a screen that is OFF or SUSPEND it lands nowhere — measured: a tap + * on the WLAN row of Settings with `powerStatus=POWER_STATUS_OFF` returned + * success and changed nothing. `describe` is honest about this same state + * (its asleep hint); the input tools must be too, or a screen timeout mid- + * session turns every later gesture into a silent no-op that reports success. + */ +export function assertHarmonyScreenAwake(display: HarmonyDisplay, action: string): void { + if (display.screenOn) return; + throw new FailureError( + `Cannot ${action} on a HarmonyOS device whose display is off: injected input lands nowhere ` + + `while the panel is suspended. Wake it with \`button\` (power), then retry.`, + { + error_code: FAILURE_CODES.HARMONY_UITEST_FAILED, + failure_stage: "harmony_screen_off", + failure_area: "tool_server", + error_kind: "validation", + } + ); +} + type HarmonyTouchCommand = "click" | "doubleClick" | "longClick"; export async function harmonyTouch( @@ -151,25 +206,12 @@ type HarmonySwipeCommand = "swipe" | "drag" | "fling"; const HARMONY_VELOCITY_MIN = 200; const HARMONY_VELOCITY_MAX = 40_000; -async function harmonySwipe( - connectKey: string, - command: HarmonySwipeCommand, - from: { x: number; y: number }, - to: { x: number; y: number }, - velocity: number -): Promise { - const v = Math.max(HARMONY_VELOCITY_MIN, Math.min(HARMONY_VELOCITY_MAX, Math.round(velocity))); - await runUitest(connectKey, `uiInput ${command} ${from.x} ${from.y} ${to.x} ${to.y} ${v}`); -} - /** - * Swipe between two normalized points over `durationMs`. - * * `uitest` takes a **velocity**, not a duration, so the duration argent's tools - * speak is converted here: velocity = pixels travelled / seconds. Doing it the - * other way — passing a fixed velocity — would make a short swipe and a - * screen-length one take wildly different times, and the callers that pace a - * scroll loop against `durationMs` would be pacing against nothing. + * speak is converted by the caller: velocity = pixels travelled / seconds. + * Doing it the other way — passing a fixed velocity — would make a short swipe + * and a screen-length one take wildly different times, and the callers that + * pace a scroll loop against `durationMs` would be pacing against nothing. * * `settle` picks the verb rather than reshaping the path. `uitest` exposes both * `swipe` (a drag that ends where it ends) and `fling` (which hands the scroller @@ -178,19 +220,15 @@ async function harmonySwipe( * between the two commands; the resulting difference in coast distance was not * measured here. */ -export async function harmonySwipeNormalized( +export async function harmonySwipe( connectKey: string, + command: HarmonySwipeCommand, from: { x: number; y: number }, to: { x: number; y: number }, - durationMs: number, - settle: boolean + velocity: number ): Promise { - const display = await harmonyDisplay(connectKey); - const fromPx = toDevicePoint(from.x, from.y, display); - const toPx = toDevicePoint(to.x, to.y, display); - const distance = Math.hypot(toPx.x - fromPx.x, toPx.y - fromPx.y); - const seconds = Math.max(durationMs, 1) / 1000; - await harmonySwipe(connectKey, settle ? "swipe" : "fling", fromPx, toPx, distance / seconds); + const v = Math.max(HARMONY_VELOCITY_MIN, Math.min(HARMONY_VELOCITY_MAX, Math.round(velocity))); + await runUitest(connectKey, `uiInput ${command} ${from.x} ${from.y} ${to.x} ${to.y} ${v}`); } export async function harmonyKeyEvent(connectKey: string, key: string): Promise { diff --git a/packages/tool-server/test/boot-device-harmony.test.ts b/packages/tool-server/test/boot-device-harmony.test.ts index aa1e64534d..256c7af1bf 100644 --- a/packages/tool-server/test/boot-device-harmony.test.ts +++ b/packages/tool-server/test/boot-device-harmony.test.ts @@ -154,9 +154,13 @@ describe("boot-device — HarmonyOS emulator path", () => { it("only counts a target once it is Connected", async () => { // A killed emulator leaves its row behind as `Offline` — measured — so a - // row existing is not the same as a device there is any point driving. - const offline = { ...emulatorTarget, state: "Offline" }; - targets([PHONE], [PHONE, offline], [PHONE, emulatorTarget]); + // row existing is not the same as a device there is any point driving. The + // Offline row and the later Connected one are given DIFFERENT keys on + // purpose: sharing one key would pass whether the code waited for + // `Connected` or adopted the `Offline` row on the earlier poll, which is + // exactly the drift this test exists to catch. + const offlineOther = { connectKey: "127.0.0.1:5559", connection: "TCP", state: "Offline" }; + targets([PHONE], [PHONE, offlineOther], [PHONE, emulatorTarget]); vi.useFakeTimers(); const pending = boot({}); @@ -212,7 +216,11 @@ describe("boot-device — HarmonyOS emulator path", () => { const result = (await pending) as { udid: string; note?: string }; expect(result.udid).toBe(`harmony-emulator-${INSTANCE}`); - expect(result.note).toBeTruthy(); + // Truthy is not enough: the note this branch returns must be the + // two-arrival refusal, not the generic "had not registered before the boot + // budget ran out" one — both targets DID register, and argent declined to + // choose. Pointing the caller at `bootTimeoutMs` would misdiagnose it. + expect(result.note).toMatch(/two|both|more than one|could not tell which/i); }); it("names the instance and says why when nothing registers within the budget", async () => { @@ -304,13 +312,16 @@ describe("boot-device — HarmonyOS emulator path", () => { vi.useFakeTimers(); const pending = boot({ bootTimeoutMs: 900_000 }); - const settled = expect(pending).rejects.toThrow( - /is not found\. Please create the device(?!.*create one in DevEco Studio)/s - ); + // A negative LOOKAHEAD anchored on `is not found…` would not catch the + // no-instances sentence prepended BEFORE the anchor, so this asserts the + // absence directly. + const settled = expect(pending).rejects.toThrow(/is not found\. Please create the device/); await vi.advanceTimersByTimeAsync(1_000); child.die(`"${INSTANCE}" is not found. Please create the device(folder): /x`, 1); await vi.advanceTimersByTimeAsync(3_000); await settled; + const msg = await pending.catch((e: unknown) => (e as Error).message); + expect(msg).not.toContain("create one in DevEco Studio"); }); it("leaves a running instance alone and says how to reach it", async () => { diff --git a/packages/tool-server/test/harmony-apps.test.ts b/packages/tool-server/test/harmony-apps.test.ts index 7dd0b0fb0a..e0344ae995 100644 --- a/packages/tool-server/test/harmony-apps.test.ts +++ b/packages/tool-server/test/harmony-apps.test.ts @@ -84,6 +84,71 @@ describe("resolveHarmonyEntry", () => { ); }); + it("resolves a short mainAbility to a fully-qualified abilityInfos entry (Photos direction)", async () => { + // Photos spells the pair the other way around from Calculator: `mainAbility` + // is the short name and the ability declares itself fully qualified. The + // match must run in BOTH directions — drop the `n.endsWith` arm and this + // bundle falls back to the bare `MainAbility`, which `aa start -a` rejects + // with `10104001 The specified ability does not exist`. + runHdcShell.mockResolvedValueOnce( + ok( + JSON.stringify({ + mainEntry: "phone", + hapModuleInfos: [ + { + name: "phone", + mainAbility: "MainAbility", + abilityInfos: [{ name: "com.huawei.hmos.photos.MainAbility" }], + }, + ], + }) + ) + ); + expect(await resolveHarmonyEntry("dev", "com.huawei.hmos.photos")).toEqual({ + mainAbility: "com.huawei.hmos.photos.MainAbility", + module: "phone", + }); + }); + + it("prefers an exact abilityInfos match over a suffix that would also match", async () => { + // Both entries match `MainAbility` under the suffix rule, but the bundle + // declares one of them verbatim — and the verbatim declaration is the + // spelling `aa start -a` accepts. Checking the suffix rule first would pick + // the qualified entry (listed first here so the wrong precedence bites). + runHdcShell.mockResolvedValueOnce( + ok( + JSON.stringify({ + mainEntry: "phone", + hapModuleInfos: [ + { + name: "phone", + mainAbility: "MainAbility", + abilityInfos: [{ name: "com.example.fake.MainAbility" }, { name: "MainAbility" }], + }, + ], + }) + ) + ); + expect((await resolveHarmonyEntry("dev", "x")).mainAbility).toBe("MainAbility"); + }); + + it("matches only on a dot boundary — `Ability` does not match `MainAbility`", async () => { + // Without the boundary, `MainAbility`.endsWith(`Ability`) would rewrite the + // ability to a name the bundle never declared. No entry matches, so the + // fallback passes `mainAbility` through unchanged. + runHdcShell.mockResolvedValueOnce( + ok( + JSON.stringify({ + mainEntry: "phone", + hapModuleInfos: [ + { name: "phone", mainAbility: "Ability", abilityInfos: [{ name: "MainAbility" }] }, + ], + }) + ) + ); + expect((await resolveHarmonyEntry("dev", "x")).mainAbility).toBe("Ability"); + }); + it("picks the module named by mainEntry, not the first one listed", async () => { runHdcShell.mockResolvedValueOnce( ok( diff --git a/packages/tool-server/test/harmony-hdc-timeout.test.ts b/packages/tool-server/test/harmony-hdc-timeout.test.ts index 8ff28179d8..0655a2a520 100644 --- a/packages/tool-server/test/harmony-hdc-timeout.test.ts +++ b/packages/tool-server/test/harmony-hdc-timeout.test.ts @@ -33,10 +33,23 @@ describe("runHdc timeout enforcement", () => { // without `killSignal`: still running 6s after a 1s timeout. const started = Date.now(); - await expect(runHdc(["list", "targets"], 800)).rejects.toThrow(); + // The rejection must name the TIMEOUT, not the dependency: an instant + // `HARMONY_HDC_NOT_FOUND` also satisfies `rejects.toThrow()` and the <6s + // bound while never having spawned the wedged child at all — so the error + // text is pinned to the kill, and the lower bound to the timeout having + // actually been waited out. + const err = await runHdc(["list", "targets"], 800).then( + () => { + throw new Error("expected a rejection, got a resolution"); + }, + (e: unknown) => e as Error + ); + const elapsed = Date.now() - started; + expect(err.message).not.toMatch(/not found|not installed/i); + expect(elapsed).toBeGreaterThanOrEqual(700); // the 800ms timeout, less timer slack // Far below the stub's own 30s sleep: the assertion is that the deadline is // enforced at all, not its precise latency on a loaded CI box. - expect(Date.now() - started).toBeLessThan(6_000); + expect(elapsed).toBeLessThan(6_000); }, 20_000); }); diff --git a/packages/tool-server/test/harmony-layout-parser.test.ts b/packages/tool-server/test/harmony-layout-parser.test.ts index b90a322190..e7ddb017c0 100644 --- a/packages/tool-server/test/harmony-layout-parser.test.ts +++ b/packages/tool-server/test/harmony-layout-parser.test.ts @@ -62,7 +62,7 @@ describe("harmonyLabel", () => { }); describe("parseHarmonyLayout", () => { - it("labels each window with the app that owns it", () => { + it("tags each window with the app that owns it, as an identifier not a label", () => { const tree = parseHarmonyLayout( root([ node({ type: "root", bundleName: "com.app", bounds: "[0,107][1216,2688]" }, [ @@ -74,8 +74,13 @@ describe("parseHarmonyLayout", () => { expect(tree.children).toHaveLength(1); expect(tree.children[0].role).toBe("Window"); // A tree spanning an app and the status bar (always a separate sceneboard - // window) is unreadable if both are anonymous stacks. - expect(tree.children[0].label).toBe("com.app"); + // window) is unreadable if both are anonymous stacks. The bundle rides as + // `identifier`, NOT `label`: the shared matcher reads `label` as visible + // text, and a full-screen node "named" `com.app` would poison every text + // wait (`exists: {text:"app"}` true on every screen, a `text` wait for the + // app's own name matching the window instead of its title). + expect(tree.children[0].identifier).toBe("com.app"); + expect(tree.children[0].label).toBeUndefined(); }); it("walks through ArkUI layout scaffolding instead of emitting it", () => { @@ -356,6 +361,84 @@ describe("parseHarmonyLayout", () => { expect(overlay.label).toMatch(/another process/); }); + it("parses a UIExtensionComponent's own subtree when the dump carries one", () => { + // On HarmonyOS 6.1.1 the same node arrives WITH the full subtree — the + // HUAWEI ID login sheet carries its TextInput and both Buttons (45 nodes + // measured), and those targets are live (their geometry taps). Replacing it + // with the placeholder discards the only controls that dismiss the sheet, + // and `uitest screenCap` does not capture the surface either. + const tree = parseHarmonyLayout( + root([ + node( + { + type: "WindowScene", + bundleName: "com.huawei.hmos.settings", + bounds: "[0,0][1216,2688]", + }, + [ + node({ type: "UIExtensionComponent", bounds: "[0,0][1216,2688]" }, [ + node({ type: "Text", text: "HUAWEI ID", bounds: "[400,200][860,280]" }), + node({ type: "TextInput", hint: "Phone number", bounds: "[200,400][1060,520]" }), + node({ + type: "Button", + text: "Back", + clickable: "true", + bounds: "[56,164][196,304]", + }), + node({ + type: "Button", + text: "Log in/Register", + clickable: "true", + bounds: "[200,2400][1060,2520]", + }), + ]), + ] + ), + ]), + SCREEN + ).tree; + const text = JSON.stringify(tree); + expect(text).toContain("HUAWEI ID"); + expect(text).toContain("Back"); + expect(text).toContain("Log in/Register"); + // The subtree's own back Button is a real, tappable target — not swallowed + // into a placeholder that says the screen is opaque. + const back = tree.children[0].children[0].children.find((c) => c.label === "Back"); + expect(back).toMatchObject({ role: "Button", clickable: true }); + }); + + it("does not let a window's bundle id read as visible text on the screen", () => { + // Regression for the matcher poisoning: with the bundle on `label`, every + // screen matched `exists: {text:"sceneboard"}` and a `text` wait for the + // app's own name tied with its real title in reading order and lost. + const tree = parseHarmonyLayout( + root([ + node( + { + type: "WindowScene", + bundleName: "com.huawei.hmos.settings", + bounds: "[0,0][1216,2688]", + }, + [node({ type: "Text", text: "Settings", bounds: "[40,120][500,200]" })] + ), + node( + { type: "WindowScene", bundleName: "com.ohos.sceneboard", bounds: "[0,0][1216,120]" }, + [node({ type: "Text", text: "12:30", bounds: "[1100,20][1220,100]" })] + ), + ]), + SCREEN + ).tree; + const labels: string[] = []; + const walk = (n: DescribeNode): void => { + if (n.label) labels.push(n.label); + n.children.forEach(walk); + }; + tree.children.forEach(walk); + expect(labels).toContain("Settings"); + expect(labels).not.toContain("com.huawei.hmos.settings"); + expect(labels).not.toContain("com.ohos.sceneboard"); + }); + it("takes the screen size from the dump's own root, not the caller's", () => { // The two are read at different instants; on a foldable that gap is enough // for an unfold to normalize every frame against the wrong axis. diff --git a/packages/tool-server/test/harmony-screen.test.ts b/packages/tool-server/test/harmony-screen.test.ts new file mode 100644 index 0000000000..96fb50c687 --- /dev/null +++ b/packages/tool-server/test/harmony-screen.test.ts @@ -0,0 +1,109 @@ +import { describe, it, expect, vi, beforeEach, afterEach } from "vitest"; +import { access, readFile, rm, writeFile } from "node:fs/promises"; +import { PNG } from "pngjs"; +import { + hdcFileRecv as realHdcFileRecv, + runHdcShell as realRunHdcShell, +} from "../src/utils/harmony-hdc"; +import { captureHarmonyScreenshotPng } from "../src/utils/harmony-screen"; + +// Only the transport is faked. `shellQuote` stays real so the assertions see +// the exact command lines the device would; the fetch writes a real (tiny) +// PNG so `captureHarmonyScreenshotPng`'s decode/scale path runs for true. +vi.mock("../src/utils/harmony-hdc", async (importOriginal) => { + const actual = await importOriginal(); + return { ...actual, runHdcShell: vi.fn(), hdcFileRecv: vi.fn() }; +}); + +const runHdcShell = vi.mocked(realRunHdcShell); +const hdcFileRecv = vi.mocked(realHdcFileRecv); + +/** A minimal valid PNG — what `uitest screenCap` would have written. */ +function tinyPng(): Buffer { + return PNG.sync.write(new PNG({ width: 4, height: 4 })); +} + +/** The remote path of every `uitest screenCap -p ''` call, in order. */ +function capturedRemotePaths(): string[] { + return runHdcShell.mock.calls + .map((c) => /screenCap -p '([^']+)'/.exec(c[1])?.[1]) + .filter((p): p is string => typeof p === "string"); +} + +const outPaths: string[] = []; + +async function capture(): Promise { + const out = await captureHarmonyScreenshotPng({ connectKey: "dev", scale: 1 }); + outPaths.push(out); + return out; +} + +beforeEach(() => { + runHdcShell.mockReset().mockResolvedValue({ stdout: "", exitCode: 0 }); + hdcFileRecv.mockReset(); +}); + +afterEach(async () => { + await Promise.all(outPaths.splice(0).map((p) => rm(p, { force: true }).catch(() => {}))); +}); + +describe("harmonyScreenCap viaDeviceTmp", () => { + it("removes the on-device capture even when the fetch fails", async () => { + // Without the finally-delete, every failed screenshot leaves a + // multi-hundred-KB PNG on /data/local/tmp, a partition nothing prunes. + hdcFileRecv.mockRejectedValue(new Error("[Fail]Error opening file: no such file")); + + await expect(captureHarmonyScreenshotPng({ connectKey: "dev", scale: 1 })).rejects.toThrow( + /Error opening file/ + ); + + const [remotePath] = capturedRemotePaths(); + expect(remotePath).toMatch(/^\/data\/local\/tmp\/argent-\d+-\d+\.png$/); + const commands = runHdcShell.mock.calls.map((c) => c[1]); + // The rm runs after the screenCap (and after the failed recv), against + // the exact path the capture wrote. + expect(commands).toContain(`rm -f '${remotePath}'`); + expect(commands.indexOf(`rm -f '${remotePath}'`)).toBeGreaterThan( + commands.findIndex((c) => c.includes("screenCap")) + ); + }); + + it("uses a distinct remote path for two concurrent captures", async () => { + // A fixed path would let one capture overwrite the other's between write + // and fetch, silently handing the loser the winner's screen. + hdcFileRecv.mockImplementation(async (_key, _remote, localPath) => { + await writeFile(localPath, tinyPng()); + }); + + const [a, b] = await Promise.all([capture(), capture()]); + + const paths = capturedRemotePaths(); + expect(paths).toHaveLength(2); + expect(paths[0]).not.toBe(paths[1]); + for (const p of paths) { + expect(p).toContain(`/data/local/tmp/argent-${process.pid}-`); + } + // Both captures completed as real, readable PNGs. + for (const out of [a, b]) { + const decoded = PNG.sync.read(await readFile(out)); + expect([decoded.width, decoded.height]).toEqual([4, 4]); + } + }); +}); + +describe("captureHarmonyScreenshotPng", () => { + it("removes the host-side raw capture when decoding it fails", async () => { + // The fetch "succeeds" but delivers garbage; the raw file must not be + // left behind in the host tmpdir on the error path. + hdcFileRecv.mockImplementation(async (_key, _remote, localPath) => { + await writeFile(localPath, Buffer.from("this is not a png")); + }); + + await expect(captureHarmonyScreenshotPng({ connectKey: "dev", scale: 1 })).rejects.toThrow(); + + // hdcFileRecv's localPath IS the raw intermediate; it must be gone. + const rawPath = hdcFileRecv.mock.calls[0]![2]; + expect(rawPath).toContain("argent-harmony-raw-"); + await expect(access(rawPath)).rejects.toThrow(); + }); +}); diff --git a/packages/tool-server/test/keyboard-error-taxonomy.test.ts b/packages/tool-server/test/keyboard-error-taxonomy.test.ts index a3cb9f17eb..2abaf8b17e 100644 --- a/packages/tool-server/test/keyboard-error-taxonomy.test.ts +++ b/packages/tool-server/test/keyboard-error-taxonomy.test.ts @@ -8,6 +8,9 @@ vi.mock("../src/utils/harmony-uitest", async (importOriginal) => ({ ...(await importOriginal()), harmonyTypeText: vi.fn(async () => {}), harmonyKeyEvent: vi.fn(async () => {}), + // The screen-awake guard reads the display first; stub it ON so these tests + // reach the validation they exercise. + harmonyDisplay: vi.fn(async () => ({ width: 1216, height: 2688, screenOn: true })), })); import { InvalidToolInputError } from "../src/utils/capability"; @@ -177,4 +180,27 @@ describe("keyboard backends — input rejection is a 400 with a uniform telemetr expect(harmonyTypeText).not.toHaveBeenCalled(); } }); + + it("harmony: newline in text → 400 + KEYBOARD_CHARACTER_UNSUPPORTED", async () => { + const harmonyDevice: DeviceInfo = { id: "harmony-KEY", platform: "harmony", kind: "device" }; + // `uitest uiInput text` validates almost nothing and answers `No Error` + // whether or not anything landed, so a newline must be rejected up front — + // every sibling backend (android above, vega) already is. + vi.mocked(harmonyTypeText).mockClear(); + await expectInvalidInput( + harmonyImpl.handler({}, { udid: harmonyDevice.id, text: "a\nb" }, harmonyDevice), + FAILURE_CODES.KEYBOARD_CHARACTER_UNSUPPORTED + ); + expect(harmonyTypeText).not.toHaveBeenCalled(); + }); + + it("harmony: a control character in text → 400 + KEYBOARD_CHARACTER_UNSUPPORTED", async () => { + const harmonyDevice: DeviceInfo = { id: "harmony-KEY", platform: "harmony", kind: "device" }; + vi.mocked(harmonyTypeText).mockClear(); + await expectInvalidInput( + harmonyImpl.handler({}, { udid: harmonyDevice.id, text: "a\x07b" }, harmonyDevice), + FAILURE_CODES.KEYBOARD_CHARACTER_UNSUPPORTED + ); + expect(harmonyTypeText).not.toHaveBeenCalled(); + }); }); diff --git a/packages/tool-server/test/keyboard-key-order.test.ts b/packages/tool-server/test/keyboard-key-order.test.ts index 21314cb7e9..db98080a9e 100644 --- a/packages/tool-server/test/keyboard-key-order.test.ts +++ b/packages/tool-server/test/keyboard-key-order.test.ts @@ -18,6 +18,9 @@ vi.mock("../src/utils/harmony-uitest", async (importOriginal) => ({ ...(await importOriginal()), harmonyTypeText: vi.fn(async () => {}), harmonyKeyEvent: vi.fn(async () => {}), + // The screen-awake guard reads the display first; stub it ON so these tests + // exercise the key path, not the panel check. + harmonyDisplay: vi.fn(async () => ({ width: 1216, height: 2688, screenOn: true })), })); import { injectVegaNamedKey, injectVegaText } from "../src/utils/vega-input"; diff --git a/packages/tool-server/test/list-devices-harmony.test.ts b/packages/tool-server/test/list-devices-harmony.test.ts new file mode 100644 index 0000000000..6c89c9c863 --- /dev/null +++ b/packages/tool-server/test/list-devices-harmony.test.ts @@ -0,0 +1,170 @@ +import { describe, it, expect, vi, beforeEach } from "vitest"; + +// Every sibling platform branch is stubbed empty at the module boundary so the +// merged list contains only what the HarmonyOS mocks define — the same +// isolation list-devices.test.ts gets from its child_process mock, but keyed +// to the two HarmonyOS discovery sources under test here. +vi.mock("../src/utils/adb", async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + listAndroidDevices: vi.fn(async () => []), + listAvds: vi.fn(async () => []), + }; +}); +vi.mock("../src/utils/ios-devices", () => ({ listIosSimulators: vi.fn(async () => []) })); +vi.mock("../src/utils/sim-remote", () => ({ + simctlListDevices: vi.fn(async () => ({ devices: {} })), +})); +vi.mock("../src/utils/chromium-discovery", async (importOriginal) => { + const actual = await importOriginal(); + return { ...actual, discoverChromiumDevices: vi.fn(async () => []) }; +}); +vi.mock("../src/utils/vega-devices", async (importOriginal) => { + const actual = await importOriginal(); + return { ...actual, listVegaDevices: vi.fn(async () => []) }; +}); +vi.mock("../src/utils/harmony-devices", () => ({ + listHarmonyInstances: vi.fn(async () => []), + listHarmonyHdcTargets: vi.fn(async () => []), +})); + +import { listDevicesTool } from "../src/tools/devices/list-devices"; +import { listHarmonyInstances, listHarmonyHdcTargets } from "../src/utils/harmony-devices"; + +const instances = vi.mocked(listHarmonyInstances); +const hdcTargets = vi.mocked(listHarmonyHdcTargets); + +type HarmonyEntry = { + platform: "harmony"; + udid: string; + name: string; + kind: "emulator" | "device"; + state: string; + connection?: string | null; + deviceType?: string | null; + osVersion?: string | null; +}; + +async function listHarmony(): Promise { + const result = await listDevicesTool.execute!({}, {}); + return result.devices.filter((d) => d.platform === "harmony") as HarmonyEntry[]; +} + +beforeEach(() => { + instances.mockReset().mockResolvedValue([]); + hdcTargets.mockReset().mockResolvedValue([]); +}); + +describe("list-devices HarmonyOS branch", () => { + it("maps a connected hdc target to kind 'device', keyed and named by its connectKey", async () => { + hdcTargets.mockResolvedValue([ + { connectKey: "FMR0223C13000654", connection: "TCP", state: "Offline" }, + { connectKey: "025DEK236V035771", connection: "USB", state: "Connected" }, + ]); + + const harmony = await listHarmony(); + + expect(harmony).toHaveLength(2); + // The id is `harmony-` — a `harmony-emulator-` id + // here is the id-swap mutation, and makes boot-device read a phone as an + // emulator instance. + const phone = harmony.find((d) => d.name === "025DEK236V035771"); + expect(phone).toEqual({ + platform: "harmony", + udid: "harmony-025DEK236V035771", + name: "025DEK236V035771", + kind: "device", + state: "Connected", + connection: "USB", + }); + // State is preserved verbatim — an Offline target must not be laundered + // into something readiness sorting would call drivable. + const offline = harmony.find((d) => d.name === "FMR0223C13000654"); + expect(offline).toMatchObject({ kind: "device", state: "Offline", connection: "TCP" }); + // Readiness: the Connected target sorts ahead of the Offline one despite + // being listed second by hdc. + expect(harmony[0]!.name).toBe("025DEK236V035771"); + }); + + it("maps DevEco instances to kind 'emulator' with running/stopped state and metadata", async () => { + instances.mockResolvedValue([ + { + name: "stopped_emu", + deviceType: "Foldable", + osVersion: "HarmonyOS 6.1.1(24)", + running: false, + }, + { + name: "running_emu", + deviceType: "Phone", + osVersion: "HarmonyOS 6.1.1(24)", + running: true, + }, + ]); + + const harmony = await listHarmony(); + + expect(harmony).toHaveLength(2); + const running = harmony.find((d) => d.name === "running_emu"); + // `harmony-emulator-` — never `harmony-`, which boot-device + // would read as a connect key and refuse to start. + expect(running).toEqual({ + platform: "harmony", + udid: "harmony-emulator-running_emu", + name: "running_emu", + kind: "emulator", + state: "running", + deviceType: "Phone", + osVersion: "HarmonyOS 6.1.1(24)", + }); + const stopped = harmony.find((d) => d.name === "stopped_emu"); + expect(stopped).toMatchObject({ + udid: "harmony-emulator-stopped_emu", + kind: "emulator", + state: "stopped", + deviceType: "Foldable", + }); + // readinessRank's harmony arm: a running instance (booted, drivable once + // registered) sorts ahead of a stopped one even though the manager listed + // the stopped one first. + expect(harmony.map((d) => d.name)).toEqual(["running_emu", "stopped_emu"]); + }); + + it("lists a running emulator under BOTH kinds — connected target and instance", async () => { + // The same booted emulator is visible to both discovery sources: hdc knows + // it by its connect key, the Emulator manager by its instance name. Both + // entries must appear, exactly as a running AVD appears in `adb devices` + // and the AVD list. + hdcTargets.mockResolvedValue([ + { connectKey: "127.0.0.1:10000", connection: "TCP", state: "Connected" }, + ]); + instances.mockResolvedValue([ + { + name: "stopped_emu", + deviceType: "Phone", + osVersion: "HarmonyOS 6.1.1(24)", + running: false, + }, + { + name: "booted_emu", + deviceType: "Phone", + osVersion: "HarmonyOS 6.1.1(24)", + running: true, + }, + ]); + + const harmony = await listHarmony(); + + expect(harmony).toHaveLength(3); + expect(harmony.filter((d) => d.kind === "device")).toHaveLength(1); + expect(harmony.filter((d) => d.kind === "emulator")).toHaveLength(2); + // The drivable entries — the Connected target and the running instance — + // both sort ahead of the stopped instance. + expect(harmony.map((d) => [d.kind, d.name, d.state])).toEqual([ + ["device", "127.0.0.1:10000", "Connected"], + ["emulator", "booted_emu", "running"], + ["emulator", "stopped_emu", "stopped"], + ]); + }); +}); From 95e81ad47671b961a92080b9f6c8e9db5e2918b0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 16:17:43 +0200 Subject: [PATCH 029/110] fix(harmony): stop open-url and restart-app reporting more than the device did A full E2E sweep of the thirteen harmony tools against the Mate 60 (HarmonyOS 6.0.1) found three defects, each reproduced on the handset and re-run after the fix: - open-url reported `opened: true` with no caveat for `tel:` and `mailto:`. Both print `start ability successfully.` and hand off to the system app selector, which puts up a modal "No options to open with" that covers the screen until it is dismissed. The caveat was gated on web URLs, so the two cases where nothing opens AND a dialog is left behind were the silent ones; it now rides on every URL, and the tool description no longer claims a HarmonyOS open fails whenever no app is registered - that is true only for a scheme nothing claims at all. - restart-app classified a bundle that is not installed as a subprocess failure, where launch-app's lookup of the same condition answers not_found. `aa force-stop` names it precisely (10104002), so the stop path classifies it the same way its twin does. - the not-running tolerance the last round added could not fire: `aa force-stop` answers `force stop process successfully.` for an app that is not running (measured against three never-launched bundles), so the branch was unreachable and its comment asserted a diagnostic the device does not produce. Removed - a cold restart-app still launches, verified by PID. Plus the pins the review round asked for and this one owes: terminateHarmonyApp had no test anywhere in the suite, and restart-app's stop-then-launch order and refusal are what the tool exists to guarantee. Both mutations - swallowing the stop, dropping the not_found classification - fail the new cases. screenshot's `scale` named iOS/Android as the platforms taking the 0.3 default while HarmonyOS and Vega take it through the same shared helper. --- .../tool-server/src/tools/open-url/index.ts | 2 +- .../src/tools/open-url/platforms/harmony.ts | 34 ++++---- .../tools/restart-app/platforms/harmony.ts | 25 ++---- .../tool-server/src/tools/screenshot/index.ts | 2 +- .../tool-server/src/utils/harmony-apps.ts | 40 ++++----- .../tool-server/test/harmony-apps.test.ts | 84 ++++++++++++++++++- .../tool-server/test/open-url-harmony.test.ts | 25 +++--- 7 files changed, 143 insertions(+), 69 deletions(-) diff --git a/packages/tool-server/src/tools/open-url/index.ts b/packages/tool-server/src/tools/open-url/index.ts index 233c7f0407..b4b4188ee4 100644 --- a/packages/tool-server/src/tools/open-url/index.ts +++ b/packages/tool-server/src/tools/open-url/index.ts @@ -59,7 +59,7 @@ export const openUrlTool: ToolDefinition = { Use to navigate to a web page or deep-link into an app. On Chromium, this navigates the primary renderer to the given URL. Cross-platform schemes: https://, tel:, mailto:. iOS also: messages://, settings://, maps://. Android also: geo:, plus any app-specific deep link. HarmonyOS resolves any scheme through \`aa start -U\`. Deep-linking caveat: an https:// link opens the native app only when an installed app is verified for the link's domain (iOS Universal Links / Android App Links) — otherwise it opens in the browser, and on iOS simulators it may open in Safari even when the owning app is installed. To reliably open an installed app, use its custom scheme (scheme://path) or launch-app with its bundle id. -Returns { opened, url, note? }. note carries the deep-linking caveat when a web URL was opened on a native device, and on HarmonyOS additionally warns that \`aa start -U\` reports success for a web URL even when nothing opens it - confirm with describe or screenshot before treating the link as followed. Fails if no app is registered to handle the URI (iOS/Android/HarmonyOS) or the renderer rejects the navigation (Chromium).`, +Returns { opened, url, note? }. note carries the deep-linking caveat when a web URL was opened on a native device; on HarmonyOS every URL carries a note, because \`aa start -U\` reports success whenever the system accepts the URI - a web URL can leave the screen unchanged, and a scheme the system routes to its app selector (tel:, mailto:) can leave a modal "No options to open with" dialog covering it - so confirm with describe or screenshot before treating the link as followed. Fails if no app is registered to handle the URI (iOS/Android, and on HarmonyOS only when nothing claims the scheme at all) or the renderer rejects the navigation (Chromium).`, zodSchema, capability, services: (params): Record => { diff --git a/packages/tool-server/src/tools/open-url/platforms/harmony.ts b/packages/tool-server/src/tools/open-url/platforms/harmony.ts index 11f93e1aaf..ee20e61f4a 100644 --- a/packages/tool-server/src/tools/open-url/platforms/harmony.ts +++ b/packages/tool-server/src/tools/open-url/platforms/harmony.ts @@ -9,22 +9,26 @@ import { httpDeepLinkNote } from "../deep-link-note"; * handler, which is exactly what this tool wants (and the one place implicit * start is the right verb; see `harmony-apps`). * - * A **custom scheme** with no registered handler is caught: `aa` prints - * `10103101 Failed to find a matching application for implicit launch`, which - * `openHarmonyUrl` turns into a throw. (It also leaves a "No options to open - * with" chooser on the device, which the thrown message says.) + * A scheme **no ability claims at all** is caught: `aa` prints `10103101 Failed + * to find a matching application for implicit launch`, which `openHarmonyUrl` + * turns into a throw. * - * A **web URL** is not, and cannot be. Measured on HarmonyOS 6.0.1: `aa start - * -U https://example.com` prints `start ability successfully.` and the - * foreground app does not change — no browser opens, no chooser appears, and - * nothing on the device distinguishes that from a handled link. So the caveat is - * stated rather than papered over; `opened: true` here means "the system - * accepted the URI", which on this platform is weaker than it sounds. + * What the system does accept is not caught, and cannot be. Measured on + * HarmonyOS 6.0.1, all printing `start ability successfully.`: + * `https://example.com` leaves the foreground app unchanged, and `tel:` / + * `mailto:` hand off to the system app selector, which puts up a modal "No + * options to open with" that covers the screen until it is dismissed. Nothing in + * `aa`'s output separates any of those from a link that was really followed, so + * the caveat rides on every URL rather than on web ones alone: `opened: true` + * here means "the system accepted the URI", which on this platform is weaker + * than it sounds. */ -const HARMONY_WEB_URL_CAVEAT = - "On HarmonyOS specifically, `aa start -U` reports success for a web URL even when nothing " + - "opens it — a device with no browser registered for https shows no chooser and stays on the " + - "current screen. Confirm with describe or screenshot before treating the link as followed."; +const HARMONY_OPEN_CAVEAT = + "On HarmonyOS specifically, `aa start -U` reports success whenever the system accepts the URI, " + + "so `opened: true` does not mean an app opened it: a web URL with no browser registered stays " + + "on the current screen, and a scheme the system hands to its app selector (measured with `tel:` " + + 'and `mailto:`) leaves a modal "No options to open with" dialog covering the screen. Confirm ' + + "with describe or screenshot, and dismiss any chooser, before treating the link as followed."; export const harmonyImpl: PlatformImpl = { requires: ["hdc"], @@ -34,7 +38,7 @@ export const harmonyImpl: PlatformImpl { const connectKey = harmonyConnectKey(device.id); - await terminateHarmonyApp(connectKey, params.bundleId).catch((err: unknown) => { - if (isHarmonyAppNotRunning(err)) return; - throw err; - }); + await terminateHarmonyApp(connectKey, params.bundleId); await launchHarmonyApp(connectKey, params.bundleId); return { restarted: true, bundleId: params.bundleId }; }, diff --git a/packages/tool-server/src/tools/screenshot/index.ts b/packages/tool-server/src/tools/screenshot/index.ts index db94e4c502..54012639dd 100644 --- a/packages/tool-server/src/tools/screenshot/index.ts +++ b/packages/tool-server/src/tools/screenshot/index.ts @@ -41,7 +41,7 @@ const zodSchema = z.object({ .max(1.0) .optional() .describe( - "Scale factor (0.01-1.0). Defaults to ARGENT_SCREENSHOT_SCALE env var, or 0.3 if unset for iOS/Android. " + + "Scale factor (0.01-1.0). Defaults to ARGENT_SCREENSHOT_SCALE env var, or 0.3 if unset for iOS/Android/HarmonyOS/Vega. " + "On Chromium the default is 1.0 (no downscale); pass <1 to opt in. Downscaling on Chromium requires the optional `sharp` dependency." ), includeImageInContext: z diff --git a/packages/tool-server/src/utils/harmony-apps.ts b/packages/tool-server/src/utils/harmony-apps.ts index da2f53c45e..5e68feb5b0 100644 --- a/packages/tool-server/src/utils/harmony-apps.ts +++ b/packages/tool-server/src/utils/harmony-apps.ts @@ -1,4 +1,4 @@ -import { FAILURE_CODES, FailureError, getFailureSignal } from "@argent/registry"; +import { FAILURE_CODES, FailureError } from "@argent/registry"; import { runHdcShell, shellQuote } from "./harmony-hdc"; /** @@ -132,19 +132,25 @@ export async function resolveHarmonyEntry( }; } -/** Stop a running app. See {@link terminateHarmonyApp} for why this is separate. */ -const AA_NOT_RUNNING_MARKER = "not running"; +/** + * `aa force-stop` says this when the package does not exist, which is + * `resolveHarmonyEntry`'s unknown-bundle case reached through the other verb. + * Both are the caller naming an app that is not installed, so both report + * `not_found` — telling an agent to fix the bundle id rather than to retry a + * subprocess that will fail identically forever. + */ +const AA_NO_SUCH_PACKAGE_CODE = "10104002"; /** * Stop every process of an app. * * `aa force-stop` takes the bundle name and, unlike `aa start`, needs no ability - * — there is nothing to resolve. - * - * Throws when the stop was refused for any reason other than the app not - * running: `restart-app` exists to guarantee a fresh process, and reporting - * `restarted: true` for an app that is still running with its old state is the - * one outcome it must not produce. + * — there is nothing to resolve. Stopping an app that is not running is not a + * failure to it: measured on 6.0.1 against three never-launched bundles, it + * answers `force stop process successfully.` So every `error:` it does print is + * a real refusal and propagates — `restart-app` exists to guarantee a fresh + * process, and reporting `restarted: true` for an app still running with its old + * state is the one outcome it must not produce. */ export async function terminateHarmonyApp(connectKey: string, bundleId: string): Promise { const { stdout } = await runHdcShell(connectKey, `aa force-stop ${shellQuote(bundleId)}`); @@ -155,26 +161,12 @@ export async function terminateHarmonyApp(connectKey: string, bundleId: string): error_code: FAILURE_CODES.HARMONY_ABILITY_START_FAILED, failure_stage: "harmony_force_stop", failure_area: "tool_server", - error_kind: "subprocess", + error_kind: stdout.includes(AA_NO_SUCH_PACKAGE_CODE) ? "not_found" : "subprocess", } ); } } -/** - * The one `aa force-stop` failure that is not a failure: stopping an app that - * is not running. A restart of an app that was never launched must still - * launch it — that is a start, not an error — so `restart-app` narrows its - * tolerance to this exact diagnostic rather than swallowing every refusal. - */ -export function isHarmonyAppNotRunning(error: unknown): boolean { - return ( - error instanceof FailureError && - getFailureSignal(error)?.failure_stage === "harmony_force_stop" && - error.message.includes(AA_NOT_RUNNING_MARKER) - ); -} - /** * The spelling of the main ability that `aa start -a` accepts — see the note on * `resolveHarmonyEntry`. diff --git a/packages/tool-server/test/harmony-apps.test.ts b/packages/tool-server/test/harmony-apps.test.ts index e0344ae995..d308709114 100644 --- a/packages/tool-server/test/harmony-apps.test.ts +++ b/packages/tool-server/test/harmony-apps.test.ts @@ -1,6 +1,13 @@ import { describe, it, expect, vi, beforeEach } from "vitest"; +import { getFailureSignal, type DeviceInfo } from "@argent/registry"; import { runHdcShell as realRunHdcShell } from "../src/utils/harmony-hdc"; -import { resolveHarmonyEntry, launchHarmonyApp, openHarmonyUrl } from "../src/utils/harmony-apps"; +import { + resolveHarmonyEntry, + launchHarmonyApp, + openHarmonyUrl, + terminateHarmonyApp, +} from "../src/utils/harmony-apps"; +import { harmonyImpl as harmonyRestartImpl } from "../src/tools/restart-app/platforms/harmony"; // Only the transport is faked: these tests are about what the module makes of // `bm`/`aa` output, and both CLIs report failure on stdout while exiting 0. @@ -233,6 +240,81 @@ describe("launchHarmonyApp", () => { }); }); +describe("terminateHarmonyApp", () => { + it("succeeds on the success line, which is also what a not-running app gets", async () => { + // Measured on 6.0.1 against three never-launched bundles: `aa force-stop` + // reports success rather than a not-running error, which is why restart-app + // tolerates nothing here. + runHdcShell.mockResolvedValueOnce(ok("force stop process successfully.")); + await expect(terminateHarmonyApp("dev", "com.huawei.hmos.calculator")).resolves.toBeUndefined(); + expect(runHdcShell.mock.calls[0][1]).toBe("aa force-stop 'com.huawei.hmos.calculator'"); + }); + + it("throws on an `error:` line, even though `aa` exits 0", async () => { + runHdcShell.mockResolvedValueOnce( + ok("error: failed to force stop process.\nError Code:16000050 Error Message:Internal error.") + ); + await expect(terminateHarmonyApp("dev", "c")).rejects.toThrow(/Failed to stop 'c'.*16000050/s); + }); + + it("reports a missing bundle as not_found, like the launch path's lookup", async () => { + // The same condition through the other verb: `resolveHarmonyEntry` answers + // not_found for a bundle that is not installed, so a stop must not tell the + // agent to retry a subprocess that can never succeed. + runHdcShell.mockResolvedValueOnce( + ok( + "error: failed to force stop process.\nError Code:10104002 Error Message:Failed to retrieve specified package information." + ) + ); + const err = await terminateHarmonyApp("dev", "com.example.nope").catch((e: unknown) => e); + expect(getFailureSignal(err as Error)?.error_kind).toBe("not_found"); + }); + + it("keeps a non-package stop failure classified as a subprocess failure", async () => { + runHdcShell.mockResolvedValueOnce( + ok("error: failed to force stop process.\nError Code:16000050 Error Message:Internal error.") + ); + const err = await terminateHarmonyApp("dev", "c").catch((e: unknown) => e); + expect(getFailureSignal(err as Error)?.error_kind).toBe("subprocess"); + }); +}); + +describe("restart-app on harmony", () => { + const device: DeviceInfo = { id: "harmony-KEY", platform: "harmony", kind: "device" }; + + it("does not launch — and does not report a restart — when the stop was refused", async () => { + // The whole point of the tool: a refused stop leaves the old process up, so + // the launch that follows would no-op on it and `restarted: true` would name + // a process that never went away. + runHdcShell.mockResolvedValueOnce( + ok("error: failed to force stop process.\nError Code:16000050 Error Message:Internal error.") + ); + await expect( + harmonyRestartImpl.handler({}, { udid: device.id, bundleId: "c" }, device) + ).rejects.toThrow(/Failed to stop 'c'/); + expect(runHdcShell).toHaveBeenCalledTimes(1); + }); + + it("stops and then launches, reporting the bundle it restarted", async () => { + runHdcShell + .mockResolvedValueOnce(ok("force stop process successfully.")) + .mockResolvedValueOnce(ok(CALCULATOR)) + .mockResolvedValueOnce(ok("start ability successfully.")); + await expect( + harmonyRestartImpl.handler( + {}, + { udid: device.id, bundleId: "com.huawei.hmos.calculator" }, + device + ) + ).resolves.toEqual({ restarted: true, bundleId: "com.huawei.hmos.calculator" }); + expect(runHdcShell.mock.calls.map((c) => c[1])).toEqual([ + "aa force-stop 'com.huawei.hmos.calculator'", + "bm dump -n 'com.huawei.hmos.calculator'", + "aa start -b 'com.huawei.hmos.calculator' -a 'CalculatorAbility' -m 'phone'", + ]); + }); +}); + describe("openHarmonyUrl", () => { it("rejects a URI no app claims", async () => { runHdcShell.mockResolvedValueOnce( diff --git a/packages/tool-server/test/open-url-harmony.test.ts b/packages/tool-server/test/open-url-harmony.test.ts index 735eae05c8..aa389baad1 100644 --- a/packages/tool-server/test/open-url-harmony.test.ts +++ b/packages/tool-server/test/open-url-harmony.test.ts @@ -28,7 +28,7 @@ beforeEach(() => { openHarmonyUrlMock.mockResolvedValue(undefined); }); -describe("open-url HarmonyOS handler surfaces the caveat only for web URLs", () => { +describe("open-url HarmonyOS handler caveats every URL it reports as opened", () => { it("attaches note for an https URL, appended to the shared web-URL note", async () => { const url = "https://bsky.app/profile/tvpworld.bsky.social"; const res = await harmonyImpl.handler({}, { udid: device.id, url }, device); @@ -39,7 +39,7 @@ describe("open-url HarmonyOS handler surfaces the caveat only for web URLs", () // both halves are load-bearing, and on this platform the caveat is the only // signal that `opened: true` may mean nothing happened on screen. expect(res.note).toContain(httpDeepLinkNote(url)); - expect(res.note).toMatch(/aa start -U. reports success for a web URL even when nothing opens/); + expect(res.note).toMatch(/aa start -U. reports success whenever the system accepts the URI/); expect(res.note?.startsWith(httpDeepLinkNote(url)!)).toBe(true); // No stray separator or stringified `undefined` from the composition. expect(res.note).not.toMatch(/undefined|^\s|\s\s|\s$/); @@ -48,16 +48,19 @@ describe("open-url HarmonyOS handler surfaces the caveat only for web URLs", () expect(openHarmonyUrlMock).toHaveBeenCalledWith(CONNECT_KEY, url); }); - it("omits note for a custom-scheme deep link", async () => { - const res = await harmonyImpl.handler( - {}, - { udid: device.id, url: "bluesky://profile/x" }, - device - ); + it("attaches the caveat alone for a non-web scheme, which the shared note skips", async () => { + // A scheme the system routes to its app selector reports success exactly + // like a handled one: measured on 6.0.1, `tel:12345` and `mailto:a@b.com` + // both print `start ability successfully.` and leave a modal "No options to + // open with" on screen. Only a scheme nothing claims at all throws + // (10103101), so `opened: true` needs the caveat here too. + const res = await harmonyImpl.handler({}, { udid: device.id, url: "tel:12345" }, device); expect(res.opened).toBe(true); - // A custom scheme no app claims fails loudly on HarmonyOS (10103101), so - // there is nothing here for a caveat to hedge about. - expect(res.note).toBeUndefined(); + expect(httpDeepLinkNote("tel:12345")).toBeUndefined(); + expect(res.note).toMatch(/aa start -U. reports success whenever the system accepts the URI/); + expect(res.note).toMatch(/No options to open with/); + // The shared web-URL note must not be pasted onto a non-web scheme. + expect(res.note).not.toMatch(/Universal Links/); }); it("propagates a device-side failure instead of resolving with a note", async () => { From 173ffe5b67122e085c8805ff1017e28cc14a6e05 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 16:23:11 +0200 Subject: [PATCH 030/110] fix(button): say which buttons the platform does take The refusal named only the button that failed. On HarmonyOS that is four of the seven - `uitest uiInput keyEvent` exposes Home, Back and Power and nothing else - so an agent that asked for volumeUp learned which one is missing and nothing about what to reach for instead. The sibling `keyboard` already names its supported keys on the same class of refusal; the set is the map this guard already reads, so it now rides along: button 'volumeUp' is not available on harmony. Supported: home, back, power Verified against the Mate 60. The platforms whose set is empty (Chromium, Vega) are refused by the capability gate before this guard, so the list is never empty here. --- packages/tool-server/src/tools/button/index.ts | 11 +++++++++-- packages/tool-server/test/button.test.ts | 6 ++++++ 2 files changed, 15 insertions(+), 2 deletions(-) diff --git a/packages/tool-server/src/tools/button/index.ts b/packages/tool-server/src/tools/button/index.ts index d32734c289..60b291e961 100644 --- a/packages/tool-server/src/tools/button/index.ts +++ b/packages/tool-server/src/tools/button/index.ts @@ -100,11 +100,18 @@ Fails if the device backend is not reachable — the simulator-server for iOS, \ }, async execute(services, params) { const device = resolveDevice(params.udid); - if (!BUTTONS_BY_PLATFORM[device.platform].has(params.button)) { + const available = BUTTONS_BY_PLATFORM[device.platform]; + if (!available.has(params.button)) { + // Name the set, as the sibling `keyboard` does for an unsupported key: + // HarmonyOS accepts three of the seven buttons, so a refusal that only + // says which one failed leaves an agent guessing at the other six. The + // platforms whose set is empty are refused by the capability gate above, + // so this list never is. throw new UnsupportedOperationError( "button", device, - `button '${params.button}' is not available on ${device.platform}` + `button '${params.button}' is not available on ${device.platform}. ` + + `Supported: ${[...available].join(", ")}` ); } if (device.platform === "android") { diff --git a/packages/tool-server/test/button.test.ts b/packages/tool-server/test/button.test.ts index b5a2e1e872..b96b03786e 100644 --- a/packages/tool-server/test/button.test.ts +++ b/packages/tool-server/test/button.test.ts @@ -162,6 +162,12 @@ describe("button tool — per-platform validation", () => { await expect( buttonTool.execute(services, { udid: harmonyUdid, button: "volumeUp" }) ).rejects.toBeInstanceOf(UnsupportedOperationError); + // The refusal names what IS available, like the sibling keyboard tool's + // unsupported-key message — four of the seven buttons are refused here, so + // a message that only names the failing one leaves the caller guessing. + await expect( + buttonTool.execute(services, { udid: harmonyUdid, button: "volumeUp" }) + ).rejects.toThrow(/Supported: home, back, power/); expect(harmonyKeyEvent).not.toHaveBeenCalled(); }); From 509deac01965527881eb0ac3cbb895cf0862db17 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 16:32:08 +0200 Subject: [PATCH 031/110] fix(harmony): match aa's own error-code line, not the bare digits The stop path read a missing package off `10104002` anywhere in stdout. The bundle id is caller input and `aa` can echo it back, so a bundle named `com.example.a10104002` failing for an unrelated reason would have been reported as not installed. It now matches `Error Code:10104002`, the prefix aa prints it under - the reason `emulatorFailure` matches verified markers rather than a bare "fail" substring. Restoring the bare number fails the new case. --- packages/tool-server/src/utils/harmony-apps.ts | 8 ++++++-- packages/tool-server/test/harmony-apps.test.ts | 13 +++++++++++++ 2 files changed, 19 insertions(+), 2 deletions(-) diff --git a/packages/tool-server/src/utils/harmony-apps.ts b/packages/tool-server/src/utils/harmony-apps.ts index 5e68feb5b0..9e75690c35 100644 --- a/packages/tool-server/src/utils/harmony-apps.ts +++ b/packages/tool-server/src/utils/harmony-apps.ts @@ -138,8 +138,12 @@ export async function resolveHarmonyEntry( * Both are the caller naming an app that is not installed, so both report * `not_found` — telling an agent to fix the bundle id rather than to retry a * subprocess that will fail identically forever. + * + * Carries `aa`'s own `Error Code:` prefix rather than the bare digits, for the + * reason `emulatorFailure` matches verified markers: the bundle id is caller + * input, and a bare number could be forged by one that contains it. */ -const AA_NO_SUCH_PACKAGE_CODE = "10104002"; +const AA_NO_SUCH_PACKAGE = "Error Code:10104002"; /** * Stop every process of an app. @@ -161,7 +165,7 @@ export async function terminateHarmonyApp(connectKey: string, bundleId: string): error_code: FAILURE_CODES.HARMONY_ABILITY_START_FAILED, failure_stage: "harmony_force_stop", failure_area: "tool_server", - error_kind: stdout.includes(AA_NO_SUCH_PACKAGE_CODE) ? "not_found" : "subprocess", + error_kind: stdout.includes(AA_NO_SUCH_PACKAGE) ? "not_found" : "subprocess", } ); } diff --git a/packages/tool-server/test/harmony-apps.test.ts b/packages/tool-server/test/harmony-apps.test.ts index d308709114..eedf4f5aef 100644 --- a/packages/tool-server/test/harmony-apps.test.ts +++ b/packages/tool-server/test/harmony-apps.test.ts @@ -277,6 +277,19 @@ describe("terminateHarmonyApp", () => { const err = await terminateHarmonyApp("dev", "c").catch((e: unknown) => e); expect(getFailureSignal(err as Error)?.error_kind).toBe("subprocess"); }); + + it("cannot be talked into not_found by a bundle id carrying the digits", async () => { + // The bundle id is caller input and `aa` can echo it, so the marker carries + // the `Error Code:` prefix — matching on the bare number would let this + // internal failure forge a not-installed verdict. + runHdcShell.mockResolvedValueOnce( + ok( + "error: failed to force stop process for com.example.a10104002.\nError Code:16000050 Error Message:Internal error." + ) + ); + const err = await terminateHarmonyApp("dev", "com.example.a10104002").catch((e: unknown) => e); + expect(getFailureSignal(err as Error)?.error_kind).toBe("subprocess"); + }); }); describe("restart-app on harmony", () => { From 64a05baf717a1223b6720d562373049da07c9ade Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 17:29:23 +0200 Subject: [PATCH 032/110] fix(harmony): resolve the key a restarted instance comes back on MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The pre-start snapshot kept every row `hdc` lists, in any state. A stopped instance leaves its row behind as `127.0.0.1:5555 TCP Offline` and comes back on the same port, so that key was already in the snapshot and could never read as an arrival: measured on the device, the second boot of an instance and every `force` restart spent the whole budget and returned `harmony-emulator-`, an id no interaction tool accepts, under a note blaming a boot that had in fact finished in seconds. The snapshot now records which keys are `Connected` — the state that means "a guest this call did not start", since by then the instance is known to be down. `Offline` is also what a connection blip leaves behind, so an arrival is confirmed against the panel the instance is configured with: `hw.lcd.single.width`/`height` are the numbers the booted guest reports back as its `render resolution`, measured identical on a 6.1.1 phone image, and they are the only join between an instance and a connect key. The check is a filter and never a requirement — an instance with no single-panel config skips it, a target that cannot be probed yet stays pending — and a target that answers with someone else's panel gets its own diagnosis rather than the budget's. Two more, from the same path: two confirmed arrivals are refused at once rather than polled to the deadline and then possibly un-refused when one flaps; and a key is only handed back once the guest answers `uitest`, since a row reaches `Connected` before the window service does and until it does every interaction tool fails against an id that looks drivable. Measured, same command and same stale-row precondition: 30.0s to an undrivable id, then 4.9s to `harmony-127.0.0.1:5555`. A forced restart takes 14.0s and `describe` on the returned id succeeds 0.7s later. --- .../src/tools/devices/boot-device.ts | 229 ++++++++++++++---- .../tool-server/src/utils/harmony-devices.ts | 28 ++- .../test/boot-device-harmony.test.ts | 217 ++++++++++++++++- .../tool-server/test/harmony-devices.test.ts | 27 ++- .../test/list-devices-harmony.test.ts | 4 + 5 files changed, 446 insertions(+), 59 deletions(-) diff --git a/packages/tool-server/src/tools/devices/boot-device.ts b/packages/tool-server/src/tools/devices/boot-device.ts index cf4b020133..196d19945c 100644 --- a/packages/tool-server/src/tools/devices/boot-device.ts +++ b/packages/tool-server/src/tools/devices/boot-device.ts @@ -1,5 +1,6 @@ import { execFile, spawn, type StdioOptions } from "node:child_process"; import { openSync, closeSync, readFileSync } from "node:fs"; +import { rm } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { promisify } from "node:util"; @@ -59,6 +60,7 @@ import { } from "../../utils/harmony-cli"; import { listHarmonyHdcTargets, listHarmonyInstances } from "../../utils/harmony-devices"; import { resolveHdc } from "../../utils/harmony-hdc"; +import { harmonyDisplay, harmonyDumpLayout } from "../../utils/harmony-uitest"; import { bootElectronApp, type ElectronBootResult } from "./boot-electron"; const execFileAsync = promisify(execFile); @@ -1544,6 +1546,28 @@ const HARMONY_AMBIGUOUS_TARGET = "tell which is the instance it started and did not guess. Call `list-devices` and drive the " + '`kind: "device"` entry for the instance, or stop the other emulator and re-run.'; +/** + * Said when every target that registered answered with some other device's + * panel. Distinct from {@link HARMONY_NO_TARGET} in the same way + * {@link HARMONY_AMBIGUOUS_TARGET} is: something did register, so a longer + * budget cannot help, and the caller needs to know a target was seen and + * declined rather than that none appeared. + */ +const HARMONY_MISMATCHED_TARGET = + "A target registered while the instance was booting, but its display is not the panel this " + + "instance is configured with, so it is another device rather than the one argent started. " + + 'Call `list-devices` and drive the `kind: "device"` entry for the instance once it appears.'; + +/** + * Said when the instance registered but never answered `uitest` in time. The + * connect key is returned regardless — it is the right id, and an interaction + * tool retried by hand may well work — so this is a caveat, not a failure. + */ +const HARMONY_NOT_DRIVABLE = + "The instance registered with `hdc` but was still not answering `uitest` when the boot budget " + + "ran out, so `describe`, `screenshot` and the gesture tools may fail for a little longer. " + + "Retry the first interaction, or give the boot longer with `bootTimeoutMs`."; + /** Said when the instance started but the connector that would reach it is missing. */ const HARMONY_NO_HDC = "The instance started, but `hdc` was not found, so argent cannot tell which target it " + @@ -1557,19 +1581,45 @@ async function connectedHarmonyTargets() { } /** - * Every connect key `hdc` lists, in ANY state. + * The connect keys `hdc` reports as `Connected` right now. + * + * The pre-start snapshot records that STATE, not mere membership in the + * listing. A stopped instance leaves its row behind indefinitely + * (`127.0.0.1:5555 TCP Offline localhost`, measured across `-stop`) and a + * restart re-registers on the same port, so a snapshot of every listed key can + * never see that instance arrive: every second boot of an instance this `hdc` + * daemon has already seen, and every `force` restart, would spend the whole + * budget concluding nothing registered and hand back an id no interaction tool + * accepts. * - * The pre-start snapshot must be taken against all rows, not only `Connected` - * ones: a target that is listed but `Offline` when the snapshot is taken reads - * as an arrival the moment it reconnects — and `Offline` is what a still- - * booting instance shows, what a stopped one leaves behind, and what any - * connection blip produces (a row was observed flipping back to `Connected` - * unattended when its guest rebooted mid-session). Adopting that reconnection - * hands back a device this call did not start. + * A key that is `Connected` here belongs to a guest this call did not start — + * by this point the instance is known to be down, either because the manager + * reported it stopped or because `-stop` was waited out — so excluding those, + * and only those, is what leaves arrival meaning something. + * + * `Offline` is also what a connection blip leaves behind, so a foreign device + * reconnecting is an arrival by this rule too. {@link waitForHarmonyTarget} + * settles that with the panel the instance is configured with. */ -async function knownHarmonyKeys(): Promise> { - const targets = await listHarmonyHdcTargets().catch(() => []); - return new Set(targets.map((t) => t.connectKey)); +async function connectedHarmonyKeys(): Promise> { + return new Set((await connectedHarmonyTargets()).map((t) => t.connectKey)); +} + +/** + * Whether two panels are the same one, whichever way round it is composited. + * + * The guest reports its display as currently oriented, the manager as + * configured, so comparing the axes pairwise would read a landscape instance as + * a different device. + */ +function sameHarmonyPanel( + a: { width: number; height: number }, + b: { width: number; height: number } +): boolean { + return ( + Math.min(a.width, a.height) === Math.min(b.width, b.height) && + Math.max(a.width, a.height) === Math.max(b.width, b.height) + ); } /** @@ -1604,7 +1654,8 @@ async function startHarmonyEmulator( if (!bin) { throw new Error( "The HarmonyOS `Emulator` manager was not found. Install DevEco Studio, or set " + - "`$DEVECO_STUDIO_HOME` to its install root, then retry." + "`$DEVECO_STUDIO_HOME` to the directory holding `tools/emulator/Emulator` (on macOS " + + "the app bundle itself works), then retry." ); } const logPath = join(tmpdir(), `argent-harmony-${instanceName.replace(/[^\w.-]/g, "_")}.log`); @@ -1659,44 +1710,107 @@ async function startHarmonyEmulator( * a start that failed after the spawn returned would be indistinguishable from * one still booting, and would burn the whole budget before saying so. * - * Exactly one *eligible* arrival, or none. Arrival alone does not identify + * Exactly one *confirmed* arrival, or none. Arrival alone does not identify * anything once a second device can produce one, and returning the wrong key is * not a failed boot — it is a drivable id, so every later tap and keystroke * lands on that device instead. So USB arrivals are excluded outright (a - * host-local emulator is not on a cable) and two eligible arrivals are refused - * rather than guessed between, as the Android path refuses when more than one - * serial appears. There is no `-avdName` equivalent to disambiguate with, so the - * caller gets the named no-target degradation and `list-devices`. + * host-local emulator is not on a cable), a candidate is confirmed against + * `expected` — the panel the instance is configured with, which the guest + * reports back as its `render resolution` — and two confirmed arrivals are + * refused rather than guessed between, as the Android path refuses when more + * than one serial appears. + * + * The panel check is a filter and never a requirement. An instance whose config + * does not describe a single panel passes `expected: null` and is not checked at + * all, and a target that cannot be probed yet is left pending for the next poll + * rather than rejected — a row reaches `Connected` before its render service + * answers. Only a target that answers with someone else's panel is out, and + * then for good: re-probing it could not change the answer and would cost a + * round trip per poll. * - * What this does NOT cover: a second emulator registering during the window but - * outside it — started from DevEco Studio, or by a concurrent `boot-device` for - * another instance, which `inFlightHarmonyBoots` does not coalesce. Both are - * loopback TCP and land seconds apart, so each boot sees a single arrival and - * the cardinality check never fires. Nothing in `hdc` or the manager ties a key - * to an instance, so closing that needs a lock across every harmony boot on the - * host, which would head-of-line block them all on one slow start. + * What this does NOT cover: two instances sharing one device profile, whose + * panels are identical by construction; and a second emulator registering + * during the window but outside it — started from DevEco Studio, or by a + * concurrent `boot-device` for another instance, which `inFlightHarmonyBoots` + * does not coalesce. Nothing in `hdc` or the manager ties a key to an instance, + * so closing those needs a lock across every harmony boot on the host, which + * would head-of-line block them all on one slow start. */ async function waitForHarmonyTarget( before: Set, deadline: number, - checkAlive: () => void -): Promise<{ key: string | null; ambiguous: boolean }> { - let ambiguous = false; + checkAlive: () => void, + expected: { width: number; height: number } | null +): Promise<{ key: string | null; ambiguous: boolean; mismatched: boolean }> { + const rejected = new Set(); for (;;) { const arrived = (await connectedHarmonyTargets()).filter( - (t) => !before.has(t.connectKey) && couldBeHarmonyEmulator(t) + (t) => !before.has(t.connectKey) && couldBeHarmonyEmulator(t) && !rejected.has(t.connectKey) ); - if (arrived.length === 1) return { key: arrived[0]!.connectKey, ambiguous: false }; - // Two eligible fresh arrivals at once is a refusal to guess, not a slow - // boot — remember it so the caller can say so instead of blaming the budget. - if (arrived.length > 1) ambiguous = true; + const confirmed: string[] = []; + for (const target of arrived) { + if (!expected) { + confirmed.push(target.connectKey); + continue; + } + const display = await harmonyDisplay(target.connectKey).catch(() => null); + if (!display) continue; + if (sameHarmonyPanel(display, expected)) confirmed.push(target.connectKey); + else rejected.add(target.connectKey); + } + if (confirmed.length === 1) { + return { key: confirmed[0]!, ambiguous: false, mismatched: false }; + } + // Two confirmed arrivals is a refusal to guess, not a slow boot, and it is + // answered here rather than latched: polling on would spend the rest of the + // budget on a decision already made, and would turn the refusal into a + // guess the moment one of the two flapped and left the other alone. + if (confirmed.length > 1) return { key: null, ambiguous: true, mismatched: false }; checkAlive(); const remaining = deadline - Date.now(); - if (remaining <= 0) return { key: null, ambiguous }; + if (remaining <= 0) { + return { key: null, ambiguous: false, mismatched: rejected.size > 0 }; + } await new Promise((r) => setTimeout(r, Math.min(HARMONY_TARGET_POLL_MS, remaining))); } } +/** + * Wait until the guest answers `uitest`, answering whether it did. + * + * A target reports `Connected` as soon as `hdc` can reach the guest's daemon, + * which is not the same as the window service being up: `uitest dumpLayout` + * answers `DumpLayout failed:Get window nodes failed` for a window after that, + * so a key returned on `Connected` alone can fail every interaction tool for + * seconds while looking drivable. Every other platform gates its boot on a + * readiness signal — `sys.boot_completed` and a first frame on Android, + * `simctl bootstatus` on iOS, `waitForVvdRunning` on Vega — and this is + * HarmonyOS'. + * + * The probe is a real `dumpLayout` rather than something cheaper because that + * is the call the gap is in; `hidumper` answers throughout it. + */ +async function waitForHarmonyDrivable(connectKey: string, deadline: number): Promise { + const probePath = join(tmpdir(), `argent-harmony-bootprobe-${process.hrtime.bigint()}.json`); + try { + for (;;) { + if ( + await harmonyDumpLayout(connectKey, probePath).then( + () => true, + () => false + ) + ) { + return true; + } + const remaining = deadline - Date.now(); + if (remaining <= 0) return false; + await new Promise((r) => setTimeout(r, Math.min(HARMONY_TARGET_POLL_MS, remaining))); + } + } finally { + await rm(probePath, { force: true }).catch(() => {}); + } +} + /** Resolve when the manager has exited, or when `graceMs` has passed. */ async function waitForHarmonyExit( emulator: { exited: () => unknown }, @@ -1778,13 +1892,17 @@ async function bootHarmonyImpl(params: { } } - // Snapshot immediately before the start, against EVERY row `hdc` lists — - // Connected or not — so a target that was already known (another emulator, a - // phone on USB, a stale Offline row) is excluded by identity rather than by - // any assumption about how an emulator's key is spelled. Only a target that - // registers FRESH after the start can be the instance just booted; a row that - // was already listed and merely reconnects is not an arrival. - const before = await knownHarmonyKeys(); + // Snapshot immediately before the start, so a target already driveable + // (another emulator, a phone on USB) is excluded by identity rather than by + // any assumption about how an emulator's key is spelled. Reaching `Connected` + // after this point is what marks the instance just booted — including on the + // key it held before, since a stopped instance's row survives as `Offline`. + const before = await connectedHarmonyKeys(); + // The panel this instance is configured with, which the guest reports back as + // its `render resolution` — the only thing that separates the instance + // reconnecting from another device doing the same. Read before the start + // because the listing is already in hand; a config's panel does not change. + const expectedPanel = instances?.find((i) => i.name === params.instanceName)?.display ?? null; const emulator = await startHarmonyEmulator(params.instanceName); @@ -1842,24 +1960,33 @@ async function bootHarmonyImpl(params: { } const outcome = hdcAvailable - ? await waitForHarmonyTarget(before, bootDeadline, assertEmulatorAlive) - : { key: null, ambiguous: false }; + ? await waitForHarmonyTarget(before, bootDeadline, assertEmulatorAlive, expectedPanel) + : { key: null, ambiguous: false, mismatched: false }; const connectKey = outcome.key; + // Registered is not driveable. The remaining budget rather than a slice of + // it: the key is already in hand, so there is nothing else left to spend it + // on, and an expiry is reported rather than walked into. + const drivable = connectKey ? await waitForHarmonyDrivable(connectKey, bootDeadline) : false; + + const note = connectKey + ? drivable + ? undefined + : HARMONY_NOT_DRIVABLE + : !hdcAvailable + ? HARMONY_NO_HDC + : outcome.ambiguous + ? HARMONY_AMBIGUOUS_TARGET + : outcome.mismatched + ? HARMONY_MISMATCHED_TARGET + : HARMONY_NO_TARGET; + return { platform: "harmony", udid: connectKey ? harmonyDeviceId(connectKey) : harmonyEmulatorId(params.instanceName), instanceName: params.instanceName, booted: true, - ...(connectKey - ? {} - : { - note: !hdcAvailable - ? HARMONY_NO_HDC - : outcome.ambiguous - ? HARMONY_AMBIGUOUS_TARGET - : HARMONY_NO_TARGET, - }), + ...(note ? { note } : {}), }; } diff --git a/packages/tool-server/src/utils/harmony-devices.ts b/packages/tool-server/src/utils/harmony-devices.ts index 030a23798d..117c44fe26 100644 --- a/packages/tool-server/src/utils/harmony-devices.ts +++ b/packages/tool-server/src/utils/harmony-devices.ts @@ -33,11 +33,25 @@ interface HarmonyInstance { /** e.g. `HarmonyOS 6.1.1(24)`. */ osVersion: string | null; running: boolean; + /** + * The panel the instance is configured with, or null when its config does not + * describe one in this shape — a multi-display profile keys its LCDs + * differently, and nothing here should assume otherwise. + * + * Load-bearing beyond metadata: these are the same numbers the booted guest + * then reports as its `render resolution` (measured 1320x2856 on a HarmonyOS + * 6.1.1 phone image, matching `hw.lcd.single.width`/`height` exactly), which + * is the only join there is between an instance and the `hdc` connect key it + * registers under. `boot-device` uses it to tell its own instance from + * another device that reconnected beside it. + */ + display: { width: number; height: number } | null; } /** * One `Emulator -list -details` record. Every value is a string, including the - * booleans — `isRunning` is `"true"`/`"false"`, never a JSON boolean, the same + * booleans and the numbers — `isRunning` is `"true"`/`"false"` and + * `hw.lcd.single.width` is `"1320"`, never a JSON boolean or number, the same * quirk `-imageList` has with `downloaded`. */ interface RawInstance { @@ -46,10 +60,19 @@ interface RawInstance { "deviceType"?: unknown; "isRunning"?: unknown; "os.osVersion"?: unknown; + "hw.lcd.single.width"?: unknown; + "hw.lcd.single.height"?: unknown; } const str = (v: unknown): string | null => (typeof v === "string" && v.length > 0 ? v : null); +/** A positive pixel count from the manager's all-strings JSON, or null. */ +const px = (v: unknown): number | null => { + if (typeof v !== "string") return null; + const n = Number.parseInt(v, 10); + return Number.isInteger(n) && n > 0 ? n : null; +}; + /** * Instances from `Emulator -list -details`. * @@ -82,11 +105,14 @@ export function parseHarmonyInstances(stdout: string): HarmonyInstance[] { for (const entry of raw as RawInstance[]) { const name = str(entry?.name) ?? str(entry?.deviceName); if (!name) continue; + const width = px(entry?.["hw.lcd.single.width"]); + const height = px(entry?.["hw.lcd.single.height"]); out.push({ name, deviceType: str(entry?.deviceType), osVersion: str(entry?.["os.osVersion"]), running: entry?.isRunning === "true", + display: width && height ? { width, height } : null, }); } return out; diff --git a/packages/tool-server/test/boot-device-harmony.test.ts b/packages/tool-server/test/boot-device-harmony.test.ts index 256c7af1bf..522c29c780 100644 --- a/packages/tool-server/test/boot-device-harmony.test.ts +++ b/packages/tool-server/test/boot-device-harmony.test.ts @@ -12,6 +12,8 @@ const listHarmonyHdcTargets = vi.fn(); const resolveHdc = vi.fn(); const ensureDep = vi.fn(); const spawnMock = vi.fn(); +const harmonyDisplay = vi.fn(); +const harmonyDumpLayout = vi.fn(); vi.mock("node:child_process", async () => { const actual = await vi.importActual("node:child_process"); @@ -37,6 +39,16 @@ vi.mock("../src/utils/harmony-hdc", async () => { ); return { ...actual, resolveHdc: (...a: unknown[]) => resolveHdc(...a) }; }); +vi.mock("../src/utils/harmony-uitest", async () => { + const actual = await vi.importActual( + "../src/utils/harmony-uitest" + ); + return { + ...actual, + harmonyDisplay: (...a: unknown[]) => harmonyDisplay(...a), + harmonyDumpLayout: (...a: unknown[]) => harmonyDumpLayout(...a), + }; +}); vi.mock("../src/utils/check-deps", async () => { const actual = await vi.importActual("../src/utils/check-deps"); @@ -55,6 +67,15 @@ const PHONE = { connectKey: "025DEK236V035771", connection: "USB", state: "Conne /** What a booted emulator registers as, measured on a HarmonyOS 6.1.1 phone image. */ const EMULATOR_KEY = "127.0.0.1:5555"; const emulatorTarget = { connectKey: EMULATOR_KEY, connection: "TCP", state: "Connected" }; +/** + * The row a stopped instance leaves behind. Measured: `-stop` takes the guest + * down but never removes the row, so `127.0.0.1:5555 TCP Offline localhost` + * survives in `hdc list targets -v` until the daemon is killed — and a restart + * comes back on that same port. + */ +const staleEmulatorTarget = { connectKey: EMULATOR_KEY, connection: "TCP", state: "Offline" }; +/** `hw.lcd.single.width`/`height` of the phone image, echoed by the guest's `render resolution`. */ +const PANEL = { width: 1320, height: 2856 }; const logPath = join(tmpdir(), `argent-harmony-${INSTANCE}.log`); /** Stands in for the detached `Emulator -start`, which normally never exits. */ @@ -97,8 +118,18 @@ beforeEach(() => { resolveHdc.mockResolvedValue("/Applications/DevEco-Studio.app/.../hdc"); runHarmonyEmulator.mockResolvedValue({ stdout: "", stderr: "" }); listHarmonyInstances.mockResolvedValue([ - { name: INSTANCE, deviceType: "Phone", osVersion: "HarmonyOS 6.1.1(24)", running: false }, + { + name: INSTANCE, + deviceType: "Phone", + osVersion: "HarmonyOS 6.1.1(24)", + running: false, + display: PANEL, + }, ]); + // Every target answers with the instance's own panel unless a case says + // otherwise, and every guest is driveable as soon as it is Connected. + harmonyDisplay.mockResolvedValue({ ...PANEL, screenOn: true }); + harmonyDumpLayout.mockResolvedValue({ attributes: {} }); targets([PHONE]); }); @@ -169,6 +200,99 @@ describe("boot-device — HarmonyOS emulator path", () => { expect(((await pending) as { udid: string }).udid).toBe(`harmony-${EMULATOR_KEY}`); }); + it("resolves the key an instance comes back on, though its row never left the listing", async () => { + // The second boot of any instance this `hdc` daemon has already seen. Its + // row is still listed — `Offline`, left there by the previous `-stop` — and + // it re-registers on the same port, so a snapshot that kept every listed key + // could never see it arrive: measured on the device as a full budget spent + // and `harmony-emulator-argent_phone` handed back, an id no interaction tool + // accepts, under a note blaming a boot that had in fact finished in seconds. + targets([PHONE, staleEmulatorTarget], [PHONE, staleEmulatorTarget], [PHONE, emulatorTarget]); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 30_000 }); + await vi.advanceTimersByTimeAsync(6_000); + const result = (await pending) as { udid: string; note?: string }; + + expect(result.udid).toBe(`harmony-${EMULATOR_KEY}`); + expect(result.note).toBeUndefined(); + }); + + it("declines a fresh arrival whose panel is another device's", async () => { + // `Offline` is also what a connection blip leaves behind, so a foreign + // device reconnecting is an arrival by state alone. The instance's + // configured panel is what separates the two: a wearable answers 466x466 + // where this phone image answers 1320x2856. + const WEARABLE = { connectKey: "127.0.0.1:5561", connection: "TCP", state: "Connected" }; + harmonyDisplay.mockImplementation((key: string) => + Promise.resolve( + key === WEARABLE.connectKey + ? { width: 466, height: 466, screenOn: true } + : { ...PANEL, screenOn: true } + ) + ); + targets([PHONE], [PHONE, WEARABLE], [PHONE, WEARABLE, emulatorTarget]); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 30_000 }); + await vi.advanceTimersByTimeAsync(6_000); + const result = (await pending) as { udid: string; note?: string }; + + expect(result.udid).toBe(`harmony-${EMULATOR_KEY}`); + expect(result.note).toBeUndefined(); + }); + + it("matches the instance's panel whichever way round the guest composites it", async () => { + // The manager reports the panel as configured, the guest as currently + // oriented, so a landscape instance would read as a different device if the + // axes were compared pairwise. + harmonyDisplay.mockResolvedValue({ width: PANEL.height, height: PANEL.width, screenOn: true }); + targets([PHONE], [PHONE, emulatorTarget]); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 30_000 }); + await vi.advanceTimersByTimeAsync(6_000); + + expect(((await pending) as { udid: string }).udid).toBe(`harmony-${EMULATOR_KEY}`); + }); + + it("says a target was seen and declined rather than that none registered", async () => { + // The panel filter's own failure mode. Blaming the budget would be the + // wrong advice twice over: something did register, and a longer one cannot + // change what it answered. + const WEARABLE = { connectKey: "127.0.0.1:5561", connection: "TCP", state: "Connected" }; + harmonyDisplay.mockResolvedValue({ width: 466, height: 466, screenOn: true }); + targets([PHONE], [PHONE, WEARABLE]); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 30_000 }); + await vi.advanceTimersByTimeAsync(31_000); + const result = (await pending) as { udid: string; note?: string }; + + expect(result.udid).toBe(`harmony-emulator-${INSTANCE}`); + expect(result.note).toMatch(/not the panel this instance is configured with/); + expect(result.note).not.toMatch(/bootTimeoutMs/); + }); + + it("leaves a target that cannot be probed yet pending rather than rejecting it", async () => { + // A row reaches `Connected` before its render service answers. Treating an + // unreadable probe as a mismatch would reject the instance permanently for + // being early. + let probes = 0; + harmonyDisplay.mockImplementation(() => + probes++ < 2 + ? Promise.reject(new Error("[Fail][E001005] Device not found or connected")) + : Promise.resolve({ ...PANEL, screenOn: true }) + ); + targets([PHONE], [PHONE, emulatorTarget]); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 30_000 }); + await vi.advanceTimersByTimeAsync(10_000); + + expect(((await pending) as { udid: string }).udid).toBe(`harmony-${EMULATOR_KEY}`); + }); + it("does not adopt a cable-attached handset that arrives during the boot", async () => { // A second HarmonyOS target reaches Connected inside the boot window - a // phone plugged in or authorised mid-boot, or a still-settling row flipping @@ -223,6 +347,77 @@ describe("boot-device — HarmonyOS emulator path", () => { expect(result.note).toMatch(/two|both|more than one|could not tell which/i); }); + it("refuses at once rather than spending the budget on a decision already made", async () => { + // The refusal is settled the moment both are seen. Polling on would bill the + // caller the whole 3-minute default for it — and would turn the refusal into + // a guess if one of the two dropped out and left the other alone. + const OTHER_EMULATOR = { connectKey: "127.0.0.1:5557", connection: "TCP", state: "Connected" }; + targets([PHONE], [PHONE, emulatorTarget, OTHER_EMULATOR]); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 180_000 }); + // Two polls' worth, three orders of magnitude short of the budget. + await vi.advanceTimersByTimeAsync(5_000); + const result = (await pending) as { udid: string; note?: string }; + + expect(result.udid).toBe(`harmony-emulator-${INSTANCE}`); + expect(result.note).toMatch(/More than one/); + }); + + it("keeps refusing after one of the two ambiguous targets drops away", async () => { + // A still-settling emulator is the likelier of the two to flap, so the + // survivor is the likelier to be the other device. Once both have been seen + // the answer cannot be un-made by one leaving. + const OTHER_EMULATOR = { connectKey: "127.0.0.1:5557", connection: "TCP", state: "Connected" }; + targets([PHONE], [PHONE, emulatorTarget, OTHER_EMULATOR], [PHONE, OTHER_EMULATOR]); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 30_000 }); + await vi.advanceTimersByTimeAsync(31_000); + const result = (await pending) as { udid: string; note?: string }; + + expect(result.udid).toBe(`harmony-emulator-${INSTANCE}`); + expect(result.note).toMatch(/More than one/); + }); + + it("waits for the guest to answer `uitest`, not merely for `hdc` to reach it", async () => { + // A target reports `Connected` as soon as the daemon is reachable; the + // window service comes up after it, and until it does every interaction tool + // fails against an id that looks drivable. + let probes = 0; + harmonyDumpLayout.mockImplementation(() => + probes++ < 3 + ? Promise.reject(new Error("DumpLayout failed:Get window nodes failed")) + : Promise.resolve({ attributes: {} }) + ); + targets([PHONE], [PHONE, emulatorTarget]); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 60_000 }); + await vi.advanceTimersByTimeAsync(20_000); + const result = (await pending) as { udid: string; note?: string }; + + expect(probes).toBeGreaterThan(3); + expect(result.udid).toBe(`harmony-${EMULATOR_KEY}`); + expect(result.note).toBeUndefined(); + }); + + it("still hands back the key when the guest never answers, and says so", async () => { + // The key is the right id either way, so an unresponsive guest is a caveat + // rather than a failure — but a silent one would put the caller's first + // interaction failure down to the wrong cause. + harmonyDumpLayout.mockRejectedValue(new Error("DumpLayout failed:Get window nodes failed")); + targets([PHONE], [PHONE, emulatorTarget]); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 30_000 }); + await vi.advanceTimersByTimeAsync(31_000); + const result = (await pending) as { udid: string; note?: string }; + + expect(result.udid).toBe(`harmony-${EMULATOR_KEY}`); + expect(result.note).toMatch(/still not answering `uitest`/); + }); + it("names the instance and says why when nothing registers within the budget", async () => { targets([PHONE]); vi.useFakeTimers(); @@ -341,16 +536,26 @@ describe("boot-device — HarmonyOS emulator path", () => { // `-stop` returns before the emulator is gone, so the restart waits on // `isRunning` — and for the whole budget, since how long the instance then // takes to go down is unpredictable (9s to ~70s measured). Reported running - // for 40s here, past any fixed grace. The snapshot is taken only once it is - // down, which is what lets a restart that lands on the port it had before - // still read as an arrival. + // for 40s here, past any fixed grace. + // + // The stopped instance is modelled as an `Offline` ROW rather than an absent + // one, because that is what a device does: `-stop` takes the guest down and + // leaves `127.0.0.1:5555 TCP Offline` behind. A fixture where the key + // vanishes and reappears cannot fail, since the key is then fresh whatever + // the snapshot kept. let listed = 0; listHarmonyInstances.mockImplementation(() => Promise.resolve([ - { name: INSTANCE, deviceType: "Phone", osVersion: null, running: listed++ < 21 }, + { + name: INSTANCE, + deviceType: "Phone", + osVersion: null, + running: listed++ < 21, + display: PANEL, + }, ]) ); - targets([PHONE], [PHONE], [PHONE, emulatorTarget]); + targets([PHONE, staleEmulatorTarget], [PHONE, staleEmulatorTarget], [PHONE, emulatorTarget]); vi.useFakeTimers(); const pending = boot({ force: true, bootTimeoutMs: 120_000 }); diff --git a/packages/tool-server/test/harmony-devices.test.ts b/packages/tool-server/test/harmony-devices.test.ts index 5cf2ca905f..27aee5be3f 100644 --- a/packages/tool-server/test/harmony-devices.test.ts +++ b/packages/tool-server/test/harmony-devices.test.ts @@ -34,6 +34,8 @@ const EMULATOR_ONE_INSTANCE = `[ "deviceName": "argent_probe", "deviceType": "Phone", "hw.hdc.port": "10000", + "hw.lcd.single.height": "2856", + "hw.lcd.single.width": "1320", "isRunning": "false", "name": "argent_probe", "os.apiVersion": "24", @@ -71,17 +73,40 @@ describe("parseHarmonyInstances", () => { expect(parseHarmonyInstances(EMULATOR_NO_INSTANCES)).toEqual([]); }); - it("reads name, form factor, OS version and running state", () => { + it("reads name, form factor, OS version, running state and panel", () => { expect(parseHarmonyInstances(EMULATOR_ONE_INSTANCE)).toEqual([ { name: "argent_probe", deviceType: "Phone", osVersion: "HarmonyOS 6.1.1(24)", running: false, + display: { width: 1320, height: 2856 }, }, ]); }); + it("reads the panel as numbers, from the strings the manager emits", () => { + // `boot-device` compares this against the `render resolution` the booted + // guest reports — measured identical on a 6.1.1 phone image — to tell its own + // instance from another device that reconnected beside it. Left as the + // manager's strings it would never equal anything. + const [one] = parseHarmonyInstances(EMULATOR_ONE_INSTANCE); + expect(one.display).toEqual({ width: 1320, height: 2856 }); + }); + + it("reports no panel rather than a broken one when the config does not describe a single LCD", () => { + // A multi-display profile keys its LCDs differently, so the keys can be + // absent on a perfectly good instance. `boot-device` skips the panel check + // entirely for a null, which is why a partial or zero reading must not + // masquerade as a measurement. + const partial = `[{ "name": "n", "isRunning": "false", "hw.lcd.single.width": "1320" }]`; + const zeroed = `[{ "name": "n", "isRunning": "false", "hw.lcd.single.width": "0", "hw.lcd.single.height": "2856" }]`; + const absent = `[{ "name": "n", "isRunning": "false" }]`; + for (const raw of [partial, zeroed, absent]) { + expect(parseHarmonyInstances(raw)[0].display).toBeNull(); + } + }); + it("reads isRunning as the string it actually is, not a JSON boolean", () => { // The manager emits every value as a string, `isRunning` included. Comparing // it as a boolean would make every instance read as stopped, so a booted diff --git a/packages/tool-server/test/list-devices-harmony.test.ts b/packages/tool-server/test/list-devices-harmony.test.ts index 6c89c9c863..1a1af7f9a4 100644 --- a/packages/tool-server/test/list-devices-harmony.test.ts +++ b/packages/tool-server/test/list-devices-harmony.test.ts @@ -94,12 +94,14 @@ describe("list-devices HarmonyOS branch", () => { deviceType: "Foldable", osVersion: "HarmonyOS 6.1.1(24)", running: false, + display: null, }, { name: "running_emu", deviceType: "Phone", osVersion: "HarmonyOS 6.1.1(24)", running: true, + display: null, }, ]); @@ -145,12 +147,14 @@ describe("list-devices HarmonyOS branch", () => { deviceType: "Phone", osVersion: "HarmonyOS 6.1.1(24)", running: false, + display: null, }, { name: "booted_emu", deviceType: "Phone", osVersion: "HarmonyOS 6.1.1(24)", running: true, + display: null, }, ]); From 57a3e8ed484f5dec2c1293c85f2f2f8996d5c3f8 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 17:29:38 +0200 Subject: [PATCH 033/110] fix(harmony): refuse a button press the suspended panel would swallow MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `button` was the one HarmonyOS input backend with no awake check, so a press against a suspended display reported `{pressed: "home"}` having done nothing — while `gesture-tap`, `gesture-swipe` and `keyboard` all refuse. A screen timeout mid-session is routine and `home`/`back` are the keys an agent recovers with, so the two disagreed exactly where it matters. `power` stays exempt: it is what the refusal tells the caller to wake the device with. `keyboard` also read the display before validating anything, so an unreachable device rewrote an unsupported-key 400 into "check that USB debugging is authorised" — cable advice for a key this backend will never support. Both pure checks now run first; nothing is injected until they pass. --- .../tool-server/src/tools/button/index.ts | 20 +++++- .../src/tools/keyboard/platforms/harmony.ts | 18 ++--- packages/tool-server/test/button.test.ts | 71 ++++++++++++++++++- .../test/keyboard-error-taxonomy.test.ts | 39 ++++++++-- 4 files changed, 130 insertions(+), 18 deletions(-) diff --git a/packages/tool-server/src/tools/button/index.ts b/packages/tool-server/src/tools/button/index.ts index 60b291e961..84b3dd246e 100644 --- a/packages/tool-server/src/tools/button/index.ts +++ b/packages/tool-server/src/tools/button/index.ts @@ -2,7 +2,11 @@ import { z } from "zod"; import type { Platform, ServiceRef, ToolCapability, ToolDefinition } from "@argent/registry"; import { simulatorServerRef, type SimulatorServerApi } from "../../blueprints/simulator-server"; import { resolveDevice, harmonyConnectKey } from "../../utils/device-info"; -import { harmonyKeyEvent } from "../../utils/harmony-uitest"; +import { + assertHarmonyScreenAwake, + harmonyDisplay, + harmonyKeyEvent, +} from "../../utils/harmony-uitest"; import { UnsupportedOperationError } from "../../utils/capability"; import { sendCommand } from "../../utils/simulator-client"; import { ANDROID_BUTTON_KEYCODES, injectAndroidKeycode } from "../../utils/android-input"; @@ -136,7 +140,19 @@ Fails if the device backend is not reachable — the simulator-server for iOS, \ // sim-server for HarmonyOS). The BUTTONS_BY_PLATFORM guard above // guarantees a key name exists for every accepted button. await ensureDep("hdc"); - await harmonyKeyEvent(harmonyConnectKey(device.id), HARMONY_BUTTON_KEYS[params.button]!); + const connectKey = harmonyConnectKey(device.id); + // `uitest uiInput keyEvent` answers `No Error` against a suspended panel + // while the press lands nowhere, so `home` and `back` share the guard the + // tap, swipe and typing backends use: a screen timeout mid-session must + // not turn the keys an agent recovers with into silent no-ops. + // + // `power` is exempt, and exempt before the display read — it is what the + // refusal tells the caller to wake the device with, and the one key that + // works while the panel is suspended. + if (params.button !== "power") { + assertHarmonyScreenAwake(await harmonyDisplay(connectKey), `press ${params.button}`); + } + await harmonyKeyEvent(connectKey, HARMONY_BUTTON_KEYS[params.button]!); return { pressed: params.button }; } const api = services.simulatorServer as SimulatorServerApi; diff --git a/packages/tool-server/src/tools/keyboard/platforms/harmony.ts b/packages/tool-server/src/tools/keyboard/platforms/harmony.ts index d797be640b..cf7a22401e 100644 --- a/packages/tool-server/src/tools/keyboard/platforms/harmony.ts +++ b/packages/tool-server/src/tools/keyboard/platforms/harmony.ts @@ -92,17 +92,19 @@ function resolveHarmonyKeycode(key: string): number { } async function typeHarmony(connectKey: string, params: KeyboardParams): Promise { - // Typing into a suspended panel reports `No Error` and lands nowhere — check - // before resolving anything, so a dead screen fails the same way a tap does. - assertHarmonyScreenAwake(await harmonyDisplay(connectKey), "type"); - // Resolve before injecting anything: a combined key+text call with an unknown - // key name must reject having typed nothing, not type the text and then 400. + // Both checks are pure and both precede the first device round trip: an + // unsupported key or an un-typeable character is the caller's mistake, and + // asking the device first lets an unreachable one rewrite that 400 into a + // connection error about a key that will never be supported. Nothing is + // injected until both pass, so a combined key+text call with an unknown key + // rejects having typed nothing. const keycode = params.key ? resolveHarmonyKeycode(params.key) : null; + if (params.text) assertTypeableHarmonyText(params.text); + // Typing into a suspended panel reports `No Error` and lands nowhere, so a + // dead screen fails the same way a tap does. + assertHarmonyScreenAwake(await harmonyDisplay(connectKey), "type"); let keysPressed = 0; if (params.text) { - // Validate before injecting: `uitest` reports `No Error` whether or not the - // text landed, so an un-typeable character must be caught here (see above). - assertTypeableHarmonyText(params.text); // `uitest uiInput text` types into whatever holds focus, in one shot — there // is no per-character injection, so `delayMs` has nothing to pace (the tool // description already lists the platforms that ignore it). diff --git a/packages/tool-server/test/button.test.ts b/packages/tool-server/test/button.test.ts index b96b03786e..0eaa32819d 100644 --- a/packages/tool-server/test/button.test.ts +++ b/packages/tool-server/test/button.test.ts @@ -1,4 +1,5 @@ -import { describe, it, expect, vi } from "vitest"; +import { beforeEach, describe, it, expect, vi } from "vitest"; +import { getFailureSignal } from "@argent/registry"; // Keep the real module (blueprints import from it too) but neutralise the // fire-and-forget WebSocket send so no real socket is opened during the test. @@ -15,10 +16,13 @@ vi.mock("../src/utils/android-input", async (importOriginal) => ({ })); // HarmonyOS presses go over `uitest uiInput keyEvent` on the device; neutralise -// it for the same reason as the adb call above. +// it for the same reason as the adb call above. `harmonyDisplay` is the power +// state the press is gated on, stubbed awake by default (see beforeEach); +// `assertHarmonyScreenAwake` stays real so the tests see the guard's own refusal. vi.mock("../src/utils/harmony-uitest", async (importOriginal) => ({ ...(await importOriginal()), harmonyKeyEvent: vi.fn(), + harmonyDisplay: vi.fn(), })); // The Android branch preflights adb via `ensureDep("adb")` before injecting. @@ -34,7 +38,7 @@ import { buttonTool, BUTTONS_BY_PLATFORM } from "../src/tools/button"; import { UnsupportedOperationError } from "../src/utils/capability"; import { ANDROID_BUTTON_KEYCODES, injectAndroidKeycode } from "../src/utils/android-input"; import { DependencyMissingError, ensureDep } from "../src/utils/check-deps"; -import { harmonyKeyEvent } from "../src/utils/harmony-uitest"; +import { harmonyDisplay, harmonyKeyEvent } from "../src/utils/harmony-uitest"; import { sendCommand } from "../src/utils/simulator-client"; const iosUdid = "AAAAAAAA-AAAA-AAAA-AAAA-AAAAAAAAAAAA"; @@ -57,6 +61,14 @@ const HARMONY_KEY_NAMES: Record = { power: "Power", }; +/** A Mate 60's render resolution, awake. */ +const HARMONY_AWAKE = { width: 1216, height: 2688, screenOn: true }; +const HARMONY_SUSPENDED = { ...HARMONY_AWAKE, screenOn: false }; + +beforeEach(() => { + vi.mocked(harmonyDisplay).mockReset().mockResolvedValue(HARMONY_AWAKE); +}); + describe("button tool — per-platform validation", () => { it("rejects `back` on iOS (no hardware back button) instead of a silent no-op", async () => { await expect( @@ -215,6 +227,59 @@ describe("button tool — per-platform validation", () => { }); }); +describe("button tool — a suspended HarmonyOS panel", () => { + // `uitest uiInput keyEvent` answers `No Error` against a display that is OFF + // or SUSPEND while the press lands nowhere: measured on a 6.1.1 emulator, a + // `Home` keyEvent injected into a suspended panel left Settings foreground + // after waking. `gesture-tap`, `gesture-swipe` and `keyboard` all refuse in + // that state, and `home`/`back` are the keys an agent reaches for to recover + // from a screen timeout. + beforeEach(() => { + vi.mocked(harmonyKeyEvent).mockClear(); + vi.mocked(ensureDep).mockClear(); + }); + + for (const button of ["home", "back"] as const) { + it(`refuses \`${button}\` while the display is suspended, injecting nothing`, async () => { + vi.mocked(harmonyDisplay).mockResolvedValue(HARMONY_SUSPENDED); + const err = await buttonTool.execute(services, { udid: harmonyUdid, button }).then( + () => { + throw new Error("expected the press to reject, but it resolved"); + }, + (e: unknown) => e + ); + // The same guard the tap refusal comes from, so the two agree at the + // moment it matters — not a second, divergent message. + expect(getFailureSignal(err)?.failure_stage).toBe("harmony_screen_off"); + expect((err as Error).message).toMatch(/display is off/); + // The refusal names the button, not "tap": the shared helper takes the + // verb from its caller. + expect((err as Error).message).toContain(`press ${button}`); + expect(harmonyKeyEvent).not.toHaveBeenCalled(); + }); + } + + it("still presses `power` while the display is suspended — it is the way back", async () => { + // The refusal above tells the caller to wake the device with `button` + // (power). Gating power on the same check would make that advice + // unfollowable, so it is carved out — and carved out before the display + // read, so waking a device does not depend on the render service answering. + vi.mocked(harmonyDisplay).mockResolvedValue(HARMONY_SUSPENDED); + await expect( + buttonTool.execute(services, { udid: harmonyUdid, button: "power" }) + ).resolves.toEqual({ pressed: "power" }); + expect(harmonyKeyEvent).toHaveBeenCalledWith(harmonyConnectKey, "Power"); + expect(harmonyDisplay).not.toHaveBeenCalled(); + }); + + it("presses `home` as usual once the display is awake", async () => { + await expect( + buttonTool.execute(services, { udid: harmonyUdid, button: "home" }) + ).resolves.toEqual({ pressed: "home" }); + expect(harmonyKeyEvent).toHaveBeenCalledWith(harmonyConnectKey, "Home"); + }); +}); + describe("button tool — service declaration", () => { it("does not declare the simulator-server service for an Android target", () => { // Android presses go over adb; declaring sim-server would needlessly resolve + diff --git a/packages/tool-server/test/keyboard-error-taxonomy.test.ts b/packages/tool-server/test/keyboard-error-taxonomy.test.ts index 2abaf8b17e..1b0c031e18 100644 --- a/packages/tool-server/test/keyboard-error-taxonomy.test.ts +++ b/packages/tool-server/test/keyboard-error-taxonomy.test.ts @@ -1,4 +1,4 @@ -import { describe, it, expect, vi } from "vitest"; +import { beforeEach, describe, it, expect, vi } from "vitest"; import { Registry, FAILURE_CODES, getFailureSignal, type DeviceInfo } from "@argent/registry"; // The harmony backend rejects an unknown key before injecting, so these tests @@ -8,16 +8,16 @@ vi.mock("../src/utils/harmony-uitest", async (importOriginal) => ({ ...(await importOriginal()), harmonyTypeText: vi.fn(async () => {}), harmonyKeyEvent: vi.fn(async () => {}), - // The screen-awake guard reads the display first; stub it ON so these tests - // reach the validation they exercise. - harmonyDisplay: vi.fn(async () => ({ width: 1216, height: 2688, screenOn: true })), + // The screen-awake guard reads the display; the beforeEach below stubs it ON + // so these tests reach the validation they exercise. + harmonyDisplay: vi.fn(), })); import { InvalidToolInputError } from "../src/utils/capability"; import { typeSimulatorServer } from "../src/tools/keyboard/simulator-server-keys"; import { makeChromiumImpl } from "../src/tools/keyboard/platforms/chromium"; import { harmonyImpl } from "../src/tools/keyboard/platforms/harmony"; -import { harmonyKeyEvent, harmonyTypeText } from "../src/utils/harmony-uitest"; +import { harmonyDisplay, harmonyKeyEvent, harmonyTypeText } from "../src/utils/harmony-uitest"; import { injectVegaNamedKey, injectVegaText } from "../src/utils/vega-input"; import { injectAndroidNamedKey, injectAndroidText } from "../src/utils/android-input"; @@ -63,6 +63,12 @@ const chromiumDevice = { kind: "app", } as unknown as DeviceInfo; +beforeEach(() => { + vi.mocked(harmonyDisplay) + .mockReset() + .mockResolvedValue({ width: 1216, height: 2688, screenOn: true }); +}); + describe("keyboard backends — input rejection is a 400 with a uniform telemetry taxonomy", () => { it("iOS: unknown key → 400 + KEYBOARD_KEY_UNSUPPORTED", async () => { await expectInvalidInput( @@ -192,6 +198,8 @@ describe("keyboard backends — input rejection is a 400 with a uniform telemetr FAILURE_CODES.KEYBOARD_CHARACTER_UNSUPPORTED ); expect(harmonyTypeText).not.toHaveBeenCalled(); + // Decided without asking the device: see the unsupported-key case below. + expect(harmonyDisplay).not.toHaveBeenCalled(); }); it("harmony: a control character in text → 400 + KEYBOARD_CHARACTER_UNSUPPORTED", async () => { @@ -202,5 +210,26 @@ describe("keyboard backends — input rejection is a 400 with a uniform telemetr FAILURE_CODES.KEYBOARD_CHARACTER_UNSUPPORTED ); expect(harmonyTypeText).not.toHaveBeenCalled(); + expect(harmonyDisplay).not.toHaveBeenCalled(); + }); + + it("harmony: an unsupported key is diagnosed without reaching the device", async () => { + const harmonyDevice: DeviceInfo = { id: "harmony-KEY", platform: "harmony", kind: "device" }; + // `tab` is one of the keys this backend deliberately does not map, so the + // answer never depends on the device — the unreachable display stubbed + // below stands in for one that is unhappy. A device round trip ahead of the + // key check turns this 400 into `hdc could not reach HarmonyOS device …: + // Device not found or connected`, sending the caller to check a cable over + // a key that will never be supported (hubgan review). + vi.mocked(harmonyKeyEvent).mockClear(); + vi.mocked(harmonyDisplay).mockRejectedValue( + new Error("[Fail][E001005] Device not found or connected") + ); + await expectInvalidInput( + harmonyImpl.handler({}, { udid: harmonyDevice.id, key: "tab" }, harmonyDevice), + FAILURE_CODES.KEYBOARD_KEY_UNSUPPORTED + ); + expect(harmonyDisplay).not.toHaveBeenCalled(); + expect(harmonyKeyEvent).not.toHaveBeenCalled(); }); }); From bc44e3a1060db74cc8b8b5c27b1d3132e60cb5ba Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 17:29:38 +0200 Subject: [PATCH 034/110] fix(harmony): keep a layout container that knows something itself MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The scaffolding hoist dropped the container's own `identifier` along with `disabled`, `focused` and `selected` — the loss `ownCarriesState` already prevents on the same-rect collapse just below it. ArkUI puts `.id()` on plain `Column`s and `Row`s, and that string is exactly what an `await-ui-element {identifier}` selector targets, so a dropped one is unrecoverable. Measured on a 6.1.1 Settings screen: 29 container-typed nodes carried a developer id and 14 were hoisted away with it. The hoist now tests the same predicate the collapse does. Pure scaffolding — no id, no label, not clickable — is still walked through. --- .../platforms/harmony/layout-parser.ts | 34 +++++++----------- .../test/harmony-layout-parser.test.ts | 36 +++++++++++++++++++ 2 files changed, 49 insertions(+), 21 deletions(-) diff --git a/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts b/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts index 4edb0b1618..62a7ff6dc6 100644 --- a/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts +++ b/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts @@ -111,15 +111,6 @@ export function harmonyLabel(attrs: Record): string { return (attrs.hint ?? "").trim(); } -function isInteractive(attrs: Record): boolean { - return ( - isTrue(attrs, "clickable") || - isTrue(attrs, "longClickable") || - isTrue(attrs, "checkable") || - isScrollable(attrs) - ); -} - interface HarmonyNode { role: string; pixelBounds: PixelRect | null; @@ -173,14 +164,6 @@ function build(node: HarmonyLayoutNode): HarmonyNode[] { } const label = harmonyLabel(attrs); - const interactive = isInteractive(attrs); - - // Scaffolding contributes nothing itself; hoist whatever survived beneath it. - // A container that is *itself* clickable or labelled is a real target wearing - // a layout type (ArkUI builds buttons out of `Stack`/`Row`), so it stays. - if (LAYOUT_CONTAINERS.has(type) && !interactive && !label) { - return children; - } if (!hasArea(bounds) && children.length === 0) return []; @@ -197,6 +180,15 @@ function build(node: HarmonyLayoutNode): HarmonyNode[] { const identifier = (attrs.id ?? "").trim() || (attrs.key ?? "").trim(); if (identifier) own.identifier = identifier; + // Scaffolding contributes nothing itself; hoist whatever survived beneath it. + // A container that is *itself* clickable, labelled, or carrying something the + // subtree beneath it does not know is a real node wearing a layout type: ArkUI + // builds buttons out of `Stack`/`Row`, and puts `.id()` — the string an + // `await-ui-element {identifier}` selects on — on plain `Column`s. + if (LAYOUT_CONTAINERS.has(type) && !label && !own.clickable && !ownCarriesState(own)) { + return children; + } + // A wrapper whose only child covers exactly the same rect is a duplicate // layer: collapse to whichever of the two carries the label. // @@ -218,12 +210,12 @@ function build(node: HarmonyLayoutNode): HarmonyNode[] { } /** - * Whether a node knows something its same-rect child cannot: an identifier to + * Whether a node knows something the subtree beneath it cannot: an identifier to * select on, or any state flag other than `clickable` (which the collapse - * merges). `clickable` alone still marks a plain duplicate layer. + * merges onto the surviving child, and which the hoist tests separately). * - * `value` is deliberately absent: it is only ever set alongside a label, which - * the caller's `!own.label` already keeps out of the collapse. + * `value` is deliberately absent: it is only ever set alongside a label, and + * both callers already check for one. */ function ownCarriesState(n: HarmonyNode): boolean { return Boolean( diff --git a/packages/tool-server/test/harmony-layout-parser.test.ts b/packages/tool-server/test/harmony-layout-parser.test.ts index e7ddb017c0..9728922b76 100644 --- a/packages/tool-server/test/harmony-layout-parser.test.ts +++ b/packages/tool-server/test/harmony-layout-parser.test.ts @@ -4,6 +4,7 @@ import { parseHarmonyBounds, parseHarmonyLayout, } from "../src/tools/describe/platforms/harmony/layout-parser"; +import { findAll } from "../src/utils/ui-tree-match"; import type { HarmonyLayoutNode } from "../src/utils/harmony-uitest"; import type { DescribeNode } from "../src/tools/describe/contract"; @@ -117,6 +118,41 @@ describe("parseHarmonyLayout", () => { expect(target.clickable).toBe(true); }); + // ArkUI puts `.id()` and the state flags on whatever component the author + // wrote, and that is routinely a plain `Column`/`Row`. Hoisting one away for + // wearing a layout type takes the only copy of that id with it. + describe("layout containers that know something themselves", () => { + const wrapping = (attrs: Record): DescribeNode => + parseHarmonyLayout( + root([ + node({ type: "root", bundleName: "com.app", bounds: "[0,0][1216,2688]" }, [ + node({ ...attrs, bounds: "[0,300][1216,440]" }, [ + node({ type: "Text", text: "Wi-Fi", bounds: "[40,340][400,400]" }), + ]), + ]), + ]), + SCREEN + ).tree; + + it("keeps a container carrying the identifier an agent selects on", () => { + const tree = wrapping({ type: "Column", id: "wifi_entry", key: "wifi_entry" }); + expect(findAll(tree, { identifier: "wifi_entry" })).toHaveLength(1); + const target = tree.children[0].children[0]; + expect(target.identifier).toBe("wifi_entry"); + // The row's own text stays reachable underneath it. + expect(target.children[0].label).toBe("Wi-Fi"); + }); + + it("keeps a container that knows it is disabled, selected or focused", () => { + // One flag per case, so each is pinned on its own rather than by an `id` + // that would have kept the node regardless. + const own = (attrs: Record) => wrapping(attrs).children[0].children[0]; + expect(own({ type: "Row", enabled: "false" }).disabled).toBe(true); + expect(own({ type: "Column", selected: "true" }).selected).toBe(true); + expect(own({ type: "Stack", focused: "true" }).focused).toBe(true); + }); + }); + // ArkUI sets `.id()` and the state flags on the OUTER component, so the // wrapper of a same-rect pair is often the only node that knows it. Collapsing // it away kept `clickable` and dropped the rest. From 4d13ab5c528fa5d2e2dfd50d3c2a843411425dee Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 17:29:38 +0200 Subject: [PATCH 035/110] fix(await-ui-element): keep a standing warning off a successful wait MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Folding the tree's `hint` into a successful result was added for HarmonyOS, where a `visible` resolved against the last composited frame is followed by a tap that lands nowhere. It sat on the platform-agnostic path, so a successful wait on any simulator not booted through argent came back carrying "You MUST call boot-device with force=true now" — an imperative to restart the simulator and discard app state, after a wait that worked. Android TV's focus-driven hint had the same shape. A success now carries only the hints that question the tree it just read: HarmonyOS' suspended panel and Chromium's node-budget truncation. Every other hint is a standing fact about the target, as true before the wait as after, and stays on the timeout note where it was. --- .../src/tools/await-ui-element/index.ts | 24 +++- .../tool-server/test/await-harmony.test.ts | 25 ++++ .../tool-server/test/await-ui-element.test.ts | 121 +++++++++++++++--- 3 files changed, 147 insertions(+), 23 deletions(-) diff --git a/packages/tool-server/src/tools/await-ui-element/index.ts b/packages/tool-server/src/tools/await-ui-element/index.ts index 322233e165..d4f4fa7be9 100644 --- a/packages/tool-server/src/tools/await-ui-element/index.ts +++ b/packages/tool-server/src/tools/await-ui-element/index.ts @@ -14,7 +14,7 @@ import { isAndroidTv } from "../../utils/adb"; import { assertSupported } from "../../utils/capability"; import { ensureDeps } from "../../utils/check-deps"; import { pollDescribeTree } from "../../utils/poll-describe-tree"; -import type { DescribeNode, DescribeTreeData } from "../describe/contract"; +import type { DescribeNode, DescribeSource, DescribeTreeData } from "../describe/contract"; import { describeIos, iosRequires } from "../describe/platforms/ios"; import { describeAndroid, androidRequires } from "../describe/platforms/android"; import { describeChromium } from "../describe/platforms/chromium"; @@ -187,6 +187,18 @@ function appendDiagnostics(base: string, lastData: DescribeTreeData | null): str return extras.length === 0 ? base : `${base} (${extras.join("; ")})`; } +// Sources whose `hint` questions the tree just read: HarmonyOS keeps dumping the +// last composited frame while the panel is suspended, so a matched element need +// not be on the live screen; Chromium's walker stops at its node budget, so the +// page is only partly there. Every other hint is a standing fact about the target +// — not booted through argent, a tvOS / Android TV device, an app that cannot be +// instrumented — as true before the wait as after, so on a success it reads as a +// verdict on what the wait saw; iOS' ends "You MUST call boot-device with +// force=true now", i.e. restart the simulator and lose the app state. Those stay +// on the timeout note. Listing rather than excluding leaves a source added later +// silent on success until someone judges its hint. +const READ_CAVEAT_SOURCES: ReadonlySet = new Set(["harmony-uitest", "cdp-dom"]); + function timeoutNote( params: Params, lastTree: DescribeNode | null, @@ -350,11 +362,11 @@ or before tapping an element that appears asynchronously.`, result.note = "condition met immediately — the selector never matched any element, " + "so it may have already been hidden before the wait, or the selector is wrong"; - } else if (data.hint) { - // A successful wait against a degraded read still owes the agent the - // caveat: on HarmonyOS the tree is the last-composited frame while - // the panel is off, so a `visible` that resolved here is followed by - // a tap that lands nowhere unless the screen is woken first. + } else if (data.hint && READ_CAVEAT_SOURCES.has(data.source)) { + // A success read off a tree that may not be the live screen still + // owes the agent that caveat: on HarmonyOS a `visible` resolved + // against the last-composited frame is followed by a tap that + // lands nowhere unless the screen is woken first. result.note = data.hint; } return { done: true, result }; diff --git a/packages/tool-server/test/await-harmony.test.ts b/packages/tool-server/test/await-harmony.test.ts index 38b98ac984..697faba246 100644 --- a/packages/tool-server/test/await-harmony.test.ts +++ b/packages/tool-server/test/await-harmony.test.ts @@ -93,6 +93,31 @@ describe("await-ui-element on HarmonyOS", () => { expect(calls()).toBeGreaterThan(1); }); + it("`visible` succeeds against a suspended panel but says the display is off", async () => { + // `uitest dumpLayout` keeps serving the last composited frame while the + // panel is suspended, so the element is "there" and the wait is right to + // resolve — but the following tap lands nowhere. Without the caveat a + // wait-then-tap loop runs to completion against a dead screen. + vi.mocked(harmonyDisplay).mockResolvedValue({ ...DISPLAY, screenOn: false }); + sequenceDumps([dumpWith("Submit")]); + const tool = createAwaitUiElementTool(harmonyRegistry()); + + const result = await tool.execute( + {}, + { + udid: HARMONY_ID, + condition: "visible", + selector: { text: "Submit" }, + timeoutMs: 2000, + pollIntervalMs: 10, + } + ); + + expect(result.success).toBe(true); + expect(result.note).toMatch(/display is off/i); + expect(result.note).toMatch(/taps land nowhere/i); + }); + it("passes the bare connect key to `uitest`, not the `harmony-` prefixed device id", async () => { sequenceDumps([dumpWith("Submit")]); const tool = createAwaitUiElementTool(harmonyRegistry()); diff --git a/packages/tool-server/test/await-ui-element.test.ts b/packages/tool-server/test/await-ui-element.test.ts index abeba4f44c..7f82556c99 100644 --- a/packages/tool-server/test/await-ui-element.test.ts +++ b/packages/tool-server/test/await-ui-element.test.ts @@ -19,9 +19,12 @@ vi.mock("../src/utils/ios-devices", async () => { ); return { ...actual, isTvOsSimulator: async () => false }; }); +// Mutable so the one Android TV case can flip the form factor; every other test +// runs on the phone shape the fixtures describe. +const adbProbe = vi.hoisted(() => ({ isTv: false })); vi.mock("../src/utils/adb", async () => { const actual = await vi.importActual("../src/utils/adb"); - return { ...actual, isAndroidTv: async () => false }; + return { ...actual, isAndroidTv: async () => adbProbe.isTv }; }); const IOS_UDID = "AAAAAAAA-AAAA-AAAA-AAAA-AAAAAAAAAAAA"; @@ -97,6 +100,7 @@ describe("await-ui-element tool", () => { beforeEach(() => { __resetDepCacheForTests(); __primeDepCacheForTests(["xcrun", "adb"]); + adbProbe.isTv = false; }); it("exposes the await-ui-element id", () => { @@ -513,6 +517,36 @@ describe("await-ui-element tool", () => { expect(result.success).toBe(true); }); + it("a success on Android TV carries no note", async () => { + // The nearest twin of the degraded-simulator case: the TV hint is a form + // factor, true of every read from this device, so a wait that resolved has + // nothing to add with it. + adbProbe.isTv = true; + const xml = + `` + + `` + + ``; + const android: AndroidDevtoolsApi = { + getHierarchy: async () => ({ xml }), + getScreenSize: async () => ({ width: 1080, height: 2400, rotation: 0 }), + } as unknown as AndroidDevtoolsApi; + const tool = createAwaitUiElementTool(makeMockRegistry({ android })); + + const result = await tool.execute( + {}, + { + udid: ANDROID_SERIAL, + condition: "visible", + selector: { text: "Sign in" }, + timeoutMs: 2000, + pollIntervalMs: 10, + } + ); + + expect(result.success).toBe(true); + expect(result.note).toBeUndefined(); + }); + it("does NOT confirm `hidden` on Android when a seen element is followed by an empty tree", async () => { // Android never sets hint / should_restart, so before the everMatched guard a // transient empty `uiautomator dump` after the element had appeared would let @@ -548,13 +582,13 @@ describe("await-ui-element tool", () => { // ── Chromium branch ────────────────────────────────────────────────────── - function makeChromiumApi(treeJson: unknown): ChromiumCdpApi { + function makeChromiumApi(treeJson: unknown, truncated = false): ChromiumCdpApi { return { refreshViewport: async () => ({ width: 1024, height: 768 }), cdp: { send: async (method: string) => { if (method === "Runtime.evaluate") { - return { result: { value: JSON.stringify({ tree: treeJson, truncated: false }) } }; + return { result: { value: JSON.stringify({ tree: treeJson, truncated }) } }; } return {}; }, @@ -562,24 +596,46 @@ describe("await-ui-element tool", () => { } as unknown as ChromiumCdpApi; } + const CHROMIUM_TREE = { + role: "html", + frame: { x: 0, y: 0, width: 1, height: 1 }, + children: [ + { + role: "button", + label: "Continue", + clickable: true, + frame: { x: 0.4, y: 0.8, width: 0.2, height: 0.05 }, + children: [], + }, + ], + }; + it("drives the Chromium CDP path and matches a DOM node", async () => { - const tree = { - role: "html", - frame: { x: 0, y: 0, width: 1, height: 1 }, - children: [ - { - role: "button", - label: "Continue", - clickable: true, - frame: { x: 0.4, y: 0.8, width: 0.2, height: 0.05 }, - children: [], - }, - ], - }; const tool = createAwaitUiElementTool(makeMockRegistry({})); const result = await tool.execute( - { chromium: makeChromiumApi(tree) }, + { chromium: makeChromiumApi(CHROMIUM_TREE) }, + { + udid: CHROMIUM_ID, + condition: "visible", + selector: { text: "Continue" }, + timeoutMs: 2000, + pollIntervalMs: 10, + } + ); + + expect(result.success).toBe(true); + expect(result.note).toBeUndefined(); + }); + + it("says the tree was partial when the walker hit its node budget", async () => { + // Truncation is a property of this read, like HarmonyOS' suspended panel: + // the page is only partly there, so the wait resolved on less than the + // screen and the caller is owed that. + const tool = createAwaitUiElementTool(makeMockRegistry({})); + + const result = await tool.execute( + { chromium: makeChromiumApi(CHROMIUM_TREE, true) }, { udid: CHROMIUM_ID, condition: "visible", @@ -590,6 +646,7 @@ describe("await-ui-element tool", () => { ); expect(result.success).toBe(true); + expect(result.note).toMatch(/PARTIAL tree/); }); it("surfaces a Chromium fetch failure in the timeout note (last tree fetch failed)", async () => { @@ -664,6 +721,36 @@ describe("await-ui-element tool", () => { expect(result.note).toMatch(/boot-device/i); }); + it.each(["visible", "exists"] as const)( + "a `%s` success on a degraded simulator carries no note", + async (condition) => { + // `degraded` means the simulator was not booted through argent — a common + // state, and the tree it returns is live and populated. The boot hint ends + // "You MUST call boot-device with force=true now", so folding it into a + // success hands the agent an order to restart the device (losing the app + // state the wait was gating) after a wait that worked. + const { api } = makeSequencedAXService( + [axResponse([{ label: "General", frame: FRAME, traits: ["button"] }])], + { degraded: true } + ); + const tool = createAwaitUiElementTool(iosRegistry(api)); + + const result = await tool.execute( + {}, + { + udid: IOS_UDID, + condition, + selector: { text: "General" }, + timeoutMs: 2000, + pollIntervalMs: 10, + } + ); + + expect(result.success).toBe(true); + expect(result.note).toBeUndefined(); + } + ); + // ── Per-fetch deadline ─────────────────────────────────────────────────── it("bounds a single slow fetch to the wait budget instead of its internal timeout", async () => { From 9664b945da6063ca0f2927d742ef6f985edff673 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 17:29:52 +0200 Subject: [PATCH 036/110] fix(harmony): try $DEVECO_STUDIO_HOME ahead of the default, not instead of it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Both resolvers read `configured || MACOS_DEVECO_ROOT`, so setting the variable replaced the macOS default rather than being tried before it — and the hint asking for "the DevEco install root" pointed at `/Applications/DevEco-Studio.app` while the constant was that path plus `/Contents`. Following the hint on a host with a working install returned zero HarmonyOS devices from `list-devices`, with no diagnostic anywhere, and a `boot-device` failure pointing back at the setting that caused it. `android-binary.ts` orders the same thing the other way, with a comment about not letting a stale env var shadow a real install. Candidate roots are now tried in order, each as given and with `Contents` appended, so both spellings of the macOS root work and a wrong variable can no longer hide a real install. The two resolvers share one implementation rather than carrying a copy each, and every hint says what the variable has to point at. --- README.md | 2 +- packages/tool-server/src/utils/check-deps.ts | 4 +- packages/tool-server/src/utils/harmony-cli.ts | 73 ++++++--- packages/tool-server/src/utils/harmony-hdc.ts | 31 +--- .../test/harmony-binary-resolve.test.ts | 147 ++++++++++++++++++ 5 files changed, 211 insertions(+), 46 deletions(-) create mode 100644 packages/tool-server/test/harmony-binary-resolve.test.ts diff --git a/README.md b/README.md index 7c9724bdfb..94418ae9c1 100644 --- a/README.md +++ b/README.md @@ -59,7 +59,7 @@ Argent drives a growing set of targets through a single toolkit, each with the r - For iOS / tvOS: macOS with **Xcode** installed (Apple TV uses tvOS simulators — Xcode downloads the tvOS runtime on demand) - For Android / Android TV: **Android SDK Platform Tools** (`adb`) on `PATH`, and the **Android Emulator** package if you want to boot AVDs from Argent. Create AVDs via Android Studio or `avdmanager`. - For Fire TV (Vega): the **Vega SDK** (`vega` CLI) on `PATH` -- For HarmonyOS: **DevEco Studio**, which ships both binaries Argent drives - `hdc` (the device connector, for phones) and the `Emulator` manager. Both are looked up inside DevEco Studio's install root - the macOS app bundle by default, `$DEVECO_STUDIO_HOME` on any other host. `hdc` also falls back to `PATH` for hosts that installed the OpenHarmony command-line tools without the IDE; `Emulator` never does, since it is too generic a name to match safely. Driving a phone needs only `hdc`. Huawei restricts the emulator image download to mainland China, so outside it no emulator instance can be created. +- For HarmonyOS: **DevEco Studio**, which ships both binaries Argent drives - `hdc` (the device connector, for phones) and the `Emulator` manager. Both are looked up inside DevEco Studio's install root: `$DEVECO_STUDIO_HOME` first when it is set, then `/Applications/DevEco-Studio.app` on macOS, each tried both as given and with `Contents/` appended so the app bundle itself works. `hdc` also falls back to `PATH` for hosts that installed the OpenHarmony command-line tools without the IDE; `Emulator` never does, since it is too generic a name to match safely. Driving a phone needs only `hdc`. Huawei restricts the emulator image download to mainland China, so outside it no emulator instance can be created. - For Electron / Chromium: nothing extra to control an already-running app - just launch it with `--remote-debugging-port`, or let Argent spawn your Electron app for you ##### Linux host: extra prerequisites for Android emulators diff --git a/packages/tool-server/src/utils/check-deps.ts b/packages/tool-server/src/utils/check-deps.ts index 93c882949b..519ef9b320 100644 --- a/packages/tool-server/src/utils/check-deps.ts +++ b/packages/tool-server/src/utils/check-deps.ts @@ -48,9 +48,9 @@ const INSTALL_HINTS: Record = { "vega": "Vega SDK CLI not found. Install the Amazon Vega SDK and run `source ~/vega/env` so `vega` (or its `kepler` alias) is on PATH; the resolver also checks `~/vega/bin/vega`. Only required for Vega (Fire TV) devices.", "harmony-emulator": - "HarmonyOS emulator manager not found. Install DevEco Studio; on a non-macOS host set `$DEVECO_STUDIO_HOME` to its install root. Only required to list or launch HarmonyOS emulators.", + "HarmonyOS emulator manager not found. Install DevEco Studio; a macOS install at `/Applications/DevEco-Studio.app` is found automatically, and anywhere else set `$DEVECO_STUDIO_HOME` to the directory holding `tools/emulator/Emulator` (the resolver also checks that path under `Contents/`, so a macOS app bundle works too). Only required to list or launch HarmonyOS emulators.", "hdc": - "`hdc`, the HarmonyOS device connector, was not found. Install DevEco Studio (it ships under `sdk/default/openharmony/toolchains`), or put `hdc` from the OpenHarmony command-line tools on PATH, or set `$DEVECO_STUDIO_HOME` to the DevEco install root. Only required to drive HarmonyOS devices.", + "`hdc`, the HarmonyOS device connector, was not found. Install DevEco Studio (it ships `hdc` under `sdk/default/openharmony/toolchains`, and a macOS install at `/Applications/DevEco-Studio.app` is found automatically), or put `hdc` from the OpenHarmony command-line tools on PATH, or set `$DEVECO_STUDIO_HOME` to the directory holding `sdk/default/openharmony/toolchains/hdc` (the resolver also checks that path under `Contents/`, so a macOS app bundle works too). Only required to drive HarmonyOS devices.", }; async function probe(dep: ToolDependency): Promise { diff --git a/packages/tool-server/src/utils/harmony-cli.ts b/packages/tool-server/src/utils/harmony-cli.ts index 5bf140531b..3e0d16f7cf 100644 --- a/packages/tool-server/src/utils/harmony-cli.ts +++ b/packages/tool-server/src/utils/harmony-cli.ts @@ -36,24 +36,40 @@ const execFileAsync = promisify(execFile); export const HARMONY_EMPTY_SENTINEL = "[Empty]"; /** - * DevEco Studio's macOS install root. Non-macOS hosts (DevEco also ships for - * Windows) are supported through `$DEVECO_STUDIO_HOME` rather than a second - * hardcoded root, because only the macOS layout has been verified here. + * DevEco Studio's default macOS install location. Non-macOS hosts (DevEco also + * ships for Windows) are reached through `$DEVECO_STUDIO_HOME` rather than a + * second hardcoded root, because only the macOS layout has been verified here. */ -const MACOS_DEVECO_ROOT = "/Applications/DevEco-Studio.app/Contents"; +const MACOS_DEVECO_APP = "/Applications/DevEco-Studio.app"; /** Path of the emulator manager relative to a DevEco Studio install root. */ const EMULATOR_RELATIVE = join("tools", "emulator", "Emulator"); -// Mirrors android-binary.ts / vega-cli.ts: memoize briefly so a burst of tool -// calls pays one lookup, but a *negative* result expires — a user who installs -// DevEco Studio mid-session recovers without restarting the tool-server. -const BINARY_TTL_MS = 60_000; -let cachedEmulator: { path: string | null; checkedAt: number } | undefined; +/** + * DevEco Studio install roots to search, in order. + * + * `$DEVECO_STUDIO_HOME` is tried ahead of the macOS default rather than in place + * of it — the same ordering `androidRoots()` gives `$ANDROID_HOME`, for the same + * reason: a stale or mis-set variable must not be able to hide a working + * install. Here that failure is silent, since both readers turn a null binary + * into an empty device list. + * + * Every root is searched with `Contents` appended too, because on macOS the + * install root a user can name is the `DevEco-Studio.app` bundle while the SDK + * and tools sit one level inside it. Both spellings name the same install. + */ +function devecoRoots(): string[] { + const roots: string[] = []; + const configured = process.env.DEVECO_STUDIO_HOME?.trim(); + if (configured) roots.push(configured); + if (process.platform === "darwin") roots.push(MACOS_DEVECO_APP); + return roots.flatMap((root) => [root, join(root, "Contents")]); +} // X_OK rather than F_OK (as in vega-cli.ts): a present-but-non-executable file -// at the DevEco path is a partial install, and returning it would surface as an -// opaque EACCES at spawn instead of the actionable not-found hint. +// at a DevEco path is a partial install, so the search moves on to the next root +// rather than returning a path that would surface as an opaque EACCES at spawn +// instead of the actionable not-found hint. async function isExecutable(p: string): Promise { try { await access(p, fsConstants.X_OK); @@ -63,6 +79,26 @@ async function isExecutable(p: string): Promise { } } +/** + * Absolute path to a binary that ships inside DevEco Studio, named by its path + * relative to the install root, or null when no candidate root holds an + * executable copy of it. Shared with `harmony-hdc.ts`: `hdc` and the emulator + * manager come out of one install, so they have to agree on where it is. + */ +export async function resolveDevecoBinary(relative: string): Promise { + for (const root of devecoRoots()) { + const candidate = join(root, relative); + if (await isExecutable(candidate)) return candidate; + } + return null; +} + +// Mirrors android-binary.ts / vega-cli.ts: memoize briefly so a burst of tool +// calls pays one lookup, but a *negative* result expires — a user who installs +// DevEco Studio mid-session recovers without restarting the tool-server. +const BINARY_TTL_MS = 60_000; +let cachedEmulator: { path: string | null; checkedAt: number } | undefined; + /** * Absolute path to the HarmonyOS emulator manager, or null when DevEco Studio * isn't installed. Not resolved from PATH: the binary is named `Emulator`, too @@ -73,13 +109,7 @@ export async function resolveHarmonyEmulator(): Promise { if (cachedEmulator && now - cachedEmulator.checkedAt < BINARY_TTL_MS) { return cachedEmulator.path; } - const configured = process.env.DEVECO_STUDIO_HOME?.trim(); - const root = configured || (process.platform === "darwin" ? MACOS_DEVECO_ROOT : null); - let path: string | null = null; - if (root) { - const candidate = join(root, EMULATOR_RELATIVE); - if (await isExecutable(candidate)) path = candidate; - } + const path = await resolveDevecoBinary(EMULATOR_RELATIVE); cachedEmulator = { path, checkedAt: now }; return path; } @@ -101,8 +131,11 @@ export async function runHarmonyEmulator( const bin = await resolveHarmonyEmulator(); if (!bin) { throw new Error( - "The HarmonyOS `Emulator` manager was not found. Install DevEco Studio, or set " + - "`$DEVECO_STUDIO_HOME` to its install root, then retry." + "The HarmonyOS `Emulator` manager was not found. Install DevEco Studio: a macOS install " + + "at /Applications/DevEco-Studio.app is found on its own, and anywhere else set " + + "`$DEVECO_STUDIO_HOME` to the directory holding `tools/emulator/Emulator` (on macOS " + + "that is the `DevEco-Studio.app` bundle, or the `Contents` directory inside it). " + + "Then retry." ); } try { diff --git a/packages/tool-server/src/utils/harmony-hdc.ts b/packages/tool-server/src/utils/harmony-hdc.ts index 31b29fc21a..57b4f74da5 100644 --- a/packages/tool-server/src/utils/harmony-hdc.ts +++ b/packages/tool-server/src/utils/harmony-hdc.ts @@ -1,10 +1,10 @@ import { execFile } from "node:child_process"; import { promisify } from "node:util"; -import { access, constants as fsConstants } from "node:fs/promises"; import { join } from "node:path"; import { FAILURE_CODES, FailureError } from "@argent/registry"; import { formatSubprocessFailure } from "./subprocess-error"; import { commandOnPath } from "./command-on-path"; +import { resolveDevecoBinary } from "./harmony-cli"; const execFileAsync = promisify(execFile); @@ -43,23 +43,12 @@ export const HDC_EMPTY_SENTINEL = "[Empty]"; */ const RC_SENTINEL = "__argent_hdc_rc"; -const MACOS_DEVECO_ROOT = "/Applications/DevEco-Studio.app/Contents"; - /** Path of `hdc` relative to a DevEco Studio install root. */ const HDC_RELATIVE = join("sdk", "default", "openharmony", "toolchains", "hdc"); const BINARY_TTL_MS = 60_000; let cachedHdc: { path: string | null; checkedAt: number } | undefined; -async function isExecutable(p: string): Promise { - try { - await access(p, fsConstants.X_OK); - return true; - } catch { - return false; - } -} - /** * Absolute path to `hdc`, or null when neither DevEco Studio nor a standalone * OpenHarmony command-line-tools install provides it. @@ -74,14 +63,7 @@ export async function resolveHdc(): Promise { if (cachedHdc && now - cachedHdc.checkedAt < BINARY_TTL_MS) { return cachedHdc.path; } - const configured = process.env.DEVECO_STUDIO_HOME?.trim(); - const root = configured || (process.platform === "darwin" ? MACOS_DEVECO_ROOT : null); - let path: string | null = null; - if (root) { - const candidate = join(root, HDC_RELATIVE); - if (await isExecutable(candidate)) path = candidate; - } - path ??= await commandOnPath("hdc"); + const path = (await resolveDevecoBinary(HDC_RELATIVE)) ?? (await commandOnPath("hdc")); cachedHdc = { path, checkedAt: now }; return path; } @@ -90,9 +72,12 @@ async function resolveHdcOrThrow(): Promise { const path = await resolveHdc(); if (!path) { throw new FailureError( - "`hdc` (the HarmonyOS device connector) was not found. Install DevEco Studio, or set " + - "`$DEVECO_STUDIO_HOME` to its install root, or put `hdc` from the OpenHarmony " + - "command-line tools on PATH.", + "`hdc` (the HarmonyOS device connector) was not found. Install DevEco Studio: a macOS " + + "install at /Applications/DevEco-Studio.app is found on its own, and anywhere else set " + + "`$DEVECO_STUDIO_HOME` to the directory holding " + + "`sdk/default/openharmony/toolchains/hdc` (on macOS that is the `DevEco-Studio.app` " + + "bundle, or the `Contents` directory inside it). Alternatively put `hdc` from the " + + "OpenHarmony command-line tools on PATH.", { error_code: FAILURE_CODES.HARMONY_HDC_NOT_FOUND, failure_stage: "harmony_hdc_resolve_binary", diff --git a/packages/tool-server/test/harmony-binary-resolve.test.ts b/packages/tool-server/test/harmony-binary-resolve.test.ts new file mode 100644 index 0000000000..fa24c850e4 --- /dev/null +++ b/packages/tool-server/test/harmony-binary-resolve.test.ts @@ -0,0 +1,147 @@ +import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; +import { chmod, mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; + +// Force the PATH probe to miss. `resolveHdc` falls back to `command -v hdc` for +// standalone command-line-tools installs, and on a host that has one it would +// return a path for every case below, hiding whether the DevEco roots were +// searched at all. +vi.mock("../src/utils/command-on-path", () => ({ + commandOnPath: vi.fn(async () => null), +})); + +// Records every candidate the resolvers stat, in order, so the tests can assert +// which roots were searched — not just which one won. Needed because the macOS +// default root is a fixed absolute path no test can populate. +const { probed } = vi.hoisted(() => ({ probed: [] as string[] })); +vi.mock("node:fs/promises", async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + access: async (path: string, mode?: number) => { + probed.push(String(path)); + return actual.access(path, mode); + }, + }; +}); + +const HDC_RELATIVE = join("sdk", "default", "openharmony", "toolchains", "hdc"); +const EMULATOR_RELATIVE = join("tools", "emulator", "Emulator"); +const MACOS_DEVECO_APP = "/Applications/DevEco-Studio.app"; + +const originalPlatform = process.platform; +const originalHome = process.env.DEVECO_STUDIO_HOME; + +function setPlatform(value: NodeJS.Platform) { + Object.defineProperty(process, "platform", { value, configurable: true }); +} + +/** + * Both resolvers memoize their answer in a module-level cache for 60s, so a + * fresh import per test is the only way to make each one probe the filesystem + * again. + */ +async function loadResolvers() { + vi.resetModules(); + const [hdc, cli] = await Promise.all([ + import("../src/utils/harmony-hdc"), + import("../src/utils/harmony-cli"), + ]); + return { resolveHdc: hdc.resolveHdc, resolveHarmonyEmulator: cli.resolveHarmonyEmulator }; +} + +async function writeExecutable(path: string): Promise { + await mkdir(join(path, ".."), { recursive: true }); + await writeFile(path, ""); + await chmod(path, 0o755); +} + +describe("DevEco Studio binary resolution", () => { + let tmpRoot: string; + + beforeEach(async () => { + tmpRoot = await mkdtemp(join(tmpdir(), "argent-deveco-root-")); + probed.length = 0; + // The macOS default root only participates on darwin, and every assertion + // below is about how `$DEVECO_STUDIO_HOME` interacts with it. + setPlatform("darwin"); + }); + + afterEach(async () => { + setPlatform(originalPlatform); + if (originalHome === undefined) delete process.env.DEVECO_STUDIO_HOME; + else process.env.DEVECO_STUDIO_HOME = originalHome; + vi.resetModules(); + await rm(tmpRoot, { recursive: true, force: true }); + }); + + describe("$DEVECO_STUDIO_HOME set to the macOS app bundle", () => { + // What Finder shows and what the install hints ask for: the `.app` itself. + // The binaries sit one level down, inside `Contents`. + it("finds hdc", async () => { + const bundle = join(tmpRoot, "DevEco-Studio.app"); + const hdc = join(bundle, "Contents", HDC_RELATIVE); + await writeExecutable(hdc); + process.env.DEVECO_STUDIO_HOME = bundle; + + const { resolveHdc } = await loadResolvers(); + expect(await resolveHdc()).toBe(hdc); + }); + + it("finds the Emulator manager", async () => { + const bundle = join(tmpRoot, "DevEco-Studio.app"); + const emulator = join(bundle, "Contents", EMULATOR_RELATIVE); + await writeExecutable(emulator); + process.env.DEVECO_STUDIO_HOME = bundle; + + const { resolveHarmonyEmulator } = await loadResolvers(); + expect(await resolveHarmonyEmulator()).toBe(emulator); + }); + }); + + it("still resolves a root that already points inside the bundle", async () => { + // The other spelling of the same install. Both have to work, or fixing one + // breaks whoever configured the other. + const contents = join(tmpRoot, "DevEco-Studio.app", "Contents"); + const hdc = join(contents, HDC_RELATIVE); + await writeExecutable(hdc); + process.env.DEVECO_STUDIO_HOME = contents; + + const { resolveHdc } = await loadResolvers(); + expect(await resolveHdc()).toBe(hdc); + }); + + describe("$DEVECO_STUDIO_HOME pointing at a root with no DevEco install", () => { + // A set-but-wrong variable must not be able to hide a working default + // install: it is tried first, then the default, exactly as `androidRoots()` + // orders `$ANDROID_HOME` ahead of the OS defaults rather than instead of + // them. Asserted on the probe list because no test can populate + // /Applications. + it("goes on to probe the macOS default root for hdc", async () => { + process.env.DEVECO_STUDIO_HOME = tmpRoot; + + const { resolveHdc } = await loadResolvers(); + await resolveHdc(); + + const envCandidate = join(tmpRoot, HDC_RELATIVE); + const defaultCandidate = join(MACOS_DEVECO_APP, "Contents", HDC_RELATIVE); + expect(probed).toContain(defaultCandidate); + expect(probed.indexOf(envCandidate)).toBeGreaterThanOrEqual(0); + expect(probed.indexOf(envCandidate)).toBeLessThan(probed.indexOf(defaultCandidate)); + }); + + it("goes on to probe the macOS default root for the Emulator manager", async () => { + process.env.DEVECO_STUDIO_HOME = tmpRoot; + + const { resolveHarmonyEmulator } = await loadResolvers(); + await resolveHarmonyEmulator(); + + const envCandidate = join(tmpRoot, EMULATOR_RELATIVE); + const defaultCandidate = join(MACOS_DEVECO_APP, "Contents", EMULATOR_RELATIVE); + expect(probed).toContain(defaultCandidate); + expect(probed.indexOf(envCandidate)).toBeGreaterThanOrEqual(0); + expect(probed.indexOf(envCandidate)).toBeLessThan(probed.indexOf(defaultCandidate)); + }); + }); +}); From 5242c037075fdf99f81545ce314b4c19444036ac Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 17:29:52 +0200 Subject: [PATCH 037/110] fix(screenshot-diff): remove the capture it has copied in MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A live capture is scratch, not an artifact: the copy under `outputDir` is what outlives the call, while the backend's own uniquely named PNG stayed in `tmpdir()` for the life of the process, one per diff, at the `scale: 1.0` this path asks for — 213KB for a single 1320x2856 frame. Every backend leaks the same way here, so the removal goes in the one caller that owns the temporary, which is the ownership `flow-pixels`' `capturePng` already takes. A caller-supplied `baselinePath`/`currentPath` never reaches this function, so a saved image cannot be removed. The HarmonyOS capture also decoded and re-encoded the frame at scale 1.0, where `scaleDecodedPng` returns its input untouched: it now moves the file `uitest` already wrote. Measured over five captures of one 1320x2856 frame, 228ms to 129ms each, same pixels. --- .../src/tools/screenshot-diff/index.ts | 8 +++ .../tool-server/src/utils/harmony-screen.ts | 16 +++++- .../tool-server/test/harmony-screen.test.ts | 55 +++++++++++++++++++ .../test/screenshot-diff-tool.test.ts | 39 +++++++++++-- 4 files changed, 111 insertions(+), 7 deletions(-) diff --git a/packages/tool-server/src/tools/screenshot-diff/index.ts b/packages/tool-server/src/tools/screenshot-diff/index.ts index 1d553f65dd..2efc1752f6 100644 --- a/packages/tool-server/src/tools/screenshot-diff/index.ts +++ b/packages/tool-server/src/tools/screenshot-diff/index.ts @@ -339,5 +339,13 @@ async function captureLiveInput(params: { const destination = path.join(params.outputDir, `${params.name}-${suffix}.live.png`); await fs.mkdir(params.outputDir, { recursive: true }); await fs.copyFile(capturedPath, destination); + // The capture is scratch, not an artifact: every backend writes a uniquely + // named PNG that nothing else prunes, so one per diff would accumulate in + // `tmpdir()` for the life of the process — and at `scale: 1.0`, which this + // path asks for, each is a full-resolution frame. `destination` is the copy + // that outlives the call. Same ownership `flow-pixels`' `capturePng` takes, + // and only reachable for a capture this function made: a caller-supplied + // `baselinePath`/`currentPath` never comes through here. + await fs.rm(capturedPath, { force: true }).catch(() => {}); return destination; } diff --git a/packages/tool-server/src/utils/harmony-screen.ts b/packages/tool-server/src/utils/harmony-screen.ts index 1b05559f66..295c573099 100644 --- a/packages/tool-server/src/utils/harmony-screen.ts +++ b/packages/tool-server/src/utils/harmony-screen.ts @@ -1,4 +1,4 @@ -import { readFile, writeFile, rm } from "node:fs/promises"; +import { readFile, rename, writeFile, rm } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { PNG } from "pngjs"; @@ -21,10 +21,22 @@ export async function captureHarmonyScreenshotPng(opts: { const rawPath = join(tmpdir(), `argent-harmony-raw-${process.hrtime.bigint()}.png`); try { await harmonyScreenCap(opts.connectKey, rawPath); + // The decode is the only proof `uitest` captured anything: `hdc` reports the + // connection's status, not the remote command's, so a failed capture arrives + // as a successful transfer of a truncated file. const decoded = PNG.sync.read(await readFile(rawPath)); const scaled = scaleDecodedPng(decoded, opts.scale); const outPath = join(tmpdir(), `harmony-screenshot-${process.hrtime.bigint()}.png`); - await writeFile(outPath, PNG.sync.write(scaled)); + if (scaled === decoded) { + // Identity — what screenshot-diff's scale 1.0 asks for. Re-encoding pixels + // the resample never touched costs ~100ms per 3.7MP frame to reproduce the + // file `uitest` already wrote, so move that one into place instead. + // Comparing against the helper's own return, rather than re-deriving the + // threshold, keeps one definition of which scales resample. + await rename(rawPath, outPath); + } else { + await writeFile(outPath, PNG.sync.write(scaled)); + } return outPath; } finally { await rm(rawPath, { force: true }).catch(() => {}); diff --git a/packages/tool-server/test/harmony-screen.test.ts b/packages/tool-server/test/harmony-screen.test.ts index 96fb50c687..1672116010 100644 --- a/packages/tool-server/test/harmony-screen.test.ts +++ b/packages/tool-server/test/harmony-screen.test.ts @@ -1,5 +1,6 @@ import { describe, it, expect, vi, beforeEach, afterEach } from "vitest"; import { access, readFile, rm, writeFile } from "node:fs/promises"; +import { createHash } from "node:crypto"; import { PNG } from "pngjs"; import { hdcFileRecv as realHdcFileRecv, @@ -23,6 +24,33 @@ function tinyPng(): Buffer { return PNG.sync.write(new PNG({ width: 4, height: 4 })); } +/** + * A valid PNG whose bytes pngjs will not reproduce: the same pixels written + * with the None row filter, as an encoder that is not pngjs — `uitest` — may + * well have written them. Re-encoding changes the bytes while leaving the image + * identical, which is what separates "moved the device's file" from "decoded it + * and wrote it out again". + */ +function devicePng(width = 8, height = 12): Buffer { + const png = new PNG({ width, height }); + for (let i = 0; i < png.data.length; i += 4) { + png.data[i] = (i * 7) % 256; + png.data[i + 1] = (i * 13) % 256; + png.data[i + 2] = (i * 29) % 256; + png.data[i + 3] = 255; + } + return PNG.sync.write(png, { filterType: 0 }); +} + +const sha256 = (bytes: Buffer): string => createHash("sha256").update(bytes).digest("hex"); + +/** Serve `bytes` as the file `uitest screenCap` captured. */ +function deviceServes(bytes: Buffer): void { + hdcFileRecv.mockImplementation(async (_key, _remote, localPath) => { + await writeFile(localPath, bytes); + }); +} + /** The remote path of every `uitest screenCap -p ''` call, in order. */ function capturedRemotePaths(): string[] { return runHdcShell.mock.calls @@ -92,6 +120,33 @@ describe("harmonyScreenCap viaDeviceTmp", () => { }); describe("captureHarmonyScreenshotPng", () => { + it("hands back the device's own bytes at full resolution", async () => { + // screenshot-diff captures at scale 1, where scaleDecodedPng returns the + // decoded image untouched — re-encoding it costs ~100ms per 3.7MP frame for + // an image identical to the one hdc already delivered. + const bytes = devicePng(); + deviceServes(bytes); + + const out = await capture(); + + expect(sha256(await readFile(out))).toBe(sha256(bytes)); + const decoded = PNG.sync.read(await readFile(out)); + expect([decoded.width, decoded.height]).toEqual([8, 12]); + }); + + it("re-encodes a downscaled capture and keeps no raw intermediate", async () => { + deviceServes(devicePng()); + + const out = await captureHarmonyScreenshotPng({ connectKey: "dev", scale: 0.5 }); + outPaths.push(out); + + const decoded = PNG.sync.read(await readFile(out)); + expect([decoded.width, decoded.height]).toEqual([4, 6]); + // hdcFileRecv's localPath IS the raw intermediate — the resample writes a + // second file, so this one has to be cleaned up behind it. + await expect(access(hdcFileRecv.mock.calls[0]![2])).rejects.toThrow(); + }); + it("removes the host-side raw capture when decoding it fails", async () => { // The fetch "succeeds" but delivers garbage; the raw file must not be // left behind in the host tmpdir on the error path. diff --git a/packages/tool-server/test/screenshot-diff-tool.test.ts b/packages/tool-server/test/screenshot-diff-tool.test.ts index 8c33c4d397..d186b7cab9 100644 --- a/packages/tool-server/test/screenshot-diff-tool.test.ts +++ b/packages/tool-server/test/screenshot-diff-tool.test.ts @@ -210,11 +210,12 @@ describe("screenshotDiffTool", () => { const baselinePath = path.join(dir, "baseline.png"); const capturedPath = path.join(dir, "captured.png"); await writePng(baselinePath, 2, 2, { r: 0, g: 0, b: 0 }); - await writePng(capturedPath, 2, 2, { r: 0, g: 0, b: 0 }); - const captureScreenshot = vi.fn(async () => ({ - url: "http://localhost/current.png", - path: capturedPath, - })); + // Re-written per call, as a real backend does: each capture writes its own + // uniquely named file, and the diff removes the one it copied from. + const captureScreenshot = vi.fn(async () => { + await writePng(capturedPath, 2, 2, { r: 0, g: 0, b: 0 }); + return { url: "http://localhost/current.png", path: capturedPath }; + }); await executeScreenshotDiffTool( { simulatorServer: { apiUrl: "http://localhost:4949" } }, @@ -236,6 +237,34 @@ describe("screenshotDiffTool", () => { expect(new Set(liveCaptures).size).toBe(2); }); + it("does not leave the backend's own capture behind once it has been copied in", async () => { + // Every backend writes its capture to a uniquely named file in `tmpdir()` + // that nothing else prunes — measured at 213KB per call for one 1320x2856 + // frame, since this path captures at `scale: 1.0`. The copy under + // `outputDir` is what outlives the call; the original is scratch. + const dir = await fs.mkdtemp(path.join(os.tmpdir(), "argent-screenshot-diff-scratch-")); + const scratch = await fs.mkdtemp(path.join(os.tmpdir(), "argent-fake-backend-")); + const baselinePath = path.join(dir, "baseline.png"); + const capturedPath = path.join(scratch, "backend-capture.png"); + await writePng(baselinePath, 2, 2, { r: 0, g: 0, b: 0 }); + const captureScreenshot = vi.fn(async () => { + await writePng(capturedPath, 2, 2, { r: 0, g: 0, b: 0 }); + return { url: "http://localhost/current.png", path: capturedPath }; + }); + + await executeScreenshotDiffTool( + { simulatorServer: { apiUrl: "http://localhost:4949" } }, + { baselinePath, captureCurrent: true, udid: "ABC", outputDir: dir }, + { artifacts: new ArtifactStore() }, + captureScreenshot as never + ); + + expect(await fs.readdir(scratch)).toEqual([]); + expect( + (await fs.readdir(dir)).filter((n) => /^current-[a-f0-9]{8}\.live\.png$/.test(n)) + ).toHaveLength(1); + }); + it("validates mutually exclusive saved and live inputs at execute time", async () => { await expect( executeScreenshotDiffTool( From c15605729ab6f2f216b2e4697f14d59211bdfb28 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 18:08:28 +0200 Subject: [PATCH 038/110] test(harmony): pin the dispatch entry each harmony tool hangs from launch-app, restart-app, open-url and keyboard each reach HarmonyOS through a single `harmony: harmonyImpl` line in their dispatchByPlatform table, and nothing executed that line: the existing harmony tests import `harmonyImpl` and call `.handler()` directly, which covers the backend and not the wiring. Deleting all four lines left the suite green while every one of those tools returned 501 on a real device. Drive each tool's `execute` with a `harmony-` udid instead, with only the leaf handler stubbed, plus the inverse case that an Android serial does not reach the harmony branch. Also cover launch-app's harmony backend, which no test file imported at all. --- .../tool-server/test/harmony-apps.test.ts | 38 ++++++ .../test/harmony-dispatch-wiring.test.ts | 115 ++++++++++++++++++ 2 files changed, 153 insertions(+) create mode 100644 packages/tool-server/test/harmony-dispatch-wiring.test.ts diff --git a/packages/tool-server/test/harmony-apps.test.ts b/packages/tool-server/test/harmony-apps.test.ts index eedf4f5aef..6948e98d1c 100644 --- a/packages/tool-server/test/harmony-apps.test.ts +++ b/packages/tool-server/test/harmony-apps.test.ts @@ -8,6 +8,7 @@ import { terminateHarmonyApp, } from "../src/utils/harmony-apps"; import { harmonyImpl as harmonyRestartImpl } from "../src/tools/restart-app/platforms/harmony"; +import { harmonyImpl as harmonyLaunchImpl } from "../src/tools/launch-app/platforms/harmony"; // Only the transport is faked: these tests are about what the module makes of // `bm`/`aa` output, and both CLIs report failure on stdout while exiting 0. @@ -292,6 +293,43 @@ describe("terminateHarmonyApp", () => { }); }); +describe("launch-app on harmony", () => { + const device: DeviceInfo = { id: "harmony-KEY", platform: "harmony", kind: "device" }; + + it("resolves the entry ability before starting, and reports the bundle it launched", async () => { + runHdcShell + .mockResolvedValueOnce(ok(CALCULATOR)) + .mockResolvedValueOnce(ok("start ability successfully.")); + await expect( + harmonyLaunchImpl.handler( + {}, + { udid: device.id, bundleId: "com.huawei.hmos.calculator" }, + device + ) + ).resolves.toEqual({ launched: true, bundleId: "com.huawei.hmos.calculator" }); + expect(runHdcShell.mock.calls.map((c) => c[1])).toEqual([ + "bm dump -n 'com.huawei.hmos.calculator'", + "aa start -b 'com.huawei.hmos.calculator' -a 'CalculatorAbility' -m 'phone'", + ]); + }); + + it("does not report a launch when `aa` refused it", async () => { + // `aa` exits 0 either way, so a missing success line is the only signal that + // separates this from the case above. + runHdcShell + .mockResolvedValueOnce(ok(CALCULATOR)) + .mockResolvedValueOnce( + ok( + "error: failed to start ability.\nError Code:10104001 Error Message:The specified ability does not exist." + ) + ); + const err = await harmonyLaunchImpl + .handler({}, { udid: device.id, bundleId: "com.huawei.hmos.calculator" }, device) + .catch((e: unknown) => e); + expect((err as Error).message).toMatch(/Error Code:10104001/); + }); +}); + describe("restart-app on harmony", () => { const device: DeviceInfo = { id: "harmony-KEY", platform: "harmony", kind: "device" }; diff --git a/packages/tool-server/test/harmony-dispatch-wiring.test.ts b/packages/tool-server/test/harmony-dispatch-wiring.test.ts new file mode 100644 index 0000000000..acec593633 --- /dev/null +++ b/packages/tool-server/test/harmony-dispatch-wiring.test.ts @@ -0,0 +1,115 @@ +import { describe, it, expect, vi } from "vitest"; + +// The four tools whose HarmonyOS support is a single `harmony: harmonyImpl` line +// in a `dispatchByPlatform` table. Every other harmony test reaches the backend +// by importing `harmonyImpl` and calling `.handler()` directly, which exercises +// the backend and never the wiring: delete all four lines and the rest of the +// suite stays green while each of these tools 501s on a real device. +// +// Only the leaf handler is stubbed, deliberately — the claim under test is +// dispatch, so a stub owning no device behaviour keeps these cases pinned to the +// wiring while the backends underneath them keep changing. +// Hoisted with the `vi.mock` factories that close over them — a plain top-level +// const is still in its temporal dead zone when the factory runs. +const { launchStub, restartStub, openUrlStub, keyboardStub } = vi.hoisted(() => { + // Full dispatch signature, so the device it was handed stays typed below. + const stub = (result: T) => ({ + handler: vi.fn( + async (_services: unknown, _params: unknown, device: { platform: string; kind: string }) => { + void device; + return result; + } + ), + }); + return { + launchStub: stub({ launched: true, bundleId: "com.huawei.hmos.calculator" }), + restartStub: stub({ restarted: true, bundleId: "com.huawei.hmos.calculator" }), + openUrlStub: stub({ opened: true, url: "https://example.com" }), + keyboardStub: stub({ typed: "hi" }), + }; +}); + +vi.mock("../src/tools/launch-app/platforms/harmony", async (orig) => ({ + ...(await orig>()), + harmonyImpl: launchStub, +})); +vi.mock("../src/tools/restart-app/platforms/harmony", async (orig) => ({ + ...(await orig>()), + harmonyImpl: restartStub, +})); +vi.mock("../src/tools/open-url/platforms/harmony", async (orig) => ({ + ...(await orig>()), + harmonyImpl: openUrlStub, +})); +vi.mock("../src/tools/keyboard/platforms/harmony", async (orig) => ({ + ...(await orig>()), + harmonyImpl: keyboardStub, +})); + +import { createLaunchAppTool } from "../src/tools/launch-app"; +import { createRestartAppTool } from "../src/tools/restart-app"; +import { openUrlTool } from "../src/tools/open-url"; +import { createKeyboardTool } from "../src/tools/keyboard"; + +const HARMONY_UDID = "harmony-025DEK236V035771"; +const registry = { resolveService: vi.fn(async () => ({})) } as never; + +const cases = [ + { + name: "launch-app", + stub: launchStub, + tool: () => createLaunchAppTool(registry), + params: { udid: HARMONY_UDID, bundleId: "com.huawei.hmos.calculator" }, + result: { launched: true, bundleId: "com.huawei.hmos.calculator" }, + }, + { + name: "restart-app", + stub: restartStub, + tool: () => createRestartAppTool(registry), + params: { udid: HARMONY_UDID, bundleId: "com.huawei.hmos.calculator" }, + result: { restarted: true, bundleId: "com.huawei.hmos.calculator" }, + }, + { + name: "open-url", + stub: openUrlStub, + tool: () => openUrlTool, + params: { udid: HARMONY_UDID, url: "https://example.com" }, + result: { opened: true, url: "https://example.com" }, + }, + { + name: "keyboard", + stub: keyboardStub, + tool: () => createKeyboardTool(registry), + params: { udid: HARMONY_UDID, text: "hi" }, + result: { typed: "hi" }, + }, +] as const; + +describe("HarmonyOS dispatch wiring", () => { + it.each(cases)( + "$name routes a harmony device to its harmony branch", + async ({ stub: impl, tool, params, result }) => { + impl.handler.mockClear(); + // Not `.toHaveBeenCalled()` alone: without the dispatch entry this rejects + // with NotImplementedOnPlatformError, so asserting on the resolved value + // pins that the branch both ran and produced the tool's result. + await expect(tool().execute!({}, params as never)).resolves.toEqual(result); + expect(impl.handler).toHaveBeenCalledOnce(); + expect(impl.handler.mock.calls[0]![2]).toMatchObject({ + platform: "harmony", + kind: "device", + }); + } + ); + + it("does not reach a harmony branch for a device of another platform", async () => { + // Guards the inverse mutation: a `harmony` entry wired onto the wrong arm of + // the table would satisfy the cases above and silently hijack Android. + keyboardStub.handler.mockClear(); + await createKeyboardTool(registry).execute!({}, { + udid: "emulator-5554", + text: "hi", + } as never).catch(() => undefined); + expect(keyboardStub.handler).not.toHaveBeenCalled(); + }); +}); From ef98e5df3318240c3e99260c0878d993d372e7ce Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 18:08:28 +0200 Subject: [PATCH 039/110] test(harmony): cover a value carrying more than one quote in shellQuote Every quote-bearing row in the table had exactly one `'`, and at one quote a first-match-only replace emits the same bytes as replaceAll - so the line standing between caller input and a device-side /bin/sh had nothing pinning it. The new row round-trips byte-identical through a 6.0.1 handset, as the others do. --- packages/tool-server/test/harmony-hdc.test.ts | 3 +++ 1 file changed, 3 insertions(+) diff --git a/packages/tool-server/test/harmony-hdc.test.ts b/packages/tool-server/test/harmony-hdc.test.ts index e932ccc608..42f28b7e03 100644 --- a/packages/tool-server/test/harmony-hdc.test.ts +++ b/packages/tool-server/test/harmony-hdc.test.ts @@ -38,6 +38,9 @@ describe("shellQuote", () => { it.each([ ["hello world", `'hello world'`], ["a'b", `'a'\\''b'`], + // The only row with two quotes: at one, a first-match-only replace produces + // identical bytes, so nothing else here separates it from `replaceAll`. + ["a'b'c", `'a'\\''b'\\''c'`], ['a"b', `'a"b'`], ["a$b", `'a$b'`], ["a;echo PWNED", `'a;echo PWNED'`], From afbe0ce1c39f36866f9e4a547a2f64d30bc5919a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 18:08:28 +0200 Subject: [PATCH 040/110] docs(await-screen-idle): describe the note returned with an unsettled screen The description listed { settled, waitedMs, polls } and stopped there, so the field that tells a device which dropped out apart from a screen that never went still was undocumented. It comes from the shared poller's lastError, so it is not HarmonyOS-only. --- packages/tool-server/src/tools/await-screen-idle/index.ts | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/packages/tool-server/src/tools/await-screen-idle/index.ts b/packages/tool-server/src/tools/await-screen-idle/index.ts index 73ff9f56b9..404b9f2aa1 100644 --- a/packages/tool-server/src/tools/await-screen-idle/index.ts +++ b/packages/tool-server/src/tools/await-screen-idle/index.ts @@ -134,8 +134,9 @@ export function createAwaitScreenIdleTool(registry: Registry): ToolDefinition Date: Fri, 14 Aug 2026 18:00:38 +0200 Subject: [PATCH 041/110] fix(screenshot-diff): drop the capture even when the copy fails The rm sat after the copy, so a diff that could not write into outputDir left a full-resolution frame in tmpdir() per attempt - the accumulation the rm was added to stop. Its cited twin, flow-pixels' capturePng, already removes in a finally. Also tighten the HarmonyOS panel parse: parseInt stops at the first character it cannot read, so "1,320" and "1e4" both parsed as 1. A panel no guest can report back is worse than none, since boot-device reads one as proof the target is another device. --- .../src/tools/screenshot-diff/index.ts | 16 ++++++---- .../tool-server/src/utils/harmony-devices.ts | 13 ++++++-- .../tool-server/test/harmony-devices.test.ts | 14 +++++++++ .../test/screenshot-diff-tool.test.ts | 31 +++++++++++++++++++ 4 files changed, 65 insertions(+), 9 deletions(-) diff --git a/packages/tool-server/src/tools/screenshot-diff/index.ts b/packages/tool-server/src/tools/screenshot-diff/index.ts index 2efc1752f6..8cac97924e 100644 --- a/packages/tool-server/src/tools/screenshot-diff/index.ts +++ b/packages/tool-server/src/tools/screenshot-diff/index.ts @@ -337,15 +337,19 @@ async function captureLiveInput(params: { const capturedPath = await params.capture(); const suffix = crypto.randomBytes(4).toString("hex"); const destination = path.join(params.outputDir, `${params.name}-${suffix}.live.png`); - await fs.mkdir(params.outputDir, { recursive: true }); - await fs.copyFile(capturedPath, destination); // The capture is scratch, not an artifact: every backend writes a uniquely // named PNG that nothing else prunes, so one per diff would accumulate in // `tmpdir()` for the life of the process — and at `scale: 1.0`, which this // path asks for, each is a full-resolution frame. `destination` is the copy - // that outlives the call. Same ownership `flow-pixels`' `capturePng` takes, - // and only reachable for a capture this function made: a caller-supplied - // `baselinePath`/`currentPath` never comes through here. - await fs.rm(capturedPath, { force: true }).catch(() => {}); + // that outlives the call, so the capture goes whether or not the copy worked. + // Same ownership `flow-pixels`' `capturePng` takes, and only reachable for a + // capture this function made: a caller-supplied `baselinePath`/`currentPath` + // never comes through here. + try { + await fs.mkdir(params.outputDir, { recursive: true }); + await fs.copyFile(capturedPath, destination); + } finally { + await fs.rm(capturedPath, { force: true }).catch(() => {}); + } return destination; } diff --git a/packages/tool-server/src/utils/harmony-devices.ts b/packages/tool-server/src/utils/harmony-devices.ts index 117c44fe26..f947dcb05a 100644 --- a/packages/tool-server/src/utils/harmony-devices.ts +++ b/packages/tool-server/src/utils/harmony-devices.ts @@ -66,11 +66,18 @@ interface RawInstance { const str = (v: unknown): string | null => (typeof v === "string" && v.length > 0 ? v : null); -/** A positive pixel count from the manager's all-strings JSON, or null. */ +/** + * A positive pixel count from the manager's all-strings JSON, or null. + * + * Digits and nothing else: `parseInt` stops at the first character it cannot + * read, so `"1,320"` and `"1e4"` would both come back as `1`. A wrong panel is + * worse here than no panel, since `boot-device` reads one no guest can match as + * proof the target is someone else's device. + */ const px = (v: unknown): number | null => { - if (typeof v !== "string") return null; + if (typeof v !== "string" || !/^\d+$/.test(v)) return null; const n = Number.parseInt(v, 10); - return Number.isInteger(n) && n > 0 ? n : null; + return Number.isSafeInteger(n) && n > 0 ? n : null; }; /** diff --git a/packages/tool-server/test/harmony-devices.test.ts b/packages/tool-server/test/harmony-devices.test.ts index 27aee5be3f..1bae6d9c29 100644 --- a/packages/tool-server/test/harmony-devices.test.ts +++ b/packages/tool-server/test/harmony-devices.test.ts @@ -107,6 +107,20 @@ describe("parseHarmonyInstances", () => { } }); + it("reports no panel rather than the digits it could read off the front of one", () => { + // A truncated read is the dangerous shape: `parseInt("1,320")` is 1, a + // panel no guest can ever report back, so `boot-device` would spend the + // whole budget concluding the instance it started is another device. Worse + // than the null, which only turns the check off. + const shapes = ["1,320", "1e4", "1320px", "1320\n2856", "999999999999999999999"]; + for (const width of shapes) { + const raw = `[{ "name": "n", "isRunning": "false", "hw.lcd.single.width": ${JSON.stringify( + width + )}, "hw.lcd.single.height": "2856" }]`; + expect(parseHarmonyInstances(raw)[0].display).toBeNull(); + } + }); + it("reads isRunning as the string it actually is, not a JSON boolean", () => { // The manager emits every value as a string, `isRunning` included. Comparing // it as a boolean would make every instance read as stopped, so a booted diff --git a/packages/tool-server/test/screenshot-diff-tool.test.ts b/packages/tool-server/test/screenshot-diff-tool.test.ts index d186b7cab9..05edec2147 100644 --- a/packages/tool-server/test/screenshot-diff-tool.test.ts +++ b/packages/tool-server/test/screenshot-diff-tool.test.ts @@ -265,6 +265,37 @@ describe("screenshotDiffTool", () => { ).toHaveLength(1); }); + it("does not leave the capture behind when the copy into `outputDir` fails", async () => { + // The failure path is the one that accumulates: a diff that cannot write + // its copy still captured a full-resolution frame, and retrying leaves one + // per attempt in `tmpdir()`. + const dir = await fs.mkdtemp(path.join(os.tmpdir(), "argent-screenshot-diff-ro-")); + const scratch = await fs.mkdtemp(path.join(os.tmpdir(), "argent-fake-backend-")); + const baselinePath = path.join(os.tmpdir(), `argent-baseline-${Date.now()}.png`); + const capturedPath = path.join(scratch, "backend-capture.png"); + await writePng(baselinePath, 2, 2, { r: 0, g: 0, b: 0 }); + const captureScreenshot = vi.fn(async () => { + await writePng(capturedPath, 2, 2, { r: 0, g: 0, b: 0 }); + return { url: "http://localhost/current.png", path: capturedPath }; + }); + await fs.chmod(dir, 0o500); + + try { + await expect( + executeScreenshotDiffTool( + { simulatorServer: { apiUrl: "http://localhost:4949" } }, + { baselinePath, captureCurrent: true, udid: "ABC", outputDir: dir }, + { artifacts: new ArtifactStore() }, + captureScreenshot as never + ) + ).rejects.toThrow(); + } finally { + await fs.chmod(dir, 0o700); + } + + expect(await fs.readdir(scratch)).toEqual([]); + }); + it("validates mutually exclusive saved and live inputs at execute time", async () => { await expect( executeScreenshotDiffTool( From d65017b49bdd5fc2c6eb064b60341487913bafc4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 18:08:41 +0200 Subject: [PATCH 042/110] fix(boot-device): stop the harmony boot latching a verdict off one early read Four things the target wait got wrong, all in the window where a guest is still settling: - The readiness wait never polled the exit latch, so an emulator dying after its target registered burned the whole budget and returned a drivable udid under a "retry the first interaction" note, throwing away the diagnostic the manager had already printed. It takes checkAlive now, as the wait for the target already did. - A target whose panel read wrong once was rejected for the rest of the boot. The reading is not final - harmonyDisplay refuses to cache it for the same reason, foldables changing resolution with the fold - so a guest mid-boot could disqualify the instance argent itself started. Each poll re-probes, and a 0x0 reading is a guest that has not composited rather than another device's panel. - A target that registered but never reported a panel came back as "had not registered with hdc", sending the caller to raise a budget that was never the problem. It gets its own note. - With no panel in the config there is nothing to check an arrival against, and the key was returned as if there were. It still is - it is the only candidate that appeared - but the note now says what it rests on, so an unchecked id cannot pass for a confirmed one. A note no longer implies the udid is the instance id: it carries whatever the boot left unproven, which the type and the tool description said the opposite of. --- .../src/tools/devices/boot-device.ts | 176 +++++++++++++----- packages/tool-server/src/utils/harmony-cli.ts | 18 +- .../test/boot-device-harmony.test.ts | 115 ++++++++++++ 3 files changed, 252 insertions(+), 57 deletions(-) diff --git a/packages/tool-server/src/tools/devices/boot-device.ts b/packages/tool-server/src/tools/devices/boot-device.ts index 196d19945c..6e786c3075 100644 --- a/packages/tool-server/src/tools/devices/boot-device.ts +++ b/packages/tool-server/src/tools/devices/boot-device.ts @@ -53,6 +53,7 @@ import { listVvdImages } from "../../utils/vega-sdk"; import { startVvd, stopVvd, isVvdRunning, waitForVvdRunning } from "../../utils/vega-vvd"; import { resolveRunningVvdSerial, listVegaDevices } from "../../utils/vega-devices"; import { + EMULATOR_NOT_FOUND, emulatorFailure, isChinaOnlyRestriction, resolveHarmonyEmulator, @@ -1452,8 +1453,13 @@ function bootVega(params: { /** * `udid` is the id to drive next: the connect-key id once the instance reached - * `hdc`, the instance id plus a `note` when it did not. Only the first is an id - * the interaction tools accept. + * `hdc`, the instance id when it did not. Only the first is an id the + * interaction tools accept. + * + * `note` does not mark which of the two it is. It carries whatever the boot + * left unproven — why no key was resolved, or what is unestablished about the + * one that was — so a caller reading it as "this is the instance id" is wrong + * exactly when the caveat matters. */ type HarmonyBootResult = { platform: "harmony"; @@ -1558,6 +1564,32 @@ const HARMONY_MISMATCHED_TARGET = "instance is configured with, so it is another device rather than the one argent started. " + 'Call `list-devices` and drive the `kind: "device"` entry for the instance once it appears.'; +/** + * Said when a target registered but never reported a panel to check it against. + * Distinct from {@link HARMONY_MISMATCHED_TARGET}, which knows the target is + * someone else's, and from {@link HARMONY_NO_TARGET}, whose "had not registered" + * would be a plain untruth here — the target did register, so a longer budget + * is not the remedy. + */ +const HARMONY_UNPROBEABLE_TARGET = + "A target registered while the instance was booting but never reported its display, so argent " + + "could not tell whether it is this instance or another device that reconnected, and did not " + + 'guess. Call `list-devices` and drive the `kind: "device"` entry for the instance.'; + +/** + * Said when the key rests on arrival alone. The instance's config does not + * describe a single panel, so the check that separates it from another device + * reconnecting in the same window had nothing to compare against. The key is + * returned because it is the only candidate that appeared and usually is the + * instance — but silently returning an unchecked id is how every later tap ends + * up on someone else's device, so the caller is told what it rests on. + */ +const HARMONY_UNCONFIRMED_TARGET = + "This instance's configuration does not describe a single panel, so argent could not check the " + + "target it returned against it and matched on arrival timing alone. If another emulator or " + + "device reconnected while the instance was booting, this may be its connect key; confirm " + + "against `list-devices` before relying on it."; + /** * Said when the instance registered but never answered `uitest` in time. The * connect key is returned regardless — it is the right id, and an interaction @@ -1651,13 +1683,7 @@ async function startHarmonyEmulator( instanceName: string ): Promise<{ exited: () => { reason: string; output: string } | null }> { const bin = await resolveHarmonyEmulator(); - if (!bin) { - throw new Error( - "The HarmonyOS `Emulator` manager was not found. Install DevEco Studio, or set " + - "`$DEVECO_STUDIO_HOME` to the directory holding `tools/emulator/Emulator` (on macOS " + - "the app bundle itself works), then retry." - ); - } + if (!bin) throw new Error(EMULATOR_NOT_FOUND); const logPath = join(tmpdir(), `argent-harmony-${instanceName.replace(/[^\w.-]/g, "_")}.log`); const logFd = openSync(logPath, "w"); const child = spawn(bin, ["-start", instanceName], { @@ -1695,6 +1721,20 @@ async function startHarmonyEmulator( return { exited: () => exit }; } +/** + * What the wait for a target concluded. + * + * `reason` says why there is no key, and is read only when there is none; + * `unconfirmed` marks a key that arrival alone picked out. Both feed the boot's + * `note`, and each has to stay distinct there: only "none" is a budget the + * caller can usefully raise. + */ +type HarmonyTargetOutcome = { + key: string | null; + unconfirmed?: boolean; + reason?: "ambiguous" | "mismatched" | "unprobeable" | "none"; +}; + /** * The connect key of the target that appears after a start, or null if none * does before `deadline`. @@ -1717,16 +1757,21 @@ async function startHarmonyEmulator( * host-local emulator is not on a cable), a candidate is confirmed against * `expected` — the panel the instance is configured with, which the guest * reports back as its `render resolution` — and two confirmed arrivals are - * refused rather than guessed between, as the Android path refuses when more - * than one serial appears. + * refused rather than guessed between. * * The panel check is a filter and never a requirement. An instance whose config - * does not describe a single panel passes `expected: null` and is not checked at - * all, and a target that cannot be probed yet is left pending for the next poll - * rather than rejected — a row reaches `Connected` before its render service - * answers. Only a target that answers with someone else's panel is out, and - * then for good: re-probing it could not change the answer and would cost a - * round trip per poll. + * does not describe a single panel passes `expected: null`, and its arrival is + * taken on arrival alone, flagged `unconfirmed` so the caller is told the id + * rests on timing rather than on identity. + * + * Nothing is rejected for good. A target that cannot be probed yet is left + * pending — a row reaches `Connected` before its render service answers — and + * so is one answering `0x0`, which is that service up but not yet composited. + * Even a target answering someone else's panel is only skipped for that poll: + * the reading is not final, since this platform's flagship form factors are + * foldables whose resolution changes with the fold, and a boot is when a + * guest's panel is least settled. Latching a verdict off one early reading + * would disqualify the instance argent itself started for the whole budget. * * What this does NOT cover: two instances sharing one device profile, whose * panels are identical by construction; and a second emulator registering @@ -1741,11 +1786,12 @@ async function waitForHarmonyTarget( deadline: number, checkAlive: () => void, expected: { width: number; height: number } | null -): Promise<{ key: string | null; ambiguous: boolean; mismatched: boolean }> { - const rejected = new Set(); +): Promise { + let sawMismatch = false; + let sawUnprobeable = false; for (;;) { const arrived = (await connectedHarmonyTargets()).filter( - (t) => !before.has(t.connectKey) && couldBeHarmonyEmulator(t) && !rejected.has(t.connectKey) + (t) => !before.has(t.connectKey) && couldBeHarmonyEmulator(t) ); const confirmed: string[] = []; for (const target of arrived) { @@ -1754,22 +1800,35 @@ async function waitForHarmonyTarget( continue; } const display = await harmonyDisplay(target.connectKey).catch(() => null); - if (!display) continue; + // A guest that has not composited yet answers `0x0`. The manager side of + // this comparison refuses zero as a panel for the same reason, so reading + // it as someone else's would have the two sides disagree about the one + // value that joins them. + if (!display || display.width <= 0 || display.height <= 0) { + sawUnprobeable = true; + continue; + } if (sameHarmonyPanel(display, expected)) confirmed.push(target.connectKey); - else rejected.add(target.connectKey); + else sawMismatch = true; } if (confirmed.length === 1) { - return { key: confirmed[0]!, ambiguous: false, mismatched: false }; + return { key: confirmed[0]!, unconfirmed: !expected }; } // Two confirmed arrivals is a refusal to guess, not a slow boot, and it is // answered here rather than latched: polling on would spend the rest of the // budget on a decision already made, and would turn the refusal into a // guess the moment one of the two flapped and left the other alone. - if (confirmed.length > 1) return { key: null, ambiguous: true, mismatched: false }; + if (confirmed.length > 1) return { key: null, reason: "ambiguous" }; checkAlive(); const remaining = deadline - Date.now(); if (remaining <= 0) { - return { key: null, ambiguous: false, mismatched: rejected.size > 0 }; + // A target that registered and was declined, or one that registered and + // never answered, are both distinct from nothing arriving at all — the + // budget is not what failed, so the caller must not be sent to raise it. + return { + key: null, + reason: sawMismatch ? "mismatched" : sawUnprobeable ? "unprobeable" : "none", + }; } await new Promise((r) => setTimeout(r, Math.min(HARMONY_TARGET_POLL_MS, remaining))); } @@ -1789,19 +1848,28 @@ async function waitForHarmonyTarget( * * The probe is a real `dumpLayout` rather than something cheaper because that * is the call the gap is in; `hidumper` answers throughout it. + * + * `checkAlive` is polled here for the same reason the wait for the target polls + * it: an emulator that dies in this window answers no probe, which is exactly + * what one still starting its window service looks like. Without it the manager + * can be gone — with its diagnostic already latched — for the whole remaining + * budget, and the boot still reports a drivable id under a note telling the + * caller to retry. */ -async function waitForHarmonyDrivable(connectKey: string, deadline: number): Promise { +async function waitForHarmonyDrivable( + connectKey: string, + deadline: number, + checkAlive: () => void +): Promise { const probePath = join(tmpdir(), `argent-harmony-bootprobe-${process.hrtime.bigint()}.json`); try { for (;;) { - if ( - await harmonyDumpLayout(connectKey, probePath).then( - () => true, - () => false - ) - ) { - return true; - } + const answered = await harmonyDumpLayout(connectKey, probePath).then( + () => true, + () => false + ); + if (answered) return true; + checkAlive(); const remaining = deadline - Date.now(); if (remaining <= 0) return false; await new Promise((r) => setTimeout(r, Math.min(HARMONY_TARGET_POLL_MS, remaining))); @@ -1959,27 +2027,39 @@ async function bootHarmonyImpl(params: { assertEmulatorAlive(); } - const outcome = hdcAvailable + const outcome: HarmonyTargetOutcome = hdcAvailable ? await waitForHarmonyTarget(before, bootDeadline, assertEmulatorAlive, expectedPanel) - : { key: null, ambiguous: false, mismatched: false }; + : { key: null }; const connectKey = outcome.key; // Registered is not driveable. The remaining budget rather than a slice of // it: the key is already in hand, so there is nothing else left to spend it // on, and an expiry is reported rather than walked into. - const drivable = connectKey ? await waitForHarmonyDrivable(connectKey, bootDeadline) : false; - - const note = connectKey - ? drivable - ? undefined - : HARMONY_NOT_DRIVABLE - : !hdcAvailable - ? HARMONY_NO_HDC - : outcome.ambiguous + const drivable = connectKey + ? await waitForHarmonyDrivable(connectKey, bootDeadline, assertEmulatorAlive) + : false; + + // Collected rather than picked: a key can be both unchecked and not yet + // answering, and dropping either caveat for the other would leave the payload + // asserting something the boot did not establish. + const caveats: string[] = []; + if (connectKey) { + if (outcome.unconfirmed) caveats.push(HARMONY_UNCONFIRMED_TARGET); + if (!drivable) caveats.push(HARMONY_NOT_DRIVABLE); + } else if (!hdcAvailable) { + caveats.push(HARMONY_NO_HDC); + } else { + caveats.push( + outcome.reason === "ambiguous" ? HARMONY_AMBIGUOUS_TARGET - : outcome.mismatched + : outcome.reason === "mismatched" ? HARMONY_MISMATCHED_TARGET - : HARMONY_NO_TARGET; + : outcome.reason === "unprobeable" + ? HARMONY_UNPROBEABLE_TARGET + : HARMONY_NO_TARGET + ); + } + const note = caveats.length > 0 ? caveats.join(" ") : undefined; return { platform: "harmony", @@ -2033,7 +2113,7 @@ Pick the platform by which argument you pass: 'udid' for an iOS simulator from l Use at the start of a session once you have picked a target. Returns a tagged payload: { platform: 'ios', udid, booted } or { platform: 'android', serial, avdName, booted } or { platform: 'vega', serial, vvdImage, booted } or { platform: 'harmony', udid, instanceName, booted, note? } or { platform: 'chromium', id, port, pid, booted } (an Electron app boots as a Chromium/CDP device). Android boots take 2–10 minutes depending on machine and cold/warm state; the tool transparently hot-boots from the AVD's default_boot snapshot when usable and falls back to cold boot otherwise. Vega starts the single SDK-managed VVD via the vega CLI (~10s) and returns once it reports running. If an Android/Electron boot stage fails, the tool terminates the device it spawned so the next retry starts clean. -HarmonyOS hands the instance to DevEco Studio's Emulator manager, then waits for it to register with 'hdc' and returns its connect key as 'udid' — the id the interaction tools drive. The manager itself reports no readiness signal and never names a port, so an instance that is already running is left alone unless you pass 'force: true'; whenever the key could not be resolved the payload names the instance instead and carries a 'note' saying why. Huawei serves the emulator images only within mainland China, so on a host outside it no instance can exist to start.`, +HarmonyOS hands the instance to DevEco Studio's Emulator manager, then waits for it to register with 'hdc' and returns its connect key as 'udid' — the id the interaction tools drive. The manager itself reports no readiness signal and never names a port, so an instance that is already running is left alone unless you pass 'force: true'; when the key could not be resolved the payload names the instance instead, and a 'note' is present whenever the boot left something unproven — no key and why, or a key that is not yet answering or could not be checked against the instance. Huawei serves the emulator images only within mainland China, so on a host outside it no instance can exist to start.`, alwaysLoad: true, searchHint: "boot start launch simulator emulator avd device session ios android vega vvd firetv harmony harmonyos deveco cold hot", diff --git a/packages/tool-server/src/utils/harmony-cli.ts b/packages/tool-server/src/utils/harmony-cli.ts index 3e0d16f7cf..ac74a341e2 100644 --- a/packages/tool-server/src/utils/harmony-cli.ts +++ b/packages/tool-server/src/utils/harmony-cli.ts @@ -124,20 +124,20 @@ const DEFAULT_TIMEOUT_MS = 30_000; /** See `HDC_KILL_SIGNAL`: a SIGTERM the child may ignore leaves `timeout` unenforced. */ const EMULATOR_KILL_SIGNAL = "SIGKILL" as const; +/** Said by every caller that needs the manager, so they cannot drift apart. */ +export const EMULATOR_NOT_FOUND = + "The HarmonyOS `Emulator` manager was not found. Install DevEco Studio: a macOS install " + + "at /Applications/DevEco-Studio.app is found on its own, and anywhere else set " + + "`$DEVECO_STUDIO_HOME` to the directory holding `tools/emulator/Emulator` (on macOS " + + "that is the `DevEco-Studio.app` bundle, or the `Contents` directory inside it). " + + "Then retry."; + export async function runHarmonyEmulator( args: string[], timeoutMs = DEFAULT_TIMEOUT_MS ): Promise { const bin = await resolveHarmonyEmulator(); - if (!bin) { - throw new Error( - "The HarmonyOS `Emulator` manager was not found. Install DevEco Studio: a macOS install " + - "at /Applications/DevEco-Studio.app is found on its own, and anywhere else set " + - "`$DEVECO_STUDIO_HOME` to the directory holding `tools/emulator/Emulator` (on macOS " + - "that is the `DevEco-Studio.app` bundle, or the `Contents` directory inside it). " + - "Then retry." - ); - } + if (!bin) throw new Error(EMULATOR_NOT_FOUND); try { const { stdout, stderr } = await execFileAsync(bin, args, { timeout: timeoutMs, diff --git a/packages/tool-server/test/boot-device-harmony.test.ts b/packages/tool-server/test/boot-device-harmony.test.ts index 522c29c780..373e3a50cd 100644 --- a/packages/tool-server/test/boot-device-harmony.test.ts +++ b/packages/tool-server/test/boot-device-harmony.test.ts @@ -418,6 +418,121 @@ describe("boot-device — HarmonyOS emulator path", () => { expect(result.note).toMatch(/still not answering `uitest`/); }); + it("reports the manager's death during the readiness wait, not a slow guest", async () => { + // The wait for the target polls the exit latch for exactly this reason, and + // the readiness wait sits in the same window: a dead emulator answers no + // probe, which is what one still starting its window service looks like. + // Without the latch the boot spends the rest of the budget on a corpse and + // then hands back a drivable id telling the caller to retry. + harmonyDumpLayout.mockImplementation(() => { + child.die("error: the emulator instance quit unexpectedly (disk image corrupted)"); + return Promise.reject(new Error("DumpLayout failed:Get window nodes failed")); + }); + targets([PHONE], [PHONE, emulatorTarget]); + + await expect(boot({ bootTimeoutMs: 30_000 })).rejects.toThrow(/disk image corrupted/); + }); + + it("says the key rests on arrival alone when the instance has no panel to check it against", async () => { + // A multi-display profile keys its LCDs differently, so `display` is null on + // a perfectly good instance — and the check that separates it from another + // device reconnecting in the same window has nothing to compare against. + // Returning the key unremarked is how a later tap lands on that device. + listHarmonyInstances.mockResolvedValue([ + { + name: INSTANCE, + deviceType: "Phone", + osVersion: "HarmonyOS 6.1.1(24)", + running: false, + display: null, + }, + ]); + targets([PHONE], [PHONE, emulatorTarget]); + + const result = (await boot({})) as { udid: string; note?: string }; + + expect(result.udid).toBe(`harmony-${EMULATOR_KEY}`); + expect(result.note).toMatch(/does not describe a single panel/); + expect(harmonyDisplay).not.toHaveBeenCalled(); + }); + + it("re-probes a target that read wrong once instead of disqualifying it for the boot", async () => { + // A guest mid-boot has not settled: this platform's flagship form factors + // are foldables, whose resolution changes with the fold. Latching the first + // reading rejects the instance argent itself started, for the whole budget. + harmonyDisplay + .mockResolvedValueOnce({ width: 1080, height: 2340, screenOn: true }) + .mockResolvedValue({ ...PANEL, screenOn: true }); + targets([PHONE], [PHONE, emulatorTarget]); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 30_000 }); + await vi.advanceTimersByTimeAsync(31_000); + const result = (await pending) as { udid: string; note?: string }; + + expect(result.udid).toBe(`harmony-${EMULATOR_KEY}`); + expect(result.note).toBeUndefined(); + expect(harmonyDisplay.mock.calls.length).toBeGreaterThan(1); + }); + + it("reads a 0x0 panel as a guest that has not composited, not as another device", async () => { + // The manager side refuses zero as a panel, so reading it as someone else's + // would have the two sides of the one joining value disagree — and would + // report a guest that never got as far as compositing as proof that the + // target belongs to some other device. + harmonyDisplay.mockResolvedValue({ width: 0, height: 0, screenOn: true }); + targets([PHONE], [PHONE, emulatorTarget]); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 30_000 }); + await vi.advanceTimersByTimeAsync(31_000); + const result = (await pending) as { udid: string; note?: string }; + + expect(result.note).toMatch(/never reported its display/); + expect(result.note).not.toMatch(/is not the panel/); + }); + + it("does not blame `hdc` registration for a target that registered but never reported a panel", async () => { + // "had not registered with `hdc`" is a plain untruth here, and it sends the + // caller to raise a budget that was never the problem. + harmonyDisplay.mockRejectedValue(new Error("hidumper produced no render resolution")); + targets([PHONE], [PHONE, emulatorTarget]); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 30_000 }); + await vi.advanceTimersByTimeAsync(31_000); + const result = (await pending) as { udid: string; note?: string }; + + expect(result.udid).toBe(`harmony-emulator-${INSTANCE}`); + expect(result.note).toMatch(/never reported its display/); + expect(result.note).not.toMatch(/had not registered/); + }); + + it("carries both caveats when the key is neither checked nor answering", async () => { + // One `note` field, two independent things left unproven. Dropping either + // for the other has the payload assert something the boot did not establish. + listHarmonyInstances.mockResolvedValue([ + { + name: INSTANCE, + deviceType: "Phone", + osVersion: "HarmonyOS 6.1.1(24)", + running: false, + display: null, + }, + ]); + harmonyDumpLayout.mockRejectedValue(new Error("DumpLayout failed:Get window nodes failed")); + targets([PHONE], [PHONE, emulatorTarget]); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 30_000 }); + await vi.advanceTimersByTimeAsync(31_000); + const result = (await pending) as { udid: string; note?: string }; + + expect(result.udid).toBe(`harmony-${EMULATOR_KEY}`); + expect(result.note).toMatch(/does not describe a single panel/); + expect(result.note).toMatch(/still not answering `uitest`/); + }); + it("names the instance and says why when nothing registers within the budget", async () => { targets([PHONE]); vi.useFakeTimers(); From 66efe56083f09d0710e4b5c7962775451b2e9f3d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 18:25:35 +0200 Subject: [PATCH 043/110] fix(boot-device): refuse the harmony boot when the pre-start snapshot cannot be read The snapshot went through a helper that swallows a failed listing as an empty one - the opposite meaning. Empty, every emulator already connected counts as this boot's arrival, and a peer off the same device profile answers the same panel, so the confirmation passes it too and the boot hands back a drivable id for the wrong device. It is retried once, since the hdc daemon restarting after a -stop is both the likely cause and one that clears, then refused while there is still nothing spawned. hdc is now resolved before the snapshot rather than after the spawn, so a host without the connector still takes the graceful path instead of failing on a listing it was never going to get. Nothing pinned any of this: deleting the whole snapshot passed all 58 harmony tests, because every fixture's pre-existing row was USB or Offline, i.e. excluded by one of the two filters that shadow it. Same for the label arm of the layout hoist and the readiness probe's temp file - both mutable to no-ops with the suite green. All three have tests now. --- .../src/tools/devices/boot-device.ts | 43 +++++++++--- .../test/boot-device-harmony.test.ts | 67 ++++++++++++++++++- .../test/harmony-layout-parser.test.ts | 19 ++++++ 3 files changed, 119 insertions(+), 10 deletions(-) diff --git a/packages/tool-server/src/tools/devices/boot-device.ts b/packages/tool-server/src/tools/devices/boot-device.ts index 6e786c3075..4f7a6b62bc 100644 --- a/packages/tool-server/src/tools/devices/boot-device.ts +++ b/packages/tool-server/src/tools/devices/boot-device.ts @@ -1632,9 +1632,32 @@ async function connectedHarmonyTargets() { * `Offline` is also what a connection blip leaves behind, so a foreign device * reconnecting is an arrival by this rule too. {@link waitForHarmonyTarget} * settles that with the panel the instance is configured with. + * + * A failed listing is not an empty one. Every later poll can shrug one off as + * "nothing new yet", but the baseline cannot: empty, it makes every emulator + * already connected count as this boot's arrival, and a peer off the same + * device profile answers the same panel — so nothing downstream catches it and + * the boot hands back a drivable id for the wrong device. Retried once, since + * the `hdc` daemon restarting after a `-stop` is both the likely cause and one + * that clears, then refused while there is still nothing spawned to clean up. */ async function connectedHarmonyKeys(): Promise> { - return new Set((await connectedHarmonyTargets()).map((t) => t.connectKey)); + for (let attempt = 0; ; attempt++) { + try { + const targets = await listHarmonyHdcTargets(); + return new Set(targets.filter((t) => t.state === "Connected").map((t) => t.connectKey)); + } catch (err) { + if (attempt > 0) { + throw new Error( + "Could not list `hdc` targets before starting the HarmonyOS instance, so the target " + + "it registers under could not have been told apart from those already connected: " + + (err instanceof Error ? err.message : String(err)), + { cause: err } + ); + } + await new Promise((r) => setTimeout(r, HARMONY_TARGET_POLL_MS)); + } + } } /** @@ -1960,12 +1983,19 @@ async function bootHarmonyImpl(params: { } } + // Asked rather than waited out: with no connector there is no target list to + // watch, so the wait would spend the whole budget concluding that nothing + // registered — blaming the emulator for the connector's absence. Asked before + // the snapshot because the snapshot is that same target list, and refuses the + // boot when it cannot be read. + const hdcAvailable = Boolean(await resolveHdc()); + // Snapshot immediately before the start, so a target already driveable // (another emulator, a phone on USB) is excluded by identity rather than by // any assumption about how an emulator's key is spelled. Reaching `Connected` // after this point is what marks the instance just booted — including on the // key it held before, since a stopped instance's row survives as `Offline`. - const before = await connectedHarmonyKeys(); + const before = hdcAvailable ? await connectedHarmonyKeys() : new Set(); // The panel this instance is configured with, which the guest reports back as // its `render resolution` — the only thing that separates the instance // reconnecting from another device doing the same. Read before the start @@ -1974,10 +2004,6 @@ async function bootHarmonyImpl(params: { const emulator = await startHarmonyEmulator(params.instanceName); - // Asked rather than waited out: with no connector there is no target list to - // watch, so the wait would spend the whole budget concluding that nothing - // registered — blaming the emulator for the connector's absence. - const hdcAvailable = Boolean(await resolveHdc()); const assertEmulatorAlive = () => { { const exit = emulator.exited(); @@ -2017,9 +2043,8 @@ async function bootHarmonyImpl(params: { // note whose first clause, "the instance started", nothing has checked. // // Asking once is not enough. Nothing between the spawn and here does any I/O - // (`resolveHdc` is a warm memo by this point, primed by the pre-start - // snapshot), so a single check runs microtasks after the spawn, before the - // child could have exited. The grace is small and bounded because a failing + // at all, so a single check runs microtasks after the spawn, before the child + // could have exited. The grace is small and bounded because a failing // `-start` prints and exits at once — this must not become the whole-budget // wait the connector-absent path exists to avoid. if (!hdcAvailable) { diff --git a/packages/tool-server/test/boot-device-harmony.test.ts b/packages/tool-server/test/boot-device-harmony.test.ts index 373e3a50cd..877f05da71 100644 --- a/packages/tool-server/test/boot-device-harmony.test.ts +++ b/packages/tool-server/test/boot-device-harmony.test.ts @@ -1,5 +1,5 @@ import { EventEmitter } from "node:events"; -import { writeFileSync } from "node:fs"; +import { existsSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { describe, it, expect, vi, beforeEach, afterEach } from "vitest"; @@ -74,6 +74,8 @@ const emulatorTarget = { connectKey: EMULATOR_KEY, connection: "TCP", state: "Co * comes back on that same port. */ const staleEmulatorTarget = { connectKey: EMULATOR_KEY, connection: "TCP", state: "Offline" }; +/** A second emulator, already up and driveable when this boot starts. */ +const foreignTarget = { connectKey: "127.0.0.1:5559", connection: "TCP", state: "Connected" }; /** `hw.lcd.single.width`/`height` of the phone image, echoed by the guest's `render resolution`. */ const PANEL = { width: 1320, height: 2856 }; const logPath = join(tmpdir(), `argent-harmony-${INSTANCE}.log`); @@ -418,6 +420,69 @@ describe("boot-device — HarmonyOS emulator path", () => { expect(result.note).toMatch(/still not answering `uitest`/); }); + it("ignores a second emulator that was connected and driveable before the start", async () => { + // The pre-start snapshot is the only thing standing between this boot and a + // peer emulator: it is TCP like ours, so the USB filter passes it, and off + // the same device profile it answers the same panel, so the confirmation + // passes it too. Every other fixture's pre-existing row is USB or + // `Offline`, i.e. excluded by something else. + targets([PHONE, foreignTarget], [PHONE, foreignTarget, emulatorTarget]); + + const result = (await boot({})) as { udid: string }; + + expect(result.udid).toBe(`harmony-${EMULATOR_KEY}`); + }); + + it("refuses rather than starting blind when the pre-start listing cannot be read", async () => { + // A failed listing read as an empty one makes every already-connected + // emulator this boot's arrival. Refused before the spawn, so there is no + // instance left running behind the error. + listHarmonyHdcTargets.mockRejectedValue(new Error("[Fail]Connect server failed")); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 30_000 }); + const assertion = expect(pending).rejects.toThrow(/could not have been told apart/); + await vi.advanceTimersByTimeAsync(31_000); + await assertion; + + expect(spawnMock).not.toHaveBeenCalled(); + }); + + it("takes the snapshot on a retry rather than failing the boot for one bad listing", async () => { + // The `hdc` daemon restarting after a `-stop` is the likely cause and it + // clears, so one refusal must not cost the caller the boot. + let call = 0; + listHarmonyHdcTargets.mockImplementation(() => { + call += 1; + if (call === 1) return Promise.reject(new Error("[Fail]Connect server failed")); + return Promise.resolve(call <= 2 ? [PHONE] : [PHONE, emulatorTarget]); + }); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 30_000 }); + await vi.advanceTimersByTimeAsync(31_000); + const result = (await pending) as { udid: string; note?: string }; + + expect(result.udid).toBe(`harmony-${EMULATOR_KEY}`); + expect(result.note).toBeUndefined(); + }); + + it("removes the readiness probe's dump instead of leaving one per boot in tmpdir", async () => { + // A full `uitest dumpLayout` JSON, and nothing else prunes it. + const probes: string[] = []; + harmonyDumpLayout.mockImplementation((_key: unknown, path: string) => { + probes.push(path); + writeFileSync(path, "{}"); + return Promise.resolve({ attributes: {} }); + }); + targets([PHONE], [PHONE, emulatorTarget]); + + await boot({}); + + expect(probes).toHaveLength(1); + expect(existsSync(probes[0]!)).toBe(false); + }); + it("reports the manager's death during the readiness wait, not a slow guest", async () => { // The wait for the target polls the exit latch for exactly this reason, and // the readiness wait sits in the same window: a dead emulator answers no diff --git a/packages/tool-server/test/harmony-layout-parser.test.ts b/packages/tool-server/test/harmony-layout-parser.test.ts index 9728922b76..fddfa95ca7 100644 --- a/packages/tool-server/test/harmony-layout-parser.test.ts +++ b/packages/tool-server/test/harmony-layout-parser.test.ts @@ -143,6 +143,25 @@ describe("parseHarmonyLayout", () => { expect(target.children[0].label).toBe("Wi-Fi"); }); + it("keeps a container carrying the label an agent selects on", () => { + // `.accessibilityText()` on a composite row is the idiomatic way to name + // one, and it lands on the `Column`/`Row` the author wrote. Hoisted away + // for wearing a layout type, the row's name is gone from the tree + // entirely — `await-ui-element {text}` and tap-by-label stop finding an + // element that is plainly on screen. One label per source attribute, + // since each reaches `label` by its own branch. + const cases: Record[] = [ + { type: "Column", description: "Wi-Fi, connected" }, + { type: "Row", hint: "Wi-Fi, connected" }, + ]; + for (const attrs of cases) { + const target = wrapping(attrs).children[0].children[0]; + expect(target.label).toBe("Wi-Fi, connected"); + // Hoisted, this position would hold the `Text` instead. + expect(target.children[0].label).toBe("Wi-Fi"); + } + }); + it("keeps a container that knows it is disabled, selected or focused", () => { // One flag per case, so each is pinned on its own rather than by an `id` // that would have kept the node regardless. From bc901b52cd432b2c0b87b3f9660c9c9f8ba91c23 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 18:47:40 +0200 Subject: [PATCH 044/110] fix(boot-device): read a refused hdc listing as a refusal, not an empty device table The previous commit's pre-start refusal could not fire for the failure it was written for. hdc exits 0 for its own errors, so listHarmonyHdcTargets turned every one into [], and only a spawn error or a SIGKILLed timeout ever reached the catch - excluding the daemon restarting after a -stop, which is the case the commit cited. Verified against a stub hdc printing [Fail]Connect server failed: the tolerant listing resolves to []. The listing now comes in two forms, and the snapshot takes the strict one. A caller polling for a change still wants [] - nothing new yet - but a baseline cannot, since an empty baseline silently means everything that appears next is new. Three notes claimed more than the code checked: - The manager-death diagnostic said the instance died "before it registered with hdc" on the readiness path, where the key had been resolved and is what the failing probe was aimed at. - The unconfirmed-target note blamed the instance's configuration for having no panel, when a listing that failed, timed out, or simply did not mention the instance reaches the same place. - The unprobeable-target note ruled out a longer budget, though a guest that has not finished compositing answers 0x0 and lands there too. Also: describe's harmony dispatch entry had nothing pinning it, though the wiring test it belongs in claims to cover exactly this class - delete the entry and the suite stayed green while the platform's primary discovery tool 501s on a real device. And the layout parser trimmed dump attributes unguarded, so one non-string value threw a bare TypeError out of describe instead of costing a single attribute. --- .../skills/argent-device-interact/SKILL.md | 2 +- .../tool-server/src/tools/describe/index.ts | 5 +- .../platforms/harmony/layout-parser.ts | 24 ++++-- .../src/tools/devices/boot-device.ts | 78 ++++++++++++------- .../src/tools/gesture-swipe/index.ts | 2 +- .../tool-server/src/utils/harmony-apps.ts | 10 ++- .../tool-server/src/utils/harmony-devices.ts | 32 +++++++- .../test/boot-device-harmony.test.ts | 35 ++++++--- .../test/harmony-dispatch-wiring.test.ts | 41 ++++++++-- .../test/harmony-hdc-targets.test.ts | 40 ++++++++++ .../test/harmony-layout-parser.test.ts | 17 ++++ 11 files changed, 227 insertions(+), 59 deletions(-) create mode 100644 packages/tool-server/test/harmony-hdc-targets.test.ts diff --git a/packages/skills/skills/argent-device-interact/SKILL.md b/packages/skills/skills/argent-device-interact/SKILL.md index 59de58ffca..abd325942b 100644 --- a/packages/skills/skills/argent-device-interact/SKILL.md +++ b/packages/skills/skills/argent-device-interact/SKILL.md @@ -21,7 +21,7 @@ For platform-specific caveats (Metro `adb reverse`, locked-screen describe error If you delegate simulator tasks to sub-agents, make sure they have MCP permissions. -Use `list-devices` to get a target id. Results are tagged with `platform` (`ios`, `ios-remote`, `android`, `chromium`, `vega`, or `harmony`); booted/ready devices come first. A `vega` target is a TV; drive it with `argent-tv-interact` (see the note above). A `harmony` target with `kind: "device"` is a HarmonyOS phone on `hdc`: the tools in this skill drive it, except `gesture-pinch`, `gesture-rotate`, `gesture-custom` and `rotate`, which are refused as unsupported on harmony — the device injects whole gestures one contact at a time, so there is no long-press or multi-finger path. Use `button` for home/back/power only. A `harmony` target with `kind: "emulator"` is a DevEco Studio instance that only `boot-device` accepts; boot it and drive the `harmony-` id it returns. Pick the first entry that matches the platform you need — if none are ready, call `boot-device` with `udid` (iOS), `avdName` (Android), or `electronAppPath` (boots an Electron app as a `chromium` device). A Chromium browser already running with a CDP port shows up directly — no `boot-device` needed. See `argent-ios-simulator-setup` / `argent-android-emulator-setup` for full setup flow. +Use `list-devices` to get a target id. Results are tagged with `platform` (`ios`, `ios-remote`, `android`, `chromium`, `vega`, or `harmony`); booted/ready devices come first. A `vega` target is a TV; drive it with `argent-tv-interact` (see the note above). A `harmony` target with `kind: "device"` is a HarmonyOS phone on `hdc`: the tools in this skill drive it, except `gesture-pinch`, `gesture-rotate`, `gesture-custom` and `rotate`, which are refused as unsupported on harmony — the device takes a whole gesture per call and exposes no multi-contact form of it. Use `button` for home/back/power only. A `harmony` target with `kind: "emulator"` is a DevEco Studio instance that only `boot-device` accepts; boot it and drive the `harmony-` id it returns. Pick the first entry that matches the platform you need — if none are ready, call `boot-device` with `udid` (iOS), `avdName` (Android), or `electronAppPath` (boots an Electron app as a `chromium` device). A Chromium browser already running with a CDP port shows up directly — no `boot-device` needed. See `argent-ios-simulator-setup` / `argent-android-emulator-setup` for full setup flow. **Load tool schemas before first use.** Gesture tools (`gesture-tap`, `gesture-swipe`, `gesture-pinch`, `gesture-rotate`, `gesture-custom`) may be deferred — their parameter schemas are not loaded until fetched. Always use ToolSearch to load the schemas of all gesture tools you plan to use **before** calling any of them. If you skip this step, parameters may be coerced to strings instead of numbers, causing validation errors. diff --git a/packages/tool-server/src/tools/describe/index.ts b/packages/tool-server/src/tools/describe/index.ts index f50395f593..e2eab9f01c 100644 --- a/packages/tool-server/src/tools/describe/index.ts +++ b/packages/tool-server/src/tools/describe/index.ts @@ -161,8 +161,9 @@ On Vega (Fire TV), reads the on-device automation toolkit (\`getPageSource\`); e (Vega is remote-driven, not touch). If describe returns an empty tree on Vega, relaunch the foreground app (the toolkit attaches at launch) and try again. On HarmonyOS, runs the device's own \`uitest dumpLayout\` (\`source: "harmony-uitest"\`); each top-level -child is a window labelled with the bundle that owns it, so an app plus the status bar reads as two -named windows. A suspended display still dumps its last-composited windows, indistinguishable in the +child is a window carrying the owning bundle as its \`identifier\` (not its label, which would make a +full-screen node match a text search for the app's own name), so an app plus the status bar reads as +two windows. A suspended display still dumps its last-composited windows, indistinguishable in the tree from a live screen, so the result carries a \`hint\` saying the panel is off — taps land nowhere until you wake it with \`button\` (power). An empty dump is hinted the same way, as an app that may still be starting. diff --git a/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts b/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts index 62a7ff6dc6..285bad931b 100644 --- a/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts +++ b/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts @@ -103,12 +103,24 @@ function isScrollable(attrs: Record): boolean { * `value`. `hint` is the placeholder of an empty field, which is the only label * such a field has. */ +/** + * One attribute as trimmed text. + * + * The dump is `JSON.parse`d and asserted to be all-strings, which is measured + * on 6.1.1 but not guaranteed by anything: a single numeric or boolean value + * would otherwise throw a bare `TypeError` out of the parser and take the whole + * `describe` down, with no failure signal, rather than costing one attribute. + */ +function attrText(value: unknown): string { + return typeof value === "string" ? value.trim() : ""; +} + export function harmonyLabel(attrs: Record): string { - const description = (attrs.description ?? "").trim(); + const description = attrText(attrs.description); if (description) return description; - const text = (attrs.text ?? "").trim(); + const text = attrText(attrs.text); if (text) return text; - return (attrs.hint ?? "").trim(); + return attrText(attrs.hint); } interface HarmonyNode { @@ -175,9 +187,9 @@ function build(node: HarmonyLayoutNode): HarmonyNode[] { if (label) own.label = label; // Only surface `text` separately when `description` already claimed the label // slot and the two genuinely differ — otherwise `value` just repeats `label`. - const text = (attrs.text ?? "").trim(); + const text = attrText(attrs.text); if (text && text !== own.label) own.value = text; - const identifier = (attrs.id ?? "").trim() || (attrs.key ?? "").trim(); + const identifier = attrText(attrs.id) || attrText(attrs.key); if (identifier) own.identifier = identifier; // Scaffolding contributes nothing itself; hoist whatever survived beneath it. @@ -403,7 +415,7 @@ export function parseHarmonyLayout( const windows: DescribeNode[] = []; for (const child of root.children ?? []) { const attrs = child.attributes ?? {}; - const bundle = (attrs.bundleName ?? "").trim(); + const bundle = attrText(attrs.bundleName); const built = build(child) .map((n) => toDescribeNode(n, screen.width, screen.height)) .filter((n) => !isFullyOffScreen(n.frame, n.children)); diff --git a/packages/tool-server/src/tools/devices/boot-device.ts b/packages/tool-server/src/tools/devices/boot-device.ts index 4f7a6b62bc..19e9345e73 100644 --- a/packages/tool-server/src/tools/devices/boot-device.ts +++ b/packages/tool-server/src/tools/devices/boot-device.ts @@ -59,7 +59,11 @@ import { resolveHarmonyEmulator, runHarmonyEmulator, } from "../../utils/harmony-cli"; -import { listHarmonyHdcTargets, listHarmonyInstances } from "../../utils/harmony-devices"; +import { + listHarmonyHdcTargets, + listHarmonyHdcTargetsStrict, + listHarmonyInstances, +} from "../../utils/harmony-devices"; import { resolveHdc } from "../../utils/harmony-hdc"; import { harmonyDisplay, harmonyDumpLayout } from "../../utils/harmony-uitest"; import { bootElectronApp, type ElectronBootResult } from "./boot-electron"; @@ -1568,27 +1572,33 @@ const HARMONY_MISMATCHED_TARGET = * Said when a target registered but never reported a panel to check it against. * Distinct from {@link HARMONY_MISMATCHED_TARGET}, which knows the target is * someone else's, and from {@link HARMONY_NO_TARGET}, whose "had not registered" - * would be a plain untruth here — the target did register, so a longer budget - * is not the remedy. + * would be a plain untruth here. + * + * A guest still composing its first frame answers `0x0` and reaches this too, + * so unlike the other refusals a longer budget CAN be the remedy — it is + * offered second, after the reading the caller can act on directly. */ const HARMONY_UNPROBEABLE_TARGET = - "A target registered while the instance was booting but never reported its display, so argent " + - "could not tell whether it is this instance or another device that reconnected, and did not " + - 'guess. Call `list-devices` and drive the `kind: "device"` entry for the instance.'; + "A target registered while the instance was booting but never reported a display argent could " + + "read, so it could not tell whether the target is this instance or another device that " + + "reconnected, and did not guess. A guest that has not finished compositing reads this way too, " + + 'so call `list-devices` and drive the `kind: "device"` entry for the instance, or give the ' + + "boot longer with `bootTimeoutMs`."; /** - * Said when the key rests on arrival alone. The instance's config does not - * describe a single panel, so the check that separates it from another device - * reconnecting in the same window had nothing to compare against. The key is - * returned because it is the only candidate that appeared and usually is the - * instance — but silently returning an unchecked id is how every later tap ends - * up on someone else's device, so the caller is told what it rests on. + * Said when the key rests on arrival alone, because no panel for this instance + * was on record to check it against. Deliberately does not say WHY there was + * none: the manager listing may have failed, or timed out, or not mentioned the + * instance, or described a profile that does not key a single LCD — and nothing + * here distinguishes those. The key is returned because it is the only + * candidate that appeared and usually is the instance, but silently returning + * an unchecked id is how every later tap ends up on someone else's device. */ const HARMONY_UNCONFIRMED_TARGET = - "This instance's configuration does not describe a single panel, so argent could not check the " + - "target it returned against it and matched on arrival timing alone. If another emulator or " + - "device reconnected while the instance was booting, this may be its connect key; confirm " + - "against `list-devices` before relying on it."; + "argent had no panel on record for this instance, so it could not check the target it returned " + + "against one and matched on arrival timing alone. If another emulator or device reconnected " + + "while the instance was booting, this may be its connect key; confirm against `list-devices` " + + "before relying on it."; /** * Said when the instance registered but never answered `uitest` in time. The @@ -1637,20 +1647,23 @@ async function connectedHarmonyTargets() { * "nothing new yet", but the baseline cannot: empty, it makes every emulator * already connected count as this boot's arrival, and a peer off the same * device profile answers the same panel — so nothing downstream catches it and - * the boot hands back a drivable id for the wrong device. Retried once, since - * the `hdc` daemon restarting after a `-stop` is both the likely cause and one - * that clears, then refused while there is still nothing spawned to clean up. + * the boot hands back a drivable id for the wrong device. Hence the STRICT + * listing: `hdc` exits 0 for its own failures, so the tolerant one reports a + * broken daemon as an empty device table. Retried once, since the daemon + * restarting after a `-stop` is both the likely cause and one that clears, + * then refused while there is still nothing spawned to clean up. */ async function connectedHarmonyKeys(): Promise> { for (let attempt = 0; ; attempt++) { try { - const targets = await listHarmonyHdcTargets(); + const targets = await listHarmonyHdcTargetsStrict(); return new Set(targets.filter((t) => t.state === "Connected").map((t) => t.connectKey)); } catch (err) { if (attempt > 0) { throw new Error( - "Could not list `hdc` targets before starting the HarmonyOS instance, so the target " + - "it registers under could not have been told apart from those already connected: " + + "Could not read `hdc`'s device table while preparing to start the HarmonyOS instance, " + + "so the target it registers under could not have been told apart from those already " + + "connected: " + (err instanceof Error ? err.message : String(err)), { cause: err } ); @@ -1749,8 +1762,9 @@ async function startHarmonyEmulator( * * `reason` says why there is no key, and is read only when there is none; * `unconfirmed` marks a key that arrival alone picked out. Both feed the boot's - * `note`, and each has to stay distinct there: only "none" is a budget the - * caller can usefully raise. + * `note`, and each has to stay distinct there, since they send the caller + * somewhere different: "ambiguous" and "mismatched" are decisions no budget + * changes, while "none" and "unprobeable" can both be a boot still in progress. */ type HarmonyTargetOutcome = { key: string | null; @@ -2004,6 +2018,9 @@ async function bootHarmonyImpl(params: { const emulator = await startHarmonyEmulator(params.instanceName); + // Set once the target wait resolves a key, so a death reported from the + // readiness wait can say which side of registration it happened on. + let resolvedKey: string | null = null; const assertEmulatorAlive = () => { { const exit = emulator.exited(); @@ -2013,11 +2030,15 @@ async function bootHarmonyImpl(params: { // printed is the only signal (see `harmony-cli.ts`). const diagnostic = emulatorFailure({ stdout: exit.output, stderr: "" }); if (!diagnostic) { + // Both waits poll this, and they sit on opposite sides of the key being + // resolved — saying "before it registered" during the readiness wait + // would deny the registration the failing probe is aimed at. + const when = resolvedKey + ? `while "${params.instanceName}" (\`${resolvedKey}\`) was still coming up` + : `before "${params.instanceName}" registered with \`hdc\``; throw new Error( - `The HarmonyOS emulator manager exited (${exit.reason}) before "${params.instanceName}" ` + - `registered with \`hdc\`, so nothing is running to drive. It printed: ${ - exit.output.trim() || "(nothing)" - }` + `The HarmonyOS emulator manager exited (${exit.reason}) ${when}` + + `, so nothing is running to drive. It printed: ${exit.output.trim() || "(nothing)"}` ); } const imageMissing = @@ -2060,6 +2081,7 @@ async function bootHarmonyImpl(params: { // Registered is not driveable. The remaining budget rather than a slice of // it: the key is already in hand, so there is nothing else left to spend it // on, and an expiry is reported rather than walked into. + resolvedKey = connectKey; const drivable = connectKey ? await waitForHarmonyDrivable(connectKey, bootDeadline, assertEmulatorAlive) : false; diff --git a/packages/tool-server/src/tools/gesture-swipe/index.ts b/packages/tool-server/src/tools/gesture-swipe/index.ts index 929be3638d..39447301cd 100644 --- a/packages/tool-server/src/tools/gesture-swipe/index.ts +++ b/packages/tool-server/src/tools/gesture-swipe/index.ts @@ -69,7 +69,7 @@ export const gestureSwipeTool: ToolDefinition = { failedMsg: ({ failureSignal }) => `Failed to swipe: ${failureSignal.error_code}`, }, description: `Execute a smooth swipe / drag touch gesture between two points on the device (iOS simulator, Android emulator, or HarmonyOS device). All from/to positions are normalized 0.0–1.0 (fractions of screen width/height, not pixels), same as gesture-tap. -Generates interpolated Move events for a natural feel (~60fps). +On iOS and Android, generates interpolated Move events for a natural feel (~60fps); HarmonyOS takes the whole gesture in one call and interpolates it on-device. Swipe up (fromY > toY) to scroll content down. Use when you need to scroll a list, dismiss a modal, drag an element, or navigate between pages. Not supported on Chromium — use gesture-scroll there instead. Pass settle:true for a momentum-free swipe that lands exactly where the finger lifts (no fling), when you need a deterministic scroll distance. Returns { swiped: true, timestampMs }. Fails if the simulator-server / emulator backend, or \`hdc\` on HarmonyOS, is not reachable for the given device.`, diff --git a/packages/tool-server/src/utils/harmony-apps.ts b/packages/tool-server/src/utils/harmony-apps.ts index 9e75690c35..d66a588e32 100644 --- a/packages/tool-server/src/utils/harmony-apps.ts +++ b/packages/tool-server/src/utils/harmony-apps.ts @@ -18,7 +18,12 @@ import { runHdcShell, shellQuote } from "./harmony-hdc"; const AA_SUCCESS = "start ability successfully."; interface HarmonyBundleEntry { - /** The ability to launch, as `bm` reports it — always fully qualified. */ + /** + * The ability to launch, in the spelling `aa start` accepts — which is the + * one `abilityInfos` uses, NOT `bm`'s `mainAbility`. The two differ on real + * bundles (Calculator: `CalculatorAbility` against a fully-qualified + * `mainAbility`), which is what {@link startableAbilityName} reconciles. + */ mainAbility: string; /** The HAP module the ability lives in; `aa start -m` needs it. */ module: string; @@ -223,7 +228,8 @@ export async function openHarmonyUrl(connectKey: string, url: string): Promise { + return listTargets(false); +} + +/** + * The same listing, refusing rather than answering `[]` when `hdc` could not + * produce one. + * + * `hdc` exits 0 whatever happens, so its `[Fail]` line is the only thing + * separating "nothing is connected" from "the device table could not be read". + * A caller polling for a change can treat those alike; one establishing a + * BASELINE to compare against cannot, since an empty baseline quietly means + * "everything that shows up next is new". + */ +export async function listHarmonyHdcTargetsStrict(): Promise { + return listTargets(true); +} + +async function listTargets(strict: boolean): Promise { if (!(await resolveHdc())) return []; const result = await runHdc(["list", "targets", "-v"], HDC_LIST_TIMEOUT_MS); - if (hdcFailure(result)) return []; + const failure = hdcFailure(result); + if (failure) { + if (strict) throw new Error(failure); + return []; + } return parseHdcTargets(result.stdout); } diff --git a/packages/tool-server/test/boot-device-harmony.test.ts b/packages/tool-server/test/boot-device-harmony.test.ts index 877f05da71..e4277377cb 100644 --- a/packages/tool-server/test/boot-device-harmony.test.ts +++ b/packages/tool-server/test/boot-device-harmony.test.ts @@ -9,6 +9,7 @@ const runHarmonyEmulator = vi.fn(); const resolveHarmonyEmulator = vi.fn(); const listHarmonyInstances = vi.fn(); const listHarmonyHdcTargets = vi.fn(); +const listHarmonyHdcTargetsStrict = vi.fn(); const resolveHdc = vi.fn(); const ensureDep = vi.fn(); const spawnMock = vi.fn(); @@ -32,6 +33,7 @@ vi.mock("../src/utils/harmony-cli", async () => { vi.mock("../src/utils/harmony-devices", () => ({ listHarmonyInstances: (...a: unknown[]) => listHarmonyInstances(...a), listHarmonyHdcTargets: (...a: unknown[]) => listHarmonyHdcTargets(...a), + listHarmonyHdcTargetsStrict: (...a: unknown[]) => listHarmonyHdcTargetsStrict(...a), })); vi.mock("../src/utils/harmony-hdc", async () => { const actual = await vi.importActual( @@ -105,9 +107,11 @@ interface HdcTargetRow { /** Successive `hdc list targets` results, the last one repeating forever. */ function targets(...rounds: HdcTargetRow[][]) { let call = 0; - listHarmonyHdcTargets.mockImplementation(() => - Promise.resolve(rounds[Math.min(call++, rounds.length - 1)]) - ); + // One counter across both: the snapshot takes the first round and the polls + // take the rest, exactly as when a single listing served both. + const next = () => Promise.resolve(rounds[Math.min(call++, rounds.length - 1)]); + listHarmonyHdcTargets.mockImplementation(next); + listHarmonyHdcTargetsStrict.mockImplementation(next); } beforeEach(() => { @@ -437,7 +441,7 @@ describe("boot-device — HarmonyOS emulator path", () => { // A failed listing read as an empty one makes every already-connected // emulator this boot's arrival. Refused before the spawn, so there is no // instance left running behind the error. - listHarmonyHdcTargets.mockRejectedValue(new Error("[Fail]Connect server failed")); + listHarmonyHdcTargetsStrict.mockRejectedValue(new Error("[Fail]Connect server failed")); vi.useFakeTimers(); const pending = boot({ bootTimeoutMs: 30_000 }); @@ -452,11 +456,13 @@ describe("boot-device — HarmonyOS emulator path", () => { // The `hdc` daemon restarting after a `-stop` is the likely cause and it // clears, so one refusal must not cost the caller the boot. let call = 0; - listHarmonyHdcTargets.mockImplementation(() => { + const next = () => { call += 1; if (call === 1) return Promise.reject(new Error("[Fail]Connect server failed")); return Promise.resolve(call <= 2 ? [PHONE] : [PHONE, emulatorTarget]); - }); + }; + listHarmonyHdcTargets.mockImplementation(next); + listHarmonyHdcTargetsStrict.mockImplementation(next); vi.useFakeTimers(); const pending = boot({ bootTimeoutMs: 30_000 }); @@ -495,7 +501,14 @@ describe("boot-device — HarmonyOS emulator path", () => { }); targets([PHONE], [PHONE, emulatorTarget]); - await expect(boot({ bootTimeoutMs: 30_000 })).rejects.toThrow(/disk image corrupted/); + // The key WAS resolved — it is what the failing probe is aimed at — so the + // message the pre-registration wait uses would deny the registration that + // had already happened. + const err = (await boot({ bootTimeoutMs: 30_000 }).catch((e: unknown) => e)) as Error; + + expect(err.message).toMatch(/disk image corrupted/); + expect(err.message).toMatch(/`127\.0\.0\.1:5555`\) was still coming up/); + expect(err.message).not.toMatch(/before "Phone_1" registered/); }); it("says the key rests on arrival alone when the instance has no panel to check it against", async () => { @@ -517,7 +530,7 @@ describe("boot-device — HarmonyOS emulator path", () => { const result = (await boot({})) as { udid: string; note?: string }; expect(result.udid).toBe(`harmony-${EMULATOR_KEY}`); - expect(result.note).toMatch(/does not describe a single panel/); + expect(result.note).toMatch(/no panel on record for this instance/); expect(harmonyDisplay).not.toHaveBeenCalled(); }); @@ -553,7 +566,7 @@ describe("boot-device — HarmonyOS emulator path", () => { await vi.advanceTimersByTimeAsync(31_000); const result = (await pending) as { udid: string; note?: string }; - expect(result.note).toMatch(/never reported its display/); + expect(result.note).toMatch(/never reported a display argent could read/); expect(result.note).not.toMatch(/is not the panel/); }); @@ -569,7 +582,7 @@ describe("boot-device — HarmonyOS emulator path", () => { const result = (await pending) as { udid: string; note?: string }; expect(result.udid).toBe(`harmony-emulator-${INSTANCE}`); - expect(result.note).toMatch(/never reported its display/); + expect(result.note).toMatch(/never reported a display argent could read/); expect(result.note).not.toMatch(/had not registered/); }); @@ -594,7 +607,7 @@ describe("boot-device — HarmonyOS emulator path", () => { const result = (await pending) as { udid: string; note?: string }; expect(result.udid).toBe(`harmony-${EMULATOR_KEY}`); - expect(result.note).toMatch(/does not describe a single panel/); + expect(result.note).toMatch(/no panel on record for this instance/); expect(result.note).toMatch(/still not answering `uitest`/); }); diff --git a/packages/tool-server/test/harmony-dispatch-wiring.test.ts b/packages/tool-server/test/harmony-dispatch-wiring.test.ts index acec593633..ad0b0b6787 100644 --- a/packages/tool-server/test/harmony-dispatch-wiring.test.ts +++ b/packages/tool-server/test/harmony-dispatch-wiring.test.ts @@ -1,17 +1,19 @@ import { describe, it, expect, vi } from "vitest"; -// The four tools whose HarmonyOS support is a single `harmony: harmonyImpl` line -// in a `dispatchByPlatform` table. Every other harmony test reaches the backend -// by importing `harmonyImpl` and calling `.handler()` directly, which exercises -// the backend and never the wiring: delete all four lines and the rest of the -// suite stays green while each of these tools 501s on a real device. +// The tools whose HarmonyOS support is a single entry in a `dispatchByPlatform` +// table. Every other harmony test reaches the backend by importing it and +// calling it directly, which exercises the backend and never the wiring: delete +// those entries and the rest of the suite stays green while each of these tools +// 501s on a real device. `describe` is the one that matters most — it is the +// platform's primary discovery tool — and it is wired as an inline handler +// rather than a `harmonyImpl`, so it is covered separately below. // // Only the leaf handler is stubbed, deliberately — the claim under test is // dispatch, so a stub owning no device behaviour keeps these cases pinned to the // wiring while the backends underneath them keep changing. // Hoisted with the `vi.mock` factories that close over them — a plain top-level // const is still in its temporal dead zone when the factory runs. -const { launchStub, restartStub, openUrlStub, keyboardStub } = vi.hoisted(() => { +const { launchStub, restartStub, openUrlStub, keyboardStub, describeStub } = vi.hoisted(() => { // Full dispatch signature, so the device it was handed stays typed below. const stub = (result: T) => ({ handler: vi.fn( @@ -26,6 +28,16 @@ const { launchStub, restartStub, openUrlStub, keyboardStub } = vi.hoisted(() => restartStub: stub({ restarted: true, bundleId: "com.huawei.hmos.calculator" }), openUrlStub: stub({ opened: true, url: "https://example.com" }), keyboardStub: stub({ typed: "hi" }), + // `describe` dispatches to a bare function, not a `{ handler }` impl. + describeStub: vi.fn(async (_connectKey: string) => ({ + tree: { + role: "Screen", + frame: { x: 0, y: 0, width: 1, height: 1 }, + pixelBounds: null, + children: [], + }, + source: "harmony-uitest" as const, + })), }; }); @@ -45,11 +57,16 @@ vi.mock("../src/tools/keyboard/platforms/harmony", async (orig) => ({ ...(await orig>()), harmonyImpl: keyboardStub, })); +vi.mock("../src/tools/describe/platforms/harmony", async (orig) => ({ + ...(await orig>()), + describeHarmony: (...a: [string]) => describeStub(...a), +})); import { createLaunchAppTool } from "../src/tools/launch-app"; import { createRestartAppTool } from "../src/tools/restart-app"; import { openUrlTool } from "../src/tools/open-url"; import { createKeyboardTool } from "../src/tools/keyboard"; +import { createDescribeTool } from "../src/tools/describe"; const HARMONY_UDID = "harmony-025DEK236V035771"; const registry = { resolveService: vi.fn(async () => ({})) } as never; @@ -102,6 +119,18 @@ describe("HarmonyOS dispatch wiring", () => { } ); + it("describe routes a harmony device to its harmony branch", async () => { + // Without the dispatch entry this rejects with NotImplementedOnPlatform, so + // resolving with the harmony source pins that the branch ran — and the + // connect key pins that the device id was unwrapped on the way in. + describeStub.mockClear(); + + await expect( + createDescribeTool(registry).execute!({}, { udid: HARMONY_UDID } as never) + ).resolves.toMatchObject({ source: "harmony-uitest" }); + expect(describeStub).toHaveBeenCalledWith("025DEK236V035771"); + }); + it("does not reach a harmony branch for a device of another platform", async () => { // Guards the inverse mutation: a `harmony` entry wired onto the wrong arm of // the table would satisfy the cases above and silently hijack Android. diff --git a/packages/tool-server/test/harmony-hdc-targets.test.ts b/packages/tool-server/test/harmony-hdc-targets.test.ts new file mode 100644 index 0000000000..1098ec2b65 --- /dev/null +++ b/packages/tool-server/test/harmony-hdc-targets.test.ts @@ -0,0 +1,40 @@ +import { afterAll, beforeAll, describe, expect, it, vi } from "vitest"; +import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { listHarmonyHdcTargets, listHarmonyHdcTargetsStrict } from "../src/utils/harmony-devices"; + +// A real stub binary reached through the documented `$DEVECO_STUDIO_HOME` +// layout, as `harmony-hdc-timeout.test.ts` does: the behaviour under test is +// `hdc` exiting 0 while reporting a failure, which a mock at the run boundary +// would simply assert rather than reproduce. +const root = mkdtempSync(join(tmpdir(), "argent-deveco-targets-")); +const binDir = join(root, "sdk", "default", "openharmony", "toolchains"); +mkdirSync(binDir, { recursive: true }); +// Verbatim from `hdc list targets -v` with the daemon down, which is the state +// a `-stop` leaves behind while the connector restarts. +writeFileSync(join(binDir, "hdc"), '#!/usr/bin/env bash\necho "[Fail]Connect server failed"\n', { + mode: 0o755, +}); + +beforeAll(() => vi.stubEnv("DEVECO_STUDIO_HOME", root)); +afterAll(() => { + vi.unstubAllEnvs(); + rmSync(root, { recursive: true, force: true }); +}); + +describe("hdc target listing when the connector itself fails", () => { + it("reports a refused listing as an empty device table to a polling caller", async () => { + // `hdc` exits 0 whatever happens, so this cannot be told from success by + // status. A caller watching for a change wants exactly this: nothing new. + await expect(listHarmonyHdcTargets()).resolves.toEqual([]); + }); + + it("raises the refusal instead, for a caller that is establishing a baseline", async () => { + // An empty baseline silently means "everything that appears next is new", + // so `boot-device` would adopt an emulator that was already connected as + // the instance it just started. Rejecting is what makes that refusable — + // and it has to happen here, since the exit status never carries it. + await expect(listHarmonyHdcTargetsStrict()).rejects.toThrow("[Fail]Connect server failed"); + }); +}); diff --git a/packages/tool-server/test/harmony-layout-parser.test.ts b/packages/tool-server/test/harmony-layout-parser.test.ts index fddfa95ca7..5d84269478 100644 --- a/packages/tool-server/test/harmony-layout-parser.test.ts +++ b/packages/tool-server/test/harmony-layout-parser.test.ts @@ -143,6 +143,23 @@ describe("parseHarmonyLayout", () => { expect(target.children[0].label).toBe("Wi-Fi"); }); + it("costs one attribute, not the whole tree, when a dump value is not a string", () => { + // The dump is `JSON.parse`d and asserted to be all-strings — measured on + // 6.1.1, promised by nothing. Unguarded, one numeric `description` threw a + // bare TypeError out of the parser and took `describe` down with it. + const odd: Record = { + type: "Text", + description: 0 as unknown as string, + text: "Submit", + id: false as unknown as string, + bounds: "[0,300][1216,440]", + }; + const target = wrapping(odd).children[0].children[0]; + + expect(target.label).toBe("Submit"); + expect(target.identifier).toBeUndefined(); + }); + it("keeps a container carrying the label an agent selects on", () => { // `.accessibilityText()` on a composite row is the idiomatic way to name // one, and it lands on the `Column`/`Row` the author wrote. Hoisted away From 24e7d0ef8c48a541218dd38fefe51a3073d0e7f2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 18:56:54 +0200 Subject: [PATCH 045/110] test(harmony): prime the hdc probe the dispatch cases do not depend on describe's harmony branch declares `requires: ["hdc"]`, so dispatchByPlatform runs the dep preflight before the handler. With nothing priming it that probe reads the host: green on a machine with DevEco Studio installed, and a DependencyMissingError on a CI runner without it, which is where it landed. Prime the cache instead, as cross-platform-tool.test.ts already does for xcrun. Reproduced by forcing resolveHdc to null - the same failure as CI - and the cases pass under that with the prime in place. --- .../tool-server/test/harmony-dispatch-wiring.test.ts | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/packages/tool-server/test/harmony-dispatch-wiring.test.ts b/packages/tool-server/test/harmony-dispatch-wiring.test.ts index ad0b0b6787..045bef6d92 100644 --- a/packages/tool-server/test/harmony-dispatch-wiring.test.ts +++ b/packages/tool-server/test/harmony-dispatch-wiring.test.ts @@ -1,4 +1,5 @@ -import { describe, it, expect, vi } from "vitest"; +import { describe, it, expect, vi, beforeEach } from "vitest"; +import { __primeDepCacheForTests, __resetDepCacheForTests } from "../src/utils/check-deps"; // The tools whose HarmonyOS support is a single entry in a `dispatchByPlatform` // table. Every other harmony test reaches the backend by importing it and @@ -68,6 +69,15 @@ import { openUrlTool } from "../src/tools/open-url"; import { createKeyboardTool } from "../src/tools/keyboard"; import { createDescribeTool } from "../src/tools/describe"; +// `describe`'s harmony branch declares `requires: ["hdc"]`, so its preflight +// runs before the handler. Prime the probe rather than leave it to the host: +// `hdc` is present on a HarmonyOS dev machine and absent on a CI runner, and +// the claim under test here is dispatch either way. +beforeEach(() => { + __resetDepCacheForTests(); + __primeDepCacheForTests(["hdc"]); +}); + const HARMONY_UDID = "harmony-025DEK236V035771"; const registry = { resolveService: vi.fn(async () => ({})) } as never; From db13bed7f785b9dbd3a4a1ffef50e46df0df76aa Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 18:59:27 +0200 Subject: [PATCH 046/110] fix(harmony): catch the connector failure hdc reports on stderr with no prefix The previous commit keyed the refusal on hdc's [Fail] prefix. Measured on hdc 3.2.0d, that prefix belongs to device-level errors; `list targets` fails at the connector, one level below, and prints a bare Connect server failed to STDERR with stdout EMPTY and exit code 0. So the prefix missed it, the status missed it, and empty stdout parsed to a clean - and entirely wrong - "no devices are connected", which is the baseline the boot compares arrivals against. The stub in the previous commit's test printed the prefixed form on stdout, so it pinned a shape the binary never produces. The listing now reads both streams, and a diagnostic is told from a row by delimiter: rows are tab-separated (or a lone connect key without -v), prose is not. Any row that parses means a real listing printed and the guard stands down, so a diagnostic trailing real targets is not a refusal. Same rule in the parser, which read `Connect server failed` as three columns and yielded a target named `Connect` - a device that does not exist, offered to list-devices and counted in a boot's baseline. --- .../tool-server/src/utils/harmony-devices.ts | 32 +++++- .../test/harmony-hdc-targets.test.ts | 102 ++++++++++++++---- 2 files changed, 112 insertions(+), 22 deletions(-) diff --git a/packages/tool-server/src/utils/harmony-devices.ts b/packages/tool-server/src/utils/harmony-devices.ts index 66955e93c8..4bdaa4061a 100644 --- a/packages/tool-server/src/utils/harmony-devices.ts +++ b/packages/tool-server/src/utils/harmony-devices.ts @@ -177,6 +177,12 @@ export function parseHdcTargets(stdout: string): HarmonyHdcTarget[] { const trimmed = line.trim(); if (trimmed.length === 0 || trimmed.startsWith(HDC_EMPTY_SENTINEL)) continue; if (trimmed.startsWith("[")) continue; + // Prose, not a row. `hdc` prints some diagnostics with no `[Fail]` prefix + // and still exits 0 — `Connect server failed` is three words, i.e. the + // shape of a `-v` row, and became a target named `Connect`. The delimiter + // is what the two never share: a row is tab-separated, or a lone connect + // key when `-v` is absent. + if (!trimmed.includes("\t") && /\s/.test(trimmed)) continue; const cols = trimmed.split(/\s+/); if (cols.length === 1) { // `hdc list targets` without `-v` prints the bare connect key. @@ -203,7 +209,7 @@ export async function listHarmonyHdcTargets(): Promise { * The same listing, refusing rather than answering `[]` when `hdc` could not * produce one. * - * `hdc` exits 0 whatever happens, so its `[Fail]` line is the only thing + * `hdc` exits 0 whatever happens, so what it printed is the only thing * separating "nothing is connected" from "the device table could not be read". * A caller polling for a change can treat those alike; one establishing a * BASELINE to compare against cannot, since an empty baseline quietly means @@ -213,10 +219,32 @@ export async function listHarmonyHdcTargetsStrict(): Promise return listTargets(true); } +/** + * What `hdc` said instead of listing targets, or null if it listed any. + * + * `[Fail]` does not cover it. Measured on hdc 3.2.0d, a client that cannot + * reach its server writes a bare `Connect server failed` to STDERR, leaves + * stdout empty and exits 0 — so both the prefix and the status miss it, and the + * empty stdout parses to a clean, entirely wrong "no devices are connected". + * Device-level errors carry the prefix; this fails at the connector, below it. + * + * Any parsed row means a listing really was printed, and nothing here + * second-guesses it — a diagnostic alongside real targets is not a refusal. + */ +function hdcDiagnostic(result: { stdout: string; stderr: string }): string | null { + if (parseHdcTargets(result.stdout).length > 0) return null; + return ( + `${result.stdout}\n${result.stderr}` + .split(/\r?\n/) + .map((l) => l.trim()) + .find((l) => l.length > 0 && !l.startsWith("[") && /\s/.test(l) && !l.includes("\t")) ?? null + ); +} + async function listTargets(strict: boolean): Promise { if (!(await resolveHdc())) return []; const result = await runHdc(["list", "targets", "-v"], HDC_LIST_TIMEOUT_MS); - const failure = hdcFailure(result); + const failure = hdcFailure(result) ?? hdcDiagnostic(result); if (failure) { if (strict) throw new Error(failure); return []; diff --git a/packages/tool-server/test/harmony-hdc-targets.test.ts b/packages/tool-server/test/harmony-hdc-targets.test.ts index 1098ec2b65..cbda87d044 100644 --- a/packages/tool-server/test/harmony-hdc-targets.test.ts +++ b/packages/tool-server/test/harmony-hdc-targets.test.ts @@ -2,20 +2,42 @@ import { afterAll, beforeAll, describe, expect, it, vi } from "vitest"; import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; -import { listHarmonyHdcTargets, listHarmonyHdcTargetsStrict } from "../src/utils/harmony-devices"; +import { + listHarmonyHdcTargets, + listHarmonyHdcTargetsStrict, + parseHdcTargets, +} from "../src/utils/harmony-devices"; // A real stub binary reached through the documented `$DEVECO_STUDIO_HOME` -// layout, as `harmony-hdc-timeout.test.ts` does: the behaviour under test is -// `hdc` exiting 0 while reporting a failure, which a mock at the run boundary -// would simply assert rather than reproduce. +// layout, as `harmony-hdc-timeout.test.ts` does: what is under test is `hdc` +// reporting a failure while exiting 0, which a mock at the run boundary would +// assert rather than reproduce — including WHICH STREAM it uses, the detail +// that defeated the first attempt at this guard. `$HDC_STUB_OUT` goes to +// stdout, `$HDC_STUB_ERR` to stderr. const root = mkdtempSync(join(tmpdir(), "argent-deveco-targets-")); const binDir = join(root, "sdk", "default", "openharmony", "toolchains"); mkdirSync(binDir, { recursive: true }); -// Verbatim from `hdc list targets -v` with the daemon down, which is the state -// a `-stop` leaves behind while the connector restarts. -writeFileSync(join(binDir, "hdc"), '#!/usr/bin/env bash\necho "[Fail]Connect server failed"\n', { - mode: 0o755, -}); +writeFileSync( + join(binDir, "hdc"), + '#!/usr/bin/env bash\nprintf %s "$HDC_STUB_OUT"\nprintf %s "$HDC_STUB_ERR" >&2\n', + { mode: 0o755 } +); + +/** Verbatim `hdc list targets -v` with one phone attached. */ +const REAL_ROW = "025DEK236V035771\t\tUSB\tConnected\tlocalhost\n"; +/** + * Verbatim hdc 3.2.0d that cannot reach its server: stdout EMPTY, this on + * stderr, exit 0, and no `[Fail]` prefix — connector-level failures print + * differently from the device-level ones that carry it. + */ +const PROSE_ERR = "Connect server failed\n"; +/** The prefixed form, as `hdc tconn` to a dead endpoint prints it. */ +const PREFIXED = "[Fail]Connect server failed\n"; + +function said(out: string, err = "") { + vi.stubEnv("HDC_STUB_OUT", out); + vi.stubEnv("HDC_STUB_ERR", err); +} beforeAll(() => vi.stubEnv("DEVECO_STUDIO_HOME", root)); afterAll(() => { @@ -23,18 +45,58 @@ afterAll(() => { rmSync(root, { recursive: true, force: true }); }); -describe("hdc target listing when the connector itself fails", () => { - it("reports a refused listing as an empty device table to a polling caller", async () => { - // `hdc` exits 0 whatever happens, so this cannot be told from success by - // status. A caller watching for a change wants exactly this: nothing new. - await expect(listHarmonyHdcTargets()).resolves.toEqual([]); +describe("hdc reporting a failure instead of listing targets", () => { + it("is not parsed as a target called `Connect`", () => { + // Split on whitespace the three words are the shape of a `-v` row, so + // without the delimiter check this is a device that does not exist — + // offered to `list-devices` and counted in a boot's baseline. + expect(parseHdcTargets("Connect server failed\n")).toEqual([]); + expect(parseHdcTargets(`${REAL_ROW}Connect server failed\n`)).toEqual([ + { connectKey: "025DEK236V035771", connection: "USB", state: "Connected" }, + ]); + }); + + it("is an empty device table to a polling caller, on either stream", async () => { + // Exit status cannot tell this from success, so a caller watching for a + // change gets what it wants either way: nothing new. + for (const [out, err] of [ + ["", PROSE_ERR], + [PREFIXED, ""], + ]) { + said(out!, err!); + await expect(listHarmonyHdcTargets()).resolves.toEqual([]); + } + }); + + it("is raised for a caller establishing a baseline, including on stderr", async () => { + // The stderr case is the one that matters: stdout is empty, so the listing + // parses to a clean `[]` and `boot-device` would adopt an emulator that was + // already connected as the instance it just started. + for (const [out, err] of [ + ["", PROSE_ERR], + [PREFIXED, ""], + ]) { + said(out!, err!); + await expect(listHarmonyHdcTargetsStrict()).rejects.toThrow(/Connect server failed/); + } + }); + + it("never displaces a listing that did print rows", async () => { + // The guard keys on prose being space-delimited where a row is tabbed, so a + // real listing must survive it — including one a diagnostic trails. + for (const [out, err] of [ + [REAL_ROW, ""], + [REAL_ROW, PROSE_ERR], + ]) { + said(out!, err!); + await expect(listHarmonyHdcTargetsStrict()).resolves.toEqual([ + { connectKey: "025DEK236V035771", connection: "USB", state: "Connected" }, + ]); + } }); - it("raises the refusal instead, for a caller that is establishing a baseline", async () => { - // An empty baseline silently means "everything that appears next is new", - // so `boot-device` would adopt an emulator that was already connected as - // the instance it just started. Rejecting is what makes that refusable — - // and it has to happen here, since the exit status never carries it. - await expect(listHarmonyHdcTargetsStrict()).rejects.toThrow("[Fail]Connect server failed"); + it("reports a genuinely empty device table as empty, not as a refusal", async () => { + said("[Empty]\n"); + await expect(listHarmonyHdcTargetsStrict()).resolves.toEqual([]); }); }); From 148402f74bec2831bc93480aca16cbcdf1a16baa Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 19:13:46 +0200 Subject: [PATCH 047/110] fix(boot-device): keep the harmony boot inside the budget it was given The waits are built on `uitest` probes that carry their own fixed timeouts and take no budget from the caller, so a poll that probes each arrival in turn ran for that timeout times the number of arrivals - 95s against a 30s bootTimeoutMs with two stale rows reconnecting beside the instance. Both waits now abandon a probe the deadline has overtaken, as the Android boot already clamps its frame wait. A `force` restart that then cannot read hdc's device table also said only that the table could not be read, leaving the caller to assume nothing had happened when the instance had in fact been stopped and not restarted. --- .../src/tools/devices/boot-device.ts | 78 +++++++++++++++++-- .../test/boot-device-harmony.test.ts | 61 +++++++++++++++ 2 files changed, 131 insertions(+), 8 deletions(-) diff --git a/packages/tool-server/src/tools/devices/boot-device.ts b/packages/tool-server/src/tools/devices/boot-device.ts index 19e9345e73..59d578d5ec 100644 --- a/packages/tool-server/src/tools/devices/boot-device.ts +++ b/packages/tool-server/src/tools/devices/boot-device.ts @@ -1651,9 +1651,16 @@ async function connectedHarmonyTargets() { * listing: `hdc` exits 0 for its own failures, so the tolerant one reports a * broken daemon as an empty device table. Retried once, since the daemon * restarting after a `-stop` is both the likely cause and one that clears, - * then refused while there is still nothing spawned to clean up. + * then refused before anything is spawned. + * + * `stoppedInstance` names the instance a `force` restart has already stopped by + * this point, since refusing here is the one path that leaves the host in a + * state the caller did not ask for: they asked for a restart and get an error, + * with the instance down and nothing started in its place. The other side of the + * `force` path says so too, and staying quiet here would have the same failure + * read as "nothing happened" depending only on which call failed. */ -async function connectedHarmonyKeys(): Promise> { +async function connectedHarmonyKeys(stoppedInstance: string | null): Promise> { for (let attempt = 0; ; attempt++) { try { const targets = await listHarmonyHdcTargetsStrict(); @@ -1664,7 +1671,11 @@ async function connectedHarmonyKeys(): Promise> { "Could not read `hdc`'s device table while preparing to start the HarmonyOS instance, " + "so the target it registers under could not have been told apart from those already " + "connected: " + - (err instanceof Error ? err.message : String(err)), + (err instanceof Error ? err.message : String(err)) + + (stoppedInstance + ? `. \`force\` had already stopped "${stoppedInstance}", which is still down — ` + + "re-run boot-device to start it" + : ""), { cause: err } ); } @@ -1818,6 +1829,36 @@ type HarmonyTargetOutcome = { * so closing those needs a lock across every harmony boot on the host, which * would head-of-line block them all on one slow start. */ +/** + * `work`, or null once `deadline` has passed. + * + * Every probe these waits are built on carries its own fixed timeout and takes + * no budget from its caller, so a poll that probes each arrival in turn runs for + * that timeout times the number of arrivals — measured at 95s against a 30s + * `bootTimeoutMs`, with two stale rows reconnecting beside the instance. The + * budget is the caller's word on how long boot-device may take: the Android boot + * above clamps its frame wait against the same deadline, "so the frame-wait + * stage cannot push total elapsed time past bootTimeoutMs". + * + * Only the waiting is abandoned. The probe's own timeout still ends the process + * it spawned, so nothing outlives the call that would not have anyway. + */ +async function withinBudget(work: Promise, deadline: number): Promise { + const remaining = deadline - Date.now(); + if (remaining <= 0) return null; + let timer: ReturnType | undefined; + try { + return await Promise.race([ + work, + new Promise((resolve) => { + timer = setTimeout(() => resolve(null), remaining); + }), + ]); + } finally { + clearTimeout(timer); + } +} + async function waitForHarmonyTarget( before: Set, deadline: number, @@ -1836,11 +1877,16 @@ async function waitForHarmonyTarget( confirmed.push(target.connectKey); continue; } - const display = await harmonyDisplay(target.connectKey).catch(() => null); + const display = await withinBudget( + harmonyDisplay(target.connectKey).catch(() => null), + deadline + ); // A guest that has not composited yet answers `0x0`. The manager side of // this comparison refuses zero as a panel for the same reason, so reading // it as someone else's would have the two sides disagree about the one - // value that joins them. + // value that joins them. A probe the budget cut short lands here too: + // unprobed and unrejected are the same thing to the poll after it, and + // once the budget is gone the rest of this round costs nothing. if (!display || display.width <= 0 || display.height <= 0) { sawUnprobeable = true; continue; @@ -1899,12 +1945,17 @@ async function waitForHarmonyDrivable( checkAlive: () => void ): Promise { const probePath = join(tmpdir(), `argent-harmony-bootprobe-${process.hrtime.bigint()}.json`); + // Non-null only while a probe is in flight, which past the deadline is a probe + // nothing is waiting for any more. + let inFlight: Promise | null = null; try { for (;;) { - const answered = await harmonyDumpLayout(connectKey, probePath).then( + inFlight = harmonyDumpLayout(connectKey, probePath).then( () => true, () => false ); + const answered = await withinBudget(inFlight, deadline); + if (answered !== null) inFlight = null; if (answered) return true; checkAlive(); const remaining = deadline - Date.now(); @@ -1912,7 +1963,14 @@ async function waitForHarmonyDrivable( await new Promise((r) => setTimeout(r, Math.min(HARMONY_TARGET_POLL_MS, remaining))); } } finally { - await rm(probePath, { force: true }).catch(() => {}); + // A probe abandoned at the deadline can still write `probePath` after this + // point, so its removal waits it out — bounded by the probe's own timeout — + // rather than racing it and leaving the dump behind in the temp directory. + // That one is not awaited, since the budget that would pay for it is what + // just ran out; on every other path the dump is gone before this returns. + const remove = () => rm(probePath, { force: true }).catch(() => {}); + if (inFlight) void inFlight.then(remove); + else await remove(); } } @@ -2009,7 +2067,11 @@ async function bootHarmonyImpl(params: { // any assumption about how an emulator's key is spelled. Reaching `Connected` // after this point is what marks the instance just booted — including on the // key it held before, since a stopped instance's row survives as `Offline`. - const before = hdcAvailable ? await connectedHarmonyKeys() : new Set(); + // `alreadyRunning` can only still be set on the `force` path — the other + // return above — so it is exactly "this call has stopped the instance". + const before = hdcAvailable + ? await connectedHarmonyKeys(alreadyRunning ? params.instanceName : null) + : new Set(); // The panel this instance is configured with, which the guest reports back as // its `render resolution` — the only thing that separates the instance // reconnecting from another device doing the same. Read before the start diff --git a/packages/tool-server/test/boot-device-harmony.test.ts b/packages/tool-server/test/boot-device-harmony.test.ts index e4277377cb..8a68c71cad 100644 --- a/packages/tool-server/test/boot-device-harmony.test.ts +++ b/packages/tool-server/test/boot-device-harmony.test.ts @@ -796,6 +796,67 @@ describe("boot-device — HarmonyOS emulator path", () => { expect(spawnMock).not.toHaveBeenCalled(); }); + it("says the instance is down when a `force` restart refuses at the snapshot", async () => { + // The one refusal that leaves the host somewhere the caller never asked to + // be: `-stop` has already run, so "the device table could not be read" + // arrives with the instance down and nothing started in its place. Read as + // "nothing happened" it costs the caller a working emulator they still + // believe is up. + let listed = 0; + listHarmonyInstances.mockImplementation(() => + Promise.resolve([ + { + name: INSTANCE, + deviceType: "Phone", + osVersion: null, + running: listed++ < 1, + display: PANEL, + }, + ]) + ); + listHarmonyHdcTargetsStrict.mockRejectedValue(new Error("Connect server failed")); + vi.useFakeTimers(); + + const pending = boot({ force: true, bootTimeoutMs: 30_000 }); + const settled = expect(pending).rejects.toThrow( + /`force` had already stopped "Phone_1", which is still down/ + ); + await vi.advanceTimersByTimeAsync(31_000); + await settled; + + expect(runHarmonyEmulator).toHaveBeenCalledWith(["-stop", INSTANCE]); + expect(spawnMock).not.toHaveBeenCalled(); + }); + + it("stops probing arrivals once the budget is spent, rather than once per arrival", async () => { + // Each probe carries its own timeout and takes no budget from the wait, so + // probing every arrival in turn multiplies `bootTimeoutMs` by however many + // rows reconnected — 95s against a 30s budget, measured. Three arrivals at + // 20s apiece is that shape: one round already outlasts two budgets. + const slowPanel = { width: 1, height: 1, screenOn: true }; + harmonyDisplay.mockImplementation( + () => new Promise((resolve) => setTimeout(() => resolve(slowPanel), 20_000)) + ); + const peers = [1, 2, 3].map((i) => ({ + connectKey: `127.0.0.1:555${i}`, + connection: "TCP", + state: "Connected", + })); + targets([PHONE], [PHONE, ...peers]); + vi.useFakeTimers(); + + const startedAt = Date.now(); + const pending = boot({ bootTimeoutMs: 30_000 }); + const elapsed = pending.then(() => Date.now() - startedAt); + await vi.advanceTimersByTimeAsync(200_000); + + // The budget is the caller's word on how long boot-device may take, so what + // is asserted is the clock, not the probe count: a boot that answers late is + // the failure, however few calls it made getting there. + expect(await elapsed).toBeLessThanOrEqual(30_000); + expect(((await pending) as { note?: string }).note).toMatch(/registered/); + }); + it("returns an id the interaction tools accept, which the instance id is not", async () => { targets([PHONE], [PHONE, emulatorTarget]); From a4dd81fa68d05ec869310e39e3fd7f5715a8d674 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 19:27:46 +0200 Subject: [PATCH 048/110] fix(harmony): report the screen-off caveat from await-screen-idle too A suspended panel keeps dumping its last composited frame, which is maximally still, so await-screen-idle settles on it instantly and reports a screen that is settled and dead. `await-ui-element` already carries that caveat off the same read; the two wait tools now share one definition of which describe sources owe it. Also names the device and the byte count when a capture will not decode, as the dumpLayout parse beside it does, and files harmony subprocess failures under `hdc` the way every other platform's are filed under theirs - the telemetry sanitiser drops a command it does not carry centrally. Covers the per-device uitest queue directly: serialization, per-device keying, and a rejection not stranding the calls behind it. --- packages/registry/src/errors.ts | 1 + .../skills/argent-device-interact/SKILL.md | 2 +- .../src/tools/await-screen-idle/index.ts | 22 +++- .../src/tools/await-ui-element/index.ts | 15 +-- .../src/tools/describe/contract.ts | 18 +++ .../tool-server/src/utils/harmony-apps.ts | 6 + packages/tool-server/src/utils/harmony-hdc.ts | 5 + .../tool-server/src/utils/harmony-screen.ts | 32 ++++- .../tool-server/src/utils/harmony-uitest.ts | 3 + .../tool-server/test/await-harmony.test.ts | 19 +++ .../test/harmony-hdc-timeout.test.ts | 11 ++ .../tool-server/test/harmony-screen.test.ts | 15 ++- .../test/harmony-uitest-queue.test.ts | 113 ++++++++++++++++++ 13 files changed, 238 insertions(+), 24 deletions(-) create mode 100644 packages/tool-server/test/harmony-uitest-queue.test.ts diff --git a/packages/registry/src/errors.ts b/packages/registry/src/errors.ts index 38c17277c9..8894e692a7 100644 --- a/packages/registry/src/errors.ts +++ b/packages/registry/src/errors.ts @@ -25,6 +25,7 @@ export const FAILURE_COMMANDS = [ "adb", "emulator", "vega", + "hdc", "xcrun_simctl", "xctrace", "native_devtools", diff --git a/packages/skills/skills/argent-device-interact/SKILL.md b/packages/skills/skills/argent-device-interact/SKILL.md index abd325942b..aa7c50a95f 100644 --- a/packages/skills/skills/argent-device-interact/SKILL.md +++ b/packages/skills/skills/argent-device-interact/SKILL.md @@ -227,7 +227,7 @@ Use after launch/navigation and before a raw tap, when an early-painted element { "udid": "", "timeoutMs": 3000, "minStableMs": 250 } ``` -On local iOS, Android, Chromium, and HarmonyOS, the tool waits for a non-empty `describe` tree to stop changing. Continue only when `settled: true`. Pair it with a destination-specific `await-ui-element`; stillness does not identify a screen. +On local iOS, Android, Chromium, and HarmonyOS, the tool waits for a non-empty `describe` tree to stop changing. Continue only when `settled: true`, and read the `note` if one came with it — a `settled: true` carrying one means the tree may not be the live screen (a suspended HarmonyOS panel settles at once on its last frame, and taps land nowhere until `button` wakes it). Pair it with a destination-specific `await-ui-element`; stillness does not identify a screen. Use it only for live diagnosis. Do not record it or put it in `run-sequence`. Flows use `await: { idle: true }`, which also compares pixels. This live tool can return during a presentation-layer animation. diff --git a/packages/tool-server/src/tools/await-screen-idle/index.ts b/packages/tool-server/src/tools/await-screen-idle/index.ts index 404b9f2aa1..b4bdeb6ad1 100644 --- a/packages/tool-server/src/tools/await-screen-idle/index.ts +++ b/packages/tool-server/src/tools/await-screen-idle/index.ts @@ -14,6 +14,7 @@ import { isAndroidTv } from "../../utils/adb"; import { assertSupported } from "../../utils/capability"; import { ensureDeps } from "../../utils/check-deps"; import { pollDescribeTree } from "../../utils/poll-describe-tree"; +import { READ_CAVEAT_SOURCES } from "../describe/contract"; import type { DescribeNode, DescribeTreeData } from "../describe/contract"; import { describeIos, iosRequires } from "../describe/platforms/ios"; import { describeAndroid, androidRequires } from "../describe/platforms/android"; @@ -69,7 +70,11 @@ interface IdleResult { waitedMs: number; /** Number of tree reads taken. */ polls: number; - /** Why a false `settled` is not a screen that stayed busy — e.g. the device went away. */ + /** + * Why a false `settled` is not a screen that stayed busy — e.g. the device + * went away — or, on a true one, why the tree it settled on may not be the + * live screen. + */ note?: string; } @@ -135,7 +140,9 @@ export function createAwaitScreenIdleTool(registry: Registry): ToolDefinition = new Set(["harmony-uitest", "cdp-dom"]); - function timeoutNote( params: Params, lastTree: DescribeNode | null, diff --git a/packages/tool-server/src/tools/describe/contract.ts b/packages/tool-server/src/tools/describe/contract.ts index f82b38b615..4832f5abd3 100644 --- a/packages/tool-server/src/tools/describe/contract.ts +++ b/packages/tool-server/src/tools/describe/contract.ts @@ -89,6 +89,24 @@ export type DescribeSource = | "harmony-uitest" | "tv-focus"; +// Sources whose `hint` questions the tree just read: HarmonyOS keeps dumping the +// last composited frame while the panel is suspended, so a matched element need +// not be on the live screen; Chromium's walker stops at its node budget, so the +// page is only partly there. Every other hint is a standing fact about the target +// — not booted through argent, a tvOS / Android TV device, an app that cannot be +// instrumented — as true before a wait as after, so on a success it reads as a +// verdict on what the wait saw; iOS' ends "You MUST call boot-device with +// force=true now", i.e. restart the simulator and lose the app state. Those stay +// on the timeout note. Listing rather than excluding leaves a source added later +// silent on success until someone judges its hint. +// +// Read by both wait tools, which owe the caveat for the same reason: each +// answers off one describe read and reports what it saw as the current screen. +export const READ_CAVEAT_SOURCES: ReadonlySet = new Set([ + "harmony-uitest", + "cdp-dom", +]); + // Internal shape produced by the per-platform adapters. The `tree` is consumed // by the formatter in `format-tree.ts` and then dropped before the tool replies // — callers see `DescribeResult` below, which surfaces only the rendered text. diff --git a/packages/tool-server/src/utils/harmony-apps.ts b/packages/tool-server/src/utils/harmony-apps.ts index d66a588e32..3784889473 100644 --- a/packages/tool-server/src/utils/harmony-apps.ts +++ b/packages/tool-server/src/utils/harmony-apps.ts @@ -79,6 +79,7 @@ export async function resolveHarmonyEntry( failure_stage: "harmony_resolve_bundle", failure_area: "tool_server", error_kind: "not_found", + failure_command: "hdc", } ); } @@ -100,6 +101,7 @@ export async function resolveHarmonyEntry( failure_stage: "harmony_resolve_bundle", failure_area: "tool_server", error_kind: "subprocess", + failure_command: "hdc", }, { cause: err as Error } ); @@ -128,6 +130,7 @@ export async function resolveHarmonyEntry( failure_stage: "harmony_resolve_bundle", failure_area: "tool_server", error_kind: "not_found", + failure_command: "hdc", } ); } @@ -171,6 +174,7 @@ export async function terminateHarmonyApp(connectKey: string, bundleId: string): failure_stage: "harmony_force_stop", failure_area: "tool_server", error_kind: stdout.includes(AA_NO_SUCH_PACKAGE) ? "not_found" : "subprocess", + failure_command: "hdc", } ); } @@ -235,6 +239,7 @@ export async function openHarmonyUrl(connectKey: string, url: string): Promise { failure_stage: "harmony_hdc_resolve_binary", failure_area: "tool_server", error_kind: "dependency_missing", + failure_command: "hdc", } ); } @@ -131,6 +132,7 @@ export async function runHdc( failure_stage: "harmony_hdc_run", failure_area: "tool_server", error_kind: "subprocess", + failure_command: "hdc", }); } } @@ -197,6 +199,7 @@ export async function runHdcShell( failure_stage: "harmony_hdc_shell", failure_area: "tool_server", error_kind: "not_found", + failure_command: "hdc", } ); } @@ -216,6 +219,7 @@ export async function runHdcShell( failure_stage: "harmony_hdc_shell", failure_area: "tool_server", error_kind: "subprocess", + failure_command: "hdc", } ); } @@ -243,6 +247,7 @@ export async function hdcFileRecv( failure_stage: "harmony_hdc_file_recv", failure_area: "tool_server", error_kind: "subprocess", + failure_command: "hdc", } ); } diff --git a/packages/tool-server/src/utils/harmony-screen.ts b/packages/tool-server/src/utils/harmony-screen.ts index 295c573099..edb3996b59 100644 --- a/packages/tool-server/src/utils/harmony-screen.ts +++ b/packages/tool-server/src/utils/harmony-screen.ts @@ -2,9 +2,29 @@ import { readFile, rename, writeFile, rm } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { PNG } from "pngjs"; +import { FAILURE_CODES, FailureError } from "@argent/registry"; import { harmonyScreenCap } from "./harmony-uitest"; import { scaleDecodedPng } from "./png-scale"; +function decodeCapture(raw: Buffer, connectKey: string): PNG { + try { + return PNG.sync.read(raw); + } catch (err) { + throw new FailureError( + `HarmonyOS device '${connectKey}' returned a screenshot that is not a readable PNG ` + + `(${raw.length} bytes).`, + { + error_code: FAILURE_CODES.HARMONY_FILE_TRANSFER_FAILED, + failure_stage: "harmony_screen_cap", + failure_area: "tool_server", + error_kind: "subprocess", + failure_command: "hdc", + }, + { cause: err as Error } + ); + } +} + /** * Capture the HarmonyOS display and return the path to a host-side PNG. * @@ -21,10 +41,14 @@ export async function captureHarmonyScreenshotPng(opts: { const rawPath = join(tmpdir(), `argent-harmony-raw-${process.hrtime.bigint()}.png`); try { await harmonyScreenCap(opts.connectKey, rawPath); - // The decode is the only proof `uitest` captured anything: `hdc` reports the - // connection's status, not the remote command's, so a failed capture arrives - // as a successful transfer of a truncated file. - const decoded = PNG.sync.read(await readFile(rawPath)); + // The capture itself is checked upstream — `uitest screenCap` exits 1 on a + // failure (measured) and `runUitest` throws on that — so what is left for + // the decode is the pull: a transfer that reported success and delivered a + // truncated file, which nothing before this point can see. `dumpLayout`'s + // parse sits in the same position for the same reason, and names the device + // and the byte count rather than leaving the decoder's own words to explain + // an empty screenshot. + const decoded = decodeCapture(await readFile(rawPath), opts.connectKey); const scaled = scaleDecodedPng(decoded, opts.scale); const outPath = join(tmpdir(), `harmony-screenshot-${process.hrtime.bigint()}.png`); if (scaled === decoded) { diff --git a/packages/tool-server/src/utils/harmony-uitest.ts b/packages/tool-server/src/utils/harmony-uitest.ts index 169f266549..5f0d0a2501 100644 --- a/packages/tool-server/src/utils/harmony-uitest.ts +++ b/packages/tool-server/src/utils/harmony-uitest.ts @@ -99,6 +99,7 @@ async function runUitest( failure_stage: "harmony_uitest", failure_area: "tool_server", error_kind: "subprocess", + failure_command: "hdc", }); } return stdout; @@ -136,6 +137,7 @@ export async function harmonyDisplay(connectKey: string): Promise { expect(result.polls).toBeGreaterThan(1); }); + it("settles instantly on a suspended panel but says the display is off", async () => { + // A frozen frame is maximally still, so this is the one screen that settles + // fastest and means it least. `await-ui-element` carries the same caveat off + // the same read — the two must not disagree about a screen they both just + // described, since the agent's next call after either is a tap. + vi.mocked(harmonyDisplay).mockResolvedValue({ ...DISPLAY, screenOn: false }); + sequenceDumps([dumpWith("Settings")]); + const tool = createAwaitScreenIdleTool(harmonyRegistry()); + + const result = await tool.execute( + {}, + { udid: HARMONY_ID, timeoutMs: 2000, pollIntervalMs: 10, minStableMs: 30 } + ); + + expect(result.settled).toBe(true); + expect(result.note).toMatch(/display is off/i); + expect(result.note).toMatch(/taps land nowhere/i); + }); + it("does not settle while the dump keeps changing (times out)", async () => { sequenceDumps(Array.from({ length: 30 }, (_, i) => dumpWith(`item-${i}`))); const tool = createAwaitScreenIdleTool(harmonyRegistry()); diff --git a/packages/tool-server/test/harmony-hdc-timeout.test.ts b/packages/tool-server/test/harmony-hdc-timeout.test.ts index 0655a2a520..54c02eb5ee 100644 --- a/packages/tool-server/test/harmony-hdc-timeout.test.ts +++ b/packages/tool-server/test/harmony-hdc-timeout.test.ts @@ -2,6 +2,7 @@ import { afterAll, beforeAll, describe, expect, it, vi } from "vitest"; import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; +import { FAILURE_COMMANDS, getFailureSignal } from "@argent/registry"; import { runHdc } from "../src/utils/harmony-hdc"; // Resolve `hdc` through the documented `$DEVECO_STUDIO_HOME` layout rather than @@ -51,5 +52,15 @@ describe("runHdc timeout enforcement", () => { // Far below the stub's own 30s sleep: the assertion is that the deadline is // enforced at all, not its precise latency on a loaded CI box. expect(elapsed).toBeLessThan(6_000); + + // Named binary, as every other platform's subprocess failures are — `adb`, + // `vega`, `xcrun_simctl`. Membership is asserted alongside the value because + // it is what decides whether the value is ever reported: the telemetry + // sanitiser validates `failure_command` against this list and silently drops + // a spelling it does not carry, which would file every `hdc` failure under + // no binary at all. + const signal = getFailureSignal(err); + expect(signal).toMatchObject({ failure_command: "hdc", failure_stage: "harmony_hdc_run" }); + expect(FAILURE_COMMANDS).toContain(signal?.failure_command); }, 20_000); }); diff --git a/packages/tool-server/test/harmony-screen.test.ts b/packages/tool-server/test/harmony-screen.test.ts index 1672116010..df01dff3e8 100644 --- a/packages/tool-server/test/harmony-screen.test.ts +++ b/packages/tool-server/test/harmony-screen.test.ts @@ -147,14 +147,21 @@ describe("captureHarmonyScreenshotPng", () => { await expect(access(hdcFileRecv.mock.calls[0]![2])).rejects.toThrow(); }); - it("removes the host-side raw capture when decoding it fails", async () => { - // The fetch "succeeds" but delivers garbage; the raw file must not be - // left behind in the host tmpdir on the error path. + it("names the device and the size when the capture will not decode", async () => { + // The fetch "succeeds" but delivers garbage — the one failure `uitest`'s own + // exit status cannot show, since the capture it reports on happened before + // the transfer. The raw file must not be left behind in the host tmpdir on + // the error path either. hdcFileRecv.mockImplementation(async (_key, _remote, localPath) => { await writeFile(localPath, Buffer.from("this is not a png")); }); - await expect(captureHarmonyScreenshotPng({ connectKey: "dev", scale: 1 })).rejects.toThrow(); + // `dumpLayout`'s parse failure names both, and for the same reason: pngjs' + // own "Invalid file signature" says nothing about which device answered or + // how much of an image arrived. + await expect(captureHarmonyScreenshotPng({ connectKey: "dev", scale: 1 })).rejects.toThrow( + /device 'dev' returned a screenshot that is not a readable PNG \(17 bytes\)/ + ); // hdcFileRecv's localPath IS the raw intermediate; it must be gone. const rawPath = hdcFileRecv.mock.calls[0]![2]; diff --git a/packages/tool-server/test/harmony-uitest-queue.test.ts b/packages/tool-server/test/harmony-uitest-queue.test.ts new file mode 100644 index 0000000000..27994f83eb --- /dev/null +++ b/packages/tool-server/test/harmony-uitest-queue.test.ts @@ -0,0 +1,113 @@ +import { describe, it, expect, vi, beforeEach } from "vitest"; +import { + hdcFileRecv as realHdcFileRecv, + runHdcShell as realRunHdcShell, +} from "../src/utils/harmony-hdc"; +import { harmonyScreenCap } from "../src/utils/harmony-uitest"; + +// Only the transport is faked, so the queue under test is the real one and the +// commands it serializes are the ones a device would receive. +vi.mock("../src/utils/harmony-hdc", async (importOriginal) => { + const actual = await importOriginal(); + return { ...actual, runHdcShell: vi.fn(), hdcFileRecv: vi.fn() }; +}); + +const runHdcShell = vi.mocked(realRunHdcShell); +const hdcFileRecv = vi.mocked(realHdcFileRecv); + +/** `start`/`end` per `uitest` call, in the order the device would see them. */ +let events: string[] = []; +/** Resolvers for the `uitest` calls currently blocked, in start order. */ +let blocked: (() => void)[] = []; + +/** Let every microtask that can run, run — without releasing any device call. */ +const settle = () => new Promise((r) => setImmediate(r)); + +beforeEach(() => { + events = []; + blocked = []; + hdcFileRecv.mockResolvedValue(undefined); + runHdcShell.mockImplementation(async (key, command) => { + // The cleanup `rm -f` runs outside the queue by design; counting it here + // would report an overlap the queue never promised to prevent. + if (!command.startsWith("uitest ")) return { stdout: "", exitCode: 0 }; + events.push(`start:${key}`); + await new Promise((resolve) => blocked.push(resolve)); + events.push(`end:${key}`); + return { stdout: "", exitCode: 0 }; + }); +}); + +/** Release the longest-blocked `uitest` call and let the queue advance. */ +async function releaseOne(): Promise { + blocked.shift()?.(); + await settle(); +} + +describe("the per-device uitest queue", () => { + it("holds a second call on one device until the first has finished", async () => { + // `uitest` is single-instance on the device: a second one launched while the + // first is running fails with `Another uitest is running`, so two argent + // calls landing together would fail whichever lost the race — a failure the + // caller cannot do anything about and did not cause. + const first = harmonyScreenCap("dev-a", "/tmp/a.png"); + const second = harmonyScreenCap("dev-a", "/tmp/b.png"); + await settle(); + + expect(events).toEqual(["start:dev-a"]); + + await releaseOne(); + expect(events).toEqual(["start:dev-a", "end:dev-a", "start:dev-a"]); + + await releaseOne(); + await Promise.all([first, second]); + }); + + it("does not hold one device's call behind another device's", async () => { + // The queue is keyed per device precisely so a phone and an emulator, or two + // emulators, still run at once. One global lock would serialize an agent's + // whole fleet behind its slowest device. + const a = harmonyScreenCap("dev-a", "/tmp/a.png"); + const b = harmonyScreenCap("dev-b", "/tmp/b.png"); + await settle(); + + expect(events).toEqual(["start:dev-a", "start:dev-b"]); + + await releaseOne(); + await releaseOne(); + await Promise.all([a, b]); + }); + + it("lets the next call through after one that failed", async () => { + // The queue tracks settlement, not the value. Chaining the successor onto + // the rejection itself would strand every later call on that device behind + // one `uitest` failure — with nothing to release it, since the failure has + // already been reported to its own caller. + // Two things this has to get right to exercise the chain at all. The failing + // call must still be IN the queue when the next one joins it — one that + // fails before the successor enqueues finds an empty queue. And the failure + // must be the transport REJECTING: a `uitest` that merely exits non-zero + // resolves the queued work and is thrown on afterwards, outside the queue, + // so it never reaches the chain this test is about. + runHdcShell.mockImplementationOnce(async (key, command) => { + events.push(`start:${key}`); + expect(command).toContain("uitest"); + await new Promise((resolve) => blocked.push(resolve)); + throw new Error("hdc could not reach HarmonyOS device 'dev-a'"); + }); + + const failed = harmonyScreenCap("dev-a", "/tmp/a.png"); + await settle(); + const next = harmonyScreenCap("dev-a", "/tmp/b.png"); + await settle(); + expect(events).toEqual(["start:dev-a"]); + + blocked.shift()?.(); + await expect(failed).rejects.toThrow(/could not reach/); + await settle(); + expect(events).toEqual(["start:dev-a", "start:dev-a"]); + + await releaseOne(); + await expect(next).resolves.toBeUndefined(); + }); +}); From cc2a9e4be35562f04a73293b751b86bbffa071f6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 19:45:47 +0200 Subject: [PATCH 049/110] test(harmony): pin the behaviours a mutation run found nothing held A 93-mutation sweep over the harmony code left 26 mutants alive. The ones that mattered are covered here: the screen-awake guard, which only `button` pinned, so deleting it from tap, swipe or keyboard left the suite green while each reported success for input that landed nowhere; every by-name instance lookup in the boot path, free variables because each fixture held exactly one instance; the spawn `error` listener, whose absence takes the tool-server down with an uncaughtException; the panel comparison, which passed on one axis alone; and the collapse guards, where dropping any one loses a label, a node's siblings, or a tenth of its frame. Also stops the readiness wait spawning a probe the budget cannot pay for - it answered to nobody and held the device's uitest queue while the caller was being told to retry. --- .../src/tools/devices/boot-device.ts | 9 +- .../test/boot-device-harmony.test.ts | 207 +++++++++++++++++- .../test/gesture-swipe-harmony.test.ts | 31 +++ .../test/gesture-tap-harmony.test.ts | 23 ++ .../test/harmony-layout-parser.test.ts | 70 ++++++ .../test/keyboard-error-taxonomy.test.ts | 6 + .../test/keyboard-key-order.test.ts | 22 +- 7 files changed, 364 insertions(+), 4 deletions(-) diff --git a/packages/tool-server/src/tools/devices/boot-device.ts b/packages/tool-server/src/tools/devices/boot-device.ts index 59d578d5ec..89fd3c3f4f 100644 --- a/packages/tool-server/src/tools/devices/boot-device.ts +++ b/packages/tool-server/src/tools/devices/boot-device.ts @@ -1606,7 +1606,7 @@ const HARMONY_UNCONFIRMED_TARGET = * tool retried by hand may well work — so this is a caveat, not a failure. */ const HARMONY_NOT_DRIVABLE = - "The instance registered with `hdc` but was still not answering `uitest` when the boot budget " + + "The instance registered with `hdc` but had not answered `uitest` before the boot budget " + "ran out, so `describe`, `screenshot` and the gesture tools may fail for a little longer. " + "Retry the first interaction, or give the boot longer with `bootTimeoutMs`."; @@ -1950,6 +1950,13 @@ async function waitForHarmonyDrivable( let inFlight: Promise | null = null; try { for (;;) { + // Asked before it is spawned, not only before it is awaited: a target + // confirmed on arrival alone (no panel to check it against) can resolve at + // the deadline itself, and a probe started then is one nothing will ever + // read — holding this device's `uitest` queue for its own timeout, which + // is exactly what the retry the caller is about to be told to make would + // then wait behind. + if (Date.now() >= deadline) return false; inFlight = harmonyDumpLayout(connectKey, probePath).then( () => true, () => false diff --git a/packages/tool-server/test/boot-device-harmony.test.ts b/packages/tool-server/test/boot-device-harmony.test.ts index 8a68c71cad..d25ca09983 100644 --- a/packages/tool-server/test/boot-device-harmony.test.ts +++ b/packages/tool-server/test/boot-device-harmony.test.ts @@ -421,7 +421,7 @@ describe("boot-device — HarmonyOS emulator path", () => { const result = (await pending) as { udid: string; note?: string }; expect(result.udid).toBe(`harmony-${EMULATOR_KEY}`); - expect(result.note).toMatch(/still not answering `uitest`/); + expect(result.note).toMatch(/had not answered `uitest`/); }); it("ignores a second emulator that was connected and driveable before the start", async () => { @@ -511,6 +511,78 @@ describe("boot-device — HarmonyOS emulator path", () => { expect(err.message).not.toMatch(/before "Phone_1" registered/); }); + it("removes the readiness probe's dump when the wait ends by throwing", async () => { + // The other exit from that wait. Cleanup hung on the return paths instead of + // a `finally` leaves a full `uitest dumpLayout` JSON per dead boot, on the + // path where boots are most likely to be retried. + const probes: string[] = []; + harmonyDumpLayout.mockImplementation((_key: unknown, path: string) => { + probes.push(path); + writeFileSync(path, "{}"); + child.die("error: the emulator instance quit unexpectedly (disk image corrupted)"); + return Promise.reject(new Error("DumpLayout failed:Get window nodes failed")); + }); + targets([PHONE], [PHONE, emulatorTarget]); + + await expect(boot({ bootTimeoutMs: 30_000 })).rejects.toThrow(/disk image corrupted/); + + expect(probes).toHaveLength(1); + expect(existsSync(probes[0]!)).toBe(false); + }); + + it("does not start a readiness probe there is no budget left to read", async () => { + // An instance with no configured panel is confirmed on arrival alone, so the + // key can resolve at the deadline itself with nothing left for the readiness + // wait. A probe launched then answers to nobody and holds this device's + // `uitest` queue for its own timeout — which the retry the note asks for + // would queue behind. + listHarmonyInstances.mockResolvedValue([ + { name: INSTANCE, deviceType: "Phone", osVersion: null, running: false, display: null }, + ]); + // Each listing costs a third of the budget, so the arrival lands on the poll + // that exhausts it. + const rounds = [[PHONE], [PHONE], [PHONE, emulatorTarget]]; + let call = 0; + const next = () => + new Promise((resolve) => + setTimeout(() => resolve(rounds[Math.min(call++, rounds.length - 1)]), 10_000) + ); + listHarmonyHdcTargets.mockImplementation(next); + listHarmonyHdcTargetsStrict.mockImplementation(next); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 30_000 }); + await vi.advanceTimersByTimeAsync(60_000); + const result = (await pending) as { udid: string; note?: string }; + + expect(result.udid).toBe(`harmony-${EMULATOR_KEY}`); + expect(harmonyDumpLayout).not.toHaveBeenCalled(); + // Both caveats, and neither claiming the guest was asked anything. + expect(result.note).toMatch(/had not answered `uitest`/); + expect(result.note).toMatch(/no panel on record/); + }); + + it("names the declined target, not the unreadable one, when it saw both", async () => { + // A target that answered someone else's panel is a firmer diagnosis than one + // that answered nothing, so it is the one worth reporting — and the note it + // selects is the only place the difference reaches the caller. + const unprobeable = { connectKey: "127.0.0.1:5557", connection: "TCP", state: "Connected" }; + harmonyDisplay.mockImplementation((key: string) => + key === unprobeable.connectKey + ? Promise.reject(new Error("hidumper: no answer")) + : Promise.resolve({ width: 466, height: 466, screenOn: true }) + ); + targets([PHONE], [PHONE, emulatorTarget, unprobeable]); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 30_000 }); + await vi.advanceTimersByTimeAsync(31_000); + const result = (await pending) as { note?: string }; + + expect(result.note).toMatch(/is not the panel/); + expect(result.note).not.toMatch(/never reported a display/); + }); + it("says the key rests on arrival alone when the instance has no panel to check it against", async () => { // A multi-display profile keys its LCDs differently, so `display` is null on // a perfectly good instance — and the check that separates it from another @@ -608,7 +680,7 @@ describe("boot-device — HarmonyOS emulator path", () => { expect(result.udid).toBe(`harmony-${EMULATOR_KEY}`); expect(result.note).toMatch(/no panel on record for this instance/); - expect(result.note).toMatch(/still not answering `uitest`/); + expect(result.note).toMatch(/had not answered `uitest`/); }); it("names the instance and says why when nothing registers within the budget", async () => { @@ -857,6 +929,137 @@ describe("boot-device — HarmonyOS emulator path", () => { expect(((await pending) as { note?: string }).note).toMatch(/registered/); }); + // Every one of these lookups is `i.name === params.instanceName` against a + // listing that, in every other fixture here, holds exactly one instance — so + // the name match itself is a free variable. A developer machine with a phone + // and a tablet profile is the ordinary case, not an exotic one. + describe("with more than one instance on the host", () => { + /** `other` first, so a lookup that takes `instances[0]` takes the wrong one. */ + const instances = (mine: { running: boolean }, other: { running: boolean }) => [ + { + name: "Tablet_9", + deviceType: "Tablet", + osVersion: null, + running: other.running, + display: { width: 2200, height: 1400 }, + }, + { + name: INSTANCE, + deviceType: "Phone", + osVersion: null, + running: mine.running, + display: PANEL, + }, + ]; + + it("checks the panel of the instance it was asked for, not the first listed", async () => { + listHarmonyInstances.mockResolvedValue(instances({ running: false }, { running: false })); + targets([PHONE], [PHONE, emulatorTarget]); + + const result = (await boot({})) as { udid: string; note?: string }; + + // The arrival answers PANEL. Read against the tablet's profile it is a + // stranger, and the boot hands back an id no interaction tool accepts. + expect(result.udid).toBe(`harmony-${EMULATOR_KEY}`); + expect(result.note).toBeUndefined(); + }); + + it("does not treat another instance being up as this one already running", async () => { + listHarmonyInstances.mockResolvedValue(instances({ running: false }, { running: true })); + targets([PHONE], [PHONE, emulatorTarget]); + + const result = (await boot({})) as { udid: string; note?: string }; + + // Without the name match this returns `booted: true` on the tablet's + // account and never starts anything. + expect(spawnMock).toHaveBeenCalledWith( + expect.any(String), + ["-start", INSTANCE], + expect.any(Object) + ); + expect(result.udid).toBe(`harmony-${EMULATOR_KEY}`); + expect(result.note).toBeUndefined(); + }); + + it("waits out only its own instance's shutdown, not another's", async () => { + // The tablet stays up throughout. A stop-wait that asks "is anything + // running" instead of "is this running" never sees the restart complete + // and spends the whole budget on a peer it was never asked about. + let listed = 0; + listHarmonyInstances.mockImplementation(() => + Promise.resolve(instances({ running: listed++ < 1 }, { running: true })) + ); + targets([PHONE, staleEmulatorTarget], [PHONE, emulatorTarget]); + vi.useFakeTimers(); + + const pending = boot({ force: true, bootTimeoutMs: 60_000 }); + await vi.advanceTimersByTimeAsync(20_000); + const result = (await pending) as { udid: string; note?: string }; + + expect(result.udid).toBe(`harmony-${EMULATOR_KEY}`); + expect(result.note).toBeUndefined(); + }); + }); + + it("refuses a target that matches the panel on one axis only", async () => { + // The comparison is min/min and max/max so a landscape guest still matches + // its portrait config. Comparing one axis would let any device sharing a + // width — 1320x2400 against this 1320x2856 — be adopted as the instance + // argent started, after which every tap lands on someone else's screen. + harmonyDisplay.mockResolvedValue({ width: PANEL.width, height: 2400, screenOn: true }); + targets([PHONE], [PHONE, emulatorTarget]); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 30_000 }); + await vi.advanceTimersByTimeAsync(31_000); + const result = (await pending) as { udid: string; note?: string }; + + expect(result.udid).toBe(`harmony-emulator-${INSTANCE}`); + expect(result.note).toMatch(/is not the panel/); + }); + + it("keeps waiting through a stop-wait listing that could not be read", async () => { + // "An unreadable listing answers nothing, so it keeps waiting rather than + // assuming the instance is gone" — read as `stopped` instead, one transient + // `Emulator -list` failure sends `-start` at a live instance, which answers + // only that it is already running. + let listed = 0; + listHarmonyInstances.mockImplementation(() => { + listed += 1; + if (listed === 1) + return Promise.resolve([ + { name: INSTANCE, deviceType: "Phone", osVersion: null, running: true, display: PANEL }, + ]); + return Promise.reject(new Error("Emulator -list failed")); + }); + vi.useFakeTimers(); + + const pending = boot({ force: true, bootTimeoutMs: 30_000 }); + const settled = expect(pending).rejects.toThrow(/still running when the 30s budget ran out/); + await vi.advanceTimersByTimeAsync(31_000); + await settled; + + expect(spawnMock).not.toHaveBeenCalled(); + }); + + it("reports a manager that could not be spawned instead of taking the server down", async () => { + // `spawn` fails asynchronously for ENOENT/EACCES. An unhandled `error` event + // is an uncaughtException, which this process routes to crashShutdown — so + // one unlaunchable emulator would stop every other device's session too. + // The Android arm has its own file pinning exactly this. + spawnMock.mockImplementation(() => { + queueMicrotask(() => child.emit("error", new Error("spawn Emulator ENOENT"))); + return child; + }); + targets([PHONE]); + vi.useFakeTimers(); + + const pending = boot({ bootTimeoutMs: 30_000 }); + const settled = expect(pending).rejects.toThrow(/could not be spawned: spawn Emulator ENOENT/); + await vi.advanceTimersByTimeAsync(31_000); + await settled; + }); + it("returns an id the interaction tools accept, which the instance id is not", async () => { targets([PHONE], [PHONE, emulatorTarget]); diff --git a/packages/tool-server/test/gesture-swipe-harmony.test.ts b/packages/tool-server/test/gesture-swipe-harmony.test.ts index 6a9f527093..bd4b3048ff 100644 --- a/packages/tool-server/test/gesture-swipe-harmony.test.ts +++ b/packages/tool-server/test/gesture-swipe-harmony.test.ts @@ -1,4 +1,5 @@ import { beforeEach, describe, expect, it, vi } from "vitest"; +import { getFailureSignal } from "@argent/registry"; // Only the hdc transport is faked. Everything between the tool and the wire - // the render-service parse, the normalized-to-pixel conversion, the duration-to- @@ -130,6 +131,36 @@ describe("gesture-swipe on HarmonyOS", () => { expect(sendCommand).not.toHaveBeenCalled(); }); + it("refuses to swipe while the display is suspended, injecting nothing", async () => { + // `uitest uiInput` answers `No Error` and exits 0 against a suspended panel + // (measured), so without the guard the swipe resolves `{swiped: true}` for a + // gesture that landed nowhere — and a scroll-until-found loop runs to its + // limit against a screen that never moved. Driven through the real + // `powerStatus` parse rather than a stubbed struct, since that string is the + // only thing standing between the two outcomes. + runHdcShell.mockImplementation(async (_connectKey, command) => + command.startsWith("hidumper") + ? { + stdout: `render resolution=${DISPLAY_WIDTH}x${DISPLAY_HEIGHT}\npowerStatus=POWER_STATUS_SUSPEND`, + exitCode: 0, + } + : { stdout: "No Error", exitCode: 0 } + ); + + const err = await gestureSwipeTool.execute(services, base).then( + () => { + throw new Error("expected the swipe to reject, but it resolved"); + }, + (e: unknown) => e + ); + + expect(getFailureSignal(err)?.failure_stage).toBe("harmony_screen_off"); + expect((err as Error).message).toMatch(/Wake it with `button` \(power\)/); + expect(runHdcShell.mock.calls.filter(([, c]) => c.startsWith("uitest uiInput"))).toHaveLength( + 0 + ); + }); + it("does not declare the simulator-server service for a HarmonyOS target", () => { // Resolving the blueprint would spawn a backend this path never uses and // block on its ready-wait before the hdc call could start. diff --git a/packages/tool-server/test/gesture-tap-harmony.test.ts b/packages/tool-server/test/gesture-tap-harmony.test.ts index 3f74e75492..45505b9bd8 100644 --- a/packages/tool-server/test/gesture-tap-harmony.test.ts +++ b/packages/tool-server/test/gesture-tap-harmony.test.ts @@ -1,4 +1,5 @@ import { beforeEach, describe, expect, it, vi } from "vitest"; +import { getFailureSignal } from "@argent/registry"; // Only the two transport calls are stubbed; `toDevicePoint` stays real so the // asserted pixel values are the ones a device would actually be handed. @@ -102,6 +103,28 @@ describe("gesture-tap on HarmonyOS", () => { expect(harmonyDisplay).toHaveBeenCalledWith(CONNECT_KEY); expect(vi.mocked(harmonyTouch).mock.calls[0][0]).toBe(CONNECT_KEY); }); + + it("refuses to tap while the display is suspended, injecting nothing", async () => { + // `uitest uiInput click` answers `No Error` and exits 0 against a suspended + // panel (measured), so without this guard the tap resolves `{tapped: true}` + // for input that landed nowhere — and the agent goes on to assert against a + // screen it believes it just touched. `button` pins the same refusal for + // its own presses; the two must not drift apart. + vi.mocked(harmonyDisplay).mockResolvedValue({ ...DISPLAY, screenOn: false }); + + const err = await gestureTapTool + .execute(noServices, { udid: HARMONY_UDID, x: 0.5, y: 0.5 }) + .then( + () => { + throw new Error("expected the tap to reject, but it resolved"); + }, + (e: unknown) => e + ); + + expect(getFailureSignal(err)?.failure_stage).toBe("harmony_screen_off"); + expect((err as Error).message).toMatch(/Wake it with `button` \(power\)/); + expect(harmonyTouch).not.toHaveBeenCalled(); + }); }); describe("gesture-tap service declaration", () => { diff --git a/packages/tool-server/test/harmony-layout-parser.test.ts b/packages/tool-server/test/harmony-layout-parser.test.ts index 5d84269478..e6a33b7605 100644 --- a/packages/tool-server/test/harmony-layout-parser.test.ts +++ b/packages/tool-server/test/harmony-layout-parser.test.ts @@ -219,6 +219,76 @@ describe("parseHarmonyLayout", () => { expect(target.children).toHaveLength(0); }); + it("reads a non-checkable Toggle as the button it behaves like", () => { + // ArkUI's `Toggle` covers switches AND push-style buttons; only the + // checkable ones are switches. Reporting both as `Switch` puts a plain + // button beyond `await-ui-element {role:"Button"}`, and tells the agent a + // control has an on/off state it does not have. + const B = "[0,0][200,100]"; + const asSwitch = wrap( + { type: "Toggle", checkable: "true", clickable: "true", id: "wifi", bounds: B }, + { type: "Text", text: "Wi-Fi", bounds: B } + ); + const asButton = wrap( + { type: "Toggle", checkable: "false", clickable: "true", id: "play", bounds: B }, + { type: "Text", text: "Play", bounds: B } + ); + + expect(asSwitch.role).toBe("Switch"); + expect(asButton.role).toBe("Button"); + }); + + it("keeps a wrapper that carries the only label in the pair", () => { + // ArkUI labels the outer component and leaves the child a bare graphic — + // `Button().accessibilityText("Submit")` over a full-bleed `Image`. + // Collapsing onto the child drops the label entirely, so `describe` shows + // an unnamed image and `await-ui-element {text:"Submit"}` can never match. + const B = "[0,0][200,100]"; + const target = wrap( + { type: "Button", clickable: "true", description: "Submit", bounds: B }, + { type: "Image", bounds: B } + ); + + expect(target.label).toBe("Submit"); + expect(target.role).toBe("Button"); + }); + + it("does not collapse a multi-child wrapper onto its first child", () => { + // A full-bleed background as child[0] is idiomatic ArkUI. Collapsing on it + // returns the background alone and every sibling — here both price and + // caption — disappears from the tree the agent reads. + const B = "[0,0][400,200]"; + const tree = parseHarmonyLayout( + root([ + node({ type: "root", bundleName: "com.app", bounds: "[0,0][1216,2688]" }, [ + node({ type: "Stack", clickable: "true", bounds: B }, [ + node({ type: "Image", bounds: B }), + node({ type: "Text", text: "Buy now", bounds: "[10,10][200,60]" }), + node({ type: "Text", text: "$4.99", bounds: "[10,70][200,120]" }), + ]), + ]), + ]), + SCREEN + ).tree; + + expect(findAll(tree, { text: "Buy now" })).toHaveLength(1); + expect(findAll(tree, { text: "$4.99" })).toHaveLength(1); + }); + + it("does not collapse onto a child that merely shares an origin", () => { + // Same x/y, a tenth of the area: an icon pinned to the top-left of its + // row. Collapsing moves the reported frame — and with it the tap centre + // the description tells the agent to compute — from (200,100) to (20,20). + const target = wrap( + { type: "Stack", clickable: "true", bounds: "[0,0][400,200]" }, + { type: "Image", bounds: "[0,0][40,40]" } + ); + + expect(target.role).toBe("Stack"); + expect(target.children).toHaveLength(1); + expect(target.children[0].role).toBe("Image"); + }); + it("keeps a wrapper that carries the identifier an agent selects on", () => { const B = "[0,0][200,100]"; const target = wrap( diff --git a/packages/tool-server/test/keyboard-error-taxonomy.test.ts b/packages/tool-server/test/keyboard-error-taxonomy.test.ts index 1b0c031e18..102a0f2ef5 100644 --- a/packages/tool-server/test/keyboard-error-taxonomy.test.ts +++ b/packages/tool-server/test/keyboard-error-taxonomy.test.ts @@ -198,6 +198,12 @@ describe("keyboard backends — input rejection is a 400 with a uniform telemetr FAILURE_CODES.KEYBOARD_CHARACTER_UNSUPPORTED ); expect(harmonyTypeText).not.toHaveBeenCalled(); + // The newline shares its failure code with every other un-typeable + // character, so the code alone cannot tell whether its own branch still + // exists — and that branch is there only for the recovery it names. + await expect( + harmonyImpl.handler({}, { udid: harmonyDevice.id, text: "a\nb" }, harmonyDevice) + ).rejects.toThrow(/Submit with `key: "enter"` after typing instead/); // Decided without asking the device: see the unsupported-key case below. expect(harmonyDisplay).not.toHaveBeenCalled(); }); diff --git a/packages/tool-server/test/keyboard-key-order.test.ts b/packages/tool-server/test/keyboard-key-order.test.ts index db98080a9e..cfd701fb55 100644 --- a/packages/tool-server/test/keyboard-key-order.test.ts +++ b/packages/tool-server/test/keyboard-key-order.test.ts @@ -24,7 +24,7 @@ vi.mock("../src/utils/harmony-uitest", async (importOriginal) => ({ })); import { injectVegaNamedKey, injectVegaText } from "../src/utils/vega-input"; -import { harmonyKeyEvent, harmonyTypeText } from "../src/utils/harmony-uitest"; +import { harmonyDisplay, harmonyKeyEvent, harmonyTypeText } from "../src/utils/harmony-uitest"; const IOS_SIM: DeviceInfo = { id: "TEST-UDID", platform: "ios", kind: "simulator" }; const CHROMIUM: DeviceInfo = { id: "chromium-cdp-9222", platform: "chromium", kind: "app" }; @@ -179,6 +179,26 @@ describe("keyboard text+key ordering", () => { expect(harmonyKeyEvent).not.toHaveBeenCalled(); }); + it("harmony: refuses to type while the display is suspended, injecting nothing", async () => { + // `uitest uiInput text` answers `No Error` and exits 0 against a suspended + // panel, so without the guard the call resolves `{ typed: "hi", keys: 2 }` + // for characters that reached no field — the same refusal `gesture-tap`, + // `gesture-swipe` and `button` make off this display read. + vi.mocked(harmonyTypeText).mockClear(); + vi.mocked(harmonyKeyEvent).mockClear(); + vi.mocked(harmonyDisplay).mockResolvedValueOnce({ + width: 1216, + height: 2688, + screenOn: false, + }); + + await expect( + harmonyImpl.handler({}, { udid: HARMONY.id, text: "hi", key: "enter" }, HARMONY) + ).rejects.toThrow(/display is off/); + expect(harmonyTypeText).not.toHaveBeenCalled(); + expect(harmonyKeyEvent).not.toHaveBeenCalled(); + }); + it("harmony: case-folds the named key, like every other backend", async () => { vi.mocked(harmonyKeyEvent).mockClear(); From cd6cf1bc656ad888724c6bdf20d13b7da3c34f80 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 20:07:41 +0200 Subject: [PATCH 050/110] fix(harmony): classify hdc's unprefixed failures and the boot's own refusals MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `hdc` exits 0 for everything and only some of its diagnostics carry the `[Fail]` prefix: a client that cannot reach its server prints bare prose to stderr and leaves stdout empty. `hdcFileRecv` read that as a completed transfer, so a screenshot or layout dump reported success over a path with nothing at it; it now matches the `FileTransfer finish` line positively, and `runHdcShell` quotes the same prose instead of blaming a device it never reached. Every HarmonyOS refusal in boot-device and the emulator wrapper threw a bare `Error`, which buckets as REGISTRY_TOOL_EXECUTION_FAILED — the Android and Vega arms of the same file all carry a granular code. They now do too, under new HARMONY_/BOOT_HARMONY_ codes and a `deveco_emulator` failure command. Windows DevEco installs ship `hdc.exe` / `Emulator.exe`, and these candidates are literal paths handed to access(), so `$DEVECO_STUDIO_HOME` — the only way to point at a non-macOS install — could never resolve one. Also corrects claims the code no longer supports: the layout parser's header (scroll-clipping HarmonyOS would drop rows that are still rendered and tappable, measured on a scrolled Settings list), `force`'s "only way to learn the connect key" against the same file's `list-devices` advice, the unconditional China restriction, open-url's caveat naming a cause that throws instead, describe's header offering `gesture-pinch` to a platform with no pinch backend, and await-screen-idle's note prose covering only HarmonyOS though it fires for Chromium too. --- packages/registry/src/errors.ts | 1 + packages/registry/src/failure-codes.ts | 7 + .../skills/argent-device-interact/SKILL.md | 2 +- .../skills/argent-test-ui-flow/SKILL.md | 2 +- .../src/tools/await-screen-idle/index.ts | 16 ++- .../src/tools/describe/format-tree.ts | 7 +- .../platforms/harmony/layout-parser.ts | 31 +++-- .../src/tools/devices/boot-device.ts | 129 ++++++++++++------ .../src/tools/keyboard/platforms/harmony.ts | 3 + .../src/tools/open-url/platforms/harmony.ts | 7 +- packages/tool-server/src/utils/harmony-cli.ts | 39 +++++- .../tool-server/src/utils/harmony-devices.ts | 19 +-- packages/tool-server/src/utils/harmony-hdc.ts | 60 +++++++- .../test/boot-device-harmony.test.ts | 42 +++++- .../test/harmony-binary-resolve.test.ts | 27 ++++ .../tool-server/test/harmony-describe.test.ts | 14 ++ .../test/harmony-hdc-classification.test.ts | 96 +++++++++++++ .../test/keyboard-error-taxonomy.test.ts | 19 +++ 18 files changed, 428 insertions(+), 93 deletions(-) create mode 100644 packages/tool-server/test/harmony-hdc-classification.test.ts diff --git a/packages/registry/src/errors.ts b/packages/registry/src/errors.ts index 8894e692a7..ba4caa5e1b 100644 --- a/packages/registry/src/errors.ts +++ b/packages/registry/src/errors.ts @@ -26,6 +26,7 @@ export const FAILURE_COMMANDS = [ "emulator", "vega", "hdc", + "deveco_emulator", "xcrun_simctl", "xctrace", "native_devtools", diff --git a/packages/registry/src/failure-codes.ts b/packages/registry/src/failure-codes.ts index 86c6feb813..4f868edea5 100644 --- a/packages/registry/src/failure-codes.ts +++ b/packages/registry/src/failure-codes.ts @@ -62,6 +62,8 @@ export const FAILURE_CODES = { VEGA_TEXT_INVALID: "VEGA_TEXT_INVALID", HARMONY_HDC_NOT_FOUND: "HARMONY_HDC_NOT_FOUND", + HARMONY_EMULATOR_NOT_FOUND: "HARMONY_EMULATOR_NOT_FOUND", + HARMONY_EMULATOR_COMMAND_FAILED: "HARMONY_EMULATOR_COMMAND_FAILED", HARMONY_HDC_COMMAND_FAILED: "HARMONY_HDC_COMMAND_FAILED", HARMONY_DEVICE_UNREACHABLE: "HARMONY_DEVICE_UNREACHABLE", HARMONY_SHELL_NO_STATUS: "HARMONY_SHELL_NO_STATUS", @@ -285,6 +287,11 @@ export const FAILURE_CODES = { BOOT_ANDROID_FIRST_FRAME_TIMEOUT: "BOOT_ANDROID_FIRST_FRAME_TIMEOUT", BOOT_ANDROID_PACKAGE_MANAGER_UNAVAILABLE: "BOOT_ANDROID_PACKAGE_MANAGER_UNAVAILABLE", BOOT_ANDROID_COLD_BOOT_FAILED: "BOOT_ANDROID_COLD_BOOT_FAILED", + BOOT_HARMONY_TARGET_LIST_FAILED: "BOOT_HARMONY_TARGET_LIST_FAILED", + BOOT_HARMONY_STOP_FAILED: "BOOT_HARMONY_STOP_FAILED", + BOOT_HARMONY_STOP_TIMEOUT: "BOOT_HARMONY_STOP_TIMEOUT", + BOOT_HARMONY_MANAGER_EXITED: "BOOT_HARMONY_MANAGER_EXITED", + BOOT_HARMONY_START_FAILED: "BOOT_HARMONY_START_FAILED", } as const; export type FailureCode = (typeof FAILURE_CODES)[keyof typeof FAILURE_CODES]; diff --git a/packages/skills/skills/argent-device-interact/SKILL.md b/packages/skills/skills/argent-device-interact/SKILL.md index aa7c50a95f..8e0699e8a1 100644 --- a/packages/skills/skills/argent-device-interact/SKILL.md +++ b/packages/skills/skills/argent-device-interact/SKILL.md @@ -227,7 +227,7 @@ Use after launch/navigation and before a raw tap, when an early-painted element { "udid": "", "timeoutMs": 3000, "minStableMs": 250 } ``` -On local iOS, Android, Chromium, and HarmonyOS, the tool waits for a non-empty `describe` tree to stop changing. Continue only when `settled: true`, and read the `note` if one came with it — a `settled: true` carrying one means the tree may not be the live screen (a suspended HarmonyOS panel settles at once on its last frame, and taps land nowhere until `button` wakes it). Pair it with a destination-specific `await-ui-element`; stillness does not identify a screen. +On local iOS, Android, Chromium, and HarmonyOS, the tool waits for a non-empty `describe` tree to stop changing. Continue only when `settled: true`, and read the `note` if one came with it — a `settled: true` carrying one means the tree it settled on is not the whole live screen (a suspended HarmonyOS panel settles at once on its last frame, and taps land nowhere until `button` wakes it; a Chromium page past the walker's node budget settles on a partial tree). Pair it with a destination-specific `await-ui-element`; stillness does not identify a screen. Use it only for live diagnosis. Do not record it or put it in `run-sequence`. Flows use `await: { idle: true }`, which also compares pixels. This live tool can return during a presentation-layer animation. diff --git a/packages/skills/skills/argent-test-ui-flow/SKILL.md b/packages/skills/skills/argent-test-ui-flow/SKILL.md index 0a9468d6cd..b4b896475c 100644 --- a/packages/skills/skills/argent-test-ui-flow/SKILL.md +++ b/packages/skills/skills/argent-test-ui-flow/SKILL.md @@ -1,6 +1,6 @@ --- name: argent-test-ui-flow -description: Autonomously test an app UI (iOS or Android) by running interact-screenshot-verify loops using argent MCP tools. Use when testing UI flows, verifying login works, testing navigation, running end-to-end UI test scenarios, manual QA steps, visible UI changes, or visual behavior. +description: Autonomously test an app UI (iOS, Android, or HarmonyOS) by running interact-screenshot-verify loops using argent MCP tools. Use when testing UI flows, verifying login works, testing navigation, running end-to-end UI test scenarios, manual QA steps, visible UI changes, or visual behavior. --- ## Platform-agnostic diff --git a/packages/tool-server/src/tools/await-screen-idle/index.ts b/packages/tool-server/src/tools/await-screen-idle/index.ts index b4bdeb6ad1..91f5a3368c 100644 --- a/packages/tool-server/src/tools/await-screen-idle/index.ts +++ b/packages/tool-server/src/tools/await-screen-idle/index.ts @@ -141,8 +141,9 @@ Polls the same accessibility / DOM tree as \`describe\` every pollIntervalMs (de has content and that content holds identical for minStableMs (default ${DEFAULT_MIN_STABLE_MS}ms), or timeoutMs (default ${DEFAULT_TIMEOUT_MS}ms) is reached. Returns { settled, waitedMs, polls }, and a note when settled=false because the tree read itself failed — that note is what separates a device that went away from a screen that never went still. A note on -settled=true instead means the tree may not be the live screen: a suspended HarmonyOS panel settles instantly on its last -composited frame, and taps land nowhere until it is woken. +settled=true instead means the tree it settled on is not the whole live screen: a suspended HarmonyOS panel settles +instantly on its last composited frame and taps land nowhere until it is woken, and a Chromium page past the walker's +node budget settles on a partial tree. Read the note before acting on what settled. Use after a launch/navigation to wait for the UI to render before screenshotting or tapping.`, searchHint: "wait until screen settles idle stable stops changing animation transition rendered ready before screenshot", @@ -209,12 +210,13 @@ Use after a launch/navigation to wait for the UI to render before screenshotting ...(poll.result !== true && poll.lastError ? { note: `last tree fetch failed: ${poll.lastError}` } : {}), - // A settled screen still owes the caveat when the tree it settled on may - // not be the live one — a suspended HarmonyOS panel keeps dumping its + // A settled screen still owes the caveat when the tree it settled on is + // not the whole live one — a suspended HarmonyOS panel keeps dumping its // last composited frame, which is maximally still and so settles at - // once. `await-ui-element` reports the same hint on the same sources for - // the same reason; without it the two wait tools disagree about a screen - // they both just read. + // once, and a truncated Chromium tree stops changing because the rest of + // the page was never walked. `await-ui-element` reports the same hint on + // the same sources for the same reason; without it the two wait tools + // disagree about a screen they both just read. ...(poll.result === true && poll.lastData?.hint && READ_CAVEAT_SOURCES.has(poll.lastData.source) diff --git a/packages/tool-server/src/tools/describe/format-tree.ts b/packages/tool-server/src/tools/describe/format-tree.ts index 7d91663e32..6dbb6eaa1e 100644 --- a/packages/tool-server/src/tools/describe/format-tree.ts +++ b/packages/tool-server/src/tools/describe/format-tree.ts @@ -205,7 +205,12 @@ export function formatDescribeTree(root: DescribeNode, opts: FormatDescribeOptio ); } else { header.push( - "Pass them straight to gesture-tap / gesture-swipe / gesture-pinch, which expect this same space." + // HarmonyOS drives touch through `uitest uiInput`, which has no multi-touch + // verb — `gesture-pinch` has no harmony backend and refuses the device, so + // naming it here would send the agent at a tool it cannot reach. + opts.source === "harmony-uitest" + ? "Pass them straight to gesture-tap / gesture-swipe, which expect this same space." + : "Pass them straight to gesture-tap / gesture-swipe / gesture-pinch, which expect this same space." ); header.push( "To tap an element, use its centre: tap_x = frame.x + frame.width / 2, tap_y = frame.y + frame.height / 2." diff --git a/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts b/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts index 285bad931b..9c11571172 100644 --- a/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts +++ b/packages/tool-server/src/tools/describe/platforms/harmony/layout-parser.ts @@ -9,9 +9,16 @@ import { clipBoundsToScreen } from "../android/uiautomator-parser"; * uiautomator dump — `[x1,y1][x2,y2]` bounds, `clickable`/`checkable` flags, a * `type` that plays the part of Android's `class` — so this mirrors the Android * trimmer's structure. It is markedly shorter for one measured reason: `uitest` - * already merges the window stack and drops invisible nodes before writing, so - * the scroll-clip pruning the Android side needs has nothing to prune (176 - * nodes for a full Settings screen, none of them `visible: false`). + * already merges the window stack and emits what is on screen, so a full + * Settings list is 177 nodes and its one `visible: false` node is a zero-area + * `ListItem`. + * + * What is deliberately NOT mirrored is the Android side's scroll clip. A `List` + * rect is not a clip rect here: scroll that same Settings screen and its + * "Log in to HUAWEI ID" row sits at y=164-308 while the `List` it belongs to + * starts at y=332, still rendered (screenshot-checked) under the collapsing + * title. Pruning by the scrolling ancestor's rect would drop six such nodes on + * one scroll, every one of them visible and tappable. * * What it must handle that Android's does not: * @@ -94,15 +101,6 @@ function isScrollable(attrs: Record): boolean { return SCROLL_TYPES.has(attrs.type ?? "") || isTrue(attrs, "scrollable"); } -/** - * The screen-reader-meaningful label. - * - * `description` (ArkUI's `accessibilityText`) wins over `text` for the same - * reason `content-desc` does on Android: when both are set they describe - * different things, and the contract carries the typed content separately as - * `value`. `hint` is the placeholder of an empty field, which is the only label - * such a field has. - */ /** * One attribute as trimmed text. * @@ -115,6 +113,15 @@ function attrText(value: unknown): string { return typeof value === "string" ? value.trim() : ""; } +/** + * The screen-reader-meaningful label. + * + * `description` (ArkUI's `accessibilityText`) wins over `text` for the same + * reason `content-desc` does on Android: when both are set they describe + * different things, and the contract carries the typed content separately as + * `value`. `hint` is the placeholder of an empty field, which is the only label + * such a field has. + */ export function harmonyLabel(attrs: Record): string { const description = attrText(attrs.description); if (description) return description; diff --git a/packages/tool-server/src/tools/devices/boot-device.ts b/packages/tool-server/src/tools/devices/boot-device.ts index 89fd3c3f4f..1909730de9 100644 --- a/packages/tool-server/src/tools/devices/boot-device.ts +++ b/packages/tool-server/src/tools/devices/boot-device.ts @@ -116,7 +116,7 @@ const zodSchema = z.object({ .boolean() .optional() .describe( - "Shut down and re-boot the device even if already running. On HarmonyOS this is also the only way to learn the `hdc` connect key of an instance that is already up, since the emulator manager never reports one: without it an already-running instance is left alone and the payload names the instance. Ignored for Electron, which always spawns a fresh process." + "Shut down and re-boot the device even if already running. On HarmonyOS it is also what ties an already-running instance to the `hdc` connect key it registered under, since the emulator manager never reports one: without it the instance is left alone and the payload names the instance rather than a key (`list-devices` lists the keys, but not which instance each belongs to). Ignored for Electron, which always spawns a fresh process." ), headless: z .boolean() @@ -1667,7 +1667,7 @@ async function connectedHarmonyKeys(stoppedInstance: string | null): Promise t.state === "Connected").map((t) => t.connectKey)); } catch (err) { if (attempt > 0) { - throw new Error( + throw new FailureError( "Could not read `hdc`'s device table while preparing to start the HarmonyOS instance, " + "so the target it registers under could not have been told apart from those already " + "connected: " + @@ -1676,7 +1676,14 @@ async function connectedHarmonyKeys(stoppedInstance: string | null): Promise setTimeout(r, HARMONY_TARGET_POLL_MS)); @@ -1730,7 +1737,15 @@ async function startHarmonyEmulator( instanceName: string ): Promise<{ exited: () => { reason: string; output: string } | null }> { const bin = await resolveHarmonyEmulator(); - if (!bin) throw new Error(EMULATOR_NOT_FOUND); + if (!bin) { + throw new FailureError(EMULATOR_NOT_FOUND, { + error_code: FAILURE_CODES.HARMONY_EMULATOR_NOT_FOUND, + failure_stage: "harmony_emulator_resolve_binary", + failure_area: "tool_server", + error_kind: "dependency_missing", + failure_command: "deveco_emulator", + }); + } const logPath = join(tmpdir(), `argent-harmony-${instanceName.replace(/[^\w.-]/g, "_")}.log`); const logFd = openSync(logPath, "w"); const child = spawn(bin, ["-start", instanceName], { @@ -1783,6 +1798,36 @@ type HarmonyTargetOutcome = { reason?: "ambiguous" | "mismatched" | "unprobeable" | "none"; }; +/** + * `work`, or null once `deadline` has passed. + * + * Every probe these waits are built on carries its own fixed timeout and takes + * no budget from its caller, so a poll that probes each arrival in turn runs for + * that timeout times the number of arrivals — measured at 95s against a 30s + * `bootTimeoutMs`, with two stale rows reconnecting beside the instance. The + * budget is the caller's word on how long boot-device may take: the Android boot + * above clamps its frame wait against the same deadline, "so the frame-wait + * stage cannot push total elapsed time past bootTimeoutMs". + * + * Only the waiting is abandoned. The probe's own timeout still ends the process + * it spawned, so nothing outlives the call that would not have anyway. + */ +async function withinBudget(work: Promise, deadline: number): Promise { + const remaining = deadline - Date.now(); + if (remaining <= 0) return null; + let timer: ReturnType | undefined; + try { + return await Promise.race([ + work, + new Promise((resolve) => { + timer = setTimeout(() => resolve(null), remaining); + }), + ]); + } finally { + clearTimeout(timer); + } +} + /** * The connect key of the target that appears after a start, or null if none * does before `deadline`. @@ -1829,36 +1874,6 @@ type HarmonyTargetOutcome = { * so closing those needs a lock across every harmony boot on the host, which * would head-of-line block them all on one slow start. */ -/** - * `work`, or null once `deadline` has passed. - * - * Every probe these waits are built on carries its own fixed timeout and takes - * no budget from its caller, so a poll that probes each arrival in turn runs for - * that timeout times the number of arrivals — measured at 95s against a 30s - * `bootTimeoutMs`, with two stale rows reconnecting beside the instance. The - * budget is the caller's word on how long boot-device may take: the Android boot - * above clamps its frame wait against the same deadline, "so the frame-wait - * stage cannot push total elapsed time past bootTimeoutMs". - * - * Only the waiting is abandoned. The probe's own timeout still ends the process - * it spawned, so nothing outlives the call that would not have anyway. - */ -async function withinBudget(work: Promise, deadline: number): Promise { - const remaining = deadline - Date.now(); - if (remaining <= 0) return null; - let timer: ReturnType | undefined; - try { - return await Promise.race([ - work, - new Promise((resolve) => { - timer = setTimeout(() => resolve(null), remaining); - }), - ]); - } finally { - clearTimeout(timer); - } -} - async function waitForHarmonyTarget( before: Set, deadline: number, @@ -2045,19 +2060,33 @@ async function bootHarmonyImpl(params: { const stopped = await runHarmonyEmulator(["-stop", params.instanceName]); const stopDiagnostic = emulatorFailure(stopped); if (stopDiagnostic) { - throw new Error( - `Failed to stop HarmonyOS emulator "${params.instanceName}" before restarting it: ${stopDiagnostic}` + throw new FailureError( + `Failed to stop HarmonyOS emulator "${params.instanceName}" before restarting it: ${stopDiagnostic}`, + { + error_code: FAILURE_CODES.BOOT_HARMONY_STOP_FAILED, + failure_stage: "boot_harmony_stop", + failure_area: "tool_server", + error_kind: "subprocess", + failure_command: "deveco_emulator", + } ); } // The whole remaining budget rather than a slice of it: an instance still up // leaves nothing worth spending the rest on, since starting it can only // report that it is already running. if (!(await waitForHarmonyInstanceStopped(params.instanceName, bootDeadline))) { - throw new Error( + throw new FailureError( `HarmonyOS emulator "${params.instanceName}" was still running when the ` + `${Math.round(params.bootTimeoutMs / 1_000)}s budget ran out after \`-stop\`, so ` + "starting it now would only report that it is already running. Re-run with a larger " + - "bootTimeoutMs, or stop the instance from DevEco Studio." + "bootTimeoutMs, or stop the instance from DevEco Studio.", + { + error_code: FAILURE_CODES.BOOT_HARMONY_STOP_TIMEOUT, + failure_stage: "boot_harmony_stop_wait", + failure_area: "tool_server", + error_kind: "timeout", + failure_command: "deveco_emulator", + } ); } } @@ -2105,9 +2134,16 @@ async function bootHarmonyImpl(params: { const when = resolvedKey ? `while "${params.instanceName}" (\`${resolvedKey}\`) was still coming up` : `before "${params.instanceName}" registered with \`hdc\``; - throw new Error( + throw new FailureError( `The HarmonyOS emulator manager exited (${exit.reason}) ${when}` + - `, so nothing is running to drive. It printed: ${exit.output.trim() || "(nothing)"}` + `, so nothing is running to drive. It printed: ${exit.output.trim() || "(nothing)"}`, + { + error_code: FAILURE_CODES.BOOT_HARMONY_MANAGER_EXITED, + failure_stage: "boot_harmony_manager_exit", + failure_area: "tool_server", + error_kind: "subprocess", + failure_command: "deveco_emulator", + } ); } const imageMissing = @@ -2119,10 +2155,17 @@ async function bootHarmonyImpl(params: { : instances?.length === 0 ? ` ${HARMONY_NO_INSTANCES}` : ""; - throw new Error( + throw new FailureError( cause ? `Failed to start HarmonyOS emulator "${params.instanceName}".${cause} The manager reported: ${diagnostic}` - : `Failed to start HarmonyOS emulator "${params.instanceName}": ${diagnostic}` + : `Failed to start HarmonyOS emulator "${params.instanceName}": ${diagnostic}`, + { + error_code: FAILURE_CODES.BOOT_HARMONY_START_FAILED, + failure_stage: "boot_harmony_start", + failure_area: "tool_server", + error_kind: "subprocess", + failure_command: "deveco_emulator", + } ); } }; @@ -2229,7 +2272,7 @@ Pick the platform by which argument you pass: 'udid' for an iOS simulator from l Use at the start of a session once you have picked a target. Returns a tagged payload: { platform: 'ios', udid, booted } or { platform: 'android', serial, avdName, booted } or { platform: 'vega', serial, vvdImage, booted } or { platform: 'harmony', udid, instanceName, booted, note? } or { platform: 'chromium', id, port, pid, booted } (an Electron app boots as a Chromium/CDP device). Android boots take 2–10 minutes depending on machine and cold/warm state; the tool transparently hot-boots from the AVD's default_boot snapshot when usable and falls back to cold boot otherwise. Vega starts the single SDK-managed VVD via the vega CLI (~10s) and returns once it reports running. If an Android/Electron boot stage fails, the tool terminates the device it spawned so the next retry starts clean. -HarmonyOS hands the instance to DevEco Studio's Emulator manager, then waits for it to register with 'hdc' and returns its connect key as 'udid' — the id the interaction tools drive. The manager itself reports no readiness signal and never names a port, so an instance that is already running is left alone unless you pass 'force: true'; when the key could not be resolved the payload names the instance instead, and a 'note' is present whenever the boot left something unproven — no key and why, or a key that is not yet answering or could not be checked against the instance. Huawei serves the emulator images only within mainland China, so on a host outside it no instance can exist to start.`, +HarmonyOS hands the instance to DevEco Studio's Emulator manager, then waits for it to register with 'hdc' and returns its connect key as 'udid' — the id the interaction tools drive. The manager itself reports no readiness signal and never names a port, so an instance that is already running is left alone unless you pass 'force: true'; when the key could not be resolved the payload names the instance instead, and a 'note' is present whenever the boot left something unproven — no key and why, or a key that is not yet answering or could not be checked against the instance. Huawei serves the emulator images only within mainland China, so a host outside it usually has no instance to start; one that already has an image is started like any other.`, alwaysLoad: true, searchHint: "boot start launch simulator emulator avd device session ios android vega vvd firetv harmony harmonyos deveco cold hot", diff --git a/packages/tool-server/src/tools/keyboard/platforms/harmony.ts b/packages/tool-server/src/tools/keyboard/platforms/harmony.ts index cf7a22401e..9b11ade9d5 100644 --- a/packages/tool-server/src/tools/keyboard/platforms/harmony.ts +++ b/packages/tool-server/src/tools/keyboard/platforms/harmony.ts @@ -26,6 +26,7 @@ function assertTypeableHarmonyText(text: string): void { { error_code: FAILURE_CODES.KEYBOARD_CHARACTER_UNSUPPORTED, failure_stage: "harmony_keyboard_validate_text", + error_kind: "unsupported", } ); } @@ -39,6 +40,7 @@ function assertTypeableHarmonyText(text: string): void { { error_code: FAILURE_CODES.KEYBOARD_CHARACTER_UNSUPPORTED, failure_stage: "harmony_keyboard_validate_text", + error_kind: "unsupported", } ); } @@ -85,6 +87,7 @@ function resolveHarmonyKeycode(key: string): number { { error_code: FAILURE_CODES.KEYBOARD_KEY_UNSUPPORTED, failure_stage: "harmony_keyboard_resolve_key", + error_kind: "unsupported", } ); } diff --git a/packages/tool-server/src/tools/open-url/platforms/harmony.ts b/packages/tool-server/src/tools/open-url/platforms/harmony.ts index ee20e61f4a..fcc7a9417f 100644 --- a/packages/tool-server/src/tools/open-url/platforms/harmony.ts +++ b/packages/tool-server/src/tools/open-url/platforms/harmony.ts @@ -25,9 +25,10 @@ import { httpDeepLinkNote } from "../deep-link-note"; */ const HARMONY_OPEN_CAVEAT = "On HarmonyOS specifically, `aa start -U` reports success whenever the system accepts the URI, " + - "so `opened: true` does not mean an app opened it: a web URL with no browser registered stays " + - "on the current screen, and a scheme the system hands to its app selector (measured with `tel:` " + - 'and `mailto:`) leaves a modal "No options to open with" dialog covering the screen. Confirm ' + + "so `opened: true` does not mean an app opened it: a web URL can leave the foreground app on " + + "screen unchanged, and a scheme the system hands to its app selector (measured with `tel:` and " + + '`mailto:`) leaves a chooser covering the screen — one listing handlers, or a modal "No options ' + + 'to open with" when none claims it. Confirm ' + "with describe or screenshot, and dismiss any chooser, before treating the link as followed."; export const harmonyImpl: PlatformImpl = { diff --git a/packages/tool-server/src/utils/harmony-cli.ts b/packages/tool-server/src/utils/harmony-cli.ts index ac74a341e2..cb62afb2da 100644 --- a/packages/tool-server/src/utils/harmony-cli.ts +++ b/packages/tool-server/src/utils/harmony-cli.ts @@ -2,6 +2,7 @@ import { execFile } from "node:child_process"; import { promisify } from "node:util"; import { access, constants as fsConstants } from "node:fs/promises"; import { join } from "node:path"; +import { FAILURE_CODES, FailureError } from "@argent/registry"; import { formatSubprocessFailure } from "./subprocess-error"; const execFileAsync = promisify(execFile); @@ -24,8 +25,9 @@ const execFileAsync = promisify(execFile); * So a failure is usually exit 0, but not always — `-start` is the outlier. Since * the code is unreliable in both directions, `runHarmonyEmulator` deliberately * does NOT reject on a non-zero exit: it returns the child's output either way - * and leaves the verdict to `emulatorFailure`, which reads stdout. That keeps one - * classification path instead of two that disagree. + * and leaves the verdict to `emulatorFailure`, which reads what was printed on + * either stream. That keeps one classification path instead of two that + * disagree. * * `hdc`, the HarmonyOS device connector, lives in `harmony-hdc.ts`: this * manager knows about instances, `hdc` knows about targets, and the two are @@ -87,12 +89,21 @@ async function isExecutable(p: string): Promise { */ export async function resolveDevecoBinary(relative: string): Promise { for (const root of devecoRoots()) { - const candidate = join(root, relative); + const candidate = join(root, relative) + BIN_EXT; if (await isExecutable(candidate)) return candidate; } return null; } +// A Windows DevEco install ships `hdc.exe` and `Emulator.exe`. These candidates +// are literal paths handed to `access()`, not PATH lookups, so nothing resolves +// the extension for them the way `where` does for `commandOnPath` — without it +// every root misses and `$DEVECO_STUDIO_HOME`, the only way to point at a +// non-macOS install, can never resolve. Same reasoning and same shape as +// `android-binary.ts`' `BIN_EXT`. Empty on POSIX, where the binaries are +// extensionless. +const BIN_EXT = process.platform === "win32" ? ".exe" : ""; + // Mirrors android-binary.ts / vega-cli.ts: memoize briefly so a burst of tool // calls pays one lookup, but a *negative* result expires — a user who installs // DevEco Studio mid-session recovers without restarting the tool-server. @@ -137,7 +148,15 @@ export async function runHarmonyEmulator( timeoutMs = DEFAULT_TIMEOUT_MS ): Promise { const bin = await resolveHarmonyEmulator(); - if (!bin) throw new Error(EMULATOR_NOT_FOUND); + if (!bin) { + throw new FailureError(EMULATOR_NOT_FOUND, { + error_code: FAILURE_CODES.HARMONY_EMULATOR_NOT_FOUND, + failure_stage: "harmony_emulator_resolve_binary", + failure_area: "tool_server", + error_kind: "dependency_missing", + failure_command: "deveco_emulator", + }); + } try { const { stdout, stderr } = await execFileAsync(bin, args, { timeout: timeoutMs, @@ -157,7 +176,17 @@ export async function runHarmonyEmulator( } // Spawn failure or timeout SIGKILL: no diagnostic to classify, so surface it // the way every other subprocess wrapper here does. - throw new Error(formatSubprocessFailure("Emulator", args, err), { cause: err }); + throw new FailureError( + formatSubprocessFailure("Emulator", args, err), + { + error_code: FAILURE_CODES.HARMONY_EMULATOR_COMMAND_FAILED, + failure_stage: "harmony_emulator_run", + failure_area: "tool_server", + error_kind: "subprocess", + failure_command: "deveco_emulator", + }, + { cause: err instanceof Error ? err : new Error(String(err)) } + ); } } diff --git a/packages/tool-server/src/utils/harmony-devices.ts b/packages/tool-server/src/utils/harmony-devices.ts index 4bdaa4061a..f5275e6a1d 100644 --- a/packages/tool-server/src/utils/harmony-devices.ts +++ b/packages/tool-server/src/utils/harmony-devices.ts @@ -4,7 +4,7 @@ import { resolveHarmonyEmulator, runHarmonyEmulator, } from "./harmony-cli"; -import { HDC_EMPTY_SENTINEL, hdcFailure, resolveHdc, runHdc } from "./harmony-hdc"; +import { HDC_EMPTY_SENTINEL, hdcFailure, hdcProse, resolveHdc, runHdc } from "./harmony-hdc"; /** * Discovery for the HarmonyOS platform, from the two sources that know about a @@ -222,23 +222,14 @@ export async function listHarmonyHdcTargetsStrict(): Promise /** * What `hdc` said instead of listing targets, or null if it listed any. * - * `[Fail]` does not cover it. Measured on hdc 3.2.0d, a client that cannot - * reach its server writes a bare `Connect server failed` to STDERR, leaves - * stdout empty and exits 0 — so both the prefix and the status miss it, and the - * empty stdout parses to a clean, entirely wrong "no devices are connected". - * Device-level errors carry the prefix; this fails at the connector, below it. - * - * Any parsed row means a listing really was printed, and nothing here + * A parsed row means a listing really was printed, and nothing here * second-guesses it — a diagnostic alongside real targets is not a refusal. + * That is the only part specific to listing; what counts as a diagnostic at all + * is {@link hdcProse}, which every hdc caller needs. */ function hdcDiagnostic(result: { stdout: string; stderr: string }): string | null { if (parseHdcTargets(result.stdout).length > 0) return null; - return ( - `${result.stdout}\n${result.stderr}` - .split(/\r?\n/) - .map((l) => l.trim()) - .find((l) => l.length > 0 && !l.startsWith("[") && /\s/.test(l) && !l.includes("\t")) ?? null - ); + return hdcProse(result); } async function listTargets(strict: boolean): Promise { diff --git a/packages/tool-server/src/utils/harmony-hdc.ts b/packages/tool-server/src/utils/harmony-hdc.ts index 643a325e02..80e406e7b5 100644 --- a/packages/tool-server/src/utils/harmony-hdc.ts +++ b/packages/tool-server/src/utils/harmony-hdc.ts @@ -22,8 +22,12 @@ const execFileAsync = promisify(execFile); * * Two consequences drive this module: * - * 1. Transport failures are classified from the `[Fail]` prefix, never the exit + * 1. Transport failures are classified from what was printed, never the exit * code, so `runHdc` returns output rather than rejecting on a non-zero exit. + * Device-level errors carry a `[Fail]` prefix; a client that cannot reach its + * own server does not, so both `hdcFailure` and `hdcProse` are needed to + * cover the two, and a command with a positive success line is read off that + * instead of off the absence of either. * 2. The remote command's own exit status never reaches the host — `hdc shell` * reports the status of the *connection*, not of what ran. `runHdcShell` * therefore appends an `echo` of `$?` and parses it back off stdout, which is @@ -149,6 +153,30 @@ export function hdcFailure(result: HdcRunResult): string | null { return line ? line.trim() : null; } +/** + * A diagnostic `hdc` printed without the `[Fail]` prefix, or null. + * + * The prefix does not cover everything. Measured on hdc 3.2.0d, a client that + * cannot reach its server writes a bare `Connect server failed` to STDERR, + * leaves stdout empty and exits 0 — so the prefix, the streams and the status + * all miss it, and every caller reading only {@link hdcFailure} takes the empty + * stdout at face value. Device-level errors carry the prefix; this one fails at + * the connector, below it. + * + * Prose is told from output by its delimiters: `hdc`'s own tabular output is + * tab-separated, and a bare connect key is one word. Neither is a line that + * holds a space and no tab. Callers whose command has a positive success signal + * should check that first — this only says what was printed instead. + */ +export function hdcProse(result: HdcRunResult): string | null { + return ( + `${result.stdout}\n${result.stderr}` + .split(/\r?\n/) + .map((l) => l.trim()) + .find((l) => l.length > 0 && !l.startsWith("[") && /\s/.test(l) && !l.includes("\t")) ?? null + ); +} + /** * Escape a string for interpolation into the single remote command line that * `hdc shell` hands to the device's `/bin/sh`. @@ -210,10 +238,16 @@ export async function runHdcShell( if (idx === -1) { // The device dropped the trailing echo — the command was killed on-device, or // the transport truncated. Either way the status is unknown, and reporting a - // fabricated 0 here would turn a dead command into a silent success. + // fabricated 0 here would turn a dead command into a silent success. An + // unprefixed connector diagnostic lands here too, since it leaves stdout + // empty; it is quoted rather than guessed at, because the two causes below + // are both wrong for it. + const prose = hdcProse(result); throw new FailureError( - `HarmonyOS device '${connectKey}' returned no exit status for \`${command}\`. ` + - `The command was terminated on the device or the hdc connection dropped mid-call.`, + prose + ? `hdc could not run \`${command}\` on HarmonyOS device '${connectKey}': ${prose}` + : `HarmonyOS device '${connectKey}' returned no exit status for \`${command}\`. ` + + `The command was terminated on the device or the hdc connection dropped mid-call.`, { error_code: FAILURE_CODES.HARMONY_SHELL_NO_STATUS, failure_stage: "harmony_hdc_shell", @@ -230,6 +264,18 @@ export async function runHdcShell( }; } +/** + * What a completed transfer prints, measured on hdc 3.2.0d: + * + * FileTransfer finish, Size:6, File count = 1, time:10ms rate:0.60kB/s + * + * Matched positively, because the absence of `[Fail]` does not mean the file + * arrived: a client that cannot reach its server prints bare prose to stderr, + * leaves stdout empty and exits 0 (see {@link hdcProse}), which read as success + * hands the caller a path with nothing at it. + */ +const HDC_TRANSFER_OK = "FileTransfer finish"; + /** Copy a file off the device. Throws with hdc's own diagnostic if it did not arrive. */ export async function hdcFileRecv( connectKey: string, @@ -238,7 +284,11 @@ export async function hdcFileRecv( timeoutMs = DEFAULT_TIMEOUT_MS ): Promise { const result = await runHdc(["-t", connectKey, "file", "recv", remotePath, localPath], timeoutMs); - const failure = hdcFailure(result); + const failure = result.stdout.includes(HDC_TRANSFER_OK) + ? null + : (hdcFailure(result) ?? + hdcProse(result) ?? + "hdc reported neither a transfer nor a diagnostic"); if (failure) { throw new FailureError( `Failed to copy '${remotePath}' off HarmonyOS device '${connectKey}': ${failure}`, diff --git a/packages/tool-server/test/boot-device-harmony.test.ts b/packages/tool-server/test/boot-device-harmony.test.ts index d25ca09983..8e4cbe4c69 100644 --- a/packages/tool-server/test/boot-device-harmony.test.ts +++ b/packages/tool-server/test/boot-device-harmony.test.ts @@ -3,7 +3,7 @@ import { existsSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { describe, it, expect, vi, beforeEach, afterEach } from "vitest"; -import type { Registry } from "@argent/registry"; +import { FAILURE_CODES, getFailureSignal, type Registry } from "@argent/registry"; const runHarmonyEmulator = vi.fn(); const resolveHarmonyEmulator = vi.fn(); @@ -850,6 +850,46 @@ describe("boot-device — HarmonyOS emulator path", () => { expect(spawnMock).not.toHaveBeenCalled(); }); + it("classifies its refusals the way every other platform in this tool does", async () => { + // Android and Vega throw `FailureError` with a granular code from this same + // file; a bare `Error` buckets as REGISTRY_TOOL_EXECUTION_FAILED, which puts + // "the manager is not installed" and "this instance would not stop" in one + // undifferentiated row — and drops the binary that failed with them. + listHarmonyInstances.mockResolvedValue([ + { name: INSTANCE, deviceType: "Phone", osVersion: null, running: true }, + ]); + runHarmonyEmulator.mockResolvedValue({ + stdout: `"${INSTANCE}" failed, emulator is not exists`, + stderr: "", + }); + const stopFailure = await boot({ force: true }).then( + () => null, + (e: unknown) => e + ); + expect(getFailureSignal(stopFailure)).toMatchObject({ + error_code: FAILURE_CODES.BOOT_HARMONY_STOP_FAILED, + failure_command: "deveco_emulator", + }); + + vi.clearAllMocks(); + ensureDep.mockResolvedValue(undefined); + resolveHdc.mockResolvedValue("/Applications/DevEco-Studio.app/.../hdc"); + listHarmonyInstances.mockResolvedValue([ + { name: INSTANCE, deviceType: "Phone", osVersion: null, running: false }, + ]); + listHarmonyHdcTargets.mockResolvedValue([]); + listHarmonyHdcTargetsStrict.mockResolvedValue([]); + resolveHarmonyEmulator.mockResolvedValue(null); + const missingBinary = await boot({}).then( + () => null, + (e: unknown) => e + ); + expect(getFailureSignal(missingBinary)).toMatchObject({ + error_code: FAILURE_CODES.HARMONY_EMULATOR_NOT_FOUND, + error_kind: "dependency_missing", + }); + }); + it("does not start an instance the stop never brought down", async () => { // One measured `-stop` had still not taken effect three minutes later, so // the instance being down is never assumable — and one still up when the diff --git a/packages/tool-server/test/harmony-binary-resolve.test.ts b/packages/tool-server/test/harmony-binary-resolve.test.ts index fa24c850e4..ccd6fb6c37 100644 --- a/packages/tool-server/test/harmony-binary-resolve.test.ts +++ b/packages/tool-server/test/harmony-binary-resolve.test.ts @@ -112,6 +112,33 @@ describe("DevEco Studio binary resolution", () => { expect(await resolveHdc()).toBe(hdc); }); + describe("on Windows, where the install ships .exe binaries", () => { + // `$DEVECO_STUDIO_HOME` is the ONLY way to point at a non-macOS install, so + // a candidate built without the extension misses every root and HarmonyOS + // is unreachable on the platform its own not-found hint sends users to. + beforeEach(() => setPlatform("win32")); + + it("finds hdc.exe", async () => { + const root = join(tmpRoot, "DevEco Studio"); + const hdc = `${join(root, HDC_RELATIVE)}.exe`; + await writeExecutable(hdc); + process.env.DEVECO_STUDIO_HOME = root; + + const { resolveHdc } = await loadResolvers(); + expect(await resolveHdc()).toBe(hdc); + }); + + it("finds Emulator.exe", async () => { + const root = join(tmpRoot, "DevEco Studio"); + const emulator = `${join(root, EMULATOR_RELATIVE)}.exe`; + await writeExecutable(emulator); + process.env.DEVECO_STUDIO_HOME = root; + + const { resolveHarmonyEmulator } = await loadResolvers(); + expect(await resolveHarmonyEmulator()).toBe(emulator); + }); + }); + describe("$DEVECO_STUDIO_HOME pointing at a root with no DevEco install", () => { // A set-but-wrong variable must not be able to hide a working default // install: it is tried first, then the default, exactly as `androidRoots()` diff --git a/packages/tool-server/test/harmony-describe.test.ts b/packages/tool-server/test/harmony-describe.test.ts index 9f694552a1..bd1d791a53 100644 --- a/packages/tool-server/test/harmony-describe.test.ts +++ b/packages/tool-server/test/harmony-describe.test.ts @@ -86,4 +86,18 @@ describe("describeHarmony", () => { expect(out).toContain("Mode: nested"); expect(out).toContain("04:39"); }); + + it("does not offer the agent a gesture this platform has no backend for", async () => { + // `gesture-pinch` declares no `harmony` capability, so it refuses the device + // — a header naming it costs the agent a round trip into a 400 with nothing + // to fall back to. The two it does name must stay named. + vi.mocked(harmonyDisplay).mockResolvedValue({ width: 1216, height: 2688, screenOn: true }); + vi.mocked(harmonyDumpLayout).mockResolvedValue(lockScreenDump()); + + const { tree, source } = await describeHarmony(CONNECT_KEY); + const out = formatDescribeTree(tree, { source }); + + expect(out).not.toContain("gesture-pinch"); + expect(out).toContain("gesture-tap / gesture-swipe"); + }); }); diff --git a/packages/tool-server/test/harmony-hdc-classification.test.ts b/packages/tool-server/test/harmony-hdc-classification.test.ts new file mode 100644 index 0000000000..fab8a10c07 --- /dev/null +++ b/packages/tool-server/test/harmony-hdc-classification.test.ts @@ -0,0 +1,96 @@ +import { afterAll, beforeAll, describe, expect, it, vi } from "vitest"; +import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { FAILURE_CODES, getFailureSignal } from "@argent/registry"; +import { hdcFileRecv, runHdcShell } from "../src/utils/harmony-hdc"; + +// Every case below is a real `hdc` outcome that exits 0 with no `[Fail]` line, +// so nothing but what was printed separates them. Measured on hdc 3.2.0d: +// a completed transfer prints `FileTransfer finish, …`, and a client that +// cannot reach its own server writes bare prose to STDERR with stdout empty. +// The stub reads the case from a file rather than argv so one binary can stand +// in for all of them. +const root = mkdtempSync(join(tmpdir(), "argent-deveco-")); +const binDir = join(root, "sdk", "default", "openharmony", "toolchains"); +const modeFile = join(root, "mode"); +mkdirSync(binDir, { recursive: true }); +writeFileSync( + join(binDir, "hdc"), + `#!/usr/bin/env bash +case "$(cat ${modeFile})" in + connect-fail) echo "Connect server failed" >&2 ;; + transfer-ok) echo "FileTransfer finish, Size:6, File count = 1, time:10ms rate:0.60kB/s" ;; + silent) ;; +esac +exit 0 +`, + { mode: 0o755 } +); +const hdcBehaves = (mode: string): void => writeFileSync(modeFile, mode); + +beforeAll(() => vi.stubEnv("DEVECO_STUDIO_HOME", root)); +afterAll(() => { + vi.unstubAllEnvs(); + rmSync(root, { recursive: true, force: true }); +}); + +async function rejection(p: Promise): Promise { + return p.then( + () => { + throw new Error("expected a rejection, got a resolution"); + }, + (e: unknown) => e as Error + ); +} + +describe("hdcFileRecv success is read off the transfer line", () => { + const remote = "/data/local/tmp/argent-screen.png"; + + it("refuses a transfer the connector never made", async () => { + // The whole failure is on stderr with no prefix, so classifying on `[Fail]` + // alone resolves — handing `harmony-screen` a path with nothing at it, which + // it then reports as a decode failure of a file that was never copied. + hdcBehaves("connect-fail"); + const err = await rejection(hdcFileRecv("127.0.0.1:5555", remote, join(root, "out.png"))); + expect(err.message).toContain("Connect server failed"); + expect(getFailureSignal(err)?.error_code).toBe(FAILURE_CODES.HARMONY_FILE_TRANSFER_FAILED); + }); + + it("refuses a silent run rather than reading silence as a copy", async () => { + hdcBehaves("silent"); + const err = await rejection(hdcFileRecv("127.0.0.1:5555", remote, join(root, "out.png"))); + expect(err.message).toMatch(/neither a transfer nor a diagnostic/); + expect(getFailureSignal(err)?.error_code).toBe(FAILURE_CODES.HARMONY_FILE_TRANSFER_FAILED); + }); + + it("accepts a completed transfer", async () => { + // The other half of the positive match: every HarmonyOS screenshot and + // layout dump goes through here, so a marker that never matches breaks all + // of them just as silently as one that always does. + hdcBehaves("transfer-ok"); + await expect( + hdcFileRecv("127.0.0.1:5555", remote, join(root, "out.png")) + ).resolves.toBeUndefined(); + }); +}); + +describe("runHdcShell without an exit status", () => { + it("quotes the connector's own diagnostic instead of blaming the device", async () => { + // Same stderr prose, one layer up: it leaves stdout empty, so the rc + // sentinel is missing and this lands in the no-status branch. Sending the + // caller to look at a device that was never reached is the wrong repair. + hdcBehaves("connect-fail"); + const err = await rejection(runHdcShell("127.0.0.1:5555", "uitest dumpLayout")); + expect(err.message).toContain("Connect server failed"); + expect(err.message).not.toMatch(/terminated on the device/); + expect(getFailureSignal(err)?.error_code).toBe(FAILURE_CODES.HARMONY_SHELL_NO_STATUS); + }); + + it("still names a truncated command when hdc said nothing at all", async () => { + hdcBehaves("silent"); + const err = await rejection(runHdcShell("127.0.0.1:5555", "uitest dumpLayout")); + expect(err.message).toMatch(/returned no exit status/); + expect(getFailureSignal(err)?.error_code).toBe(FAILURE_CODES.HARMONY_SHELL_NO_STATUS); + }); +}); diff --git a/packages/tool-server/test/keyboard-error-taxonomy.test.ts b/packages/tool-server/test/keyboard-error-taxonomy.test.ts index 102a0f2ef5..4dec7bcf89 100644 --- a/packages/tool-server/test/keyboard-error-taxonomy.test.ts +++ b/packages/tool-server/test/keyboard-error-taxonomy.test.ts @@ -238,4 +238,23 @@ describe("keyboard backends — input rejection is a 400 with a uniform telemetr expect(harmonyDisplay).not.toHaveBeenCalled(); expect(harmonyKeyEvent).not.toHaveBeenCalled(); }); + + it("harmony: every unsupported-input rejection carries the shared `unsupported` kind", async () => { + const harmonyDevice: DeviceInfo = { id: "harmony-KEY", platform: "harmony", kind: "device" }; + // The android twin tags all three of these `unsupported` + // (utils/android-input.ts), and telemetry groups by kind before code — so a + // harmony rejection without one lands in the untyped bucket and the same + // failure reads as two different things depending on the platform. + for (const params of [{ text: "a\nb" }, { text: "a\x07b" }, { key: "tab" }]) { + const err = await harmonyImpl + .handler({}, { udid: harmonyDevice.id, ...params }, harmonyDevice) + .then( + () => { + throw new Error("expected the call to reject, but it resolved"); + }, + (e: unknown) => e + ); + expect(getFailureSignal(err)?.error_kind).toBe("unsupported"); + } + }); }); From cc8791654efe7087204165184826b748809fdca7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 20:11:36 +0200 Subject: [PATCH 051/110] fix(harmony): tell a cut-off command from a connector that never reached one `runHdcShell` reaches its no-status branch two ways, and they need opposite repairs. Any output at all means the command ran and its trailing status echo was lost; an empty stdout is a call that produced nothing, which is the shape an unprefixed `hdc` diagnostic has. Quoting the prose unconditionally read a truncated `uitest`'s own success line back at the caller as hdc's verdict. --- packages/tool-server/src/utils/harmony-hdc.ts | 14 +++++++++----- .../test/harmony-hdc-classification.test.ts | 12 ++++++++++++ 2 files changed, 21 insertions(+), 5 deletions(-) diff --git a/packages/tool-server/src/utils/harmony-hdc.ts b/packages/tool-server/src/utils/harmony-hdc.ts index 80e406e7b5..5c2524b4c5 100644 --- a/packages/tool-server/src/utils/harmony-hdc.ts +++ b/packages/tool-server/src/utils/harmony-hdc.ts @@ -238,11 +238,15 @@ export async function runHdcShell( if (idx === -1) { // The device dropped the trailing echo — the command was killed on-device, or // the transport truncated. Either way the status is unknown, and reporting a - // fabricated 0 here would turn a dead command into a silent success. An - // unprefixed connector diagnostic lands here too, since it leaves stdout - // empty; it is quoted rather than guessed at, because the two causes below - // are both wrong for it. - const prose = hdcProse(result); + // fabricated 0 here would turn a dead command into a silent success. + // + // A connector that never reached the device lands here too, and needs the + // opposite repair, so the two are told apart by what came back: any output + // at all means the command ran and was cut off, while an empty stdout is a + // call that produced nothing — the shape an unprefixed `hdc` diagnostic has + // (see {@link hdcProse}). Reading a truncated command's own output as that + // diagnostic would quote the app back at the caller as hdc's verdict. + const prose = result.stdout.trim().length === 0 ? hdcProse(result) : null; throw new FailureError( prose ? `hdc could not run \`${command}\` on HarmonyOS device '${connectKey}': ${prose}` diff --git a/packages/tool-server/test/harmony-hdc-classification.test.ts b/packages/tool-server/test/harmony-hdc-classification.test.ts index fab8a10c07..728f92de00 100644 --- a/packages/tool-server/test/harmony-hdc-classification.test.ts +++ b/packages/tool-server/test/harmony-hdc-classification.test.ts @@ -21,6 +21,7 @@ writeFileSync( case "$(cat ${modeFile})" in connect-fail) echo "Connect server failed" >&2 ;; transfer-ok) echo "FileTransfer finish, Size:6, File count = 1, time:10ms rate:0.60kB/s" ;; + truncated) echo "DumpLayout saved to:/data/local/tmp/dump.json" ;; silent) ;; esac exit 0 @@ -93,4 +94,15 @@ describe("runHdcShell without an exit status", () => { expect(err.message).toMatch(/returned no exit status/); expect(getFailureSignal(err)?.error_code).toBe(FAILURE_CODES.HARMONY_SHELL_NO_STATUS); }); + + it("does not quote a cut-off command's own output back as hdc's verdict", async () => { + // `uitest`'s success line is prose by every test {@link hdcProse} applies — + // spaces, no tab, no prefix. Output that arrived without the sentinel is a + // command that ran and was truncated, which is the opposite repair from a + // connector that never reached the device. + hdcBehaves("truncated"); + const err = await rejection(runHdcShell("127.0.0.1:5555", "uitest dumpLayout")); + expect(err.message).toMatch(/returned no exit status/); + expect(err.message).not.toContain("DumpLayout saved to"); + }); }); From 21187f9d0015d15fc3bb84738a15d6761e61eaff Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 20:16:12 +0200 Subject: [PATCH 052/110] test(harmony): pin the uitest queue, sentinel parsing and role table --- .../test/boot-device-harmony.test.ts | 44 ++++++++++++ .../test/harmony-hdc-classification.test.ts | 69 ++++++++++++++++++- .../test/harmony-layout-parser.test.ts | 56 +++++++++++++++ .../test/harmony-uitest-queue.test.ts | 63 +++++++++++++++++ 4 files changed, 231 insertions(+), 1 deletion(-) diff --git a/packages/tool-server/test/boot-device-harmony.test.ts b/packages/tool-server/test/boot-device-harmony.test.ts index 8e4cbe4c69..a89cabf6e2 100644 --- a/packages/tool-server/test/boot-device-harmony.test.ts +++ b/packages/tool-server/test/boot-device-harmony.test.ts @@ -1121,6 +1121,50 @@ describe("boot-device — HarmonyOS emulator path", () => { ).toThrow(/not supported on harmony emulator/); }); + it("coalesces two concurrent boots of one instance into a single start", async () => { + // `Emulator -start` twice for one instance is two managers racing over the + // same instance directory; the second reports "already running" and the + // caller that asked first can no longer tell whose emulator it got. + targets([PHONE], [PHONE, emulatorTarget]); + + const [a, b] = await Promise.all([boot({}), boot({})]); + + expect(spawnMock).toHaveBeenCalledTimes(1); + expect(a).toEqual(b); + }); + + it("does not let a `force` boot join a plain one that would skip the restart", async () => { + // The restart is the whole point of `force`: joined to a plain boot, the + // caller is handed a payload for an instance that was never restarted — and + // on this platform that also means no connect key was resolved, since only + // a boot argent performed can be matched to the key it registers under. + let listed = 0; + listHarmonyInstances.mockImplementation(() => + Promise.resolve([ + { + name: INSTANCE, + deviceType: "Phone", + osVersion: null, + // Running for both boots' opening check, down once the restart's + // stop-wait starts polling. + running: listed++ < 2, + display: PANEL, + }, + ]) + ); + targets([PHONE, staleEmulatorTarget], [PHONE, staleEmulatorTarget], [PHONE, emulatorTarget]); + vi.useFakeTimers(); + + const pending = Promise.all([boot({}), boot({ force: true })]); + await vi.advanceTimersByTimeAsync(60_000); + const [plain, forced] = (await pending) as { udid: string; note?: string }[]; + + expect(runHarmonyEmulator).toHaveBeenCalledWith(["-stop", INSTANCE]); + expect(plain.udid).toBe(`harmony-emulator-${INSTANCE}`); + expect(plain.note).toMatch(/already running/); + expect(forced.udid).toBe(`harmony-${EMULATOR_KEY}`); + }); + it("starts the instance behind a harmony-emulator- udid", async () => { targets([PHONE], [PHONE, emulatorTarget]); diff --git a/packages/tool-server/test/harmony-hdc-classification.test.ts b/packages/tool-server/test/harmony-hdc-classification.test.ts index 728f92de00..8f9dc608d2 100644 --- a/packages/tool-server/test/harmony-hdc-classification.test.ts +++ b/packages/tool-server/test/harmony-hdc-classification.test.ts @@ -3,7 +3,7 @@ import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; import { FAILURE_CODES, getFailureSignal } from "@argent/registry"; -import { hdcFileRecv, runHdcShell } from "../src/utils/harmony-hdc"; +import { hdcFileRecv, hdcProse, runHdcShell } from "../src/utils/harmony-hdc"; // Every case below is a real `hdc` outcome that exits 0 with no `[Fail]` line, // so nothing but what was printed separates them. Measured on hdc 3.2.0d: @@ -22,6 +22,10 @@ case "$(cat ${modeFile})" in connect-fail) echo "Connect server failed" >&2 ;; transfer-ok) echo "FileTransfer finish, Size:6, File count = 1, time:10ms rate:0.60kB/s" ;; truncated) echo "DumpLayout saved to:/data/local/tmp/dump.json" ;; + shell-ok) printf 'line one\nline two\n__argent_hdc_rc=0\n' ;; + shell-decoy) printf '__argent_hdc_rc=99 is device output, not the echo\nreal output\n__argent_hdc_rc=3\n' ;; + shell-inline-token) printf 'saved to /tmp/__argent_hdc_rc=0.json\n' ;; + shell-garbled-status) printf 'work done\n__argent_hdc_rc=x\n' ;; silent) ;; esac exit 0 @@ -106,3 +110,66 @@ describe("runHdcShell without an exit status", () => { expect(err.message).not.toContain("DumpLayout saved to"); }); }); + +describe("runHdcShell recovering the remote exit status", () => { + it("returns the command's own output, without the echo that carried the status", () => { + // The sentinel line is argent's, not the command's. Left in, it lands in + // whatever parses this output — `resolveHarmonyEntry` runs `JSON.parse` on + // it, so a trailing sentinel breaks every launch-app and open-url. + hdcBehaves("shell-ok"); + return expect(runHdcShell("127.0.0.1:5555", "echo hi")).resolves.toEqual({ + stdout: "line one\nline two", + exitCode: 0, + }); + }); + + it("takes the last status echo, not a line the command printed that looks like one", async () => { + hdcBehaves("shell-decoy"); + const result = await runHdcShell("127.0.0.1:5555", "cat /tmp/log"); + expect(result.exitCode).toBe(3); + expect(result.stdout).toContain("real output"); + }); + + it("does not accept the token buried in a line as the status", async () => { + // Matched at the start of a line only: a device path or log line that merely + // contains the token is not the echo, and reading it as one fabricates an + // exit status for a command whose real one never came back. + hdcBehaves("shell-inline-token"); + const err = await rejection(runHdcShell("127.0.0.1:5555", "uitest dumpLayout")); + expect(getFailureSignal(err)?.error_code).toBe(FAILURE_CODES.HARMONY_SHELL_NO_STATUS); + }); + + it("reports an unreadable status as failure rather than as 0", async () => { + hdcBehaves("shell-garbled-status"); + const result = await runHdcShell("127.0.0.1:5555", "uitest dumpLayout"); + expect(result.exitCode).toBe(-1); + }); +}); + +describe("hdcProse tells a diagnostic from output", () => { + const of = (stdout: string, stderr = "") => hdcProse({ stdout, stderr }); + + it("reads a bare diagnostic off either stream", () => { + expect(of("", "Connect server failed\n")).toBe("Connect server failed"); + expect(of("Connect server failed\n")).toBe("Connect server failed"); + }); + + it("does not read hdc's own tabular output as prose", () => { + // `list targets -v` rows are tab-separated and hold spaces in no column that + // matters; taken as a diagnostic, a healthy device table would read as a + // connector failure and hide every attached device. + expect(of("025DEK236V035771\t\tUSB\tConnected\tlocalhost\n")).toBeNull(); + }); + + it("does not read a bare connect key as prose", () => { + expect(of("127.0.0.1:5555\n")).toBeNull(); + }); + + it("leaves the bracketed forms to their own readers", () => { + // `[Fail]` is hdcFailure's, `[Empty]` means "none" — neither is an + // unprefixed diagnostic, and claiming them here would double-report one + // failure and turn an empty device list into an error. + expect(of("[Fail]Not match target founded, check connect-key please\n")).toBeNull(); + expect(of("[Empty]\n")).toBeNull(); + }); +}); diff --git a/packages/tool-server/test/harmony-layout-parser.test.ts b/packages/tool-server/test/harmony-layout-parser.test.ts index e6a33b7605..39a6277107 100644 --- a/packages/tool-server/test/harmony-layout-parser.test.ts +++ b/packages/tool-server/test/harmony-layout-parser.test.ts @@ -84,6 +84,62 @@ describe("parseHarmonyLayout", () => { expect(tree.children[0].label).toBeUndefined(); }); + // `role` is half of what an agent selects on (`await-ui-element {role}`, the + // flow matchers), and it is derived from a type name the app author chose — + // so a mapping quietly reverting to the `type || "Group"` default is invisible + // in a rendered tree yet breaks every selector aimed at it. One row per arm. + it.each([ + ["Text", "StaticText"], + ["Span", "StaticText"], + ["TextClock", "StaticText"], + ["TextInput", "TextField"], + ["TextArea", "TextField"], + ["SearchField", "TextField"], + ["Search", "TextField"], + ["Image", "Image"], + ["SymbolGlyph", "Image"], + ["Checkbox", "Checkbox"], + ["Slider", "Slider"], + ["List", "ScrollView"], + ["Grid", "ScrollView"], + ["Scroll", "ScrollView"], + ["WaterFlow", "ScrollView"], + ["ListItem", "Cell"], + ["GridItem", "Cell"], + ["Swiper", "Pager"], + ["Dialog", "Dialog"], + ["Button", "Button"], + // Unmapped: the author's own component name beats a generic label. + ["CustomWidget", "CustomWidget"], + ])("maps the ArkUI %s onto %s", (type, role) => { + const tree = parseHarmonyLayout( + root([ + node({ type: "root", bundleName: "com.app", bounds: "[0,0][1216,2688]" }, [ + node({ type, text: "x", bounds: "[10,10][210,110]" }), + ]), + ]), + SCREEN + ).tree; + expect(flatten(tree)).toEqual(["Screen", "Window", role]); + }); + + it("reads a Toggle's role off what it can actually do", () => { + // The one arm that is not a lookup: the same ArkUI component is a switch or + // a plain button depending on `checkable`, and only the switch has a state + // an agent can assert. + const build = (checkable: string) => + parseHarmonyLayout( + root([ + node({ type: "root", bundleName: "com.app", bounds: "[0,0][1216,2688]" }, [ + node({ type: "Toggle", checkable, text: "Wi-Fi", bounds: "[10,10][210,110]" }), + ]), + ]), + SCREEN + ).tree; + expect(flatten(build("true"))).toEqual(["Screen", "Window", "Switch"]); + expect(flatten(build("false"))).toEqual(["Screen", "Window", "Button"]); + }); + it("walks through ArkUI layout scaffolding instead of emitting it", () => { const tree = parseHarmonyLayout( root([ diff --git a/packages/tool-server/test/harmony-uitest-queue.test.ts b/packages/tool-server/test/harmony-uitest-queue.test.ts index 27994f83eb..1cada9e195 100644 --- a/packages/tool-server/test/harmony-uitest-queue.test.ts +++ b/packages/tool-server/test/harmony-uitest-queue.test.ts @@ -1,4 +1,5 @@ import { describe, it, expect, vi, beforeEach } from "vitest"; +import { FAILURE_CODES, getFailureSignal } from "@argent/registry"; import { hdcFileRecv as realHdcFileRecv, runHdcShell as realRunHdcShell, @@ -110,4 +111,66 @@ describe("the per-device uitest queue", () => { await releaseOne(); await expect(next).resolves.toBeUndefined(); }); + + it("keeps serializing when a call joins a queue whose head has already finished", async () => { + // The map entry is dropped when the queue drains, and only then: the tail is + // replaced on every enqueue, so a settled call deleting the entry + // unconditionally would hand the next arrival an empty queue while the call + // behind it is still on the device — two `uitest` processes at once, which + // is the one thing the queue exists to prevent. + const first = harmonyScreenCap("dev-a", "/tmp/a.png"); + await settle(); + const second = harmonyScreenCap("dev-a", "/tmp/b.png"); + await settle(); + + await releaseOne(); // `first` settles; `second` is now the one on the device + expect(events).toEqual(["start:dev-a", "end:dev-a", "start:dev-a"]); + + const third = harmonyScreenCap("dev-a", "/tmp/c.png"); + await settle(); + expect(events).toEqual(["start:dev-a", "end:dev-a", "start:dev-a"]); + + await releaseOne(); + await releaseOne(); + await Promise.all([first, second, third]); + }); + + it("reports a `uitest` that ran and failed, without its usage block", async () => { + // `uitest` exits non-zero for every on-device refusal there is — the harmony + // tap, swipe, keyboard, screenshot and describe paths all reach the device + // through this one check. Reading its status as success reports a tap that + // never landed. It prints 12 lines of usage after the diagnostic, so only + // the leading line is surfaced; the rest buries it in the agent's context. + runHdcShell.mockResolvedValueOnce({ + stdout: "error: no such file or directory\nusage: uitest screenCap -p \n -p path", + exitCode: 1, + }); + + const err = await harmonyScreenCap("dev-a", "/tmp/a.png").then( + () => null, + (e: unknown) => e as Error + ); + + expect(getFailureSignal(err)).toMatchObject({ + error_code: FAILURE_CODES.HARMONY_UITEST_FAILED, + failure_command: "hdc", + }); + expect(err?.message).toContain("error: no such file or directory"); + expect(err?.message).not.toContain("usage:"); + }); + + it("lets the next call through after a `uitest` that exited non-zero", async () => { + // The failure is thrown outside the queue, so the queued unit resolves — but + // only if the throw really is outside it. Moved inside, every later call on + // that device would chain onto a rejection with nothing to release it. + runHdcShell.mockResolvedValueOnce({ stdout: "error: device is asleep", exitCode: 1 }); + + await expect(harmonyScreenCap("dev-a", "/tmp/a.png")).rejects.toThrow(/device is asleep/); + + const next = harmonyScreenCap("dev-a", "/tmp/b.png"); + await settle(); + expect(events).toEqual(["start:dev-a"]); + await releaseOne(); + await expect(next).resolves.toBeUndefined(); + }); }); From ed61b888f53b70208adf528b16cd6cb2cd5c3c1a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 20:38:17 +0200 Subject: [PATCH 053/110] fix(harmony): keep the boot's own listings inside the caller's budget `hdc list targets` and the instance listing each carry a ceiling of their own (8s and 6s), spent on top of `bootTimeoutMs` rather than inside it: a poll round opening with 6s left still handed `hdc` 8, so a slow daemon returned the wait past the deadline the caller set. Both are now bounded by what is left of the budget - bounded rather than abandoned, since a listing that lands at the deadline still names the key the wait exists to find. Pins the five boot refusals that had a `FailureError` code but no test asserting it, and the manager run killed with no diagnostic to classify. --- .../src/tools/devices/boot-device.ts | 60 ++++++-- packages/tool-server/src/utils/harmony-cli.ts | 5 +- .../tool-server/src/utils/harmony-devices.ts | 30 ++-- .../tool-server/src/utils/harmony-uitest.ts | 12 +- .../test/boot-device-harmony.test.ts | 130 +++++++++++++++++- packages/tool-server/test/check-deps.test.ts | 41 ++++++ .../test/harmony-binary-resolve.test.ts | 30 ++++ .../tool-server/test/harmony-devices.test.ts | 49 ++++++- .../test/harmony-hdc-classification.test.ts | 5 +- .../test/harmony-uitest-queue.test.ts | 9 +- .../test/list-devices-harmony.test.ts | 22 ++- 11 files changed, 352 insertions(+), 41 deletions(-) diff --git a/packages/tool-server/src/tools/devices/boot-device.ts b/packages/tool-server/src/tools/devices/boot-device.ts index 1909730de9..fbae435d6c 100644 --- a/packages/tool-server/src/tools/devices/boot-device.ts +++ b/packages/tool-server/src/tools/devices/boot-device.ts @@ -60,12 +60,14 @@ import { runHarmonyEmulator, } from "../../utils/harmony-cli"; import { + HARMONY_LIST_TIMEOUT_MS, + HDC_LIST_TIMEOUT_MS, listHarmonyHdcTargets, listHarmonyHdcTargetsStrict, listHarmonyInstances, } from "../../utils/harmony-devices"; import { resolveHdc } from "../../utils/harmony-hdc"; -import { harmonyDisplay, harmonyDumpLayout } from "../../utils/harmony-uitest"; +import { UITEST_TIMEOUT_MS, harmonyDisplay, harmonyDumpLayout } from "../../utils/harmony-uitest"; import { bootElectronApp, type ElectronBootResult } from "./boot-electron"; const execFileAsync = promisify(execFile); @@ -1617,8 +1619,8 @@ const HARMONY_NO_HDC = "connector (or put `hdc` on PATH), then call `list-devices`."; /** The targets `hdc` can drive right now. */ -async function connectedHarmonyTargets() { - const targets = await listHarmonyHdcTargets().catch(() => []); +async function connectedHarmonyTargets(timeoutMs?: number) { + const targets = await listHarmonyHdcTargets(timeoutMs).catch(() => []); return targets.filter((t) => t.state === "Connected"); } @@ -1811,10 +1813,15 @@ type HarmonyTargetOutcome = { * * Only the waiting is abandoned. The probe's own timeout still ends the process * it spawned, so nothing outlives the call that would not have anyway. + * + * Takes a thunk rather than a promise: an argument is evaluated before the call, + * so a promise passed in has already spawned its `hdc` client by the time the + * clock is read here — one per arrival, every one of them past the deadline. */ -async function withinBudget(work: Promise, deadline: number): Promise { +async function withinBudget(start: () => Promise, deadline: number): Promise { const remaining = deadline - Date.now(); if (remaining <= 0) return null; + const work = start(); let timer: ReturnType | undefined; try { return await Promise.race([ @@ -1883,7 +1890,19 @@ async function waitForHarmonyTarget( let sawMismatch = false; let sawUnprobeable = false; for (;;) { - const arrived = (await connectedHarmonyTargets()).filter( + // The listing is bounded by the budget too, not just the probes it feeds: + // `hdc list targets` carries its own 8s ceiling, so a daemon slow enough to + // reach it would return this wait a quarter past `bootTimeoutMs`. Bounded + // rather than abandoned, since a listing that lands at the deadline still + // names the key this whole wait exists to find. + const budget = deadline - Date.now(); + if (budget <= 0) { + return { + key: null, + reason: sawMismatch ? "mismatched" : sawUnprobeable ? "unprobeable" : "none", + }; + } + const arrived = (await connectedHarmonyTargets(Math.min(HDC_LIST_TIMEOUT_MS, budget))).filter( (t) => !before.has(t.connectKey) && couldBeHarmonyEmulator(t) ); const confirmed: string[] = []; @@ -1893,7 +1912,7 @@ async function waitForHarmonyTarget( continue; } const display = await withinBudget( - harmonyDisplay(target.connectKey).catch(() => null), + () => harmonyDisplay(target.connectKey).catch(() => null), deadline ); // A guest that has not composited yet answers `0x0`. The manager side of @@ -1968,15 +1987,23 @@ async function waitForHarmonyDrivable( // Asked before it is spawned, not only before it is awaited: a target // confirmed on arrival alone (no panel to check it against) can resolve at // the deadline itself, and a probe started then is one nothing will ever - // read — holding this device's `uitest` queue for its own timeout, which - // is exactly what the retry the caller is about to be told to make would - // then wait behind. - if (Date.now() >= deadline) return false; - inFlight = harmonyDumpLayout(connectKey, probePath).then( + // read. + const budget = deadline - Date.now(); + if (budget <= 0) return false; + // Capped by the budget as well as by `uitest`'s own ceiling: a probe this + // wait abandons at the deadline still holds the device's `uitest` queue + // until its client is killed, and the retry the caller is about to be told + // to make is what waits behind it. + const probe = harmonyDumpLayout( + connectKey, + probePath, + Math.min(UITEST_TIMEOUT_MS, budget) + ).then( () => true, () => false ); - const answered = await withinBudget(inFlight, deadline); + inFlight = probe; + const answered = await withinBudget(() => probe, deadline); if (answered !== null) inFlight = null; if (answered) return true; checkAlive(); @@ -2023,7 +2050,14 @@ async function waitForHarmonyExit( */ async function waitForHarmonyInstanceStopped(name: string, deadline: number): Promise { for (;;) { - const instances = await listHarmonyInstances().catch(() => null); + // Bounded for the same reason as the arrival wait's listing: this one + // carries a 6s ceiling of its own, and a `force` restart spends one budget + // on the shutdown and the boot that follows it. + const budget = deadline - Date.now(); + if (budget <= 0) return false; + const instances = await listHarmonyInstances(Math.min(HARMONY_LIST_TIMEOUT_MS, budget)).catch( + () => null + ); if (instances && !instances.some((i) => i.name === name && i.running)) return true; const remaining = deadline - Date.now(); if (remaining <= 0) return false; diff --git a/packages/tool-server/src/utils/harmony-cli.ts b/packages/tool-server/src/utils/harmony-cli.ts index cb62afb2da..59087cc6f8 100644 --- a/packages/tool-server/src/utils/harmony-cli.ts +++ b/packages/tool-server/src/utils/harmony-cli.ts @@ -193,8 +193,9 @@ export async function runHarmonyEmulator( /** * Emulator-image downloads are restricted by Huawei to mainland China; outside it * `Emulator -install` prints exactly this and exits 0. Without an image no - * instance can be created, so this is the wall every non-China host hits — worth - * naming precisely instead of reporting the generic create failure it causes. + * instance can be created, so this is the wall a host outside it hits until one + * is obtained — worth naming precisely instead of reporting the generic create + * failure it causes. */ const CHINA_ONLY_MARKER = "available only in the Chinese mainland"; diff --git a/packages/tool-server/src/utils/harmony-devices.ts b/packages/tool-server/src/utils/harmony-devices.ts index f5275e6a1d..5a261b0c9a 100644 --- a/packages/tool-server/src/utils/harmony-devices.ts +++ b/packages/tool-server/src/utils/harmony-devices.ts @@ -135,10 +135,18 @@ export function parseHarmonyInstances(stdout: string): HarmonyInstance[] { */ export const HARMONY_LIST_TIMEOUT_MS = 6_000; -/** Emulator instances, or [] when DevEco Studio isn't installed. */ -export async function listHarmonyInstances(): Promise { +/** + * Emulator instances, or [] when DevEco Studio isn't installed. + * + * `timeoutMs` is for a caller polling this on a deadline of its own: the + * default is a ceiling for one call, not a promise about when the call returns + * to a caller that has less time than that left. + */ +export async function listHarmonyInstances( + timeoutMs = HARMONY_LIST_TIMEOUT_MS +): Promise { if (!(await resolveHarmonyEmulator())) return []; - const result = await runHarmonyEmulator(["-list", "-details"], HARMONY_LIST_TIMEOUT_MS); + const result = await runHarmonyEmulator(["-list", "-details"], timeoutMs); if (emulatorFailure(result)) return []; return parseHarmonyInstances(result.stdout); } @@ -201,8 +209,10 @@ export function parseHdcTargets(stdout: string): HarmonyHdcTarget[] { * Callers that cannot read "no devices" and "no answer" as the same thing want * {@link listHarmonyHdcTargetsStrict}. */ -export async function listHarmonyHdcTargets(): Promise { - return listTargets(false); +export async function listHarmonyHdcTargets( + timeoutMs = HDC_LIST_TIMEOUT_MS +): Promise { + return listTargets(false, timeoutMs); } /** @@ -215,8 +225,10 @@ export async function listHarmonyHdcTargets(): Promise { * BASELINE to compare against cannot, since an empty baseline quietly means * "everything that shows up next is new". */ -export async function listHarmonyHdcTargetsStrict(): Promise { - return listTargets(true); +export async function listHarmonyHdcTargetsStrict( + timeoutMs = HDC_LIST_TIMEOUT_MS +): Promise { + return listTargets(true, timeoutMs); } /** @@ -232,9 +244,9 @@ function hdcDiagnostic(result: { stdout: string; stderr: string }): string | nul return hdcProse(result); } -async function listTargets(strict: boolean): Promise { +async function listTargets(strict: boolean, timeoutMs: number): Promise { if (!(await resolveHdc())) return []; - const result = await runHdc(["list", "targets", "-v"], HDC_LIST_TIMEOUT_MS); + const result = await runHdc(["list", "targets", "-v"], timeoutMs); const failure = hdcFailure(result) ?? hdcDiagnostic(result); if (failure) { if (strict) throw new Error(failure); diff --git a/packages/tool-server/src/utils/harmony-uitest.ts b/packages/tool-server/src/utils/harmony-uitest.ts index 5f0d0a2501..1867a3c44c 100644 --- a/packages/tool-server/src/utils/harmony-uitest.ts +++ b/packages/tool-server/src/utils/harmony-uitest.ts @@ -43,7 +43,12 @@ import { hdcFileRecv, runHdcShell, shellQuote } from "./harmony-hdc"; /** Where on-device artifacts are staged before being copied to the host. */ const REMOTE_TMP = "/data/local/tmp"; -const UITEST_TIMEOUT_MS = 20_000; +/** + * Per-`uitest` ceiling. Exported because a caller working to a deadline of its + * own has to cap its probe against it: a probe outliving the call that made it + * holds this device's queue, and the next call is what waits. + */ +export const UITEST_TIMEOUT_MS = 20_000; /** * `uitest` does not tolerate overlapping invocations: a second call on the @@ -290,10 +295,11 @@ export interface HarmonyLayoutNode { */ export async function harmonyDumpLayout( connectKey: string, - localPath: string + localPath: string, + timeoutMs = UITEST_TIMEOUT_MS ): Promise { await viaDeviceTmp(connectKey, ".json", localPath, async (remotePath) => { - await runUitest(connectKey, `dumpLayout -p ${shellQuote(remotePath)}`); + await runUitest(connectKey, `dumpLayout -p ${shellQuote(remotePath)}`, timeoutMs); }); const raw = await readFile(localPath, "utf8"); try { diff --git a/packages/tool-server/test/boot-device-harmony.test.ts b/packages/tool-server/test/boot-device-harmony.test.ts index a89cabf6e2..9a3279e831 100644 --- a/packages/tool-server/test/boot-device-harmony.test.ts +++ b/packages/tool-server/test/boot-device-harmony.test.ts @@ -30,11 +30,17 @@ vi.mock("../src/utils/harmony-cli", async () => { resolveHarmonyEmulator: (...a: unknown[]) => resolveHarmonyEmulator(...a), }; }); -vi.mock("../src/utils/harmony-devices", () => ({ - listHarmonyInstances: (...a: unknown[]) => listHarmonyInstances(...a), - listHarmonyHdcTargets: (...a: unknown[]) => listHarmonyHdcTargets(...a), - listHarmonyHdcTargetsStrict: (...a: unknown[]) => listHarmonyHdcTargetsStrict(...a), -})); +vi.mock("../src/utils/harmony-devices", async () => { + const actual = await vi.importActual( + "../src/utils/harmony-devices" + ); + return { + ...actual, + listHarmonyInstances: (...a: unknown[]) => listHarmonyInstances(...a), + listHarmonyHdcTargets: (...a: unknown[]) => listHarmonyHdcTargets(...a), + listHarmonyHdcTargetsStrict: (...a: unknown[]) => listHarmonyHdcTargetsStrict(...a), + }; +}); vi.mock("../src/utils/harmony-hdc", async () => { const actual = await vi.importActual( "../src/utils/harmony-hdc" @@ -489,6 +495,69 @@ describe("boot-device — HarmonyOS emulator path", () => { expect(existsSync(probes[0]!)).toBe(false); }); + it("caps the readiness probe at what is left of the budget", async () => { + // The probe outlives the wait that abandons it: `uitest` is serialized per + // device, so a 20s client still running after boot-device returned is 20s + // the caller's first interaction spends queued — and `HARMONY_NOT_DRIVABLE` + // is precisely the answer that tells them to retry it. + const probes: { at: number; timeoutMs?: number }[] = []; + harmonyDumpLayout.mockImplementation((_key: unknown, path: string, timeoutMs?: number) => { + probes.push({ at: Date.now(), timeoutMs }); + writeFileSync(path, "{}"); + return new Promise(() => {}); // never answers, like a guest still coming up + }); + targets([PHONE], [PHONE, emulatorTarget]); + vi.useFakeTimers(); + + // Under `uitest`'s own 20s ceiling, so a probe taking that ceiling would + // outlive the boot — which is the whole failure being pinned. + const startedAt = Date.now(); + const pending = boot({ bootTimeoutMs: 10_000 }); + const settled = expect(pending).resolves.toMatchObject({ + note: expect.stringMatching(/uitest/), + }); + await vi.advanceTimersByTimeAsync(11_000); + await settled; + + expect(probes.length).toBeGreaterThan(0); + for (const probe of probes) { + expect(probe.timeoutMs).toBeGreaterThan(0); + // Killed by the deadline at the latest — never `uitest`'s own ceiling + // measured from whenever this probe happened to start. + expect(probe.at + probe.timeoutMs!).toBeLessThanOrEqual(startedAt + 10_000); + } + }); + + it("caps the target listing at what is left of the budget too", async () => { + // The listing opens every poll round, so its own 8s ceiling is spent on top + // of the budget rather than inside it: the round that starts with 6s left + // still hands `hdc` 8, and the boot answers 2s late every time the last + // round lands there. + const listings: { at: number; timeoutMs?: number }[] = []; + listHarmonyHdcTargetsStrict.mockResolvedValue([PHONE]); + listHarmonyHdcTargets.mockImplementation((timeoutMs?: number) => { + listings.push({ at: Date.now(), timeoutMs }); + // Long enough that rounds start at 0/6s/12s/…, so one of them opens + // inside the last 8s of the budget with the ceiling still to pay. + return new Promise((resolve) => setTimeout(() => resolve([PHONE]), 4_000)); + }); + vi.useFakeTimers(); + + const startedAt = Date.now(); + const pending = boot({ bootTimeoutMs: 30_000 }); + const settled = expect(pending).resolves.toMatchObject({ + note: expect.stringMatching(/bootTimeoutMs/), + }); + await vi.advanceTimersByTimeAsync(40_000); + await settled; + + expect(listings.length).toBeGreaterThan(1); + for (const listing of listings) { + expect(listing.timeoutMs).toBeGreaterThan(0); + expect(listing.at + listing.timeoutMs!).toBeLessThanOrEqual(startedAt + 30_000); + } + }); + it("reports the manager's death during the readiness wait, not a slow guest", async () => { // The wait for the target polls the exit latch for exactly this reason, and // the readiness wait sits in the same window: a dead emulator answers no @@ -509,6 +578,10 @@ describe("boot-device — HarmonyOS emulator path", () => { expect(err.message).toMatch(/disk image corrupted/); expect(err.message).toMatch(/`127\.0\.0\.1:5555`\) was still coming up/); expect(err.message).not.toMatch(/before "Phone_1" registered/); + expect(getFailureSignal(err)).toMatchObject({ + error_code: FAILURE_CODES.BOOT_HARMONY_MANAGER_EXITED, + failure_command: "deveco_emulator", + }); }); it("removes the readiness probe's dump when the wait ends by throwing", async () => { @@ -762,6 +835,11 @@ describe("boot-device — HarmonyOS emulator path", () => { child.die(`"${INSTANCE}" is not found. Please create the device(folder): /x`, 1); await vi.advanceTimersByTimeAsync(3_000); await settled; + + expect(getFailureSignal(await pending.catch((e: unknown) => e))).toMatchObject({ + error_code: FAILURE_CODES.BOOT_HARMONY_START_FAILED, + failure_command: "deveco_emulator", + }); }); it("does not claim the host has no instances when the listing itself failed", async () => { @@ -906,6 +984,42 @@ describe("boot-device — HarmonyOS emulator path", () => { expect(runHarmonyEmulator).toHaveBeenCalledWith(["-stop", INSTANCE]); expect(spawnMock).not.toHaveBeenCalled(); + expect(getFailureSignal(await pending.catch((e: unknown) => e))).toMatchObject({ + error_code: FAILURE_CODES.BOOT_HARMONY_STOP_TIMEOUT, + failure_command: "deveco_emulator", + }); + }); + + it("caps the stop-wait listing at what is left of the budget", async () => { + // Same shape one stage earlier: a `force` restart pays for the shutdown out + // of the boot's budget, so the listing that watches for the instance to go + // down must not run past the deadline the caller set for the whole restart. + const listings: { at: number; timeoutMs?: number }[] = []; + listHarmonyInstances.mockImplementation((timeoutMs?: number) => { + listings.push({ at: Date.now(), timeoutMs }); + return new Promise((resolve) => + setTimeout( + () => resolve([{ name: INSTANCE, deviceType: "Phone", osVersion: null, running: true }]), + 4_000 + ) + ); + }); + vi.useFakeTimers(); + + const startedAt = Date.now(); + const pending = boot({ force: true, bootTimeoutMs: 30_000 }); + const settled = expect(pending).rejects.toThrow(/still running when the 30s budget ran out/); + await vi.advanceTimersByTimeAsync(40_000); + await settled; + + // The pre-stop lookup runs unbounded; every wait round after it is the + // one on the clock. + const waitRounds = listings.slice(1); + expect(waitRounds.length).toBeGreaterThan(1); + for (const listing of waitRounds) { + expect(listing.timeoutMs).toBeGreaterThan(0); + expect(listing.at + listing.timeoutMs!).toBeLessThanOrEqual(startedAt + 30_000); + } }); it("says the instance is down when a `force` restart refuses at the snapshot", async () => { @@ -938,6 +1052,12 @@ describe("boot-device — HarmonyOS emulator path", () => { expect(runHarmonyEmulator).toHaveBeenCalledWith(["-stop", INSTANCE]); expect(spawnMock).not.toHaveBeenCalled(); + // `hdc`, not the manager: this refusal is the device table being unreadable, + // and the binary named is what the caller has to go and check. + expect(getFailureSignal(await pending.catch((e: unknown) => e))).toMatchObject({ + error_code: FAILURE_CODES.BOOT_HARMONY_TARGET_LIST_FAILED, + failure_command: "hdc", + }); }); it("stops probing arrivals once the budget is spent, rather than once per arrival", async () => { diff --git a/packages/tool-server/test/check-deps.test.ts b/packages/tool-server/test/check-deps.test.ts index a265494ea0..52a8f60c11 100644 --- a/packages/tool-server/test/check-deps.test.ts +++ b/packages/tool-server/test/check-deps.test.ts @@ -20,6 +20,21 @@ vi.mock("../src/utils/android-binary", () => ({ __resetAndroidBinaryCacheForTesting: () => {}, })); +// The two HarmonyOS deps ship inside DevEco Studio rather than on PATH, so they +// probe through their own resolvers. Mocked for the same reason as the Android +// pair above: a dev machine with DevEco installed would report them available +// however the test stubs PATH. +const resolveHdcMock = vi.fn(); +const resolveHarmonyEmulatorMock = vi.fn(); +vi.mock("../src/utils/harmony-hdc", async (importOriginal) => ({ + ...(await importOriginal()), + resolveHdc: () => resolveHdcMock(), +})); +vi.mock("../src/utils/harmony-cli", async (importOriginal) => ({ + ...(await importOriginal()), + resolveHarmonyEmulator: () => resolveHarmonyEmulatorMock(), +})); + import { DependencyMissingError, __resetDepCacheForTests, @@ -47,6 +62,8 @@ describe("check-deps", () => { __resetDepCacheForTests(); commandOnPathMock.mockReset(); resolveAndroidBinaryMock.mockReset(); + resolveHdcMock.mockReset().mockResolvedValue("/deveco/hdc"); + resolveHarmonyEmulatorMock.mockReset().mockResolvedValue("/deveco/Emulator"); }); it("returns without throwing when all deps are on PATH", async () => { @@ -94,4 +111,28 @@ describe("check-deps", () => { stubProbe(["xcrun"]); await expect(ensureDep("xcrun")).rejects.toBeInstanceOf(DependencyMissingError); }); + + // Neither HarmonyOS dep is a PATH binary on an IDE install: `hdc` lives under + // the SDK and the manager is named `Emulator`, too generic to match on PATH. + // Probed through `commandOnPath` they read as missing on every machine that + // has DevEco Studio, gating the whole platform off behind a 424. + it("probes the HarmonyOS deps through the DevEco resolvers, not PATH", async () => { + stubProbe(["hdc", "Emulator", "harmony-emulator"]); + + await expect(ensureDeps(["hdc", "harmony-emulator"])).resolves.toBeUndefined(); + expect(commandOnPathMock).not.toHaveBeenCalledWith("hdc"); + expect(commandOnPathMock).not.toHaveBeenCalledWith("harmony-emulator"); + }); + + it("reports each HarmonyOS dep as missing when its own resolver finds nothing", async () => { + stubProbe([]); + resolveHdcMock.mockResolvedValue(null); + resolveHarmonyEmulatorMock.mockResolvedValue(null); + + // The hint is the whole value of the 424: it names DevEco Studio and + // `$DEVECO_STUDIO_HOME`, which is the only way to point at a non-macOS + // install. A caller told merely "hdc is missing" has nowhere to go. + await expect(ensureDep("hdc")).rejects.toThrow(/DEVECO_STUDIO_HOME/); + await expect(ensureDep("harmony-emulator")).rejects.toBeInstanceOf(DependencyMissingError); + }); }); diff --git a/packages/tool-server/test/harmony-binary-resolve.test.ts b/packages/tool-server/test/harmony-binary-resolve.test.ts index ccd6fb6c37..245363ad2b 100644 --- a/packages/tool-server/test/harmony-binary-resolve.test.ts +++ b/packages/tool-server/test/harmony-binary-resolve.test.ts @@ -2,6 +2,7 @@ import { afterEach, beforeEach, describe, expect, it, vi } from "vitest"; import { chmod, mkdir, mkdtemp, rm, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; import { join } from "node:path"; +import { FAILURE_CODES, getFailureSignal } from "@argent/registry"; // Force the PATH probe to miss. `resolveHdc` falls back to `command -v hdc` for // standalone command-line-tools installs, and on a host that has one it would @@ -139,6 +140,35 @@ describe("DevEco Studio binary resolution", () => { }); }); + describe("running the manager once it has been resolved", () => { + it("classifies a run killed with nothing to say as a subprocess failure", async () => { + // The one `Emulator` failure with no diagnostic to read: killed at the + // timeout, so `emulatorFailure`'s marker list has nothing to match and the + // caller cannot classify it downstream. Left a bare `Error` it buckets as + // REGISTRY_TOOL_EXECUTION_FAILED, which is where every unclassified throw + // in the server already sits — and drops the binary that hung with it. + const bundle = join(tmpRoot, "DevEco-Studio.app"); + const emulator = join(bundle, "Contents", EMULATOR_RELATIVE); + await mkdir(join(emulator, ".."), { recursive: true }); + await writeFile(emulator, "#!/usr/bin/env bash\nsleep 30\n"); + await chmod(emulator, 0o755); + process.env.DEVECO_STUDIO_HOME = bundle; + + vi.resetModules(); + const { runHarmonyEmulator } = await import("../src/utils/harmony-cli"); + const err = await runHarmonyEmulator(["-list"], 100).then( + () => null, + (e: unknown) => e + ); + + expect(getFailureSignal(err)).toMatchObject({ + error_code: FAILURE_CODES.HARMONY_EMULATOR_COMMAND_FAILED, + error_kind: "subprocess", + failure_command: "deveco_emulator", + }); + }); + }); + describe("$DEVECO_STUDIO_HOME pointing at a root with no DevEco install", () => { // A set-but-wrong variable must not be able to hide a working default // install: it is tried first, then the default, exactly as `androidRoots()` diff --git a/packages/tool-server/test/harmony-devices.test.ts b/packages/tool-server/test/harmony-devices.test.ts index 1bae6d9c29..48768d9019 100644 --- a/packages/tool-server/test/harmony-devices.test.ts +++ b/packages/tool-server/test/harmony-devices.test.ts @@ -1,5 +1,25 @@ -import { describe, it, expect } from "vitest"; -import { parseHarmonyInstances, parseHdcTargets } from "../src/utils/harmony-devices"; +import { describe, it, expect, vi, beforeEach } from "vitest"; +import { + HARMONY_LIST_TIMEOUT_MS, + HDC_LIST_TIMEOUT_MS, + listHarmonyHdcTargets, + listHarmonyInstances, + parseHarmonyInstances, + parseHdcTargets, +} from "../src/utils/harmony-devices"; +import { runHarmonyEmulator as realRunHarmonyEmulator } from "../src/utils/harmony-cli"; +import { runHdc as realRunHdc } from "../src/utils/harmony-hdc"; + +vi.mock("../src/utils/harmony-cli", async (importOriginal) => ({ + ...(await importOriginal()), + resolveHarmonyEmulator: vi.fn(async () => "/deveco/Emulator"), + runHarmonyEmulator: vi.fn(async () => ({ stdout: "[Empty]", stderr: "" })), +})); +vi.mock("../src/utils/harmony-hdc", async (importOriginal) => ({ + ...(await importOriginal()), + resolveHdc: vi.fn(async () => "/deveco/hdc"), + runHdc: vi.fn(async () => ({ stdout: "[Empty]", stderr: "" })), +})); import { HARMONY_EMULATOR_ID_PREFIX, HARMONY_ID_PREFIX, @@ -221,3 +241,28 @@ describe("HarmonyOS device ids", () => { expect(harmonyConnectKey("025DEK236V035771")).toBe("025DEK236V035771"); }); }); + +// Both wrappers default to 30s, which is itself above `list-devices`' +// BRANCH_DEADLINE_MS — so a discovery call that forgets to pass its own timeout +// is not merely slower, it is a branch the backstop truncates while it is still +// working, dropping every HarmonyOS device from the list. The deadline test +// compares the two constants; this is what ties them to the calls. +describe("discovery calls are bounded by the constants list-devices was sized against", () => { + beforeEach(() => vi.clearAllMocks()); + + it("bounds `Emulator -list -details`", async () => { + await listHarmonyInstances(); + expect(vi.mocked(realRunHarmonyEmulator)).toHaveBeenCalledWith( + ["-list", "-details"], + HARMONY_LIST_TIMEOUT_MS + ); + }); + + it("bounds `hdc list targets`", async () => { + await listHarmonyHdcTargets(); + expect(vi.mocked(realRunHdc)).toHaveBeenCalledWith( + ["list", "targets", "-v"], + HDC_LIST_TIMEOUT_MS + ); + }); +}); diff --git a/packages/tool-server/test/harmony-hdc-classification.test.ts b/packages/tool-server/test/harmony-hdc-classification.test.ts index 8f9dc608d2..4a18609b91 100644 --- a/packages/tool-server/test/harmony-hdc-classification.test.ts +++ b/packages/tool-server/test/harmony-hdc-classification.test.ts @@ -54,8 +54,9 @@ describe("hdcFileRecv success is read off the transfer line", () => { it("refuses a transfer the connector never made", async () => { // The whole failure is on stderr with no prefix, so classifying on `[Fail]` - // alone resolves — handing `harmony-screen` a path with nothing at it, which - // it then reports as a decode failure of a file that was never copied. + // alone resolves — handing `harmony-screen` a path with nothing at it, where + // `readFile` throws a bare ENOENT naming a tmp file, the connector's own + // diagnostic having already been dropped. hdcBehaves("connect-fail"); const err = await rejection(hdcFileRecv("127.0.0.1:5555", remote, join(root, "out.png"))); expect(err.message).toContain("Connect server failed"); diff --git a/packages/tool-server/test/harmony-uitest-queue.test.ts b/packages/tool-server/test/harmony-uitest-queue.test.ts index 1cada9e195..4cacab11e0 100644 --- a/packages/tool-server/test/harmony-uitest-queue.test.ts +++ b/packages/tool-server/test/harmony-uitest-queue.test.ts @@ -47,10 +47,11 @@ async function releaseOne(): Promise { describe("the per-device uitest queue", () => { it("holds a second call on one device until the first has finished", async () => { - // `uitest` is single-instance on the device: a second one launched while the - // first is running fails with `Another uitest is running`, so two argent - // calls landing together would fail whichever lost the race — a failure the - // caller cannot do anything about and did not cause. + // `uitest` does not tolerate overlapping invocations: a second one launched + // while the first is running blocks, and is SIGKILLed if that carries it + // past the 20s timeout — measured as one 20s failure out of two concurrent + // `dumpLayout`s. Two argent calls landing together take that loss on a race + // the caller did not cause and cannot do anything about. const first = harmonyScreenCap("dev-a", "/tmp/a.png"); const second = harmonyScreenCap("dev-a", "/tmp/b.png"); await settle(); diff --git a/packages/tool-server/test/list-devices-harmony.test.ts b/packages/tool-server/test/list-devices-harmony.test.ts index 1a1af7f9a4..24dc2441a4 100644 --- a/packages/tool-server/test/list-devices-harmony.test.ts +++ b/packages/tool-server/test/list-devices-harmony.test.ts @@ -29,7 +29,7 @@ vi.mock("../src/utils/harmony-devices", () => ({ listHarmonyHdcTargets: vi.fn(async () => []), })); -import { listDevicesTool } from "../src/tools/devices/list-devices"; +import { BRANCH_DEADLINE_MS, listDevicesTool } from "../src/tools/devices/list-devices"; import { listHarmonyInstances, listHarmonyHdcTargets } from "../src/utils/harmony-devices"; const instances = vi.mocked(listHarmonyInstances); @@ -171,4 +171,24 @@ describe("list-devices HarmonyOS branch", () => { ["emulator", "stopped_emu", "stopped"], ]); }); + + it("returns the rest of the fleet when a HarmonyOS probe never answers", async () => { + // Both harmony probes are branches of the same `Promise.all` as every other + // platform's, so one that hangs — a wedged `hdc` daemon, a stuck manager — + // takes the whole `alwaysLoad` tool with it unless the branch deadline + // stands behind its per-call timeout. The listing must come back without + // the harmony entries rather than not come back. + vi.useFakeTimers(); + hdcTargets.mockImplementation(() => new Promise(() => {})); + instances.mockResolvedValue([ + { name: "emu", deviceType: "Phone", osVersion: null, running: false, display: null }, + ]); + + const pending = listDevicesTool.execute!({}, {}); + await vi.advanceTimersByTimeAsync(BRANCH_DEADLINE_MS + 1_000); + const devices = (await pending).devices.filter((d) => d.platform === "harmony"); + + expect(devices.map((d) => d.kind)).toEqual(["emulator"]); + vi.useRealTimers(); + }); }); From 5a53003651587cd0fbda1d37c102704269619cf7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 20:38:24 +0200 Subject: [PATCH 054/110] fix(await-screen-idle): report why the screen was never read A wait that never settles usually ends on a screen the reader could not see - a degraded AX tree, a suspended panel, an app whose native inspection needs a restart - and the empty tree it leaves behind says none of that. `await-ui-element` already folds those caveats onto its own timeout note off the same read; here they were dropped, leaving the caller a bare "did not settle" and a longer timeout as the only move. The shared clause builder moves to `poll-describe-tree`, next to the loop both wait tools run. Also corrects the prose the HarmonyOS support left behind: describe's frame line named `gesture-pinch` on platforms that refuse it, the test-ui-flow skill still dispatched to iOS and Android only, and the README's prerequisite still read the emulator image restriction as absolute. --- README.md | 2 +- .../skills/argent-device-interact/SKILL.md | 2 +- .../skills/argent-test-ui-flow/SKILL.md | 15 +++-- .../src/tools/await-screen-idle/index.ts | 65 +++++++++++-------- .../src/tools/await-ui-element/index.ts | 11 +--- .../src/tools/describe/format-tree.ts | 24 +++++-- .../tool-server/src/tools/describe/index.ts | 5 +- .../src/utils/poll-describe-tree.ts | 20 ++++++ .../tool-server/test/await-harmony.test.ts | 19 ++++++ .../test/describe-format-tree.test.ts | 25 ++++++- 10 files changed, 133 insertions(+), 55 deletions(-) diff --git a/README.md b/README.md index 94418ae9c1..e6ab81ff32 100644 --- a/README.md +++ b/README.md @@ -59,7 +59,7 @@ Argent drives a growing set of targets through a single toolkit, each with the r - For iOS / tvOS: macOS with **Xcode** installed (Apple TV uses tvOS simulators — Xcode downloads the tvOS runtime on demand) - For Android / Android TV: **Android SDK Platform Tools** (`adb`) on `PATH`, and the **Android Emulator** package if you want to boot AVDs from Argent. Create AVDs via Android Studio or `avdmanager`. - For Fire TV (Vega): the **Vega SDK** (`vega` CLI) on `PATH` -- For HarmonyOS: **DevEco Studio**, which ships both binaries Argent drives - `hdc` (the device connector, for phones) and the `Emulator` manager. Both are looked up inside DevEco Studio's install root: `$DEVECO_STUDIO_HOME` first when it is set, then `/Applications/DevEco-Studio.app` on macOS, each tried both as given and with `Contents/` appended so the app bundle itself works. `hdc` also falls back to `PATH` for hosts that installed the OpenHarmony command-line tools without the IDE; `Emulator` never does, since it is too generic a name to match safely. Driving a phone needs only `hdc`. Huawei restricts the emulator image download to mainland China, so outside it no emulator instance can be created. +- For HarmonyOS: **DevEco Studio**, which ships both binaries Argent drives - `hdc` (the device connector, for phones) and the `Emulator` manager. Both are looked up inside DevEco Studio's install root: `$DEVECO_STUDIO_HOME` first when it is set, then `/Applications/DevEco-Studio.app` on macOS, each tried both as given and with `Contents/` appended so the app bundle itself works. `hdc` also falls back to `PATH` for hosts that installed the OpenHarmony command-line tools without the IDE; `Emulator` never does, since it is too generic a name to match safely. Driving a phone needs only `hdc`. Huawei restricts the emulator **image** download to mainland China, so on a host outside it no emulator instance can be created until an image is obtained. - For Electron / Chromium: nothing extra to control an already-running app - just launch it with `--remote-debugging-port`, or let Argent spawn your Electron app for you ##### Linux host: extra prerequisites for Android emulators diff --git a/packages/skills/skills/argent-device-interact/SKILL.md b/packages/skills/skills/argent-device-interact/SKILL.md index 8e0699e8a1..fb515b3dff 100644 --- a/packages/skills/skills/argent-device-interact/SKILL.md +++ b/packages/skills/skills/argent-device-interact/SKILL.md @@ -227,7 +227,7 @@ Use after launch/navigation and before a raw tap, when an early-painted element { "udid": "", "timeoutMs": 3000, "minStableMs": 250 } ``` -On local iOS, Android, Chromium, and HarmonyOS, the tool waits for a non-empty `describe` tree to stop changing. Continue only when `settled: true`, and read the `note` if one came with it — a `settled: true` carrying one means the tree it settled on is not the whole live screen (a suspended HarmonyOS panel settles at once on its last frame, and taps land nowhere until `button` wakes it; a Chromium page past the walker's node budget settles on a partial tree). Pair it with a destination-specific `await-ui-element`; stillness does not identify a screen. +On local iOS, Android, Chromium, and HarmonyOS, the tool waits for a non-empty `describe` tree to stop changing. Continue only when `settled: true`, and read the `note` if one came with it — on `settled: true` it means the tree it settled on is not the whole live screen (a suspended HarmonyOS panel settles at once on its last frame, and taps land nowhere until `button` wakes it; a Chromium page past the walker's node budget settles on a partial tree), and on `settled: false` it says why the screen was never read rather than never still, which is what tells you a longer `timeoutMs` is not the answer. Pair it with a destination-specific `await-ui-element`; stillness does not identify a screen. Use it only for live diagnosis. Do not record it or put it in `run-sequence`. Flows use `await: { idle: true }`, which also compares pixels. This live tool can return during a presentation-layer animation. diff --git a/packages/skills/skills/argent-test-ui-flow/SKILL.md b/packages/skills/skills/argent-test-ui-flow/SKILL.md index b4b896475c..4b8bbac2e6 100644 --- a/packages/skills/skills/argent-test-ui-flow/SKILL.md +++ b/packages/skills/skills/argent-test-ui-flow/SKILL.md @@ -5,16 +5,17 @@ description: Autonomously test an app UI (iOS, Android, or HarmonyOS) by running ## Platform-agnostic -The interaction tool names are identical on iOS and Android — `gesture-tap`, `gesture-swipe`, `describe`, `screenshot`, `launch-app`, etc. — and the tool-server auto-dispatches based on the `udid` you pass (UUID-shape → iOS, adb serial → Android). +The interaction tool names are identical on iOS, Android and HarmonyOS — `gesture-tap`, `gesture-swipe`, `describe`, `screenshot`, `launch-app`, etc. — and the tool-server auto-dispatches based on the `udid` you pass (UUID-shape → iOS, `harmony-` → HarmonyOS, adb serial → Android). HarmonyOS takes one contact per call, so `gesture-pinch`, `gesture-rotate`, `gesture-custom` and `rotate` are refused there; every other tool in this workflow applies. **Before testing, resolve which device to test on.** Call `list-devices` and follow ``: prefer a running device on any platform; Once a platform is chosen, the per-platform setup skill takes over: -| Platform | Setup skill | Find devices with | -| -------- | ------------------------------- | ----------------------------------------------------------- | -| iOS | `argent-ios-simulator-setup` | `list-devices` → `boot-device` with `udid` if none booted | -| Android | `argent-android-emulator-setup` | `list-devices` → `boot-device` with `avdName` if none ready | +| Platform | Setup skill | Find devices with | +| --------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------- | +| iOS | `argent-ios-simulator-setup` | `list-devices` → `boot-device` with `udid` if none booted | +| Android | `argent-android-emulator-setup` | `list-devices` → `boot-device` with `avdName` if none ready | +| HarmonyOS | `argent-device-interact` | `list-devices` → a phone on `hdc` is ready as-is; `boot-device` with a `harmony-emulator-` id otherwise | ## 1. Workflow @@ -25,7 +26,7 @@ For implementation tasks that modify visible UI, this workflow can also serve as 1. **Baseline screenshot**: Call `screenshot` to see the current UI state. For visual regression comparison or UI change verification, capture the baseline at `scale: 1.0` with `includeImageInContext: false` and keep the returned `path` before editing whenever feasible. 2. **Find target**: Before tapping, use a discovery tool to get element coordinates: - **React Native apps**: use `debugger-component-tree` — it returns component names with (tap: x,y) coordinates. This is the preferred tool for RN apps on either platform. To use it, resolve the `argent-react-native-app-workflow` skill for setup; on Android you must also run `adb -s reverse tcp:8081 tcp:8081` so Metro is reachable from the device. - - **Standard app screens and in-app modals**: use `describe`. On iOS this returns the AX tree (falls back to native-devtools when AX is empty); on Android it returns the uiautomator tree in the same DescribeNode shape. + - **Standard app screens and in-app modals**: use `describe`. On iOS this returns the AX tree (falls back to native-devtools when AX is empty); on Android it returns the uiautomator tree, and on HarmonyOS the `uitest dumpLayout` tree, both in the same DescribeNode shape. - **Permission prompts / system modal overlays**: try `describe` first. Fall back to `screenshot` only if the overlay is not exposed reliably. When the app raises its own permission dialog, answer it here — that's the real flow under test. To take a prompt _out_ of the flow (pre-grant/deny before launch, re-enable a permission the user already denied, or reset it so the dialog reappears), use the `argent-settings-permissions` skill during setup instead of interacting with the dialog. - **Fallback**: use `screenshot` to estimate where the desired component is, then verify immediately after the action. 3. **Interact**: Perform the action (`gesture-tap`, `gesture-swipe`, `keyboard`, `button`, ...) — you receive a screenshot automatically. @@ -122,7 +123,7 @@ Steps: | Skill | When to use | | ---------------------------------- | -------------------------------------------------------- | -| `argent-device-interact` | Tool usage for tapping, swiping, typing (iOS + Android) | +| `argent-device-interact` | Tool usage for tapping, swiping, typing (every platform) | | `argent-screenshot-diff` | Visual regression and before/after screenshot comparison | | `argent-ios-simulator-setup` | Booting and connecting an iOS simulator | | `argent-android-emulator-setup` | Booting and connecting an Android emulator | diff --git a/packages/tool-server/src/tools/await-screen-idle/index.ts b/packages/tool-server/src/tools/await-screen-idle/index.ts index 91f5a3368c..3c7b0f26d7 100644 --- a/packages/tool-server/src/tools/await-screen-idle/index.ts +++ b/packages/tool-server/src/tools/await-screen-idle/index.ts @@ -13,7 +13,11 @@ import { isTvOsSimulator } from "../../utils/ios-devices"; import { isAndroidTv } from "../../utils/adb"; import { assertSupported } from "../../utils/capability"; import { ensureDeps } from "../../utils/check-deps"; -import { pollDescribeTree } from "../../utils/poll-describe-tree"; +import { + pollDescribeTree, + readCaveats, + type PollDescribeTreeResult, +} from "../../utils/poll-describe-tree"; import { READ_CAVEAT_SOURCES } from "../describe/contract"; import type { DescribeNode, DescribeTreeData } from "../describe/contract"; import { describeIos, iosRequires } from "../describe/platforms/ios"; @@ -103,6 +107,25 @@ function treeSignature(root: DescribeNode): string { return parts.join("\n"); } +// Why a wait ended somewhere other than on a screen that plainly settled. +// +// A screen that never settles is usually one the reader could not see — a +// degraded AX tree, a panel that is off, an app whose native inspection needs a +// restart — and none of that is recoverable from the empty tree left behind. A +// failed fetch outranks those, since a device that went away and a screen that +// stayed busy are opposite diagnoses. On a success only +// {@link READ_CAVEAT_SOURCES} applies, for the reason given there. +function idleNote(poll: PollDescribeTreeResult): string | undefined { + if (poll.result !== true) { + if (poll.lastError) return `last tree fetch failed: ${poll.lastError}`; + const caveats = readCaveats(poll.lastData); + return caveats.length === 0 ? undefined : caveats.join("; "); + } + const settledOn = poll.lastData; + if (!settledOn?.hint || !READ_CAVEAT_SOURCES.has(settledOn.source)) return undefined; + return settledOn.hint; +} + // `await-screen-idle` waits for the screen to *settle* — render content and stop // changing — rather than for a named element like `await-ui-element`. The MCP // layer uses it to time its auto-screenshot: capture once the screen is stable @@ -139,11 +162,13 @@ export function createAwaitScreenIdleTool(registry: Registry): ToolDefinition> = { + "harmony-uitest": "gesture-tap / gesture-swipe", + "cdp-dom": "gesture-tap / gesture-scroll / gesture-drag", +}; + +const DEFAULT_GESTURE_TOOLS = "gesture-tap / gesture-swipe / gesture-pinch"; + export function formatDescribeTree(root: DescribeNode, opts: FormatDescribeOptions): string { // iOS providers (ax-service, native-devtools) emit a flat list under a // synthetic root, so the flat renderer is correct. Sources that produce @@ -205,12 +222,7 @@ export function formatDescribeTree(root: DescribeNode, opts: FormatDescribeOptio ); } else { header.push( - // HarmonyOS drives touch through `uitest uiInput`, which has no multi-touch - // verb — `gesture-pinch` has no harmony backend and refuses the device, so - // naming it here would send the agent at a tool it cannot reach. - opts.source === "harmony-uitest" - ? "Pass them straight to gesture-tap / gesture-swipe, which expect this same space." - : "Pass them straight to gesture-tap / gesture-swipe / gesture-pinch, which expect this same space." + `Pass them straight to ${GESTURE_TOOLS_BY_SOURCE[opts.source] ?? DEFAULT_GESTURE_TOOLS}, which expect this same space.` ); header.push( "To tap an element, use its centre: tap_x = frame.x + frame.width / 2, tap_y = frame.y + frame.height / 2." diff --git a/packages/tool-server/src/tools/describe/index.ts b/packages/tool-server/src/tools/describe/index.ts index e2eab9f01c..b1db648e4e 100644 --- a/packages/tool-server/src/tools/describe/index.ts +++ b/packages/tool-server/src/tools/describe/index.ts @@ -173,8 +173,9 @@ with tap coordinates. When no dialog is present, it returns the foreground app's Returns \`{ description, source }\` where \`description\` is a text rendering of the UI tree — one line per element with its role, label/value/id, interactivity flags, and frame. Frame coordinates -are normalized [0,1] fractions of the screen / window width/height (not pixels) — the same space as -gesture-tap / gesture-swipe / gesture-pinch. +are normalized [0,1] fractions of the screen / window width/height (not pixels) — the same space the +gesture tools take. The response header names the ones this device accepts, which on a platform +without multi-touch is fewer than the full set. To tap an element use the centre of its frame: \`tap_x = frame.x + frame.width / 2\`, \`tap_y = frame.y + frame.height / 2\`. The same formula appears in the response header so it diff --git a/packages/tool-server/src/utils/poll-describe-tree.ts b/packages/tool-server/src/utils/poll-describe-tree.ts index 4bad10ab2c..2fb9245f08 100644 --- a/packages/tool-server/src/utils/poll-describe-tree.ts +++ b/packages/tool-server/src/utils/poll-describe-tree.ts @@ -50,6 +50,26 @@ export interface PollDescribeTreeResult { lastError?: string; } +/** + * What the last read said about itself, as clauses for a wait's timeout note: + * a tree the adapter flagged as degraded or partial, and an app whose native + * inspection needs a restart before it can be seen at all. A wait that ends + * without its condition met ends on one of these far more often than on a + * screen that genuinely kept moving, and neither wait tool can tell them apart + * from the tree alone. + */ +export function readCaveats(lastData: DescribeTreeData | null): string[] { + if (!lastData) return []; + const caveats: string[] = []; + if (lastData.should_restart) { + caveats.push( + "the foreground app may need a restart for native inspection — call restart-app and retry" + ); + } + if (lastData.hint) caveats.push(lastData.hint); + return caveats; +} + export async function pollDescribeTree( args: PollDescribeTreeArgs ): Promise> { diff --git a/packages/tool-server/test/await-harmony.test.ts b/packages/tool-server/test/await-harmony.test.ts index 2df673ce99..3637568ca8 100644 --- a/packages/tool-server/test/await-harmony.test.ts +++ b/packages/tool-server/test/await-harmony.test.ts @@ -262,6 +262,25 @@ describe("await-screen-idle on HarmonyOS", () => { expect(result.note).toMatch(/taps land nowhere/i); }); + it("says why the screen was never readable, not just that it never settled", async () => { + // A windowless dump can never settle — the loop wants content before it + // starts timing stability — so this ends as a timeout with the reason + // already in hand. Dropped, "did not settle" reads as a busy screen and the + // caller waits longer for a screen that is not being drawn; the sibling + // `await-ui-element` puts the same hint on its own timeout note, off the + // very same read. + sequenceDumps([dumpWith()]); + const tool = createAwaitScreenIdleTool(harmonyRegistry()); + + const result = await tool.execute( + {}, + { udid: HARMONY_ID, timeoutMs: 40, pollIntervalMs: 10, minStableMs: 10 } + ); + + expect(result.settled).toBe(false); + expect(result.note).toMatch(/no windows/i); + }); + it("does not settle while the dump keeps changing (times out)", async () => { sequenceDumps(Array.from({ length: 30 }, (_, i) => dumpWith(`item-${i}`))); const tool = createAwaitScreenIdleTool(harmonyRegistry()); diff --git a/packages/tool-server/test/describe-format-tree.test.ts b/packages/tool-server/test/describe-format-tree.test.ts index 1a000f2e32..df1bfcbe48 100644 --- a/packages/tool-server/test/describe-format-tree.test.ts +++ b/packages/tool-server/test/describe-format-tree.test.ts @@ -1,6 +1,6 @@ import { describe, expect, it } from "vitest"; import { formatDescribeTree } from "../src/tools/describe/format-tree"; -import type { DescribeNode } from "../src/tools/describe/contract"; +import type { DescribeNode, DescribeSource } from "../src/tools/describe/contract"; function leaf( partial: Partial & { role: string; frame: DescribeNode["frame"] } @@ -277,8 +277,8 @@ describe("formatDescribeTree", () => { }); // The header text is part of the agent-visible response, so it must keep - // pointing at gesture-tap / gesture-swipe / gesture-pinch and the centre - // formula. If this drifts again the runtime help is silently misleading. + // pointing at the gesture tools the source's platform actually has, plus the + // centre formula. If this drifts the runtime help is silently misleading. it("renders the coordinate-space + tap-formula header on every call", () => { const empty: DescribeNode = { role: "AXGroup", @@ -292,6 +292,25 @@ describe("formatDescribeTree", () => { expect(out).toContain("tap_y = frame.y + frame.height / 2"); }); + it("names only the gesture tools the source's own platform supports", () => { + // Every tool named here is one the agent is invited to call next, and each + // platform's capability gate refuses the ones it has no backend for — so a + // fixed list spends a round trip on a 400 the header could have avoided. + // Chromium is the sharp case: it has neither swipe nor pinch, and the two + // tools it does have (scroll, drag) went unmentioned. + const empty: DescribeNode = { + role: "AXGroup", + frame: { x: 0, y: 0, width: 1, height: 1 }, + children: [], + }; + const header = (source: DescribeSource) => formatDescribeTree(empty, { source }).split("\n")[3]; + + expect(header("cdp-dom")).toContain("gesture-tap / gesture-scroll / gesture-drag"); + expect(header("harmony-uitest")).toContain("gesture-tap / gesture-swipe,"); + expect(header("ax-service")).toContain("gesture-tap / gesture-swipe / gesture-pinch"); + expect(header("uiautomator")).toContain("gesture-tap / gesture-swipe / gesture-pinch"); + }); + // Bluesky-style names mix emoji, ZWJ sequences, and bidirectional isolate // markers (U+202A/U+202C). Those must pass through escapeForLine unchanged // — only ASCII control chars (\\n, \\r, \\t) get backslash-escaped. From 7e7315627f19e0d3a2d13839284b4fad3e9c0c74 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 20:45:44 +0200 Subject: [PATCH 055/110] test(wait): pin the caveats both wait tools fold onto a timeout Neither tool had a test for the restart clause, so the shared builder could lose it silently. Stubbing the describe adapter puts both flags on one read, which pins the clause for `await-ui-element` and `await-screen-idle` at once. Also drops `gesture-pinch` from describe's frame line: which gestures a device takes differs by platform for more than one reason, and the response header already names them per device. --- .../tool-server/src/tools/describe/index.ts | 6 +- .../test/wait-tools-read-caveats.test.ts | 71 +++++++++++++++++++ 2 files changed, 74 insertions(+), 3 deletions(-) create mode 100644 packages/tool-server/test/wait-tools-read-caveats.test.ts diff --git a/packages/tool-server/src/tools/describe/index.ts b/packages/tool-server/src/tools/describe/index.ts index b1db648e4e..c3d15da673 100644 --- a/packages/tool-server/src/tools/describe/index.ts +++ b/packages/tool-server/src/tools/describe/index.ts @@ -173,9 +173,9 @@ with tap coordinates. When no dialog is present, it returns the foreground app's Returns \`{ description, source }\` where \`description\` is a text rendering of the UI tree — one line per element with its role, label/value/id, interactivity flags, and frame. Frame coordinates -are normalized [0,1] fractions of the screen / window width/height (not pixels) — the same space the -gesture tools take. The response header names the ones this device accepts, which on a platform -without multi-touch is fewer than the full set. +are normalized [0,1] fractions of the screen / window width/height (not pixels) — the space every +gesture tool takes. Which gestures a device accepts differs by platform, so the response header names +them for the device you called. To tap an element use the centre of its frame: \`tap_x = frame.x + frame.width / 2\`, \`tap_y = frame.y + frame.height / 2\`. The same formula appears in the response header so it diff --git a/packages/tool-server/test/wait-tools-read-caveats.test.ts b/packages/tool-server/test/wait-tools-read-caveats.test.ts new file mode 100644 index 0000000000..486754e5b8 --- /dev/null +++ b/packages/tool-server/test/wait-tools-read-caveats.test.ts @@ -0,0 +1,71 @@ +import { beforeEach, describe, expect, it, vi } from "vitest"; +import type { DescribeTreeData } from "../src/tools/describe/contract"; +import { __primeDepCacheForTests, __resetDepCacheForTests } from "../src/utils/check-deps"; + +// Both wait tools answer off one `describe` read, so a read that came with a +// diagnosis of its own has to reach the timeout note either way — it is what +// decides the caller's next move. Stubbing the adapter puts both flags on one +// fixture, which makes this a test of the shared clause builder rather than of +// the iOS path that happens to feed it. +const read = vi.hoisted(() => ({ data: null as DescribeTreeData | null })); +vi.mock("../src/tools/describe/platforms/ios", () => ({ + describeIos: async () => read.data, + iosRequires: [], +})); + +const { createAwaitUiElementTool } = await import("../src/tools/await-ui-element"); +const { createAwaitScreenIdleTool } = await import("../src/tools/await-screen-idle"); + +const IOS_UDID = "AAAAAAAA-AAAA-AAAA-AAAA-AAAAAAAAAAAA"; + +/** No service is resolved on this path — the adapter above is the whole read. */ +const registry = { + resolveService: vi.fn(async (urn: string) => { + throw new Error(`unexpected service: ${urn}`); + }), +} as never; + +beforeEach(() => { + __resetDepCacheForTests(); + __primeDepCacheForTests(["xcrun", "adb"]); + read.data = { + tree: { role: "AXApplication", frame: { x: 0, y: 0, width: 1, height: 1 }, children: [] }, + source: "ax-service", + should_restart: true, + hint: "the simulator was not booted through argent", + }; +}); + +describe("a wait that times out on a read with a diagnosis of its own", () => { + it("tells await-ui-element's caller to restart the app, and why the tree was empty", async () => { + const tool = createAwaitUiElementTool(registry); + + const result = await tool.execute( + {}, + { + udid: IOS_UDID, + condition: "visible", + selector: { text: "Submit" }, + timeoutMs: 40, + pollIntervalMs: 10, + } + ); + + expect(result.success).toBe(false); + expect(result.note).toMatch(/restart-app/); + expect(result.note).toMatch(/not booted through argent/); + }); + + it("tells await-screen-idle's caller the same, rather than only that it did not settle", async () => { + const tool = createAwaitScreenIdleTool(registry); + + const result = await tool.execute( + {}, + { udid: IOS_UDID, timeoutMs: 40, pollIntervalMs: 10, minStableMs: 10 } + ); + + expect(result.settled).toBe(false); + expect(result.note).toMatch(/restart-app/); + expect(result.note).toMatch(/not booted through argent/); + }); +}); From e8bf073e04930096f1188a09cb349fde5814eb48 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 21:34:31 +0200 Subject: [PATCH 056/110] fix(harmony): give a describe read only the wait's remaining budget MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A `uitest` client holds the device's queue until it is killed, so a read left on its own 20s ceiling and abandoned at a 300ms deadline is time the caller's NEXT call spends queued — measured at 0.8s per auto-screenshot on a warm emulator, and the client's death frees the queue at once. `describeHarmony` now takes the whole read's budget and splits it across the panel query, the dump and the fetch that follows it; both wait tools hand `pollDescribeTree`'s per-poll remainder down. On a success note, `await-ui-element` reports a `hidden` met on sight AND the caveat of the tree it met it on — a suspended panel is the wait that produces both, and the caveat is what stops the tap that follows from landing nowhere. Also fixes the test file that broke CI on 7e731562: `tsconfig.test.json` rejects top-level `await`, and vitest hoists `vi.mock` above the static imports anyway. --- .../src/tools/await-screen-idle/index.ts | 7 +- .../src/tools/await-ui-element/index.ts | 33 ++++--- .../tool-server/src/tools/describe/index.ts | 2 +- .../tools/describe/platforms/harmony/index.ts | 20 ++++- .../tool-server/src/utils/harmony-uitest.ts | 59 ++++++++++--- .../src/utils/poll-describe-tree.ts | 13 ++- .../tool-server/test/await-harmony.test.ts | 66 +++++++++++++- .../test/harmony-uitest-queue.test.ts | 36 +++++++- .../test/wait-tools-read-caveats.test.ts | 86 ++++++++++++++++++- 9 files changed, 276 insertions(+), 46 deletions(-) diff --git a/packages/tool-server/src/tools/await-screen-idle/index.ts b/packages/tool-server/src/tools/await-screen-idle/index.ts index 3c7b0f26d7..057bd1dc73 100644 --- a/packages/tool-server/src/tools/await-screen-idle/index.ts +++ b/packages/tool-server/src/tools/await-screen-idle/index.ts @@ -135,7 +135,8 @@ export function createAwaitScreenIdleTool(registry: Registry): ToolDefinition, isTvOs: boolean, - androidIsTv: boolean + androidIsTv: boolean, + budgetMs: number ): Promise { if (device.platform === "ios") { return describeIos(registry, device, {}, { isTvOs }); @@ -144,7 +145,7 @@ export function createAwaitScreenIdleTool(registry: Registry): ToolDefinition({ - fetchTree: () => fetchTree(device, services, isTvOs, androidIsTv), + fetchTree: (budgetMs) => fetchTree(device, services, isTvOs, androidIsTv, budgetMs), timeoutMs: params.timeoutMs ?? DEFAULT_TIMEOUT_MS, pollIntervalMs: params.pollIntervalMs ?? DEFAULT_POLL_INTERVAL_MS, signal: ctx?.signal, diff --git a/packages/tool-server/src/tools/await-ui-element/index.ts b/packages/tool-server/src/tools/await-ui-element/index.ts index ce345ce6f9..7405eb1a43 100644 --- a/packages/tool-server/src/tools/await-ui-element/index.ts +++ b/packages/tool-server/src/tools/await-ui-element/index.ts @@ -89,7 +89,7 @@ const zodSchema = z .string() .optional() .describe( - "Optional iOS app bundle id, passed to the describe fallback (see `describe`). Ignored on Android / Chromium." + "Optional iOS app bundle id, passed to the describe fallback (see `describe`). Ignored on every other platform." ), timeoutMs: z .number() @@ -230,7 +230,8 @@ export function createAwaitUiElementTool(registry: Registry): ToolDefinition, isTvOs: boolean, - androidIsTv: boolean + androidIsTv: boolean, + budgetMs: number ): Promise { if (device.platform === "ios") { return describeIos(registry, device, { bundleId: params.bundleId }, { isTvOs }); @@ -242,7 +243,7 @@ export function createAwaitUiElementTool(registry: Registry): ToolDefinition({ - fetchTree: () => fetchTree(device, params, services, isTvOs, androidIsTv), + fetchTree: (budgetMs) => fetchTree(device, params, services, isTvOs, androidIsTv, budgetMs), timeoutMs, pollIntervalMs, signal, @@ -340,17 +341,21 @@ or before tapping an element that appears asynchronously.`, const blind = isBlindRead(data, everMatched); if (!blind && evaluateMatches(params, matches)) { const result: WaitResult = { success: true, elapsed: Date.now() - start }; - if (params.condition === "hidden" && !everMatched) { - result.note = - "condition met immediately — the selector never matched any element, " + - "so it may have already been hidden before the wait, or the selector is wrong"; - } else if (data.hint && READ_CAVEAT_SOURCES.has(data.source)) { + // Both, not the first that applies: a `hidden` that met its + // condition on sight is exactly the wait a suspended panel resolves + // — the selector matches nothing in a frame composited before the + // screen went dark — and the caveat is what stops the tap that + // follows from landing nowhere. + const notes = [ + params.condition === "hidden" && !everMatched + ? "condition met immediately — the selector never matched any element, " + + "so it may have already been hidden before the wait, or the selector is wrong" + : null, // A success read off a tree that may not be the live screen still - // owes the agent that caveat: on HarmonyOS a `visible` resolved - // against the last-composited frame is followed by a tap that - // lands nowhere unless the screen is woken first. - result.note = data.hint; - } + // owes the agent that caveat. + data.hint && READ_CAVEAT_SOURCES.has(data.source) ? data.hint : null, + ].filter((n): n is string => n !== null); + if (notes.length > 0) result.note = notes.join(" "); return { done: true, result }; } return { done: false }; diff --git a/packages/tool-server/src/tools/describe/index.ts b/packages/tool-server/src/tools/describe/index.ts index c3d15da673..33ccaf71a7 100644 --- a/packages/tool-server/src/tools/describe/index.ts +++ b/packages/tool-server/src/tools/describe/index.ts @@ -48,7 +48,7 @@ const zodSchema = z.object({ .describe( "Optional app bundle ID. Used as a target hint on iOS when the AX-service returns no elements " + "and the describe tool falls back to native-devtools inspection. " + - "If omitted, the fallback auto-detects the frontmost connected app. Ignored on Android / Chromium." + "If omitted, the fallback auto-detects the frontmost connected app. Ignored on every other platform." ), }); diff --git a/packages/tool-server/src/tools/describe/platforms/harmony/index.ts b/packages/tool-server/src/tools/describe/platforms/harmony/index.ts index f7600e102a..d07da9784e 100644 --- a/packages/tool-server/src/tools/describe/platforms/harmony/index.ts +++ b/packages/tool-server/src/tools/describe/platforms/harmony/index.ts @@ -31,12 +31,26 @@ const EMPTY_HINT = * The dump is written on the device and copied back, so it needs somewhere to * land on the host; the file is temporary and removed once parsed, since * describe's contract is the rendered text and nothing downstream reads it. + * + * `timeoutMs` is the whole read's budget, split across the panel query and the + * dump rather than given to each. A caller polling to a deadline needs it: the + * `uitest` client this spawns holds the device's queue until it is killed, so a + * read left on its own 20s ceiling and abandoned at a 300ms deadline is 20s the + * caller's NEXT call spends queued — measured at 0.8s of it on a warm emulator, + * on every auto-screenshot that follows an interaction. Killing the client + * frees the queue at once: the on-device `uitest` goes with it (measured — a + * dump run 0.3s after one was killed mid-flight costs the same as a cold one). */ -export async function describeHarmony(connectKey: string): Promise { +export async function describeHarmony( + connectKey: string, + timeoutMs?: number +): Promise { const localPath = join(tmpdir(), `argent-harmony-dump-${process.hrtime.bigint()}.json`); - const display = await harmonyDisplay(connectKey); + const deadline = timeoutMs === undefined ? undefined : Date.now() + timeoutMs; + const left = () => (deadline === undefined ? undefined : Math.max(1, deadline - Date.now())); + const display = await harmonyDisplay(connectKey, left()); try { - const raw = await harmonyDumpLayout(connectKey, localPath); + const raw = await harmonyDumpLayout(connectKey, localPath, left()); const { tree, screen } = parseHarmonyLayout(raw, { width: display.width, height: display.height, diff --git a/packages/tool-server/src/utils/harmony-uitest.ts b/packages/tool-server/src/utils/harmony-uitest.ts index 1867a3c44c..f5a6cdb7f5 100644 --- a/packages/tool-server/src/utils/harmony-uitest.ts +++ b/packages/tool-server/src/utils/harmony-uitest.ts @@ -127,11 +127,14 @@ interface HarmonyDisplay { * fold. The call is a local service dump measured at 50-190ms, cheap enough to * pay per gesture rather than risk that. */ -export async function harmonyDisplay(connectKey: string): Promise { +export async function harmonyDisplay( + connectKey: string, + timeoutMs = UITEST_TIMEOUT_MS +): Promise { const { stdout } = await runHdcShell( connectKey, "hidumper -s RenderService -a screen", - UITEST_TIMEOUT_MS + timeoutMs ); const res = /render resolution=(\d+)x(\d+)/.exec(stdout); if (!res) { @@ -194,7 +197,7 @@ export function assertHarmonyScreenAwake(display: HarmonyDisplay, action: string ); } -type HarmonyTouchCommand = "click" | "doubleClick" | "longClick"; +type HarmonyTouchCommand = "click" | "doubleClick"; export async function harmonyTouch( connectKey: string, @@ -204,7 +207,7 @@ export async function harmonyTouch( await runUitest(connectKey, `uiInput ${command} ${point.x} ${point.y}`); } -type HarmonySwipeCommand = "swipe" | "drag" | "fling"; +type HarmonySwipeCommand = "swipe" | "fling"; /** * `uitest` rejects a velocity outside this range with `Invalid parameters.`, so @@ -261,22 +264,45 @@ async function viaDeviceTmp( connectKey: string, suffix: string, localPath: string, - producer: (remotePath: string) => Promise + producer: (remotePath: string) => Promise, + deadline?: number ): Promise { const remotePath = `${REMOTE_TMP}/argent-${process.pid}-${process.hrtime.bigint()}${suffix}`; try { await producer(remotePath); - await hdcFileRecv(connectKey, remotePath, localPath); + // The fetch is bounded together with the producer rather than left on the + // 30s default: a caller working to a deadline spends it across BOTH round + // trips, and one that has already run out asks for a transfer that fails at + // once instead of one that runs 30s past it. The delete keeps the default — + // it runs outside the queue, and a spent budget is no reason to leave a + // capture behind on the device. + await hdcFileRecv( + connectKey, + remotePath, + localPath, + deadline === undefined ? undefined : Math.max(1, deadline - Date.now()) + ); } finally { await runHdcShell(connectKey, `rm -f ${shellQuote(remotePath)}`).catch(() => {}); } } /** Capture the display to `localPath` as a PNG. */ -export async function harmonyScreenCap(connectKey: string, localPath: string): Promise { - await viaDeviceTmp(connectKey, ".png", localPath, async (remotePath) => { - await runUitest(connectKey, `screenCap -p ${shellQuote(remotePath)}`); - }); +export async function harmonyScreenCap( + connectKey: string, + localPath: string, + timeoutMs = UITEST_TIMEOUT_MS +): Promise { + const deadline = Date.now() + timeoutMs; + await viaDeviceTmp( + connectKey, + ".png", + localPath, + async (remotePath) => { + await runUitest(connectKey, `screenCap -p ${shellQuote(remotePath)}`, timeoutMs); + }, + deadline + ); } /** A node of the tree `uitest dumpLayout` writes. */ @@ -298,9 +324,16 @@ export async function harmonyDumpLayout( localPath: string, timeoutMs = UITEST_TIMEOUT_MS ): Promise { - await viaDeviceTmp(connectKey, ".json", localPath, async (remotePath) => { - await runUitest(connectKey, `dumpLayout -p ${shellQuote(remotePath)}`, timeoutMs); - }); + const deadline = Date.now() + timeoutMs; + await viaDeviceTmp( + connectKey, + ".json", + localPath, + async (remotePath) => { + await runUitest(connectKey, `dumpLayout -p ${shellQuote(remotePath)}`, timeoutMs); + }, + deadline + ); const raw = await readFile(localPath, "utf8"); try { return JSON.parse(raw) as HarmonyLayoutNode; diff --git a/packages/tool-server/src/utils/poll-describe-tree.ts b/packages/tool-server/src/utils/poll-describe-tree.ts index 2fb9245f08..553155c19a 100644 --- a/packages/tool-server/src/utils/poll-describe-tree.ts +++ b/packages/tool-server/src/utils/poll-describe-tree.ts @@ -22,8 +22,15 @@ import { settleWithin, sleepOrAbort } from "./timing"; export type PollVerdict = { done: true; result: R } | { done: false }; export interface PollDescribeTreeArgs { - /** Read the current tree. Called once per poll; must be read-only. */ - fetchTree: () => Promise; + /** + * Read the current tree. Called once per poll; must be read-only. + * + * `budgetMs` is what is left before the wait's deadline. This loop abandons a + * read that overruns it but cannot cancel one, so a backend whose read holds + * a device-side resource has to bound itself with this to keep the abandoned + * read from charging the caller's next call for it. + */ + fetchTree: (budgetMs: number) => Promise; timeoutMs: number; pollIntervalMs: number; signal?: AbortSignal; @@ -95,7 +102,7 @@ export async function pollDescribeTree( // Bound each fetch to the time left before the deadline. const remaining = Math.max(0, deadline - Date.now()); - const settled = await settleWithin(fetchTree(), remaining, signal); + const settled = await settleWithin(fetchTree(remaining), remaining, signal); polls += 1; if (settled.type === "aborted") return outcome(undefined, true); diff --git a/packages/tool-server/test/await-harmony.test.ts b/packages/tool-server/test/await-harmony.test.ts index 3637568ca8..bd3cdff872 100644 --- a/packages/tool-server/test/await-harmony.test.ts +++ b/packages/tool-server/test/await-harmony.test.ts @@ -133,8 +133,12 @@ describe("await-ui-element on HarmonyOS", () => { } ); - expect(harmonyDisplay).toHaveBeenCalledWith(CONNECT_KEY); - expect(harmonyDumpLayout).toHaveBeenCalledWith(CONNECT_KEY, expect.any(String)); + expect(harmonyDisplay).toHaveBeenCalledWith(CONNECT_KEY, expect.any(Number)); + expect(harmonyDumpLayout).toHaveBeenCalledWith( + CONNECT_KEY, + expect.any(String), + expect.any(Number) + ); }); it("`hidden` succeeds once the element leaves a still-populated screen", async () => { @@ -158,6 +162,30 @@ describe("await-ui-element on HarmonyOS", () => { expect(result.note ?? "").not.toMatch(/never matched/i); }); + it("`hidden` met on sight against a suspended panel still says the display is off", async () => { + // The two notes are not alternatives: a screen that went dark holds a frame + // the selector never matched, so this wait meets its condition at once AND + // is the one most likely to be followed by a tap into a dead panel. + vi.mocked(harmonyDisplay).mockResolvedValue({ ...DISPLAY, screenOn: false }); + sequenceDumps([dumpWith("Content")]); + const tool = createAwaitUiElementTool(harmonyRegistry()); + + const result = await tool.execute( + {}, + { + udid: HARMONY_ID, + condition: "hidden", + selector: { text: "Spinner" }, + timeoutMs: 2000, + pollIntervalMs: 10, + } + ); + + expect(result.success).toBe(true); + expect(result.note).toMatch(/never matched/i); + expect(result.note).toMatch(/display is off/i); + }); + // ── Unreadable screen must never confirm `hidden` ───────────────────────── // Same guarantee as the iOS "AX backend is down" and Android "empty dump" // cases, reached differently: harmony's fetcher throws instead of returning a @@ -281,6 +309,32 @@ describe("await-screen-idle on HarmonyOS", () => { expect(result.note).toMatch(/no windows/i); }); + it("gives the read only what is left of the wait, so an abandoned dump frees the queue", async () => { + // The wait abandons a read it cannot finish but cannot cancel one, and a + // `uitest` client holds this device's queue until it is killed. Left on its + // own 20s ceiling, a read abandoned at a 300ms deadline is time the caller's + // NEXT call spends queued — measured at 0.8s of it on a warm emulator, on + // the auto-screenshot that follows every interaction. + const budgets: (number | undefined)[] = []; + vi.mocked(harmonyDumpLayout).mockImplementation(async (_key, _path, timeoutMs) => { + budgets.push(timeoutMs); + return dumpWith("Content"); + }); + const tool = createAwaitScreenIdleTool(harmonyRegistry()); + + await tool.execute( + {}, + { udid: HARMONY_ID, timeoutMs: 500, pollIntervalMs: 10, minStableMs: 10 } + ); + + expect(budgets.length).toBeGreaterThan(0); + for (const budget of budgets) { + expect(budget).toBeGreaterThan(0); + expect(budget).toBeLessThanOrEqual(500); + } + expect(harmonyDisplay).toHaveBeenCalledWith(CONNECT_KEY, expect.any(Number)); + }); + it("does not settle while the dump keeps changing (times out)", async () => { sequenceDumps(Array.from({ length: 30 }, (_, i) => dumpWith(`item-${i}`))); const tool = createAwaitScreenIdleTool(harmonyRegistry()); @@ -302,7 +356,11 @@ describe("await-screen-idle on HarmonyOS", () => { { udid: HARMONY_ID, timeoutMs: 2000, pollIntervalMs: 10, minStableMs: 0 } ); - expect(harmonyDisplay).toHaveBeenCalledWith(CONNECT_KEY); - expect(harmonyDumpLayout).toHaveBeenCalledWith(CONNECT_KEY, expect.any(String)); + expect(harmonyDisplay).toHaveBeenCalledWith(CONNECT_KEY, expect.any(Number)); + expect(harmonyDumpLayout).toHaveBeenCalledWith( + CONNECT_KEY, + expect.any(String), + expect.any(Number) + ); }); }); diff --git a/packages/tool-server/test/harmony-uitest-queue.test.ts b/packages/tool-server/test/harmony-uitest-queue.test.ts index 4cacab11e0..670ec0c95e 100644 --- a/packages/tool-server/test/harmony-uitest-queue.test.ts +++ b/packages/tool-server/test/harmony-uitest-queue.test.ts @@ -1,10 +1,13 @@ import { describe, it, expect, vi, beforeEach } from "vitest"; +import { writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; import { FAILURE_CODES, getFailureSignal } from "@argent/registry"; import { hdcFileRecv as realHdcFileRecv, runHdcShell as realRunHdcShell, } from "../src/utils/harmony-hdc"; -import { harmonyScreenCap } from "../src/utils/harmony-uitest"; +import { harmonyDumpLayout, harmonyScreenCap } from "../src/utils/harmony-uitest"; // Only the transport is faked, so the queue under test is the real one and the // commands it serializes are the ones a device would receive. @@ -175,3 +178,34 @@ describe("the per-device uitest queue", () => { await expect(next).resolves.toBeUndefined(); }); }); + +describe("a caller's own ceiling reaches the device", () => { + // A `timeoutMs` accepted here and dropped leaves every clamp upstream — + // boot-device's probe, the wait tools' polls — computing a bound nothing + // enforces, which is the state this whole parameter exists to leave behind. + // Both round trips count: the fetch is the half that hangs when a device + // starts to go away, and it defaults to 30s, above every caller's budget. + it("bounds the `uitest` call and the fetch that follows it", async () => { + const shellTimeouts: (number | undefined)[] = []; + runHdcShell.mockImplementation(async (_key, command, timeoutMs) => { + if (command.startsWith("uitest ")) shellTimeouts.push(timeoutMs); + return { stdout: "", exitCode: 0 }; + }); + const fetchTimeouts: (number | undefined)[] = []; + hdcFileRecv.mockImplementation(async (_key, _remote, localPath, timeoutMs) => { + fetchTimeouts.push(timeoutMs); + writeFileSync(localPath, "{}"); + }); + + await harmonyDumpLayout( + "dev-a", + join(tmpdir(), `argent-uitest-budget-${process.pid}.json`), + 900 + ); + + expect(shellTimeouts).toEqual([900]); + expect(fetchTimeouts).toHaveLength(1); + expect(fetchTimeouts[0]).toBeGreaterThan(0); + expect(fetchTimeouts[0]).toBeLessThanOrEqual(900); + }); +}); diff --git a/packages/tool-server/test/wait-tools-read-caveats.test.ts b/packages/tool-server/test/wait-tools-read-caveats.test.ts index 486754e5b8..12d6d28b1d 100644 --- a/packages/tool-server/test/wait-tools-read-caveats.test.ts +++ b/packages/tool-server/test/wait-tools-read-caveats.test.ts @@ -7,14 +7,23 @@ import { __primeDepCacheForTests, __resetDepCacheForTests } from "../src/utils/c // decides the caller's next move. Stubbing the adapter puts both flags on one // fixture, which makes this a test of the shared clause builder rather than of // the iOS path that happens to feed it. -const read = vi.hoisted(() => ({ data: null as DescribeTreeData | null })); +const read = vi.hoisted(() => ({ + data: null as DescribeTreeData | null, + /** Set to make every read after the first fail, as a device that goes away does. */ + errorAfterFirst: null as string | null, + calls: 0, +})); vi.mock("../src/tools/describe/platforms/ios", () => ({ - describeIos: async () => read.data, + describeIos: async () => { + read.calls += 1; + if (read.errorAfterFirst !== null && read.calls > 1) throw new Error(read.errorAfterFirst); + return read.data; + }, iosRequires: [], })); -const { createAwaitUiElementTool } = await import("../src/tools/await-ui-element"); -const { createAwaitScreenIdleTool } = await import("../src/tools/await-screen-idle"); +import { createAwaitUiElementTool } from "../src/tools/await-ui-element"; +import { createAwaitScreenIdleTool } from "../src/tools/await-screen-idle"; const IOS_UDID = "AAAAAAAA-AAAA-AAAA-AAAA-AAAAAAAAAAAA"; @@ -28,6 +37,8 @@ const registry = { beforeEach(() => { __resetDepCacheForTests(); __primeDepCacheForTests(["xcrun", "adb"]); + read.calls = 0; + read.errorAfterFirst = null; read.data = { tree: { role: "AXApplication", frame: { x: 0, y: 0, width: 1, height: 1 }, children: [] }, source: "ax-service", @@ -68,4 +79,71 @@ describe("a wait that times out on a read with a diagnosis of its own", () => { expect(result.note).toMatch(/restart-app/); expect(result.note).toMatch(/not booted through argent/); }); + + it("reports the read that failed instead, when the device stopped answering", async () => { + // Opposite diagnoses: "the app needs a restart" sends the caller back to the + // same device, and it is the tree BEFORE the device went away that says so. + read.errorAfterFirst = "device AAAAAAAA-AAAA-AAAA-AAAA-AAAAAAAAAAAA not booted"; + const tool = createAwaitScreenIdleTool(registry); + + const result = await tool.execute( + {}, + { udid: IOS_UDID, timeoutMs: 40, pollIntervalMs: 10, minStableMs: 10 } + ); + + expect(read.calls).toBeGreaterThan(1); + expect(result.note).toMatch(/last tree fetch failed: .*not booted/); + expect(result.note).not.toMatch(/restart-app/); + }); +}); + +describe("a wait that settles", () => { + /** Non-empty, so the first sample can settle it; still by definition. */ + const rendered = { role: "AXButton", frame: { x: 0, y: 0, width: 1, height: 1 }, children: [] }; + + it("carries a hint that questions the tree it settled on into the note", async () => { + read.data = { + tree: { + role: "AXApplication", + frame: { x: 0, y: 0, width: 1, height: 1 }, + children: [rendered], + }, + source: "harmony-uitest", + hint: "The display is off. This tree is what was last composited", + }; + const tool = createAwaitScreenIdleTool(registry); + + const result = await tool.execute( + {}, + { udid: IOS_UDID, timeoutMs: 200, pollIntervalMs: 10, minStableMs: 0 } + ); + + expect(result.settled).toBe(true); + expect(result.note).toMatch(/last composited/); + }); + + it("leaves a standing fact about the target off the note, having settled on a real screen", async () => { + // A settle is a verdict on what the wait saw. iOS' hint ends in an order to + // reboot the simulator and lose the app state — reporting that beside a + // screen that did settle would have the caller undo a passing wait. + read.data = { + tree: { + role: "AXApplication", + frame: { x: 0, y: 0, width: 1, height: 1 }, + children: [rendered], + }, + source: "ax-service", + should_restart: true, + hint: "the simulator was not booted through argent", + }; + const tool = createAwaitScreenIdleTool(registry); + + const result = await tool.execute( + {}, + { udid: IOS_UDID, timeoutMs: 200, pollIntervalMs: 10, minStableMs: 0 } + ); + + expect(result.settled).toBe(true); + expect(result.note).toBeUndefined(); + }); }); From dacdb2c7cbfe632cd642f43394c856ec62f2b2d0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 21:34:39 +0200 Subject: [PATCH 057/110] fix(harmony): refuse a boot target the host does not have MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `boot-device` spent the budget starting an instance that does not exist and answered with the manager's own words, naming nothing the caller could act on. The listing that names the typo is already in hand, so the lookup is made there — the refusal Android makes off `emulator -list-avds`, with the instances the host does have. A listing that could not be read still lets the start be attempted, since it answers nothing. `harmonyConnectKey` refuses an instance id rather than stripping it: `harmony-emulator-` carries the target prefix too, so a slice invents `emulator-`, a key no target holds. The capability gate turns that id away at the HTTP edge, but a flow step reaches `execute` through the registry, which does not gate it. Both waits keep their clamps: the arrival probe is never spawned past the deadline, and the stop-wait gives up on the deadline round rather than opening a listing with nothing left to bound it. --- packages/registry/src/failure-codes.ts | 2 + .../src/tools/devices/boot-device.ts | 73 +++++++-- packages/tool-server/src/utils/device-info.ts | 24 ++- .../test/boot-device-harmony.test.ts | 153 +++++++++++++----- .../tool-server/test/harmony-devices.test.ts | 25 +++ 5 files changed, 223 insertions(+), 54 deletions(-) diff --git a/packages/registry/src/failure-codes.ts b/packages/registry/src/failure-codes.ts index 4f868edea5..8c3afa2e90 100644 --- a/packages/registry/src/failure-codes.ts +++ b/packages/registry/src/failure-codes.ts @@ -65,6 +65,7 @@ export const FAILURE_CODES = { HARMONY_EMULATOR_NOT_FOUND: "HARMONY_EMULATOR_NOT_FOUND", HARMONY_EMULATOR_COMMAND_FAILED: "HARMONY_EMULATOR_COMMAND_FAILED", HARMONY_HDC_COMMAND_FAILED: "HARMONY_HDC_COMMAND_FAILED", + HARMONY_DEVICE_ID_INVALID: "HARMONY_DEVICE_ID_INVALID", HARMONY_DEVICE_UNREACHABLE: "HARMONY_DEVICE_UNREACHABLE", HARMONY_SHELL_NO_STATUS: "HARMONY_SHELL_NO_STATUS", HARMONY_FILE_TRANSFER_FAILED: "HARMONY_FILE_TRANSFER_FAILED", @@ -287,6 +288,7 @@ export const FAILURE_CODES = { BOOT_ANDROID_FIRST_FRAME_TIMEOUT: "BOOT_ANDROID_FIRST_FRAME_TIMEOUT", BOOT_ANDROID_PACKAGE_MANAGER_UNAVAILABLE: "BOOT_ANDROID_PACKAGE_MANAGER_UNAVAILABLE", BOOT_ANDROID_COLD_BOOT_FAILED: "BOOT_ANDROID_COLD_BOOT_FAILED", + BOOT_HARMONY_INSTANCE_NOT_FOUND: "BOOT_HARMONY_INSTANCE_NOT_FOUND", BOOT_HARMONY_TARGET_LIST_FAILED: "BOOT_HARMONY_TARGET_LIST_FAILED", BOOT_HARMONY_STOP_FAILED: "BOOT_HARMONY_STOP_FAILED", BOOT_HARMONY_STOP_TIMEOUT: "BOOT_HARMONY_STOP_TIMEOUT", diff --git a/packages/tool-server/src/tools/devices/boot-device.ts b/packages/tool-server/src/tools/devices/boot-device.ts index fbae435d6c..8e54bed579 100644 --- a/packages/tool-server/src/tools/devices/boot-device.ts +++ b/packages/tool-server/src/tools/devices/boot-device.ts @@ -54,6 +54,7 @@ import { startVvd, stopVvd, isVvdRunning, waitForVvdRunning } from "../../utils/ import { resolveRunningVvdSerial, listVegaDevices } from "../../utils/vega-devices"; import { EMULATOR_NOT_FOUND, + EMULATOR_TIMEOUT_MS, emulatorFailure, isChinaOnlyRestriction, resolveHarmonyEmulator, @@ -1518,7 +1519,7 @@ const HARMONY_IMAGE_RESTRICTION = "the image; an instance has to be created in DevEco Studio on a host that can download one."; /** - * Said when the start failed and the manager lists no instances at all. Zero + * Said in place of the available-instance list when there is none to give. Zero * instances is equally what a host that simply has not created one yet looks * like — including inside mainland China, where the download works — so this * states what was observed and offers the restriction as a possible cause @@ -1662,13 +1663,24 @@ async function connectedHarmonyTargets(timeoutMs?: number) { * `force` path says so too, and staying quiet here would have the same failure * read as "nothing happened" depending only on which call failed. */ -async function connectedHarmonyKeys(stoppedInstance: string | null): Promise> { +async function connectedHarmonyKeys( + stoppedInstance: string | null, + deadline: number +): Promise> { for (let attempt = 0; ; attempt++) { + // The snapshot is spent from the same budget as everything after it. Left + // on its own ceilings it runs 18s past the deadline on the `force` path — + // a shutdown is given "the whole remaining budget" by design, so arriving + // here with none left is the ordinary case, not the odd one — and the wait + // it feeds then returns on its first line. A budget too small to trace the + // key is refused here rather than overspent: the message below is the one + // that says the instance is down and the restart has to be re-run. + const budget = Math.max(1, deadline - Date.now()); try { - const targets = await listHarmonyHdcTargetsStrict(); + const targets = await listHarmonyHdcTargetsStrict(Math.min(HDC_LIST_TIMEOUT_MS, budget)); return new Set(targets.filter((t) => t.state === "Connected").map((t) => t.connectKey)); } catch (err) { - if (attempt > 0) { + if (attempt > 0 || Date.now() >= deadline) { throw new FailureError( "Could not read `hdc`'s device table while preparing to start the HarmonyOS instance, " + "so the target it registers under could not have been told apart from those already " + @@ -1688,7 +1700,9 @@ async function connectedHarmonyKeys(stoppedInstance: string | null): Promise setTimeout(r, HARMONY_TARGET_POLL_MS)); + await new Promise((r) => + setTimeout(r, Math.min(HARMONY_TARGET_POLL_MS, Math.max(0, deadline - Date.now()))) + ); } } } @@ -2077,7 +2091,30 @@ async function bootHarmonyImpl(params: { // start fails — whether the host has any at all. `null` is a listing that // itself failed and answers neither, so it neither blocks a start nor lets // one be blamed on a host with no instances. - const instances = await listHarmonyInstances().catch(() => null); + const instances = await listHarmonyInstances( + Math.min(HARMONY_LIST_TIMEOUT_MS, Math.max(1, bootDeadline - Date.now())) + ).catch(() => null); + // A name the manager does not know can only fail, and the listing that names + // the caller's typo is already in hand — the same refusal Android makes off + // `emulator -list-avds`. Left to `-start`, the boot instead spends the budget + // and answers with the manager's own words, which name nothing that exists. + // Only a listing that was read can refuse; a failed one answers neither + // question (above), so it still lets the start be attempted. + if (instances && !instances.some((i) => i.name === params.instanceName)) { + throw new FailureError( + `HarmonyOS emulator instance "${params.instanceName}" not found. ` + + (instances.length === 0 + ? HARMONY_NO_INSTANCES + : `Available: ${instances.map((i) => i.name).join(", ")}.`), + { + error_code: FAILURE_CODES.BOOT_HARMONY_INSTANCE_NOT_FOUND, + failure_stage: "boot_harmony_instance_lookup", + failure_area: "tool_server", + error_kind: "not_found", + } + ); + } + const alreadyRunning = instances?.some((i) => i.name === params.instanceName && i.running) ?? false; @@ -2091,7 +2128,14 @@ async function bootHarmonyImpl(params: { }; } if (alreadyRunning) { - const stopped = await runHarmonyEmulator(["-stop", params.instanceName]); + // Bounded by the budget like every wait after it: `-stop` returns as soon as + // it has asked, but its own 30s ceiling is the whole of a minimum budget, + // and a manager slow to answer would leave nothing for the shutdown this + // call still has to wait out. + const stopped = await runHarmonyEmulator( + ["-stop", params.instanceName], + Math.min(EMULATOR_TIMEOUT_MS, Math.max(1, bootDeadline - Date.now())) + ); const stopDiagnostic = emulatorFailure(stopped); if (stopDiagnostic) { throw new FailureError( @@ -2140,7 +2184,7 @@ async function bootHarmonyImpl(params: { // `alreadyRunning` can only still be set on the `force` path — the other // return above — so it is exactly "this call has stopped the instance". const before = hdcAvailable - ? await connectedHarmonyKeys(alreadyRunning ? params.instanceName : null) + ? await connectedHarmonyKeys(alreadyRunning ? params.instanceName : null, bootDeadline) : new Set(); // The panel this instance is configured with, which the guest reports back as // its `render resolution` — the only thing that separates the instance @@ -2180,14 +2224,11 @@ async function bootHarmonyImpl(params: { } ); } - const imageMissing = - isChinaOnlyRestriction(diagnostic) || diagnostic.includes("Cannot find image"); - // Only the manager's own words justify blaming the region; an empty - // instance list is merely consistent with it. - const cause = imageMissing - ? ` ${HARMONY_IMAGE_RESTRICTION}` - : instances?.length === 0 - ? ` ${HARMONY_NO_INSTANCES}` + // Only the manager's own words justify blaming the region. A host with no + // instances at all is answered before the start, by the lookup above. + const cause = + isChinaOnlyRestriction(diagnostic) || diagnostic.includes("Cannot find image") + ? ` ${HARMONY_IMAGE_RESTRICTION}` : ""; throw new FailureError( cause diff --git a/packages/tool-server/src/utils/device-info.ts b/packages/tool-server/src/utils/device-info.ts index 14f06a973e..836480f5d2 100644 --- a/packages/tool-server/src/utils/device-info.ts +++ b/packages/tool-server/src/utils/device-info.ts @@ -1,3 +1,4 @@ +import { FAILURE_CODES, FailureError } from "@argent/registry"; import type { DeviceInfo, DeviceKind, Platform } from "@argent/registry"; /** @@ -93,8 +94,29 @@ export function harmonyInstanceName(udid: string): string { : udid; } -/** The `hdc` connect key behind a `harmony-` id. */ +/** + * The `hdc` connect key behind a `harmony-` id. + * + * An instance id is refused rather than stripped: `harmony-emulator-` + * carries the `harmony-` prefix too, so slicing it yields `emulator-` — a + * key no target holds, which `hdc` answers with `[Fail]Not match target` naming + * a device the caller never asked for. The capability gate turns an instance id + * away at the HTTP edge, but a flow step reaches `execute` through the registry, + * which does not gate it. + */ export function harmonyConnectKey(udid: string): string { + if (udid.startsWith(HARMONY_EMULATOR_ID_PREFIX)) { + throw new FailureError( + `"${udid}" names a HarmonyOS emulator instance, not a device to drive. Start it with ` + + "boot-device and drive the `harmony-` id that returns.", + { + error_code: FAILURE_CODES.HARMONY_DEVICE_ID_INVALID, + failure_stage: "harmony_connect_key", + failure_area: "tool_server", + error_kind: "validation", + } + ); + } return udid.startsWith(HARMONY_ID_PREFIX) ? udid.slice(HARMONY_ID_PREFIX.length) : udid; } diff --git a/packages/tool-server/test/boot-device-harmony.test.ts b/packages/tool-server/test/boot-device-harmony.test.ts index 9a3279e831..e78637ba57 100644 --- a/packages/tool-server/test/boot-device-harmony.test.ts +++ b/packages/tool-server/test/boot-device-harmony.test.ts @@ -824,42 +824,61 @@ describe("boot-device — HarmonyOS emulator path", () => { await settled; }); - it("points at creating an instance when the start failed and none exist", async () => { + it("points at creating an instance when the host has none", async () => { listHarmonyInstances.mockResolvedValue([]); targets([PHONE]); - vi.useFakeTimers(); - const pending = boot({ bootTimeoutMs: 900_000 }); - const settled = expect(pending).rejects.toThrow(/create one in DevEco Studio/); - await vi.advanceTimersByTimeAsync(1_000); - child.die(`"${INSTANCE}" is not found. Please create the device(folder): /x`, 1); - await vi.advanceTimersByTimeAsync(3_000); - await settled; + const failed = await boot({ bootTimeoutMs: 900_000 }).catch((e: unknown) => e); - expect(getFailureSignal(await pending.catch((e: unknown) => e))).toMatchObject({ - error_code: FAILURE_CODES.BOOT_HARMONY_START_FAILED, - failure_command: "deveco_emulator", + expect((failed as Error).message).toMatch(/create one in DevEco Studio/); + expect(getFailureSignal(failed)).toMatchObject({ + error_code: FAILURE_CODES.BOOT_HARMONY_INSTANCE_NOT_FOUND, }); + expect(spawnMock).not.toHaveBeenCalled(); + }); + + it("names the instances the host does have, rather than starting one it does not", async () => { + // The name is the caller's to spell and the listing is already in hand, so + // a typo is answered here — not by spending the budget on a start whose + // failure quotes the manager naming nothing that exists. + listHarmonyInstances.mockResolvedValue([ + { name: "Tablet_9", deviceType: "Tablet", osVersion: null, running: false }, + { name: INSTANCE, deviceType: "Phone", osVersion: null, running: false }, + ]); + targets([PHONE]); + + const failed = await boot({ harmonyInstance: "Phone1", bootTimeoutMs: 900_000 }).catch( + (e: unknown) => e + ); + + expect((failed as Error).message).toMatch( + /"Phone1" not found\. Available: Tablet_9, Phone_1\./ + ); + expect(spawnMock).not.toHaveBeenCalled(); }); - it("does not claim the host has no instances when the listing itself failed", async () => { + it("still starts the instance when the listing itself could not be read", async () => { // An unreadable list answers nothing, so it must not be read as "none" — - // that would send the caller off creating an instance they already have. + // that would refuse a boot of an instance the host has. listHarmonyInstances.mockRejectedValue(new Error("Emulator: spawn EACCES")); targets([PHONE]); vi.useFakeTimers(); const pending = boot({ bootTimeoutMs: 900_000 }); - // A negative LOOKAHEAD anchored on `is not found…` would not catch the - // no-instances sentence prepended BEFORE the anchor, so this asserts the - // absence directly. const settled = expect(pending).rejects.toThrow(/is not found\. Please create the device/); await vi.advanceTimersByTimeAsync(1_000); + expect(spawnMock).toHaveBeenCalledWith( + expect.any(String), + ["-start", INSTANCE], + expect.any(Object) + ); child.die(`"${INSTANCE}" is not found. Please create the device(folder): /x`, 1); await vi.advanceTimersByTimeAsync(3_000); await settled; - const msg = await pending.catch((e: unknown) => (e as Error).message); - expect(msg).not.toContain("create one in DevEco Studio"); + expect(getFailureSignal(await pending.catch((e: unknown) => e))).toMatchObject({ + error_code: FAILURE_CODES.BOOT_HARMONY_START_FAILED, + failure_command: "deveco_emulator", + }); }); it("leaves a running instance alone and says how to reach it", async () => { @@ -905,7 +924,7 @@ describe("boot-device — HarmonyOS emulator path", () => { await vi.advanceTimersByTimeAsync(60_000); const result = (await pending) as { udid: string; note?: string }; - expect(runHarmonyEmulator).toHaveBeenCalledWith(["-stop", INSTANCE]); + expect(runHarmonyEmulator).toHaveBeenCalledWith(["-stop", INSTANCE], expect.any(Number)); expect(spawnMock).toHaveBeenCalledWith( expect.any(String), ["-start", INSTANCE], @@ -982,7 +1001,7 @@ describe("boot-device — HarmonyOS emulator path", () => { await vi.advanceTimersByTimeAsync(61_000); await settled; - expect(runHarmonyEmulator).toHaveBeenCalledWith(["-stop", INSTANCE]); + expect(runHarmonyEmulator).toHaveBeenCalledWith(["-stop", INSTANCE], expect.any(Number)); expect(spawnMock).not.toHaveBeenCalled(); expect(getFailureSignal(await pending.catch((e: unknown) => e))).toMatchObject({ error_code: FAILURE_CODES.BOOT_HARMONY_STOP_TIMEOUT, @@ -1000,7 +1019,7 @@ describe("boot-device — HarmonyOS emulator path", () => { return new Promise((resolve) => setTimeout( () => resolve([{ name: INSTANCE, deviceType: "Phone", osVersion: null, running: true }]), - 4_000 + 3_500 ) ); }); @@ -1020,6 +1039,62 @@ describe("boot-device — HarmonyOS emulator path", () => { expect(listing.timeoutMs).toBeGreaterThan(0); expect(listing.at + listing.timeoutMs!).toBeLessThanOrEqual(startedAt + 30_000); } + // 3.5s per listing and a 2s poll between them leave the last round 1s of + // budget, so its sleep is clamped and the loop comes back around exactly on + // the deadline — the round that has to give up rather than open a listing + // with nothing left to bound it. + expect(waitRounds.at(-1)!.at).toBeLessThan(startedAt + 30_000); + }); + + it("pays for a `force` restart's stop and snapshot out of the budget, not on top of it", async () => { + // The shutdown is given the whole remaining budget by design, so reaching + // the pre-start snapshot with almost none left is the ordinary `force` case + // rather than the odd one. Left on their own ceilings from there, `-stop` + // (30s) and the snapshot listing (8s, and again after a retry) answer a + // third of a minute past the budget the caller set — and the wait they feed + // then returns on its first line, so the overspend buys nothing. + const calls: { at: number; timeoutMs?: number }[] = []; + runHarmonyEmulator.mockImplementation((args: string[], timeoutMs?: number) => { + if (args[0] === "-stop") calls.push({ at: Date.now(), timeoutMs }); + return Promise.resolve({ stdout: "", stderr: "" }); + }); + listHarmonyHdcTargetsStrict.mockImplementation((timeoutMs?: number) => { + calls.push({ at: Date.now(), timeoutMs }); + return new Promise((resolve) => setTimeout(() => resolve([PHONE]), 4_000)); + }); + // Four seconds per listing and a 2s poll between them puts the instance's + // shutdown at 26s of a 30s budget. + let listed = 0; + listHarmonyInstances.mockImplementation( + () => + new Promise((resolve) => + setTimeout( + () => + resolve([ + { + name: INSTANCE, + deviceType: "Phone", + osVersion: null, + running: listed++ < 4, + display: PANEL, + }, + ]), + 4_000 + ) + ) + ); + vi.useFakeTimers(); + + const startedAt = Date.now(); + const pending = boot({ force: true, bootTimeoutMs: 30_000 }); + await vi.advanceTimersByTimeAsync(90_000); + await pending; + + expect(calls).toHaveLength(2); + for (const call of calls) { + expect(call.timeoutMs).toBeGreaterThan(0); + expect(call.at + call.timeoutMs!).toBeLessThanOrEqual(startedAt + 30_000); + } }); it("says the instance is down when a `force` restart refuses at the snapshot", async () => { @@ -1050,7 +1125,7 @@ describe("boot-device — HarmonyOS emulator path", () => { await vi.advanceTimersByTimeAsync(31_000); await settled; - expect(runHarmonyEmulator).toHaveBeenCalledWith(["-stop", INSTANCE]); + expect(runHarmonyEmulator).toHaveBeenCalledWith(["-stop", INSTANCE], expect.any(Number)); expect(spawnMock).not.toHaveBeenCalled(); // `hdc`, not the manager: this refusal is the device table being unreadable, // and the binary named is what the caller has to go and check. @@ -1066,9 +1141,11 @@ describe("boot-device — HarmonyOS emulator path", () => { // rows reconnected — 95s against a 30s budget, measured. Three arrivals at // 20s apiece is that shape: one round already outlasts two budgets. const slowPanel = { width: 1, height: 1, screenOn: true }; - harmonyDisplay.mockImplementation( - () => new Promise((resolve) => setTimeout(() => resolve(slowPanel), 20_000)) - ); + const probes: number[] = []; + harmonyDisplay.mockImplementation(() => { + probes.push(Date.now()); + return new Promise((resolve) => setTimeout(() => resolve(slowPanel), 20_000)); + }); const peers = [1, 2, 3].map((i) => ({ connectKey: `127.0.0.1:555${i}`, connection: "TCP", @@ -1087,6 +1164,12 @@ describe("boot-device — HarmonyOS emulator path", () => { // the failure, however few calls it made getting there. expect(await elapsed).toBeLessThanOrEqual(30_000); expect(((await pending) as { note?: string }).note).toMatch(/registered/); + // Abandoning the wait is not enough: the first probe eats the whole budget, + // so the two arrivals behind it must never be spawned at all. An `hdc` + // client started at the deadline is one nothing will ever read, still + // holding the guest for its own 20s ceiling. + expect(probes.length).toBeGreaterThan(0); + for (const at of probes) expect(at).toBeLessThan(startedAt + 30_000); }); // Every one of these lookups is `i.name === params.instanceName` against a @@ -1279,7 +1362,7 @@ describe("boot-device — HarmonyOS emulator path", () => { await vi.advanceTimersByTimeAsync(60_000); const [plain, forced] = (await pending) as { udid: string; note?: string }[]; - expect(runHarmonyEmulator).toHaveBeenCalledWith(["-stop", INSTANCE]); + expect(runHarmonyEmulator).toHaveBeenCalledWith(["-stop", INSTANCE], expect.any(Number)); expect(plain.udid).toBe(`harmony-emulator-${INSTANCE}`); expect(plain.note).toMatch(/already running/); expect(forced.udid).toBe(`harmony-${EMULATOR_KEY}`); @@ -1302,19 +1385,15 @@ describe("boot-device — HarmonyOS emulator path", () => { }); it("does not read a connect-key id as an instance name", async () => { - vi.useFakeTimers(); - - const pending = boot({ + // A device id passed where a boot target belongs is refused under the + // spelling it was given — nothing strips the prefix and starts whatever is + // left of it, which would boot an instance the caller never named. + const failed = await boot({ harmonyInstance: `harmony-${PHONE.connectKey}`, bootTimeoutMs: 30_000, - }); - await vi.advanceTimersByTimeAsync(31_000); - await pending; + }).catch((e: unknown) => e); - expect(spawnMock).toHaveBeenCalledWith( - expect.any(String), - ["-start", `harmony-${PHONE.connectKey}`], - expect.any(Object) - ); + expect((failed as Error).message).toContain(`"harmony-${PHONE.connectKey}" not found`); + expect(spawnMock).not.toHaveBeenCalled(); }); }); diff --git a/packages/tool-server/test/harmony-devices.test.ts b/packages/tool-server/test/harmony-devices.test.ts index 48768d9019..568c6e4638 100644 --- a/packages/tool-server/test/harmony-devices.test.ts +++ b/packages/tool-server/test/harmony-devices.test.ts @@ -240,6 +240,17 @@ describe("HarmonyOS device ids", () => { expect(harmonyInstanceName("Phone_1")).toBe("Phone_1"); expect(harmonyConnectKey("025DEK236V035771")).toBe("025DEK236V035771"); }); + + it("refuses an instance id rather than driving the key that stripping it invents", () => { + // `harmony-emulator-Phone_1` carries the target prefix too, so a slice + // yields `emulator-Phone_1` — a key no target holds. The capability gate + // stops an instance id at the HTTP edge, but a flow step goes through the + // registry, which does not gate it, and `hdc` would answer for a device the + // caller never named. + expect(() => harmonyConnectKey(harmonyEmulatorId("Phone_1"))).toThrow( + /names a HarmonyOS emulator instance/ + ); + }); }); // Both wrappers default to 30s, which is itself above `list-devices`' @@ -265,4 +276,18 @@ describe("discovery calls are bounded by the constants list-devices was sized ag HDC_LIST_TIMEOUT_MS ); }); + + // The caller's ceiling has to reach the process, not just the argument list. + // A `timeoutMs` accepted and dropped here leaves `boot-device`'s clamp + // computing a bound nothing enforces — and the callers that pass one are + // polling to a deadline, so the ceiling they hand over is the whole point. + it("hands a caller's own ceiling to `Emulator`, not just the default", async () => { + await listHarmonyInstances(1_500); + expect(vi.mocked(realRunHarmonyEmulator)).toHaveBeenCalledWith(["-list", "-details"], 1_500); + }); + + it("hands a caller's own ceiling to `hdc`, not just the default", async () => { + await listHarmonyHdcTargets(1_500); + expect(vi.mocked(realRunHdc)).toHaveBeenCalledWith(["list", "targets", "-v"], 1_500); + }); }); From fda201cf12fddabb2c1de0c998f5aab87969ad6b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 21:34:46 +0200 Subject: [PATCH 058/110] fix(harmony): classify a killed CLI as a timeout, take the keys the other backends take MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `hdc` and the `Emulator` manager reported every rejection as `subprocess`, so a client killed at its ceiling was counted as a command that ran and failed — and neither carried the exit code, signal or spawn code the `adb` and `vega` wrappers attach. Both now read the kind off the child and go through `subprocessFailureMetadata`. `return` and `delete` are the names iOS and Android take for two keys HarmonyOS already presses, so a step written against either platform no longer stops here. `longClick` and `drag` leave `uitest`'s command unions: nothing produces them. Prose: the skill claimed every tool in its workflow applies on HarmonyOS (five do not), `bundleId`'s "Ignored on Android / Chromium" predates two platforms, `open-url`'s note is unconditional there, and run-sequence's table now qualifies `keyboard` the way it already qualifies `button`. --- .../skills/argent-device-interact/SKILL.md | 2 +- .../skills/argent-test-ui-flow/SKILL.md | 22 ++++++++-------- .../tool-server/src/tools/keyboard/index.ts | 2 +- .../src/tools/keyboard/platforms/harmony.ts | 9 +++++++ .../tool-server/src/tools/open-url/types.ts | 9 ++++--- .../src/tools/run-sequence/index.ts | 2 +- packages/tool-server/src/utils/harmony-cli.ts | 26 ++++++++++++++----- packages/tool-server/src/utils/harmony-hdc.ts | 17 +++++++++--- .../test/harmony-binary-resolve.test.ts | 7 +++-- .../test/harmony-hdc-timeout.test.ts | 10 ++++++- .../test/keyboard-key-order.test.ts | 4 +++ 11 files changed, 79 insertions(+), 31 deletions(-) diff --git a/packages/skills/skills/argent-device-interact/SKILL.md b/packages/skills/skills/argent-device-interact/SKILL.md index fb515b3dff..6fca303c87 100644 --- a/packages/skills/skills/argent-device-interact/SKILL.md +++ b/packages/skills/skills/argent-device-interact/SKILL.md @@ -170,7 +170,7 @@ Values: `home`, `back`, `power`, `volumeUp`, `volumeDown`, `appSwitch`, `actionB { "udid": "", "text": "search query", "key": "enter" } ``` -Special keys: `enter`, `escape`, `backspace`, `tab`, `space`, `arrow-up`, `arrow-down`, `arrow-left`, `arrow-right`, `f1`–`f12` — on HarmonyOS only `enter`, `backspace`, `space`, `arrow-left` and `arrow-right` exist; any other key is refused with 400 KEYBOARD_KEY_UNSUPPORTED. Optional: `"delayMs": 100` between keystrokes (default 50ms) — applies to the iOS simulator and Chromium; it is ignored on Android phones/tablets (typed via `adb input text`, no per-key cadence), on HarmonyOS (`uitest uiInput text` types in one shot), on Vega, and on TV targets. +Special keys: `enter`, `escape`, `backspace`, `tab`, `space`, `arrow-up`, `arrow-down`, `arrow-left`, `arrow-right`, `f1`–`f12` — on HarmonyOS only `enter` (`return`), `backspace` (`delete`), `space`, `arrow-left` and `arrow-right` exist; any other key is refused with 400 KEYBOARD_KEY_UNSUPPORTED. Optional: `"delayMs": 100` between keystrokes (default 50ms) — applies to the iOS simulator and Chromium; it is ignored on Android phones/tablets (typed via `adb input text`, no per-key cadence), on HarmonyOS (`uitest uiInput text` types in one shot), on Vega, and on TV targets. **Typing secrets.** To enter a credential without its plaintext ever entering your context, transcript, or logs, use a secret placeholder in `text` (works in `keyboard`, `paste`, `run-sequence` keyboard steps, and flow `type` steps): diff --git a/packages/skills/skills/argent-test-ui-flow/SKILL.md b/packages/skills/skills/argent-test-ui-flow/SKILL.md index 4b8bbac2e6..43b4a240d7 100644 --- a/packages/skills/skills/argent-test-ui-flow/SKILL.md +++ b/packages/skills/skills/argent-test-ui-flow/SKILL.md @@ -5,7 +5,7 @@ description: Autonomously test an app UI (iOS, Android, or HarmonyOS) by running ## Platform-agnostic -The interaction tool names are identical on iOS, Android and HarmonyOS — `gesture-tap`, `gesture-swipe`, `describe`, `screenshot`, `launch-app`, etc. — and the tool-server auto-dispatches based on the `udid` you pass (UUID-shape → iOS, `harmony-` → HarmonyOS, adb serial → Android). HarmonyOS takes one contact per call, so `gesture-pinch`, `gesture-rotate`, `gesture-custom` and `rotate` are refused there; every other tool in this workflow applies. +The interaction tool names are identical on iOS, Android and HarmonyOS — `gesture-tap`, `gesture-swipe`, `describe`, `screenshot`, `launch-app`, etc. — and the tool-server auto-dispatches based on the `udid` you pass (UUID-shape → iOS, `harmony-` → HarmonyOS, adb serial → Android). HarmonyOS drives the tap / swipe / type / read loop; everything outside it is refused there — `gesture-pinch`, `gesture-rotate` and `gesture-custom` (one contact per call, so no pinch, no rotate, and no long-press), `gesture-drag`, `gesture-scroll`, `rotate`, and the JS / native inspectors `debugger-component-tree`, `debugger-evaluate`, `debugger-log-registry`, `native-describe-screen` and `view-network-logs`. **Before testing, resolve which device to test on.** Call `list-devices` and follow ``: prefer a running device on any platform; @@ -114,19 +114,19 @@ Steps: ## Tips - **Wait on the UI, don't poll.** When a step needs the screen to change first, gate it with `await-ui-element` (block until an element is `visible`/`hidden` or contains `text`) rather than repeated `screenshot` calls with fixed sleeps. See the `await-ui-element` section of `argent-device-interact`. -- **Use `gesture-custom` for long-press** context menus (800ms hold). +- **Use `gesture-custom` for long-press** context menus (800ms hold). Not on HarmonyOS — it has no long-press tool. - **Report clearly**: state what you expected, what you saw, and the verdict. - **Permission modals**: try `describe` first. Use `screenshot` only as fallback, tap one visible button at a time, and verify with the returned screenshot before continuing. - **Record for replay**: If a tested flow is likely to be repeated, use the `argent-create-flow` skill to record it as a `.yaml` script. This lets you replay the entire sequence later with a single `flow-execute` call instead of re-running each step manually. ## Related Skills -| Skill | When to use | -| ---------------------------------- | -------------------------------------------------------- | -| `argent-device-interact` | Tool usage for tapping, swiping, typing (every platform) | -| `argent-screenshot-diff` | Visual regression and before/after screenshot comparison | -| `argent-ios-simulator-setup` | Booting and connecting an iOS simulator | -| `argent-android-emulator-setup` | Booting and connecting an Android emulator | -| `argent-react-native-app-workflow` | Starting the app, Metro, build issues | -| `argent-metro-debugger` | Breakpoints, console logs, JS evaluation | -| `argent-create-flow` | Record a test sequence as a replayable flow | +| Skill | When to use | +| ---------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------- | +| `argent-device-interact` | Tool usage for tapping, swiping, typing (iOS, Android, HarmonyOS, Chromium — TV targets are focus-driven, see `argent-tv-interact`) | +| `argent-screenshot-diff` | Visual regression and before/after screenshot comparison | +| `argent-ios-simulator-setup` | Booting and connecting an iOS simulator | +| `argent-android-emulator-setup` | Booting and connecting an Android emulator | +| `argent-react-native-app-workflow` | Starting the app, Metro, build issues | +| `argent-metro-debugger` | Breakpoints, console logs, JS evaluation | +| `argent-create-flow` | Record a test sequence as a replayable flow | diff --git a/packages/tool-server/src/tools/keyboard/index.ts b/packages/tool-server/src/tools/keyboard/index.ts index fe77fac74c..4bd99eb1dc 100644 --- a/packages/tool-server/src/tools/keyboard/index.ts +++ b/packages/tool-server/src/tools/keyboard/index.ts @@ -33,7 +33,7 @@ const zodSchema = z.object({ .string() .optional() .describe( - "Named key to press: enter, escape, backspace, tab, space, arrow-up, arrow-down, arrow-left, arrow-right, f1–f12. On HarmonyOS only enter, backspace, space, arrow-left and arrow-right exist — any other key fails with 400 KEYBOARD_KEY_UNSUPPORTED. When combined with `text`, the key is pressed AFTER the text is typed (so text + enter types and submits). Not supported on TV targets — move focus with `tv-remote` (up/down/left/right) instead." + "Named key to press: enter, escape, backspace, tab, space, arrow-up, arrow-down, arrow-left, arrow-right, f1–f12. On HarmonyOS only enter (return), backspace (delete), space, arrow-left and arrow-right exist — any other key fails with 400 KEYBOARD_KEY_UNSUPPORTED. When combined with `text`, the key is pressed AFTER the text is typed (so text + enter types and submits). Not supported on TV targets — move focus with `tv-remote` (up/down/left/right) instead." ), delayMs: z .number() diff --git a/packages/tool-server/src/tools/keyboard/platforms/harmony.ts b/packages/tool-server/src/tools/keyboard/platforms/harmony.ts index 9b11ade9d5..e8d154ca46 100644 --- a/packages/tool-server/src/tools/keyboard/platforms/harmony.ts +++ b/packages/tool-server/src/tools/keyboard/platforms/harmony.ts @@ -57,6 +57,13 @@ function assertTypeableHarmonyText(text: string): void { * `abX Y`, which pins both directions; `enter` submitted the search and * replaced the screen. * + * `return` and `delete` are the same two presses under the names the other + * backends also take (`NAMED_KEYS`, `ANDROID_NAMED_KEYCODES`): a named key means + * the same thing on every platform, so a step written against iOS or Android + * must not be refused here for spelling `enter` the other way. `delete` aliases + * backspace rather than forward-delete, for the reason spelled out beside the + * Android table. + * * `tab`, `escape` and the vertical arrows are deliberately absent. `uitest` * accepts their documented keycodes and reports `No Error`, but nothing * observable happened in a text field, so there is no evidence they do the @@ -66,7 +73,9 @@ function assertTypeableHarmonyText(text: string): void { */ const HARMONY_KEYCODES: Record = { "enter": 2054, + "return": 2054, "backspace": 2055, + "delete": 2055, "space": 2050, "arrow-left": 2014, "arrow-right": 2015, diff --git a/packages/tool-server/src/tools/open-url/types.ts b/packages/tool-server/src/tools/open-url/types.ts index 6922b20434..f072f1a204 100644 --- a/packages/tool-server/src/tools/open-url/types.ts +++ b/packages/tool-server/src/tools/open-url/types.ts @@ -7,9 +7,12 @@ export interface OpenUrlResult { opened: boolean; url: string; /** - * Present when the URL was a web URL (http/https) opened on a native device: - * a caveat that it may have opened in the browser rather than deep-linked into - * a native app. Absent for custom schemes and for Chromium navigations. + * What `opened: true` leaves unsaid. On iOS and Android it is the web-URL + * caveat — an http/https link may have opened in the browser rather than + * deep-linked into a native app — and a custom scheme carries none. On + * HarmonyOS every URL carries one, since `aa start -U` reports success for any + * URI the system accepts (see that platform's impl). Absent for Chromium + * navigations, which land on the page they were given. */ note?: string; } diff --git a/packages/tool-server/src/tools/run-sequence/index.ts b/packages/tool-server/src/tools/run-sequence/index.ts index df7674cb48..90e724d2ed 100644 --- a/packages/tool-server/src/tools/run-sequence/index.ts +++ b/packages/tool-server/src/tools/run-sequence/index.ts @@ -113,7 +113,7 @@ Allowed tools and their args (udid is auto-injected, do NOT include it in args): gesture-pinch: { centerX: number, centerY: number, startDistance: number, endDistance: number, endCenterX?: number, endCenterY?: number, angle?: number, durationMs?: number } [ios/android] gesture-rotate: { centerX: number, centerY: number, radius?: number, radiusX?: number, radiusY?: number, startAngle: number, endAngle: number, durationMs?: number } [ios/android] button: { button: "home"|"back"|"power"|"volumeUp"|"volumeDown"|"appSwitch"|"actionButton" } [ios/android/harmony — harmony: home/back/power only] - keyboard: { text?: string, key?: string, delayMs?: number } (key pressed after text; TV: text only) [ios/android/chromium/vega/tv/harmony] + keyboard: { text?: string, key?: string, delayMs?: number } (key pressed after text; TV: text only; harmony: enter/backspace/space/arrow-left/arrow-right only) [ios/android/chromium/vega/tv/harmony] text supports {{secret:}} placeholders, resolved server-side from ARGENT_SECRET_ env vars or an argent secrets file — credentials never enter agent context rotate: { orientation: "Portrait"|"LandscapeLeft"|"LandscapeRight"|"PortraitUpsideDown" } [ios/android] tv-remote: { button: , repeat?: number } [apple tv/android tv/vega] diff --git a/packages/tool-server/src/utils/harmony-cli.ts b/packages/tool-server/src/utils/harmony-cli.ts index 59087cc6f8..a1b8eda32c 100644 --- a/packages/tool-server/src/utils/harmony-cli.ts +++ b/packages/tool-server/src/utils/harmony-cli.ts @@ -2,7 +2,7 @@ import { execFile } from "node:child_process"; import { promisify } from "node:util"; import { access, constants as fsConstants } from "node:fs/promises"; import { join } from "node:path"; -import { FAILURE_CODES, FailureError } from "@argent/registry"; +import { FAILURE_CODES, FailureError, subprocessFailureMetadata } from "@argent/registry"; import { formatSubprocessFailure } from "./subprocess-error"; const execFileAsync = promisify(execFile); @@ -130,7 +130,12 @@ export interface HarmonyRunResult { stderr: string; } -const DEFAULT_TIMEOUT_MS = 30_000; +/** + * Per-`Emulator` ceiling. Exported for the same reason `UITEST_TIMEOUT_MS` is: a + * caller on a deadline of its own has to cap against it, since a manager call + * left on this ceiling can outlast the budget the whole boot was given. + */ +export const EMULATOR_TIMEOUT_MS = 30_000; /** See `HDC_KILL_SIGNAL`: a SIGTERM the child may ignore leaves `timeout` unenforced. */ const EMULATOR_KILL_SIGNAL = "SIGKILL" as const; @@ -145,7 +150,7 @@ export const EMULATOR_NOT_FOUND = export async function runHarmonyEmulator( args: string[], - timeoutMs = DEFAULT_TIMEOUT_MS + timeoutMs = EMULATOR_TIMEOUT_MS ): Promise { const bin = await resolveHarmonyEmulator(); if (!bin) { @@ -165,7 +170,13 @@ export async function runHarmonyEmulator( }); return { stdout, stderr }; } catch (err) { - const e = err as { killed?: boolean; code?: unknown; stdout?: string; stderr?: string }; + const e = err as { + killed?: boolean; + signal?: string | null; + code?: unknown; + stdout?: string; + stderr?: string; + }; // The child ran and exited non-zero (`-start` on a missing instance does // this) — its diagnostic is on stdout, so hand the output back and let the // caller classify it exactly as it would an exit-0 failure. A numeric `code` @@ -175,15 +186,16 @@ export async function runHarmonyEmulator( return { stdout: e.stdout ?? "", stderr: e.stderr ?? "" }; } // Spawn failure or timeout SIGKILL: no diagnostic to classify, so surface it - // the way every other subprocess wrapper here does. + // the way every other subprocess wrapper here does — including the kind, so + // a manager killed at its ceiling is not counted as one that failed. throw new FailureError( formatSubprocessFailure("Emulator", args, err), { error_code: FAILURE_CODES.HARMONY_EMULATOR_COMMAND_FAILED, failure_stage: "harmony_emulator_run", failure_area: "tool_server", - error_kind: "subprocess", - failure_command: "deveco_emulator", + error_kind: e.killed || e.signal ? "timeout" : "subprocess", + ...subprocessFailureMetadata(err, "deveco_emulator"), }, { cause: err instanceof Error ? err : new Error(String(err)) } ); diff --git a/packages/tool-server/src/utils/harmony-hdc.ts b/packages/tool-server/src/utils/harmony-hdc.ts index 5c2524b4c5..bb4788c752 100644 --- a/packages/tool-server/src/utils/harmony-hdc.ts +++ b/packages/tool-server/src/utils/harmony-hdc.ts @@ -1,7 +1,7 @@ import { execFile } from "node:child_process"; import { promisify } from "node:util"; import { join } from "node:path"; -import { FAILURE_CODES, FailureError } from "@argent/registry"; +import { FAILURE_CODES, FailureError, subprocessFailureMetadata } from "@argent/registry"; import { formatSubprocessFailure } from "./subprocess-error"; import { commandOnPath } from "./command-on-path"; import { resolveDevecoBinary } from "./harmony-cli"; @@ -127,16 +127,25 @@ export async function runHdc( }); return { stdout, stderr }; } catch (err) { - const e = err as { killed?: boolean; code?: unknown; stdout?: string; stderr?: string }; + const e = err as { + killed?: boolean; + signal?: string | null; + code?: unknown; + stdout?: string; + stderr?: string; + }; if (!e.killed && typeof e.code === "number" && (e.stdout != null || e.stderr != null)) { return { stdout: e.stdout ?? "", stderr: e.stderr ?? "" }; } + // Kind from what happened to the child, as `adb`'s wrapper reads it: a + // client killed at its ceiling is a timeout, not a command that failed, and + // the two want different answers from whoever reads the telemetry. throw new FailureError(formatSubprocessFailure("hdc", args, err), { error_code: FAILURE_CODES.HARMONY_HDC_COMMAND_FAILED, failure_stage: "harmony_hdc_run", failure_area: "tool_server", - error_kind: "subprocess", - failure_command: "hdc", + error_kind: e.killed || e.signal ? "timeout" : "subprocess", + ...subprocessFailureMetadata(err, "hdc"), }); } } diff --git a/packages/tool-server/test/harmony-binary-resolve.test.ts b/packages/tool-server/test/harmony-binary-resolve.test.ts index 245363ad2b..6a96ab8769 100644 --- a/packages/tool-server/test/harmony-binary-resolve.test.ts +++ b/packages/tool-server/test/harmony-binary-resolve.test.ts @@ -141,12 +141,14 @@ describe("DevEco Studio binary resolution", () => { }); describe("running the manager once it has been resolved", () => { - it("classifies a run killed with nothing to say as a subprocess failure", async () => { + it("classifies a run killed at its ceiling as a timeout, naming the signal", async () => { // The one `Emulator` failure with no diagnostic to read: killed at the // timeout, so `emulatorFailure`'s marker list has nothing to match and the // caller cannot classify it downstream. Left a bare `Error` it buckets as // REGISTRY_TOOL_EXECUTION_FAILED, which is where every unclassified throw // in the server already sits — and drops the binary that hung with it. + // The kind is read off the child, as `adb`'s wrapper does it: a manager + // that ran out of time is not one that failed. const bundle = join(tmpRoot, "DevEco-Studio.app"); const emulator = join(bundle, "Contents", EMULATOR_RELATIVE); await mkdir(join(emulator, ".."), { recursive: true }); @@ -163,8 +165,9 @@ describe("DevEco Studio binary resolution", () => { expect(getFailureSignal(err)).toMatchObject({ error_code: FAILURE_CODES.HARMONY_EMULATOR_COMMAND_FAILED, - error_kind: "subprocess", + error_kind: "timeout", failure_command: "deveco_emulator", + failure_signal: "SIGKILL", }); }); }); diff --git a/packages/tool-server/test/harmony-hdc-timeout.test.ts b/packages/tool-server/test/harmony-hdc-timeout.test.ts index 54c02eb5ee..485176e9e5 100644 --- a/packages/tool-server/test/harmony-hdc-timeout.test.ts +++ b/packages/tool-server/test/harmony-hdc-timeout.test.ts @@ -59,8 +59,16 @@ describe("runHdc timeout enforcement", () => { // sanitiser validates `failure_command` against this list and silently drops // a spelling it does not carry, which would file every `hdc` failure under // no binary at all. + // The kind and the signal come off the child the same way `adb`'s wrapper + // reads them, so a wedged daemon is counted as a timeout rather than as a + // command that ran and failed. const signal = getFailureSignal(err); - expect(signal).toMatchObject({ failure_command: "hdc", failure_stage: "harmony_hdc_run" }); + expect(signal).toMatchObject({ + failure_command: "hdc", + failure_stage: "harmony_hdc_run", + error_kind: "timeout", + failure_signal: "SIGKILL", + }); expect(FAILURE_COMMANDS).toContain(signal?.failure_command); }, 20_000); }); diff --git a/packages/tool-server/test/keyboard-key-order.test.ts b/packages/tool-server/test/keyboard-key-order.test.ts index cfd701fb55..e25d9773a5 100644 --- a/packages/tool-server/test/keyboard-key-order.test.ts +++ b/packages/tool-server/test/keyboard-key-order.test.ts @@ -218,6 +218,10 @@ describe("keyboard text+key ordering", () => { "space": "2050", "arrow-left": "2014", "arrow-right": "2015", + // The aliases iOS and Android take for these two. A step that spells the + // submit key `return` runs on both of them and must not stop here. + "return": "2054", + "delete": "2055", }; for (const [key, keyId] of Object.entries(expected)) { From e981ec5e6bac0c61f24fd0fd921dd1da653664ea Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 21:44:06 +0200 Subject: [PATCH 059/110] fix(harmony): only refuse a boot against a listing that has rows MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `listHarmonyInstances` answers `[]` for a `-list` that ran and printed a diagnostic as well as for a host with nothing configured — "Cannot find image" on a host whose images were never downloaded reads the same way. Refusing the boot on an empty listing therefore blocks an instance the host has, on the strength of a listing that could not see it. The refusal now needs rows to name: a listing WITH instances refuses a name that is not among them, and an empty one is left to the start, whose failure already says the manager lists none. --- .../src/tools/devices/boot-device.ts | 29 +++++++++++-------- .../test/boot-device-harmony.test.ts | 29 +++++++++++++++---- 2 files changed, 40 insertions(+), 18 deletions(-) diff --git a/packages/tool-server/src/tools/devices/boot-device.ts b/packages/tool-server/src/tools/devices/boot-device.ts index 8e54bed579..3d90f16ce3 100644 --- a/packages/tool-server/src/tools/devices/boot-device.ts +++ b/packages/tool-server/src/tools/devices/boot-device.ts @@ -1519,7 +1519,7 @@ const HARMONY_IMAGE_RESTRICTION = "the image; an instance has to be created in DevEco Studio on a host that can download one."; /** - * Said in place of the available-instance list when there is none to give. Zero + * Said when the start failed and the manager lists no instances at all. Zero * instances is equally what a host that simply has not created one yet looks * like — including inside mainland China, where the download works — so this * states what was observed and offers the restriction as a possible cause @@ -2098,14 +2098,16 @@ async function bootHarmonyImpl(params: { // the caller's typo is already in hand — the same refusal Android makes off // `emulator -list-avds`. Left to `-start`, the boot instead spends the budget // and answers with the manager's own words, which name nothing that exists. - // Only a listing that was read can refuse; a failed one answers neither - // question (above), so it still lets the start be attempted. - if (instances && !instances.some((i) => i.name === params.instanceName)) { + // + // Only a listing WITH ROWS refuses. An empty one is not the same claim: a + // `-list` that ran but printed a diagnostic — "Cannot find image" on a host + // whose images were never downloaded — comes back from `listHarmonyInstances` + // as `[]` too, and refusing on that would block a boot of an instance the host + // has. That case is still left to the start, whose failure says as much. + if (instances?.length && !instances.some((i) => i.name === params.instanceName)) { throw new FailureError( `HarmonyOS emulator instance "${params.instanceName}" not found. ` + - (instances.length === 0 - ? HARMONY_NO_INSTANCES - : `Available: ${instances.map((i) => i.name).join(", ")}.`), + `Available: ${instances.map((i) => i.name).join(", ")}.`, { error_code: FAILURE_CODES.BOOT_HARMONY_INSTANCE_NOT_FOUND, failure_stage: "boot_harmony_instance_lookup", @@ -2224,11 +2226,14 @@ async function bootHarmonyImpl(params: { } ); } - // Only the manager's own words justify blaming the region. A host with no - // instances at all is answered before the start, by the lookup above. - const cause = - isChinaOnlyRestriction(diagnostic) || diagnostic.includes("Cannot find image") - ? ` ${HARMONY_IMAGE_RESTRICTION}` + const imageMissing = + isChinaOnlyRestriction(diagnostic) || diagnostic.includes("Cannot find image"); + // Only the manager's own words justify blaming the region; an empty + // instance list is merely consistent with it. + const cause = imageMissing + ? ` ${HARMONY_IMAGE_RESTRICTION}` + : instances?.length === 0 + ? ` ${HARMONY_NO_INSTANCES}` : ""; throw new FailureError( cause diff --git a/packages/tool-server/test/boot-device-harmony.test.ts b/packages/tool-server/test/boot-device-harmony.test.ts index e78637ba57..81a7febbd1 100644 --- a/packages/tool-server/test/boot-device-harmony.test.ts +++ b/packages/tool-server/test/boot-device-harmony.test.ts @@ -824,17 +824,31 @@ describe("boot-device — HarmonyOS emulator path", () => { await settled; }); - it("points at creating an instance when the host has none", async () => { + it("points at creating an instance when the start failed and none exist", async () => { + // An empty listing is not a refusal of its own: `listHarmonyInstances` + // answers `[]` for a `-list` that ran and printed a diagnostic too, so a + // host whose images are missing must still reach the start rather than be + // told it has no instances by a listing that could not see them. listHarmonyInstances.mockResolvedValue([]); targets([PHONE]); + vi.useFakeTimers(); - const failed = await boot({ bootTimeoutMs: 900_000 }).catch((e: unknown) => e); + const pending = boot({ bootTimeoutMs: 900_000 }); + const settled = expect(pending).rejects.toThrow(/create one in DevEco Studio/); + await vi.advanceTimersByTimeAsync(1_000); + expect(spawnMock).toHaveBeenCalledWith( + expect.any(String), + ["-start", INSTANCE], + expect.any(Object) + ); + child.die(`"${INSTANCE}" is not found. Please create the device(folder): /x`, 1); + await vi.advanceTimersByTimeAsync(3_000); + await settled; - expect((failed as Error).message).toMatch(/create one in DevEco Studio/); - expect(getFailureSignal(failed)).toMatchObject({ - error_code: FAILURE_CODES.BOOT_HARMONY_INSTANCE_NOT_FOUND, + expect(getFailureSignal(await pending.catch((e: unknown) => e))).toMatchObject({ + error_code: FAILURE_CODES.BOOT_HARMONY_START_FAILED, + failure_command: "deveco_emulator", }); - expect(spawnMock).not.toHaveBeenCalled(); }); it("names the instances the host does have, rather than starting one it does not", async () => { @@ -854,6 +868,9 @@ describe("boot-device — HarmonyOS emulator path", () => { expect((failed as Error).message).toMatch( /"Phone1" not found\. Available: Tablet_9, Phone_1\./ ); + expect(getFailureSignal(failed)).toMatchObject({ + error_code: FAILURE_CODES.BOOT_HARMONY_INSTANCE_NOT_FOUND, + }); expect(spawnMock).not.toHaveBeenCalled(); }); From bc7629a8e495592cbf405b496fa813505c8a580e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 22:25:48 +0200 Subject: [PATCH 060/110] fix(harmony): bound an interaction to one budget, and refuse a panel with no coordinates Every HarmonyOS interaction read the display on a 20s ceiling and then injected on another, so a tap's worst case was 40s plus an unbounded wait for the device's `uitest` queue. None of these tools declares `longRunning`, so the MCP client aborts at 30s and replays the call while the abandoned `hdc` children keep running and keep holding that queue: for a tap the replay is a second touch for one the caller believes never happened. One 20s whole-call ceiling now covers the read, the queue and every injection, with the render-service read cut to 5s (measured at 50-190ms, slowest `hdc shell` round trip 0.8s). `viaDeviceTmp` claimed the deadline was spent across both round trips but handed the producer a fresh full ceiling, so with two callers on one connect key the fetch reliably got `Math.max(1, ...)` = 1ms, was SIGKILLed, and came back as a device that timed out - with the capture still on the device. Each leg now takes what is left, and a leg that starts with nothing left says the budget ran out instead of dying as an unexplained kill. The clock also starts before the queue rather than after it, so time spent queued is charged to the caller that is waiting. The cleanup delete keeps running past the budget, but on a 5s ceiling of its own rather than `hdc`'s 30s default. The render service prints `render resolution=0x0` while the guest compositor is still coming up, and `toDevicePoint` clamps every coordinate against it: a tap at (0.83, 0.42) went out as `uiInput click 0 0` and came back `{ tapped: true }`. `assertHarmonyScreenAwake` becomes `assertHarmonyDisplayReady` and refuses that read before the power state, the way `boot-device` already refuses it as unreadable. An empty `keyboard` call now no-ops with no device traffic, as every other backend does, rather than paying for a display read it has no use for. The wait loop's final poll runs AT the deadline and is handed no budget, so a backend that refuses an empty budget outright would report that refusal as the reason the wait failed - displacing what the polls that did have time found on the device. That answer is now kept only when nothing else was ever read. --- .../tool-server/src/tools/button/index.ts | 10 +- .../tools/describe/platforms/harmony/index.ts | 27 ++- .../src/tools/gesture-swipe/index.ts | 20 +- .../src/tools/gesture-tap/index.ts | 18 +- .../src/tools/keyboard/platforms/harmony.ts | 22 +- .../tool-server/src/utils/harmony-uitest.ts | 188 ++++++++++++++---- .../src/utils/poll-describe-tree.ts | 8 +- .../tool-server/test/await-harmony.test.ts | 3 +- packages/tool-server/test/button.test.ts | 61 +++++- .../test/gesture-swipe-harmony.test.ts | 61 ++++++ .../test/gesture-tap-harmony.test.ts | 56 +++++- .../tool-server/test/harmony-describe.test.ts | 3 +- packages/tool-server/test/harmony-hdc.test.ts | 61 +++++- .../test/harmony-uitest-queue.test.ts | 99 +++++++-- .../test/keyboard-key-order.test.ts | 86 +++++++- 15 files changed, 624 insertions(+), 99 deletions(-) diff --git a/packages/tool-server/src/tools/button/index.ts b/packages/tool-server/src/tools/button/index.ts index 84b3dd246e..7f9c1ab091 100644 --- a/packages/tool-server/src/tools/button/index.ts +++ b/packages/tool-server/src/tools/button/index.ts @@ -3,7 +3,8 @@ import type { Platform, ServiceRef, ToolCapability, ToolDefinition } from "@arge import { simulatorServerRef, type SimulatorServerApi } from "../../blueprints/simulator-server"; import { resolveDevice, harmonyConnectKey } from "../../utils/device-info"; import { - assertHarmonyScreenAwake, + HARMONY_INTERACTION_TIMEOUT_MS, + assertHarmonyDisplayReady, harmonyDisplay, harmonyKeyEvent, } from "../../utils/harmony-uitest"; @@ -141,6 +142,9 @@ Fails if the device backend is not reachable — the simulator-server for iOS, \ // guarantees a key name exists for every accepted button. await ensureDep("hdc"); const connectKey = harmonyConnectKey(device.id); + // One deadline for the display read and the press, so the pair stays under + // the MCP layer's abort-and-replay cap. + const deadline = Date.now() + HARMONY_INTERACTION_TIMEOUT_MS; // `uitest uiInput keyEvent` answers `No Error` against a suspended panel // while the press lands nowhere, so `home` and `back` share the guard the // tap, swipe and typing backends use: a screen timeout mid-session must @@ -150,9 +154,9 @@ Fails if the device backend is not reachable — the simulator-server for iOS, \ // refusal tells the caller to wake the device with, and the one key that // works while the panel is suspended. if (params.button !== "power") { - assertHarmonyScreenAwake(await harmonyDisplay(connectKey), `press ${params.button}`); + assertHarmonyDisplayReady(await harmonyDisplay(connectKey), `press ${params.button}`); } - await harmonyKeyEvent(connectKey, HARMONY_BUTTON_KEYS[params.button]!); + await harmonyKeyEvent(connectKey, HARMONY_BUTTON_KEYS[params.button]!, deadline - Date.now()); return { pressed: params.button }; } const api = services.simulatorServer as SimulatorServerApi; diff --git a/packages/tool-server/src/tools/describe/platforms/harmony/index.ts b/packages/tool-server/src/tools/describe/platforms/harmony/index.ts index d07da9784e..6df7acc7f6 100644 --- a/packages/tool-server/src/tools/describe/platforms/harmony/index.ts +++ b/packages/tool-server/src/tools/describe/platforms/harmony/index.ts @@ -3,7 +3,12 @@ import { join } from "node:path"; import { rm } from "node:fs/promises"; import type { ToolDependency } from "@argent/registry"; import type { DescribeTreeData } from "../../contract"; -import { harmonyDisplay, harmonyDumpLayout } from "../../../../utils/harmony-uitest"; +import { + harmonyDisplay, + harmonyDumpLayout, + remainingBudget, + HARMONY_INTERACTION_TIMEOUT_MS, +} from "../../../../utils/harmony-uitest"; import { parseHarmonyLayout } from "./layout-parser"; /** `uitest` runs on the device; reaching it needs only the connector. */ @@ -33,7 +38,9 @@ const EMPTY_HINT = * describe's contract is the rendered text and nothing downstream reads it. * * `timeoutMs` is the whole read's budget, split across the panel query and the - * dump rather than given to each. A caller polling to a deadline needs it: the + * dump rather than given to each, and defaulting to the one-interaction ceiling + * so a caller with no deadline of its own is bounded under the MCP client's + * abort-and-replay cap. A caller polling to a deadline needs it: the * `uitest` client this spawns holds the device's queue until it is killed, so a * read left on its own 20s ceiling and abandoned at a 300ms deadline is 20s the * caller's NEXT call spends queued — measured at 0.8s of it on a warm emulator, @@ -43,14 +50,20 @@ const EMPTY_HINT = */ export async function describeHarmony( connectKey: string, - timeoutMs?: number + timeoutMs: number = HARMONY_INTERACTION_TIMEOUT_MS ): Promise { const localPath = join(tmpdir(), `argent-harmony-dump-${process.hrtime.bigint()}.json`); - const deadline = timeoutMs === undefined ? undefined : Date.now() + timeoutMs; - const left = () => (deadline === undefined ? undefined : Math.max(1, deadline - Date.now())); - const display = await harmonyDisplay(connectKey, left()); + const deadline = Date.now() + timeoutMs; + const display = await harmonyDisplay( + connectKey, + remainingBudget(connectKey, deadline, "the display read") + ); try { - const raw = await harmonyDumpLayout(connectKey, localPath, left()); + const raw = await harmonyDumpLayout( + connectKey, + localPath, + remainingBudget(connectKey, deadline, "the layout dump") + ); const { tree, screen } = parseHarmonyLayout(raw, { width: display.width, height: display.height, diff --git a/packages/tool-server/src/tools/gesture-swipe/index.ts b/packages/tool-server/src/tools/gesture-swipe/index.ts index 39447301cd..6b242cce84 100644 --- a/packages/tool-server/src/tools/gesture-swipe/index.ts +++ b/packages/tool-server/src/tools/gesture-swipe/index.ts @@ -3,7 +3,8 @@ import type { ServiceRef, ToolCapability, ToolDefinition } from "@argent/registr import { simulatorServerRef, type SimulatorServerApi } from "../../blueprints/simulator-server"; import { resolveDevice, harmonyConnectKey } from "../../utils/device-info"; import { - assertHarmonyScreenAwake, + HARMONY_INTERACTION_TIMEOUT_MS, + assertHarmonyDisplayReady, harmonyDisplay, harmonySwipe, toDevicePoint, @@ -95,14 +96,25 @@ Pass settle:true for a momentum-free swipe that lands exactly where the finger l if (device.platform === "harmony") { await ensureDep("hdc"); const connectKey = harmonyConnectKey(device.id); + // One deadline for the geometry read and the injection it feeds, so the + // pair stays under the MCP layer's abort-and-replay cap. + const deadline = Date.now() + HARMONY_INTERACTION_TIMEOUT_MS; const display = await harmonyDisplay(connectKey); - // A swipe against a suspended panel reports `No Error` and lands nowhere. - assertHarmonyScreenAwake(display, "swipe"); + // A swipe against a panel that is suspended, or that the render service + // could not size, reports `No Error` and lands nowhere. + assertHarmonyDisplayReady(display, "swipe"); const fromPx = toDevicePoint(params.fromX, params.fromY, display); const toPx = toDevicePoint(params.toX, params.toY, display); const distance = Math.hypot(toPx.x - fromPx.x, toPx.y - fromPx.y); const seconds = Math.max(duration, 1) / 1000; - await harmonySwipe(connectKey, settle ? "swipe" : "fling", fromPx, toPx, distance / seconds); + await harmonySwipe( + connectKey, + settle ? "swipe" : "fling", + fromPx, + toPx, + distance / seconds, + deadline - Date.now() + ); return { swiped: true, timestampMs }; } const api = services.simulatorServer as SimulatorServerApi; diff --git a/packages/tool-server/src/tools/gesture-tap/index.ts b/packages/tool-server/src/tools/gesture-tap/index.ts index de665022c6..688f0f6c04 100644 --- a/packages/tool-server/src/tools/gesture-tap/index.ts +++ b/packages/tool-server/src/tools/gesture-tap/index.ts @@ -5,7 +5,8 @@ import { chromiumCdpRef, type ChromiumCdpApi } from "../../blueprints/chromium-c import { assertChromiumWindowVisible } from "../../utils/chromium-visibility"; import { resolveDevice, harmonyConnectKey } from "../../utils/device-info"; import { - assertHarmonyScreenAwake, + HARMONY_INTERACTION_TIMEOUT_MS, + assertHarmonyDisplayReady, harmonyDisplay, harmonyTouch, toDevicePoint, @@ -110,18 +111,23 @@ async function tapHarmony( y: number, clickCount: number ): Promise { + // One deadline for the read and every injection that follows it, so a + // multi-tap cannot spend a fresh ceiling per click and outlive the MCP layer's + // abort-and-replay cap — where the replay is another tap on the same spot. + const deadline = Date.now() + HARMONY_INTERACTION_TIMEOUT_MS; const display = await harmonyDisplay(connectKey); - // A tap against a suspended panel reports `No Error` and lands nowhere — - // refuse it rather than report { tapped: true } for a touch that did nothing. - assertHarmonyScreenAwake(display, "tap"); + // A tap against a panel that is suspended, or that the render service could + // not size, reports `No Error` and lands nowhere — refuse it rather than + // report { tapped: true } for a touch that did nothing. + assertHarmonyDisplayReady(display, "tap"); const point = toDevicePoint(x, y, display); if (clickCount === 2) { - await harmonyTouch(connectKey, "doubleClick", point); + await harmonyTouch(connectKey, "doubleClick", point, deadline - Date.now()); return; } for (let i = 0; i < clickCount; i++) { if (i > 0) await sleep(MULTI_TAP_GAP_MS); - await harmonyTouch(connectKey, "click", point); + await harmonyTouch(connectKey, "click", point, deadline - Date.now()); } } diff --git a/packages/tool-server/src/tools/keyboard/platforms/harmony.ts b/packages/tool-server/src/tools/keyboard/platforms/harmony.ts index e8d154ca46..a7724ef879 100644 --- a/packages/tool-server/src/tools/keyboard/platforms/harmony.ts +++ b/packages/tool-server/src/tools/keyboard/platforms/harmony.ts @@ -3,7 +3,8 @@ import type { PlatformImpl } from "../../../utils/cross-platform-tool"; import { InvalidToolInputError } from "../../../utils/capability"; import { harmonyConnectKey } from "../../../utils/device-info"; import { - assertHarmonyScreenAwake, + HARMONY_INTERACTION_TIMEOUT_MS, + assertHarmonyDisplayReady, harmonyDisplay, harmonyKeyEvent, harmonyTypeText, @@ -112,21 +113,30 @@ async function typeHarmony(connectKey: string, params: KeyboardParams): Promise< // rejects having typed nothing. const keycode = params.key ? resolveHarmonyKeycode(params.key) : null; if (params.text) assertTypeableHarmonyText(params.text); - // Typing into a suspended panel reports `No Error` and lands nowhere, so a - // dead screen fails the same way a tap does. - assertHarmonyScreenAwake(await harmonyDisplay(connectKey), "type"); + // Neither key nor text is schema-valid (both are optional, with no refinement) + // and injects nothing, so it costs no device round trip — the contract every + // sibling backend follows. Reaching the device first would let a suspended + // panel fail a step that was never going to type anything. + if (keycode === null && !params.text) return { typed: "", keys: 0 }; + // One deadline for the display read and both injections, so a type-then-submit + // call stays under the MCP layer's abort-and-replay cap. + const deadline = Date.now() + HARMONY_INTERACTION_TIMEOUT_MS; + // Typing into a panel that is suspended, or that the render service could not + // size, reports `No Error` and lands nowhere, so a dead screen fails the same + // way a tap does. + assertHarmonyDisplayReady(await harmonyDisplay(connectKey), "type"); let keysPressed = 0; if (params.text) { // `uitest uiInput text` types into whatever holds focus, in one shot — there // is no per-character injection, so `delayMs` has nothing to pace (the tool // description already lists the platforms that ignore it). - await harmonyTypeText(connectKey, params.text); + await harmonyTypeText(connectKey, params.text, deadline - Date.now()); keysPressed += [...params.text].length; } // Key after text, so a combined call means "type, then submit". Pressing first // would submit an empty field. if (keycode !== null) { - await harmonyKeyEvent(connectKey, String(keycode)); + await harmonyKeyEvent(connectKey, String(keycode), deadline - Date.now()); keysPressed++; } return { typed: params.text ?? params.key ?? "", keys: keysPressed }; diff --git a/packages/tool-server/src/utils/harmony-uitest.ts b/packages/tool-server/src/utils/harmony-uitest.ts index f5a6cdb7f5..ec66331294 100644 --- a/packages/tool-server/src/utils/harmony-uitest.ts +++ b/packages/tool-server/src/utils/harmony-uitest.ts @@ -50,6 +50,34 @@ const REMOTE_TMP = "/data/local/tmp"; */ export const UITEST_TIMEOUT_MS = 20_000; +/** + * Whole-call ceiling for one interaction — the display read, the wait for this + * device's `uitest` queue and every injection come out of it together. + * + * None of the interaction tools declares `longRunning`, so the MCP client aborts + * a call at 30s and *replays* it while the abandoned `hdc` children keep running + * and keep holding the queue. For a tap that replay is a second touch for one + * the caller believes never happened, so the tool has to fail on its own budget + * first — which two legs on {@link UITEST_TIMEOUT_MS} each cannot do. + */ +export const HARMONY_INTERACTION_TIMEOUT_MS = 20_000; + +/** + * Ceiling for one render-service read, and the reason the whole interaction fits + * inside one `uitest` ceiling: the read is measured at 50-190ms and the slowest + * `hdc shell` round trip measured on an emulator is 0.8s, so this is ~6x headroom + * for a loaded host rather than a bound a healthy read approaches. + */ +export const HARMONY_DISPLAY_TIMEOUT_MS = 5_000; + +/** + * Ceiling for the on-device cleanup delete, which runs after the caller's budget + * is already spent. `runHdcShell`'s 30s default would add that much again past a + * deadline the caller has exhausted; a `rm -f` is the lightest shell round trip + * there is, against the 0.1-0.8s measured for one. + */ +const REMOTE_CLEANUP_TIMEOUT_MS = 5_000; + /** * `uitest` does not tolerate overlapping invocations: a second call on the * same device blocks until the first finishes, and if that takes past the @@ -89,14 +117,45 @@ function uitestDiagnostic(stdout: string): string { return first?.trim() ?? "uitest failed without a diagnostic"; } +/** + * Milliseconds left of a shared deadline, or a refusal when there are none. + * + * A leg started with nothing left is killed the moment it begins and reported as + * a device that hung, which is the wrong repair for a budget that was gone + * before it — and a spent budget cannot simply be passed on, since `execFile` + * reads a timeout of 0 as *no* timeout. + */ +export function remainingBudget(connectKey: string, deadline: number, step: string): number { + const left = deadline - Date.now(); + if (left > 0) return left; + throw new FailureError( + `Ran out of time before ${step} on HarmonyOS device '${connectKey}': the call's budget was ` + + `spent by the steps before it, waiting for this device's \`uitest\` queue included. ` + + `Retry once the device is idle.`, + { + error_code: FAILURE_CODES.HARMONY_HDC_COMMAND_FAILED, + failure_stage: "harmony_budget_exhausted", + failure_area: "tool_server", + error_kind: "timeout", + failure_command: "hdc", + } + ); +} + /** Run a `uitest` subcommand, throwing with its own diagnostic if it exits non-zero. */ -async function runUitest( - connectKey: string, - args: string, - timeoutMs = UITEST_TIMEOUT_MS -): Promise { +async function runUitest(connectKey: string, args: string, timeoutMs: number): Promise { + // The ceiling spans the queue as well as the call. A clock started once this + // device's queue hands over cannot see the wait in front of it, so a caller + // queued behind another — the one case that makes an interaction outlive its + // budget — would be charged against no ceiling at all. + const deadline = Date.now() + timeoutMs; const { stdout, exitCode } = await enqueueUitest(connectKey, () => - runHdcShell(connectKey, `uitest ${args}`, timeoutMs) + runHdcShell( + connectKey, + `uitest ${args}`, + // The subcommand alone — the arguments carry the text being typed. + remainingBudget(connectKey, deadline, `\`uitest ${args.split(" ")[0]}\``) + ) ); if (exitCode !== 0) { throw new FailureError(`uitest ${args} failed on ${connectKey}: ${uitestDiagnostic(stdout)}`, { @@ -129,7 +188,7 @@ interface HarmonyDisplay { */ export async function harmonyDisplay( connectKey: string, - timeoutMs = UITEST_TIMEOUT_MS + timeoutMs = HARMONY_DISPLAY_TIMEOUT_MS ): Promise { const { stdout } = await runHdcShell( connectKey, @@ -174,16 +233,42 @@ export function toDevicePoint( } /** - * Refuse an input injection against a suspended panel. + * Refuse an input injection against a display that cannot receive one. + * + * `uitest uiInput` reports `No Error` whether or not the touch landed, so both + * states it cannot land in are argent's to catch, off the one read every input + * tool already makes: * - * `uitest uiInput` reports `No Error` whether or not the touch landed, and - * against a screen that is OFF or SUSPEND it lands nowhere — measured: a tap - * on the WLAN row of Settings with `powerStatus=POWER_STATUS_OFF` returned - * success and changed nothing. `describe` is honest about this same state - * (its asleep hint); the input tools must be too, or a screen timeout mid- - * session turns every later gesture into a silent no-op that reports success. + * - **A non-positive panel.** The render service prints `render resolution=0x0` + * while the guest's compositor is still coming up, and every normalized + * coordinate then clamps to the origin — a tap at (0.83, 0.42) goes out as + * `uiInput click 0 0` and comes back as the tap that was asked for. + * `boot-device` refuses the same read as unreadable rather than act on it. + * - **A suspended panel.** Measured: a tap on the WLAN row of Settings with + * `powerStatus=POWER_STATUS_OFF` returned success and changed nothing. + * `describe` is honest about this state (its asleep hint); the input tools + * must be too, or a screen timeout mid-session turns every later gesture into + * a silent no-op that reports success. */ -export function assertHarmonyScreenAwake(display: HarmonyDisplay, action: string): void { +export function assertHarmonyDisplayReady(display: HarmonyDisplay, action: string): void { + // Before the power state, which is read off the same dump: a service that + // could not name a panel has said nothing trustworthy about that panel's + // power either. + if (display.width <= 0 || display.height <= 0) { + throw new FailureError( + `Cannot ${action} on a HarmonyOS device whose render service reports a ` + + `${display.width}x${display.height} display: there are no coordinates to aim at, and ` + + `every position would collapse onto the top-left pixel. The panel has not composited ` + + `yet, or the render service answered with nothing usable — retry once the device has ` + + `finished booting.`, + { + error_code: FAILURE_CODES.HARMONY_UITEST_FAILED, + failure_stage: "harmony_display_zero", + failure_area: "tool_server", + error_kind: "validation", + } + ); + } if (display.screenOn) return; throw new FailureError( `Cannot ${action} on a HarmonyOS device whose display is off: injected input lands nowhere ` + @@ -202,9 +287,10 @@ type HarmonyTouchCommand = "click" | "doubleClick"; export async function harmonyTouch( connectKey: string, command: HarmonyTouchCommand, - point: { x: number; y: number } + point: { x: number; y: number }, + timeoutMs: number ): Promise { - await runUitest(connectKey, `uiInput ${command} ${point.x} ${point.y}`); + await runUitest(connectKey, `uiInput ${command} ${point.x} ${point.y}`, timeoutMs); } type HarmonySwipeCommand = "swipe" | "fling"; @@ -235,19 +321,32 @@ export async function harmonySwipe( command: HarmonySwipeCommand, from: { x: number; y: number }, to: { x: number; y: number }, - velocity: number + velocity: number, + timeoutMs: number ): Promise { const v = Math.max(HARMONY_VELOCITY_MIN, Math.min(HARMONY_VELOCITY_MAX, Math.round(velocity))); - await runUitest(connectKey, `uiInput ${command} ${from.x} ${from.y} ${to.x} ${to.y} ${v}`); + await runUitest( + connectKey, + `uiInput ${command} ${from.x} ${from.y} ${to.x} ${to.y} ${v}`, + timeoutMs + ); } -export async function harmonyKeyEvent(connectKey: string, key: string): Promise { - await runUitest(connectKey, `uiInput keyEvent ${key}`); +export async function harmonyKeyEvent( + connectKey: string, + key: string, + timeoutMs: number +): Promise { + await runUitest(connectKey, `uiInput keyEvent ${key}`, timeoutMs); } /** Type into whatever currently holds focus. */ -export async function harmonyTypeText(connectKey: string, text: string): Promise { - await runUitest(connectKey, `uiInput text ${shellQuote(text)}`); +export async function harmonyTypeText( + connectKey: string, + text: string, + timeoutMs: number +): Promise { + await runUitest(connectKey, `uiInput text ${shellQuote(text)}`, timeoutMs); } /** @@ -264,26 +363,33 @@ async function viaDeviceTmp( connectKey: string, suffix: string, localPath: string, - producer: (remotePath: string) => Promise, - deadline?: number + producer: (remotePath: string, timeoutMs: number) => Promise, + deadline: number ): Promise { const remotePath = `${REMOTE_TMP}/argent-${process.pid}-${process.hrtime.bigint()}${suffix}`; try { - await producer(remotePath); - // The fetch is bounded together with the producer rather than left on the - // 30s default: a caller working to a deadline spends it across BOTH round - // trips, and one that has already run out asks for a transfer that fails at - // once instead of one that runs 30s past it. The delete keeps the default — - // it runs outside the queue, and a spent budget is no reason to leave a - // capture behind on the device. + // A caller working to a deadline spends it across BOTH round trips, so each + // leg is handed what is left of it rather than a ceiling of its own — a + // producer given the full budget leaves the fetch a sliver, and a fetch left + // on `hdc`'s 30s default runs that far past a deadline the caller has + // already exhausted. + await producer(remotePath, remainingBudget(connectKey, deadline, "the capture")); await hdcFileRecv( connectKey, remotePath, localPath, - deadline === undefined ? undefined : Math.max(1, deadline - Date.now()) + remainingBudget(connectKey, deadline, "copying the result off the device") ); } finally { - await runHdcShell(connectKey, `rm -f ${shellQuote(remotePath)}`).catch(() => {}); + // The delete is outside the shared budget — it runs after that budget is + // spent, and leaving a multi-hundred-KB capture on a partition nothing + // prunes is not the way to save the time — but on a ceiling of its own, so a + // wedged daemon cannot add 30s to a call that is already over. + await runHdcShell( + connectKey, + `rm -f ${shellQuote(remotePath)}`, + REMOTE_CLEANUP_TIMEOUT_MS + ).catch(() => {}); } } @@ -293,15 +399,14 @@ export async function harmonyScreenCap( localPath: string, timeoutMs = UITEST_TIMEOUT_MS ): Promise { - const deadline = Date.now() + timeoutMs; await viaDeviceTmp( connectKey, ".png", localPath, - async (remotePath) => { - await runUitest(connectKey, `screenCap -p ${shellQuote(remotePath)}`, timeoutMs); + async (remotePath, leftMs) => { + await runUitest(connectKey, `screenCap -p ${shellQuote(remotePath)}`, leftMs); }, - deadline + Date.now() + timeoutMs ); } @@ -324,15 +429,14 @@ export async function harmonyDumpLayout( localPath: string, timeoutMs = UITEST_TIMEOUT_MS ): Promise { - const deadline = Date.now() + timeoutMs; await viaDeviceTmp( connectKey, ".json", localPath, - async (remotePath) => { - await runUitest(connectKey, `dumpLayout -p ${shellQuote(remotePath)}`, timeoutMs); + async (remotePath, leftMs) => { + await runUitest(connectKey, `dumpLayout -p ${shellQuote(remotePath)}`, leftMs); }, - deadline + Date.now() + timeoutMs ); const raw = await readFile(localPath, "utf8"); try { diff --git a/packages/tool-server/src/utils/poll-describe-tree.ts b/packages/tool-server/src/utils/poll-describe-tree.ts index 553155c19a..2a912e0755 100644 --- a/packages/tool-server/src/utils/poll-describe-tree.ts +++ b/packages/tool-server/src/utils/poll-describe-tree.ts @@ -102,6 +102,12 @@ export async function pollDescribeTree( // Bound each fetch to the time left before the deadline. const remaining = Math.max(0, deadline - Date.now()); + // The final poll runs AT the deadline (see the sleep clamp below), so it is + // handed nothing. A backend that refuses an empty budget outright then fails + // for a reason of the loop's own making, which must not displace what the + // polls that had time actually found — the same reason the timeout branch + // below only speaks up when no tree was ever read. + const unbudgeted = remaining === 0; const settled = await settleWithin(fetchTree(remaining), remaining, signal); polls += 1; @@ -116,7 +122,7 @@ export async function pollDescribeTree( break; } if (settled.type === "error") { - lastError = settled.error; + if (!unbudgeted || (lastError === undefined && lastData === null)) lastError = settled.error; } else { lastData = settled.value; lastError = undefined; diff --git a/packages/tool-server/test/await-harmony.test.ts b/packages/tool-server/test/await-harmony.test.ts index bd3cdff872..9710c765a1 100644 --- a/packages/tool-server/test/await-harmony.test.ts +++ b/packages/tool-server/test/await-harmony.test.ts @@ -11,7 +11,8 @@ import { __primeDepCacheForTests, __resetDepCacheForTests } from "../src/utils/c // Stub the `uitest`-over-`hdc` transport rather than `describeHarmony` itself, // so the wait tools poll through the real dump parser and the real hint rules — // those are what decide whether an unreadable screen may confirm `hidden`. -vi.mock("../src/utils/harmony-uitest", () => ({ +vi.mock("../src/utils/harmony-uitest", async (importOriginal) => ({ + ...(await importOriginal()), harmonyDisplay: vi.fn(), harmonyDumpLayout: vi.fn(), })); diff --git a/packages/tool-server/test/button.test.ts b/packages/tool-server/test/button.test.ts index 0eaa32819d..d3a7e5daf0 100644 --- a/packages/tool-server/test/button.test.ts +++ b/packages/tool-server/test/button.test.ts @@ -16,9 +16,9 @@ vi.mock("../src/utils/android-input", async (importOriginal) => ({ })); // HarmonyOS presses go over `uitest uiInput keyEvent` on the device; neutralise -// it for the same reason as the adb call above. `harmonyDisplay` is the power -// state the press is gated on, stubbed awake by default (see beforeEach); -// `assertHarmonyScreenAwake` stays real so the tests see the guard's own refusal. +// it for the same reason as the adb call above. `harmonyDisplay` is the read the +// press is gated on, stubbed awake by default (see beforeEach); +// `assertHarmonyDisplayReady` stays real so the tests see the guard's own refusal. vi.mock("../src/utils/harmony-uitest", async (importOriginal) => ({ ...(await importOriginal()), harmonyKeyEvent: vi.fn(), @@ -38,7 +38,11 @@ import { buttonTool, BUTTONS_BY_PLATFORM } from "../src/tools/button"; import { UnsupportedOperationError } from "../src/utils/capability"; import { ANDROID_BUTTON_KEYCODES, injectAndroidKeycode } from "../src/utils/android-input"; import { DependencyMissingError, ensureDep } from "../src/utils/check-deps"; -import { harmonyDisplay, harmonyKeyEvent } from "../src/utils/harmony-uitest"; +import { + HARMONY_INTERACTION_TIMEOUT_MS, + harmonyDisplay, + harmonyKeyEvent, +} from "../src/utils/harmony-uitest"; import { sendCommand } from "../src/utils/simulator-client"; const iosUdid = "AAAAAAAA-AAAA-AAAA-AAAA-AAAAAAAAAAAA"; @@ -196,7 +200,11 @@ describe("button tool — per-platform validation", () => { expect(expectedKey, `no uitest key name for "${button}"`).toBeTypeOf("string"); // The connect key, not the `harmony-` prefixed device id — `uitest` is // addressed by what `hdc list targets` reports. - expect(harmonyKeyEvent).toHaveBeenCalledWith(harmonyConnectKey, expectedKey); + expect(harmonyKeyEvent).toHaveBeenCalledWith( + harmonyConnectKey, + expectedKey, + expect.any(Number) + ); // hdc is preflighted so a missing connector fails with a 424 install hint. expect(ensureDep).toHaveBeenCalledWith("hdc"); // Neither of the other backends is touched — in particular the press must @@ -268,7 +276,7 @@ describe("button tool — a suspended HarmonyOS panel", () => { await expect( buttonTool.execute(services, { udid: harmonyUdid, button: "power" }) ).resolves.toEqual({ pressed: "power" }); - expect(harmonyKeyEvent).toHaveBeenCalledWith(harmonyConnectKey, "Power"); + expect(harmonyKeyEvent).toHaveBeenCalledWith(harmonyConnectKey, "Power", expect.any(Number)); expect(harmonyDisplay).not.toHaveBeenCalled(); }); @@ -276,7 +284,46 @@ describe("button tool — a suspended HarmonyOS panel", () => { await expect( buttonTool.execute(services, { udid: harmonyUdid, button: "home" }) ).resolves.toEqual({ pressed: "home" }); - expect(harmonyKeyEvent).toHaveBeenCalledWith(harmonyConnectKey, "Home"); + expect(harmonyKeyEvent).toHaveBeenCalledWith(harmonyConnectKey, "Home", expect.any(Number)); + }); + + it("refuses `home` when the render service reports a 0x0 display", async () => { + // A guest whose compositor has not come up answers `render resolution=0x0`, + // and nothing about that read is trustworthy — including the power state + // parsed off the same dump. The tap and swipe paths refuse it through the + // same helper; a press that went ahead would inject against a device that + // has no screen yet and still resolve `{ pressed: "home" }`. + vi.mocked(harmonyDisplay).mockResolvedValue({ width: 0, height: 0, screenOn: true }); + + const err = await buttonTool.execute(services, { udid: harmonyUdid, button: "home" }).then( + () => { + throw new Error("expected the press to reject, but it resolved"); + }, + (e: unknown) => e + ); + + expect(getFailureSignal(err)?.failure_stage).toBe("harmony_display_zero"); + expect((err as Error).message).toContain("press home"); + expect((err as Error).message).toContain("0x0 display"); + expect(harmonyKeyEvent).not.toHaveBeenCalled(); + }); + + it("gives the press what is left of ONE budget shared with the display read", async () => { + // Both legs on `UITEST_TIMEOUT_MS` put a press's worst case at 40s, above + // the 30s at which the MCP client aborts a call and REPLAYS it — a second + // `Back` for one the caller believes never happened. The read is charged + // against the same deadline, so the press gets what it left behind. + const READ_MS = 60; + vi.mocked(harmonyDisplay).mockImplementation(async () => { + await new Promise((r) => setTimeout(r, READ_MS)); + return HARMONY_AWAKE; + }); + + await buttonTool.execute(services, { udid: harmonyUdid, button: "home" }); + + const budget = vi.mocked(harmonyKeyEvent).mock.calls[0][2]; + expect(budget).toBeLessThanOrEqual(HARMONY_INTERACTION_TIMEOUT_MS - READ_MS); + expect(budget).toBeGreaterThan(0); }); }); diff --git a/packages/tool-server/test/gesture-swipe-harmony.test.ts b/packages/tool-server/test/gesture-swipe-harmony.test.ts index bd4b3048ff..9a29fcb196 100644 --- a/packages/tool-server/test/gesture-swipe-harmony.test.ts +++ b/packages/tool-server/test/gesture-swipe-harmony.test.ts @@ -28,6 +28,10 @@ vi.mock("../src/utils/check-deps", async (importOriginal) => ({ import { gestureSwipeTool } from "../src/tools/gesture-swipe"; import { ensureDep } from "../src/utils/check-deps"; import { runHdcShell as realRunHdcShell } from "../src/utils/harmony-hdc"; +import { + HARMONY_DISPLAY_TIMEOUT_MS, + HARMONY_INTERACTION_TIMEOUT_MS, +} from "../src/utils/harmony-uitest"; import { sendCommand } from "../src/utils/simulator-client"; const runHdcShell = vi.mocked(realRunHdcShell); @@ -161,6 +165,63 @@ describe("gesture-swipe on HarmonyOS", () => { ); }); + it("refuses to swipe when the render service reports a 0x0 render resolution", async () => { + // What a guest prints while its compositor is still coming up — parsed here + // rather than stubbed, since `render resolution=0x0` is a line the parser + // accepts without complaint. Both endpoints would clamp onto the origin, so + // the swipe would go out as `uiInput fling 0 0 0 0 `: a gesture from the + // corner to itself, reported as `{ swiped: true }`. + runHdcShell.mockImplementation(async (_connectKey, command) => + command.startsWith("hidumper") + ? { stdout: `render resolution=0x0\npowerStatus=POWER_STATUS_ON`, exitCode: 0 } + : { stdout: "No Error", exitCode: 0 } + ); + + const err = await gestureSwipeTool.execute(services, base).then( + () => { + throw new Error("expected the swipe to reject, but it resolved"); + }, + (e: unknown) => e + ); + + expect(getFailureSignal(err)?.failure_stage).toBe("harmony_display_zero"); + expect((err as Error).message).toContain("0x0 display"); + expect(runHdcShell.mock.calls.filter(([, c]) => c.startsWith("uitest uiInput"))).toHaveLength( + 0 + ); + }); + + it("reads the display on its own small ceiling and gives the injection the rest", async () => { + // Two legs on `UITEST_TIMEOUT_MS` put a swipe's worst case at 40s, above the + // 30s at which the MCP client (`FETCH_TIMEOUT_MS` in argent-mcp) aborts a + // call and REPLAYS it — a second scroll of a list the caller believes never + // moved. The read gets a ceiling sized to what it was measured at, and the + // injection gets what is left of the one shared budget. + const READ_MS = 60; + runHdcShell.mockImplementation(async (_connectKey, command) => { + if (!command.startsWith("hidumper")) return { stdout: "No Error", exitCode: 0 }; + await new Promise((r) => setTimeout(r, READ_MS)); + return { + stdout: `render resolution=${DISPLAY_WIDTH}x${DISPLAY_HEIGHT}\npowerStatus=POWER_STATUS_ON`, + exitCode: 0, + }; + }); + + await gestureSwipeTool.execute(services, { ...base, durationMs: 500 }); + + const read = runHdcShell.mock.calls.find(([, c]) => c.startsWith("hidumper")); + expect(read?.[2]).toBe(HARMONY_DISPLAY_TIMEOUT_MS); + const injection = runHdcShell.mock.calls.find(([, c]) => c.startsWith("uitest uiInput")); + expect(injection?.[2]).toBeLessThanOrEqual(HARMONY_INTERACTION_TIMEOUT_MS - READ_MS); + expect(injection?.[2]).toBeGreaterThan(0); + // The magnitudes themselves, not just the wiring: the whole interaction has + // to finish inside the MCP client's 30s cap, and the read has to stay a + // small slice of it — it was measured at 50-190ms, against an `hdc shell` + // round trip of 0.1-0.8s. + expect(HARMONY_INTERACTION_TIMEOUT_MS).toBeLessThanOrEqual(25_000); + expect(HARMONY_DISPLAY_TIMEOUT_MS).toBeLessThanOrEqual(5_000); + }); + it("does not declare the simulator-server service for a HarmonyOS target", () => { // Resolving the blueprint would spawn a backend this path never uses and // block on its ready-wait before the hdc call could start. diff --git a/packages/tool-server/test/gesture-tap-harmony.test.ts b/packages/tool-server/test/gesture-tap-harmony.test.ts index 45505b9bd8..7e49fbfb6e 100644 --- a/packages/tool-server/test/gesture-tap-harmony.test.ts +++ b/packages/tool-server/test/gesture-tap-harmony.test.ts @@ -17,7 +17,11 @@ vi.mock("../src/utils/check-deps", async (importOriginal) => ({ })); import { gestureTapTool } from "../src/tools/gesture-tap"; -import { harmonyDisplay, harmonyTouch } from "../src/utils/harmony-uitest"; +import { + HARMONY_INTERACTION_TIMEOUT_MS, + harmonyDisplay, + harmonyTouch, +} from "../src/utils/harmony-uitest"; import { ensureDep } from "../src/utils/check-deps"; const CONNECT_KEY = "025DEK236V035771"; @@ -125,6 +129,56 @@ describe("gesture-tap on HarmonyOS", () => { expect((err as Error).message).toMatch(/Wake it with `button` \(power\)/); expect(harmonyTouch).not.toHaveBeenCalled(); }); + + it("refuses to tap when the render service reports a 0x0 display", async () => { + // What the render service prints while the guest's compositor is still + // coming up. `toDevicePoint` clamps every normalized coordinate into a + // 0-wide, 0-tall panel, so the tap below would go out as `uiInput click 0 0` + // — a corner tap the caller never asked for, reported as `{ tapped: true }`. + vi.mocked(harmonyDisplay).mockResolvedValue({ width: 0, height: 0, screenOn: true }); + + const err = await gestureTapTool + .execute(noServices, { udid: HARMONY_UDID, x: 0.83, y: 0.42 }) + .then( + () => { + throw new Error("expected the tap to reject, but it resolved"); + }, + (e: unknown) => e + ); + + expect(getFailureSignal(err)?.failure_stage).toBe("harmony_display_zero"); + expect((err as Error).message).toContain("0x0 display"); + expect(harmonyTouch).not.toHaveBeenCalled(); + }); + + it("spends ONE budget across the display read and every click of a multi-tap", async () => { + // A display read on `UITEST_TIMEOUT_MS` plus a fresh ceiling per injection + // puts a triple-tap at 80s, and the MCP client aborts at 30s and REPLAYS the + // call — more taps on the same point, for taps the caller believes never + // landed. Every leg comes out of one deadline instead, so each click is + // handed strictly less than the one before it. + const READ_MS = 60; + vi.mocked(harmonyDisplay).mockImplementation(async () => { + await new Promise((r) => setTimeout(r, READ_MS)); + return DISPLAY; + }); + + await gestureTapTool.execute(noServices, { + udid: HARMONY_UDID, + x: 0.5, + y: 0.5, + clickCount: 3, + }); + + const budgets = vi.mocked(harmonyTouch).mock.calls.map((c) => c[3]); + expect(budgets).toHaveLength(3); + // The read is charged first… + expect(budgets[0]).toBeLessThanOrEqual(HARMONY_INTERACTION_TIMEOUT_MS - READ_MS); + // …and so is each 100ms inter-tap gap, so the last click cannot be handed a + // ceiling that ignores the two before it. + expect(budgets[2]).toBeLessThanOrEqual(budgets[0] - 2 * 100); + expect(budgets[2]).toBeGreaterThan(0); + }); }); describe("gesture-tap service declaration", () => { diff --git a/packages/tool-server/test/harmony-describe.test.ts b/packages/tool-server/test/harmony-describe.test.ts index bd1d791a53..a2beebd892 100644 --- a/packages/tool-server/test/harmony-describe.test.ts +++ b/packages/tool-server/test/harmony-describe.test.ts @@ -3,7 +3,8 @@ import { harmonyDisplay, harmonyDumpLayout } from "../src/utils/harmony-uitest"; import { describeHarmony } from "../src/tools/describe/platforms/harmony"; import { formatDescribeTree } from "../src/tools/describe/format-tree"; -vi.mock("../src/utils/harmony-uitest", () => ({ +vi.mock("../src/utils/harmony-uitest", async (importOriginal) => ({ + ...(await importOriginal()), harmonyDisplay: vi.fn(), harmonyDumpLayout: vi.fn(), })); diff --git a/packages/tool-server/test/harmony-hdc.test.ts b/packages/tool-server/test/harmony-hdc.test.ts index 42f28b7e03..22d7380880 100644 --- a/packages/tool-server/test/harmony-hdc.test.ts +++ b/packages/tool-server/test/harmony-hdc.test.ts @@ -1,6 +1,7 @@ import { describe, it, expect } from "vitest"; +import { getFailureSignal } from "@argent/registry"; import { hdcFailure, shellQuote } from "../src/utils/harmony-hdc"; -import { toDevicePoint } from "../src/utils/harmony-uitest"; +import { assertHarmonyDisplayReady, toDevicePoint } from "../src/utils/harmony-uitest"; describe("hdcFailure", () => { it("reports the `[Fail]` line hdc printed", () => { @@ -80,4 +81,62 @@ describe("toDevicePoint", () => { it("puts the origin at the first pixel", () => { expect(toDevicePoint(0, 0, display)).toEqual({ x: 0, y: 0 }); }); + + it("collapses every point onto the origin when the display has no size", () => { + // Not a behaviour to rely on — the reason `assertHarmonyDisplayReady` refuses + // a 0x0 read before any of this runs. The clamp has no in-range pixel to + // pick, so a tap anywhere on the screen becomes a tap on the top-left one. + expect(toDevicePoint(0.83, 0.42, { width: 0, height: 0 })).toEqual({ x: 0, y: 0 }); + }); +}); + +describe("assertHarmonyDisplayReady", () => { + const awake = { width: 1216, height: 2688, screenOn: true }; + + it("passes a panel that has a size and is on", () => { + expect(() => assertHarmonyDisplayReady(awake, "tap")).not.toThrow(); + }); + + it("refuses a non-positive panel, naming the size the render service reported", () => { + // `render resolution=0x0` is what a guest prints while its compositor is + // still coming up, and `harmonyDisplay` parses it without complaint. Every + // coordinate then collapses onto the origin (above) while `uitest uiInput` + // answers `No Error`, so the injection reports the tap that was asked for + // and lands somewhere else entirely. + const err = (() => { + try { + assertHarmonyDisplayReady({ width: 0, height: 0, screenOn: true }, "tap"); + } catch (e) { + return e as Error; + } + throw new Error("expected a 0x0 display to be refused"); + })(); + + expect(getFailureSignal(err)?.failure_stage).toBe("harmony_display_zero"); + expect(err.message).toContain("0x0 display"); + expect(err.message).toContain("Cannot tap"); + }); + + it("refuses a half-read panel too", () => { + // One dimension is enough: a 1216x0 read makes every y collapse onto the top + // row while x still scales, which reads as a working tap on the wrong element + // rather than an obviously broken one. + expect(() => assertHarmonyDisplayReady({ ...awake, height: 0 }, "swipe")).toThrow( + /1216x0 display/ + ); + }); + + it("refuses a suspended panel with the wake-it advice", () => { + const err = (() => { + try { + assertHarmonyDisplayReady({ ...awake, screenOn: false }, "type"); + } catch (e) { + return e as Error; + } + throw new Error("expected a suspended display to be refused"); + })(); + + expect(getFailureSignal(err)?.failure_stage).toBe("harmony_screen_off"); + expect(err.message).toMatch(/Wake it with `button` \(power\)/); + }); }); diff --git a/packages/tool-server/test/harmony-uitest-queue.test.ts b/packages/tool-server/test/harmony-uitest-queue.test.ts index 670ec0c95e..09011b987c 100644 --- a/packages/tool-server/test/harmony-uitest-queue.test.ts +++ b/packages/tool-server/test/harmony-uitest-queue.test.ts @@ -180,32 +180,103 @@ describe("the per-device uitest queue", () => { }); describe("a caller's own ceiling reaches the device", () => { + const dumpPath = (tag: string) => + join(tmpdir(), `argent-uitest-budget-${process.pid}-${tag}.json`); + + /** Serve a `uitest` call that takes `runMs`, recording the ceiling it was given. */ + function deviceTakes(runMs: number): { + shellTimeouts: (number | undefined)[]; + fetchTimeouts: (number | undefined)[]; + } { + const shellTimeouts: (number | undefined)[] = []; + const fetchTimeouts: (number | undefined)[] = []; + runHdcShell.mockImplementation(async (_key, command, timeoutMs) => { + if (!command.startsWith("uitest ")) return { stdout: "", exitCode: 0 }; + shellTimeouts.push(timeoutMs); + await new Promise((r) => setTimeout(r, runMs)); + return { stdout: "", exitCode: 0 }; + }); + hdcFileRecv.mockImplementation(async (_key, _remote, localPath, timeoutMs) => { + fetchTimeouts.push(timeoutMs); + writeFileSync(localPath, "{}"); + }); + return { shellTimeouts, fetchTimeouts }; + } + // A `timeoutMs` accepted here and dropped leaves every clamp upstream — // boot-device's probe, the wait tools' polls — computing a bound nothing // enforces, which is the state this whole parameter exists to leave behind. // Both round trips count: the fetch is the half that hangs when a device // starts to go away, and it defaults to 30s, above every caller's budget. it("bounds the `uitest` call and the fetch that follows it", async () => { - const shellTimeouts: (number | undefined)[] = []; + const RUN_MS = 200; + const { shellTimeouts, fetchTimeouts } = deviceTakes(RUN_MS); + + await harmonyDumpLayout("dev-a", dumpPath("one"), 900); + + // Nothing ran before it, so the capture gets ~the whole budget… + expect(shellTimeouts).toHaveLength(1); + expect(shellTimeouts[0]).toBeGreaterThan(800); + expect(shellTimeouts[0]).toBeLessThanOrEqual(900); + // …and the fetch gets what the capture left, rather than a sliver above zero + // or a fresh 30s default. + expect(fetchTimeouts).toHaveLength(1); + expect(fetchTimeouts[0]).toBeLessThanOrEqual(900 - RUN_MS); + expect(fetchTimeouts[0]).toBeGreaterThan(900 - RUN_MS - 150); + }); + + it("charges the wait for another caller's `uitest` to the budget, not to the fetch alone", async () => { + // Two callers on one device, each with the same budget. The second waits out + // the first in `enqueueUitest` — time nothing was charging — and then, if the + // capture is handed a fresh full ceiling, spends the whole budget on the + // capture and leaves the fetch a 1ms SIGKILL reported as a device that hung, + // with the capture still sitting on the device. + const BUDGET_MS = 500; + const RUN_MS = 300; + const { shellTimeouts, fetchTimeouts } = deviceTakes(RUN_MS); + + const first = harmonyDumpLayout("dev-a", dumpPath("first"), BUDGET_MS); + const second = harmonyDumpLayout("dev-a", dumpPath("second"), BUDGET_MS); + + await expect(first).resolves.toEqual({}); + const outcome = await second.then( + () => null, + (e: unknown) => e as Error + ); + + // The queued caller's capture is bounded by what its budget had left after + // the wait, not by the budget it started with. + expect(shellTimeouts).toHaveLength(2); + expect(shellTimeouts[1]).toBeLessThanOrEqual(BUDGET_MS - RUN_MS); + // And once that is spent, the fetch is not attempted at all: the caller is + // told the budget ran out instead of being handed a killed transfer. + expect(fetchTimeouts).toHaveLength(1); + expect(getFailureSignal(outcome)).toMatchObject({ + error_code: FAILURE_CODES.HARMONY_HDC_COMMAND_FAILED, + failure_stage: "harmony_budget_exhausted", + error_kind: "timeout", + }); + expect(outcome?.message).toContain("Ran out of time"); + }); + + it("bounds the cleanup delete, which runs after the budget is spent", async () => { + // The delete is deliberately outside the shared budget — a spent budget is + // no reason to leave a multi-hundred-KB capture on the device — but on + // `runHdcShell`'s 30s default a wedged daemon adds 30s to a call whose + // caller has already run out of time, on top of every ceiling before it. + const rmTimeouts: (number | undefined)[] = []; runHdcShell.mockImplementation(async (_key, command, timeoutMs) => { - if (command.startsWith("uitest ")) shellTimeouts.push(timeoutMs); + if (command.startsWith("rm -f ")) rmTimeouts.push(timeoutMs); return { stdout: "", exitCode: 0 }; }); - const fetchTimeouts: (number | undefined)[] = []; - hdcFileRecv.mockImplementation(async (_key, _remote, localPath, timeoutMs) => { - fetchTimeouts.push(timeoutMs); + hdcFileRecv.mockImplementation(async (_key, _remote, localPath) => { writeFileSync(localPath, "{}"); }); - await harmonyDumpLayout( - "dev-a", - join(tmpdir(), `argent-uitest-budget-${process.pid}.json`), - 900 - ); + await harmonyDumpLayout("dev-a", dumpPath("cleanup"), 900); - expect(shellTimeouts).toEqual([900]); - expect(fetchTimeouts).toHaveLength(1); - expect(fetchTimeouts[0]).toBeGreaterThan(0); - expect(fetchTimeouts[0]).toBeLessThanOrEqual(900); + // Sized against the 0.1-0.8s an `hdc shell` round trip was measured at, and + // well under the MCP client's 30s abort. + expect(rmTimeouts).toEqual([5_000]); }); }); diff --git a/packages/tool-server/test/keyboard-key-order.test.ts b/packages/tool-server/test/keyboard-key-order.test.ts index e25d9773a5..38a09cc73e 100644 --- a/packages/tool-server/test/keyboard-key-order.test.ts +++ b/packages/tool-server/test/keyboard-key-order.test.ts @@ -24,7 +24,12 @@ vi.mock("../src/utils/harmony-uitest", async (importOriginal) => ({ })); import { injectVegaNamedKey, injectVegaText } from "../src/utils/vega-input"; -import { harmonyDisplay, harmonyKeyEvent, harmonyTypeText } from "../src/utils/harmony-uitest"; +import { + HARMONY_INTERACTION_TIMEOUT_MS, + harmonyDisplay, + harmonyKeyEvent, + harmonyTypeText, +} from "../src/utils/harmony-uitest"; const IOS_SIM: DeviceInfo = { id: "TEST-UDID", platform: "ios", kind: "simulator" }; const CHROMIUM: DeviceInfo = { id: "chromium-cdp-9222", platform: "chromium", kind: "app" }; @@ -160,10 +165,14 @@ describe("keyboard text+key ordering", () => { ); expect(order).toEqual(["text", "key"]); - expect(harmonyTypeText).toHaveBeenCalledWith(HARMONY_CONNECT_KEY, "hi"); + expect(harmonyTypeText).toHaveBeenCalledWith(HARMONY_CONNECT_KEY, "hi", expect.any(Number)); // The keyID, not the key name — `uitest` names only Home/Back/Power and // takes a raw number for everything else. - expect(harmonyKeyEvent).toHaveBeenCalledWith(HARMONY_CONNECT_KEY, HARMONY_ENTER_KEYID); + expect(harmonyKeyEvent).toHaveBeenCalledWith( + HARMONY_CONNECT_KEY, + HARMONY_ENTER_KEYID, + expect.any(Number) + ); // Two characters plus the key press. expect(result.keys).toBe(3); }); @@ -204,7 +213,11 @@ describe("keyboard text+key ordering", () => { await harmonyImpl.handler({}, { udid: HARMONY.id, key: "Enter" }, HARMONY); - expect(harmonyKeyEvent).toHaveBeenCalledWith(HARMONY_CONNECT_KEY, HARMONY_ENTER_KEYID); + expect(harmonyKeyEvent).toHaveBeenCalledWith( + HARMONY_CONNECT_KEY, + HARMONY_ENTER_KEYID, + expect.any(Number) + ); }); it("harmony: sends each supported key its own measured keyID", async () => { @@ -227,7 +240,70 @@ describe("keyboard text+key ordering", () => { for (const [key, keyId] of Object.entries(expected)) { vi.mocked(harmonyKeyEvent).mockClear(); await harmonyImpl.handler({}, { udid: HARMONY.id, key }, HARMONY); - expect(harmonyKeyEvent, key).toHaveBeenCalledWith(HARMONY_CONNECT_KEY, keyId); + expect(harmonyKeyEvent, key).toHaveBeenCalledWith( + HARMONY_CONNECT_KEY, + keyId, + expect.any(Number) + ); } }); + + it("harmony: no-ops on an empty request (neither key nor text), with zero device traffic", async () => { + // The schema leaves both `key` and `text` optional with no refinement, so an + // empty request is a no-op returning { typed:"", keys:0 } — the same + // contract every other keyboard backend follows. Reaching the device first + // costs a round trip for a step that injects nothing, and fails the whole + // sequence when the panel happens to be suspended. + vi.mocked(harmonyDisplay).mockClear(); + vi.mocked(harmonyTypeText).mockClear(); + vi.mocked(harmonyKeyEvent).mockClear(); + + const result = await harmonyImpl.handler({}, { udid: HARMONY.id }, HARMONY); + + expect(result).toEqual({ typed: "", keys: 0 }); + expect(harmonyDisplay).not.toHaveBeenCalled(); + expect(harmonyTypeText).not.toHaveBeenCalled(); + expect(harmonyKeyEvent).not.toHaveBeenCalled(); + }); + + it("harmony: refuses to type when the render service reports a 0x0 display", async () => { + // A guest whose compositor has not come up answers `render resolution=0x0`. + // `uitest uiInput text` would report `No Error` for characters that reached + // no field, so this read is refused for typing exactly as it is for a tap. + vi.mocked(harmonyTypeText).mockClear(); + vi.mocked(harmonyKeyEvent).mockClear(); + vi.mocked(harmonyDisplay).mockResolvedValueOnce({ width: 0, height: 0, screenOn: true }); + + await expect( + harmonyImpl.handler({}, { udid: HARMONY.id, text: "hi", key: "enter" }, HARMONY) + ).rejects.toThrow(/0x0 display/); + expect(harmonyTypeText).not.toHaveBeenCalled(); + expect(harmonyKeyEvent).not.toHaveBeenCalled(); + }); + + it("harmony: spends ONE budget across the display read, the text and the key", async () => { + // Three legs on a ceiling each put a type-then-submit at 60s, and the MCP + // client aborts at 30s and REPLAYS — retyping into a field it cannot see. + // Each leg is charged against the one deadline, so the key gets strictly + // less than the text that ran before it. + const LEG_MS = 60; + vi.mocked(harmonyDisplay).mockImplementationOnce(async () => { + await new Promise((r) => setTimeout(r, LEG_MS)); + return { width: 1216, height: 2688, screenOn: true }; + }); + vi.mocked(harmonyTypeText) + .mockClear() + .mockImplementationOnce(async () => { + await new Promise((r) => setTimeout(r, LEG_MS)); + }); + vi.mocked(harmonyKeyEvent).mockClear(); + + await harmonyImpl.handler({}, { udid: HARMONY.id, text: "hi", key: "enter" }, HARMONY); + + const textBudget = vi.mocked(harmonyTypeText).mock.calls[0][2]; + const keyBudget = vi.mocked(harmonyKeyEvent).mock.calls[0][2]; + expect(textBudget).toBeLessThanOrEqual(HARMONY_INTERACTION_TIMEOUT_MS - LEG_MS); + expect(keyBudget).toBeLessThanOrEqual(textBudget - LEG_MS); + expect(keyBudget).toBeGreaterThan(0); + }); }); From 2322bfc20a1d243eb189e66a6e5642d7f07a0cb3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 22:26:03 +0200 Subject: [PATCH 061/110] fix(harmony): fail a stop closed, and stop reading an unreadable listing as an answer `terminateHarmonyApp` classified success by the absence of a lowercase `error:`, where both siblings in the same module - and the module header's own stated policy - require the success line. `aa force-stop` has one: measured over `hdc shell`, a running app and an already-stopped one both answer `force stop process successfully.`, while a rejected invocation prints a bare `usage:` line and a coded failure puts its headline under a capital `Error Code:`. Both of those read as a successful stop, so `restart-app` ran `aa start` on the app it had not stopped and reported `restarted: true` for the old process. Nothing pinned it: all five existing cases stay green under the positive form. `waitForHarmonyInstanceStopped` had the same conflation the pre-start lookup was fixed for one commit ago: `listHarmonyInstances` answers `[]` for a `-list` that ran and printed a diagnostic, and a stopped instance keeps a row of its own, so an answer with no rows is the unreadable one - not proof the instance went down. Read as a shutdown, a `force` restart went straight on to `-start` an instance still running, which answers only that it is already running. `HARMONY_NO_INSTANCES` stops asserting the host has no instances for the same reading. The per-instance `-start` log mapped every unsafe character onto `_`, so "Phone 1" and "Phone_1" shared one file that each boot opens `"w"`: a second boot truncated the first's diagnostic mid-start, and the failure that followed was reported with nothing printed. Escaping instead keeps the mapping one-to-one. Also: the `aa`/`bm` steps take explicit ceilings (8s stop, 6s dump, 10s start - `aa start` measured at 50-100ms, and it returns once the ability is asked for, not once it is drawn), so restart-app's worst case is 24s rather than 90s, under the MCP client's 30s abort-and-replay cap; and `openHarmonyUrl` discriminates `not_found` from `subprocess` off the coded line (`10103101` measured for a URI no installed ability claims) rather than calling every failure a missing app. --- .../src/tools/devices/boot-device.ts | 33 ++++-- .../tool-server/src/utils/harmony-apps.ts | 102 ++++++++++++++---- .../test/boot-device-harmony.test.ts | 64 ++++++++++- .../tool-server/test/harmony-apps.test.ts | 81 ++++++++++++++ 4 files changed, 249 insertions(+), 31 deletions(-) diff --git a/packages/tool-server/src/tools/devices/boot-device.ts b/packages/tool-server/src/tools/devices/boot-device.ts index 3d90f16ce3..a8569c7b80 100644 --- a/packages/tool-server/src/tools/devices/boot-device.ts +++ b/packages/tool-server/src/tools/devices/boot-device.ts @@ -1519,15 +1519,18 @@ const HARMONY_IMAGE_RESTRICTION = "the image; an instance has to be created in DevEco Studio on a host that can download one."; /** - * Said when the start failed and the manager lists no instances at all. Zero - * instances is equally what a host that simply has not created one yet looks - * like — including inside mainland China, where the download works — so this - * states what was observed and offers the restriction as a possible cause - * rather than asserting it the way {@link HARMONY_IMAGE_RESTRICTION} does. + * Said when the start failed and the listing came back empty. Empty is what a + * host that simply has not created an instance yet looks like — including + * inside mainland China, where the download works — and equally what a `-list` + * that ran and printed a diagnostic looks like, since `listHarmonyInstances` + * answers `[]` for that too. So this states what was observed and offers the + * restriction as a possible cause rather than asserting it the way + * {@link HARMONY_IMAGE_RESTRICTION} does. */ const HARMONY_NO_INSTANCES = - "The emulator manager lists no HarmonyOS instances at all; create one in DevEco Studio before " + - "starting it. If creating one fails for want of an image, note that Huawei serves HarmonyOS " + + "The emulator manager listed no HarmonyOS instances — either none has been created, or the " + + "listing itself failed; `Emulator -list -details` shows which. Create one in DevEco Studio if " + + "there is none. If creating one fails for want of an image, note that Huawei serves HarmonyOS " + "emulator images only within mainland China."; /** Said when the instance was already running, its target therefore unidentifiable. */ @@ -1762,7 +1765,14 @@ async function startHarmonyEmulator( failure_command: "deveco_emulator", }); } - const logPath = join(tmpdir(), `argent-harmony-${instanceName.replace(/[^\w.-]/g, "_")}.log`); + // Escaped rather than blanked to `_`, because the log is opened `"w"`: + // collapsing every unsafe character onto one would point two instances the + // manager keeps apart ("Phone 1", "Phone_1") at a single file, and a boot of + // the second truncates the first's diagnostic mid-start — leaving a failure + // to be reported with nothing printed. `%` is itself escaped, so the mapping + // stays one-to-one. + const safeName = instanceName.replace(/[^\w.-]/g, (c) => `%${c.codePointAt(0)!.toString(16)}`); + const logPath = join(tmpdir(), `argent-harmony-${safeName}.log`); const logFd = openSync(logPath, "w"); const child = spawn(bin, ["-start", instanceName], { detached: true, @@ -2072,7 +2082,12 @@ async function waitForHarmonyInstanceStopped(name: string, deadline: number): Pr const instances = await listHarmonyInstances(Math.min(HARMONY_LIST_TIMEOUT_MS, budget)).catch( () => null ); - if (instances && !instances.some((i) => i.name === name && i.running)) return true; + // Only a listing WITH ROWS can say the instance stopped. A `-list` that ran + // and printed a diagnostic resolves to `[]` rather than throwing, and the + // stopped instance itself is still a row (with `isRunning` false), so an + // empty answer is the unreadable one — and reading it as "gone" would + // `-start` into an instance still running. + if (instances?.length && !instances.some((i) => i.name === name && i.running)) return true; const remaining = deadline - Date.now(); if (remaining <= 0) return false; await new Promise((r) => setTimeout(r, Math.min(HARMONY_TARGET_POLL_MS, remaining))); diff --git a/packages/tool-server/src/utils/harmony-apps.ts b/packages/tool-server/src/utils/harmony-apps.ts index 3784889473..66ec0053da 100644 --- a/packages/tool-server/src/utils/harmony-apps.ts +++ b/packages/tool-server/src/utils/harmony-apps.ts @@ -17,6 +17,38 @@ import { runHdcShell, shellQuote } from "./harmony-hdc"; /** Printed verbatim by `aa start` on success. */ const AA_SUCCESS = "start ability successfully."; +/** + * Printed verbatim by `aa force-stop` on success — measured over `hdc shell` on + * a HarmonyOS emulator, for a running app and an already-stopped one alike. + * + * Matched positively because `aa`'s failures share no marker: the same emulator + * answers a rejected invocation with a bare `usage: aa force-stop …` line, and + * writes its coded diagnostic under a capital `Error Code:` that a scan for the + * lowercase `error:` headline misses. + */ +const AA_STOP_SUCCESS = "force stop process successfully."; + +/** + * Per-call ceilings for the `aa`/`bm` steps. + * + * None of launch-app, restart-app or open-url declares `longRunning`, so the MCP + * client aborts a call at 30s and *replays* it while the abandoned `hdc` + * children keep running — a second `aa start` racing the first. `runHdcShell`'s + * own 30s default would give restart-app's stop-dump-start 90s between them, so + * each step is capped instead: 24s worst case for restart-app, 16s for + * launch-app, 10s for open-url. + * + * The split follows the Android twins' magnitudes (`am force-stop` 15s, + * `cmd package resolve-activity` 10s, `am start -W` 30s) scaled to fit all three + * in one budget. Every one of them is far above the work itself — measured over + * `hdc shell`, `aa start` answers in 50-100ms, returning once the ability has + * been asked for rather than once it is drawn — so what they bound is a wedged + * daemon, not a slow app. + */ +const AA_STOP_TIMEOUT_MS = 8_000; +const BM_DUMP_TIMEOUT_MS = 6_000; +const AA_START_TIMEOUT_MS = 10_000; + interface HarmonyBundleEntry { /** * The ability to launch, in the spelling `aa start` accepts — which is the @@ -67,7 +99,11 @@ export async function resolveHarmonyEntry( connectKey: string, bundleId: string ): Promise { - const { stdout } = await runHdcShell(connectKey, `bm dump -n ${shellQuote(bundleId)}`); + const { stdout } = await runHdcShell( + connectKey, + `bm dump -n ${shellQuote(bundleId)}`, + BM_DUMP_TIMEOUT_MS + ); const start = stdout.indexOf("{"); if (start === -1) { // `bm dump` prints a prose line for an unknown bundle rather than JSON. @@ -159,25 +195,33 @@ const AA_NO_SUCH_PACKAGE = "Error Code:10104002"; * `aa force-stop` takes the bundle name and, unlike `aa start`, needs no ability * — there is nothing to resolve. Stopping an app that is not running is not a * failure to it: measured on 6.0.1 against three never-launched bundles, it - * answers `force stop process successfully.` So every `error:` it does print is - * a real refusal and propagates — `restart-app` exists to guarantee a fresh - * process, and reporting `restarted: true` for an app still running with its old - * state is the one outcome it must not produce. + * answers `force stop process successfully.` So {@link AA_STOP_SUCCESS} is the + * whole verdict, read the way the launch path reads its own — a coded refusal, a + * diagnostic `runHdcShell` never saw because `aa` wrote it to stderr, or nothing + * at all are equally a stop that did not happen. `restart-app` exists to + * guarantee a fresh process, and reporting `restarted: true` for an app still + * running with its old state is the one outcome it must not produce. + * + * The bundle name is positional: `aa force-stop -b ` answers `10104002 + * … not installed` even for an installed, running app. */ export async function terminateHarmonyApp(connectKey: string, bundleId: string): Promise { - const { stdout } = await runHdcShell(connectKey, `aa force-stop ${shellQuote(bundleId)}`); - if (stdout.includes("error:")) { - throw new FailureError( - `Failed to stop '${bundleId}' on HarmonyOS device '${connectKey}': ${firstLine(stdout)}`, - { - error_code: FAILURE_CODES.HARMONY_ABILITY_START_FAILED, - failure_stage: "harmony_force_stop", - failure_area: "tool_server", - error_kind: stdout.includes(AA_NO_SUCH_PACKAGE) ? "not_found" : "subprocess", - failure_command: "hdc", - } - ); - } + const { stdout } = await runHdcShell( + connectKey, + `aa force-stop ${shellQuote(bundleId)}`, + AA_STOP_TIMEOUT_MS + ); + if (stdout.includes(AA_STOP_SUCCESS)) return; + throw new FailureError( + `Failed to stop '${bundleId}' on HarmonyOS device '${connectKey}': ${firstLine(stdout)}`, + { + error_code: FAILURE_CODES.HARMONY_ABILITY_START_FAILED, + failure_stage: "harmony_force_stop", + failure_area: "tool_server", + error_kind: stdout.includes(AA_NO_SUCH_PACKAGE) ? "not_found" : "subprocess", + failure_command: "hdc", + } + ); } /** @@ -213,11 +257,23 @@ export async function launchHarmonyApp(connectKey: string, bundleId: string): Pr const entry = await resolveHarmonyEntry(connectKey, bundleId); const { stdout } = await runHdcShell( connectKey, - `aa start -b ${shellQuote(bundleId)} -a ${shellQuote(entry.mainAbility)} -m ${shellQuote(entry.module)}` + `aa start -b ${shellQuote(bundleId)} -a ${shellQuote(entry.mainAbility)} -m ${shellQuote(entry.module)}`, + AA_START_TIMEOUT_MS ); assertAbilityStarted(stdout, connectKey, bundleId); } +/** + * What `aa start -U` says when no installed ability claims the scheme — the + * open-url counterpart of {@link AA_NO_SUCH_PACKAGE}, and the only failure here + * that is about something not existing. Every other code comes from a handler + * that does exist and refused, which is a subprocess failure. + * + * Prefixed like {@link AA_NO_SUCH_PACKAGE}: the URI is caller input, and a bare + * number could be forged by output quoting it. + */ +const AA_NO_IMPLICIT_MATCH = "Error Code:10103101"; + /** * Open a URI through whichever app claims it. * @@ -228,7 +284,11 @@ export async function launchHarmonyApp(connectKey: string, bundleId: string): Pr * discover a dialog sitting on the screen. */ export async function openHarmonyUrl(connectKey: string, url: string): Promise { - const { stdout } = await runHdcShell(connectKey, `aa start -U ${shellQuote(url)}`); + const { stdout } = await runHdcShell( + connectKey, + `aa start -U ${shellQuote(url)}`, + AA_START_TIMEOUT_MS + ); if (!stdout.includes(AA_SUCCESS)) { throw new FailureError( `HarmonyOS device '${connectKey}' could not open '${url}': ${firstLine(stdout)}. ` + @@ -238,7 +298,7 @@ export async function openHarmonyUrl(connectKey: string, url: string): Promise { vi.useFakeTimers(); const pending = boot({ bootTimeoutMs: 900_000 }); - const settled = expect(pending).rejects.toThrow(/create one in DevEco Studio/); + const settled = expect(pending).rejects.toThrow(/Create one in DevEco Studio if there is none/); await vi.advanceTimersByTimeAsync(1_000); expect(spawnMock).toHaveBeenCalledWith( expect.any(String), @@ -1302,6 +1306,64 @@ describe("boot-device — HarmonyOS emulator path", () => { expect(spawnMock).not.toHaveBeenCalled(); }); + it("keeps waiting through a stop-wait listing that answered no rows at all", async () => { + // The sibling case: `listHarmonyInstances` resolves `[]` for a `-list` that + // ran and printed a diagnostic, not only for a host with no instances — and + // an instance that stopped keeps a row of its own, with `isRunning` false. + // So no rows is the unreadable answer, and taking it for a shutdown sends + // `-start` at an instance still up. + let listed = 0; + listHarmonyInstances.mockImplementation(() => { + listed += 1; + if (listed === 1) + return Promise.resolve([ + { name: INSTANCE, deviceType: "Phone", osVersion: null, running: true, display: PANEL }, + ]); + return Promise.resolve([]); + }); + vi.useFakeTimers(); + + const pending = boot({ force: true, bootTimeoutMs: 30_000 }); + const settled = expect(pending).rejects.toThrow(/still running when the 30s budget ran out/); + await vi.advanceTimersByTimeAsync(31_000); + await settled; + + expect(spawnMock).not.toHaveBeenCalled(); + }); + + it("gives two instances that differ only in an unsafe character their own start logs", async () => { + // The log is opened `"w"`, so collapsing every unsafe character onto `_` + // would point "Phone 1" and "Phone_1" at one file — and a boot of the second + // truncates the first's diagnostic while its manager is still dying, leaving + // that failure to be reported with nothing printed. + const children: FakeEmulator[] = []; + spawnMock.mockImplementation(() => { + const spawned = new FakeEmulator(); + children.push(spawned); + return spawned; + }); + listHarmonyInstances.mockResolvedValue([ + { name: SPACED_INSTANCE, deviceType: "Phone", osVersion: null, running: false }, + { name: INSTANCE, deviceType: "Phone", osVersion: null, running: false }, + ]); + vi.useFakeTimers(); + + const spaced = boot({ harmonyInstance: SPACED_INSTANCE, bootTimeoutMs: 900_000 }); + const spacedSettled = expect(spaced).rejects.toThrow(/Cannot find image for the profile/); + await vi.advanceTimersByTimeAsync(1_000); + writeFileSync(spacedLogPath, "Cannot find image for the profile\n"); + + const underscored = boot({ bootTimeoutMs: 900_000 }).catch((e: Error) => e); + await vi.advanceTimersByTimeAsync(1_000); + expect(children).toHaveLength(2); + + children[0].emit("exit", 1, null); + children[1].emit("exit", 1, null); + await vi.advanceTimersByTimeAsync(3_000); + await spacedSettled; + await underscored; + }); + it("reports a manager that could not be spawned instead of taking the server down", async () => { // `spawn` fails asynchronously for ENOENT/EACCES. An unhandled `error` event // is an uncaughtException, which this process routes to crashShutdown — so diff --git a/packages/tool-server/test/harmony-apps.test.ts b/packages/tool-server/test/harmony-apps.test.ts index 6948e98d1c..f1cbe1b645 100644 --- a/packages/tool-server/test/harmony-apps.test.ts +++ b/packages/tool-server/test/harmony-apps.test.ts @@ -291,6 +291,17 @@ describe("terminateHarmonyApp", () => { const err = await terminateHarmonyApp("dev", "com.example.a10104002").catch((e: unknown) => e); expect(getFailureSignal(err as Error)?.error_kind).toBe("subprocess"); }); + + it("fails a stop that printed nothing recognisable", async () => { + // The success line is the only evidence the stop happened. `aa`'s `error:` + // headline is not a reliable stand-in for its absence: `runHdcShell` returns + // stdout only, so a diagnostic written to stderr arrives here as an empty + // string, and reading that as success hands `restart-app` a live process. + runHdcShell.mockResolvedValueOnce(ok("")); + await expect(terminateHarmonyApp("dev", "c")).rejects.toThrow( + "Failed to stop 'c' on HarmonyOS device 'dev': the ability assistant printed nothing" + ); + }); }); describe("launch-app on harmony", () => { @@ -364,6 +375,34 @@ describe("restart-app on harmony", () => { "aa start -b 'com.huawei.hmos.calculator' -a 'CalculatorAbility' -m 'phone'", ]); }); + + it("does not launch when the stop reported neither success nor an error", async () => { + // The outcome the tool exists to prevent: `aa start` on a still-running app + // foregrounds it and prints success, so a stop taken on faith turns into + // `restarted: true` for a process that never went away. + runHdcShell.mockResolvedValueOnce(ok("")); + await expect( + harmonyRestartImpl.handler({}, { udid: device.id, bundleId: "c" }, device) + ).rejects.toThrow(/Failed to stop 'c'/); + expect(runHdcShell).toHaveBeenCalledTimes(1); + }); + + it("bounds each step so the whole restart stays inside the MCP client's 30s cap", async () => { + // Past 30s the client aborts and replays the call while the abandoned hdc + // children keep running, so a second `aa start` races the first. Nothing + // here declares `longRunning`, and `runHdcShell`'s own default would give + // these three steps 90s between them. + runHdcShell + .mockResolvedValueOnce(ok("force stop process successfully.")) + .mockResolvedValueOnce(ok(CALCULATOR)) + .mockResolvedValueOnce(ok("start ability successfully.")); + await harmonyRestartImpl.handler({}, { udid: device.id, bundleId: "c" }, device); + const budgets = runHdcShell.mock.calls.map((c) => c[2]); + expect(budgets).toEqual([8_000, 6_000, 10_000]); + expect(budgets.reduce((total, ms) => total + (ms ?? Number.NaN), 0)).toBeLessThan( + 30_000 + ); + }); }); describe("openHarmonyUrl", () => { @@ -381,4 +420,46 @@ describe("openHarmonyUrl", () => { await openHarmonyUrl("dev", "app://x?a=1&b=2;id"); expect(runHdcShell.mock.calls[0][1]).toBe("aa start -U 'app://x?a=1&b=2;id'"); }); + + it("bounds the `aa start -U` call well inside the MCP client's 30s cap", async () => { + // open-url is one hdc call, but it takes its budget from the same ceiling as + // the launch path rather than `runHdcShell`'s 30s default — at the default a + // slow open is aborted and replayed, and the replay opens the URI twice. + runHdcShell.mockResolvedValueOnce(ok("start ability successfully.")); + await openHarmonyUrl("dev", "app://x"); + expect(runHdcShell.mock.calls[0][2]).toBe(10_000); + }); + + it("reports a URI no app claims as not_found", async () => { + runHdcShell.mockResolvedValueOnce( + ok( + "error: failed to start ability.\nError Code:10103101 Error Message:Failed to find a matching application for implicit launch." + ) + ); + const err = await openHarmonyUrl("dev", "nope://x").catch((e: unknown) => e); + expect(getFailureSignal(err as Error)?.error_kind).toBe("not_found"); + }); + + it("reports a claimed URI whose handler refused as a subprocess failure", async () => { + // A handler exists and would not start — `10104001` is the same code the + // launch path calls a subprocess failure. Calling it not_found tells the + // agent to go fix a URI that is fine. + runHdcShell.mockResolvedValueOnce( + ok( + "error: failed to start ability.\nError Code:10104001 Error Message:The specified ability does not exist." + ) + ); + const err = await openHarmonyUrl("dev", "app://x").catch((e: unknown) => e); + expect(getFailureSignal(err as Error)?.error_kind).toBe("subprocess"); + }); + + it("matches the coded line, not the digits wherever they appear", async () => { + runHdcShell.mockResolvedValueOnce( + ok( + "error: failed to start ability.\nAbility name: com.example.a10103101\nError Code:10104001 Error Message:The specified ability does not exist." + ) + ); + const err = await openHarmonyUrl("dev", "app://x").catch((e: unknown) => e); + expect(getFailureSignal(err as Error)?.error_kind).toBe("subprocess"); + }); }); From 7f734ebf0d64731259e5a31b0c8eab953dcda648 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ignacy=20=C5=81=C4=85tka?= Date: Fri, 14 Aug 2026 22:26:18 +0200 Subject: [PATCH 062/110] docs(harmony): say which tools the platform actually has, and name its devices The device-interact skill listed four tools refused on HarmonyOS and then routed an agent whose `describe` came back empty straight at four more that are refused the same way - `native-describe-screen`, `native-view-at-point`, `native-user-interactable-view-at-point` and `debugger-component-tree` declare no harmony support, so each answers 400 before running. On HarmonyOS an empty tree is the expected read while an app starts, and the recovery is to call `describe` again. `await-ui-element` now returns a `note` on a successful match too - a suspended HarmonyOS panel still serves its last composited frame, so the selector matches a screen that is not live - but both prose sites still said a note only comes with a timeout, which is exactly the reading that makes an agent tap into a dead panel. The test-ui-flow skill advertises HarmonyOS, and its "record for replay" tip sent agents into a flow engine that has no harmony arm at all: a harmony device is never auto-resolved, `when: { platform: harmony }` is rejected at parse time, and a selector step dies in ui-tree matching. The tip now names the platforms that do replay. `flow-pixels`' capture comment claimed to route "exactly as the `screenshot` tool routes it", which stopped being true when screenshot gained its `uitest screenCap` arm. The always-on device-selection rule says to prefer a running device, and a plugged-in HarmonyOS phone sorts to the front of `list-devices` like a booted simulator - while the router entry says to pick one only when the user asked for HarmonyOS. The rule now says so where the sorting is described. Also: the README lede and capabilities bullet enumerate the platforms without the one the table below them lists; `screenshot`'s `rotation`, `screen-recording`'s unsupported list, `gesture-scroll`/`gesture-drag`'s "use gesture-swipe instead", `restart-app`'s `bundleId`, `run-sequence`'s keyboard row and `stop-all-simulator-servers`' unmatched-id reading all had a HarmonyOS-shaped hole; and `flow-device` rendered a harmony or remote-iOS device as `?` in the one message whose job is to name what the caller can pass, since both key their entries by `udid` rather than `serial`. --- README.md | 4 +- packages/skills/rules/argent.md | 2 +- .../skills/argent-device-interact/SKILL.md | 20 +++---- .../skills/argent-test-ui-flow/SKILL.md | 4 +- .../src/tools/await-ui-element/index.ts | 8 +-- .../src/tools/flows/flow-device.ts | 10 +++- .../src/tools/flows/flow-pixels.ts | 12 +++-- .../src/tools/gesture-drag/index.ts | 2 +- .../src/tools/gesture-scroll/index.ts | 2 +- .../tool-server/src/tools/keyboard/index.ts | 4 +- .../src/tools/restart-app/index.ts | 2 +- .../src/tools/run-sequence/index.ts | 2 +- .../screen-recording-start.ts | 2 +- .../simulator/stop-all-simulator-servers.ts | 2 +- .../tool-server/test/flow-device-ids.test.ts | 52 +++++++++++++++++++ server.json | 2 +- 16 files changed, 96 insertions(+), 34 deletions(-) create mode 100644 packages/tool-server/test/flow-device-ids.test.ts diff --git a/README.md b/README.md index e6ab81ff32..1382ed54f2 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@ [![Ad](https://swm-delivery.com/www/images/zone-gh-argent-2?n=1)](https://swm-delivery.com/www/delivery/ck-slug.php?zoneid=zone-gh-argent-2&n=1) [![Ad](https://swm-delivery.com/www/images/zone-gh-argent-3?n=1)](https://swm-delivery.com/www/delivery/ck-slug.php?zoneid=zone-gh-argent-3&n=1) -**[Argent](https://argent.swmansion.com)** is an **agentic toolkit** that gives your AI assistant direct access to iOS Simulators, Android emulators and physical devices, TVs (Apple TV, Android TV, Fire TV) and Electron/Chromium desktop and web apps. Ask it to tap a button, run a profiler or reproduce an issue manually - all from within your CLI, without switching context. +**[Argent](https://argent.swmansion.com)** is an **agentic toolkit** that gives your AI assistant direct access to iOS Simulators, Android emulators and physical devices, HarmonyOS phones and emulators, TVs (Apple TV, Android TV, Fire TV) and Electron/Chromium desktop and web apps. Ask it to tap a button, run a profiler or reproduce an issue manually - all from within your CLI, without switching context. ```bash npx @swmansion/argent@latest init @@ -33,7 +33,7 @@ Argent drives a growing set of targets through a single toolkit, each with the r ## Capabilities -- **Autonomous mobile, TV and desktop development** - Allow your agent to work with iOS, Android, TV and Electron/web apps on its own - let it build, open, interact with the app and debug it. Ask for reproducing issues, testing features manually, profiling your app and much more, without ever interrupting your work. +- **Autonomous mobile, TV and desktop development** - Allow your agent to work with iOS, Android, HarmonyOS, TV and Electron/web apps on its own - let it build, open, interact with the app and debug it. Ask for reproducing issues, testing features manually, profiling your app and much more, without ever interrupting your work. - **UI interaction** - Give your agent the full control toolkit - tapping, swiping, pinching, typing, gestures and hardware buttons on mobile; the directional remote on TV; mouse, scroll and drag on desktop/web. Let it navigate your app exactly as a user would, without lifting a finger. - **Record & replay flows** - Capture a sequence of interactions once and let your agent replay it deterministically, so manual repros and smoke tests become repeatable. - **Visual regression** - Diff two screenshots (or a saved baseline against a live capture) with OCR- and font-aware comparison to catch unintended UI changes. diff --git a/packages/skills/rules/argent.md b/packages/skills/rules/argent.md index 60b52779cf..3d89d54837 100644 --- a/packages/skills/rules/argent.md +++ b/packages/skills/rules/argent.md @@ -60,7 +60,7 @@ Before booting, running, or interacting with any app, call `list-devices` first Decision order: 1. **Explicit user intent** - choose the user named platform or device. Look for words "simulator" and "emulator". -2. **Prefer a running device.** iOS simulators - state `Booted` and Android devices - `state: "device"` come first in `list-devices`; Chromium (CDP) apps appear as `platform: "chromium"`, `state: "Running"`. +2. **Prefer a running device.** iOS simulators - state `Booted` and Android devices - `state: "device"` come first in `list-devices`; Chromium (CDP) apps appear as `platform: "chromium"`, `state: "Running"`. A plugged-in HarmonyOS phone - `platform: "harmony"`, `state: "Connected"` - is listed as ready alongside them, but take it only when the user asked for HarmonyOS. 3. **Single-platform project:** (per `argent-environment-inspector` flags `is_native_ios`/`is_native_android`, or RN with only one platform configured) → boot that platform. diff --git a/packages/skills/skills/argent-device-interact/SKILL.md b/packages/skills/skills/argent-device-interact/SKILL.md index 6fca303c87..cb25eb0983 100644 --- a/packages/skills/skills/argent-device-interact/SKILL.md +++ b/packages/skills/skills/argent-device-interact/SKILL.md @@ -21,7 +21,7 @@ For platform-specific caveats (Metro `adb reverse`, locked-screen describe error If you delegate simulator tasks to sub-agents, make sure they have MCP permissions. -Use `list-devices` to get a target id. Results are tagged with `platform` (`ios`, `ios-remote`, `android`, `chromium`, `vega`, or `harmony`); booted/ready devices come first. A `vega` target is a TV; drive it with `argent-tv-interact` (see the note above). A `harmony` target with `kind: "device"` is a HarmonyOS phone on `hdc`: the tools in this skill drive it, except `gesture-pinch`, `gesture-rotate`, `gesture-custom` and `rotate`, which are refused as unsupported on harmony — the device takes a whole gesture per call and exposes no multi-contact form of it. Use `button` for home/back/power only. A `harmony` target with `kind: "emulator"` is a DevEco Studio instance that only `boot-device` accepts; boot it and drive the `harmony-` id it returns. Pick the first entry that matches the platform you need — if none are ready, call `boot-device` with `udid` (iOS), `avdName` (Android), or `electronAppPath` (boots an Electron app as a `chromium` device). A Chromium browser already running with a CDP port shows up directly — no `boot-device` needed. See `argent-ios-simulator-setup` / `argent-android-emulator-setup` for full setup flow. +Use `list-devices` to get a target id. Results are tagged with `platform` (`ios`, `ios-remote`, `android`, `chromium`, `vega`, or `harmony`); booted/ready devices come first. A `vega` target is a TV; drive it with `argent-tv-interact` (see the note above). A `harmony` target with `kind: "device"` is a HarmonyOS phone on `hdc`: most of the tools in this skill drive it. `gesture-pinch`, `gesture-rotate`, `gesture-custom` and `rotate` are refused as unsupported on harmony — the device takes a whole gesture per call and exposes no multi-contact form of it — and so are the app-scoped inspectors `native-describe-screen`, `native-view-at-point`, `native-user-interactable-view-at-point` and `debugger-component-tree`, which declare no harmony support and answer `Tool '' is not supported on harmony device` before running, leaving `describe` and `screenshot` as the whole discovery surface there. Use `button` for home/back/power only. A `harmony` target with `kind: "emulator"` is a DevEco Studio instance that only `boot-device` accepts; boot it and drive the `harmony-` id it returns. Pick the first entry that matches the platform you need — if none are ready, call `boot-device` with `udid` (iOS), `avdName` (Android), or `electronAppPath` (boots an Electron app as a `chromium` device). A Chromium browser already running with a CDP port shows up directly — no `boot-device` needed. See `argent-ios-simulator-setup` / `argent-android-emulator-setup` for full setup flow. **Load tool schemas before first use.** Gesture tools (`gesture-tap`, `gesture-swipe`, `gesture-pinch`, `gesture-rotate`, `gesture-custom`) may be deferred — their parameter schemas are not loaded until fetched. Always use ToolSearch to load the schemas of all gesture tools you plan to use **before** calling any of them. If you skip this step, parameters may be coerced to strings instead of numbers, causing validation errors. @@ -72,7 +72,7 @@ Common schemes: `messages://`, `settings://`, `maps://?q=`, `tel://