Repository navigation
Merge upstream mobilecli through 1.0.22 and the keyboard commands - #2
Merged
Merged
Conversation
…le-next#419) * test: cover webview and app install/uninstall on both platforms The e2e suite never installed or uninstalled an app, and never touched a webview. It also reported far less coverage than it actually had: device commands run inside the daemon, and the daemon was still alive when `go tool covdata` read the counter directory, so none of its work counted. Tests, one group per platform against the playground app: - install and uninstall: uninstall, assert the package is gone from `apps list`, install the release build from disk, then assert every field `apps list` and `apps install` report today, so a dropped `appName` or `versionCode` fails here instead of shipping. - webview: tap into the playground webview screen, then list, url, title, eval, content, query, wait, goto, back, forward and reload, plus negative paths for an unknown webview id and an unknown device. - a separate group launches the system settings app and asserts `webview list` fails on a build the agent cannot attach to. It owns its setup, since `webview list` reads the foreground app and would otherwise break the state the playground group sets up once. `goto`, `back` and `forward` return before the navigation commits, and `wait --state load` can observe the outgoing page's load event, so url assertions poll instead of sleeping. The app restores its last webview url between launches, so the setup navigates back to the sample page rather than assuming a fresh one. Fixes: - Makefile stops the daemon before the suites, so the one they spawn inherits GOCOVERDIR, and again afterwards, so it exits cleanly and flushes its counters before covdata reads them. - `test:ios` ran `--project=simulator`, which made it a silent duplicate of `test:simulator` and left real-device coverage unreachable. The two are now one honest `test:ios-simulator`, and the Makefile runs it instead of the commented-out pair. Coverage: 26.3% -> 43.1% overall, with no change to the product code. commands/webview.go 0% -> 9 of 10 functions at 100%, commands/apps.go 0% -> 44-86%, devices/{android,ios}_webview.go 0% -> 74.6%. * test: poll the device instead of sleeping, and address review The android settings tests slept for a fixed 3 to 5 seconds and then asserted. On a slow emulator that guess was wrong often enough that unmodified main failed 1, 2 and 3 of them across three consecutive runs. test/poll.ts reads the device up to ten times, a second apart, and returns as soon as it agrees. Applied to every place that was sleeping on a guess: waiting for an app to reach the foreground, for the launcher to come back, for text to appear on screen, for the playground menu to draw, and for a webview to be reported. That made the suite both stable and faster: the emulator project went from about 1.1 minutes to 34 seconds, and it exposed a real race the fixed sleeps had been papering over, where a webview is listed before its page has a title. The webview setup now settles the page before any test reads it. Review fixes: - The Makefile ran the suites as separate recipe lines, so a failing suite skipped the daemon stop and lost its counters. They now run in one shell under an EXIT trap. - webViewCommandError parsed stdout as json unconditionally, so a timeout or a missing binary surfaced as "Unexpected end of JSON input" instead of the real failure. Empty stdout now rethrows the original. - The uninspectable-app group skips real devices and clears the settings task afterwards on android, and terminates settings on ios, so neither leaves a device in that state. - The two groups that walk a device through an ordered sequence are marked serial, so a failure retries the group rather than one step out of context. Not taken: replacing the mobilewright.dev sample page with a local fixture. The playground app hardcodes that url and loads it itself, so the dependency exists with or without these tests.
…e-next#423) getForegroundComponent() stopped at the first mCurrentFocus line in `dumpsys window displays`. A device with more than one display reports one mCurrentFocus per display and the unfocused ones read "mCurrentFocus=null", so on such a device the first line is frequently the null and foreground detection failed outright with "could not determine foreground app". Seen on Genymotion (Android 15), where it takes down every webview command, since all of them resolve the foreground package first. Keep scanning until a line actually parses. Extracted the parsing into parseForegroundComponent() so it can be tested without a device. Co-authored-by: Somkiat Khitwongwattana <somkiat.k@lmwn.com>
…obile-next#430) Recordings from the current devicekit-ios extension decoded as one frame with a duration of hundreds of hours. - the timecode SEI precedes the picture it stamps, but grouping attached it to the previous access unit, shifting every timestamp by one frame and emitting SPS/PPS as a sample without a slice - High profile streams contain B-frames, so timestamps arrive in decode order; writing pts == dts made the unsigned deltas wrap Timestamps are now held until the next slice claims them, parameter sets stay with their key frame, and dts walks the sorted timestamps with pts pushed back by the reorder delay so dts <= pts always holds.
…on, picker foreground, slow dump) (mobile-next#431)
…mobile-next#433) `apps install --force-resign` signed Frameworks/ and PlugIns/ but not dylibs in a bundle's root. Xcode debug builds keep the app's code there (<Name>.debug.dylib next to a stub executable, plus __preview.dylib), and signing the bundle does not sign them, so every re-signed debug build died at launch: Library not loaded: @rpath/Runner.debug.dylib ... mapped file has no cdhash, completely unsigned? Code has to be at least ad-hoc signed. Sign them before the bundle that contains them, for the app and for each app extension.
* fix(android): name the local port in adb forward AWS Device Farm runs the adb server on another host. Its adb wrapper tunnels each forwarded port to the test host by number, so a forward to "tcp:0" tunnels port 0 and nothing listens on the port adb reports. Every agent behind such a forward looked dead: the device server "did not start within 5s" although it was listening 200ms after launch, and live bitrate and key frame requests were refused. Pick a free local port ourselves and pass it to adb, in all four places that forwarded to tcp:0. * fix(android): check the listener address type * fix(android): retry the forward when the port is taken The local port is free when we probe it, but adb binds it a moment later, and with a remote adb server on another host. When adb reports it cannot bind the port, pick another one, at most three times. Other adb errors still fail at once.
Screen capture mirrors the display through the hidden static DisplayManager.createVirtualDisplay(name, w, h, displayId, surface). That method does not exist on Android 12 and 13: getMethod throws NoSuchMethodException, AvcServer exits before its first frame, and there is no video. Seen on a Pixel 6, a Galaxy A53 and a Pixel 7a. Fall back to SurfaceControl.createDisplay there. It is the older way to mirror a display from the shell user, and Android 14 removed it, so each version has exactly one of the two.
A sync frame is made from the encoder's next input frame. On a static screen the only input is REPEAT_PREVIOUS_FRAME_AFTER, and on Pixel encoders that stops after a moment. A key frame request then sets the flag and nothing comes out until the screen changes: a viewer that joins an idle phone waits for the status bar clock to tick. When a key frame is requested and the encoder has been silent for 500ms, recreate the mirror display on the same encoder surface. A new display is drawn once when it appears, which gives the encoder the frame it needs. A moving screen is left alone. Re-attaching the surface to the existing display was tried first and draws nothing on Android 14.
Set errcheck check-blank to false so '_ =' marks a reviewed, deliberate ignore, then handle the remaining sites by severity: - return errors that mean data loss or weak security: daemon socket chmod, unzip MkdirAll, Close() on files we just wrote, flutter VM request marshal - log leaked adb forwards and failed app termination before a data wipe - verbose-log failed HTTP/websocket writes and device-side cleanup - check type assertions instead of panicking
…le-next#449) * feat(avc2mp4): stream the muxer from disk ConvertFile muxes an Annex B stream from an io.ReadSeeker in two passes, timestamps first and samples second, so peak memory is one access unit plus the sample table, not the whole recording. NALSplitter cuts a chunked stream into whole NAL units and ScanNALUnits reads one from an io.Reader; both find the same units as ParseNALUnits. Convert stays for in-memory callers. * feat(screenrecord): stop recording at a size limit Recordings spool whole NAL units and stop before the input would pass 4,000,000,000 bytes, keeping the MP4 under the 32-bit size boundary. The capture unsubscribes, later frames are ignored, and the response reports endReason "size_limit". The recorded stream is now muxed with ConvertFile instead of being read into memory, and a failed disk write is reported instead of silently ending the recording. * test: cover restarting a recording right after stop A recording stopped and started again must free the one-at-a-time slot, and the new file must begin at parameter sets and a key frame while another subscriber keeps the shared stream running. * fix(screenrecord): bound the unterminated NAL unit A stream whose last unit never gets a closing start code kept growing the splitter buffer past the size limit and the scanner's 64 MiB cap. Stop the recording once it passes MaxNALSize and drop that unit, keeping every whole unit written before it.
mobile-next#451) Flutter debug-build dumps now carry a `key` field with the nearest enclosing `Key('...')` (ValueKey<String>), read from each render object's debugCreator chain. Widget keys never reach the semantics or platform accessibility trees, yet Flutter tooling (find.byKey, Patrol, flutter_driver) uses them as test ids. Also fix a lost identifier: a Semantics(identifier: ...) wrapping a control is its own id-only node above the control's node, and the inner node replaced it. The inner node now inherits the identifier unless it sets its own.
Replace context-less stdlib calls with their context-aware equivalents using context.Background(), matching stdlib's own implementations so behavior is unchanged.
* feat: report clickable on android elements * feat: report pane titles on android elements (mobile-next#525)
Brings in the 83 upstream commits since f714858, through 1.0.21. Conflicts: - cli/dump.go: upstream added `snapshot` next to our `--source` flag. Keep both, and give `snapshot` `--source` too, since it runs dump ui's RunE and would otherwise always read the default tree. - devices/ios_device_flutter.go: take upstream's mDNS fast path for real devices (mobile-next#432) and thread the TreeSource through it, so `--source semantics` no longer has to inject the agent over LLDB to find the VM service. mobile-next#432 also calls resolveDartVMServiceMDNS, which 6a3c6fd had removed when the simulator stopped letting mDNS choose the port. Rather than restore a second copy of the dns-sd loop, both callers now share lookupDartVMServiceMDNS: the simulator accepts only the port its process already owns, and the real device takes the first answer, as upstream does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…bile-next#546) * fix: re-resolve Offset.zero when it expires during a flutter dump The render-tree walk resolves Offset.zero once and passes its object id to every localToGlobal call. Each invoke and getObject adds ids to the isolate's VM-service id ring, so on a large tree the Offset.zero id is evicted partway through. Every later localToGlobal then fails with -32602 "invalid 'argumentIds' parameter", and emitLeaf drops the node without a trace. When localToGlobal fails that way, re-resolve Offset.zero (once for all concurrent walkers) and retry. Read the error's data.details so the cause can be told apart from other invalid params, and log dropped nodes in verbose mode. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix: re-resolve Offset.zero when invoke returns the Expired sentinel --------- Co-authored-by: Andreas Lüdeke <aluedeke@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: gmegidish <gil@megidish.net>
…hing one (mobile-next#547) * fix(android): build the in-app agent for x86_64 too and push the matching one The JVMTI agent was built only for arm64-v8a. On an x86_64 emulator, attaching it fails with `dlopen failed: ... is for EM_AARCH64 (183) instead of EM_X86_64 (62)`, so the agent never starts: no Flutter render tree and no webview agent, and every dump waits 5 s for it. Build mobilecli.so for arm64-v8a and x86_64, embed both, and push the one that matches the app process: the package's primaryCpuAbi, or ro.product.cpu.abi for an app without native code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(android): pick the agent by the device ABI, not the package's An arm64-only app on an x86_64 emulator runs as an x86_64 process with its native code translated, and ART won't load a JVMTI agent through that translation. Only a 32-bit package ABI overrides the device ABI. * fix(android): treat armeabi as a 32-bit package ABI too --------- Co-authored-by: Andreas Lüdeke <aluedeke@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: gmegidish <gil@megidish.net>
…or dump (mobile-next#523) * perf: stop shelling out to simctl on every simulator ui dump A dump of a simulator first asks whether the foreground app is Flutter, and that question cost two simctl calls every time: GetForegroundApp lists every installed app only to enrich a name and version the dump throws away, and isFlutterAppBundle reads the app container to look for Flutter.framework. Together ~0.6s, more than the accessibility dump of a light screen, paid again by every locator query, visibility poll and tap-by-ref. Ask the agent for the foreground app instead, which the dump needs anyway, and remember the Flutter verdict per process id, the way the real-device path already does. A relaunched or reinstalled app gets a new pid and is looked at again. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * perf: don't ask which view controller is on screen before every dump A dump first asks the agent which app is in front, to decide whether to read a Flutter render tree instead. That answer carries the class name of the view controller on screen, which the agent finds by snapshotting the hierarchy 60 levels deep — on an app with a large one that costs as much as the dump itself, around 5s, and the dump path throws it away. device.apps.foreground now takes a flag for it, so the Flutter probes ask for the app's identity alone. Callers that report the view controller, such as device.apps.foreground on the command line, are unchanged, and an older agent that ignores the flag still answers correctly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * refactor: request foreground app fields and do not cache failed bundle lookups GetActiveAppIdentity now asks the agent for fields [bundleId, pid] instead of a viewController flag, matching mobile-next/devicekit-ios#105. GetActiveAppInfo sends no fields, so the agent reports everything as before. isFlutterAppBundle returns an error when simctl cannot find the app bundle, and that is no longer remembered as a not-Flutter verdict, which would have kept a Flutter app on the accessibility dump until it was relaunched. * chore: update devicekit-ios agent to 0.0.36 0.0.36 includes mobile-next/devicekit-ios#105, which honours the fields this branch sends to device.apps.foreground. --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: gmegidish <gil@megidish.net>
…obile-next#550) govulncheck reported 10 standard library vulnerabilities reachable from our code (net/http, net/textproto, crypto/tls, crypto/x509), fixed in go1.26.9 and go1.26.10, plus golang.org/x/net issues fixed in v0.60.0 and v0.62.0.
Brings in the 11 upstream commits after 1.0.21, through the keyboard commands that followed 1.0.22. Conflicts: - agents/ios-real/agent.m: upstream removed it (mobile-next#559). Real devices now load the same agents/ios agent as the simulator over debugserver, with no LLDB expression evaluation, so 32f0d02's Xcode 26 LLDB fixes have nothing left to apply to. Take the deletion. - devices/ios_device_flutter.go: take upstream's comment, keep logging the TreeSource the dump used. - devices/simulator_flutter.go: take upstream's pid-cached Flutter check (mobile-next#523) and keep the TreeSource parameter. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
ローカル保管用の PR です。upstream (
mobile-next/mobilecli) には出しません。概要
fork 後に upstream で進んだ 94 コミット (1.0.11〜1.0.22 と、その後のキーボード系コマンド) を 2 回に分けてマージしました。
135aca7bbb50a8衝突の解消 (fork 側で手を入れた箇所)
cli/dump.gosnapshotとこちらの--sourceを両方残し、snapshotにも--sourceを付けた。snapshotはdump uiの処理をそのまま呼ぶので、フラグがないと常に既定の tree になるdevices/ios_device_flutter.goTreeSourceを通したdevices/simulator_flutter.goTreeSourceを残したdevices/simulator_flutter_process.goresolveDartVMServiceMDNSは 6a3c6fd で消していた。復活させるとdns-sdの読み取りが二重になるので、共通のlookupDartVMServiceMDNSにまとめた。シミュレータはプロセスが持つポートだけを受け入れ (6a3c6fd の挙動を維持)、実機は upstream と同じく最初の答えを使うagents/ios-real/agent.magents/iosを debugserver 経由で読み込むようになり、LLDB で式を評価しなくなったので、32f0d02 (Xcode 26 LLDB 対応) は不要になった。削除を採用取り込んだ主な upstream 変更
Offset.zeroの ID が失効し、以降の要素が黙って落ちていた問題の修正Key('...')を出すclickableを付けるMOBILECLI_DEVICE、snapshot、io fill、io keyboard status|hide確認方法
go build ./...とgo vet ./...は通りました。go test ./...は全パッケージ通りました。端末では、ジャンプTOON stg を対象にマージ後のバイナリで確認しました。iOS シミュレータ (iPhone 17 Pro / iOS 26.3)
ax/semantics/render/ 既定Offset.zeroの再取得が 21 回記録されており、mobile-next#546 で直った分semanticsの所要時間snapshot --source semanticsio tap @ref --source semanticsio keyboard status/hideio fillabcをテストに置き換えた--disable-service-auth-codesなし)lookupDartVMServiceMDNSは、ポートが一致すれば auth code を返し、違うポートでは返さないことを直接確認したAndroid エミュレータ (Pixel_8 / arm64 / Android 16)
ax/semantics/render/ 既定clickableOnboardingStartButtonにclickable: trueが付いたiOS 実機 (MB-1094 / iPhone 12 / iOS 18.6.2)
ax、tap、スクリーンショットaxは 2 回目以降 0.57s)semantics/render)NSBonjourServicesがない。そのため Flutter の取得は判定の段階でスキップされる (仕様どおりで、今使っているバイナリも同じ)未検証のもの
io keyboard hide。upstream によるとデバッグ可能なアプリでのみ動く注意
make agentsは、Makefile の既定値 (NDK 29.0.13113456、build-tools 36.0.0) がこのマシンに入っていないため、NDK_VERSION=29.0.14206865 BUILD_TOOLS_VERSION=36.1.0を指定して実行した。Android のエージェントはmobilecli-arm64-v8a.soとmobilecli-x86_64.soに分かれたので、古いagents/android/mobilecli.soは不要mobilecli agent install --force)。確認に使ったシミュレータは 0.0.26 から 0.0.37 に上げた。今使っているバイナリも 0.0.37 のエージェントで動くことを確認済み今後の改善
mobile-next#523 で Flutter かどうかの判定は速くなったが、fork で書き直した VM service の探索 (simctl、プロセス検索、
dns-sd) は、Flutter アプリを開いている間は dump のたびに走る。前面アプリのプロセス ID が取れるようになったので、見つけた URI をプロセス ID ごとに覚えておけば省ける。🤖 Generated with Claude Code