Skip to content
This repository was archived by the owner on Nov 9, 2026. It is now read-only.

Latest commit

 

History

History
208 lines (155 loc) · 12.8 KB

README.md

File metadata and controls

208 lines (155 loc) · 12.8 KB

English | 简体中文
Logo

iphone-control-toolkit

Let an AI agent operate a real iPhone plugged into a Linux machine — no Mac, no paid developer account, no jailbreak.

Release Docker iOS MCP CI License

🚀 Quick Start  ·  📖 Docs  ·  📦 Releases  ·  🐛 Issues

Important

We recommend iphone-control instead of this project. It is the successor to iphone-control-toolkit: one Rust binary instead of a container of separate tools, runs on Linux or macOS, supports iPhone and iPad, and keeps the MCP tools compatible, so existing prompts and clients keep working.

Note

Early release, tested on an iPhone 15 Pro Max (iOS 27.0.1) with an x86_64 Debian host. Other models, iOS 17–26 and arm64 hosts have not been tested yet — reports are welcome.

Overview

AI agents can drive browsers and desktops, but a real iPhone is harder: Apple's automation tools need a Mac and Xcode, and the open-source alternatives come as separate pieces that each need their own setup, signing and babysitting.

iphone-control-toolkit puts it all in one Docker container on Linux. It builds go-ios, pymobiledevice3, plumesign and WebDriverAgent from pinned sources, keeps them running, and exposes the phone to Claude Code or any MCP client.

Claude Code / any MCP client
        │  MCP over stdio (docker exec)
        ▼
┌──────────── Docker container on Linux ────────────┐
│  MCP server ─┬─ WebDriverAgent        (primary)    │
│              └─ CoreDevice HID        (fallback)   │
│  supervisor: reconnect · restart · re-sign · alert │
└───────────────────────┬────────────────────────────┘
                        │ USB
                     iPhone

Features

  • See and touch the screen: screenshots with region zoom, tap, swipe, long-press, text input (including Chinese), hardware buttons
  • Work with UI elements: read the element tree, tap buttons by name, launch and close apps, open URLs
  • Automatic fallback: when WebDriverAgent is down, screenshots, taps, swipes and keys switch to Apple's CoreDevice HID service (iOS 27+, no signing needed)
  • Runs unattended: recovers from replugging, phone reboots, tunnel drops and WebDriverAgent crashes; re-signs WebDriverAgent with a free Apple ID before the 7-day profile expires
  • Alerts: Telegram or Bark notifications when re-signing fails, the phone goes offline, or a passcode is needed after a reboot
  • Optional auto-unlock: the phone can lock normally; every passcode digit is verified, and a wrong passcode disables auto-unlock after one attempt
  • Live viewer: iphonectl viewer opens a desktop window with the phone's screen (up to ~25 fps over USB) that you control with mouse and keyboard, while the agent keeps working
  • Optional WebView debugging: Safari tabs and inspectable WebViews (Capacitor, Cordova, your own WKWebView) are exposed over the Chrome DevTools Protocol, so tools can read the DOM and console directly instead of going through screenshots
  • Diagnostics: iphonectl doctor checks every layer from USB to WebDriverAgent and tells you how to fix each failure

Quick Start

You need a Linux host with Docker (Compose v2.24+), an iPhone on iOS 17+ connected by a data USB cable, and a free Apple ID (use a dedicated one, not your main account).

git clone https://github.com/Derrors/iphone-control-toolkit.git
cd iphone-control-toolkit
bin/iphonectl setup          # guided setup; safe to re-run
bin/iphonectl mcp install    # register with Claude Code

setup pulls ghcr.io/derrors/iphone-control-toolkit, then walks you through pairing, Developer Mode, mounting the developer disk image, signing in to your Apple ID, and signing and installing WebDriverAgent. Steps that are already done are skipped.

Open a new Claude Code session and ask, for example:

Open the Clock app, switch to the stopwatch, and show me a screenshot.

Tip

Other MCP clients: the server speaks stdio through docker exec and opens no network port.

{ "mcpServers": { "iphone": { "command": "docker", "args": ["exec", "-i", "iphone-control-toolkit", "/opt/ictl/mcp-server.sh"] } } }

To build the image from source instead of pulling it, set IMAGE=iphone-control-toolkit:local in .env.

MCP Tools

Tool Description
iphone_status Connection, lock state, foreground app, screen size
iphone_screenshot Screenshot scaled to points; pass x/y/width/height to zoom into a region
iphone_list_elements Elements with text and their center coordinates
iphone_tap / iphone_tap_element Tap by coordinates or by element name
iphone_swipe / iphone_long_press Swipe, long-press
iphone_type_text Type into the focused field
iphone_press_button home, volume_up, volume_down, lock
iphone_list_apps / iphone_launch_app / iphone_terminate_app Manage apps by name or bundle ID
iphone_open_url Open a URL or URL scheme

Screenshot pixels are tap coordinates, so the agent can tap what it sees. Games, maps and video players draw their own UI and have no element tree — use screenshots and coordinates there.

Live Viewer

bin/iphonectl viewer

A window on the host shows the phone's screen live, streamed from WebDriverAgent's MJPEG server, and passes your input on to the phone. It runs next to the MCP server and shares its WebDriverAgent session, so you can watch the agent work or take over.

Input On the phone
Click / hold Tap / long press
Drag, mouse wheel Swipe, scroll
Keyboard Types into the focused field
Esc, right click, ⌂ Home Home button
✕ Close app Ends the foreground app, like swiping it away in the app switcher

iOS ignores system gestures that do not come from a finger, such as the swipe up from the bottom edge, so the window has buttons for them. Resize it freely. Options such as frame rate and resolution: iphonectl viewer --help.

It needs Python 3.11+ with Tk and Pillow on the host: sudo apt install python3-tk python3-pil.imagetk (Debian/Ubuntu) or sudo pacman -S tk python-pillow (Arch).

If the phone is plugged into a headless host, run the viewer on your desktop from a checkout of this repository:

ssh -N -L 8100:127.0.0.1:8100 -L 9100:127.0.0.1:9100 your-host &
PYTHONPATH=src python3 -m iphone_viewer

Debugging WebViews

This is off by default. Set CDP_PORT=9222 in .env and run iphonectl up, and the container serves the phone's Safari tabs and inspectable WebViews over the Chrome DevTools Protocol on 127.0.0.1:9222: curl 127.0.0.1:9222/json lists them, and anything that speaks CDP can attach to read the DOM, run JavaScript, or follow console output and network requests. For hybrid apps that is faster and more precise than the element tree or screenshots.

Warning

While it is on, any process on the host can read and script the phone's Safari tabs, including sites you are logged in to. The bridge has no authentication and does not check where a request comes from, so turn it on only while you need it and only on a host you trust.

  1. On the phone, turn on Settings → Apps → Safari → Advanced → Web Inspector.
  2. Your app's WKWebView must be inspectable (isInspectable = true, iOS 16.4+). Safari is always inspectable.

Tip

Capacitor apps built without Xcode (for example with xtool): Capacitor only makes the WebView inspectable in debug builds, which it detects through a CAPACITOR_DEBUG Info.plist key that only Xcode's build settings fill in. Set it explicitly in capacitor.config.ts, for development builds only:

ios: { webContentsDebuggingEnabled: !isReleaseBuild },

iphonectl doctor reports how many pages are inspectable. Both the bridge and the live stream listen on 127.0.0.1 only and have no authentication, like WebDriverAgent itself; set MJPEG_PORT=0 in .env to turn the stream off. To reach them from another machine, use SSH port forwarding rather than binding them to other interfaces.

Everyday Commands

Command What it does
iphonectl status Container status and recent supervisor log
iphonectl doctor Layer-by-layer health check with a fix for each failure
iphonectl viewer Live view of the screen, controlled with mouse and keyboard
iphonectl logs [name] [-f] Logs: supervisor, wda, tunnel, resign, notify, …
iphonectl resign [--force] Re-sign WebDriverAgent if it expires soon (or now)
iphonectl login Sign in to the Apple ID again when the session expires
iphonectl passcode set|clear Turn auto-unlock on or off
iphonectl pull && iphonectl up Upgrade to the latest image

Settings live in .env (created from .env.example): proxy, Telegram / Bark, data directory owner. All state lives in data/ — pairing records, the Apple ID session and signing key, certificates and the passcode. Back it up; never commit or share it.

Documentation

Design Architecture: components, the two control paths, supervisor and re-signing
Operations Troubleshooting · Security
Development Development guide: tests, upgrading pinned upstream versions
Licensing THIRD_PARTY.md: bundled components, versions and licenses

The detailed docs are in Chinese for now.

Limitations

  • One phone per host; don't connect two instances to the same phone
  • Free Apple ID profiles last 7 days. Re-signing is automatic, but an expired Apple ID session needs iphonectl login
  • After the phone reboots, you must enter the passcode once on the phone to allow UI automation — iOS requires this
  • Elements hidden behind a popup may still be listed, because visibility is judged by position
  • System gestures (home indicator swipe, Control Center, Notification Center) cannot be injected from outside; use the Home button, closing apps by bundle ID, and URL schemes instead

Warning

This toolkit can fully control your phone. An agent may tap payment, delete or send buttons — keep tool-call confirmation on in your MCP client. Not affiliated with Apple; use it only with devices you own.

Contributing

Bug reports and suggestions are welcome in Issues; please include the output of iphonectl doctor with your UDID removed. Pull requests are welcome too — see the development guide.

License

MIT for this project's code. The container image also includes upstream components under other licenses, including GPL-3.0; each release attaches their complete corresponding source. See THIRD_PARTY.md.

(back to top)