Under the Hood

One daemon per phone runs on your Mac. It reads the screen as an accessibility tree (text, not pixels), sends taps and typing through a small XCTest runner it installs on the iPhone, and checks that the screen has settled before it answers. Agents and people go through the same daemon, and one of them holds the phone at a time.

 Claude Code / Codex / any MCP client ──MCP──┐
 scripts and other agents ─────────────HTTP──┤
                                             ▼
                          iphone-use daemon on your Mac ──USB or Wi-Fi──▶ XCTest runner on the iPhone
                                             ▲
 you, in a browser or the iOS app ───────────┘

The daemon

The daemon runs as a per-user LaunchAgent and listens on 127.0.0.1:44321 behind a password. It offers the same control in three forms: an MCP server for agents, an HTTP API under /agent/* for scripts, and a web page plus a native iOS app for people (Connect).

• The screen as text. /agent/elements returns a flattened accessibility tree with a snapshot token; element taps are bound to that snapshot, so a changed screen fails with stale_element_snapshot instead of tapping the wrong thing.
• Honest outcomes. Every action reports whether it was applied, not sent, or unknown, and whether a retry is safe. A tap on an element covered by another control is refused (element_occluded).
• Settle check. After an action the daemon can read the tree again and return what changed plus a settle block (stable, budget_exhausted or observation_failed).
• The Mac stays still. It never touches the Mac's screen, cursor or focus.

The XCTest device runner

iphone-use drives the phone through its own XCTest UI-test bundle (runner/). It replaced WebDriverAgent in v0.14.0 and speaks the same WebDriverAgent HTTP API. It does not use iPhone Mirroring. The runner is built for speed:

• It reads the screen with one accessibility snapshot of a fixed attribute set, instead of walking the element tree attribute by attribute.
• It synthesizes touches directly and does not wait for the app to go idle before acting; the daemon checks afterwards that the screen has settled.
• It is one small app with a built-in HTTP server and MJPEG stream; nothing else is installed on the phone.

iphone-use setup builds and signs the runner with your Xcode team, starts it on the phone, and pins two loopback relays: 8100 for control and 9100 for the screen stream. The relays run over macOS's own usbmuxd (iphone-use relay), so the daemon only ever talks to localhost and no Homebrew packages are needed.

USB and Wi-Fi

• USB is preferred whenever the phone is plugged in.
• Wi-Fi (iOS 17 and later). Off the cable, setup, relaunch and the same relays reach the phone through CoreDevice's encrypted Wi-Fi tunnel, with no flag (IPU_RUNNER_TRANSPORT=auto, the default). That tunnel is routed by this Mac only. Status then reports transport: "wifi-tunnel".
• iOS 15 and 16 need no update, but Xcode 26 and later cannot see them, so setup takes its own legacy path: it mounts the pinned Developer Disk Image for that iOS, installs the runner and starts it over lockdown's TLS session on Wi-Fi when the phone answers there. Driving it off the cable goes over a LAN relay that is signed but not encrypted: someone watching the Wi-Fi can see the screen but cannot drive the phone. This path needs a paid developer account's App Store Connect API key.
• A plain unencrypted LAN relay (IPU_RUNNER_ALLOW_LAN=1) is an explicit last resort only.

Request signing

The runner's ports on the phone refuse (401, no work done) any request not signed with the token setup generates at each runner launch: HMAC-SHA256 over the method, target, body, a timestamp and a one-time nonce. The token stays on the Mac (mode 0600) and is never sent, so a captured request can be neither replayed nor altered. A runner launched without a token refuses everything. Signing does not encrypt; see Privacy & Security.

Lifecycle: idle release and pre-warm

• Idle release. After 5 minutes without agent activity or a live viewer, the daemon stops the runner and parks it (IPHONE_USE_IDLE_RELEASE_SECS changes the window). The next agent request brings it back from the cached build; unlock the phone if asked.
• Passcode phones keep their runner. Every runner launch on a phone with a passcode asks for it on the phone, so on such a phone idle release leaves a running runner up and reports keep_runner_alive.
• Pre-warm. Bringing a parked runner back takes about 5 s with the screen on and 10–25 s once iOS turned it off. To hide that wait, the daemon starts the bring-up in the background when an MCP server starts or a session takes the owner lease, but only for a phone an agent drove within the last hour and never on a locked phone.
• /agent/status is the source of truth: drivable, device_state, and a hint plus setup_blocked_on when something blocks the phone (locked, trust, Developer Mode, WARP, an Xcode too old for the phone's iOS…).

Giving the phone back

• Hold both volume keys to end iOS UI automation; the runner parks until an explicit agent or web request.
• iphone-use pause persists a human handoff; iphone-use resume (or the web panel's agent mode) ends it.
• The hand back button in the web toolbar (or POST /agent/mode {"mode":"human"}) stops the runner; agent input then answers 409 phone_handed_to_human instead of restarting it under your fingers.
• One owner at a time. A session claims the phone with an owner lease (X-Phone-Owner, 300 s by default, refreshed per request); others get 409 phone_owned with the owner's name. This is coordination between cooperating sessions, not a security boundary.

Live view: browser and iOS app

The /phone page shows the live screen as H.264 encoded on the phone by the runner (MJPEG where the browser lacks WebCodecs). Click taps, drag swipes, hold long-presses, the wheel scrolls, and typing goes to the phone through your own input method. An unchanged screen is not re-encoded.

Auto mode watches in quality from this Mac or its LAN and in performance from anywhere else. On screens an app hides from capture (banking, payments), you see a live wireframe of the accessibility tree instead of a blank picture, and can still tap through it.

The native SwiftUI iOS app (minimum deployment target iOS 15) pairs by scanning a QR code on the Mac's page and plays the H.264 feed with the hardware decoder. It is not in the App Store: iphone-use setup-app builds it with your own signing team.

On-phone daemon and local control

The daemon on the phone is the same program as on the Mac, linked into the runner. With a paid developer team, the iOS app's packet-tunnel extension starts the runner on the phone itself, so the phone serves taps, typing, screen reads and the agent API with no Mac at run time (iOS 17.4 or later; the first pairing still needs the USB cable once). A free Apple ID gets the remote app, which drives phones that a Mac serves. Goals handed to the daemon as agent tasks keep running there while another app is in front.

More than one phone

One daemon drives one phone. A named instance (./install.sh --instance lab --udid <UDID>) gets its own copy of the app, state directory, launchd labels, token and ports, and leaves the default install alone. Each phone has its own daemon and runner, so two phones driven at the same time run as fast as either one alone. MCP calls can pick an installed instance per call with instance.

Simulators (preview)

The runner also runs on an iOS Simulator, with no phone, Apple ID or signing: iphone-use simulator start "iPhone 16 Pro" builds and starts everything, then IPHONE_USE_INSTANCE=sim points an MCP client at it. Simulators have no camera, App Store apps, real push notifications or Face ID, so tasks that need those only work on a phone.

Speed

The project's own measurements, from its README (October 2026, same iPhone 13 over USB):

Through the full agent API, a tap by label that returns the settled change takes about 1.9 s on an iPhone 17 Pro Max; the WebDriverAgent-based release took 4.2 s. The live view streams at 27–28 fps. These are maintainer measurements, not an independent benchmark.

Deeper reading: the full guide, the device runner and the architecture notes.