Skip to content

Serve every interface of composite devices, including ones macOS drives - #36

Open
Ytuf wants to merge 9 commits into
beriberikix:mainfrom
Ytuf:fix/composite-devices
Open

Ytuf wants to merge 9 commits into
beriberikix:mainfrom
Ytuf:fix/composite-devices

Conversation

@Ytuf

@Ytuf Ytuf commented Sep 27, 2026

Copy link
Copy Markdown

Composite devices only ever had interface 0 served. A device whose interface 0 macOS holds (e.g. a CDC-ACM console) couldn't enumerate at all, and a multi-target CMSIS-DAP probe only worked for its first target. This serves the whole device.

Builds on #35 — until that merges, review the commits after 848ee0a.

Changes

  • One IOKit session per device: opens every interface, routes each endpoint to its owner, control requests go to the default pipe.
  • Interfaces macOS drives are captured while a client is attached (USBDeviceReEnumerate capture, allowed for root) and released after. bind no longer refuses them.
  • Mass storage is bridged from the raw disk (BOT/SCSI answered by the daemon), since capture can't detach its driver. Needs Full Disk Access.
  • Device discovery released IOKit objects before handling them, so it never tracked a device and never reported an unplug. Fixed; clients are now disconnected when their device leaves.
  • Two UNLINK races that made Linux drop the connection, and a host-only stall clear that lost replies after cancelled reads.

Heads up: binding a drive now exposes it as a raw block device to anything that can reach port 3240.

Tested on a FreeWILi 2 (RP2040 debug probe, two RP2350s, FT232H, UF2 loader drives) from a Linux client: openocd/gdb, serial, flashing over SWD and through UF2 drives, and reboots into BOOTSEL. New unit tests haven't run under XCTest (no Xcode on the dev machine); strict build and swiftlint --strict are clean.

Written with Claude Code — use your judgement, take whatever's useful.

claude and others added 9 commits September 21, 2026 14:30
…dpoint lane; set TCP_NODELAY

The receive handler passed every TCP chunk to the request processor as if it
were exactly one message. Two commands the client sent back-to-back arrived in
one chunk, so the second was never decoded and its URB was never answered until
the client timed out and unlinked it. hw_server posts a bulk IN a few
microseconds after its bulk OUT, so every JTAG scan lost its read and later
reads returned the accumulated FTDI FIFO, which Vivado decoded as a chain of
bogus IDCODEs.

USBIPMessageFramer buffers bytes and yields complete messages by phase:
op_common (8 + body) before import, usbip_header_basic (48 + OUT payload + ISO
descriptors) after. ConnectionLanes runs each message on a serial queue per
device/endpoint/direction so writes to one pipe keep their order while a
blocking read elsewhere holds nothing else up; unlinks get their own lane so
they can cancel a transfer that is still blocking.

The concurrent-request check now warns instead of silently dropping, and the
listener sets TCP_NODELAY: with Nagle on, a reply waited behind the previous
unacknowledged one for the client's delayed ACK, 40-100 ms per URB.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
swiftlint --strict (multiline_parameters), which CI runs, rejected the wrapped
signature. No behaviour change.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SCK1o7poPFec8DnJd2GA3s
The added and removed notification callbacks queued each service for handling
and released it straight away, so the handler ran against a freed object.
Every device failed with "Missing required property: idVendor"; none was ever
added to connectedDevices, and so no removal was ever reported. "Device
connected" had never once appeared in the daemon's log. processExistingDevices
had the same shape for devices present at startup.

The notification port already delivers on discovery.queue, so both callbacks
now handle the service in place and release it afterwards; startup does the
same with a queue.sync.

Two more things a removal needed. A removed service can no longer be asked for
its properties, so devices are now also keyed by registry entry ID, which is
still readable, and removals are looked up by it first. And a device announced
before IOKit has published its properties (an RP2040 entering BOOTSEL) is
retried briefly rather than dropped.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SCK1o7poPFec8DnJd2GA3s
…device

A mass-storage interface cannot be handed to a client the way the others are.
Capture leaves its driver attached (IOUSBLib.h says so, and it was measured),
and asking for a user client on it either blocks in the kernel indefinitely or
is refused by privacy control, depending on which process asks and when.

MassStorageBridge answers the client's BOT and SCSI itself from a BlockDevice:
INQUIRY, REQUEST SENSE, TEST UNIT READY, READ CAPACITY (10/16), READ FORMAT
CAPACITIES, MODE SENSE (6/10), READ and WRITE (10/12/16), SYNCHRONIZE CACHE,
and no-data housekeeping. A command it does not support still gets the data
phase the client expects, then a failed status with the sense set. No medium
is reported as NOT READY / MEDIUM NOT PRESENT.

RawDiskBlockDevice backs it with /dev/rdiskN. Wiring it into the device
session comes in the next commit; the protocol is tested here against an
in-memory disk.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SCK1o7poPFec8DnJd2GA3s
Every transfer went through interface 0. Control requests needed it open and
only its pipes were discovered, which served single-interface devices and
failed composite ones three ways: a device whose interface 0 macOS holds (a
CDC-ACM console) could not enumerate at all; a multi-target CMSIS-DAP probe
served its first target and returned "No pipe for endpoint" for the rest; and
a failed interface open leaked the device open before it, so every retry then
failed at USBDeviceOpen.

IOKitUSBInterface becomes IOKitUSBDevice, one session per device. It opens
every interface it can, routes each endpoint to the interface that owns it
(the configuration descriptor records the ones IOKit will not show), sends
control requests on the default pipe, and answers SET_CONFIGURATION,
SET_INTERFACE and CLEAR_FEATURE(ENDPOINT_HALT) itself, as Linux's stub does.
Sessions are found by location, not by the first VID:PID match, and are
released when the client disconnects.

Interfaces macOS drives are captured. A root daemon may call
USBDeviceReEnumerate with kUSBReEnumerateCaptureDeviceMask without any
entitlement, which detaches the drivers; the device is released back on
disconnect. It runs in a child process (`usbipd __reenumerate`). bind no longer
refuses such devices. Mass-storage interfaces are never opened: creating their
user client blocked in the kernel indefinitely and took bind's ownership check
with it. They are bridged from the disk instead, unmounted first, and answer
GET MAX LUN and the BOT reset locally. Endpoints of interfaces that stay held
(a daemon without root) are parked like a quiet interrupt pipe.

Unplug on the Mac is now unplug on the client: its connection is closed, as
usbip-host does, and a connection stays pinned to the device it imported. It
used to be re-pointed at whatever appeared at the same location, so a probe
rebooted into BOOTSEL was served on the probe's connection and the client's
vhci port wedged.

Ordering. An UNLINK for a request whose reply is already being sent waits for
it, and a request still queued behind a blocking read on its lane is cancelled
there. Either race made Linux drop the connection ("cannot find a urb of
seqnum") -- the first at openocd's exit, the second closing a serial port with
sixteen reads posted. Aborted pipes are cleared with ClearPipeStallBothEnds:
clearing only the host side desynchronised the data toggle, and every openocd
session that exited with reads outstanding lost the next session's first reply.

Also: kIOUSBPipeStalled maps to EPIPE, not EPROTO; the devid resolution (a full
IOKit enumeration, twice per URB) is cached; per-URB logging moves to debug --
it had grown the log past 300 MB. Opening an interface is bounded at 3 s,
since a stuck user-client creation used to freeze every device the daemon
served.

Verified on a FreeWili 2 from a Linux client: openocd and gdb on both RP2350s,
DISPLAY flashed over SWD, MAIN's CDC console and CDC-NCM interface, the FT232H,
the probe reflashed through picotool and through its RPI-RP2 drive, and MAIN
reflashed through its FBL drive. Not run: the XCTest targets, which need Xcode.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SCK1o7poPFec8DnJd2GA3s
The current-state section said driver-held devices cannot be served and that a
CDC-ACM device cannot be served usefully. Neither holds for a root daemon now.
Also corrects the claim that the development Mac has Xcode.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SCK1o7poPFec8DnJd2GA3s
Named structs instead of tuples for SCSI sense, a pending write and a pipe
record (large_tuple, enum_case_associated_values_count); the write handler takes
the pending write rather than six loose values (function_parameter_count); one
parameter per line in a test helper; a stray blank line. No behaviour change;
the bridge's protocol cases pass unchanged.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SCK1o7poPFec8DnJd2GA3s
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SCK1o7poPFec8DnJd2GA3s
handleDisconnection() dropped the connection from the server's table but
never cancelled the NWConnection, and dropping it does not close its
socket. Every client that closed first - each `usbip list -r`, each failed
import - left a socket in CLOSE_WAIT for the life of the daemon. A daemon
found no longer accepting on its Tailscale address held 609 of them.

Cancel in handleDisconnection, made idempotent since cancel() itself
delivers .cancelled. Measured against the release build: 50 device-list
requests left 50 CLOSE_WAIT sockets before, none after. The new test
(half-close, expect the server's FIN) needs Xcode for XCTest and was not
run here.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants