Streams & ports

There are five lanes. Two feed the app under test, two capture from it, one drives the OS. Each gets its own dedicated port so the hot path is a single binary WebSocket with no demultiplexing and no head-of-line blocking between audio and video (ADR 0002).

The lanes

CLI name	Canonical	Direction	Port	State
audio-in	mic	harness → app	9331	driver built, not loadable
audio-out	speaker	app → harness	9332	frames flow, silent without the grant
video-in	cam	harness → app	9333	not built
video-out	screen	app → harness	9334	working
input	input	harness → OS	9335	working

The CLI accepts either name, plus kbd and mouse as aliases for input. The control plane speaks the canonical names. Port 9330 is the control port.

The send/receive naming is deliberate: audio-in is audio going in to the app, which means it is the harness writing. If you think of it from the app’s point of view — “this is my input device” — it lines up.

Standalone: one stream, one fixed port

device-bridge start <stream> --port N [--token T]

The media port is gated by the token directly, passed as a query parameter:

ws://127.0.0.1:9334/?token=…

--port is required; there is no default for the standalone path. This is the ergonomic shape for a CI job that only needs one lane, and it is what the quickstart uses.

Control plane: negotiate and allocate

device-bridge serve [--port N] [--token T]

The control port is the only stateful channel. A client sends hello with the token, then start / stop / list. Ports are allocated on demand at start and released at stop, and the reply carries the port that was actually bound plus a one-time ticket the media port will demand.

{ "type": "start", "stream": "input" }

The control plane only starts input today

Spike S6 measured this. standalone.go wires speaker and screen, so device-bridge start audio-out and start video-out work — but the control plane’s own inventory still reports all four media streams as available: false, "not implemented yet", and its start refuses everything except input. So the negotiated path is real for input and not yet real for audio or video; use the standalone path for those. Recorded, not yet fixed.

The same spike caught a harder one: start minted a per-stream ticket and the media port enforced it, but the reply carried no ticket field — the daemon handed a client a port and then refused it forever. It failed closed, so it was never a hole; it was an end-to-end path that had never worked. Fixed, with regression tests.

Availability

device-bridge devices

input      available=true
audio-in   available=false not implemented yet
audio-out  available=false not implemented yet
video-in   available=false not implemented yet
video-out  available=false not implemented yet

devices under-reports today

The verb checks Accessibility for input and hard-codes the other four as unavailable. audio-out and video-out do in fact serve frames on the standalone path — start them and see. The status page is the accurate source until this is fixed.

Frames

Every media message is one binary WebSocket frame: a fixed 16-byte header then the raw payload. Control and input stay JSON, because they are low-rate and ergonomics wins there.

• audio-out delivers Kind=2 (speaker), Format=2 (interleaved float32 LE).
• video-out delivers Kind=4 (screen), Format=11 (packed BGRA).

The header’s stamp is a producer monotonic millisecond counter, so a harness measures round-trip latency without touching a wall clock — which is exactly how the 0.047 ms and 5.6 ms figures in spike S0 were obtained.