device-bridge

Your app under test thinks it is talking to a microphone, a camera, a screen and a keyboard. It is talking to your test harness, over loopback, at 0.047 ms per audio frame.

Quickstart What actually works GitHub

device-bridge serve --port 9330 --token "$TOKEN"   # control plane, 127.0.0.1 only
device-bridge start video-out --port 9334 --token "$TOKEN"
device-bridge type "hello"                         # drive the UI
device-bridge screenshot --out shot.png            # read the result back

Testing a voice or video product means proving what it heard and what it showed. A browser driver cannot do that: it can click, but it cannot be a microphone. device-bridge sits on the other side of the device layer — it feeds the app audio and video, captures what the app plays and renders, and injects the keyboard and mouse events in between.

Audio frame: 0.047 msp50 round-trip, 10 ms PCM buffer on loopback (spike S0)
720p frame: 5.6 msp50 round-trip, 3.6 MB raw BGRA (spike S0)
Screenshot: 110 ms1920×1080 PNG, cold one-shot via ScreenCaptureKit (spike S5)
Frame header: 16 bytesfixed-width, allocation-free, no per-frame JSON

The loop

The app under test never learns that anything is unusual. It enumerates devices, picks one, and reads or writes frames. On the other end of each lane is your harness.

App under test

Unmodified. No SDK, no shim, no instrumentation. It just opens a device.

→audio-invirtual mic — driver built, not loadable yet9331

←audio-outspeaker loopback, process tap9332

→video-invirtual camera — not built9333

←video-outscreen capture, ScreenCaptureKit9334

→inputkeyboard + mouse, CGEvent9335

device-bridge

One Go binary. A token-gated control plane on 9330, one dedicated port per stream, everything bound to 127.0.0.1.

→ harness feeds the app← harness captures from the appdashed = designed and specified, not working yet

Latency was the first thing measured, not the last

The whole design hangs on one question: can a socket carry a device? Spike S0 measured it before any device shim was written, and the transport decision in ADR 0002 followed the numbers.

Audio, 10 ms PCM~2 KB, p500.047 ms

Video, 720p BGRA3.6 MB, p505.6 ms

Video, 720p BGRA3.6 MB, p997.1 ms

60 fps budgetone frame period16.6 ms

30 fps budgetone frame period33.3 ms

Linear scale against the 30 fps frame budget, measured on Apple silicon, macOS 26.4 (spike S0). The audio bar is a hairline because that is the result: a 10 ms audio buffer costs 0.5% of one buffer period to move. Raw 1080p60 (~480 MB/s) is the one case the numbers do not cover — that is why ADR 0002 keeps a shared-memory fast path in reserve.

What actually works

Two of the five lanes do not work today. Here is the whole truth, and what each one is waiting on.

Surface	State	Detail
Keyboard, mouse, chords, drag	working	CGEvent injection; round-trip verified through the input port (spike S1)
Screenshot	working	ScreenCaptureKit; 1920×1080 PNG in 110 ms cold (spike S5)
Screen capture stream (video-out)	working	Persistent `SCStream`, raw BGRA frames with a 16-byte header
Audio device list / set default	working	CoreAudio; reversible — spike S7 switched the default input and reverted it
Speaker loopback (audio-out)	frames flow, silent	The process tap delivered 237,568 samples with no third-party device — but peak 0.0000. Non-silent capture needs the system-audio permission granted to the calling terminal (spike S3)
Virtual microphone (audio-in)	driver built	Our own CoreAudio AudioServerPlugIn, code-complete. `coreaudiod` refuses to load it: it rejects signatures that are not Developer ID, so this needs a notarized Developer ID build
Virtual camera (video-in)	not built	CoreMediaIO system extension, designed in ADR 0006. `install camera` exits with a clear error rather than pretending
Windows	not built	Every surface is designed for it (WASAPI loopback, Media Foundation virtual camera, `SendInput`), none is implemented. ADR 0009 additionally defers the kernel-mode virtual mic indefinitely
Linux	not built	Not designed either. Non-darwin builds compile and report each stream as unavailable

Every row is backed by a runnable spike in spikes/ and an ADR saying why the decision went that way — including the ones that came back negative. The full evidence table is on What actually works.

Ports are boring on purpose

One control port, one dedicated port per stream, no multiplexing. A media socket carries binary frames and nothing else, so there is no demux on the hot path and no head-of-line blocking between audio and video (ADR 0002).

9330controlJSON. Token handshake, list, start, stop.
9331audio-inPCM to the app’s microphone.
9332audio-outPCM captured from the speakers.
9333video-inFrames to the app’s camera.
9334video-outFrames captured from the screen.
9335inputJSON. Keyboard and mouse events.

Two ways in

device-bridge serve --port 9330 --token "$TOKEN"

One authenticated connection, then start and stop streams by name. The daemon allocates the port and mints a one-time ticket per stream; the media port refuses anything without it.

{ "type": "hello", "token": "…" }
{ "type": "start", "stream": "input" }

Today this path starts input only — its inventory still reports the four media streams as unimplemented even though two of them serve on the standalone path. Measured in spike S6, written down, not yet fixed.

device-bridge start video-out --port 9334 --token "$TOKEN"

One stream, one fixed port, gated by the token directly (?token=). No control connection to hold open — the shape you want inside a CI job that only needs one lane.

device-bridge type "hello"
device-bridge key cmd+shift+t
device-bridge click --x 400 --y 300
device-bridge drag --x1 100 --y1 100 --x2 400 --y2 300
device-bridge screenshot --out shot.png
device-bridge audio list --json

Run and exit. No daemon, no token, no socket — these talk to the OS directly and are the fastest way to script a desktop.

It is locked by default, because of what it is

A process that mirrors your microphone and injects keystrokes is a remote-takeover primitive if it is ever exposed. So:

Loopback only. Every listener binds 127.0.0.1, never 0.0.0.0 — enforced at net.Listen time, not by configuration.
No token, no start. serve and start refuse to run without one.
One-time tickets. The control plane mints a ticket per stream; the media port rejects a connection that does not carry it.
Nothing long-lived and unauthenticated. Remote exposure, if ever used, is a deliberate tunnel that gets torn down — never a standing listener.

Read the whole model in Security and ADR 0008.

Built spike-first

Every capability here has, in this order: a runnable spike that proved it on real hardware, an ADR recording why the decision went that way, an OpenSpec slice, and then an implementation. Spike S3 is the reason that discipline exists — the process tap delivered a quarter of a million samples and every one of them was zero. A green signal that measured nothing. It is written down as a negative result, and the site says so above.

MIT licensed. Start here →