What actually works
This page is the source of truth. If a claim anywhere else on this site conflicts with it, this page is right.
Capability matrix
Section titled “Capability matrix”| Surface | State | What is true |
|---|---|---|
| Keyboard, mouse, chords, drag | working | CGEvent injection on macOS. type, key, click, move-mouse, scroll, drag, plus the input stream. Gated on Accessibility, which reports correctly. |
| Screenshot | working | ScreenCaptureKit one-shot; whole display, a chosen display, or one window by CG id. 1920×1080 PNG in 110 ms cold. |
| Screen capture stream | working | Persistent SCStream, raw BGRA frames with a 16-byte header. Standalone defaults: native size, 10 fps. |
| Audio device control | working | audio list, audio default-in, audio default-out. CoreAudio, reversible. |
| Control plane + tickets | input only | serve with token handshake, start/stop/list, on-demand port allocation and a one-time ticket per stream — all verified by spike S6 (21/21). But its start refuses every stream except input, while the standalone path happily serves audio-out and video-out. That divergence is open. |
| Speaker loopback (audio-out) | frames flow, silent | The Core Audio process tap runs and delivers PCM with no third-party device installed. Every sample is zero until the calling process holds the system-audio capture grant — macOS returns silence rather than an error. |
| Virtual microphone (audio-in) | driver built | Our own CoreAudio AudioServerPlugIn (com.deemwar.windowctl.audio) is code-complete, signed with a hardened runtime and a timestamp, with a notarization script ready. coreaudiod still refuses it: it rejects Apple Development and Apple Distribution signatures. Blocked on a Developer ID Application certificate. |
| Virtual camera (video-in) | not built | CoreMediaIO system extension, designed in ADR 0006 and feasibility-checked in spike S4. No code. install camera exits non-zero with the reason. |
| Windows | not built | Designed throughout — WASAPI loopback, MFCreateVirtualCamera, Windows.Graphics.Capture, SendInput — and none of it implemented. ADR 0009 separately defers the kernel-mode virtual mic indefinitely in favour of a user-mode router. |
| Linux | not built | Not designed. Non-darwin builds compile; every device call reports unavailable. |
| Automated tests | control plane only | The wire protocol and the whole control plane are under go test (promoted from spike S6). The cgo device layer is not testable in CI and is covered by spikes on real hardware instead (ADR 0013). |
Spike results
Section titled “Spike results”Run on Apple silicon, macOS 26.4 (build 25E246), 2026-07-22. Sources in spikes/.
| Spike | Surface | ADR | Result |
|---|---|---|---|
| S0 | transport latency | 0002 | PASS — audio negligible, video fine at 30/60 fps |
| S1 | keyboard/mouse inject | 0003 | PASS — cursor moved via CGEvent through the input port |
| S2 | virtual mic (HAL) | 0004 | FEASIBLE — full install needs sudo + signing |
| S3 | audio-out loopback | 0005 | MECHANISM PASS, non-silent pending the audio TCC grant |
| S4 | virtual camera (extension) | 0006 | FEASIBLE — full approval needs signing + activation |
| S5 | screen capture | 0007 | PASS — 1920×1080 via ScreenCaptureKit |
| S6 | control plane: auth, tickets, lifecycle, bind scope | 0008, 0013 | PASS (21/21) — and it caught a total outage of serve |
| S7 | default audio device | 0010 | PASS — switched default input and reverted |
| S10 | distribution / cross-compile matrix | 0014 | PASS (11/11) — all six targets emit the right machine type |
| S11 | product site deploy | 0012 | PASS (9/9) — this site |
S0 — latency
Section titled “S0 — latency”WebSocket localhost round-trip, measured with the header’s stamp field:
- 10 ms PCM audio buffer (~2 KB): p50 0.047 ms, p99 0.167 ms.
- 720p BGRA video frame (3.6 MB): p50 5.6 ms, p99 7.1 ms.
Audio is effectively free — half a percent of one buffer period. Raw video is comfortable at 30 fps (33.3 ms budget) and at 60 fps (16.6 ms). Raw 1080p60 is ~480 MB/s and is the one case these numbers do not cover; ADR 0002 keeps a shared-memory fast path in reserve for it.
S6 — the outage nobody had noticed
Section titled “S6 — the outage nobody had noticed”ADR 0008 was marked Accepted without ever being run against the implementation.
Spike S6 is that run: it boots the real server on an ephemeral loopback port and
drives it over a real WebSocket client — wrong token rejected, command-before-hello
rejected, list inventory, dynamic port allocation, start idempotent, media port
refusing a missing and a wrong ticket, full handshake with the reply’s ticket,
stop idempotent, port released, and the control port unreachable on the host’s
LAN address. 21/21.
It found that start minted a ticket and the media port enforced it, but the
reply struct had no ticket field — the daemon told a client which port to use
and then refused it, forever. It failed closed, so it was never a vulnerability;
it was a headline path that had never worked. An earlier draft of the spike read
the ticket out of the server’s private state and therefore proved the gate worked
while the real client was locked out. Tests must obtain credentials the way a
client must.
S3 — the negative result that matters
Section titled “S3 — the negative result that matters”The process tap was created, a private aggregate device was built, and the IOProc delivered 237,568 samples with no third-party audio device installed. The mechanism works. But the measured peak was 0.0000 — every sample was zero.
macOS does not error when the calling process lacks the system-audio capture grant; it hands back silence. So the pipeline showed green while measuring nothing. That is why loopback is listed as partial rather than working, and why this repo runs a spike before an implementation.
S5 — screen capture
Section titled “S5 — screen capture”SCScreenshotManager.captureImage produced a 1920×1080 PNG in 110 ms. That is
a cold one-shot dominated by ScreenCaptureKit setup; the streaming path uses a
persistent SCStream and costs far less per frame.
S7 — default device control
Section titled “S7 — default device control”Enumerated every input and output device, set the default input from the built-in microphone to a USB device (status 0, verified by read-back), then reverted. Audio default control is real and reversible.
macOS has no system-wide default camera — camera selection is per-app. That is a platform fact, recorded in ADR 0010, not something a future release fixes.
The two gates
Section titled “The two gates”Gate A — load the virtual audio driver. coreaudiod refuses plugins that are
not signed with a Developer ID Application certificate; spctl -a -t install
rejects both the Apple Development and Apple Distribution signatures. The fix is a
Developer ID certificate plus notarization, and the build and notarization
scripts are already written. Until then the mic lane does not exist to an app.
Gate B — the system-audio grant. Speaker loopback needs the calling process to hold the audio-capture permission, a manual System Settings gesture with the same shape as the Screen Recording gate. Until then loopback delivers correctly framed silence.
Neither gate is a design problem. Both are ceremonies, and both are written down rather than papered over.