scrcpy --serial R58N71M14ZE and you have a phone on your screen, mirrored at low latency, keyboard and mouse working. It is one of the most-starred tools on GitHub — around 148,000 stars — and for one device the story ends there.
The interesting part starts at the second device, and everything that is actually hard shows up by the twentieth. This is what we learned building that for NOISEY, against a rack of twenty Samsung S20 boards, including the four handshake details that each cost a day and the bug that only exists when a tile is dead.
What everyone tries second
A shell script with twenty scrcpy --serial lines in it. It works, in the sense that twenty windows appear. Four things then go wrong at once:
- Window management becomes the job. Twenty OS windows, each independently placed, none of which know they are part of a grid.
- Twenty independent decoders. Each process decodes and presents alone; nothing pools, nothing throttles, nothing coordinates.
- No shared state. There is no “selected”, no group, no way to know that six of them died.
- No dispatch. The thing you actually want — do this on all of them, then tell me which ones failed — has nowhere to live.
This is the gap the GUI wrappers fill, and why so many of them exist. They are launchers: they arrange the windows and they do not change what each window is.
The fork: twenty processes, or one process twenty times
scrcpy is two halves — a server that runs on the phone as a plain Java process, and a client that speaks its protocol over a socket. Nothing stops one client from holding twenty of those conversations at once. That is the road we took: the desktop app pushes the same server jar to every handset, opens two sockets per phone, and owns all of them in one process with one grid, one selection and one place a failure can show up.
The cost of that road is the protocol. It is not documented as a public API — it is an implementation detail of a tool that ships its own client — so every ordering constraint below was found by hitting it.
Four handshake details that each cost a day
The TCP connect succeeds before the server exists
In tunnel-forward mode you ask adb to forward a local port onto an abstract socket on the device, then start the server. Connect to that port and adb accepts immediately and then closes the connection, because adb is listening, not the server. A client that treats “connected” as success gets a socket that dies under it a moment later, and the error surfaces somewhere else entirely.
The only real signal is the dummy byte. Ours connects with a 500 ms timeout, attempts to read exactly one byte, and retries every 100 ms against a 15-second deadline; only 0x00 counts as the server being up. Fifteen seconds sounds generous until an old handset JITs the server for the first time.
The control socket must connect before the device name arrives
The server accepts all of its sockets before it sends anything further. So the natural code — connect the video socket, read the 64-byte device name, then connect control — deadlocks. Not an error, not a refusal: the read simply blocks until your timeout fires, which means you go looking for the fault in the wrong place. Connect control first, then read the name.
SPS/PPS is not a packet, it is a prefix
The stream carries config packets — the H.264 parameter sets — flagged separately from media. They must be prepended to the next media packet rather than handed to the decoder alone. Feed them separately and you get a decoder that never produces a frame while every counter says data is arriving.
The same rule survives the move into a browser engine. With WebCodecs, the first EncodedVideoChunk must carry the parameter sets; and configure the decoder without a description, which is what selects Annex-B mode — the byte format scrcpy actually sends.
Taps travel in stream space, not device space
This is the one that bites after everything else works. The server validates touch coordinates against the size of the stream it announced, not the phone's real resolution. Our sessions cap the larger dimension at 800 px, so a 1440 × 3200 board streams at 360 × 800 — and a tap computed in device pixels saturates the clamp and lands in the far corner of the screen, every time, on every device.
Rotation comes free from the same rule: a rotated device arrives as a new session header with a new size, so nothing needs to track orientation separately. And if you have read our iPhone post, this is the same lesson in different clothes — there WebDriverAgent speaks points while screenshots are pixels at 3×. Two platforms, one bug class: a coordinate is meaningless without the space it belongs to.
Twenty H.264 streams in a webview
We measured this properly before committing, because “twenty tiles will need hardware decode” was an assumption we had inherited rather than tested. Two builds of the same vertical slice, both against the same twenty boards, both decoding with WebCodecs, 25-second window:
- Zero frames dropped before decode. Zero decode-queue depth. On both. Queue depth is the early-warning signal — it grows long before fps visibly falls — and neither build ever registered it.
- 3,052 and 2,858 chunks decoded respectively, with a median decode latency of 1.1 ms and 2.0 ms. The chunk difference is screen activity, not throughput: scrcpy only encodes when the screen changes.
- The source side alone, headless, launched all 20 boards in 931 ms and sustained 214 fps aggregate at 5.44 Mbit/s in 81 MB of RSS.
The old pipeline had struggled at three boards, and the reason turned out not to be decode throughput at all — it decoded to raw frames and re-encoded every one to JPEG before showing it. Deleting the JPEG round trip is what bought the headroom. If you are stuck at a handful of tiles, look for a re-encode before you look for a faster decoder.
Two WebCodecs details that cost an evening each:
- A decoder cannot start mid-GOP. Drop every chunk until the first key frame arrives, or every delta frame errors until one happens along.
VideoFrameholds GPU memory. Forget to close one and the decoder stalls within a second or two — which looks exactly like a throughput problem and is not one.
The failures that only exist at twenty
Everything above is true at one device too. These are not.
The unread control socket. The device pushes messages back on the control socket — clipboard content, acknowledgements, UHID output. A client that only ever writes will eventually wedge the server when its send buffer fills. It takes minutes to appear, it is invisible in any single-device test, and there is no diagnostic pointing at it. We run a drain thread per session that reads and frames those messages and acts on none of them.
The tile that looks alive and is dead. Ours ended a session, left its channel registered, and so the poll that restarts dropped sessions skipped it forever. The grid hid it completely: a tile without a live session falls back to a screenshot every three seconds, which on a mostly-static phone is indistinguishable from mirroring. Opening the phone full-screen was the first place the corpse was visible. The lesson generalises past scrcpy — a fallback that looks like success is worse than no fallback, and every fallback needs a visible marker saying which one you are looking at.
The hub browns out. Installing an app four devices at a time is fine. Doing it with a split bundle — X ships as 53 parts and 398 MB, pushed through a package-manager session — took a nineteen-board rack down: adb started reporting failed to write, then device not found, and a board dropped off the bus. Result was 0 of 19 on two apps. The fix was arithmetic, not cleverness: four-wide for a plain APK, two-wide for a bundle over eight parts, and a retry that distinguishes transport errors from a package manager's actual verdict. INSTALL_FAILED_* is a real answer and retrying it is pointless; failed to write is the cable complaining and deserves another go.
At twenty tiles you stop watching and start dispatching
This is the part that surprised us. Mirroring twenty phones is impressive for about a day, and then you notice you are not looking at the grid. What you want is to send work and be told what failed. Mirroring is the debugger, not the product.
Two consequences worth stealing even if you never use NOISEY. First, anything that shells out per device needs batching: adb shell input tap spawns a process for every tap, at 150–250 ms a round trip, so the low-latency path has to go over the control socket instead — and reading a phone's properties in one adb shell invocation instead of four round trips is 20 process spawns across a rack instead of 80. Second, a batch operation must never let one device take the batch down: our screenshot-many returns an empty image for a phone that is asleep, because one asleep handset should not blank the grid.
In the product those turn into words a user says rather than plumbing: a group of phones is a named set an automation targets as one, and seconds between phones spaces the jobs so twenty simultaneous uploads through one computer do not become a self-inflicted traffic jam. That spacing is a courtesy to your own bandwidth — it is not a randomiser, and it is not there to make anything look human.
What we have not measured
Stated so the numbers above are not over-read. The runs are 25 seconds. Long enough to expose queue growth, not long enough to expose a slow leak; a multi-hour soak on this rack still does not exist. Windows is unverified — everything here ran on macOS, WebView2 is not WKWebView, and the desktop app is Mac-only today. Tap-to-pixel latency end to end has never been measured, only decode latency. And USB topology has hard limits we have bumped into rather than characterised: the brownout above is the evidence that a shared hub is a real constraint, not a measurement of where it sits.
One product number, because it would be dishonest to leave it out: the twenty-board rack is our test bed, not a shipped entitlement. NOISEY registers 2 phones on the free plan and 10 on the paid plan. The architecture is built for twenty because the hard problems only appear there.
You can build this yourself
Honestly: you can. The protocol is readable, the server jar ships with scrcpy, and the handshake fits in about two hundred lines once you know the four things above — which is the entire reason for writing them down. What takes the time is not the streaming; it is everything after it, when you discover that mirroring twenty phones was the easy half and driving them reliably is the rest of the work.
If you would rather not, that is what the desktop app is: plug a phone in, turn on USB debugging, and the grid is there. A routine is what a phone does, an automation decides when it runs, and the phones stay in front of you — our servers never touch a device, structurally.