SplotchScrapbook

Harness snapshot · September 5, 2026

How Splotch measures itself on real screens

The profiling rig, one surface at a time. Something has to touch the glass. Something has to carry the numbers home. The platform can add what the page cannot see. Then Node turns raw rows into a verdict. This page builds that up piece by piece, for every surface the app ships on.

11 surfaces 4 input transports 6 layers 20 cells per surface 0 metrics computed on the device

Start here

Two channels, and everything else is choosing them

Picture a phone on a desk. A robot finger scribbles on it. Meanwhile the web page keeps a diary of every frame it drew and every touch it felt. When the scribble ends, a courier carries the diary back to the Mac, and the Mac does all the arithmetic. That is the whole rig. Every surface differs only in who the finger is and who the courier is.

1

A finger that touches the glass

Called the input transport. It can be an automation framework, an operating-system command, or fake events fired from inside the page. Which one matters more than anything else on this page.

2

A diary the page keeps

Called the probe. A small script injected into the app that appends numbers: when each frame ran, when each touch arrived, how long the engine spent. It records and never judges.

3

A courier that brings it home

Called the measurement channel. A debugger connection, an HTTP upload, or a file copied out of the app. Node reads the diary and computes every percentile and every pass or fail.

Follow one capture

Select a stage to see what happens there.

Build: compile the marks in. A production build with PERF_MARKS=true wraps the engine's hot paths in performance.mark and performance.measure calls, and keeps function names readable. Release builds strip all of it, and a post-build check proves that they did.
The one rule that shaped everything. Input and measurement are separate channels. When the same connection both drives the finger and reads the diary, the driver can slow the page it is measuring, and the number that comes back describes the driver, not the app. ADR-0135 made the split explicit after exactly that happened on Android.

The stack

Six layers, bottom to top

Most confusion about the rig comes from mixing two of these up. Each layer answers one question and hands the next one a clean input. Select a brick to read what it does, what it is made of, and how it fails.

The surfaces

Eleven surfaces, one rig each

A surface is a platform, a piece of hardware, and a runtime: an iPad or a phone or a Mac; a real device, a simulator, or the host itself; the app in a browser or the app in its native shell. The campaign plan module names all eleven and says how each is driven. Pick one and the rig below rebuilds itself.

iPad device · web

🖥️ Capture host (Mac)

📱 Device

What this surface is for

Commands

    Worth knowing

      Three tiers, never interchangeable. Physical devices are the only rows that gate a release. Simulators and emulators are a rejection tier: a failure there is a useful lead, a pass proves nothing. The Mac rows are advisory calibration, because nothing on a Mac claims to be a hand. Never relabel one tier's result as another tier's approval (ADR-0156).

      The fingers

      Four transports, one at a time

      A transport is a pairing: a way to make a touch and a way to read the diary back. Only four survived. Each card shows the path a touch takes from the Mac to the glass, then the path the numbers take home.

      appium iPad, and Android taps

      Node runner→Appium server :4723→XCUITest / UiAutomator2→WebDriverAgent :8100→trusted touch
      Input
      Apple's own UI-testing framework injects touches through the real event path, so events arrive trusted and shaped like a digitizer. On Android the same role is played by UiAutomator2.
      Read back
      The same Appium session's script channel. It switches between the web context, where it reads the probe's tables, and the native context, where it draws.
      Why it stays
      The only transport that gives an app shell plus context switching, which is what native work needs. It also taps perfectly well, so it kept the Android action sweeps.
      Costs
      On iOS 17 and newer, device discovery needs a root-owned RemoteXPC tunnel whose password prompt a human must answer once per host boot.

      split all Android drawing

      Node runner→adb shell input swipe→OS touch stream→the page draws⇢HTTP upload→probe host :4175
      Input
      The platform's own trusted injection. On Android that is an adb swipe, which is an operating-system touch stream, the kind Chrome's display boost responds to. On iPad it is WebDriverAgent's HTTP API reached over USB with iproxy.
      Read back
      A separate HTTP host that proxies the preview page and injects one same-origin script. The page uploads its tables to it when the gesture ends.
      Why it exists
      No debugger channel and no tunnel, so it runs unattended. ADR-0135. It is the transport that made Android cells scoreable at all.
      Guards
      A nonce on the plan, proof the route hydrated, and proof a touch at the canvas centre would hit the paper. Each one caught a capture that would otherwise have been scored.

      cdp Android Chrome actions

      Playwright→adb forward→DevTools socket→Input.dispatchTouchEvent
      Input
      Chrome DevTools Protocol touch events over a forwarded DevTools socket. Trusted, and direct.
      Read back
      The same CDP connection reads the in-page probe. The connection also carries Tracing, the only instrument here that attributes cost to page work.
      Why it exists
      ADR-0092. Appium's browser path intermittently paused frame presentation: a 75 ms drawer read 16.8 ms through CDP and 166 to 350 ms through Appium. A full CDP sweep cut the failing actions from 34 of 46 to nine with no product change.
      Limits
      Android browser only. No iOS device exposes a CDP endpoint, which is the single fact that gives iOS its own transport family.

      desktop Mac Chrome, Safari, Firefox

      Playwright→browser in process→probe fires its own pointer events
      Input
      The probe's own synthetic pointer events for drawing, and Playwright's mouse for actions. Untrusted, and honest about it.
      Read back
      Same process. Nothing crosses a cable.
      Why it exists
      A frame-pacing baseline costs a command instead of a USB cable. A stall that reproduces here is a cheap regression signal.
      Limits
      A stall that does not reproduce here says nothing about a device. Desktop calibration stays advisory, and the Mac rows never gate a release.

      Two asymmetries tell the whole history

      Android draws and taps over different transports

      Appium under-drove the drawing stream so badly that its cells could not be scored, yet it taps a button perfectly well. So drawing moved to the split transport and the action sweeps stayed where they were: on direct CDP for Chrome, on Appium for the native WebView.

      The iPad rows are the only ones that still need a person

      Appium discovers a modern iPad through a root-owned tunnel. Its password prompt goes to the macOS GUI, and an overnight run cannot type into it. One human, once per host boot, is the exact cost an unattended run cannot pay.

      Targets outside the campaign table also exist. perf:web, perf:web:mount, perf:web:settings and perf:web:undo drive headless Chromium under CPU throttle as the local regression tier, and perf:web:undo:webkit is the CI commit gate in Playwright WebKit.

      Why the split

      Input cadence is a result, not a detail

      Chrome raises the phone's display to 120 Hz only while touch keeps arriving. Drive the page too slowly and the panel falls back to 60 Hz, the gaps between sparse touches get priced as dropped frames, and a healthy build reads as a catastrophe. This is what the first Android campaign published before anyone measured the finger.

      Contact moves per second, by who made the touch

      Measured on the campaign devices

      automated, passes the floorautomated, below the floora real fingerthe retired 100–170 band

      The band was set from the automation, not from a hand. When people finally drew on both devices they produced 178 on the phone and 268 on the iPad, and the gate rejected a real finger on the very iPad it was calibrated from. Cadence is now a floor only, and it gates on moves per frame rather than a rate (ADR-0141, ADR-0145).

      What the same build scored

      Android device, browser, lost frame time

      Ten to twelve percent lost frame time looks like a regression. It was the gaps between 47 touches a second, priced as frames the page failed to produce. The published rows made that way were not a measurement of the product at all.

      Under-driving does not error. The capture completes, the artifact parses, the number is plausible. The only defence is a fidelity verdict on every capture, read before the result, and never widened to let a transport pass.

      Try it: moves per frame

      Slide the input rate and switch the panel's refresh rate. The check is density, not speed.

      —moves per frame
      —gap between moves
      —cadence check (≥ 0.9)

      A rate gate would have passed 116 moves per second at 60 Hz and failed it at 120 Hz, because the rate encodes the panel, not the stream. Moves per frame asks the question the metric cares about: was the page given something to draw in each frame it could draw?

      The diary

      Inside the probe

      The probe is one file, injected into the real app at /, pasteable by hand into a Web Inspector console. It hooks pointer events, requestAnimationFrame and the engine's performance marks, and appends numeric rows. It computes nothing, on purpose: a capture must outlive the metric definitions it was taken with.

      Three timestamps make the two numbers that matter

      One touch, one handler, one frame

      Timeline showing a touch timestamp, the handler running later, and the next animation frame after that Display frames, one rAF callback each frame nn+1n+2n+3 input queue delay paint latency: move → next frame Input touch happenedevent.timeStamp handler ranperformance.now()

      Input queue delay is main-thread congestion a child feels as lag with every frame on time. Paint latency is how stale the ink is when a frame finally runs. A frame gap over four observed beats, with at least two trusted moves inside it and engine work covering under a tenth of it, is a starvation episode: the page was given input, produced nothing, and the engine was not why.

      Six tables come back, and they are read by position

      frames[]

      One row per animation-frame callback: time, delta, and whether a finger was down.

      events[]

      Every pointer event with its own timestamp, the handler's time, trust, pressure, contact size, and coalesced count.

      measures[]

      The engine's performance.measure spans: draw, commit, undo, resize, empty-scan.

      history[]

      One row per finger-lift from the read-only undo seam: entries, patch bytes, folded base bytes.

      liftLatencies[]

      Finger-up to halo-gone, the whole lift path in one number.

      phases[]

      Which condition owned which span of time, so suppressions can be compared.

      What the build has to carry

      Two flags decide whether a page can be measured at all. PERF_MARKS=true compiles the engine's marks in. PUBLIC_ENABLE_DEV_HARNESS=true unlocks the /dev/engine harness route and the read-only seams the probe reads through. The drawing seam is read-only on purpose: a probe that can change the renderer can invalidate its own measurement.

      Release builds compile both out, and tools/check-release-seams.mjs scans the shipped client for the seams and the mark names. If a report says the build had no engine measures, the build was not an instrumented one, and the capture has to be thrown away.

      tools/perf/probes/real-screen-probe.jsthe contract 
      // It is a RECORDER, not an analyzer. It appends
// numeric rows and hands them back verbatim;
// every percentile, verdict and comparison is
// computed in Node by analyze-frame-capture.mjs,
// where it can be unit-tested.

frames.push([t, dt, contact]);
events.push([stamp, at, type, id, buttons,
  coalesced, onCanvas, kind, trusted,
  pressure, width, height, …]);
      Why "recorder, not analyzer" is not a style choice. Four metric definitions were wrong in the first device capture: the assumed frame budget, move gaps that spanned stroke boundaries, paint latency that counted the idle after a lift, and a lift into an idle page reported as a 2.4 second hitch. All four were corrected against that same capture with no redrawing, because the rows were still there. perf:rescore later re-priced an entire corpus three times in one campaign.

      The platform's eyes

      What the page cannot see

      The probe knows when its animation-frame callback ran. It does not know whether a pixel reached the glass, or whether the app was slow or simply descheduled. Those answers live in the operating system, and the two platforms give very different amounts of them up. Select an instrument to see how far along the frame pipeline it can look.

      Touchdigitizer → OS → browser
      Scriptengine.draw, Svelte
      Style / layoutrecalc, reflow
      Paint / rastercanvas → GPU
      Compositor commitrenderer hands off
      System compositorSurfaceFlinger, WindowServer
      Presentthe glass
      sees this stagepartly, or only for some targetsblind

      When two instruments disagree, the disagreement is the finding

      A frame the probe scores as lost is one the page failed to produce. A frame gfxinfo scores as janky may have been produced on time and missed its deadline somewhere the page cannot observe. The probe and CDP see the web content; gfxinfo and Perfetto see the platform.

      Android gives up far more than iOS

      The whole-device tracer, the per-frame view-system stages and a renderer trace are all reachable over adb. On an iPad the only frame-level instrument is xctrace, correlated to the probe through timeOrigin, and the question "which function made crayon expensive" has no answer there at all.

      Does the instrument move the number? Measure it.

      Same gesture, scored by the app's own probe

      Instruments on iPad, lost frame time %

      Time Profiler attached to one process or to every process changed nothing the probe could see. Input cadence and marked script work were identical, so the app was not slowed. That result covers sampling templates only: Perfetto's sched is closer to instrumenting than sampling, and has to be measured on its own.

      The probe against a bare counter, Android, frames per second

      The probe cannot score the arm that has no probe, so a minimal frame counter rides in both arms and cancels. In steady state the probe's cost is unmeasurable. A single worst-frame asymmetry appeared once and inverted on the next run: three samples of a single-frame statistic, which is exactly the shape that looks consistent by chance.

      From rows to verdicts

      Scoring happens on the Mac, after the fact

      Every number in a report is derived in Node from the raw tables, so a corpus can be re-priced when a gate turns out to be wrong. Four ideas carry most of the weight: the beat, the refresh regime, the lost-frame share, and the fidelity verdict that decides whether a capture may be scored at all.

      The beat

      The capture's frame interval, derived as the dominant interval rather than a percentile, because a percentile drags toward doubled intervals and charges the app for on-time frames (ADR-0134). Safari gives web content a 60 Hz beat even on a 120 Hz ProMotion iPad, so a loop pacing at 17 ms there is at the ceiling, not failing.

      The refresh regime

      The presentation rate a surface is scored against, declared per target from measured captures. A beat inside 16 to 17 ms is the 60 Hz regime; 8.3 to 8.42 ms is 120 Hz; a beat outside every band is no regime, not the nearest one. It exists because the same iPad cell minutes apart produced two correctly derived numbers that differed sixfold.

      Lost frame time share

      Lost time over in-contact time, priced against the observed beat. A late frame that the next frame gives back is not charged: on a ProMotion iPad, 93% of late frames were the long half of a pair summing to two beats (ADR-0136). It is a share of time, not of frames; the frame share is roughly double.

      The fidelity verdict

      Whether the capture may be scored. Two universal checks: trustedTouch, every event went through the real touch path, and cadence, at least 0.9 moves per frame with a p95 gap under 25 ms. A failed check is a bad run. An uncalibrated check is a silent instrument. A check that is absent was measured and found to carry no information.

      What each runtime is asked

      From tools/perf/lib/input-fidelity.mjs

      Capture runtime trustedTouch cadence pressure contact geometry coalescing
      ios-safari checked checked none reported finger-sized witness only
      ios-capacitor-webview checked checked none reported finger-sized witness only
      android-chrome checked checked not applicable not applicable witness only
      android-capacitor-webview checked checked not applicable not applicable witness only
      desktop-playwright never passes checked uncalibrated uncalibrated witness only

      Android's pressure and contact-geometry checks were dropped by measurement, not by widening: a real finger and an adb swipe on the same phone report them identically, so they cannot tell a hand from a robot (ADR-0141). A not-applicable check is absent from the verdict rather than present and true, so silence can never be mistaken for a pass. Coalescing is recorded as a witness in every runtime because the value tracks page delivery, not input (ADR-0144).

      The drawing gates, once fidelity passes

      ≤ 20 mspaint latency, P95move → next frame
      ≤ 33 mspaint latency, P99two 60 Hz frames
      ≤ 50 mspaint latency, worstthree frames
      ≤ 1%lost frame timeshare of in-contact time

      Undo adds engine P95 ≤ 20 ms, next-frame P95 ≤ 33 ms and next-frame max ≤ 50 ms. The action sweeps gate frame P95 at 20 ms and first or worst frame at 33.5 ms, with a worst-frame breach counting only when two of the three scored repeats show it (ADR-0156). Every gate is a constant in tools/perf/lib/, not a number in prose.

      Not-applicable versus uncalibrated is the distinction people get wrong. An uncalibrated check is a gap the instrument can still close by measuring; recapturing changes nothing, and it is not a pass. A not-applicable check has been measured and found to carry no information, and is left out of the recorded checks entirely. Both are named in the artifact so a reader can tell which one they are looking at.

      Putting it together

      A campaign, end to end

      A campaign captures one surface's whole grid: four modes, five items each, then folds the results into the published performance matrix. The plan module owns the queue shape, the retry policy and the resume ledger, so the only host-specific inputs are device identity and two URLs.

      Preflightdevices, tunnel, ports, input, rotationperf:preflight
      Serveinstrumented preview + probe hostperf:serve · perf:device:serve
      Queue20 cells, 3 attempts each, resumableperf:campaign
      Statusdone or not, from artifacts on diskperf:campaign:status
      Foldwhole modes into the manifestperf:campaign:sources
      Regenerateuntouched cells stay preservedgen:performance-matrix
      Keepone capture per target × brushperf:evidence:keep

      One surface's grid

      Press play to watch a queue run with the outcomes a real ledger records

      Reference cell, crayon, first mode:
      bankedfailed fidelity, retriedattempts exhausted

      Every drawing cell is driven at 10 gesture repeats and the pen cell adds 10 undos; action cells take one warmup and three scored repeats. A cell whose capture fails fidelity spends an attempt but is recorded as failed-input-fidelity, never as a missing artifact. A reference cell repeats at the start, middle and end so drift inside the session has a control inside the session.

      The unit of folding is a mode

      A mode is rewritten only when all four brushes and its action sweep are present and were captured through the target's own transport. A partially captured mode is not a captured one.

      The instrument is fingerprinted

      An instrument.json beside the ledger hashes the modules that decide what a capture measures. Resuming across a change is refused with the changed files named. Scorers are outside the fingerprint on purpose: they re-derive at fold time.

      Preserved is not current

      The matrix carries cells forward from its own previous output, so a gate correction changes the gates block and nothing else. A matrix number and a fresh capture of the same cell can legitimately disagree by more than the gate.

      All the bricks at once

      The whole rig on one page

      Every earlier section is a piece of this picture. One Mac drives three kinds of screen. Solid arrows are the fingers, one colour per transport. Dashed arrows are the couriers. Eyes mark what the platform can add. Everything comes home to the same Node scorer.

      Composite diagram of the capture host, the three device classes, the four transports and the shared scorer CAPTURE HOST · MAC Instrumented buildPERF_MARKS · dev harness seams Preview server :4173serves the page over the LAN Probe host :4175proxies the page, receives uploads Appium :4723 + tunnelXCUITest, UiAutomator2, one human adb serverinput swipe out, DevTools forward in Playwrightdesktop engines in process, CDP client SCORING · PURE NODE beat → refresh regime → fidelity verdict → gates → campaign fold → matrix perf:rescore re-prices any corpus IPAD · DEVICE OR SIMULATOR Safari tabweb row, LAN preview WKWebView appnative row, bundled page page + probesix tables, no arithmetic 👁 xctraceframe lifetimes, hitches ANDROID · DEVICE OR EMULATOR Chrome tabweb row WebView appnative row, server.url page + probeuploads with the run nonce 👁 gfxinfo · Perfetto+ CDP Tracing in the renderer SAME MAC · A BROWSER WINDOW Chromium · WebKit · Firefoxadvisory rows, no hand claimed page + probefires its own pointer events 👁 CDP TracingChromium only appium · XCUITest via WebDriverAgent back over the Appium script channel, or a plist split · adb shell input swipe cdp · taps for Chrome actions back as an HTTP upload to the probe host desktop · synthetic events, same process back in process, nothing crosses a cable
      appiumsplitcdpdesktopmeasurement back to the Mac👁 platform instruments

      Read it top to bottom for fidelity

      The iPad row is the surface users touch, with real WebKit, the Apple GPU and a trusted digitizer path. Android is where the platform gives the most back. The Mac window is a baseline that costs a command.

      Read it left to right for trust

      Everything on the left can be wrong in ways that do not error: a stale build, a cached shell, a persisted brush, a degraded adb server. Everything on the right records what actually happened. The verdict on the far left decides whether the two agree.

      Read the arrows for the split

      Where a solid and a dashed arrow share a channel, the driver can disturb the measurement. Where they do not, the capture can run overnight. That difference is why Android draws over adb and the iPad still needs one person at the tunnel prompt.

      Numbers that look fine and are wrong

      The traps that do not raise an error

      Every entry here shares one shape: the capture completes, writes a well-formed artifact, and reports a plausible number. One August campaign hit six of them, and three were each enough on their own to fail every physical Android drawing cell. Filter by surface.

      Before believing a result

      The fidelity verdict passed and the cadence is in band.Read it before the number, every time.
      The served build is the one you meant, verified rather than assumed.A resolving manifest proves a server is self-consistent, not whose build it is.
      The committed brush matches the requested one.The brush persists across captures on a shared origin.
      At least three samples per cell.The within-configuration spread on a physical device is routinely as large as the effect being measured.
      The previous run of the same cell, for comparison.A single absolute number from this gate has been wrong more often than right.
      The commit the cell was captured at.A red cell can be a faithful measurement of a build nobody runs any more. Rebuild that commit before treating it as a product problem.

      Vocabulary

      Every term on this page

      Grouped by what kind of thing it is. Type to filter.

      Read the real thing

      Where this page came from

      Everything above is a summary. These are the documents and modules it summarizes, and the decisions that shaped them.

      SourceWhat it owns
      docs/PROFILING-MECHANICS.mdThe six-layer model, the transport table, the platform instruments, every ruled-out driver, and the glossary this page's vocabulary follows.
      docs/PROFILING.mdWhat each npm run perf:* command measures, how capture works, and how to read a report into a bottleneck.
      docs/PROFILING-IPAD.mdThe physical iPad runbook: one-time setup, the tunnel, the gates run, the real-screen capture, the bundled app path, and the Timeline recording that stays manual.
      docs/PROFILING-ANDROID.mdThe Android toolchain: gfxinfo framestats, Perfetto, and CDP Tracing, and which one answers which question.
      docs/PROFILING-CAMPAIGNS.mdThe full catalogue of setup mistakes that produce plausible wrong numbers, the campaign loop, and the before-believing checklist.
      tools/perf/lib/campaign-plan.mjsThe eleven targets, their transports, fidelity runtimes and refresh regimes, and the queue shape. A test fails if the mechanics doc disagrees with it.
      tools/perf/lib/input-fidelity.mjsThe per-runtime expectations table, the density floor, and the not-applicable versus uncalibrated distinction.
      tools/perf/probes/real-screen-probe.jsThe recorder itself, with the six table layouts.
      tools/perf/split-capture/README.mdThe split transport's mechanics, its three guards, and how the Android fidelity gate was closed by measurement.
      The deployment-target performance matrixWhat the rig produces: every surface, every mode, every brush and action, scored against these gates.

      Decisions behind the rig

      ADR-0032 · the harness ADR-0079 · WebKit Inspector Protocol ADR-0084 · trusted XCUITest input ADR-0090 · tiered device gates ADR-0092 · direct CDP for Android actions ADR-0134 · the beat ADR-0135 · split input and measurement ADR-0136 · lost-frame gate ADR-0138 · evidence subset ADR-0139 · per-runtime fidelity ADR-0141 · cadence is a floor ADR-0143 · pin the panel for actions ADR-0144 · coalescing is a witness ADR-0145 · density, not rate ADR-0151 · bundled iOS reports ADR-0156 · physical rows gate ADR-0159 · staleness reported