How the drawing engine works

1Start here

What a drawing engine has to do

Every drawing app, from a paint program to a toddler doodle app, solves the same four problems. The solutions differ. The problems do not.

How Splotch answers them, in one paragraph

Splotch keeps the drawing in two forms at once. Recent strokes stay as vector instructions ("a blue path, 18 px wide, through these points"). Older strokes are baked into pixels. The visible paper is not one big canvas but a grid of twenty smaller ones, so a stroke only repaints the tiles it crosses. Before a stroke changes a tile, the engine copies the pixels that are about to change, so Undo is a paste rather than a replay. Saving hands the finished tiles to a worker thread, which assembles and encodes the PNG while the child keeps drawing.

2Try it

One stroke, start to finish

Pick a stage to read what happens, then draw on the paper below to watch it happen. The lab runs a simplified copy of the real engine's rules: the same 4 × 5 grid, the same midpoint curves, the same one-op-per-frame batching, and the same "copy before you change it" undo. The lab keeps 4 undo steps so folding is visible quickly. The real app keeps 20.

3The parts

One conductor, several kinds of memory

The engine is a plain TypeScript module, not a Svelte component. The UI pushes tool choices in and receives a few callbacks out. Pointer handling and canvas pixels never go through the reactive render loop, because a reactive update per pointer event would be far too slow.

While drawing

1. Pointer eventstouch · pen · mouse
2. Enginecoordinates · fingers · tools
3. Opsdot · path · clear
4. Shared renderersame brush rules everywhere
5. Live tilespixels on screen this frame

The same ops also go to

1. Vector tailthe newest 20 commands + their patches
2. Pixel baseolder commands, baked when idle
3. Save workerpaper + texture + tiles + line art → PNG

The screen is one view of the drawing. It is not the only copy.

The layer stack you actually see

From top to bottom, the drawing area is four DOM layers. The warm paper is CSS, not a canvas. Only the ink and the pixel base use the tile grid.

Not in the DOM: twenty more offscreen canvases hold the pixel base, the baked older strokes. They use the same 4 × 5 layout so a tile can be folded into its base tile with one copy.

4Input

Many fingers, one undoable stroke

Each finger on the glass gets its own record: position, last midpoint, width, brush flags, and a small speed history. But every finger that is down at the same time writes into one shared command. Lifting one finger does not end the command. Lifting the last one does.

const activePointers = new Map<number, PointerState>();

function beginStrokeGroup() {
  if (groupHasDrawn) return;
  beginTiledCommand(canvasEmpty);
  groupHasDrawn = true;
}

// A finger lifting does not commit while another finger remains.
activePointers.delete(event.pointerId);
if (activePointers.size === 0) finishStrokeGroup();

From dots to a smooth line

The browser usually delivers one pointermove per display frame, but the touch hardware samples faster than that. The extra samples are available through getCoalescedEvents(), and Splotch reads all of them. A fast scribble on an iPad produces about two to four samples per frame.

For each sample, the engine draws a quadratic curve from the previous midpoint to the new midpoint, using the previous raw sample as the control point. Neighbouring curves share a tangent where they meet, so corners look smooth with no curve-fitting step. Turn on "Straight chords" in the lab to see what the raw samples look like without this.

for (const { x, y } of points) {
  const midX = (pointer.x + x) / 2;
  const midY = (pointer.y + y) / 2;
  op.segs.push({ cx: pointer.x, cy: pointer.y, x: midX, y: midY });
  pointer.x = x; pointer.y = y;
}
// later, when the op is painted:
context.quadraticCurveTo(seg.cx, seg.cy, seg.x, seg.y);

One op per frame, not one per event

Painting inside the event handler would repaint the same frame two to four times, and every pass pays the per-tile cost again. Instead the engine queues the samples and drains them once per animation frame into a single path op. The pixels are identical, because the op still carries every sample as its own curve segment. The per-op bookkeeping is paid once. Watch the "Ops" counter in the lab grow more slowly than "Samples".

5Tiles

Why the paper is cut into twenty pieces

On a large iPad, the JavaScript finished painting quickly and the screen still froze for hundreds of milliseconds. The browser was flushing one huge changed canvas to the display. The variable that mattered was the size of the largest surface that changed, not the total number of pixels painted. Splitting the paper into a grid of smaller canvases kept the same resolution and removed the freeze.

Cull first, paint second

Every op carries paper-space geometry. Before painting, the renderer computes a box around the op's points and control points, pads it by half the stroke width plus a 2 px anti-aliasing bleed, and tests that box against each tile. This is culling. Only tiles the box touches allocate memory, get an undo copy, become visible, and run the renderer. Each tile's drawing transform subtracts the tile's origin, so every op stays in one global paper coordinate system.

for (const [index, tile] of liveTiles.entries()) {
  if (!geometryIntersectsTile(op, tile)) continue;

  ensureNormalTileBacking(tile);
  prepareTileForMutation(tile, index);
  if (command) undoPatches.capture(command, tile, index, opDeviceBounds(tile, op));
  renderOp(tile.ctx, op);
}

How the grid size was chosen

The grid started as 4 × 4. A later sweep on the physical iPad compared one smaller and two larger grids, using the crayon brush because it stresses surfaces the most. Each row is three captures of the same gesture. "Lost frames" is the share of frames the screen failed to present on time.

The slow 4 × 4 frames carried at most 1 ms of engine JavaScript. The time was in the browser's own surface work, which is why the fix was topology rather than code.

Knowing when the paper is blank

Undo, Clear, and the camera button disable themselves when the paper is empty. Checking that means reading pixels back, which is slow on a GPU-backed canvas and gets slower if you do it often. So each tile is copied at quarter size onto one small CPU-side scratch canvas and scanned there. Even that costs a few milliseconds across twenty tiles, so the scan waits until 400 ms after the last input rather than running during a stroke.

6Undo

Vectors on top, pixels underneath

Undo is a hybrid. Recent commands are kept as vector ops and own cropped pixel patches of what was under them. Old commands lose their patches, then fold into the pixel base while the child is not drawing. The lab's history strip shows the same three states.

The life of one command

tile.ctx.save();
tile.ctx.setTransform(1, 0, 0, 1, 0, 0);
tile.ctx.clearRect(snapshot.x, snapshot.y, snapshot.canvas.width, snapshot.canvas.height);
tile.ctx.drawImage(snapshot.canvas, snapshot.x, snapshot.y); // the restoring BLIT
tile.ctx.restore();

Clear is one command spread across frames

Clearing many full tiles and copying them all in one go caused long frames. Clear now hides the affected tiles immediately, so it looks instant. Then it copies at most one tile per animation frame for the undo patch, and wipes each hidden tile two frames after its copy. An Undo that arrives mid-way either restores the finished patches or simply unhides the tiles that were never wiped.

7Pixels

Vectors, rasters, and what "BLIT" means

The four op kinds

Brush flags on a dot or path: erase, magic with its captured sheet, and crayon with its texture seed.

8Brushes

One geometry pipeline, four paint styles

Every brush produces the same dot and path ops. The flags on the op decide how the shared renderer turns that shape into pixels. This is what keeps undo, export, and rotation brush-agnostic.

Magic: paint through a color sheet

The colored version of a coloring page, or a generated rainbow, is fitted into paper space to make a "sheet". Where the platform allows it, a worker fetches, decodes, and rasterizes that sheet and hands back an ImageBitmap. A Magic stroke does not paint a color. It paints a window onto the sheet, so scribbling reveals the picture underneath.

• Each tile crops its own pattern. WebKit samples a full-page pattern slowly, so every tile gets a pattern source cut to its own rectangle and offset to line up.
• The first paint captures the sheet. A Magic op remembers which sheet it painted from. Changing pages later does not silently recolor old ink.
• Page changes recode. Applying or removing a coloring page recodes the retained Magic ops to the new sheet as one undoable command. Ink that has already folded into the pixel base is rebuilt from a kept raster baseline plus the folded ops, in draw order.
• Not ready yet means paint nothing. An op created before its sheet arrives paints nothing, and one repaint resolves it when the sheet lands.
• Rainbows are pooled. Ten random gradients are generated up front, each with a random angle, 5 to 8 stops, and a hue sweep that wraps past 360° so every rainbow misses a different color.

9Paper space

Coordinates, sharpness, rotation, and recovery

Ops are never tied to the phone's current orientation. They live in paper space, a fixed pixel coordinate system adopted from the viewport when drawing starts. The screen is a view onto that paper.

1. Browser client x/yCSS pixels in the viewport
2. Backing x/ycached rect × render scale
3. Inverse viewundo the contain-fit transform
4. Paper x/ystored in every op
5. Tile-local x/ypaper − tile origin

The paper locks when it has ink

On a blank canvas, a resize adopts the new viewport size. Once there is ink, a rotation keeps the old paper size. The paper sheet, the line art, and the whole tile wrapper receive one CSS contain-fit transform that scales and centers them in the new viewport. Pointer input goes through the inverse of the same matrix. No tile is resized and no ink is replayed. Rotating back reveals the exact original pixels.

Two scales

The engine starts before the page is interactive

The home page is prerendered HTML. As soon as the engine module loads, it finds the existing input canvas and starts listening for pointers. Later, when Svelte hydrates, the drawing component adopts the running engine and wires up its callbacks. Without this, the canvas looked ready for several hundred milliseconds while it could not draw.

When the browser throws the pixels away

A backgrounded tab or app can lose its canvas contents and drawing state. This is context loss. An Android phone came back from the background with a blank paper where new ink appeared only as small clipped fragments, because each tile's transform had reset to identity while the app kept running. The engine now listens for the browser's loss and restore events on every surface, and on every resume it also probes each context for its expected round line caps and tile transform, since some resets are silent. Recovery rebinds all contexts, reapplies their state, and repaints from the pixel base plus the vector tail.

10Save

Saving without freezing the drawing

The saved PNG contains the paper color, the paper texture, the ink, and the coloring-page line art. Assembling a full-size page and encoding it is slow, so the whole job runs on a worker whenever the browser supports the pieces.

1. Snapshotsynchronous, before any await
2. Tile bitmapsonly visible, settled tiles
3. Worker composesOffscreenCanvas at export scale
4. Stackpaper → texture → tiles → line art
5. PNG encode15 s timeout, one retry on context loss

const scale = currentExportScale();

// Must happen before loading the export module.
const snapshots = snapshotStrokes(scale, capturePreview);

const { composeExportPng } = await import('./exportDrawing');
return composeExportPng(snapshots.export, scale, overlaySource, options);

Taking the snapshot before the first await is what makes save-on-delete safe. The Clear button starts an export and wipes the paper in the same tap. Because the snapshot is already taken, the saved picture is complete even though the live tiles are empty by the time the worker runs.

11The numbers

What the measurements said

Almost everything above exists because a physical iPad said the previous design was too slow. Each change landed with before-and-after captures on the real device, driven by trusted synthetic touch and scored from a recording of the screen itself. Emulators and headless browsers passed while the real screen froze.

Platform facts you cannot see from a laptop

Working code, retired by a number

12Glossary

The words, explained

Each entry says what the thing is, why it matters here, and where it shows up in Splotch. Written in plain technical English.

Anti-aliasing

The browser makes the edge of a curve look smooth by painting partly transparent pixels along it. Those edge pixels reach slightly outside the exact shape. Splotch adds 2 px to every bounding box so culling and undo patches include them.

Animation frame

The browser presents a new image to the screen at a steady rate, usually 60 times a second. requestAnimationFrame runs your code once before each frame. Splotch drains queued samples, captures clear patches, and warms caches one step per frame so no single frame does too much.

Backing store

The real bitmap behind a canvas, measured in device pixels. It can be a different size from the canvas's CSS size on screen. Splotch's input canvas has a 1 × 1 backing store but a full-size CSS box.

BLIT

Short for "bit block transfer". A copy of a rectangle of existing pixels from one bitmap to another, with no drawing math. In Canvas 2D it is a drawImage call with a canvas or bitmap as the source. Splotch uses BLITs for undo, folding, and export.

Canvas

An HTML element that holds a bitmap you can draw on with JavaScript. The "2D context" is the drawing API: paths, fills, strokes, transforms, and compositing rules. Splotch uses about sixty on-screen canvases and twenty offscreen ones.

Coalesced events

Touch hardware samples position more often than the browser fires pointermove. The skipped samples are attached to the event and available through getCoalescedEvents(). Reading them keeps fast scribbles curved instead of turning them into long straight chords.

Command

One undoable unit. In Splotch it is every finger that was down at the same time, from the first touch to the last lift. A Clear is also one command.

Compositing

The rule for combining new pixels with the pixels already there. The rule is set by globalCompositeOperation. Splotch uses source-over for pen and Magic, destination-out for the eraser, and darken for crayon mixing.

Contain-fit

Scale something by min(boxWidth / paperWidth, boxHeight / paperHeight) so all of it fits inside a box, then center it. The same matrix presents the paper after rotation and maps pointer input back onto it.

Context loss

The operating system can take a canvas's graphics memory away, usually when the app is in the background. The pixels and the drawing state are gone when it comes back. Splotch listens for the browser's loss and restore events, probes for silent resets on resume, and repaints from history.

Culling

Skipping work that cannot affect the result. Here it means skipping every tile whose rectangle does not touch an op's padded bounding box. A short stroke usually touches one or two tiles out of twenty.

Destination-out

A compositing rule where the new shape removes alpha from what is already there instead of adding color. It is how the eraser works: the paper shows through the hole.

Device pixel ratio

How many physical screen pixels make up one CSS pixel. A phone with a ratio of 3 has three hardware pixels per CSS pixel in each direction. Splotch draws at no more than 2 and saves at no less than 2.

Dirty rectangle

The smallest box that safely contains every pixel a command may have changed. Splotch grows it as ops arrive and uses it to crop undo patches. It is conservative: a little too big is fine, too small is a bug.

Fold

Take an old vector command, paint it into the offscreen pixel base, and delete its vector form. This bounds how much history is kept. Folding only runs after 1.5 s of no input, one command at a time.

Frame and frame budget

One image presented to the screen. At 60 Hz there are 16.7 ms between frames, and all input handling, drawing, and browser work for that frame must fit inside it. A frame that misses is a "lost frame" and shows up as a stutter.

Hydration

When a prerendered HTML page becomes interactive because the JavaScript framework attaches to it. It can take a few hundred milliseconds. Splotch starts the engine before hydration so the first touch is never lost.

ImageBitmap

A decoded picture that can be handed to another thread without copying. Splotch transfers tile bitmaps to the export worker and Magic sheets back from the sheet worker this way.

Main thread

The single thread that runs page JavaScript, handles input, and lays out the page. Anything slow on it delays the next frame. Splotch moves PNG encoding and sheet rasterization to workers to keep it free.

OffscreenCanvas

A canvas that is not in the page and can live on a worker. The export worker builds the full-page picture on one so the main thread never owns a tablet-sized surface.

Op (operation)

One drawing instruction. Splotch has four kinds: dot, path, crayonFlush, and clear. A path op holds a start point, a list of curve segments, a width, and brush flags. Ops are what history stores and what the renderer paints.

Paper space

The fixed coordinate system the drawing lives in, in device pixels of the paper as it was adopted. Ops, tiles, patches, and Magic sheets all use it. Rotation changes how paper space is shown, never what is in it.

Patch

A cropped copy of a tile's pixels from just before a command changed them. Undo pastes the patch back. Patches are cheap to restore but cost memory, so a byte budget drops the oldest ones.

Pattern

A canvas fill made from an image instead of a color. Magic strokes use a pattern cut from the color sheet, aligned so the picture appears in place. Crayon uses patterns of wax texture.

Pixel base

Twenty offscreen tiles that hold every command too old to undo, as plain pixels. It is the bottom layer of history. Live tiles show the base plus the vector tail plus whatever is being drawn.

Pointer capture

Telling the browser to keep sending a pointer's events to one element even when the pointer moves over something else. Without it, dragging across a button would end the stroke.

Pointer event

The browser's single event type for mouse, touch, and pen. Each contact has a pointer id, and the events are pointerdown, pointermove, and pointerup. Splotch handles all input through them.

P95

The 95th percentile. If paint time P95 is 16 ms, then 95 out of 100 frames painted in 16 ms or less. It describes the typical bad case rather than the average, which hides stutter.

Quadratic curve

A curve defined by a start point, one control point, and an end point. The line bends toward the control point without passing through it. Splotch draws one per input sample, from midpoint to midpoint, with the raw sample as the control point.

Raster, rasterize

A raster is a grid of colored pixels. To rasterize is to turn a shape description into that grid. Canvas backing stores, patches, the pixel base, and PNG files are all rasters.

Source-over

The default compositing rule: put the new pixels on top, using their transparency to blend with what is there. Pen and Magic use it.

Tile

One cell of the 4 × 5 grid that covers the paper. Each tile is its own canvas with its own backing store. Twenty live tiles are in the page; twenty base tiles are offscreen.

Transform

A small matrix that moves, scales, or rotates coordinates before drawing. Each tile's context carries a transform that subtracts the tile's origin, so an op in paper coordinates lands in the right spot on that tile.

Vector

A drawing stored as shapes and instructions rather than pixels. It can be redrawn at any size. Splotch's recent history is vector so it can be replayed for export at a different scale or after a recovery.

Worker

A separate JavaScript thread with no access to the page. It cannot block input or frames. Splotch uses one to rasterize Magic sheets and one to compose and encode PNGs.

Where to read the code

Decision records behind this page