How data moves between the shaders in webpaint.ing
By releases ·
webpaint.ing is the drawing app that sits beside this site. This note follows the data: which pictures and buffers exist, who writes and reads each, in what order and in which frame, and where data crosses between the graphics card and ordinary memory. Where something is only planned, it says so.
One rule that shapes everything
A shader cannot read the picture it is writing. And a picture drawn in one frame cannot be relied on by another picture drawn in that same frame, so it is read in a later one. Everything below follows from those two facts.
- Two pictures taking turns. If a step must build on its own last result, it reads picture A and writes picture B, and the next step reads B and writes A.
- Work spread over frames. A job that needs a finished picture is queued in one frame and runs in the next. A stroke at less than full strength is gathered in a scratch picture first and laid onto its layer a frame later.
- A result read a frame later. The card answers questions about its pictures late: on WebGPU, frames later. So a question is asked, the app keeps drawing, and the answer is collected when it arrives.
Only two things in the app are compute shaders: the flood behind Fill and Select by colour, and the export enlarger. The rest of the card's work is drawing shaders. The turn-taking pattern appears in both worlds: the flood uses it with compute passes, and the Watercolour brush uses it with drawing shaders, keeping its record of how much each pixel is covered in one of two pictures that each new stroke reads and writes in turn.
The flood's data, in order
Three kinds of picture matter for a fill, all on the card. The layer's own picture, drawn in an earlier frame. Two state pictures the size of the sheet, called A and B here, which take turns. And a tiny flags picture, one pixel for each of the sixteen sweeps. Each state pixel holds three numbers: whether it is in the region so far, the layer pixel's opacity, and whether it is close enough in colour to the pressed pixel. A few whole numbers (the pressed position, the tolerance, the sheet's size, the sweep's direction) are handed to each pass directly.
All of the passes are recorded in one batch, in one frame, with a barrier between passes so that each starts only when the one before it has finished.
- Before the batch. Clears the flags picture. All on the card.
- Close-enough pass. Reads the layer's picture. Writes A: every pixel's closeness and opacity, and a mark on the pressed pixel alone. On the card.
- Sweeps 1, 3, 5 and so on (along rows). Read A, write B, and write their own pixel of the flags picture if they added anything to the region. On the card.
- Sweeps 2, 4, 6 and so on (along columns). Read B, write A, and flag in the same way. Sixteen is even, so the newest picture ends up in A.
- The next frame. The fill is drawn straight from A (see below), and the app asks the card to copy A and the flags into ordinary memory.
- A few frames later. The copies arrive. The app checks that the pressed pixel is in the answer, so a late reply to another question is not mistaken for this one. If it is not, or no answer comes, nothing is filled rather than the wrong pixels.
- The check. If neither of the last two sweeps flagged a change, the region is finished and is taken from A. If one did (a maze, a spiral), the answer carries no region, and ordinary code finishes the job from the closeness values, which are also the definition of the right answer.
The fill then stores its pixels as a list of runs, made from the copy in ordinary memory.
How a compute result reaches a drawing shader
A compute shader writes into a picture. A drawing shader samples pictures. Nothing says these must be different pictures, so the app wraps state picture A, the card's own object, as an ordinary texture that the drawing side can use. The fill preview is then one small drawing shader that paints the fill colour wherever that texture says "in the region". No pixels move: the same memory on the card is first written by compute and then sampled by drawing. It works from the frame after the batch was recorded (the later-frame rule again) and stays good until the next fill reuses A. It is drawn into the buffer a stroke in progress uses, and ends when the fill's real operation lands on the layer.
Where data crosses to ordinary memory
The boundary is crossed rarely because the card answers late and the copies are large: one picture of a 2048 by 2048 sheet is 32 MB. Today there are these crossings.
- Going to the card: the pressed pixel. The press and tolerance travel as a handful of numbers.
- Coming back: the flags and the final region. Described above. This is the only copy a fill needs, and it is the part not yet moved onto the card.
- Coming back: a layer, for saving. The layer's picture is read back, cut into tiles and packed in ordinary code. The colour picker and copying also read pixels back.
- Going to the card: tiles, when a drawing opens. A layer that has not been needed for a while is kept as packed tiles in ordinary memory, so that a long animation fits. Opening a saved drawing, or looking at such a layer again, unpacks its tiles and copies them up onto the card.
Everything else stays on the card: strokes, the live stroke buffer, and the small undo patches. A patch is the rectangle an operation changed, kept as it was before, cut from a second copy of the layer that runs one operation behind. Undoing puts a patch back with no trip through ordinary memory.
The export enlarger is a separate flow
Enlarging an export by 2, 4 or 8 is the second compute shader, and it shares no pictures with the flood. It belongs to the page, not to the drawing engine, and uses the browser's own WebGPU device, which the page asks the browser for once and keeps.
- In. The frame arrives as an ordinary browser image or canvas and is copied into a small texture on that device.
- Compute. One pass reads that texture and writes a large storage texture: each pixel of the large picture takes the colour of the small pixel it lies in.
- Out. The large texture is copied onto a canvas shown by the browser, and the export takes that canvas as it takes the engine's own pictures.
- The trial first. A card's mistakes are reported late, so before an export depends on it the enlarger is run once on a picture of four plain colours, given as plain pixels. The result is checked at the middle of each square. A card that fails is not used, a plain canvas enlarges instead, and the Export window says which way was taken.
Planned, not built
- A selection kept as a picture on the card, so that adding, subtracting, growing and shrinking become compute passes. Today a selection's shape is a list of runs worked out in ordinary code.
- Fills that store the press instead of the pixels, replayed by running the flood again on the card. That would remove the fill's read-back entirely.
- Layers as tiles that exist only where there is paint.