Where a drawing lives in webpaint.ing: device, memory and graphics card
By releases ·
webpaint.ing is the drawing app that sits beside this site. A drawing is a lot of pixels, and a computer has three places to keep them, each faster and smaller than the one before. This note says what is kept where today, what moves between the places and when, and what each move costs. Anything not built yet is marked as planned.
The three places
- The device (the disk). What survives closing the page. It holds the drawing's document, a journal of every operation (each stroke, fill and move, in order), and the pictures of every layer cut into small square tiles saved as PNG files. The browser keeps these in its private file system for the site. In a browser that cannot write files there, the app works but keeps nothing between visits, and says so.
- Ordinary memory (RAM). Layers that are not being looked at are kept here as packed tiles : the same tiles, each squeezed without loss. A packed picture comes back as exactly the same bytes.
- The graphics card (graphics memory). A layer that is on screen is a whole picture here, the size of the sheet. So are the working pictures: the stroke being drawn, a scratch picture for strokes that are see-through, and the copies undo uses.
A cel is one layer on one frame of an animation. Everything below is counted per cel.
Tiles
A layer's picture is cut into squares of 256 by 256 pixels (smaller along the right and bottom edges). A tile with nothing on it, fully transparent, is not kept at all, so a line drawing on a big sheet is a handful of small tiles. Memory holds each kept tile compressed with Zstandard. The device holds each as a PNG. Nothing on the card is compressed: a picture there is plain pixels, so it can be drawn into at full speed. Packed tiles are never changed in place, which is why a copied frame can start by sharing the very same tiles as the frame it was copied from.
What a picture costs on the card
The engine counts a layer's picture as the width times the height times eight bytes: four for the pixels and four for a second buffer the drawing surface carries. That is 32 MB at 2048 by 2048, 128 MB at 4096 by 4096 and 512 MB at 8192 by 8192. Every limit below is in those terms.
- Pictures kept ready. Layers on screen always get a picture. Beyond them, pictures are kept as a cache of layers looked at lately, up to 512 MB, but never fewer than four. That is 16 pictures at 2048, 4 at 4096 and 4 at 8192.
- Undo copies. Each layer may keep whole-sheet copies to go back to, up to 256 MB and at most four. That is four at 2048, two at 4096 and none at 8192.
- Undo patches. For each operation, the rectangle it changed is kept as it was before. A layer's patches may total 256 MB and number 64, oldest first to go. One patch may be as large as the whole sheet, so the last operation can always be undone by putting one picture back.
- Working pictures. The two extra scratch pictures a stroke may need are each a whole sheet. They are made when first needed and given up after 180 frames with nothing being drawn, about three seconds. The Watercolour brush's two cover pictures are kept longer, about half a minute, because a wash can be joined until it dries.
- Large sheets. From 2896 by 2896 up, where one picture is 128 MB or more, a layer that is off screen gives up its undo patches and its second copy at once. Otherwise a few frames of an animation held gigabytes nothing was using. The cost is that an undo on such a layer, after it comes back, redraws from history instead.
Moves between the places
- Opening: device, then memory, then card. The page reads the saved files from the device and hands the engine the document, tiles and journal. The engine decodes each PNG tile and packs it into memory. A tile that is damaged, missing or the wrong size leaves its place blank and is reported at once, rather than refusing the whole drawing. Only layers that are needed get a picture on the card, and their tiles are copied up onto it. Operations in the journal that are newer than the saved tiles are then replayed.
- A layer leaves the screen: card to memory. When the card runs over its budget, the layer unused for longest is given up. Layers on screen, with operations still waiting, or being drawn this frame are never given up, and neither are the onion-skin and coming-up frames unless operations need the room. If the layer has not changed since it was last packed, its picture is simply freed. If it has, it is read back from the card first. On WebGPU the card answers frames later, so this is done one layer at a time in the background, and the packing is spread out, about four milliseconds a frame, because a large sheet is a thousand tiles and doing them at once froze the app. If the layer is drawn on meanwhile, the stale result is thrown away. Playing an animation never reads back.
- A layer comes back: memory to card. Its tiles are unpacked and laid onto a new picture in the coming frame, before the sheet is drawn, so it shows in the frame it was asked for.
- Saving: card to memory to device, changed tiles only. Every change is already with the page the moment it is made, in the journal, so a save only brings the stored pictures up to date. A save starts after two idle seconds; or after 30 changes once the pen has been still for 0.6 seconds; or after 400 changes whatever the pen is doing. It first waits for every operation to land on its layer. Then it reads back only the layers that changed, all in one frame so they are of one moment; a layer that is already packed needs no read at all. Only the tiles that changed are encoded as PNG, a few milliseconds a frame and none in a frame where the pen is down, and are sent in batches. The page writes them under a new revision, commits by replacing one small file in a single step, trims the journal of what the tiles now hold, and keeps the last three revisions. A save that dies halfway leaves the previous one whole.
- Undo: patches stay on the card. Undoing the last operation puts one patch back, with no read and nothing drawn. A patch is cut after the operation lands, in an idle frame, from a second copy of the layer that runs one operation behind, so drawing never waits for it. When no patch or copy can answer (an older operation, a patch dropped for room, a layer that left the screen and came back), the layer is redrawn from the nearest copy or from where it was last packed, replaying the operations after it. This is slower and gives the same picture.
Planned, not built
- Layers as tiles that exist only where there is paint, on the card too. Today a layer on screen is a whole picture the size of the sheet, even when it holds one line, and only memory and the device keep sparse tiles.