The .xgis compiler pipeline: from style text to GPU programs

A polygon category comes out the right colour in one tile and the wrong colour in the tile right next to it — a seam that moves as you pan, and it traces back to a single design decision inside the compiler that turns a declarative map style into GPU programs. The first post on this blog claimed that an X-GIS style is source code — compiled once, not interpreted per frame; this post is the anatomy of that compiler in @xgis/compiler.

The stages

The IR is a Scene: sources, an array of render nodes, symbols. Each render node carries its paint as typed value unions (ColorValue, SizeValue, …) — a constant, a data-driven match, a zoom interpolate — which is exactly the shape the optimizer and codegen dispatch on.

The optimizer

A small LLVM-flavored pass manager runs over the Scene. Most of its passes are the obvious folds — a zoom interpolate whose stops are all identical collapses to a constant, a match() whose arms all yield the same literal collapses to that literal, scalar constant folding runs wherever expressions are classified. The value is mundane: paint that can be static becomes static before any GPU decision, so the expensive machinery below only fires for paint that genuinely varies.

The one pass worth dwelling on is the one that has to iterate: dead-layer-elim and dead-source-elim run as a fixpoint group, not a single sweep, because the two eliminations feed each other. Consider a source countries consumed by exactly one layer, and that layer is dead — every arm folded to transparent, say, so nothing it draws is visible. Pass one removes the layer. Only now is countries an orphan: it has zero consumers. A one-shot pipeline would stop there and hand the runtime a source nothing reads — a tile fetch and upload for no pixels. Running the group to a fixpoint lets the next iteration see the freshly-orphaned source and drop it too. The ordering isn’t decorative; the second removal only becomes legal because the first one happened.

Rule one: codegen emits IR, not strings

Codegen’s output is typed shader-dsl IR — ModuleDecl fragments and expression Nodes — never WGSL text. The former raw-WGSL escape hatch was deliberately removed. Three things fall out:

• Type safety across the seam. A fill expression is a NodeLike<'vec4<f32>'>; producing the wrong type is a TypeScript error in the compiler, not a GPU compile error in a user’s browser.
• Backend neutrality. The same emitted IR lowers to WGSL for WebGPU and GLSL ES 3.00 for the WebGL2 fallback, and the DSL’s CPU oracle can execute it for parity tests.
• Structural deduplication. Compute kernels are fingerprinted on the serialized IR module — two layers whose match() compiles to the same kernel share one, by reference, across fill and stroke.

Where data-driven paint goes

Paint routing splits on what an expression depends on:

match(.CONTINENT) { … } — depends on feature data, invariant per feature. It compiles to a per-feature compute kernel (workgroup size 64) that evaluates the match once per feature and packs the resulting RGBA into a storage buffer with pack4x8unorm. The render shader then does a single buffer read per vertex — the per-frame cost of data-driven paint collapses to a lookup.

The same match AST also has a second emitter that inlines it into a fragment shader when compute isn’t available — two emitters, one canonical AST, which keeps the two paths provably equivalent (the CPU oracle executes the same IR both derive from).

The bug that categoryOrder exists to kill

match(.class) doesn’t compare strings on the GPU — the shader compares integer IDs. So somewhere a string like "hospital" becomes an i32 the shader’s switch has a case for, and two pieces of code have to pick the same integer: the codegen that emits the case table, and the runtime packer that writes each feature’s ID into the storage buffer. If they ever disagree, the shader compiles fine and draws the wrong colour.

The tempting way to assign IDs is from the data — number the category values you actually see, in order. It’s also the trap, because vector tiles arrive per tile. A tile containing {cemetery, hospital, school} numbers hospital 1; the neighbouring tile, which happens to hold only {cemetery, hospital}, numbers the same hospital… also 1 here, but in a tile with {cemetery, hospital, railway, school} it might land on 2. The shader’s case table is fixed, so the same category comes out the right colour in one tile and the wrong colour in the tile next to it — a seam of miscoloured polygons that moves as you pan.

The fix is a single authority: IDs come not from the tile’s data but from the style’s arm patterns, sorted alphabetically once — ['cemetery', 'hospital', 'railway', 'school'] → 0,1,2,3 — and that list (categoryOrder) is carried on the shader variant. Codegen builds its case i table from it; the runtime packer maps each feature string to its index in the exact same list (a miss packs -1, so the shader’s else fires the default colour). Neither side ever looks at what a particular tile happens to contain, so tiles can’t drift. The comparison chain and the packer agree byte-for-byte because they read one sorted map.

interpolate over zoom — continuous in a frame-varying input. Zoom changes essentially every frame while the user navigates, so treating a zoom ramp like feature paint — re-dispatching a compute kernel to re-evaluate the ramp whenever zoom moves — pays a per-gradient dispatch every frame, and there’s no discrete “zoom granularity” you can precompute at because zoom is continuous. So the ramp is baked once, at compile time, into a gradient atlas: an N×1 texture row per ramp, sampled with hardware linear filtering at the current zoom. The entire runtime cost of zoom-dependent paint collapses to one textureSampleLevel — the hardware sampler does the interpolation the rejected kernel would have recomputed frame after frame.

The handoff

The compiler’s artifact is a SceneCommands list — load/show commands, each carrying its shader variant (a module preamble plus the fill/stroke expression nodes), the palette, and the compute plan. The runtime composes these fragments with its own geometry pipelines and only then does shader-dsl emit WGSL. No shader text exists until the last step, on the consumer’s side of the seam.

Pinned by tests

The pipeline is snapshot- and parity-gated: IR snapshots per fixture, WGSL snapshots for the compute kernels, per-pass statistics tests, and end-to-end conversion tests against real MapLibre/Mapbox styles (including full basemaps) that assert the compiled scene, not just “it didn’t throw”. When the optimizer grew GPU-execution parity gates, the compiler’s output was already IR those gates could run.

What generalizes

The lesson that outlives this compiler is to classify paint by what it depends on — constant, per-feature, or per-frame — rather than by what it is, because each dependency class has a different cheapest home (fold it away, precompute it once in a kernel, or bake it into a texture), and reaching for one general mechanism to serve all three is exactly what makes data-driven paint cost more than it needs to.