A WebGL2 flicker fix (#1139) was render-parity-verified on exactly one scenario — the zoom-OUT hold-tiles case that reproduced the original flicker, retained-content 0.19 → 0.79 matching WebGPU — and never on zoom-in or per-frame cost. It shipped an unmemoized per-frame classifyTile run twice per tile plus synchronous uploads, and stuttered with progressively-black fills on a Seoul zoom-in. Reverted 36 minutes later (#1140). A hot-path fix must be verified in its worst regime, not the one that showed the original bug.
A draped landcover polygon rendered hundreds of kilometres offshore — but only on WebGPU. Every vector slice of one source shares one Material uniform pool, and each slice resets the pool cursor to 0, so a later slice's tile i overwrote an earlier slice's tile-i buffer. WebGPU's queue.writeBuffer defers to submit (last-writer-wins), so every draw read the final bytes; WebGL2's immediate bufferSubData never showed it — and the differential test was structurally blind, because both frames aliased identically.
Half the globe rendered as background again — same symptom as the drape-suppression bug, opposite root cause. This time the trans-antimeridian tiles were never selected: the overzoom fast-path unprojects the viewport corners to a lon/lat box, but on a small globe disc the corners miss the sphere, the box collapses to the sub-camera point, and it emits the single tile column under the camera — which a contiguous longitude range can never wrap across the dateline. Told apart from the draw bug by seam location and persistence.
A jq command over a saved MCP result printed nothing and exited 0 — no data, no error, no failure. Every one of those was manufactured by the caller: a wrong schema emptied stdout, a defensive 2>/dev/null ate the one diagnostic line, and | head laundered jq exit code 5 into 0. A Unix pipeline has three signal channels; habits acquired as defenses can close all three at once.
A refactor rerouted the WebGPU frame shell through RHI wrappers and claimed byte-identical rendering. We did not diff pixels against a tolerance — we hashed three captures and got cd3097…9b0f three times. A same-code re-run proved the noise floor was zero, upgrading the gate from diff-below-tolerance to bit equality. Determinism is a property you engineer into the harness; once you have it, verification collapses from statistics to md5sum.
A sequel scar. Piping a merge commit into head -3 closed the pipe after three lines and SIGPIPE-killed lint-staged mid-run — orphaning its backup stash onto the shared stack and leaving the merge half-done. The worktrees post predicted this pile collects scars; today added one through a mechanism nobody had listed. Treat a hook-running commit stdout as a transaction log, not a stream to sample.
An icon-padding feature grew three compiler files past their LOC ceiling. The PR raised the ratchet it knew about and went green — then CI failed on a SECOND ratchet tracking the same three files with the same numbers. A repo that preaches single-authority had let one invariant grow two authorities, and a local gate nobody knew existed is a gate nobody can run.
Labels decayed 6980 → 2138 → 76 → 0 pixels across four probe runs — in commit order, looking exactly like a regression gradient. Every HTTP fetch returned 200. The frame trace said all 46 labels placed. The real bug: an upload staging buffer invisible to the render-on-demand idle predicate, a deadlock that fossilized the map mid-load on any machine. One line made the buffer visible; labels went 0 → 7,081 pixels with zero interaction.
One debugging day, seven instrument failures: a probe that double-counted its own bookkeeping into '93 duplicate requests', a WebGPU canvas that reads back as all zeros, a parity test that passed by comparing two empty frames, a screenshot taken 7.5 seconds too early, and three more. Each with the tell that gave it away and the cross-check that caught it — because the same day also proved that one of those 'flakes' was hiding a real bug.
I spec'd a drift guard for the globe's camera-relative ECEF offset: force the ellipsoid constant to a sphere, assert the result shifts >10 km. The implementer measured it: 0.7 m. Turning the WHOLE computation spherical barely moves a relative offset, because both endpoints carry the same 21 km bias and a subtraction cancels it. The regression that actually bites is the mixed frame — ellipsoid tile, spherical camera — and only a guard that breaks the symmetry the same way sees it.
A building went missing from the 3D layer. Tile clipping had split its footprint into two pieces; the tiler flattened both outers into one rings array, and the extrusion consumer read everything after rings[0] as a hole — so piece #2 was earcut-punched out of piece #1's roof. Walls survived, roofs vanished, n pieces rendered n-1 roofs. The 2D fill was pixel-perfect the whole time, and zero tests had ever fed the consumer more than one polygon.
A static map view produced 10+ MB/s bursts and a warm phone. The render loop was innocent, our first probe fabricated a '93 duplicate requests' finding, and the real villain was the availability feature that keeps fast-pan from blanking — politely fetching an 85-tile, 33 MB world pyramid on every load and pinning it against eviction forever. Same view after the fix: 35.33 MB → 6.27 MB.
Half the globe rendered as pure background, cut exactly at the 0-meridian z1 tile boundary. The fallback ancestors for the missing half were computed, CPU-cached, and force-uploaded to the GPU — then drawn nowhere, because a drape suppression flag was scoped to the renderer instance instead of one dispatch. Verified by making the network slow on purpose, after the first pixel readback lied.
A row of country labels floated in the sky above the globe's horizon. Round 1 — an angular margin, provably safe at every zoom — died because floaters and healthy labels interleave in angular depth (Nigeria 0.332 floats, Kenya 0.329 doesn't). Round 2's screen-space limb fixed Chad and missed two-line Burkina Faso by exactly one line of text. The gate only held once it moved to where the quad height actually exists.
A displaced vector layer, a red/blue checkerboard ocean, a 16-gon planet, and intermittent flicker — four user-visible globe defects on WebGL2, and not one root cause in shared code. Every camera rendered correctly on WebGPU. On why a hand-maintained backend twin is scaffolding that must die, and the 20.7 km test witness that keeps its seam from drifting while it lives.
A headless session with no GPU had thirteen open issues and a hard rule against claiming a render is correct without a pixel diff. The split that worked sorted the backlog not by difficulty but by whether each fix's correctness reduces to something provable without a raster.
A concave country's label rendered offshore because the anchor was the bounding-box centre — a point the polygon does not contain. The fix is a guaranteed-interior point, but the harder call was shipping it from a session with no GPU to re-check the pixels.
Six agents committed in parallel from six worktrees of one repo — and every pre-commit hook stashed into the single stack they all share, finding its entry by list index. What worktrees actually share, plus the orphaned-stash scar that proves the race.
We broke instanced batching on purpose to watch a brand-new CI gate go red — 100,000 draw calls — before trusting its green. Why new gates need a demonstrated failure, and why this one asserts dc(10k) === dc(100k) instead of pinning an exact count.
Ticking 'include stress battery' on a conformance page froze an RTX 2080 desktop so hard the window's own close button stopped responding. The df64 stress shaders over-ran the 2 s Windows TDR, and a 4 s JS compile timeout couldn't help — you can't cancel a GPU op you already submitted.
A PR body cited a CI gate that no workflow references and a waiver clause absent from the issue it named — both written by an AI implementer, both refuted by one grep. Plus the real bug beside them, and the derived artifact a freshness gate caught.
Routing Earth constants out of GPU shader source through one planetary Body authority, under a hard bar: emitted bytes must not change. Goldens captured from pre-change code, a literal ratchet, a 211-hit census — and a "wrong" eccentricity kept on purpose.
An issue prescribed relocating a context type to kill an adapter-to-adapter dependency edge. By pickup, that mechanism meant reverting a merged refactor. We cut the edge with composition-root injection instead, and locked it with a shrink-only ratchet.
Adjacent map tiles projected the same vertex 40,075 km apart — one full world circumference. A rotated projection's unavoidable branch cut, resolved per tile, scattered the seam across every tile join; the fix relocates it to the one edge geometry expects.
A missing hemisphere, labels floating off the globe, and tile seams at z6.73 were filed as three engine bugs. All three were one unsynced camera field in the demo harness — and after the fix, review found the identical write one harness file over.
The same render test passed twice: once at 38.5% painted pixels on real WebGPU, once at 93.6% on a silent WebGL2 fallback the assertion never noticed. Why a designed-in fallback chain turns every green output test ambiguous — until the test also pins which backend produced the frame.
I edited one file in @xgis/map and ran that package's whole suite — 1246 green. Then the full monorepo run turned up two failures, both in @xgis/runtime, a package I never opened. Mid-extraction, the sufficient gate is the whole repo, not the package.
A fix for per-feature fill opacity came with a fail-before/pass-after test: 0.25 and 0.75 baked to alpha 64 and 191, two values where there had been one. It was green and I reverted the change — because the alpha it proved feeds the outline, not the fill.
A package extraction moved the render code to a new package. It changed no behavior, so every open bug report's symptom stayed valid — and every report's file:line pointed at a path that no longer existed. The precise-looking part of a report rots first.
A single-authority migration for on-screen scale left the globe branch returning the raw, uncapped value under a comment reading 'UNVERIFIED'. A sibling branch of identical shape was a non-bug — and only a probe of which value is read told them apart.
A bug report for one-way arrows drifting off the road came with a diagnosis and a one-line fix. The diagnosis was refuted by reading the anchor code, and the proposed fix was a revert of a change the repo had already tried and backed out. The real fix was neither.
Making the line outline's projection precise, but not its polygon fill, turned a shared invisible jitter into a ~3px fill≠outline seam at deep zoom. The fix isn't more f64 — it's the same camera-relative reframe, which the linear term needs no emulation for.
Two engines rendered the same style differently because one honoured a metadata field the other deliberately skipped. The comment defending the skip described a danger an existing clamp had already neutralized — and the "working" half of the cull turned out to be silently keyed past its own lookup.
Line strokes shook at deep zoom in non-Mercator projections: the shader rebuilt an absolute longitude in f32 degrees, then a later stage subtracted the camera again — catastrophic cancellation. The fix subtracts first; the proof needs no GPU.
A coordinate-space migration carried one assumption — "absolute geometry needs no per-copy shift" — that was true for the globe and false for flat maps. Applied per-renderer, it silently dropped world-copy fan-out path by path. The fix is one ten-line router.
Two overlapping map labels swapped survivors on pan because the collision pass's tie-break was tile-dispatch order — reversed. The fix is a stable identity fed to the greedy allocator, and the gate is a permutation test.
We adjudicated eight open bug reports in one batch. Two were already fixed on main and one's premise was refuted by a write-only uniform lane. In every stale case the deliverable that survived was a regression test, not a patch.
Fast-math deleted our emulated-double multiply on Apple GPUs, and every float-side guard — including hardware FMA — died with it. So we rebuilt twoSum and twoProd in u32 bit arithmetic: same pair format, same compositions, exact by construction. The device that zeroed every float variant returned the golden answer.
A hand audit of a 16-workspace import graph concluded the backend adapters were clean. It was wrong twice — once because `[a-z-]` doesn't match a `2`, once because `from '…'` can't see a dynamic import. The 166-line CI gate built FROM that audit found the missed edge on its first run.
The GPU has no double. There are four ways to get one anyway — double-float error-free transforms, integer emulation, reference-point rendering, and perturbation — and they are not interchangeable. A decision guide, from a survey of Thall, QD/CAMPARY, Cesium, deck.gl, and fractal deep-zoom engines.
I shipped four in-shader fixes for the df64 multiply collapsing to zero on an Apple GPU. Every one failed on-device. Then the research settled it: Metal's default fast-math deletes the error term by construction, no opaque-guard trick reliably survives, and the durable answer is to restructure the math so the high-precision multiply never happens — which is why deck.gl deprecated emulated fp64.
The emulated-double add and multiply are the same algorithm family, but on an Apple GPU add read 0.5 and multiply read exactly 0. The bug was not in either — it was a renormalization step the textbook multiply omits and the battle-tested one keeps.
The same df64 value—1e8+0.5—minus 1e8. Computed on the GPU it reads 0.5; loaded from a uniform it reads 0.0, with the fast-math guard bound in both. To a reassociating compiler a loaded low word and a computed one are not the same number, and the fix is to launder one into the other.
GPU double-single splits floats with 8193; the textbook says 4097. I was sure 8193 was cargo-culted from a wider type — until two million float32 products came back error-free for both. The split constant that breaks is one too small; the real bug was never the split.
A shader drew correctly on one GPU and returned the clear colour on another — not precision, but layout:auto deleting a binding it declared but never read. When an interface is derived from usage, an unused declaration is a mismatch only the strict backend reveals.
A one-file docs PR sat on "Expected — waiting for status to be reported" with no checks coming. Two GitHub mechanisms hid required checks: a paths filter that skipped the whole workflow, and a matrix job skipped at the job level — evaluated before the matrix expands, so its per-leg contexts never post. Why "skipped" is not "never ran."
After a texel-fetched guard fixed the fast-math collapse, isolated df64 ops passed on the reporter's phone but composed ones still broke. There are two failure classes, not one — and the second lives in the ALU, where no value you thread into the shader can reach it. df64 is per-vendor, and our CI's SwiftShader is structurally blind to half of it.
217 GPURenderPipeline references stood between @xgis/map and backend-neutrality. Most of them turned out not to be pipelines at all — they were labels being matched. Collapsing them to a neutral { label } handle, and how prove-or-refute kept us from doing the wrong migration.
Moving map's render passes behind the RHI one at a time meant two encoders — native and RHI — had to share a single frame and a single submit. Why createCommandEncoder was the wrong tool, and the wrapper that let a native encoder be driven through the RHI interface for one pass at a time.
@xgis/map named backend types in ~700 places. You cannot cut that in one commit and stay honest. The method: sort every reference into layers by how it's actually used, find the one clean move, and prove each slice compiles to identical bytes before the next.
A refactor step I left uncommitted on one machine had already been authored and pushed to the shared branch by an autonomous agent, under a different SHA. Why 'is it done?' on an agent-shared branch is a distributed-consensus question — and why, before redoing anything, you fetch and read the agent's commits first.
Emitted WGSL/GLSL ships to gl.shaderSource with its authored vocabulary intact — no JS minifier reaches it. A Vite/Webpack-style emit-plugin pass (mangle + minify) that compacts and obfuscates the shader text, the ABI boundary it must never cross, and why compile-and-link is not enough to trust it.
A post-mortem on a refactor that went the wrong way. To strip @webgpu/types from @xgis/map we re-exported the WebGPU types through a barrel — and made the coupling worse, not better. The revert, and the rule that would have caught it up front.
Relocating a module is editing the dependency graph, and text search sees only some of its edges. It misses dynamic imports and re-exports, and a regex-dialect gap can make a guard silently lie. Grep narrows; the compiler decides.
A recursive no-op Proxy let a WebGPU-typed engine boot on WebGL2 by making every native GPU call return a harmless dummy. It also silently swallowed every place a fence was missing. Replacing it with a fail-loud stub — and what the stub screamed about.
The emit path of @xgis/shader-dsl grew a Vite/Webpack-style plugin seam so ship-time transforms compose without weighing on the core — and a build-only subpath so runtime-emit consumers bundle zero bytes of them. The test: dropping in a third transform (call-graph inlining) that touched no core line.
The verification stack behind shader-dsl's fp64: a correctly-rounding f32 machine built from Math.fround, a metamorphic oracle gate, byte goldens, and discriminative real-GPU known answers.
Incident report: the fp64 demo alternated between correct and f32-collapsed frames under byte-identical inputs. Inferred cause: driver pipeline re-optimization specializing on uniform values. Fixes: render-on-demand and a texel-fetched guard.
f64 and vecN<f64> as first-class shader-dsl types, emulated as two-f32 double-float pairs — same authoring syntax as f32, one lowering pass, ~48 significand bits.
We rebuilt ten classic ShaderToy effects in X-GIS's typed shader DSL. The renders were the easy part: authoring them surfaced eleven concrete DX issues — an elliptical sun no gate could see, a reversed smoothstep every GPU hides — each closed with a gate so it stays fixed.
Our CI runs real shaders on SwiftShader — but only some render claims survive a CPU rasterizer. The dividing line we use (compute parity and shader compilation: yes; full-pipeline pixel parity: no), and the red-baseline incident that taught us to blame the software GPU last.
Three rounds of CI failures where every test was green and the job was red — vitest's worker→main reporter RPC timing out under file-count pressure. The ~110-file floor we split on (set below the lowest count we watched fail), the matrix design that contains it, and the playbook for the next regrowth.
How a declarative map style becomes GPU work: lexer → parser → a Scene IR → optimization passes → codegen that emits typed shader-dsl IR (never WGSL strings), per-feature compute kernels for match(), and a gradient atlas for zoom-dependent paint.
The program index for X-GIS's WebGL2 port: package-enforced backend neutrality, a single shader IR emitting both WGSL and GLSL, and pixel gates instead of promises — with links to the per-subsystem deep-dives. Shared shader math still can't stop pipeline state and target conventions from diverging.
Three real bugs from making WebGL2 match WebGPU on a 117-layer basemap: a dedup map nobody cleared, a depth jitter that out-voted painter's order, and a vertically mirrored composite hidden by a symmetric test fixture.
Our CI container's browser can't reach the internet, but the final render gate needs a real 117-layer basemap. On mirroring styles, tiles, glyphs and sprites locally — and the traps: SPA fallback serving HTML as tiles, unencoded fontstacks, and settling on signals instead of sleeps.
X-GIS compiles a declarative style language into optimized WebGPU shaders — through a real compiler and a typed shader IR. Here's the shape of the system and why each layer exists.
