One model.
Thousands of GPUs.
Zero collapse.
The Grid is our federated fine-tuning fabric. Every client trains a mathematically orthogonal subspace of the same base model — so updates sum instead of averaging away.
See it running
The Grid, live.
How it works
Four ideas. One fabric.
Orthogonal SVD subspaces
The server decomposes W = UΣVᵀ and hands each client a strictly orthogonal slice. ⟨ΔW_A, ΔW_B⟩ = 0 by construction.
Zero-gated adapters (SB-ZGA)
Adapters start as a perfect identity (tanh(0) = 0). Cayley-parametrised weights stay orthogonal, so gradients never explode.
Execution-state graph loss
We don't tune on tokens. We tune on the DAG of tool calls the agent actually ran — Wasserstein-aligned to successful paths.
Non-destructive summation
The server sums updates instead of averaging. W_new = W_base + ΔW_A + ΔW_B + … Millions of nodes contribute without clashing.
SB-ZGA adapters
Spectrally-bounded, zero-gated, Cayley-orthogonal. 4-bit QLoRA-friendly. Fits on an 8 GB consumer card.
TIES-merge fallback
When subspaces overlap, the server trims noise, elects consensus signs and averages only sign-aligned updates.
Wavefront-aligned by design.
The AMD pipeline targets only q_proj, v_proj and gate_proj, freezes every norm layer, and pins the LoRA rank to exactly r = 64 — the size of a native RDNA/CDNA wavefront.
The result: matrix cores stay saturated, register spills disappear, and consumer Radeons train real 7B models without OOMing.
AMD architecture docsGRID
Orthogonal · Federated · Non-destructive