Overview

The Grid is the RAYS Foundation's federated fine-tuning fabric. It lets thousands of consumer machines contribute training signal to a single global model without the collapse modes that break naive federated learning.

The three problems it solves

• Catastrophic interference. Two nodes learning unrelated behaviours pull the base weights in opposite directions. Naive FedAvg ((W_A + W_B) / 2) destroys both updates.
• Behavioural forgetting. Fine-tuning on JSON tool-calling text erodes the model's conversational reasoning and general knowledge.
• Consumer VRAM limits. Adam optimiser state alone blows past 8 GB. Local training on a laptop GPU is usually impossible for 7B+ models.

The three ideas that fix them

• Execution-state graph loss. We tune on the DAG of tool calls the agent actually ran, not on next-token cross-entropy. Grammar stays intact.
• Server-side SVD allocation. The server decomposes W = U Σ Vᵀ and hands each client a strictly orthogonal subspace. Updates can only live inside that subspace.
• Non-destructive summation. Because the subspaces are orthogonal by construction, the server sums deltas instead of averaging them: W_new = W_base + ΔW_A + ΔW_B + …

Where to go next

• FOGR math — the full derivation of the loss, SVD routing, and TIES fallback.
• RAYS Studio — the desktop daemon that runs the local half of the pipeline.
• AMD pipeline — the wavefront-aligned path for ROCm hardware.