Systems

Learning systems and where they run.

A system is a model, a learner, a run policy and a pairing with a device, compiled whole into one executable. Each state below links to the record behind it. As of build 5f12a3db.

Runs on hardware

A recorded run on the device: train, resume from a checkpoint, predict.

Verified, run pending

The executable builds and verifies; a fresh run on the device is still to come.

In development

Maintained source that does not yet run end to end on a device.

Systems

The active systems

Each is a program under systems/. Model code is hardware-independent; the pairing chooses the device. Sixteen earlier systems were moved out of the live tree on 8 Nov 2026 and are no longer built or checked; the archive lists them.

Trains on hardware

GPT-2

GPT-2 small (124M): learned positions, LayerNorm, GELU, tied embeddings, trained from fresh weights. The same system states nanochat's d24 (1.38B parameters: value embeddings, sliding windows, ReLU², softcap, Muon and AdamW, FP8) as GPT2.Speedrun.

Runs on
x86-64 host → sm_86 (RTX 3070) and sm_90 (H100 SXM)
Status
GPT-2 small trained on 9.03B tokens of OpenWebText on the RTX 3070, then on llm.c's FineWeb recipe; on one H100 an update takes a median 11.01 ms against llm.c's 11.37. The d24 speedrun trained the record's full schedule on two H100s (5,568 steps); its CORE of 0.2089 is still short of the record's 0.2626.
Namespaces
GPT2 · GPT2.Build · GPT2.Speedrun · GPT2.SpeedrunBuild
Runs on hardware

AlphaOS

A GPU-resident operating system Alpha compiles into one sm_86 program, launched once per GPU: its nodes claim tasks from run queues and dataflow graphs, programs are installed while it runs, and the host is its network.

Runs on
x86-64 host → sm_86 (RTX 3070, RTX 3090)
Status
The whole system runs in Alpha's sm_86 CPU emulator. On 23 Oct 2026 it ran on two RTX 3070s with Midget learning; GPT-2 on AlphaOS has since trained on an RTX 3090. Running every qualification root on the cards is still open (#136).
Namespaces
AlphaOS
Trains on hardware

GPT-2 on AlphaOS

GPT-2 small trained as one resident AlphaOS task graph instead of kernel launches, aiming at GPT-2 quality on one consumer GPU by moving data rather than launching kernels.

Runs on
x86-64 host → AlphaOS on sm_86 (RTX 3070, RTX 3090)
Status
Trained from scratch with bit-exact arithmetic on an RTX 3070 and, to 30,000 updates (held-out loss 4.77), on an RTX 3090 in October 2026; the knots research runs on it. GPT-2 small's quality (FineWeb validation loss 3.28) is the goal, not yet reached.
Namespaces
GPT2AlphaOS
Trains on hardware

Midget

A byte-level transformer written for AlphaOS that learns as its tasks: its learner, updater, sampler and evaluator are loaded programs, and the trainer publishes each step's tasks on the device.

Runs on
x86-64 host → AlphaOS on sm_86 (RTX 3070)
Status
Trained inside AlphaOS on one RTX 3070 and on two (23 Oct 2026), with every host command recorded.
Namespaces
Midget

Targets

Hosts, GPU architectures and cards

Emitting code for an architecture and running a model on a particular card are different claims, so they have separate rows. Hardware has a page for each card.

TargetLayerStateWhat it doesEvidence
x86-64 LinuxELF64 / x86-64 / LinuxHost outputEmits and runs

The default host target: static executables with no libc, including the host side of every GPU training executable.

A host target says nothing about a GPU. GPU execution also needs a device image the card admits.
Sources
AArch64 LinuxELF64 / AArch64 / LinuxHost outputEmits and runs

The host of the DGX Spark: direct lowering to static AArch64 executables.

Direct lowering only; the evaluator-carrying artifact and native object output remain x86-64 only. The executables that ran here belong to systems archived on 2026-11-08 (archive/README.md); no active system has a recorded run on this host yet.
Sources
sm_86 (Ampere)NVIDIA Ampere / compute capability 8.6Accelerator architectureTrains models

The primary GPU architecture: instruction encoding, scoreboard checking, launch descriptors and command streams for the RTX 3070 and 3090. AlphaOS is one resident sm_86 program.

Built through each system's build roots rather than the generic --target flag.
Sources
sm_90 (Hopper)NVIDIA Hopper / compute capability 9.0Accelerator architectureTrains models

Hopper's instruction set as an Alpha architecture package: warpgroup matrix instructions (FP8 included), the tensor memory copy engine, transaction barriers and clusters. GPT-2's sm_86 kernels are lowered to it, and the d24 trainer's products and attention are written for it.

There is no public sm_90 assembler: every encoding is derived, round-trip proved and checked against nvdisasm as an oracle.
Sources
sm_121 (Blackwell)NVIDIA Blackwell / compute capability 12.1Accelerator architectureTrains models

A compiler back end lowers the sm_86 kernels to sm_121 and emits the Blackwell launch descriptors, so the same model source runs on the GB10.

Kernels are written for sm_86 and lowered; nothing is yet tuned specifically for sm_121. The training runs that qualified it belong to systems archived on 2026-11-08 (archive/README.md); no active system has a recorded run on sm_121 yet.
Sources
sm_89 envelope (Ada)NVIDIA Ada / compute capability 8.9Accelerator architectureVerified · run pending

Runs the sm_86 instructions on Ada cards under sm_89 launch descriptors and driver contracts.

The executables that passed compiler and independent envelope verification belong to systems archived on 2026-11-08 (archive/README.md). No active system targets sm_89 yet; a fresh physical run is pending.
Sources
GeForce RTX 3070GA104 / sm_86 / 8 GiBPhysical deviceQualified

GPT-2 small trained here (OpenWebText, then llm.c's FineWeb recipe). AlphaOS ran here with Midget learning on one card and on two, and GPT-2 on AlphaOS trained here from scratch.

The smallest-memory qualified card.
Sources
GeForce RTX 3090GA102 / sm_86 / 24 GiBPhysical deviceQualified

GPT-2 on AlphaOS trained here to 30,000 updates, the model the knots research runs on.

AlphaOS's resumable nodes, which let a run outlive the host's quarter-hourly context switch, were qualified on this card.
Sources
GeForce RTX 4090AD102 / sm_89 / 24 GiBPhysical deviceVerified · run pending

The Ada card: sm_86 instructions run under sm_89 launch descriptors and driver contracts.

No active system has run here. The executables verified for it belong to systems archived on 2026-11-08 (archive/README.md), and a fresh train, resume and predict run is pending: the rented cards available so far failed to open /dev/nvidia-uvm.
Sources
H100 SXMGH100 / sm_90 / 80 GB HBM3Physical deviceQualified

GPT-2 small at a median 11.01 ms an update on one card. nanochat d24 trained through the record's full schedule on two NVLinked cards, whose ranks talk through peer memory Alpha maps itself.

The d24 run reproduces the record's schedule, data and kernels, not yet its result: CORE 0.2089 against the record's 0.2626.
Sources
DGX Spark (GB10)20-core AArch64 / sm_121 / 128 GB unified memoryPhysical deviceQualified

A Blackwell GPU sharing 128 GB of memory with a 20-core Arm CPU, reached through the compiler's sm_121 back end.

The GPU shares the host's memory, so bandwidth and capacity are shared with the CPU. Its qualifying training runs belong to systems archived on 2026-11-08 (archive/README.md); no active system has a recorded run here yet.
Sources

Compatibility

Which system runs where

Derived from the rows above: the host each system emits, the GPU architectures it targets and the cards with a record for it.