Training
Models training on real hardware.
Each dashboard reads the run's own records: loss at every step, predictions, where each step's device time goes, optimizer internals, and GPU and host telemetry. The executables come from the Alpha compiler alone, with no CUDA runtime in the path.
AlphaOS on two RTX 3070s
two rented RTX 3070s (RunPod) · sm_86 · one launch per GPU
An operating system compiled by Alpha, resident on each GPU: dataflow tasks, programs loaded while it runs, and Midget training on one card and on two, with every host command recorded.
Open run record →Finishednanochat d24 speedrun on H100s
rented H100 SXMs (RunPod) · sm_90 · ClimbMix, FP8
Karpathy's record recipe (1.38B parameters, GPT-2 grade), every step trained by an Alpha-compiled trainer across the pod's cards. 5,568 steps on 2 H100s.
Open live dashboard →StoppedGPT-2 small on the RTX 3070
a rented RTX 3070 (RunPod) · sm_86 · OpenWebText, then FineWeb
The newest snapshot is 49671 minutes old.
Open dashboard →Milestones
Recorded results
Each links to the run record it summarizes.
- GPT-2 trained as AlphaOS tasks
A GPT-2-small-shaped model trained from scratch as resident AlphaOS tasks rather than kernel launches, with bit-exact arithmetic: on an RTX 3070, then to 30,000 updates on an RTX 3090 (held-out loss 4.77). The knots research runs on it.
- AlphaOS runs on two RTX 3070s
A GPU-resident operating system compiled by Alpha boots once per card, schedules dataflow tasks, installs programs while it runs and trains Midget on one card and on two. The first runs found a predicate-latency bug and a reserved-register hang, both now fixed.
- nanochat d24 trained end to end on two H100s
The record's full schedule, 5,568 steps in 6 h 32 min at 247,865 tokens a second, every step trained by an Alpha-compiled trainer across two NVLinked H100s. CORE 0.2089 against the record's 0.2626: the schedule, data and kernels are reproduced, the result not yet.
- GPT-2 small on Alpha's own Hopper kernels
GPT-2 124M on one H100 through the sm_90 back end: a median 11.01 ms an update over four alternating rounds, against 11.37 ms for llm.c with cuDNN attention, on the same model, data, start and recipe.
- GPT-2 small scored in bits per byte
Every retained checkpoint of Alpha's GPT-2 small scored on the same 18.5 MB of held-out OpenWebText on the RTX 3070: 1.396 bits per byte after 9B tokens at a constant learning rate, 1.223 after llm.c's recipe on FineWeb.
- GPT-2 small trains on the RTX 3070
9.03 billion tokens of OpenWebText through Alpha's sm_86 trainer at 36,000 tokens a second, then llm.c's 124M recipe on FineWeb, from the same weights.