All training runs / AlphaOS / 2 × RTX 3070
AlphaOS On the GPU
One launch per GPU · 46 nodes, one per SM · programs loaded while it runs · the host as its internet
AlphaOS is an operating system that Alpha compiles into one resident sm_86 program. The host launches it once per GPU. From then on the GPU's 46 nodes claim tasks from queues and dataflow graphs, install programs the host sends, and run them. The nodes talk to the host only through protocol words in shared memory. No CUDA, no kernels, no relaunch.
- GPUs
- 2 × RTX 3070 GA104 · sm_86 · PCI 0000:46:00.0, 0000:81:00.0
- Nodes per GPU
- 46 online 46 · booted 46 · halted 46
- Midget on one card
- 5.58 → 3.41 nats per byte over 8 steps
- Held-out loss
- 5.65 → 2.99 on text the learner never trained on
What the host said to the GPU
Each card host recorded its own conversation with AlphaOS. Every write is a protocol word or a program's code going into memory the GPU reads. Every wait is the host holding until the GPU's nodes answered. Choose a run, then replay it.
exec: programs loaded at run time, CALL.REL/RET.REL · 17 host events over 84.3 ms · exit 0
- 0.000 mshosthost process starts
- 29.583 mshostGPU channel ready (RM client, device, memory)
- 32.645 mshostAlphaOS image and tables written to the GPU
- 35.677 mshostAlphaOS launched (the one launch of this GPU's life)
- 38.587 mswaitwaited for GPU 0 booted = 46 · the GPU answered
- 41.499 mswritewrote GPU 0 code area +0x0 · 48 bytes
- 44.756 mswritewrote GPU 0 program table +0x0 · 32 bytes
- 48.499 mswritewrote GPU 0 code area +0x400 · 176 bytes
- 51.407 mswritewrote GPU 0 program table +0x20 · 32 bytes
- 54.280 mswritewrote GPU 0 code area +0x800 · 208 bytes
- 57.154 mswritewrote GPU 0 program table +0x40 · 32 bytes
- 60.102 mswritewrote GPU 0 task ring +0x0 · 1,920 bytes
- 63.044 mswritewrote GPU 0 ring tail = 120 · 8 bytes
- 73.809 mswaitwaited for GPU 0 done = 120 · the GPU answered
- 77.118 mswritewrote GPU 0 command = 1 · 8 bytes
- 80.921 mshostevery node halted; the completion fence released
- 84.292 mshostmailbox read back to stdout
Every run on the cards
Each run is one Alpha-compiled executable, started on the pod: one launch, one conversation, one read-back of what the GPU left in its mailbox.
| Run | What it exercises | Exit | Seconds | Host events |
|---|---|---|---|---|
| AlphaOSBoot | boot/1: 46 nodes online, booted, halted | 0 | <1 | 8 |
| AlphaOSRunQueue | run queue: CAS claim, BRX dispatch | 0 | <1 | 11 |
| AlphaOSExec | exec: programs loaded at run time, CALL.REL/RET.REL | 0 | <1 | 17 |
| AlphaOSCat | gpu-file and gpu-term: a file read and printed | 0 | <1 | 63 |
| AlphaOSGraph | graph/1: 240 dataflow tasks | 0 | <1 | 17 |
| AlphaOSProcesses | proc/1 and event/1: exit, kill, events | 0 | 1 | 25 |
| AlphaOSLoader | loader/1: one image installed, nine refused by name | 0 | <1 | 24 |
| AlphaOSSignal | signal/1: a program supplies a task's input | 0 | <1 | 21 |
| AlphaOSObject | object/1: versioned objects, no torn snapshot | 0 | <1 | 16 |
| MidgetLearns | Midget (tiny) learns inside AlphaOS | 0 | 12 | 27 |
| MidgetDeepLearns | Midget.Deep learns; held-out loss | 0 | 86 | 28 |
| AlphaOSNet | net/1: a cross-GPU pipe by remote name (two cards) | 0 | 1 | 145 |
| MidgetPeers | two RTX 3070s train one Midget (relay/1) | 0 | 15 | 106 |
The loader test sent ten program images. The card installed one, refused nine by name, and ran the installed one: cube(3) = 27. The refusal codes it returned were 0, 1, 2, 5, 4, 3, 6, 7, 7, 8.
Midget learns on the GPU
Midget is a byte-level transformer written for AlphaOS. Its trainer is itself a program running on the GPU: it publishes every learner, evaluator and updater task, and the nodes claim them. Training text is Shakespeare's Sonnets and Thus Spake Zarathustra.
- Samples
- Generated on the card, greedily, from the same prompt. Before training:
nnnn····························································
After 8 steps:␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣␣
After 8 steps of 8 sequences of 16 bytes, Midget has learned the commonest byte, the space (␣); it needs far more steps before it writes words. A non-printing byte shows as ·. - Two cards agree
- Both GPUs ended with the same parameters and wrote the same sample.
Where the 46 nodes ran
Each node records the SM it found itself on (SR_VIRTUALSMID). The card fills one SM of each TPC first: 0, 2, 4 … then 1, 3, 5 ….
- node 0SM 0
- node 1SM 2
- node 2SM 4
- node 3SM 6
- node 4SM 8
- node 5SM 10
- node 6SM 12
- node 7SM 14
- node 8SM 16
- node 9SM 18
- node 10SM 20
- node 11SM 22
- node 12SM 24
- node 13SM 26
- node 14SM 28
- node 15SM 30
- node 16SM 32
- node 17SM 34
- node 18SM 36
- node 19SM 38
- node 20SM 40
- node 21SM 42
- node 22SM 44
- node 23SM 1
- node 24SM 3
- node 25SM 5
- node 26SM 7
- node 27SM 9
- node 28SM 11
- node 29SM 13
- node 30SM 15
- node 31SM 17
- node 32SM 19
- node 33SM 21
- node 34SM 23
- node 35SM 25
- node 36SM 27
- node 37SM 29
- node 38SM 31
- node 39SM 33
- node 40SM 35
- node 41SM 37
- node 42SM 39
- node 43SM 41
- node 44SM 43
- node 45SM 45
What the cards taught us
The first runs found four things that the CPU emulator had accepted. Each was fixed in the compiler or the emulator, then confirmed on the card.
- Every lane stayed
- The first boot counted 1,472 nodes online instead of 46: all 32 threads of every node had run past their exit. The exit read its predicate 7 cycles after the comparison that wrote it; on the card a branch or exit needs 13. Alpha's assembler now waits 13, and the emulator reports the hazard.
- Two reserved registers
- The signal test hung at 100% with its producer stuck before its first store. The producer used R70:R71, the two registers the hardware reserves above a 72-register allocation. The image now allocates what its programs need, and the emulator faults on a reserved register.
- A new driver
- The pod ran driver 590, which Alpha had never run on. Every structure the host sends the driver has the same layout in NVIDIA's 590 and 580 sources, so the 590 branch is now admitted. Before that change, the host refused 590 by name.
- Cards by their real names
- The container exposed the GPUs as /dev/nvidia2 and /dev/nvidia5, not 0 and 1. Each AlphaOS build now names the cards it runs on.
- Record
- Run record ↗ · Where AlphaOS is going ↗