Anatomy of a learning system#

A system is a model, a learner, a run policy and a pairing with a device. The first three are hardware-independent semantics; only the pairing knows which card it is for. Every byte of the final executable is then the compiler's job: a system declares typed regions, launches and schedules, and shared packages derive the device program and the host program from them.

This page takes Bob apart. Bob is the smallest GPU system in the repository: it predicts the next raw byte from the previous one with a learned 256 × 256 FP32 logits table and no hidden state. It is small enough to read in an afternoon, and it has the same structure as the large transformers.

The layout#

systems/bob/
  alpha.package               the package: name, namespace, source and test roots
  src/Bob/
    Model.alpha               what is computed
    Learner.alpha             how parameters change
    TrainingRun.alpha         the run policy: updates per invocation, inputs, checkpoint
    Prediction.alpha          greedy next-byte prediction
    System.alpha              the hardware-independent system
    ArenaPlan.alpha           where buffers sit in device memory
    Build/                    everything that knows about a device
      SM86.alpha              what the system requires of an SM86 card
      SM86Programs.alpha      which kernels it runs
      PhysicalComponents.alpha  regions, launches and submission order
      NativeHost.alpha        what the host program must do
      DirectArtifact.alpha    the whole-program artifact value
      RTX3090.alpha           the pairing with one card
      RTX3090Artifact.alpha   the build root for that pairing
  tests/
    alpha-tests               the package's declared suite
    qualification/Proof/      proof roots

The boundary is the Build/ directory. Nothing above it imports a hardware package; Bob.System says so in its own comment, and the package graph enforces it.

The package#

format alpha-package-v2
name bob-system
area systems
namespace Bob
src src
tests tests
test-suite tests/alpha-tests
maintained true

alpha.project at the repository root lists the package, so alpha test --package bob-system runs its suite. Workspace and packages explains the manifest.

Model: what is computed#

Bob.Model fixes the shapes (bobVocabularySize is 256, bobContextLength is 1) and builds the model from shared components: bobTableComponent is a linear component with no bias, and bobModel is the whole model. The objective is exact full-vocabulary cross-entropy. Nothing here names a numeric layout in device memory or a kernel.

Learner: how parameters change#

Bob.Learner selects the shared AdamW contract (LearnerContract LearnerAdamW) and states its hyperparameters: learning rate 0.05, β₁ 0.9, β₂ 0.999, ε 1e-8, weight decay 0. The per-step scalars the update kernel reads are computed in Alpha from those hyperparameters and the step number, not pasted in as constants.

Learners are built from named mechanisms in packages/learning, not from a monolithic optimizer. Learning.Learner.MomentNormalized exposes firstMomentValue, secondMomentValue, momentDecayPowerNext, correctMomentBias and normalizedStepValue, and composes them as momentNormalizedUpdate: the update usually called Adam, without weight decay. The CPU systems Manfred and Ambrosio call these mechanisms per parameter. Learning.Checked.Muon and Learning.Checked.AdamW are the checked meanings of their kernels. The convention is deliberate: an established name like Adam belongs in a comment, while the code names the operations, so a new learner can recombine them.

Run policy: what one invocation does#

Bob.TrainingRun is the contract of a run: ten updates per invocation (bobUpdatesPerInvocation), the input record layout, the checkpoint schema, and bobRunAllowedHostFallbacks = 0. The checkpoint is the shared Checkpoint.Envelope. It is checked before the card is opened and published through a temporary file, so an interrupted run never leaves a half-written checkpoint.

Build: meeting a device#

Everything device-specific is under Bob.Build:

Requirements. SM86.alpha states what Bob needs from an SM86 card. NativeHost.alpha states the host's compatibility demands. Neither names a product.
Kernels. SM86Programs.alpha takes shared kernel images (AdamW, embedding gather, greedy argmax) from packages/realizations, plus Bob's own bigram loss gradient.
The whole-program plan. PhysicalComponents.alpha declares four device regions, the launches with their parameters, and the submission order. The NVIDIA backend places, prunes and encodes the program and command tables from it.
The host. NativeHost.alpha declares the host's layout, recipes and request schedule. Platform.Linux.Nvidia.PlanHost derives the host program from that declaration.
The pairing. RTX3090.alpha checks Bob's demands against the RTX 3090 profile's supplies. RTX3090Artifact.alpha is the root you build.

Targets, devices and pairings explains admission and the derived host in detail.

Build it#

alpha build bob-rtx3090.elf --profile release --artifact-value artifact \
  --root systems/bob/src/Bob/Build/RTX3090Artifact.alpha

--artifact-value artifact evaluates the checked Bytes definition named artifact in the root and writes those bytes as the executable. Nothing is assembled outside the compiler: no generated packager, no intermediate component files. With the compiler of 2026-10-10 this writes an 800,804-byte static ELF in about 24 seconds, with a receipt under .alpha-cache/receipts/.

Other useful commands:

Task	Command
Check the package's suite	`alpha test --package bob-system`
Check one root's closure	`alpha check --root systems/bob/src/Bob/Build/RTX3090Artifact.alpha`
See what the ELF contains	`alpha inspect artifact bob-rtx3090.elf --root …`
Check the ELF structurally	`alpha verify elf bob-rtx3090.elf`

Run it#

Each executable implements a versioned train/predict request protocol. One request names the input batch, the checkpoint to read (an empty file means a fresh start), the checkpoint to write, and the result files. The system's README gives the exact commands, and scripts/alpha/pod-qualify.sh runs fresh training, a continuation and a separate prediction in order. On a machine without the card, the host stops at the first device call and exits with status 120. It never pretends to train.

The recorded RTX 3090 run (under tests/observability/import/bob-rtx3090/) reduces the loss from 5.545 to 4.552 to 3.351 across fresh training and continuation, and reproduces the same prediction receipts in a separate process.

Tests that come with it#

tests/alpha-tests lists the package's check cases. tests/qualification/Proof/ holds proof roots: BobSemanticContract, BobCompatibility (admission refuses a card without unified memory, and an sm_89 build on Ampere), BobCheckpointPrefix and BobAdamWScheduleContract, among others. Larger systems such as Coppelius and Baguette also keep a tests/negative/ corpus of programs that must be refused. Tests, gates and CI covers how all of these run.

From Bob to the large systems#

Coppelius (a 57.7M-parameter attention transformer), Baguette and GPT-2 have the same parts at a larger scale: more components in the model, more kernels in the plan, more launches in the schedule, the same AdamW contract, and one pairing module per device. The Learning systems page lists every system, what it learns, and where it has run.