What the kernels leave in the staging region for the host to return (the
R6 result layout, R9 appended the host timestamp table): the loss block
at 0x180000 (16 KiB, four 4 KiB windows: the last step's per-row
cross-entropy -- the target logit gathered into `correct` makes the
cross-entropy rows the loss; before that they held the log-sum-exp --
and its embedding gradient, then the embedding parameters and their AdamW
first moment after the final step), the three next-token predictions at
0x1F0000 (three u64), and the result record at 0x200000 -- its first
147,456 bytes are written after the forward submission, before training
(the marker below is put at its start first), the next 49,152 after the
run. The result file is, in the order the host writes it: the first
147,456 bytes of the record (the forward on the run's first context,
before any step); every step's per-row cross-entropy (1024 binary32
words, one 4 KiB window per step, appended as the step completes: the
update's loss copy leaves it at 0x190000); then, after the run, the
checkpoint input's stat (160 bytes, with gpGet, gpPut and the first
semaphore word at 96, 100 and 104), the loss block, 49,152 bytes of the
record (row 1023's logits after the final step), the semaphore region
(32 bytes per submission of the plan: the push segment's final release,
the fence the host waits for, and its pre-launch release, each with the
device's timestamp -- a submission issued again leaves its last issue's),
64 bytes of the error notifier, the host's 160-byte status block, the
timestamp table (64 bytes per submission the run issues, in issue order).
A step's loss is the forward on a context no earlier step has trained
on: the loss on unseen text.
149def coppeliusResultWindowBytes : Nat = 0x1000The compiler supplied declaration spans and resolved links from this source snapshot. This page does not assert that this file belongs to a checked closure.