The token stream the host reads the records from: a token cache (the
tokenizer's output), a 16-byte header (the source's modification time
and size) then little-endian u32 token ids. Context c is ids
[1024 c, 1024 c + 1024]: its inputs the first 1024, its targets the last
1024 -- the same window one token on.
116def coppeliusTokenStreamHeaderBytes : Nat = 16The compiler supplied declaration spans and resolved links from this source snapshot. This page does not assert that this file belongs to a checked closure.