Per-coordinate ownership (Runtime.NativeDataParallelOwned: optimizer state
and, with p = 1, parameters) over a bounded-fan-out hierarchy
(Representation.ParticipantTree) instead of a star:
combine: the sum and the assembly run on the hierarchy's tree
(Runtime.NativeCombine), so a participant exchanges with at most
fanout + 1 peers per combine;
publication: Checkpoint.NativeTreePublication. Every owner writes the
shards of its own words (its coordinates' optimizer state and, when
partitioned, parameters); the root also writes the replicated words
(replicated parameters, shared state). Each commitment carries the
child's capture prefix and replicated words, which the parent compares
with its own, so diverged replicas publish no root, and no owner ever
holds another owner's words;
restore: every owner reads its manifest path, its own shards and, below
the root, only the root's replicated shards.
The checkpoint's identity is the star's for replicated parameters with the
hierarchy's combine order, and for partitioned parameters the learner's
placement-independent identity tagged with the hierarchy's manifest tree.
61field unrestricted dataParallelOwnedTreeRootShards : (family NativeRegionCheckpoint)The compiler supplied declaration spans and resolved links from this source snapshot. This page does not assert that this file belongs to a checked closure.