Per-coordinate ownership (Runtime.NativeDataParallelOwned: optimizer state
and, with p = 1, parameters) over a bounded-fan-out hierarchy
(Representation.ParticipantTree) instead of a star:
combine: the sum and the assembly run on the hierarchy's tree
(Runtime.NativeCombine), so a participant exchanges with at most
fanout + 1 peers per combine;
publication: Checkpoint.NativeTreePublication. Every owner writes the
shards of its own words (its coordinates' optimizer state and, when
partitioned, parameters); the root also writes the replicated words
(replicated parameters, shared state). Each commitment carries the
child's capture prefix and replicated words, which the parent compares
with its own, so diverged replicas publish no root, and no owner ever
holds another owner's words;
restore: every owner reads its manifest path, its own shards and, below
the root, only the root's replicated shards.
The checkpoint's identity is the star's for replicated parameters with the
hierarchy's combine order, and for partitioned parameters the learner's
placement-independent identity tagged with the hierarchy's manifest tree.
52family NativeDataParallelOwnedTreeLayout : Type 0
53constructor NativeDataParallelOwnedTreeLayoutOf
54-- template frames, planned for the edge of role 1; every edge's frames share
55-- this storage
56field unrestricted dataParallelOwnedTreeCommitment : (family RegionFrameLayout)
57field unrestricted dataParallelOwnedTreeRelease : (family RegionFrameLayout)
58field unrestricted dataParallelOwnedTreeCheckpoint : (family NativeTreeCheckpoint)
59-- below the root, the root's replicated shards as this owner restores them
60-- (none at the root, whose replicated words are its own shards)
61field unrestricted dataParallelOwnedTreeRootShards : (family NativeRegionCheckpoint)The compiler supplied declaration spans and resolved links from this source snapshot. This page does not assert that this file belongs to a checked closure.