Source/Packages

Model.LearnedPositionDecoder

packages/components/src/Model/LearnedPositionDecoder.alpha

233 lines48 declarations12.8 KiBSHA-256 58c267358560

def · lines 77–88

learnedPositionDecoderAdmitted

Full file
the heads tile the width; padding never removes a real class
77def learnedPositionDecoderAdmitted =
78  (lambda unrestricted geometry : LearnedPositionDecoderGeometry .
79    eliminate LearnedPositionDecoderGeometry
80      (lambda unrestricted current : LearnedPositionDecoderGeometry . Nat) geometry
81      (branch LearnedPositionDecoderGeometryValue vocab tokens context layers width heads head feed .
82        nest naturalAnd (naturalNonzero tokens)
83          (naturalLessOrEqual tokens vocab)
84            (naturalNonzero context)
85              (naturalNonzero layers)
86                (naturalNonzero heads)
87                  (naturalEqual width (naturalMultiply heads head))
88                    (naturalNonzero feed)))

The compiler supplied declaration spans and resolved links from this source snapshot. This page does not assert that this file belongs to a checked closure.