Source/Packages

Component.CubicTanhGateSigmoid

packages/components/src/Component/CubicTanhGateSigmoid.alpha

179 lines15 declarations6.9 KiBSHA-256 d73f403dc6e0

def · lines 122–145

propagateCubicTanhSigmoidGatePointwise

Full file
122def propagateCubicTanhSigmoidGatePointwise =
123  (lambda erased inputs : Nat .
124    unrestricted x : (family PointwiseExpression StdU32 inputs) .
125    unrestricted upstream : (family PointwiseExpression StdU32 inputs) .
126    let unrestricted multiply =
127      (pointwiseBinary StdU32 inputs PointwiseMultiply)
128      in
129      let unrestricted add =
130        (pointwiseBinary StdU32 inputs PointwiseAdd)
131        in
132        let unrestricted s =
133          (sigmoidPointwise inputs (cubicTanhSigmoidArgumentPointwise inputs x))
134          in
135          let unrestricted complement =
136            (add
137              (cubicTanhSigmoidConstant inputs sigmoidWeightedGateOne)
138              (pointwiseUnary StdU32 inputs PointwiseNegate s))
139            in
140            let unrestricted slope =
141              (add
142                (cubicTanhSigmoidConstant inputs cubicTanhSigmoidScale)
143                (nest multiply (cubicTanhSigmoidConstant inputs cubicTanhSigmoidSlopeCubic) x x))
144              in
145              multiply upstream (add s (multiply (nest multiply x s complement) slope)))

The compiler supplied declaration spans and resolved links from this source snapshot. This page does not assert that this file belongs to a checked closure.