Source/Packages

Component.CubicTanhGateSigmoid

packages/components/src/Component/CubicTanhGateSigmoid.alpha

179 lines15 declarations6.9 KiBSHA-256 d73f403dc6e0

def · lines 153–174

propagateCubicTanhSigmoidGateWithSigmoidPointwise

Full file
propagateCubicTanhSigmoidGatePointwise with its sigmoid given: the tree names s = sigmoid(v) three times, and a compiler evaluates each node of a tree once, so the same sigmoid would be evaluated three times. Given as an operand (an input already holding the sigmoid tree's value), every node computes the same word from the same operands: the pullback's value is unchanged.
153def propagateCubicTanhSigmoidGateWithSigmoidPointwise =
154  (lambda erased inputs : Nat .
155    unrestricted x : (family PointwiseExpression StdU32 inputs) .
156    unrestricted upstream : (family PointwiseExpression StdU32 inputs) .
157    unrestricted s : (family PointwiseExpression StdU32 inputs) .
158    let unrestricted multiply =
159      (pointwiseBinary StdU32 inputs PointwiseMultiply)
160      in
161      let unrestricted add =
162        (pointwiseBinary StdU32 inputs PointwiseAdd)
163        in
164        let unrestricted complement =
165          (add
166            (cubicTanhSigmoidConstant inputs sigmoidWeightedGateOne)
167            (pointwiseUnary StdU32 inputs PointwiseNegate s))
168          in
169          let unrestricted slope =
170            (add
171              (cubicTanhSigmoidConstant inputs cubicTanhSigmoidScale)
172              (nest multiply (cubicTanhSigmoidConstant inputs cubicTanhSigmoidSlopeCubic) x x))
173            in
174            multiply upstream (add s (multiply (nest multiply x s complement) slope)))

The compiler supplied declaration spans and resolved links from this source snapshot. This page does not assert that this file belongs to a checked closure.