propagateCubicTanhSigmoidGatePointwise with its sigmoid given: the tree
names s = sigmoid(v) three times, and a compiler evaluates each node of a
tree once, so the same sigmoid would be evaluated three times. Given as
an operand (an input already holding the sigmoid tree's value), every
node computes the same word from the same operands: the pullback's value
is unchanged.
153def propagateCubicTanhSigmoidGateWithSigmoidPointwise =
154 (lambda erased inputs : Nat .
155 unrestricted x : (family PointwiseExpression StdU32 inputs) .
156 unrestricted upstream : (family PointwiseExpression StdU32 inputs) .
157 unrestricted s : (family PointwiseExpression StdU32 inputs) .
158 let unrestricted multiply =
159 (pointwiseBinary StdU32 inputs PointwiseMultiply)
160 in
161 let unrestricted add =
162 (pointwiseBinary StdU32 inputs PointwiseAdd)
163 in
164 let unrestricted complement =
165 (add
166 (cubicTanhSigmoidConstant inputs sigmoidWeightedGateOne)
167 (pointwiseUnary StdU32 inputs PointwiseNegate s))
168 in
169 let unrestricted slope =
170 (add
171 (cubicTanhSigmoidConstant inputs cubicTanhSigmoidScale)
172 (nest multiply (cubicTanhSigmoidConstant inputs cubicTanhSigmoidSlopeCubic) x x))
173 in
174 multiply upstream (add s (multiply (nest multiply x s complement) slope)))The compiler supplied declaration spans and resolved links from this source snapshot. This page does not assert that this file belongs to a checked closure.