Source/Packages

Component.SigmoidWeightedGate

packages/components/src/Component/SigmoidWeightedGate.alpha

181 lines14 declarations6.6 KiBSHA-256 7302f612c9f7

def · lines 42–52

propagateSigmoidOutputWith

Full file
The logistic function's pullback from its saved output s: (dy (1 - s)) s, PyTorch's sigmoid_backward order (a * (1 - b) * b). Using the output rather than recomputing exp keeps the forward's approximation and the backward's in agreement.
42def propagateSigmoidOutputWith =
43  (lambda unrestricted operations : Float32Operations .
44    unrestricted sigmoid : Nat .
45    unrestricted upstream : Nat .
46    f32Multiply
47      operations
48      (f32Multiply
49        operations
50        upstream
51        (f32Add operations sigmoidWeightedGateOne (f32Negate operations sigmoid)))
52      sigmoid)

The compiler supplied declaration spans and resolved links from this source snapshot. This page does not assert that this file belongs to a checked closure.