The logistic function's pullback from its saved output s: (dy (1 - s)) s,
PyTorch's sigmoid_backward order (a * (1 - b) * b). Using the output
rather than recomputing exp keeps the forward's approximation and the
backward's in agreement.
42def propagateSigmoidOutputWith =
43 (lambda unrestricted operations : Float32Operations .
44 unrestricted sigmoid : Nat .
45 unrestricted upstream : Nat .
46 f32Multiply
47 operations
48 (f32Multiply
49 operations
50 upstream
51 (f32Add operations sigmoidWeightedGateOne (f32Negate operations sigmoid)))
52 sigmoid)The compiler supplied declaration spans and resolved links from this source snapshot. This page does not assert that this file belongs to a checked closure.