Bob's learner computes them in binary32 (Bob.Learner.bobAdamStepScalars);
the host continues the step count from the checkpoint's header.
666def bobHostAdamW : (family NvidiaPlanHostAdamW) =
667 (constructor
668 NvidiaPlanHostAdamW
669 NvidiaPlanHostAdamWValue
670 (constructor NvidiaPlanHostArithmetic NvidiaPlanHostBinary32)
671 (constructor
672 NvidiaPlanHostConstant
673 NvidiaPlanHostBinary32Bits
674 (nvidiaPlanHostBinary32Of bobLearningRate))
675 (constructor
676 NvidiaPlanHostConstant
677 NvidiaPlanHostBinary32Bits
678 (nvidiaPlanHostBinary32Of bobBeta1))
679 (constructor
680 NvidiaPlanHostConstant
681 NvidiaPlanHostBinary32Bits
682 (nvidiaPlanHostBinary32Of bobBeta2))
683 (constructor
684 NvidiaPlanHostConstant
685 NvidiaPlanHostBinary32Bits
686 (nvidiaPlanHostBinary32Of bobEpsilon)))The compiler supplied declaration spans and resolved links from this source snapshot. This page does not assert that this file belongs to a checked closure.