In launch order: initialize; the default checkpoint saved and the
request's loaded (chunk by chunk); the half copies; the prediction; one
update's shared part; each update's AdamW (which writes the half
copies); the final forward; the checkpoint saved; the report put back;
the loss record.
coppeliusSubmissionSchedule (below) submits them.
2619def cgBeforeAdamW : (family NvidiaLaunchSchedule) =
2620 (cgThen
2621 cgInitialization
2622 (cgThen
2623 (cgCheckpointTransfer 0)
2624 (cgThen
2625 (cgCheckpointTransfer 1)
2626 (cgThen cgRecast (cgThen cgRotaryTables (cgThen cgPrediction cgUpdate))))))The compiler supplied declaration spans and resolved links from this source snapshot. This page does not assert that this file belongs to a checked closure.