In launch order: initialize; the default checkpoint saved and the
request's loaded (chunk by chunk); the half copies; the prediction; one
update's shared part; each update's AdamW (which writes the half
copies); the final forward; the checkpoint saved; the report put back;
the loss record.
coppeliusSubmissionSchedule (below) submits them.
2620def cgBeforeAdamW : (family NvidiaLaunchSchedule) =
2621 (cgThen
2622 cgInitialization
2623 (cgThen
2624 (cgCheckpointTransfer 0)
2625 (cgThen
2626 (cgCheckpointTransfer 1)
2627 (cgThen cgRecast (cgThen cgRotaryTables (cgThen cgPrediction cgUpdate))))))The compiler supplied declaration spans and resolved links from this source snapshot. This page does not assert that this file belongs to a checked closure.