AlphaGitHub
Source/Packages

Realization.Nvidia.SM86.StreamingAttentionSM86

packages/realizations/cooperative/nvidia-sm86/src/Realization/Nvidia/SM86/StreamingAttentionSM86.alpha

3,230 lines493 declarations193.3 KiBSHA-256 cc0366b3d857
Download exact fileView at revisionbcc11a4e7647d31628

def · lines 3137–3143

streamingAttentionKeyHeadsFirstProjectionSM86

Full file
The same storing dK into a fused projection's gradient (skProjectionEpilogue).
3137def streamingAttentionKeyHeadsFirstProjectionSM86 =
3138  (lambda unrestricted seq : Nat . (lambda unrestricted heads : Nat .
3139    (lambda unrestricted keyValueHeads : Nat . (lambda unrestricted width : Nat .
3140      (lambda erased admitted : (equal Nat (streamingAttentionKeyProjectionAdmitted seq heads keyValueHeads width) 1) .
3141        (sm86GuardLateReads
3142          (sm86TensorProductsFixed 12
3143            (streamingAttentionKeyGroupedOrderedWithSM86 (skProjectionEpilogue width) 1 seq heads keyValueHeads 0 1))))))))

The compiler supplied declaration spans and resolved links from this source snapshot. This page does not assert that this file belongs to a checked closure.