AlphaGitHub
Source/Packages

Realization.Nvidia.SM86.StreamingAttentionSM86

packages/realizations/cooperative/nvidia-sm86/src/Realization/Nvidia/SM86/StreamingAttentionSM86.alpha

3,230 lines493 declarations193.3 KiBSHA-256 cc0366b3d857
Download exact fileView at revisionbcc11a4e7647d31628

def · lines 2607–2613

streamingAttentionKeyProjectionAdmitted

Full file
a power-of-two sequence, the heads within the width, its binary32 byte offsets within 32 bits
2607def streamingAttentionKeyProjectionAdmitted =
2608  (lambda unrestricted seq : Nat . (lambda unrestricted heads : Nat .
2609    (lambda unrestricted keyValueHeads : Nat . (lambda unrestricted width : Nat .
2610      (naturalAnd (streamingAttentionGroupedSM86Admitted seq heads keyValueHeads)
2611      (naturalAnd (naturalEqual (naturalPowerOfTwo (sqrLog2 seq)) seq)
2612      (naturalAnd (naturalLessOrEqual (naturalMultiply heads saHeadWidth) width)
2613        (naturalLess (naturalMultiply seq (naturalMultiply width 4)) (naturalPowerOfTwo 32)))))))))

The compiler supplied declaration spans and resolved links from this source snapshot. This page does not assert that this file belongs to a checked closure.