AlphaGitHub
Source/Packages

Realization.Nvidia.SM86.StreamingAttentionSM86

packages/realizations/cooperative/nvidia-sm86/src/Realization/Nvidia/SM86/StreamingAttentionSM86.alpha

3,230 lines493 declarations193.3 KiBSHA-256 cc0366b3d857
Download exact fileView at revisionbcc11a4e7647d31628

def · lines 1877–1883

streamingAttentionQueryProjectionAdmitted

Full file
a power-of-two sequence, the query heads within the projection's width, its binary32 byte offsets within 32 bits
1877def streamingAttentionQueryProjectionAdmitted =
1878  (lambda unrestricted seq : Nat . (lambda unrestricted heads : Nat .
1879    (lambda unrestricted keyValueHeads : Nat . (lambda unrestricted width : Nat .
1880      (naturalAnd (streamingAttentionGroupedSM86Admitted seq heads keyValueHeads)
1881      (naturalAnd (naturalEqual (naturalPowerOfTwo (sqrLog2 seq)) seq)
1882      (naturalAnd (naturalLessOrEqual (naturalMultiply heads saHeadWidth) width)
1883        (naturalLess (naturalMultiply seq (naturalMultiply width 4)) (naturalPowerOfTwo 32)))))))))

The compiler supplied declaration spans and resolved links from this source snapshot. This page does not assert that this file belongs to a checked closure.