AlphaGitHub
Source/Packages

Realization.Nvidia.SM86.AttentionHeadLayoutSM86

packages/realizations/cooperative/nvidia-sm86/src/Realization/Nvidia/SM86/AttentionHeadLayoutSM86.alpha

1,710 lines162 declarations85.3 KiBSHA-256 ad8cef82929b
Download exact fileView at revisionf5034773ad4bb52133

def · lines 1644–1649

attentionHeadLayoutSM86InverseTransposeValueAdmitted

Full file
Pull back the transposed V operand into its token-major projection slice. The incoming dV^T is f32 [head][component][position]; no rotation applies to V. The two positions owned by one thread have disjoint output words.
1644def attentionHeadLayoutSM86InverseTransposeValueAdmitted =
1645  (lambda unrestricted seq : Nat . (lambda unrestricted heads : Nat .
1646    (lambda unrestricted projectionWidth : Nat . (lambda unrestricted planeOffset : Nat .
1647      (naturalAnd
1648        (attentionHeadLayoutSM86TransposeValueAdmitted seq heads projectionWidth planeOffset)
1649        (naturalLessOrEqual (naturalMultiply heads (naturalMultiply seq 256)) 4294967295))))))

The compiler supplied declaration spans and resolved links from this source snapshot. This page does not assert that this file belongs to a checked closure.