daniel/h3-blackwell-runtime
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
h3-blackwell-runtime/research/vortex_exact_attention/PHASE2B_ALIGNED_REPORT.md
Daniel Maddern 67e05f517c Validate aligned Vortex attention prototype
2026-08-26 15:51:39 +07:00

4.7 KiB
Raw Blame History

Phase 2B Aligned Numerical Prototype

Decision

VEA-B passes the isolated fixed-shape Phase 2B gate on NVIDIA GB10. The selected kernel computes B=1, H=1, Q=128, KV=192, D=128 from prequantized Q/K/V with three increasing K64 epochs. It is retained as research only. Ragged tails, preparation, complete H3 shapes, block integration, trajectories, and production dispatch remain out of scope.

Exactness

The four-warp checkpoint oracle reproduces the installed public SageAttention 2.2.0 output. The ten-warp VEA-B kernel matches the oracle byte-for-byte for:

  • final BF16 output, SHA-256 75d4169c9ae0692b24a463a64b538f852acba5a5567404728c8af60c16fb8e1b;
  • INT32-to-FP32 QK scores;
  • per-epoch online-softmax m and d;
  • FP8 E4M3 probability words and FP32 rescale values;
  • FP16 PV partial words and FP32 RO after every epoch;
  • final reciprocal values.

The selected source also completes 1,000 consecutive fast launches with the same output hash and no deadlock.

Selected Kernel

  • 320 threads: two K/V producer warps, four QK/softmax owners, and four PV/output owners.
  • Two ready, two free, and two K-ready CUDA block-scope mbarriers.
  • Two K/V and probability slots with strict epoch reuse.
  • Final reciprocal publication is folded into the final score-ready handoff.
  • Probability and V fragments are processed as ordered 8-column MMA pairs.
  • A per-lane volatile shared retirement slot aliases scale metadata only after a warp phase boundary. It constrains ptxas scheduling without changing arithmetic or allocating additional shared memory.

Ptxas and runtime attributes report 168 registers/thread, zero stack frame, zero spill stores/loads, zero local bytes/thread, 51.2 KiB dynamic shared memory, and one resident CTA/SM. NCU independently reports zero local-memory spilling requests, 168 registers/thread, one register/shared-memory-limited block per SM, and ten theoretical active warps (20.83%).

Synchronization And Safety

  • Compute Sanitizer memcheck: 0 errors.
  • Compute Sanitizer racecheck: 0 hazards, 0 errors, 0 warnings.
  • QK epoch 1 overlaps PV epoch 0 by 49,317 clocks.
  • QK epoch 2 overlaps PV epoch 1 by 46,408 clocks.

The retirement alias initially produced racecheck WAR warnings because lanes could overwrite scale metadata before sibling lanes completed their reads. The selected variant adds an explicit warp phase boundary before the first write.

Timing

The final 2,000-sample single-launch run measures VEA-B p50 0.016288 ms and public Sage2 p50 0.019840 ms, a prototype-only 1.218x ratio. Single-launch CV is 5.95%, reflecting a bimodal microsecond-scale distribution.

The authoritative stability run uses 500 event samples with 20 identical launches per sample and reports per-launch values. VEA-B p50 is 0.014426 ms, CV 0.87%; public Sage2 p50 is 0.014871 ms, CV 1.23%; the ratio is 1.031x. This is fixed-shape screening evidence, not a complete-attention speedup claim.

Rejected Variants

Variant Result
Initial combined ownership Exact, 352 local bytes/thread
Sequential PV fragments Exact, material spills remained
Union role state Exact, 208 local bytes/thread
Role-separated epoch loops Exact, 40 local bytes/thread
setmaxnreg redistribution Compiled at 200 registers, rejected at launch for excessive resources
Shared persistent RO fragment Final BF16 exact, FP32 RO diverged at epoch 1 and stack usage increased
Inner-pair warp fence Exact, but increased the fast stack frame
Selected shared retirement alias Exact, zero local memory; race-clean after warp phase boundary

Evidence

  • /home/daniel/StoryStudioAssets/H3-output/h3-blackwell-runtime/research/vortex_exact_attention/phase2b/probes/authoritative-batched-final.json
  • /home/daniel/StoryStudioAssets/H3-output/h3-blackwell-runtime/research/vortex_exact_attention/phase2b/probes/authoritative-final.json
  • /home/daniel/StoryStudioAssets/H3-output/h3-blackwell-runtime/research/vortex_exact_attention/phase2b/probes/memcheck-final.log
  • /home/daniel/StoryStudioAssets/H3-output/h3-blackwell-runtime/research/vortex_exact_attention/phase2b/probes/racecheck-final.log
  • /home/daniel/StoryStudioAssets/H3-output/h3-blackwell-runtime/research/vortex_exact_attention/phase2b/vea-b-final.ncu-rep
  • /home/daniel/StoryStudioAssets/H3-output/h3-blackwell-runtime/research/vortex_exact_attention/phase2b/fixtures/aligned-q128-kv192-seed73021/manifest.json

Advancement Limit

Phase 2B authorizes no production integration. The next phase must add ragged tails and complete-shape work distribution, then pass canonical attention, blocks 0/24/49, trajectory, resident-service, sanitizer, and complete-block performance gates before any dispatch proposal.