Document NVFP4 native pack status

This commit is contained in:
Daniel Maddern 2026-08-15 03:00:06 +07:00
parent 8d90cec133
commit c2fb5c1e3f

View file

@ -81,6 +81,13 @@ Kernel candidates to test, in order:
3. CUTLASS/CuTe block-scaled NVFP4 GEMM prototype for owned hot-path kernels, exact layouts, fused epilogues, and persistent-kernel experiments.
4. cuBLASLt NVFP4/grouped GEMM where CUDA exposes a supported operation on the current Blackwell target.
Current NVFP4 activation quantization status:
- `H3_NVFP4_SCALE_BACKEND=vortex` is parity-safe for BF16 H3 activations and is the useful optimization seam. The tuned scale kernel uses 128 threads / 256 blocks and reaches about 88-90% of DGX Spark's advertised LPDDR5X bandwidth on FC1/FC2 scale discovery.
- Standalone scale discovery is effectively done for Spark; remaining theoretical headroom is too small to justify more absmax-only tuning.
- `vortex_native` BF16->NVFP4 pack is parity-safe for real H3 FC1/FC2 activation shapes, including qdata, cuBLAS tiled block-scale layout, signed zero, tie-to-even FP4 rounding, and downstream `scaled_mm_nvfp4` linear output.
- Keep Comfy Kitchen as the default production packer. Pack-only sweep `8d90cec` shows native 512-thread pack is essentially tied on FC1 (`1.090 ms` vs CK `1.098 ms`) but still slower on FC2 (`2.977 ms` vs CK `2.886 ms`). Use `vortex_native` only as an experimental benchmark path until the wide-FC2 pack kernel is redesigned.
Validation and profiling sequence:
1. Use `tools/profile_h3_block.py` to measure one representative H3 block before writing kernels. Record QKV, RoPE/RMS, attention kernel, output projection, MLP fc1, activation, MLP fc2, modulation/gating, and total block time.