Document NVFP4 native pack status
This commit is contained in:
parent
8d90cec133
commit
c2fb5c1e3f
1 changed files with 7 additions and 0 deletions
7
PLAN.md
7
PLAN.md
|
|
@ -81,6 +81,13 @@ Kernel candidates to test, in order:
|
|||
3. CUTLASS/CuTe block-scaled NVFP4 GEMM prototype for owned hot-path kernels, exact layouts, fused epilogues, and persistent-kernel experiments.
|
||||
4. cuBLASLt NVFP4/grouped GEMM where CUDA exposes a supported operation on the current Blackwell target.
|
||||
|
||||
Current NVFP4 activation quantization status:
|
||||
|
||||
- `H3_NVFP4_SCALE_BACKEND=vortex` is parity-safe for BF16 H3 activations and is the useful optimization seam. The tuned scale kernel uses 128 threads / 256 blocks and reaches about 88-90% of DGX Spark's advertised LPDDR5X bandwidth on FC1/FC2 scale discovery.
|
||||
- Standalone scale discovery is effectively done for Spark; remaining theoretical headroom is too small to justify more absmax-only tuning.
|
||||
- `vortex_native` BF16->NVFP4 pack is parity-safe for real H3 FC1/FC2 activation shapes, including qdata, cuBLAS tiled block-scale layout, signed zero, tie-to-even FP4 rounding, and downstream `scaled_mm_nvfp4` linear output.
|
||||
- Keep Comfy Kitchen as the default production packer. Pack-only sweep `8d90cec` shows native 512-thread pack is essentially tied on FC1 (`1.090 ms` vs CK `1.098 ms`) but still slower on FC2 (`2.977 ms` vs CK `2.886 ms`). Use `vortex_native` only as an experimental benchmark path until the wide-FC2 pack kernel is redesigned.
|
||||
|
||||
Validation and profiling sequence:
|
||||
|
||||
1. Use `tools/profile_h3_block.py` to measure one representative H3 block before writing kernels. Record QKV, RoPE/RMS, attention kernel, output projection, MLP fc1, activation, MLP fc2, modulation/gating, and total block time.
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue