diff --git a/PLAN.md b/PLAN.md index c1c340c..35a9ec8 100644 --- a/PLAN.md +++ b/PLAN.md @@ -81,6 +81,13 @@ Kernel candidates to test, in order: 3. CUTLASS/CuTe block-scaled NVFP4 GEMM prototype for owned hot-path kernels, exact layouts, fused epilogues, and persistent-kernel experiments. 4. cuBLASLt NVFP4/grouped GEMM where CUDA exposes a supported operation on the current Blackwell target. +Current NVFP4 activation quantization status: + +- `H3_NVFP4_SCALE_BACKEND=vortex` is parity-safe for BF16 H3 activations and is the useful optimization seam. The tuned scale kernel uses 128 threads / 256 blocks and reaches about 88-90% of DGX Spark's advertised LPDDR5X bandwidth on FC1/FC2 scale discovery. +- Standalone scale discovery is effectively done for Spark; remaining theoretical headroom is too small to justify more absmax-only tuning. +- `vortex_native` BF16->NVFP4 pack is parity-safe for real H3 FC1/FC2 activation shapes, including qdata, cuBLAS tiled block-scale layout, signed zero, tie-to-even FP4 rounding, and downstream `scaled_mm_nvfp4` linear output. +- Keep Comfy Kitchen as the default production packer. Pack-only sweep `8d90cec` shows native 512-thread pack is essentially tied on FC1 (`1.090 ms` vs CK `1.098 ms`) but still slower on FC2 (`2.977 ms` vs CK `2.886 ms`). Use `vortex_native` only as an experimental benchmark path until the wide-FC2 pack kernel is redesigned. + Validation and profiling sequence: 1. Use `tools/profile_h3_block.py` to measure one representative H3 block before writing kernels. Record QKV, RoPE/RMS, attention kernel, output projection, MLP fc1, activation, MLP fc2, modulation/gating, and total block time.