31 lines
1.2 KiB
Markdown
31 lines
1.2 KiB
Markdown
|
# NVFP4 SwiGLU Producer Fusion
|
||
|
|
|
||
|
|
## Implementation
|
||
|
|
|
||
|
|
The H3 MLP now recomputes its exact BF16 SwiGLU values inside the Vortex global
|
||
|
|
scale and NVFP4 pack passes instead of materializing the 37,810x14,336 BF16
|
||
|
|
activation. The FC2 Comfy Kitchen NVFP4 GEMM and its reduction order are
|
||
|
|
unchanged.
|
||
|
|
|
||
|
|
The fused producer preserves both BF16 boundaries:
|
||
|
|
|
||
|
|
1. `silu(gate)` rounded to BF16.
|
||
|
|
2. The BF16 SiLU result multiplied by BF16 `up` and rounded to BF16.
|
||
|
|
|
||
|
|
It then preserves the established Vortex tensor-scale compatibility cast,
|
||
|
|
E4M3 block scales, E2M1 encoding, approximate reciprocal, and Comfy physical
|
||
|
|
scale layout.
|
||
|
|
|
||
|
|
## Validation
|
||
|
|
|
||
|
|
- Randomized BF16 tensor scale, QDATA, and SFA are byte-exact.
|
||
|
|
- Real block-24 FC1 output is byte-exact; producer median is 21.93 ms.
|
||
|
|
- Blocks 0, 24, and 49 are bit-exact and improve by 2.14-2.26%.
|
||
|
|
- Warmed two-step video/audio tensors are bit-exact.
|
||
|
|
- Warmed canonical 12-step tensors are bit-exact and improve from 289.14 s to
|
||
|
|
277.36 s, or 4.07%.
|
||
|
|
|
||
|
|
Spark single-GPU deployments enable the path with
|
||
|
|
`H3_NVFP4_SWIGLU_FUSION=1`. Active FC2 LoRA, full-precision, pre-scale,
|
||
|
|
autograd, and unsupported dtype/layout cases use the materialized fallback.
|