h3-blackwell-runtime/research/fc2_nvfp4_scheduling/RESULTS.md

FC2 NVFP4 Scheduling Results

Status: production accepted for the canonical descriptor shape behind H3_NVFP4_FC2_LT_SPLITK1; all other shapes retain the existing fallback.

Decision

The canonical GB10 FC2 slowdown is caused by the cuBLASLt heuristic selecting a _stream_k kernel with poor traversal locality and heavy synchronization polling. A documented public split-K-1 configuration is byte-exact and removes the regression. Because this library candidate passed the complete validation ladder, the custom persistent-kernel branch is closed.

The retained candidate is:

Field	Value
Algorithm ID	70
Tile ID	20
Stages ID	37
Split-K	1
Reduction scheme	0
Custom option / CTA swizzle	0 / 0
Required workspace	0 bytes
Supplied workspace	64 MiB sweep; 0 bytes block/trajectory gates

The baseline heuristic's negative split-K value is an undocumented library sentinel. CUDA 13 has no documented public attribute that directly selects Stream-K, so the sentinel itself is not interpreted as a public Stream-K control. The launched baseline symbol does end in _stream_k.

Canonical Operation

• Logical GEMM: M=37,810, N=5,376, K=14,336.
• Descriptor GEMM after accepted producer padding: M=37,824.
• Inputs: packed NVFP4 E2M1 activation and weight with vector-16 UE4M3 scales.
• Accumulation/scalars: FP32; output: BF16.
• Accepted activation producer: vortex_native_quantize_swiglu_nvfp4.
• Image: sha256:1d340e14cb6fc45ccfdbe63dde8db2a2b3aea94b493702c8a08e1f8d5b4f7b83.

Timing

The 20-round isolated sweep alternated AB/BA order in one loaded model.

Measurement	Baseline	Split-K-1	Result
FC2 p50	53.618 ms	15.636 ms	3.43x
Dense throughput p50	108.70 TFLOP/s	372.74 TFLOP/s	3.43x
Producer + FC2 p50	76.737 ms	38.823 ms	49.41% faster

The negative-sentinel candidate reproduced the production-equivalent baseline within noise. Explicit split-K factors 2, 4, 8, and 16 were rejected by cublasLtMatmulAlgoCheck for the tested configuration space.

Complete 20-round block gates replaced only block.mlp.fc2.forward_swiglu:

Block	Baseline p50	Candidate p50	Improvement
0	457.891 ms	420.536 ms	8.16%
24	459.928 ms	420.243 ms	8.63%
49	458.054 ms	417.364 ms	8.88%

All block outputs were byte-exact against both the paired baseline and retained traversal.

Trajectory	Baseline	Candidate	Improvement	Correctness
Two-step	45.962 s	42.516 s	7.50%	video/audio BF16 exact
Canonical 12-step	278.201 s	255.371 s	8.21%	video/audio BF16 exact

Trajectory values are single paired runs in baseline-then-candidate order, not repeated medians or AB/BA trials. Their approximately 38 ms per-FC2 savings agree with the alternating isolated and complete-block gates, but the precise end-to-end percentages retain run-order uncertainty.

The 12-step saving of 22.829 s matches approximately 600 FC2 invocations multiplied by the isolated roughly 38 ms saving.

Production Integration

The packaged path lives in src/h3_blackwell_runtime/fc2_lt.py and is called from Nvfp4Linear.forward_swiglu only after the accepted native producer. It requires the FC2 role, unsharded K=14,336/N=5,376, BF16 output, no bias or active LoRA, GB10/SM121, CUDA 13.0, cuBLAS 13.1 headers, cuBLASLt 13.0 runtime, logical M=37,810, and packed M=37,824. A shape-specific cublasLtMatmulAlgoCheck must pass with zero workspace. Load, check, or execution failure falls back through the already produced NVFP4 bytes without quantizing twice; strict mode is available through H3_NVFP4_FC2_LT_STRICT=1. Production startup prepares the extension and caches its measured runtime compatibility result before inference.

The 20-round production-method gate exercised Nvfp4Linear.forward_swiglu directly and recorded 23 successful dispatches with zero fallbacks per block:

Block	Baseline p50	Production p50	Improvement
0	464.153 ms	420.714 ms	9.36%
24	460.184 ms	422.656 ms	8.16%
49	456.183 ms	420.335 ms	7.86%

Production-method trajectories remained exact:

Trajectory	Baseline	Production	Improvement	Dispatch
Two-step	47.301 s	43.728 s	7.55%	100/100, zero fallback
Canonical 12-step	286.431 s	262.979 s	8.19%	600/600, zero fallback

The row-shape gate prevents unsafe generalization. Algorithm 70 was exact for the tested M=37,774, 37,809, 37,810, 37,838, 37,966, and 38,222 inputs, but differed in two BF16 elements at each of M=37,711, 37,725, and 37,742 despite passing AlgoCheck. Production therefore intentionally keeps the canonical-only guard rather than inferring exactness from nearby shapes. The retained shape artifact records compile-time CUDA/cuBLAS versions but predates runtime-version fields in build_info(). The final extension separately measured CUDA runtime 13000 and cuBLASLt runtime 130000; the shape artifact alone is not runtime-ABI evidence.

Traffic Attribution

The one-pass distinct-data footprint is:

Category	Bytes
Activation QDATA + scales	305,012,736
Weight QDATA + scales	43,352,064
Padded BF16 output	406,683,648
Total	755,048,448

Both schedules issue the same nominal operand requests:

Requested category	Bytes	Geometric reuse
Weight QDATA + scales	12,832,210,944	296x
Activation QDATA + scales	12,832,210,944	about 42.1x
Total L2 operand reads	25,664,424,960	unchanged

The schedule does not eliminate CTA-level rereading. It changes whether those rereads remain cache-resident:

NCU metric	Baseline _stream_k	Split-K-1
Main-kernel duration	55.057 ms	17.031 ms
Combined L2 hit rate	53.32%	91.10%
L2 read-miss bytes	11.762 GB	1.909 GB
L2 miss-byte proxy	12.169 GB	2.316 GB
Sysmem traffic proxy	12.176 GB	2.327 GB
Off-chip amplification	16.12x	3.07x
SM throughput	25.11%	81.79%
Tensor-pipe active share	25.01%	82.42%
Eligible warps/scheduler	0.073	0.220
Issue rate	0.057	0.168

GB10 does not expose the usual discrete-GPU DRAM byte counters in these captures. L2 misses and the reported sysmem fill/write sectors are used as the off-chip proxy.

The 9.853 GB reduction in L2 read misses is avoided operand rereading. NCU aggregates the two TMA input descriptors, so it cannot defensibly assign exact miss-byte totals separately to weights and activations. The request geometry does prove that each input family accounts for 12.832 GB of requested reads.

There is no material global partial-accumulator or output-reduction traffic:

• Candidate workspace is zero.
• Neither kernel issues global atomics or global reduction operations.
• No auxiliary reduction kernel is launched.
• Candidate writes exactly one padded BF16 output; baseline writes only 73,728 bytes more.
• Baseline-only local stack traffic is about 22.35 MB at L1 and almost none reaches sysmem.

The baseline's dominant scheduler stall is sleeping. Source-correlated samples land in NANOSLEEP.SYNCS polling around synchronization phase checks. Launch resources are otherwise the same: 12,432 CTAs, 384 threads/CTA, 168 registers per thread, 89,088 bytes allocated shared memory, and 25% theoretical occupancy. The gain therefore comes from traversal locality and reduced synchronization waiting, not occupancy or a reduction workspace.

Evidence

• benchmarks/gb10-fc2-nvfp4-library-sweep-20260825.json
• benchmarks/gb10-fc2-nvfp4-block-gate-20260825.json
• benchmarks/gb10-fc2-nvfp4-trajectory-2step-20260825.json
• benchmarks/gb10-fc2-nvfp4-trajectory-12step-20260825.json
• benchmarks/gb10-fc2-nvfp4-baseline-profile-20260825.json
• benchmarks/gb10-fc2-nvfp4-splitk1-profile-20260825.json
• benchmarks/gb10-fc2-nvfp4-baseline-20260825.csv
• benchmarks/gb10-fc2-nvfp4-splitk1-20260825.csv
• benchmarks/gb10-fc2-nvfp4-baseline-20260825.ncu-rep
• benchmarks/gb10-fc2-nvfp4-splitk1-20260825.ncu-rep
• benchmarks/gb10-fc2-nvfp4-production-block-gate-20260825.json
• benchmarks/gb10-fc2-nvfp4-production-trajectory-2step-20260825.json
• benchmarks/gb10-fc2-nvfp4-production-trajectory-12step-20260825.json
• benchmarks/gb10-fc2-nvfp4-shape-gate-20260825.json
• src/h3_blackwell_runtime/csrc/fc2_nvfp4_lt.cpp

The Spark hot runtime remains stopped while performance work continues. Resident-service validation is intentionally deferred until the performance program is finished.