h3-blackwell-runtime/research/fc2_nvfp4_scheduling/README.md

# FC2 NVFP4 Library Scheduling

This directory contains the first isolated library-scheduling study for H3 FC2.
The validated extension source moved to
`src/h3_blackwell_runtime/csrc/fc2_nvfp4_lt.cpp` when the guarded production
integration was added. The research harness and production loader share that
source.

The completed measurements and decision are in `RESULTS.md`. Algorithm 70 with
public split-K 1 is byte-exact and passes the block, two-step, and canonical
12-step gates. Canonical-only production integration is complete behind an
opt-in flag; resident-service deployment validation remains deferred.

## Exact Operation

The extension reproduces Comfy Kitchen 0.2.31's
`cublas_gemm_nvfp4.cu` descriptors for row-major packed activation `[M,K]`
times packed weight `[N,K]` to BF16 `[M,N]`:

- cuBLASLt sees column-major `weight.T @ activation`, so Lt `m=N`, `n=M`, and
  `k=K`; the output storage is the row-major `[M,N]` view.
- A and B are `CUDA_R_4F_E2M1`, with
  `CUBLASLT_MATMUL_MATRIX_SCALE_VEC16_UE4M3` block scales.
- Compute, scalar scale type, alpha, and beta are FP32. Alpha and zero beta are
  device pointers.
- Output is BF16, epilogue is default, and there is no bias.
- The activation producer is the deployed, accepted
  `vortex_native_quantize_swiglu_nvfp4`. Alpha is its FP32 tensor scale times
  FC2's FP32 weight tensor scale.

The canonical 37,810-row input pads to 37,824 rows in the accepted producer.
The report distinguishes logical `[M,N,K]` from descriptor/padded dimensions;
comparison slices back to the logical output exactly as Comfy Kitchen does.

## Current Baseline

The accepted production path fuses exact BF16 SwiGLU into NVFP4 production and
retains Comfy Kitchen's cuBLASLt FC2 GEMM. Existing validated metadata reports:

- Producer tensor scale, QDATA, and SFA are byte-identical at blocks 0, 24, and
  49.
- Canonical block FC2 input is logically `37,810 x 14,336` BF16 before packing.
- Fully fused block-24 profiling attributes 26.87% of kernel time to all four
  NVFP4 GEMMs; this is not claimed as an FC2-only percentage.
- Spark's accepted image is
  `sha256:1d340e14cb6fc45ccfdbe63dde8db2a2b3aea94b493702c8a08e1f8d5b4f7b83`
  and enables `H3_NVFP4_SCALE_BACKEND=vortex`,
  `H3_FUSED_ELEMENTWISE=1`, `H3_NVFP4_MODULATE_FUSION=1`, and
  `H3_NVFP4_SWIGLU_FUSION=1`.

Every run also records a fresh profiler-derived baseline kernel list. Kernel
names and times in that list are run metadata, not hard-coded claims.

## Commands

Run inside image `1d340e14cb6f...` from the Spark checkout, without starting the
hot service. Preserve its production environment switches, for example:

```bash
export H3_NVFP4_SCALE_BACKEND=vortex H3_NVFP4_SCALE_VERSION=1
export H3_FUSED_ELEMENTWISE=1 H3_NVFP4_MODULATE_FUSION=1
export H3_NVFP4_SWIGLU_FUSION=1 H3_SAGE_QKV_LAYOUT=strided_nhd
```

Compile only:

```bash
python tools/benchmark_fc2_nvfp4_algorithms.py --mode compile --verbose-build \
  --build-directory /tmp/fc2-nvfp4-build
```

Default characterization enumerates workspace budgets 0, 4, 8, 16, 32, and
64 MiB, checks a bounded set of public split-K configurations, and times one
valid probe with alternating AB/BA order:

```bash
python tools/benchmark_fc2_nvfp4_algorithms.py --mode characterize \
  --build-directory /tmp/fc2-nvfp4-build \
  --output /output/h3-blackwell-runtime/benchmarks/fc2-nvfp4-characterize.json
```

The same workload can come from an existing capture:

```bash
python tools/benchmark_fc2_nvfp4_algorithms.py --mode characterize \
  --capture /artifacts/capture --build-directory /tmp/fc2-nvfp4-build \
  --output /output/h3-blackwell-runtime/benchmarks/fc2-nvfp4-capture.json
```

A bounded timing sweep defaults to eight candidates, not a full combinatorial
gate:

```bash
python tools/benchmark_fc2_nvfp4_algorithms.py --mode sweep \
  --candidate-limit 8 --rounds 6 --workspace-bytes 33554432 \
  --build-directory /tmp/fc2-nvfp4-build \
  --output /output/h3-blackwell-runtime/benchmarks/fc2-nvfp4-sweep.json
```

Re-run one selected candidate. `--candidate` accepts an inline JSON object, a
JSON file containing one object, or `enumerate:<budget-index>:<result-index>`:

```bash
python tools/benchmark_fc2_nvfp4_algorithms.py --mode selected \
  --candidate '{"algorithm_id":23,"tile_id":42,"stages_id":35,"split_k":1,"reduction_scheme":0}' \
  --workspace-bytes 33554432 --build-directory /tmp/fc2-nvfp4-build \
  --output /output/h3-blackwell-runtime/benchmarks/fc2-nvfp4-selected.json
```

NCU capture uses `cudaProfilerStart/Stop`; only the selected FC2 cuBLASLt call
is inside the range:

```bash
ncu --target-processes all --profile-from-start off --set full \
  --export /output/h3-blackwell-runtime/benchmarks/fc2-nvfp4-selected \
  python tools/benchmark_fc2_nvfp4_algorithms.py --mode profile \
  --candidate /tmp/fc2-candidate.json --workspace-bytes 33554432 \
  --build-directory /tmp/fc2-nvfp4-build \
  --output /output/h3-blackwell-runtime/benchmarks/fc2-nvfp4-ncu.json
```

Complete block gate for blocks 0, 24, and 49:

```bash
python tools/benchmark_fc2_nvfp4_algorithms.py --mode block-gate \
  --candidate /tmp/fc2-candidate.json --rounds 6 \
  --workspace-bytes 33554432 --build-directory /tmp/fc2-nvfp4-build \
  --output /output/h3-blackwell-runtime/benchmarks/fc2-nvfp4-block-gate.json
```

Block-gate monkeypatches only `block.mlp.fc2.forward_swiglu`. It still invokes
the accepted SwiGLU producer on the actual FC1 result. Norm, FC1, modulation,
gate, residual, attention, and all other code remain the accepted path.

## Requested NCU Traffic Counters

Use `ncu --query-metrics` on the installed NCU version before replacing `--set
full` with an explicit list. Request the available equivalents of:

```text
gpu__time_duration.sum
dram__bytes_read.sum
dram__bytes_write.sum
lts__t_bytes.sum
lts__t_sector_hit_rate.pct
sm__throughput.avg.pct_of_peak_sustained_elapsed
smsp__inst_executed.sum
smsp__pipe_tensor_cycles_active.avg.pct_of_peak_sustained_active
launch__registers_per_thread
launch__shared_mem_per_block_allocated
```

Metric spelling and availability vary with NCU and GB10. Missing counters must
be reported as unavailable rather than silently substituted. Also retain
kernel duration, grid/cluster dimensions, achieved occupancy, waves, required
workspace, and the extension's tile/stage/inner/cluster configuration.

## Validation Ladder

1. Compile against the image's CUDA 13 headers and print `build_info()`.
2. Characterize heuristics and verify every executed candidate passes
   `cublasLtMatmulAlgoCheck` within caller-supplied workspace.
3. Compare direct-probe candidate FC2 against a cloned
   `fc2.forward_swiglu(gate_up)` output byte-for-byte.
4. Alternate baseline/candidate AB and BA each round; record p50, p95, dense
   TFLOP/s, producer-plus-FC2 time, workspace, and output SHA-256.
5. NCU one FC2 call and inspect traffic/resource counters.
6. Gate complete blocks 0, 24, and 49 with only FC2 GEMM replaced. Require
   BF16 exactness against both the paired baseline and retained traversal.
7. Only after those gates should a separate, explicitly authorized trajectory
   experiment be considered.

That trajectory authorization was granted for the retained candidate. Both the
two-step and canonical 12-step gates passed with byte-exact video and audio
latents; see `RESULTS.md` and the linked benchmark artifacts.

## Prohibited Experiments

These restrictions governed the isolated study. Production integration was
authorized only after the library candidate passed the complete validation
ladder; the canonical-only guard and automatic fallback preserve that boundary.

- Do not alter production source, configuration, dispatch, or existing
  research files.
- Do not expose this extension through `Nvfp4Linear` or any production operator.
- Do not start or perturb the hot service.
- Do not add bias, accumulation, host scalars, another output dtype, a different
  producer, or approximate validation to this study.
- Do not claim a checkpoint hash unless `--checkpoint-sha256` actually computes
  it.
- Do not label undocumented behavior Stream-K. CUDA 13 exposes no documented
  public `cublasLtMatmulAlgoConfig` attribute that directly selects Stream-K;
  the report states that limitation explicitly.
- Do not promote a direct-GEMM result without the complete block gate and later
  separately authorized trajectory validation.

## Limitations

- Heuristics are library, driver, GPU, shape, and workspace specific.
- Explicit split-K is attempted only through documented `AlgoInit`,
  `ConfigSet`, and `AlgoCheck`. Unsupported factors/reduction schemes are
  recorded; they are not emulated.
- Negative `SPLITK_NUM` values returned by the heuristic are retained as raw
  signed library sentinels. They are not interpreted or labeled as Stream-K.
- Characterization deliberately bounds explicit checks and timing candidates.
  Increase limits consciously because each full canonical FC2 call is costly.
- Caller-owned output and workspace are reused during timing. The accepted
  producer still owns its quantized activation allocations.
- A capture supplies packed-denoiser inputs, not pre-captured FC2 operands; the
  harness traverses the loaded H3 model once to construct exact current
  boundaries.