h3-blackwell-runtime/PERFORMANCE_ROADMAP.md

H3 Performance Roadmap

This document is the canonical performance plan for the direct MiniMax H3 runtime on GB10/SM121, RTX PRO 6000/SM120, and B200/SM100. It separates measured results from projections and requires numerical and visual gates before an approximate optimization becomes a default.

Executive Decision

The first component, bit-exact H3 modulation and residual-gate fusion on GB10, is complete.

This is the best first engineering target because:

1. The current path spends about 39 ms per representative block in modulation and residual gates, before counting the separate 25.8 ms SwiGLU operation.
2. A Triton reference with explicit BF16 rounding already exists and has been validated bit-exact against eager H3 block behavior.
3. The optimization does not alter attention semantics, weights, precision, or scheduler behavior.
4. The same elementwise design applies to SM100, SM120, and SM121.
5. It provides a low-risk test of the profiling, fallback, parity, and deployment machinery required by later custom kernels.

The next component is NVFP4 scale, activation packing, GEMM, and epilogue fusion. Component 1 established the numerical contracts and validation infrastructure required to attempt it safely. A custom attention target follows this component and should preserve SageAttention 2.2.0's validated numerical policy while adopting zero-copy strided QKV input and SM-specific fused execution. Sol sparsity must remain experimental until full latent, audio, and visual quality gates pass.

Completed result: blocks 0, 24, and 49 are bit-exact and 3.6-3.9% faster; the canonical 12-step sampling run is checksum-identical and improves from 310.11 s to 301.05 s (3.0%). Peak allocated memory falls by 408,825,856 bytes. Evidence is in benchmarks/gb10-h3-fused-elementwise-1344x768-124f-seed440420.json.

Internal closeout summary:

Vortex Component 1 is complete. We implemented bit-exact H3 modulation and residual-gate fusion on GB10, reducing canonical 12-step sampling from 310.11 seconds to 301.05 seconds while retaining identical video and audio checksums. The fused runtime improves individual transformer blocks by 3.6-3.9%, reduces peak allocated memory by approximately 390 MiB, and passes 32 deployed numerical and integration tests. This establishes that Vortex can remove meaningful H3 execution overhead without changing the model's numerical program or generated output.

This is measured engineering evidence, not a projected optimization.

Measured GB10 Baseline

The representative workload is 1344x768, 124 frames, 100 synthetic refined-text tokens, and 37,810 packed tokens. Measurements use block 24 unless noted.

Warmed top-level sampling step:

Backend	Step time	Relative to Sage2
SageAttention 2.2.0	25.414 s	baseline
Sol BF16, tau 1.3	21.573 s	1.18x
Sol residual-INT8 Q/K, tau 1.3	19.774 s	1.29x

The denoiser consumes effectively all warmed-step time. Packing takes about 15 ms; output transformation and the RES update together take about 1.4 ms.

Representative Sage2 block:

Component	P50	Approximate block share
Attention	259.9 ms	50.5%
NVFP4 QKV/out/FC1/FC2 projections	151.1 ms	29.3%
Modulation, residual gates, and SwiGLU	64.9 ms	12.6%
QKV and output layout materialization	20.3 ms	4.0%
Norm and RoPE	19.7 ms	3.8%
Full module forward	514.9 ms	baseline

SageAttention 2.2.0 internals:

Component	P50
Q/K INT8 quantization	6.45 ms
V FP8 quantization	10.19 ms
Attention forward	239.00 ms
K smoothing and framework estimate	2.98 ms

Sol residual-INT8 Q/K internals at tau 1.3 with exact conditioning KV:

Component	P50
K summary and residual quantization	4.45 ms
V summary	2.45 ms
Q quantization and routing threshold	4.27 ms
Total preparation	11.53 ms
Routed forward estimate	114.88 ms
Total	126.42 ms

INT8 P-by-V is not useful on GB10: it increases total Sol attention time to 138.65 ms.

Quality Boundary

Current Sol routing is a performance bound, not an accepted Sage2 replacement.

Sol policy	Relative L2 versus Sage2	Performance
Fully dense, tau -100	0.0171	3.2x slower than Sage2
Exact conditioning KV, tau 0.8	about 0.471	about 1.12x faster
Exact conditioning KV, tau 1.3	about 0.626	about 1.87x faster in isolated attention

No measured Sol setting is both faster than Sage2 and numerically Sage-like. Exact conditioning sinks protect prompt/audio rows but do not remove sparse video-attention error. A timestep or block hybrid therefore needs complete latent, audio, lip-sync, and visual evaluation; attention-output timing alone cannot approve it.

Opportunity Ranking

The recoverable ranges below overlap and must not be added directly.

Priority	Target	Measured cost	Plausible saving	Risk	Portability
Complete	Fuse modulation and residual gates	39.1 ms/block projected scope	3.6-3.9% measured block gain	Low	Validated on SM121
1	Fuse NVFP4 scale, pack, GEMM, and epilogue	151.1 ms/block projections	30-75 ms/block	Medium	Architecture-specific
2	Retain Vortex scale discovery	up to 21 ms/block versus Comfy scale path	3-9% block gain already measured	Low	SM100/120/121, retune launch
4	Sage-compatible zero-copy dense attention	259.9 ms/block attention plus 20.3 ms layout	70-140 ms/block	High	Separate SM100/120/121 policy
5	Fuse SwiGLU with FC1/FC2 boundaries	25.8 ms/block plus projection traffic	5-15 ms/block	Medium	SM100/120/121
6	CUDA graphs or persistent denoiser execution	Not isolated yet	likely 1-5% sampling	Medium	Shape-bucket-specific
7	Quality-gated sparse scheduling	up to 140 ms/block attention bound	Unknown accepted saving	Very high	Backend-specific

Native Sage NHD was tested and rejected as a standalone optimization. It was numerically exact but did not materially improve median path time and had worse tail latency. Eliminating layout cost requires fusion with QKV production, RMSNorm/RoPE, or the attention kernel rather than changing Sage's layout flag.

Implementation Phases

Phase 1: Exact H3 Elementwise Fusion

Status: complete on GB10/SM121. Spark deployments enable H3_FUSED_ELEMENTWISE=1; other deployments retain the eager default until validated on their architecture.

Implement an opt-in direct-runtime path for:

1. RMSNorm output plus segmented AdaLN scale/shift modulation.
2. Attention residual gate/add.
3. MLP segmented scale/shift modulation.
4. MLP residual gate/add.
5. A request-layout segment-index cache reused by all 50 blocks and sampling steps.

Preserve explicit BF16 rounding boundaries. Keep the eager implementation as a fallback until all gates pass.

The achieved 3.0% sampling gain is materially below the original theoretical 20-35 ms/block opportunity. Before extending this fusion, profile:

1. Which modulation, gate, or add launches remain outside the fused path.
2. Whether tensor conversions or intermediate allocations remain.
3. Whether the fused kernels are launch-bound or bandwidth-bound.
4. Whether generated kernels perform unnecessary loads or stores.
5. Which remaining elementwise operations can move into exact NVFP4 GEMM epilogues.

The GB10 Nsight follow-up is complete. One warmed block has 53 kernel launches. The fused path contains exactly two modulation and two residual gate/add launches; no modulation, gate, or add launch remains outside it. Those four kernels still consume about 20.48 ms, showing that the residual is tensor traffic rather than launch overhead. The four NVFP4 projections account for 32 launches across scale reduction/finalization, scalar conversions, block-scale initialization, activation packing, and GEMM. Their current intermediate tensor sizes sum to approximately 7.73 GB per block when counted at each projection boundary. See benchmarks/gb10-component2-nsys-summary-1344x768-124f-seed440420.json.

Success criteria:

• Bit-exact outputs at blocks 0, 24, and 49.
• Bit-exact one-step full-denoiser video and audio outputs.
• No change to a complete 12-step latent checksum.
• At least 5% median block improvement or 3% warmed sampling improvement.
• No peak-memory regression.

Phase 2: Architecture Baselines

Capture the same component report on each architecture before writing a shared kernel abstraction:

Architecture	Required baseline
SM121 GB10	Sage2 2.2, current Vortex scale path, 38K and 74K tokens
SM120 RTX PRO 6000	SDPA, packaged Sage2 2.2, 38K and 74K tokens
SM100 B200	SDPA, forced cuDNN SDPA, Sage2 if supported, 38K and 74K tokens

Record actual kernel names, clocks, power, SM utilization, achieved bandwidth, and achieved tensor throughput. Do not infer one architecture's policy from another's result.

Phase 3: NVFP4 Fused Projection Prototype

Status: active next component on GB10/SM121.

The first profile also exposed and fixed a native activation-packer layout bug: the previous width-specific block-scale swizzle failed at the attention output's 7168-feature width. The replacement general 128-row by 4-scale-column mapping is packed-bit exact at H3 widths 5376, 7168, and 14336, and all four linear outputs now match Comfy exactly. Standalone native packing is not consistently faster, so it remains prototype infrastructure rather than a deployed backend.

Component 2 execution order, revised from the measured Nsight profile:

1. Make QKV projection land directly in the attention backend's required layout, with Q/K normalization and RoPE operating there.
2. Make attention output land directly in the token-major layout consumed by the output projection.
3. Eliminate complete NVFP4 packed-activation and block-scale materialization through a CUTLASS/CuTe producer-consumer or persistent design.
4. Integrate reference-exact residual gates into output-projection and FC2 epilogues.
5. Optimize the standalone packer only after the boundary-removing paths exist.
6. Recapture the complete block profile after every accepted boundary removal.

The first two layout items are complete for single-GPU Sage2 on GB10. Sage2 accepts the projection-strided NHD Q/K/V views directly, including in-place Q/K normalization and RoPE, and emits contiguous NHD output that reshapes to token-major rows without a copy. This removes three QKV copies and the attention output copy without replacing the projection GEMM. Blocks 0, 24, and 49 and the two-step and canonical 12-step trajectories are bit-exact. Canonical sampling improves from the Component 1 baseline of 301.05 s to 290.23 s (3.6%), with cumulative improvement of 6.4% from the original 310.11 s baseline. The accepted Nsight recapture falls from 53 to 49 launches and contains none of the four large BF16 layout-copy kernels. Evidence is in benchmarks/gb10-sage-strided-nhd-summary-1344x768-124f-seed440420.json.

The target NVFP4 dataflow is not another wrapper around scaled_mm_nvfp4. Tensor-core GEMMs require exact packed-data and block-scale layouts, so avoiding durable HBM intermediates likely requires an owned CUTLASS/CuTe collective with tile-level producer-consumer execution.

Interface investigation confirms this boundary. Comfy Kitchen 0.2.31 allocates complete QDATA, block-scale, and BF16 output tensors before invoking a cuBLAS blockwise FP4 GEMM with fixed pointers. It cannot consume incremental tiles. CUTLASS DSL 4.6.2's stock SM121 E2M1/E4M3 persistent kernel passes on GB10, but its public mainloop also TMA-loads prebuilt A and SFA. The owned implementation will follow NVFP4_STREAMING_DESIGN.md: preserve the mandatory global-scale reduction, replace the A/SFA producer in one fixed tile, then measure multi-N reuse before attempting full H3 shapes.

P0 is complete for the selected streamed roles. The owned CuTe bridge and alpha-before-BF16 epilogue are bit-exact for QKV, attention output, and FC1, proving E2M1, E4M3, and global-scale interoperability. FC2 remains non-exact because its reference uses a different reduction policy (max_abs=16 after the correct epilogue). Resolve that policy or retain an explicit FC2 cuBLAS fallback before the streaming A producer begins.

FC2 fallback is selected. The streamed backend will reject FC2 dispatch and use the existing Comfy/cuBLAS implementation there; QKV, attention output, and FC1 advance to the P1 software A/SFA producer.

The fixed 128-row P1 producer-consumer checkpoint is complete. The stock kernel's 32-thread DMA warp produces four rows per lane directly into staged E2M1 A and E4M3 SFA shared memory; only B and SFB retain TMA loads. Every 128-K tile has zero packed-data and scale-byte differences for real QKV, attention-output, and FC1 activations, and all three complete GEMM outputs are bit-exact after the fused alpha epilogue. The streamed kernel receives BF16 A plus its tensor scale and has no global activation QDATA or SFA input.

The final arithmetic contract includes Comfy's --use_fast_math behavior: encode scale must use PTX rcp.approx.ftz.f32. Correctly rounded division changes FP4 midpoint decisions for real H3 values. CUTLASS DSL 4.6.2 also requires a static contiguous destination view for vector FP4 stores. Evidence is in benchmarks/gb10-cute-p1-stream-a-summary.json.

The P1 timing gate rejects direct per-output-N-CTA streaming. At 128 rows, the exact streamed kernel is 9.14x slower than the complete reference projection for QKV, 12.98x slower for attention output, and 12.44x slower for FC1. Producer overhead is linear at approximately 0.56-0.60 us per (N,K) tile. Break-even against the removed quantizer would require theoretical N reuse of 78, 28, and 102 tiles respectively, which cannot be implemented by retaining duplicate accumulator state under the current register budget.

The next P2 prototype is a bounded global packed-tile ring or persistent work queue. It must produce each (M,K) A tile once, expose it to all N consumers, and recycle the slot, trading bounded global packed traffic for elimination of the measured repeated BF16 conversion. Do not add canonical-shape runtime dispatch for the rejected schedule. FC2 remains an explicit Comfy/cuBLAS fallback. Evidence is in benchmarks/gb10-cute-p1-stream-a-timing-summary.json.

The bounded-ring P2 checkpoint now uses caller-owned native QDATA/SFA buffers and an allocation-free _into producer. A capacity sweep with one full-activation scale selected 2048 rows. The ring occupies 6.19 MB for QKV/FC1 and 8.26 MB for attention output. Measured chunk latency improves QKV by 10.7%, attention output by 14.4%, and FC1 by 11.3%; modeled canonical totals improve by 10.1-12.5%. Every packed byte, scale byte, and BF16 output matches the reference. The 4096-row point regresses, confirming that larger buffers are not monotonically better. Evidence is in benchmarks/gb10-cute-p2-ring-capacity-summary.json.

Complete 37,810-row projection parity now passes for QKV, attention output, and FC1 in blocks 0, 24, and 49. Every one of the 19 chunks, including the final 946-row tail, is bit-exact. QKV has a stable approximately 32 ms ring time and a positive result across all three blocks. Attention-output timing is mixed. FC1 timing is not usable from the duplicate-model harness because multi-gigabyte output pressure caused large variance and one container stop.

The opt-in QKV runtime gate is complete and rejects the current bounded-ring schedule. An alternating baseline/ring benchmark inside one loaded block removes the clock, allocator, input, and model-copy bias present in separate-process profiles. Blocks 0, 24, and 49 remain bit-exact, but the 2048-row ring regresses median block time by 0.81%, 0.52%, and 0.52%, respectively. On block 24, 3072 rows regress by 1.00%, 4096 by 0.24%, 8192 by 4.80%, and a 37888-row full workspace by 14.49%. The isolated projection win does not survive the chunk launch and scheduler overhead. Keep the implementation opt-in and disabled; do not spend trajectory-validation compute until a launch-fused work queue or a different persistent scheduler passes this same block gate. Attention output and FC1 remain experimental, and FC2 remains the explicit Comfy/cuBLAS fallback. Evidence is in benchmarks/gb10-cute-qkv-runtime-block-gate-summary.json.

The post-optimization profile identifies Sage2's mainloop as the largest cost. Canonical sampling is 288.93 s; block 24 is 468.22 ms, and its production NHD Sage2 call is 258.47 ms. The existing SM89 mainloop alone takes 238.81 ms, while Q/K and V quantization together take 18.34 ms. A direct Hopper WGMMA retarget was rejected by SM121 ptxas, and CUTLASS SM120/121 UMMA does not support the INT8 QK operation needed for exact Sage2 parity. Do not substitute Sage3's different FP4 algorithm under the exact optimization contract.

The next implementable exact target was AdaLN modulation into NVFP4 production. The MSA and MLP modulation passes cost 10.47 ms and 10.60 ms and materialize their values in the RMSNorm buffers solely for QKV and FC1. The deployed fusion recomputes the exact BF16 values inside Vortex scale/pack and retains Comfy GEMMs. Complete producer bytes match in blocks 0, 24, and 49. Alternating block medians improve by 0.28-0.79%; warmed two-step and 12-step runs improve by 0.52% and 0.56%, with bit-identical outputs. Spark deployments enable H3_NVFP4_MODULATE_FUSION=1. Evidence is in benchmarks/gb10-nvfp4-modulate-fusion-summary.json.

SwiGLU-to-FC2 producer fusion is complete. Recomputing the exact BF16 SiLU and multiply boundaries inside Vortex scale/pack removes the complete intermediate activation while retaining the reference Comfy FC2 GEMM. The real producer is byte-exact and takes 21.93 ms. Alternating blocks 0, 24, and 49 improve by 2.14-2.26%; the warmed canonical 12-step trajectory improves by 4.07% with identical video and audio tensors. Enable H3_NVFP4_SWIGLU_FUSION=1 for Spark single-GPU deployments. Evidence is in benchmarks/gb10-nvfp4-swiglu-fusion-summary.json.

The mandatory post-fusion profile supersedes the earlier approximately 515 ms block distribution. With all production flags enabled, fresh block-24 timing is 458.78 ms median without Nsight and 465.78 ms across the Nsight GPU span. The 41 kernels have only 0.084 ms total inter-kernel idle time, so CPU launch gaps are not a block-level bottleneck. A complete warmed sampling step takes 23.708 s, contains 2,694 kernels, and has 12.995 ms total inter-kernel idle time.

The fresh block kernel-time distribution is:

Component	Time	Share
Sage2 preparation and mainloop	268.22 ms	57.60%
Four NVFP4 GEMMs	125.11 ms	26.87%
NVFP4 scale and packing	41.53 ms	8.92%
Norm and RoPE	19.86 ms	4.27%
Two residual gate/add kernels	10.94 ms	2.35%

The one-step distribution independently matches these shares within 0.5 percentage points. The standalone modulation and SwiGLU kernels are gone. Their remaining producer work is 13.04 ms for modulated QKV/FC1 packing and 23.04 ms for SwiGLU-to-FC2 packing. The block still records 8,009,578,496 positive self-allocated bytes, but there are no large activation-layout copy kernels or explicit CUDA memcpy operations. Remaining conversions are small AdaLN table and scalar operations.

GB10 exposes no direct physical DRAM-byte counter. Nsight Compute measures 320.999 GB of L2-request traffic for the block. With cache control disabled, L2 read/write miss sectors imply a 42.117 GB off-chip request proxy: NVFP4 GEMMs account for 56.25%, packing 18.25%, Sage2 15.09%, norm/RoPE 6.54%, and gates 3.87%. This separates the time bottleneck, Sage2, from the off-chip traffic bottleneck, the NVFP4 GEMMs. Do not select a new kernel from the old profile. See benchmarks/gb10-fully-fused-fresh-nsight-summary.json.

The follow-up real block-24 Sage2 decomposition now selects the next exact kernel experiment. Manual preparation plus the existing prequantized mainloop is byte-exact against public SageAttention 2.2.0. Uninstrumented median phase times are 2.37 ms K smoothing, 7.63 ms combined Q/K quantization, 10.61 ms combined V transpose/quantization, and 237.09 ms for the fused mainloop.

The full-counter mainloop capture is profiler-perturbed to 258.92 ms, so its duration is not used as the baseline. Its ratios establish the bottleneck:

Mainloop metric	Result
Registers per thread	255
Dynamic shared memory per CTA	32 KiB
Achieved occupancy	16.83% / 8.08 warps per SM
Scheduler cycles with no eligible warp	63.53%
INT8 QK tensor-pipe utilization	37.77% of elapsed cycles
FP8 PV tensor-pipe utilization	37.77% of elapsed cycles
Combined tensor-pipe utilization	75.54%
Memory throughput	31.61%
L2 hit rate	98.84%
L2 request traffic	161.50 GB
Off-chip request proxy	1.85 GB
Excess shared-memory wavefronts	7.68%

This explains why high SM activity did not imply a saturated tensor pipeline. QK and PV alternate on separate INT and FP tensor sub-pipelines, while online softmax, scaling, conversion, and synchronization occupy scalar pipelines. Only 0.46 warps per scheduler are eligible on average. Fixed-latency dependencies consume 2.01 of the 5.48 cycles between issued instructions, and math-pipe throttle consumes another 1.24; memory scoreboards are much smaller. The 255-register footprint limits the kernel to two CTAs per SM, so it cannot hide these dependencies. Q and KV tail sweeps each add less than 1 ms and exclude tail scheduling as the primary target.

The source-identical classic SM89 P0 is complete and rejected. The exact three-CTA cliff is 168 registers/thread, not approximately 170: 170 still has two CTAs and 16.67% theoretical occupancy, while 168 reaches 25% at the cost of 4.95 billion local spill requests and 78.79% no-eligible cycles. Caps from 240 through 170 never change residency and progressively worsen scheduler eligibility.

Byte-exact source variants also missed the 3% gate. Narrowed scopes reduced static spills from 44/44 to 12/12 bytes and dynamic spill requests by 73%, but improved interleaved latency by only 0.06%. In-place score storage was +0.01%; early K prefetch was -0.30%; independent softmax-chain interleaving was -0.02%. The persistent 128-register FP32 output fragment plus exact QK or instantaneous PV fragments prevents a source-only lifetime cleanup from reaching the next residency tier.

The shared-memory follow-up is also closed. Source attribution maps all 626,970,624 excessive wavefronts to four repeated V-staging LDGSTS.128 instructions. A 128-byte padded V stride raised dynamic shared memory from 32 to 40 KiB but left the excess count exactly unchanged and changed latency by -0.04%. No complete-block benchmark or deployment was run because no mainloop variant crossed 3%. Evidence is in benchmarks/gb10-sage2-p0-register-scheduler-analysis.json, the per-variant latency JSON files, and the P0 NCU reports.

Materializing the optional BF16 LoRA input while these fused producers pack NVFP4 was also tested against the official Turbo-4 adapter. The isolated 1344x768, 124-frame trajectory was bit-exact but regressed from 131.11 s to 135.14 s (3.08%). Reject this implementation: its extra BF16 writes cost more than the removed standalone producer. Active LoRA must retain the existing materialized fallback unless a future design consumes the producer values without global-memory materialization. Evidence is in benchmarks/gb10-nvfp4-lora-producer-fusion-turbo4-isolated.json.

An exact residual-gate epilogue must preserve this numerical program:

1. Complete NVFP4 GEMM accumulation.
2. Convert to the reference BF16 GEMM output.
3. Apply the reference-equivalent gate multiplication and its BF16 boundary.
4. Add the BF16 residual using the reference operation order.
5. Round the final output to BF16.

Bit-exact GEMM output alone is insufficient. Validate randomized epilogues, representative real blocks, all 50 blocks in one denoiser step, a two-step trajectory, and the canonical 12-step trajectory before enabling an epilogue.

The direct-layout target extends naturally to distributed execution: QKV tiles must be able to land in rank-destined Ulysses buffers, and attention output must land in the output projection's local token-major layout without an intervening transpose or contiguous copy.

Prototype behind Nvfp4Linear so model code remains unchanged. Compare:

1. Comfy Kitchen baseline.
2. Vortex scale plus Comfy pack/GEMM.
3. Fused scale and activation pack.
4. Fused scale, pack, GEMM, bias, and output epilogue.
5. Transformer Engine or CUTLASS reference where supported.

Gate every variant on packed activation identity, linear output error, block output error, full denoiser-step error, and final media quality.

Phase 4: Dense Sage-Compatible Attention

The first owned attention kernel should remain dense and target Sage2 behavior:

1. Consume strided BSHD views directly from fused QKV output.
2. Preserve Sage2 per-warp INT8 Q/K quantization and FP8 V policy.
3. Preserve its accumulation and K-smoothing behavior.
4. Fuse Q/K/V preparation with the attention launch where profitable.
5. Emit token-major output suitable for the NVFP4 output projection without an intermediate transpose/contiguous allocation.
6. Tune independently for SM100, SM120, and SM121.

The initial goal is 1.5x attention speedup with bit-exact Sage2 output where the operation order permits it; any nonzero tolerance must be declared and quality-gated before implementation. A 2x attention target is stretch.

Phase 5: Sparse Hybrid Research

Only after the dense backend passes:

1. Keep early quality-sensitive sampling steps dense.
2. Keep first/last sensitive transformer blocks dense.
3. Force text, reference, and audio conditioning KV blocks exact.
4. Sweep tau by timestep using complete generated media, not random attention tensors alone.
5. Reject any policy that degrades prompt adherence, identity, lip sync, audio onset, or temporal stability.

Performance Targets

GB10 / SM121

The current two-step profile implies roughly 305 s for a warmed 12-step 1344x768/124-frame sampling run.

Target	Estimated sampling	Speedup
Conservative	215-240 s	1.3-1.4x
Engineering	170-195 s	1.6-1.8x
Stretch	130-145 s	2.1-2.3x
Aggressive ceiling	about 120 s	about 2.5x

Use 1.5x as the commitment target, 1.8x as the engineering target, and 2.3x as stretch. Do not plan around a 5x end-to-end gain.

RTX PRO 6000 / SM120

These projections have lower confidence because the measured RTX baseline uses PyTorch SDPA rather than packaged Sage2.

Workload	Current SDPA	Engineering target	Stretch target
1344x768, 124 frames	97.94 s	45-60 s	35-45 s
1344x768, 243 frames	311.02 s	120-170 s	90-120 s

Packaging and profiling Sage2 2.2 on SM120 is the first RTX action. Expected overall improvement is 1.6-2.2x at 124 frames and 1.8-2.6x at 243 frames; stretch ranges are 2.2-2.8x and 2.6-3.5x respectively. These are planning ranges, not commitments.

B200 / SM100

Measured B200 SDPA sampling is 79.72 s at 124 frames and 257.86 s at 243 frames, only 1.23x and 1.21x faster than RTX PRO 6000 despite much larger theoretical FP4 and memory-bandwidth capability. No optimized B200 target is assigned until component profiling confirms actual SDPA dispatch, NVFP4 kernel selection, clocks, power, utilization, and achieved throughput.

The current B200 result measures this runtime path, not B200's hardware ceiling.

Evidence

• benchmarks/gb10-sampling-component-profile-1344x768-124f-seed440420.json
• benchmarks/gb10-37810-token-optimization-profile-seed440420.json
• benchmarks/rtxpro6000-server-1v2-sdpa-scaling-seed440420.json
• benchmarks/b200-vs-rtxpro6000-server-sdpa-1344x768-seed440420.json
• tools/profile_h3_block.py
• tools/profile_attention_components.py
• tools/profile_hybrid_attention.py
• tools/profile_sampling_stages.py

Component timings use synchronized probes for attribution. End-to-end claims must use separate uninstrumented runs. Resident services remained loaded during GB10 profiling, so medians are preferred over means and final claims require an isolated rerun.