h3-blackwell-runtime/VORTEX_RESEARCH_AGENDA.md

Vortex Research Agenda

Flagship Research Project

Vortex: Reference-Exact, Communication-Avoiding Video Diffusion on Commodity PCIe GPUs

The core research question:

Can a large multimodal video diffusion model achieve near-NVLink scaling on ordinary PCIe hardware while preserving reference-exact output?

The existing 86.1% two-GPU efficiency is the starting evidence, not the final contribution.

Novel Architecture

Build an end-to-end H3 block where:

• NVFP4 QKV projections write directly into rank-destined communication buffers.
• Q, K, and V are never separately materialized or repacked.
• Ulysses communication begins at tile granularity while later projection tiles are still computing.
• Video, audio, text, and conditioning tokens use different sharding policies.
• Small conditioning segments are replicated when replication is cheaper than communication.
• Received attention output writes directly into the output-projection layout.
• Reference BF16 rounding boundaries are preserved exactly.
• The scheduler automatically accounts for PIX, PXB, PHB, NODE, and SYS topology.

The target pipeline is:

NVFP4 projection
      |
      v
rank-destined QKV tiles
      |
      v overlapping
PCIe all-to-all
      |
      v overlapping
attention
      |
      v
output projection layout

There is no intermediate QKV pack, full synchronization point, or redundant memory round trip.

Communication and computation overlap has already been explored by systems such as Ulysses Unbound and topology-aware systems such as SwiftFusion. The contribution therefore cannot simply be "we overlapped an all-to-all."

The novel combination is:

• Direct NVFP4 GEMM-to-collective dataflow.
• Ragged multimodal sequence parallelism.
• Segment-aware replication and sharding.
• Reference-exact numerical behavior.
• Automatic PCIe topology adaptation.
• Scaling across inexpensive non-NVLink hardware.

That combination appears meaningfully ahead of public H3 implementations.

Rounding-Boundary-Aware Exact Fusion

Create a compiler or kernel-generation framework that understands where the original model performs BF16 rounding.

Ordinary fusion can silently change output because fused multiply-add performs one rounding where separate operations perform two. NVIDIA documents this numerical distinction in its CUDA floating-point guidance.

The system would:

• Trace the reference H3 graph.
• Mark mandatory numerical materialization boundaries.
• Fuse everything between safe boundaries.
• Generate Triton or CUDA kernels.
• Prove equivalence with adversarial and randomized testing.
• Preserve bitwise behavior across SM100, SM120, and SM121 where possible.

Apply it to:

• AdaLN modulation.
• Residual gates.
• SwiGLU.
• QK normalization and RoPE.
• NVFP4 scaling and packing.
• GEMM epilogues.

This could be a broader contribution than Vortex itself. It would answer:

How much of a mixed-precision transformer can be fused without altering its numerical program?

That is a serious systems and numerical-computing paper.

Trajectory-Bounded Sparse Attention

This is the highest-risk project.

The Sol results show why simple threshold sparsity is not enough:

• It becomes fast at higher thresholds.
• Attention-level relative error rises to roughly 0.5-0.6.
• Local attention error does not reveal what happens to the final video.

Instead of using one global threshold, allocate an error budget across:

• Denoising timestep.
• Transformer block.
• Attention head.
• Video, audio, text, and conditioning segments.
• Spatial and temporal regions.

The runtime would choose dense or sparse attention dynamically, with exact fallback when its predicted error exceeds the remaining trajectory budget.

Existing research already covers distributed sparse attention, including DSA, and feature reuse such as FasterCache. The potential Vortex contribution is different:

Optimize sparsity against final multimodal trajectory error, not isolated attention-tensor error.

That requires proper evaluation of image quality, temporal consistency, audio quality, lip synchronization, and prompt adherence.

Practical Ranking

Research direction	Novelty	Technical risk	Commercial value	Recommendation
PCIe-native ragged Ulysses	Very high	Medium	Exceptional	First
Rounding-aware exact fusion	Very high	Medium	Exceptional	Build alongside
Trajectory-bounded sparsity	Very high	Very high	Potentially exceptional	Later
NVFP4 block superkernel	Medium-high	Medium	Very high	Core component
Topology-autonomous planner	Medium	Low-medium	High	Supporting work
Conventional caching	Medium-low	Medium	High	Only with a novel error guarantee

Landmark Result

A compelling target is:

Scale	Target efficiency	Interconnect
2 GPUs	Above 90%	PCIe, no NVLink
4 GPUs	Above 80%	PCIe
8 GPUs	Above 70%	PCIe or cross-NUMA
Output	Reference-exact or formally bounded	All scales
Portability	SM100, SM120, SM121	Same runtime

Also measure:

• End-to-end latency, not sampling alone.
• Exposed communication time.
• Bytes transferred per block.
• HBM traffic and achieved bandwidth.
• Tensor-core utilization.
• Cost per generated second.
• Energy per generated second.
• PIX/NODE/PHB/SYS topology sensitivity.
• Comparison against SGLang, xDiT, and LightX2V.

Recommendation

Do not lead with sparse attention yet. Lead with:

Reference-exact, communication-avoiding ragged Ulysses with direct NVFP4 projection-to-collective dataflow.

The first fused modulation kernel establishes the exact-fusion infrastructure. Use that infrastructure next to build the direct QKV-to-Ulysses path.

The measured single-GPU implementation order is direct QKV-to-attention layout, direct attention-to-output-projection layout, streamed NVFP4 scale/pack/GEMM, and exact residual-gate epilogues. This order attacks the measured 105.83 ms layout boundary first while preserving the larger projection-to-collective architecture as the distributed destination.

If strong four-GPU and eight-GPU scaling over PCIe can be demonstrated while retaining exact output, this stops being merely an excellent H3 runtime. It becomes credible new research into how large generative models should be executed without proprietary high-bandwidth interconnects.