30 KiB
Buyer Questions: Current Evidence Status
Internal - do not share without review Evidence date: 2026-08-25 Purpose: Track what Vortex can currently answer, what is only partially evidenced, and what remains open.
Status Summary
| Status | Count | Meaning |
|---|---|---|
| Answerable | 1 | The current product state supports a direct, evidence-backed answer. The answer may be negative. |
| Partially answerable | 13 | Useful measurements or implementation facts exist, but they do not completely answer the buyer's exact question. |
| Open | 11 | The required comparison, integration, evaluation, or operating evidence does not exist. |
| Total | 25 | 14 performance questions and 11 quality questions. |
These classifications measure whether the buyer's full question can be answered, not whether Vortex has done related technical work. Component-level checksums and kernel timings are useful evidence, but they are not substitutes for end-to-end performance, reliability, or perceptual-quality evidence.
Evidence Rules
- Same-workload cross-hardware measurements must not be presented as Vortex-versus-official-runtime comparisons.
- Scalar checksums do not prove complete tensor equality.
- Kernel and block parity do not prove final video or audio parity.
- Native sampling resolution must be distinguished from latent-upscaled output.
- A demonstrated workload is not a proven maximum-safe workload.
- Projected performance is not measured performance.
- Historical Markdown can describe superseded states.
CURRENT_STATE.mdis the canonical status source where documents conflict. - Much of the newest evidence is currently preserved in the working tree rather than an earlier Git commit. Release claims should cite the eventual preservation commit.
Performance Questions
1. What models are supported?
Status: Open
Current answer
The implemented runtime currently supports the MiniMax H3 stack centered on the pruned FL2VA NVFP4 denoiser and Qwen3-VL 32B NVFP4-AWQ conditioner. Implemented H3 modes include prompt-only T2VA, first-frame I2VA, last-frame L2VA, first/last-frame FL2VA, joint video and stereo audio, base beta/RES sampling, official Turbo 4-step v1.1 and 8-step v1.0 adapters, and an optional learned latent upscaler.
Full arbitrary Ref2VA video, audio, identity, and voice conditioning is not implemented.
This does not meet the required buyer-facing model scope. The support matrix must also include:
- Wan, with exact model families, variants, revisions, and task modes still to be selected.
- LTX 2.3.
- LTX 2.5.
- Any additional strategically selected model family.
Existing evidence
README.mdCURRENT_STATE.mdTURBO.mdH3_LATENT_UPSCALER.mdsrc/h3_blackwell_runtime/runtime.py
Remaining gap
There is no implemented or validated Vortex runtime for the required Wan and LTX versions. The phrase "supports Wan" is also too broad until exact checkpoints and modes are pinned.
Next action
Define a versioned model-support contract. For every selected model, record checkpoint revision and SHA-256, T2V/I2V capabilities, audio support, precision modes, valid step counts, native resolutions, duration limits, supported GPUs, parity status, and measured performance. Then integrate and validate Wan and LTX 2.3/2.5 against that contract.
2. What is Vortex versus the official H3 runtime on the same RTX PRO 6000?
Status: Open
Current answer
No same-RTX-PRO-6000 comparison against the official H3 runtime exists. The measured 4.44x result compares the same direct SDPA workload on RTX PRO 6000 and GB10. It is a cross-hardware result, not a Vortex-versus-official-runtime result.
The closest older same-hardware evidence is a GB10 direct-versus-Comfy warm comparison around 149.304s versus 150.26s at 960x544x124. It is not an isolated RTX comparison and does not establish a Vortex advantage.
Existing evidence
benchmarks/rtxpro6000-server-vs-gb10-sdpa-864x480-141f-base12-seed440420.jsonPARITY.md
Remaining gap
There is no licensed or pinned official runtime running beside Vortex on the same RTX GPU, checkpoint, prompt, seed, precision, attention backend, and output settings.
Next action
Run alternated cold and warm trials of official H3 and Vortex on one RTX PRO 6000. Retain full stage timings, final video/audio latents, decoded media, environment versions, and checkpoint hashes.
3. What is Vortex versus reference on the same B200 or GB300 GPU?
Status: Open
Current answer
B200 has been tested only as a hardware comparison against RTX PRO 6000 using the same direct SDPA runner. At 1344x768x124, B200 sampling was 79.7213s versus RTX 97.9439s, a 1.2286x speedup. At 1344x768x243, B200 was 257.8620s versus RTX 311.0163s, a 1.2061x speedup. Under the recorded provider prices, B200 sampling cost was approximately 2.64-2.69x the RTX cost.
These results do not compare Vortex with the official runtime on B200. No GB300 evidence exists.
Existing evidence
benchmarks/b200-vs-rtxpro6000-server-sdpa-1344x768-seed440420.json
Remaining gap
There is no same-B200 official-runtime baseline, no architecture-matched attribution explaining the modest B200 speedup, and no GB300 run.
Next action
Run official H3 and Vortex on the same B200, then repeat the unchanged protocol on GB300. Record clocks, power, kernels, precision, complete timings, latents, and media.
4. How much gain comes from hardware, NVFP4, attention, fusion, runtime residency, and distributed execution?
Status: Partially answerable
Current answer
GB10 block composition and several isolated software gains are measured. In a fully fused block-24 profile, Sage2 represented approximately 57.60% of block time, NVFP4 GEMMs 26.87%, NVFP4 scale and packing 8.92%, norm/RoPE 4.27%, and gates 2.35%.
Measured controlled improvements include:
- Fused elementwise path: approximately
3.0%over the 12-step run and about 390 MiB lower peak allocation. - Strided-NHD QKV copy removal: approximately
3.6%over its baseline. - NVFP4 modulation fusion: approximately
0.56%. - NVFP4 SwiGLU producer fusion: approximately
4.07%. - Two-GPU RTX Ulysses:
1.21x,1.55x, and1.72xas workload size increased.
These numbers do not form a complete additive decomposition. NVFP4's share of runtime is not the gain caused by NVFP4, and independently measured percentages cannot safely be summed.
Existing evidence
benchmarks/gb10-fully-fused-fresh-nsight-summary.jsonbenchmarks/gb10-h3-fused-elementwise-1344x768-124f-seed440420.jsonbenchmarks/gb10-sage-strided-nhd-summary-1344x768-124f-seed440420.jsonbenchmarks/gb10-nvfp4-modulate-fusion-summary.jsonbenchmarks/gb10-nvfp4-swiglu-fusion-summary.jsonbenchmarks/rtxpro6000-server-1v2-sdpa-scaling-seed440420.json
Remaining gap
There is no controlled BF16-versus-NVFP4 model comparison, no complete residency ablation, and no factorial matrix across hardware, attention, fusion, precision, and distribution.
Next action
Add explicit component toggles and run a fixed hardware x precision x attention x fusion x residency x GPU-count matrix with parity gates and uninstrumented end-to-end timing.
5. What are cold-start and warm-start times?
Status: Partially answerable
Current answer
A historical GB10 deployment recorded approximately 30.73s for startup warmup and 2.322s for a later resident 320x192x22 smoke request. RTX H3 model loading was approximately 4.33-4.82s; B200 model loading was approximately 10.71-11.83s.
These values measure different scopes and workloads. They do not provide controlled process-launch-to-ready, first-request, and warmed-request timings for one configuration.
Existing evidence
benchmarks/gb10-post-lora-revert-deployment-smoke.jsonbenchmarks/gb10-nvfp4-modulate-fusion-deployment-smoke.jsonbenchmarks/gb10-nvfp4-swiglu-fusion-deployment-smoke.jsonbenchmarks/rtxpro6000-server-vs-gb10-sdpa-864x480-141f-base12-seed440420.jsonbenchmarks/b200-vs-rtxpro6000-server-sdpa-1344x768-seed440420.json
Remaining gap
Container startup, model loading, CUDA compilation, warmup, conditioning, sampling, decoding, encoding, and muxing have not been separated under one controlled protocol.
Next action
Restart the service and issue one fixed cold request followed by at least ten identical warm requests. Record every stage separately.
6. What are p50, p95, and p99 latencies?
Status: Partially answerable
Current answer
Percentiles exist for selected blocks and kernels. For example, fully fused GB10 block-24 p50 was approximately 458.775ms. Some three-iteration component artifacts report interpolated p95 and p99 values, and a 50-sample Sage2 study reports stable isolated medians.
There are no buyer-grade service-request or end-to-end p50, p95, and p99 measurements. Microkernel percentiles must not be presented as request latency.
Existing evidence
benchmarks/gb10-fully-fused-fresh-nsight-summary.jsonbenchmarks/gb10-cute-qkv-block0-baseline.jsonbenchmarks/gb10-sage2-p3-temporal-pair-analysis.jsonPERFORMANCE_ROADMAP.md
Remaining gap
There is no sufficiently large resident-request distribution, queue timing, or full-media-completion distribution.
Next action
Run at least 100 serialized resident requests per supported operating point and report p50/p95/p99 for queue time, request wall time, sampling, and full media completion.
7. How does performance change across 4, 8, 12, and higher step counts?
Status: Partially answerable
Current answer
Official Turbo 4-step and 8-step modes and base 12-step mode are operational. One 960x544x124 first-frame case measured 60.4s sampling for Turbo-4, 121.1s for Turbo-8, and 148.1s for the stated base 8-step comparison. Other canonical base results use 12 steps.
Turbo and base runs use different adapters and schedules. They are not a controlled step-count-only sweep, and no buyer-grade result above 12 steps exists.
Existing evidence
TURBO.mdtests/test_turbo.pytools/serve_hot_runtime.pybenchmarks/gb10-h3-fused-elementwise-1344x768-124f-seed440420.json
Remaining gap
No matched 4/8/12/16+ latency and quality sweep exists with adapter and sampler held or explicitly accounted for.
Next action
Run a declared 4/8/12/16-step matrix and score both performance and final quality. Separate the effect of fewer steps from the effect of the Turbo adapter.
8. How does it scale with 5-, 10-, 20-, and 30-second videos?
Status: Partially answerable
Current answer
Approximately five-second and ten-second workloads have been measured. At 24 fps, 124 frames is approximately 5.17s and 243 frames is approximately 10.13s. Source guidance describes roughly 124-362 frames, or about 5-15 seconds, as the trained range, with longer output untested.
No 20- or 30-second performance, memory, completion, audio-continuity, or quality evidence exists.
Existing evidence
benchmarks/b200-vs-rtxpro6000-server-sdpa-1344x768-seed440420.jsonbenchmarks/rtxpro6000-server-1v2-sdpa-scaling-seed440420.jsonsrc/h3_blackwell_runtime/upstream_nodes.py
Remaining gap
Longer requests may exceed memory or the model's trained range. That failure boundary is not known.
Next action
Run native 5/10/20/30-second sweeps with peak memory, latency, completion rate, latent health, audio continuity, and temporal-quality review.
9. What happens at native 720p, 1080p, 2K, and 4K?
Status: Partially answerable
Current answer
Demonstrated native sampling resolutions include 864x480, 960x544, and 1344x768. A 1920x1088 result exists after 2x latent upscaling from 960x544. It is not native 1080p generation. No native 2K or 4K evidence exists.
The term "2K delivery" must therefore be described as an upscaled output workflow, not native 2K sampling.
Existing evidence
benchmarks/rtxpro6000-server-vs-gb10-sdpa-864x480-141f-base12-seed440420.jsonbenchmarks/b200-vs-rtxpro6000-server-sdpa-1344x768-seed440420.jsonH3_LATENT_UPSCALER.mdTURBO.md
Remaining gap
Exact native 1280x720 and 1920x1080 have not been validated, and memory-safe native 2K/4K limits are unknown.
Next action
Test exact native 720p and 1080p first. Attempt native 2K and 4K only with explicit memory, duration, quality, and failure criteria. Keep upscaled controls separately labeled.
10. What are maximum safe frame and token counts?
Status: Partially answerable
Current answer
The largest demonstrated complete sampling case is 1344x768x243 frames. The canonical 1344x768x124 workload contains 37,810 packed tokens. On B200, the 243-frame sampling case recorded approximately 22.96 GB peak sampling allocation.
These are demonstrated points, not maximum-safe limits. The direct API does not currently enforce a validated maximum frame or token count.
Existing evidence
benchmarks/b200-vs-rtxpro6000-server-sdpa-1344x768-seed440420.jsonbenchmarks/gb10-h3-fused-elementwise-1344x768-124f-seed440420.jsontools/serve_hot_runtime.pysrc/h3_blackwell_runtime/upstream_nodes.py
Remaining gap
No push-to-failure matrix defines safe limits by GPU, model, resolution, duration, conditioning load, or concurrency.
Next action
Increase frames and conditioning tokens until a predeclared memory, latency, quality, or error boundary is reached. Repeatedly validate the preceding safe point and enforce it in the API.
11. Can one rack run many independent jobs efficiently, or is it optimized only for one distributed job?
Status: Partially answerable
Current answer
The resident HTTP service accepts threaded connections but serializes generation through a single runtime lock. It therefore executes one independent generation at a time per process. Distributed Ulysses or tensor execution instead shards one generation across GPUs.
On two RTX PRO 6000 GPUs, one Ulysses job achieved speedups of 1.21x, 1.55x, and 1.72x as workload size increased. This does not answer whether two independent one-GPU workers would provide better rack throughput or economics.
Existing evidence
tools/serve_hot_runtime.pyDISTRIBUTED.mdbenchmarks/rtxpro6000-server-1v2-sdpa-scaling-seed440420.json
Remaining gap
There is no rack scheduler, intentional concurrency contract, or equal-work comparison between independent workers and distributed jobs.
Next action
On identical GPUs, compare concurrent one-GPU workers with one multi-GPU Ulysses job. Report aggregate throughput, p95 latency, VRAM, queue behavior, and cost.
12. Does throughput remain stable after hours or days of continuous operation?
Status: Open
Current answer
No claim can currently be made. The repository contains repeatability studies, isolated 50-sample kernel tests, short benchmark matrices, and deployment smoke tests. It does not contain an hours- or days-long resident soak.
Existing evidence
benchmarks/rtxpro6000-server-vs-gb10-sdpa-864x480-141f-base12-seed440420.jsonbenchmarks/gb10-sage2-p3-temporal-pair-analysis.jsoncompose.spark-stack.yml
Remaining gap
No sustained throughput, memory-leak, thermal, clock, checksum, restart, or failure trend exists.
Next action
Run a 24-hour soak followed by a 72-hour mixed-workload soak. Capture throughput, p50/p95/p99, host/GPU memory, allocator state, temperature, clocks, power, checksums, failures, and restarts.
13. What happens when GPU memory becomes fragmented?
Status: Open
Current answer
Selected experiments report peak allocated and reserved memory, including an elementwise-fusion reduction of approximately 408.8 MB allocated and 130.0 MB reserved. These are single-run peaks and do not measure fragmentation.
A duplicate-model FC1 experiment caused a pressure-related container stop, but its timing was invalid and it does not establish allocator behavior.
Existing evidence
benchmarks/gb10-h3-fused-elementwise-1344x768-124f-seed440420.jsonPERFORMANCE_ROADMAP.md
Remaining gap
There is no mixed-size workload, allocator fragmentation telemetry, largest-free-block tracking, or recovery test.
Next action
Run hundreds of alternating small and large requests while recording allocated, reserved, inactive split blocks, largest free allocation, failures, and post-request recovery.
14. What are failure and retry rates?
Status: Partially answerable
Current answer
The service returns HTTP 400 for selected validation failures and HTTP 500 for other exceptions. There is no automatic generation retry logic. Individual records mention a pre-sampling dependency failure and a container stop under experimental memory pressure, but no rate can be calculated.
Existing evidence
tools/serve_hot_runtime.pybenchmarks/b200-vs-rtxpro6000-server-sdpa-1344x768-seed440420.jsonCURRENT_STATE.mdPERFORMANCE_ROADMAP.md
Remaining gap
Failure frequency, timeout behavior, cancellation, idempotency, cleanup, retry success, and durable job recovery are unmeasured.
Next action
Add structured request outcome logging and run a fault-injection matrix covering invalid input, OOM, missing checkpoints, CUDA errors, FFmpeg failure, client disconnect, timeout, and process restart.
Quality Questions
15. Are complete per-step latents equal to the official reference?
Status: Partially answerable
Current answer
One small fixed prompt-only FL2VA fixture at 320x192x22, seed 440204, and 12 steps has extensive ComfyUI-oracle replay evidence. The ledger reports exact H3 calls and RES updates at selected boundaries, with some wrapper and state-boundary qualifications.
This does not prove complete per-step equality for the current production runtime across T2VA, I2VA, L2VA, FL2VA, Turbo, different prompts, or different hardware. Exact QDATA/SFA, projection, block, or encoder hashes prove bounded seams rather than complete end-to-end equality.
Existing evidence
PARITY.mdtools/compare_fl2va_steps.pyCURRENT_STATE.md
Remaining gap
The complete official initial state, conditioning, every scheduler pre/post state, and final video/audio latent set are not preserved in one self-contained assertion-based fixture.
Next action
Capture and assert torch.equal at every step for at least base T2VA, FL2VA, and Turbo-4 using pinned official-reference artifacts.
16. Are final video and audio latents equal across hardware?
Status: Open
Current answer
Cross-hardware final-latent equality has not been tested. RTX repeat runs recorded identical scalar checksums, but complete latents were not retained. B200 records timing and scalar values without matching RTX or GB10 tensors. One-rank Ulysses and tensor modes were equal on one GB10, but that tests execution paths on one hardware class rather than cross-hardware equality.
Existing evidence
benchmarks/rtxpro6000-server-vs-gb10-sdpa-864x480-141f-base12-seed440420.jsonbenchmarks/b200-vs-rtxpro6000-server-sdpa-1344x768-seed440420.jsonDISTRIBUTED.md
Remaining gap
Complete matching video/audio tensors are absent across GB10, RTX PRO 6000, B200, and GB300.
Next action
Save full final video and audio latents for identical requests on each available GPU. Compare raw-byte SHA-256, torch.equal, maximum and mean error, RMSE, and cosine similarity.
17. Does NVFP4 introduce perceptual degradation relative to BF16?
Status: Open
Current answer
No defensible perceptual answer exists. The extensive NVFP4 parity work compares optimized implementations with the existing NVFP4/Comfy Kitchen path. It does not compare the pruned NVFP4 model's output quality with the official BF16 base.
Turbo adapters were released for a BF16 base; use over the pruned NVFP4 base is operationally validated but not upstream-certified.
Existing evidence
PLAN.mdPERFORMANCE_ROADMAP.mdTURBO.mdresearch/EXPERIMENT_REGISTRY.md
Remaining gap
There is no matched BF16/NVFP4 prompt-and-seed corpus, latent comparison, video/audio metric set, or blinded preference study.
Next action
Run official BF16 and pruned NVFP4 models at equal prompts and seeds. Retain all latents and decoded media, score objective quality, and conduct blinded human comparison.
18. Are results stable across hundreds of prompts and seeds?
Status: Open
Current answer
No hundreds-of-prompts-and-seeds campaign exists. The strongest related evidence is a narrow ten-seed audio-format investigation and a ten-prompt text-conditioning repeatability tool. Neither is a broad end-to-end generation study.
Existing evidence
benchmarks/audio-dialogue-format-sweep-sage2-seeds440420-440429.jsontools/compare_text_repeatability.pytests/
Remaining gap
There is no representative prompt taxonomy, broad seed matrix, retained media set, failure count, collapse rate, or quality distribution.
Next action
Run at least 100 prompts across at least three seeds, spanning dialogue, music, action, static scenes, keyframes, identities, text, faces, hands, and difficult camera motion.
19. What are temporal consistency, motion quality, and identity-preservation results?
Status: Open
Current answer
No systematic results exist. One Turbo comparison contains informal observations about composition, motion, and exposure drift. There are no standard temporal or identity metrics and no multi-seed evaluation.
Full arbitrary identity/reference conditioning is also outside the currently implemented H3 direct-runtime scope.
Existing evidence
TURBO.mdVORTEX_RESEARCH_AGENDA.mdCURRENT_STATE.md
Remaining gap
There is no subject-tracking suite, motion-adherence test, temporal perceptual score, optical-flow consistency score, or identity-embedding drift measurement.
Next action
Build a fixed multi-seed suite with prescribed subjects and motion. Add temporal perceptual, flow-consistency, subject/face embedding, motion-adherence, and human-review scoring.
20. How well does it handle text, hands, faces, and complex camera motion?
Status: Open
Current answer
The repository contains prompt guidance and isolated diagnostic prompts, not category-level quality measurements. No hand/finger benchmark, OCR suite, face-quality suite, or camera-trajectory evaluation exists.
The latent upscaler is explicitly described as unsafe for text, QR codes, and identity-critical content, and it can alter faces.
Existing evidence
H3_PROMPT_GUIDE.mdH3_LATENT_UPSCALER.mdbenchmarks/t2va-dialogue-quoted-864x480-141f-base12-sage2-seed440420.json
Remaining gap
There is no curated hard-case corpus, automated task scoring, defect taxonomy, or blinded review.
Next action
Create four explicit test buckets: readable text/OCR, hand-object interaction, close and multi-angle faces, and prescribed static/pan/dolly/handheld camera motion.
21. What are speech quality, voice consistency, and lip-sync scores?
Status: Partially answerable
Current answer
Stereo 32 kHz audio generation, decoding, and muxing are operational. A ten-seed quoted-dialogue diagnostic found useful prompt-format behavior and received subjective WAV review. Quoted prompts removed immediate startup activity in all ten recorded cases.
Speech accuracy, voice consistency, and full-video lip-sync remain unscored. Signal levels and successful muxing are not speech-quality evidence.
Existing evidence
benchmarks/audio-dialogue-format-sweep-sage2-seeds440420-440429.jsonAUDIO_BOUNDARY_INVESTIGATION.mdCURRENT_STATE.mdPARITY.md
Remaining gap
There are no ASR/WER results, speaker-embedding measurements, onset/offset accuracy, lip-sync metrics, or structured human scores.
Next action
Generate full dialogue videos with known scripts and visible speakers. Retain lossless PCM and score WER, speaker consistency, speech timing, automated lip-sync, and blinded human lip-sync.
22. Does the latent upscaler introduce ringing, texture, or chromatic artifacts?
Status: Partially answerable
Current answer
Yes, known examples show harsher illustrated texture, ringing or chromatic edges, and facial changes. One principal source-to-downscaled comparison recorded SSIM 0.924986. The upscaler is not considered fidelity-safe for text, QR codes, or identity-critical content.
What is unknown is how often these defects occur, their severity distribution, and whether they are preferable to conventional scaling.
Existing evidence
H3_LATENT_UPSCALER.mdCURRENT_STATE.mdtests/test_latent_upscaler.py
Remaining gap
There is no representative multi-clip artifact study, edge-overshoot metric, color-fringe metric, LPIPS distribution, or blind crop review.
Next action
Evaluate at least 20 clips containing text, high-contrast edges, faces, skin, hair, and fine texture against bicubic/Lanczos and native-resolution controls.
23. Has there been a blind human A/B evaluation?
Status: Open
Current answer
No randomized, blinded, multi-rater A/B evaluation is documented. Existing audio, Turbo, and upscaler observations are informal and do not record blindness, randomization, rater count, inter-rater agreement, or confidence intervals.
Existing evidence
benchmarks/audio-dialogue-format-sweep-sage2-seeds440420-440429.jsonTURBO.mdH3_LATENT_UPSCALER.md
Remaining gap
There is no evaluation protocol, recruited panel, anonymized media set, scoring rubric, or statistical analysis.
Next action
Randomize and anonymize at least 30 matched clip pairs, recruit at least three independent raters, collect preference and defect labels, and report confidence intervals and agreement.
24. Does the four-step Turbo model materially reduce quality?
Status: Partially answerable
Current answer
Turbo-4 is operational and materially faster, but its general quality delta is unknown. On one matched 960x544x124 case, Turbo-4 sampled in 60.4s, Turbo-8 in 121.1s, and the stated base comparison in 148.1s. Informal review found Turbo-4 somewhat more compositionally stable in that example, Turbo-8 showed more motion, and both showed exposure drift.
One example cannot establish population-level quality. There is no scored comparison with base 12-step output or official LightX2V BF16 Turbo.
Existing evidence
TURBO.mdbenchmarks/gb10-nvfp4-lora-producer-fusion-turbo4-isolated.jsontests/test_turbo.py
Remaining gap
No matched prompt suite, latent-quality analysis, video/audio metrics, or blind preference result exists.
Next action
Compare base 12-step, Turbo-8, and Turbo-4 over at least 20 prompts and three seeds, then separately compare NVFP4 Turbo-4 with the official BF16 path.
25. Can customers choose between reference-quality and maximum-throughput modes?
Status: Answerable
Current answer
Not as coherent, named, validated product modes today.
The runtime exposes low-level controls for attention backend, Turbo adapter, upscaling, cache behavior, VAE dtype, and distributed mode. An expert can approximate a conservative path using base 12-step, Sage2, and no cache/upscaler, or a faster path using Turbo-4 and selected acceleration controls. Production fusions and layouts are also selected through startup environment variables.
These combinations are not immutable buyer-facing reference_quality and maximum_throughput contracts. "Reference quality" cannot currently mean proven official-BF16 equivalence because that quality comparison remains open.
Existing evidence
src/h3_blackwell_runtime/runtime.pysrc/h3_blackwell_runtime/attention.pytools/serve_hot_runtime.pycompose.spark.ymlCURRENT_STATE.md
Remaining gap
There are no named profiles, frozen configuration contracts, per-profile regression tests, quality floors, or supported-model matrix.
Next action
Define immutable reference_quality and maximum_throughput profiles per supported model. Validate the reference profile against the official oracle and the throughput profile against explicit latency, reliability, and perceptual-quality thresholds.
Recommended Closure Order
Immediate evidence work on available hardware
- End-to-end p50/p95/p99 resident-request harness.
- Controlled cold-start and warm-start instrumentation.
- Full per-step latent capture and assertion fixture.
- Five-, ten-, twenty-, and thirty-second duration sweep.
- Maximum safe frame/token push-to-failure matrix.
- Structured failure logging and fault injection.
- Fixed 100-prompt, three-seed stability suite.
- Upscaler artifact benchmark.
Highest-leverage engineering work
- Pin and implement the Wan and LTX 2.3/2.5 support matrix.
- Build a shared video/audio quality-scoring pipeline.
- Add controlled precision, attention, fusion, residency, and distribution ablations.
- Define named reference-quality and maximum-throughput profiles.
- Add allocator telemetry and mixed-workload soak support.
External or additional-hardware work
- Official H3 versus Vortex on the same RTX PRO 6000.
- Official H3 versus Vortex on the same B200 and GB300.
- Independent-job versus distributed-job rack-scale testing.
- Four-, eight-, and higher-GPU scaling.
- Blind human A/B evaluation.
Primary Evidence Index
CURRENT_STATE.md: canonical implementation and open-work status.README.md: runtime scope and operating modes.PARITY.md: historical correctness ledger; interpret contradictory rows cautiously.PERFORMANCE_ROADMAP.md: profiles, measured opportunities, and projected targets.DISTRIBUTED.md: distributed implementation, topology, timing, and parity gaps.TURBO.md: Turbo revisions, schedules, timing, and limited subjective observations.H3_LATENT_UPSCALER.md: upscaler implementation, timing, and known quality defects.AUDIO_BOUNDARY_INVESTIGATION.md: audio diagnostics and unresolved quality work.research/EXPERIMENT_REGISTRY.md: permanent performance-experiment registry.research/REPRODUCIBILITY.md: environment, command, and reproducibility gaps.benchmarks/: raw and summarized timing, parity, memory, profiler, and smoke evidence.