daniel/h3-blackwell-runtime
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Plan persistent hot runtime

Browse source
This commit is contained in:
Daniel Maddern 2026-08-14 20:02:47 +07:00
parent d1cdd4b6b3
commit 54dd649ccf
1 changed files with 5 additions and 4 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

9
PLAN.md
View file

@ -54,10 +54,11 @@ Prompt-only FL2VA is now at warm Comfy parity with the direct Sage2 baseline. Fe
1. Validate and benchmark the existing `sage3` backend against the same cat prompt, seed, dimensions, and FP16 VAE runtime path used for Sage2 parity. 1. Validate and benchmark the existing `sage3` backend against the same cat prompt, seed, dimensions, and FP16 VAE runtime path used for Sage2 parity.
- First cat benchmark result: Sage3 runs successfully but is slower than Sage2 in this direct path. Sampling was `123.675s` versus Sage2 `114.414s`; warm after text conditioning was `158.111s` versus Sage2 `149.304s`. Same-seed MP4 frame diff versus Sage2 was mean `46.563`, max `255`, so keep Sage3 experimental pending human visual review and stricter tensor gates. - First cat benchmark result: Sage3 runs successfully but is slower than Sage2 in this direct path. Sampling was `123.675s` versus Sage2 `114.414s`; warm after text conditioning was `158.111s` versus Sage2 `149.304s`. Same-seed MP4 frame diff versus Sage2 was mean `46.563`, max `255`, so keep Sage3 experimental pending human visual review and stricter tensor gates.
2. Add exact memory/lifetime optimizations next: `kj_head_sliced` and `kj_chunked_ffn`. These must preserve the validated direct outputs before being kept. 2. Build a persistent hot runtime service instead of measuring only process-per-run CLIs. A warm container must preload and retain Qwen, H3, video VAE, and audio VAE in GPU memory for the selected attention backend, then accept video jobs without model-load latency. Add explicit startup warmup, readiness reporting, backend selection, and timing fields that separate resident-model request latency from cold startup.
3. Evaluate prior H3-tested attention candidates as standalone adapters: `sol_attn` and `kj_sage`. 3. Add exact memory/lifetime optimizations next: `kj_head_sliced` and `kj_chunked_ffn`. These must preserve the validated direct outputs before being kept.
4. Evaluate approximate denoiser caches only after exact baselines are recorded: `easycache` and `h3_cache`. 4. Evaluate prior H3-tested attention candidates as standalone adapters: `sol_attn` and `kj_sage`.
5. Keep every backend explicit per run, with separate quality and timing records for sampling, VAE, audio, and end-to-end output. 5. Evaluate approximate denoiser caches only after exact baselines are recorded: `easycache` and `h3_cache`.
6. Keep every backend explicit per run, with separate quality and timing records for sampling, VAE, audio, and end-to-end output.
## Non-Negotiable Validation ## Non-Negotiable Validation