diff --git a/PLAN.md b/PLAN.md index 3400ae1..8f6bb63 100644 --- a/PLAN.md +++ b/PLAN.md @@ -54,10 +54,11 @@ Prompt-only FL2VA is now at warm Comfy parity with the direct Sage2 baseline. Fe 1. Validate and benchmark the existing `sage3` backend against the same cat prompt, seed, dimensions, and FP16 VAE runtime path used for Sage2 parity. - First cat benchmark result: Sage3 runs successfully but is slower than Sage2 in this direct path. Sampling was `123.675s` versus Sage2 `114.414s`; warm after text conditioning was `158.111s` versus Sage2 `149.304s`. Same-seed MP4 frame diff versus Sage2 was mean `46.563`, max `255`, so keep Sage3 experimental pending human visual review and stricter tensor gates. -2. Add exact memory/lifetime optimizations next: `kj_head_sliced` and `kj_chunked_ffn`. These must preserve the validated direct outputs before being kept. -3. Evaluate prior H3-tested attention candidates as standalone adapters: `sol_attn` and `kj_sage`. -4. Evaluate approximate denoiser caches only after exact baselines are recorded: `easycache` and `h3_cache`. -5. Keep every backend explicit per run, with separate quality and timing records for sampling, VAE, audio, and end-to-end output. +2. Build a persistent hot runtime service instead of measuring only process-per-run CLIs. A warm container must preload and retain Qwen, H3, video VAE, and audio VAE in GPU memory for the selected attention backend, then accept video jobs without model-load latency. Add explicit startup warmup, readiness reporting, backend selection, and timing fields that separate resident-model request latency from cold startup. +3. Add exact memory/lifetime optimizations next: `kj_head_sliced` and `kj_chunked_ffn`. These must preserve the validated direct outputs before being kept. +4. Evaluate prior H3-tested attention candidates as standalone adapters: `sol_attn` and `kj_sage`. +5. Evaluate approximate denoiser caches only after exact baselines are recorded: `easycache` and `h3_cache`. +6. Keep every backend explicit per run, with separate quality and timing records for sampling, VAE, audio, and end-to-end output. ## Non-Negotiable Validation