From 75ee9ba4ca76a22fcf04e8f5ed330c09d0ae3f35 Mon Sep 17 00:00:00 2001 From: Daniel Maddern Date: Fri, 14 Aug 2026 20:39:20 +0700 Subject: [PATCH] Document backend implementation status --- PLAN.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/PLAN.md b/PLAN.md index 366f3cb..94893bc 100644 --- a/PLAN.md +++ b/PLAN.md @@ -57,7 +57,10 @@ Prompt-only FL2VA is now at warm Comfy parity with the direct Sage2 baseline. Fe 2. Build a persistent hot runtime service instead of measuring only process-per-run CLIs. A warm container must preload and retain Qwen, H3, video VAE, and audio VAE in GPU memory, then accept video jobs without model-load latency. Add explicit startup warmup, readiness reporting, request-level attention selection (`sage2`, `sdpa`, `sage3` initially), and timing fields that separate resident-model request latency from cold startup. 3. Add exact memory/lifetime optimizations next: `kj_head_sliced` and `kj_chunked_ffn`. These must preserve the validated direct outputs before being kept. 4. Evaluate prior H3-tested attention candidates as standalone adapters: `sol_attn` and `kj_sage`. + - `kj_sage` is implemented as explicit SageAttention mode backends: `kj_sage_cuda`, `kj_sage_triton`, `kj_sage_fp8`, and `kj_sage_fp8pp`; all passed hot-runtime smoke tests. + - `sol_attn` is still blocked on locating/adding the standalone Sol-Attn source or package. It is not installed in the Spark image and is not present in this repository. 5. Evaluate approximate denoiser caches only after exact baselines are recorded: `easycache` and `h3_cache`. + - Initial direct cache modes are implemented as opt-in approximate sampler modes. They reuse cached denoised deltas and report skipped-step stats; full-size quality/threshold sweeps are still required before using them for production output. 6. Keep every backend explicit per run, with separate quality and timing records for sampling, VAE, audio, and end-to-end output. ## Non-Negotiable Validation