daniel/h3-blackwell-runtime
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
163 commits 3 branches 0 tags 34 MiB
Commit graph

71 commits

Author SHA1 Message Date
Daniel Maddern
6d8a1c3c2b Record RTX Pro 6000 SDPA baseline 2026-08-22 15:38:09 +07:00
Daniel Maddern
49a9d0d1a1 Fix RunPod API client requests 2026-08-22 14:13:15 +07:00
Daniel Maddern
bd92baeb46 Add distributed H3 execution and validation 2026-08-22 14:09:45 +07:00
Daniel Maddern
1d5faa8f16 Expand H3 runtime acceleration and validation 2026-08-20 22:08:52 +07:00
Daniel Maddern
837c16871d Add resident FL2VA Turbo variants 2026-08-20 19:13:23 +07:00
Daniel Maddern
6d8c9ca4cf Add selectable hot attention backends 2026-08-20 17:39:44 +07:00
Daniel Maddern
9bb96a26e8 Fix FL2VA conditioning parity end to end 2026-08-20 16:43:22 +07:00
Daniel Maddern
807bd64a82 Add direct first/last-frame (fl2va) keyframe conditioning 
Wire full fl2va into the direct H3 runtime so first/last keyframes flow
through VAE encode -> Qwen vision tokens -> DiT cond segments:

- vae_encoder.py: direct encoder-only H3 video VAE (causal 3D convs,
  reflect spatial padding, causal temporal padding, single-frame tap
  truncation, tiling, FP32 moments + mean/std normalization).
- qwen3vl_vision.py: Qwen3-VL-32B visual tower (27 blocks, 2D rotary,
  deepstack mergers) ported to match the Comfy reference exactly
  (head_dim=72, no-bias proj, LayerNorm blocks, split-half apply_rope),
  plus Qwen image preprocess, mrope ids/freqs, DiT token tags, keyframe
  resize (first=stretch / last=center cover-crop matching Comfy
  common_upscale), and build_fl2va_presentation.
- qwen3vl_text.py: split-half apply_rope, _embed_rows/_run_layers,
  optional mrope position_ids + DeepStack injection at the first three
  decoder layers at visual positions.
- packing.py: H3PromptPacker builds [text | cond | audio | video] with
  tag-run text spans, cond rows (first/last cond_t anchors,
  VISUAL_COND_TIMESTEP=0.999 noise augmentation via CPU-seeded RNG),
  three-timestep row table (t_row*3 + modality_tag), and rope positions.
- runtime.py: load VAE encoder + vision tower; generate() accepts
  first_frame/last_frame, builds the fl2va presentation, encodes keyframes,
  and passes text_token_tags/cond_latents/frame_count/seed to the sampler.
- sampler.py: thread pack kwargs + seed.
- serve_hot_runtime.py / direct_t2v_preview.py: /generate and
  --first-frame/--last-frame accept image paths or base64.
 2026-08-19 20:17:41 +07:00
Daniel Maddern
8730920634 Default Spark runs to Sol attention 2026-08-15 03:37:50 +07:00
Daniel Maddern
731a2813fd Add Sol native attention baseline 2026-08-15 03:35:59 +07:00
Daniel Maddern
8d90cec133 Fix pack profiler imports 2026-08-15 02:57:21 +07:00
Daniel Maddern
ddc0c5a2b7 Add NVFP4 pack geometry profiler 2026-08-15 02:56:19 +07:00
Daniel Maddern
0e855b8e55 Prototype Vortex native NVFP4 quantizer 2026-08-15 02:22:37 +07:00
Daniel Maddern
03dbe1456d Prototype Vortex NVFP4 quantizer seam 2026-08-15 01:47:03 +07:00
Daniel Maddern
d6eabb150e Add NVFP4 linear profiler 2026-08-15 01:37:42 +07:00
Daniel Maddern
7cc03f3e57 Add H3 block profiler plan 2026-08-15 01:32:18 +07:00
Daniel Maddern
c3cc04e98d Add approximate H3 cache modes 2026-08-14 20:38:48 +07:00
Daniel Maddern
fb257ef982 Add exact memory backend options 2026-08-14 20:34:41 +07:00
Daniel Maddern
eaf9324145 Add KJ Sage attention backends 2026-08-14 20:28:56 +07:00
Daniel Maddern
1c0883a54b Add persistent hot runtime service 2026-08-14 20:13:48 +07:00
Daniel Maddern
a388aa64ff Default video VAE decode to FP16 2026-08-14 19:50:36 +07:00
Daniel Maddern
f6166ede68 Add VAE decode benchmark tool 2026-08-14 14:36:42 +07:00
Daniel Maddern
ea4f4a2dad Add Comfy cat benchmark submitter 2026-08-14 14:20:45 +07:00
Daniel Maddern
53cd8bd4f4 Make runtime diagnostics opt in 2026-08-14 14:11:36 +07:00
Daniel Maddern
59b8302d4e Add direct H3 audio VAE decode 2026-08-14 13:39:46 +07:00
Daniel Maddern
4696a3e72c Add safetensors shape inspector 2026-08-14 13:32:01 +07:00
Daniel Maddern
81714ed50f Trace VAE temporal blend metadata 2026-08-14 12:56:36 +07:00
Daniel Maddern
feeb4b75c9 Trace VAE blend inputs 2026-08-14 01:50:14 +07:00
Daniel Maddern
4e17e866a0 Add VAE temporal assembly tracer 2026-08-14 01:33:33 +07:00
Daniel Maddern
6dd0b0e737 Add VAE tiled clip comparator 2026-08-14 01:20:27 +07:00
Daniel Maddern
9658faa4be Add full VAE decode comparator 2026-08-14 01:12:12 +07:00
Daniel Maddern
a262d44632 Compare VAE decoder projection output 2026-08-14 00:34:36 +07:00
Daniel Maddern
b55887808d Add VAE decoder clip comparator 2026-08-14 00:31:49 +07:00
Daniel Maddern
d8fd04bce4 Add frame directory comparison tool 2026-08-14 00:28:00 +07:00
Daniel Maddern
1059b9f5d6 Add Comfy upstream decode runner 2026-08-14 00:23:37 +07:00
Daniel Maddern
addc42a452 Add upstream VAE latent decode mode 2026-08-14 00:20:14 +07:00
Daniel Maddern
6b0050adfa Add standalone H3 latent decode tooling 2026-08-14 00:11:17 +07:00
Daniel Maddern
c3d72d9f1e Timestamp direct memory profile output 2026-08-13 23:49:41 +07:00
Daniel Maddern
8ed9eecf17 Add fastsafetensors loader option 2026-08-13 23:35:01 +07:00
Daniel Maddern
ce022f06e9 Add direct mmap and memory profiling controls 2026-08-13 23:31:30 +07:00
Daniel Maddern
5f68f1dce9 Decode preview VAE in inference mode 2026-08-13 23:29:29 +07:00
Daniel Maddern
d6e83440c3 Add Qwen text repeatability check 2026-08-13 22:53:36 +07:00
Daniel Maddern
2ef5d00270 Use Comfy model sigma in free-run comparator 2026-08-13 22:39:47 +07:00
Daniel Maddern
d289b3c497 Allow captured H3 timesteps in preview 2026-08-13 22:23:07 +07:00
Daniel Maddern
9d55aef8a6 Add H3 free-run parity comparator 2026-08-13 22:18:13 +07:00
Daniel Maddern
f5fe966a7e Match Comfy audio sampler boundary 2026-08-13 21:11:41 +07:00
Daniel Maddern
a2c49ad93b Match Comfy sampler BF16 output boundary 2026-08-13 16:28:51 +07:00
Daniel Maddern
1e6a65489a Capture Comfy denoised wrapper boundary 2026-08-13 16:23:08 +07:00
Daniel Maddern
09f5d2ef65 Match Comfy joint AV noise initialization 2026-08-13 16:12:10 +07:00
Daniel Maddern
15ac9b4c9c Decode final sampled latent in VAE gate 2026-08-13 15:59:25 +07:00