daniel/h3-blackwell-runtime
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
122 commits 3 branches 0 tags 34 MiB
Commit graph

68 commits

Author SHA1 Message Date
Daniel Maddern
807bd64a82 Add direct first/last-frame (fl2va) keyframe conditioning 
Wire full fl2va into the direct H3 runtime so first/last keyframes flow
through VAE encode -> Qwen vision tokens -> DiT cond segments:

- vae_encoder.py: direct encoder-only H3 video VAE (causal 3D convs,
  reflect spatial padding, causal temporal padding, single-frame tap
  truncation, tiling, FP32 moments + mean/std normalization).
- qwen3vl_vision.py: Qwen3-VL-32B visual tower (27 blocks, 2D rotary,
  deepstack mergers) ported to match the Comfy reference exactly
  (head_dim=72, no-bias proj, LayerNorm blocks, split-half apply_rope),
  plus Qwen image preprocess, mrope ids/freqs, DiT token tags, keyframe
  resize (first=stretch / last=center cover-crop matching Comfy
  common_upscale), and build_fl2va_presentation.
- qwen3vl_text.py: split-half apply_rope, _embed_rows/_run_layers,
  optional mrope position_ids + DeepStack injection at the first three
  decoder layers at visual positions.
- packing.py: H3PromptPacker builds [text | cond | audio | video] with
  tag-run text spans, cond rows (first/last cond_t anchors,
  VISUAL_COND_TIMESTEP=0.999 noise augmentation via CPU-seeded RNG),
  three-timestep row table (t_row*3 + modality_tag), and rope positions.
- runtime.py: load VAE encoder + vision tower; generate() accepts
  first_frame/last_frame, builds the fl2va presentation, encodes keyframes,
  and passes text_token_tags/cond_latents/frame_count/seed to the sampler.
- sampler.py: thread pack kwargs + seed.
- serve_hot_runtime.py / direct_t2v_preview.py: /generate and
  --first-frame/--last-frame accept image paths or base64.
 2026-08-19 20:17:41 +07:00
Daniel Maddern
731a2813fd Add Sol native attention baseline 2026-08-15 03:35:59 +07:00
Daniel Maddern
ddc0c5a2b7 Add NVFP4 pack geometry profiler 2026-08-15 02:56:19 +07:00
Daniel Maddern
d4c243c9ef Match NVFP4 tie-to-even rounding 2026-08-15 02:48:06 +07:00
Daniel Maddern
11d3c4164e Match NVFP4 midpoint rounding 2026-08-15 02:45:16 +07:00
Daniel Maddern
9c0992350f Match NVFP4 signed zero encoding 2026-08-15 02:43:03 +07:00
Daniel Maddern
ef84a2786c Match H3 NVFP4 row scale swizzle 2026-08-15 02:40:09 +07:00
Daniel Maddern
fbfb3453de Zero initialize native NVFP4 block scales 2026-08-15 02:38:08 +07:00
Daniel Maddern
7f9b1560e4 Add H3 NVFP4 scale column swizzles 2026-08-15 02:34:52 +07:00
Daniel Maddern
1200152d79 Match wide NVFP4 block scale swizzle 2026-08-15 02:30:41 +07:00
Daniel Maddern
b3a2544936 Match NVFP4 block scale swizzle 2026-08-15 02:26:48 +07:00
Daniel Maddern
57b84b8d6b Match NVFP4 block scale padding 2026-08-15 02:24:09 +07:00
Daniel Maddern
0e855b8e55 Prototype Vortex native NVFP4 quantizer 2026-08-15 02:22:37 +07:00
Daniel Maddern
1fec77d2e9 Tune Vortex NVFP4 scale geometry 2026-08-15 02:15:19 +07:00
Daniel Maddern
1cc0ffdd00 Add Vortex NVFP4 scale v1 2026-08-15 02:11:47 +07:00
Daniel Maddern
684b645546 Match Torch NVFP4 scale precision 2026-08-15 01:57:25 +07:00
Daniel Maddern
0fda84502a Add Vortex NVFP4 scale kernel 2026-08-15 01:54:02 +07:00
Daniel Maddern
03dbe1456d Prototype Vortex NVFP4 quantizer seam 2026-08-15 01:47:03 +07:00
Daniel Maddern
3d0c093168 Add Sol-Attn backend 2026-08-14 21:04:32 +07:00
Daniel Maddern
c3cc04e98d Add approximate H3 cache modes 2026-08-14 20:38:48 +07:00
Daniel Maddern
fb257ef982 Add exact memory backend options 2026-08-14 20:34:41 +07:00
Daniel Maddern
eaf9324145 Add KJ Sage attention backends 2026-08-14 20:28:56 +07:00
Daniel Maddern
1c0883a54b Add persistent hot runtime service 2026-08-14 20:13:48 +07:00
Daniel Maddern
a388aa64ff Default video VAE decode to FP16 2026-08-14 19:50:36 +07:00
Daniel Maddern
ea7712b4b2 Use paired kitchen RoPE in VAE fast path 2026-08-14 14:58:11 +07:00
Daniel Maddern
8595dd875e Add optional kitchen VAE fast ops 2026-08-14 14:54:38 +07:00
Daniel Maddern
53cd8bd4f4 Make runtime diagnostics opt in 2026-08-14 14:11:36 +07:00
Daniel Maddern
fea69673cd Return native H3 audio latents 2026-08-14 13:57:30 +07:00
Daniel Maddern
59b8302d4e Add direct H3 audio VAE decode 2026-08-14 13:39:46 +07:00
Daniel Maddern
b73d08b231 Compute upstream VAE temporal overlap constants 2026-08-14 13:04:58 +07:00
Daniel Maddern
28a4121df9 Match upstream VAE blend implementation 2026-08-14 01:41:53 +07:00
Daniel Maddern
49bae4ab74 Match upstream VAE temporal padding 2026-08-14 01:25:53 +07:00
Daniel Maddern
f85627a9fd Use SageAttention fallback for direct VAE 2026-08-14 01:00:16 +07:00
Daniel Maddern
9c9e0857c2 Load direct VAE weights as float32 2026-08-14 00:52:57 +07:00
Daniel Maddern
e4f145e9d5 Mirror upstream VAE block math 2026-08-14 00:50:03 +07:00
Daniel Maddern
e043e9af82 Use Comfy attention for VAE when available 2026-08-14 00:41:20 +07:00
Daniel Maddern
e96af86e72 Use fused VAE RMS RoPE when available 2026-08-14 00:38:05 +07:00
Daniel Maddern
d45773053e Match upstream H3 VAE temporal decode 2026-08-14 00:29:31 +07:00
Daniel Maddern
c3d72d9f1e Timestamp direct memory profile output 2026-08-13 23:49:41 +07:00
Daniel Maddern
bc0cff88ce Normalize fastsafetensors CUDA device 2026-08-13 23:41:25 +07:00
Daniel Maddern
8ed9eecf17 Add fastsafetensors loader option 2026-08-13 23:35:01 +07:00
Daniel Maddern
ce022f06e9 Add direct mmap and memory profiling controls 2026-08-13 23:31:30 +07:00
Daniel Maddern
4018fbea15 Match Comfy flow model sigma round trip 2026-08-13 22:36:58 +07:00
Daniel Maddern
9d55aef8a6 Add H3 free-run parity comparator 2026-08-13 22:18:13 +07:00
Daniel Maddern
d0f4dd0ebc Match Comfy H3 timestep precision 2026-08-13 21:50:53 +07:00
Daniel Maddern
c4d64d0d72 Match Comfy BF16 audio carry input 2026-08-13 21:47:02 +07:00
Daniel Maddern
3e56fbc32c Match Comfy BF16 latent packing 2026-08-13 21:29:52 +07:00
Daniel Maddern
b491589418 Use carried audio sampler state directly 2026-08-13 21:23:18 +07:00
Daniel Maddern
f5fe966a7e Match Comfy audio sampler boundary 2026-08-13 21:11:41 +07:00
Daniel Maddern
a2c49ad93b Match Comfy sampler BF16 output boundary 2026-08-13 16:28:51 +07:00