daniel/h3-blackwell-runtime
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
142 commits 3 branches 0 tags 34 MiB
Commit graph

142 commits

This branch
This branch
All branches
Author SHA1 Message Date
Daniel Maddern
ffb3c9e729 Debug print in vision tower forward 2026-08-19 22:18:33 +07:00
Daniel Maddern
e4d9a6f8ff Fix position_embeddings unpack (3-tuple of cos/sin/-sin) 2026-08-19 22:12:48 +07:00
Daniel Maddern
a19f051800 Cast qkv to common dtype before SDPA (q/k/v match) 2026-08-19 22:10:39 +07:00
Daniel Maddern
efc5fd9bc1 Fix col_idx expand/reshape precedence in rot_pos_emb 2026-08-19 22:08:21 +07:00
Daniel Maddern
67e4b12246 Make process_image return 2D grid_thw [[t,h,w]] for multi-image support 2026-08-19 22:04:12 +07:00
Daniel Maddern
6866263631 Fix F.conv3d call: remove invalid kernel_size kwarg (uses weight.shape) 2026-08-19 22:01:12 +07:00
Daniel Maddern
453aa86328 Use PIL Lanczos resize in resize_keyframe (F.interpolate no-4D-lanczos) 2026-08-19 21:56:32 +07:00
Daniel Maddern
0b7217485c nin_shortcut is a plain 1x1x1 conv (no causal padding) 2026-08-19 21:49:40 +07:00
Daniel Maddern
ea6ab87a34 Decouple spatial_padding from temporal_causal in causal conv 2026-08-19 21:47:00 +07:00
Daniel Maddern
b557cba173 Fix Downsample3D spatial reflect pad to (W,H) dims only 2026-08-19 21:41:16 +07:00
Daniel Maddern
136297f4ae Fix F.pad dim order (spatial reflect + no T pad) for 1-frame path 2026-08-19 21:36:40 +07:00
Daniel Maddern
c2d507691b Restore front-zero temporal pad for multi-frame causal conv; two-step F.pad (reflect spatial + constant T) 2026-08-19 21:35:07 +07:00
Daniel Maddern
f7c48b067a Remove permute bug: rely on F.conv3d spatial padding 2026-08-19 21:33:08 +07:00
Daniel Maddern
6fc7cf0ad6 Fix causal conv: correct F.pad spatial dim order + 1-frame kernel truncation 2026-08-19 21:31:05 +07:00
Daniel Maddern
3386975326 Fix VAE: use same multi-frame causal path for keyframes, add frame_pre_padding so 1 keyframe -> 1 latent 2026-08-19 21:20:32 +07:00
Daniel Maddern
a0274a9868 Fix tiled_encode latent x-overlap off-by-one (use [j-1] for left neighbor) 2026-08-19 21:13:40 +07:00
Daniel Maddern
65e80be1cf Thread single_frame flag so keyframe truncation does not apply to 1-frame tiles 2026-08-19 21:11:43 +07:00
Daniel Maddern
06fd79a8fd Fix causal temporal padding to match reference (2k-1 front zeros when spatial_padding>0) 2026-08-19 21:09:03 +07:00
Daniel Maddern
390135fca6 Fix VAE pixel normalization in-place bug 2026-08-19 21:07:34 +07:00
Daniel Maddern
9bec53ac36 Fix VAE encoder to load canonical checkpoint key names 2026-08-19 21:04:30 +07:00
Daniel Maddern
807bd64a82 Add direct first/last-frame (fl2va) keyframe conditioning 
Wire full fl2va into the direct H3 runtime so first/last keyframes flow
through VAE encode -> Qwen vision tokens -> DiT cond segments:

- vae_encoder.py: direct encoder-only H3 video VAE (causal 3D convs,
  reflect spatial padding, causal temporal padding, single-frame tap
  truncation, tiling, FP32 moments + mean/std normalization).
- qwen3vl_vision.py: Qwen3-VL-32B visual tower (27 blocks, 2D rotary,
  deepstack mergers) ported to match the Comfy reference exactly
  (head_dim=72, no-bias proj, LayerNorm blocks, split-half apply_rope),
  plus Qwen image preprocess, mrope ids/freqs, DiT token tags, keyframe
  resize (first=stretch / last=center cover-crop matching Comfy
  common_upscale), and build_fl2va_presentation.
- qwen3vl_text.py: split-half apply_rope, _embed_rows/_run_layers,
  optional mrope position_ids + DeepStack injection at the first three
  decoder layers at visual positions.
- packing.py: H3PromptPacker builds [text | cond | audio | video] with
  tag-run text spans, cond rows (first/last cond_t anchors,
  VISUAL_COND_TIMESTEP=0.999 noise augmentation via CPU-seeded RNG),
  three-timestep row table (t_row*3 + modality_tag), and rope positions.
- runtime.py: load VAE encoder + vision tower; generate() accepts
  first_frame/last_frame, builds the fl2va presentation, encodes keyframes,
  and passes text_token_tags/cond_latents/frame_count/seed to the sampler.
- sampler.py: thread pack kwargs + seed.
- serve_hot_runtime.py / direct_t2v_preview.py: /generate and
  --first-frame/--last-frame accept image paths or base64.
 2026-08-19 20:17:41 +07:00
Daniel Maddern
8730920634 Default Spark runs to Sol attention 2026-08-15 03:37:50 +07:00
Daniel Maddern
731a2813fd Add Sol native attention baseline 2026-08-15 03:35:59 +07:00
Daniel Maddern
c2fb5c1e3f Document NVFP4 native pack status 2026-08-15 03:00:06 +07:00
Daniel Maddern
8d90cec133 Fix pack profiler imports 2026-08-15 02:57:21 +07:00
Daniel Maddern
ddc0c5a2b7 Add NVFP4 pack geometry profiler 2026-08-15 02:56:19 +07:00
Daniel Maddern
d4c243c9ef Match NVFP4 tie-to-even rounding 2026-08-15 02:48:06 +07:00
Daniel Maddern
11d3c4164e Match NVFP4 midpoint rounding 2026-08-15 02:45:16 +07:00
Daniel Maddern
9c0992350f Match NVFP4 signed zero encoding 2026-08-15 02:43:03 +07:00
Daniel Maddern
ef84a2786c Match H3 NVFP4 row scale swizzle 2026-08-15 02:40:09 +07:00
Daniel Maddern
fbfb3453de Zero initialize native NVFP4 block scales 2026-08-15 02:38:08 +07:00
Daniel Maddern
7f9b1560e4 Add H3 NVFP4 scale column swizzles 2026-08-15 02:34:52 +07:00
Daniel Maddern
1200152d79 Match wide NVFP4 block scale swizzle 2026-08-15 02:30:41 +07:00
Daniel Maddern
b3a2544936 Match NVFP4 block scale swizzle 2026-08-15 02:26:48 +07:00
Daniel Maddern
57b84b8d6b Match NVFP4 block scale padding 2026-08-15 02:24:09 +07:00
Daniel Maddern
0e855b8e55 Prototype Vortex native NVFP4 quantizer 2026-08-15 02:22:37 +07:00
Daniel Maddern
1fec77d2e9 Tune Vortex NVFP4 scale geometry 2026-08-15 02:15:19 +07:00
Daniel Maddern
1cc0ffdd00 Add Vortex NVFP4 scale v1 2026-08-15 02:11:47 +07:00
Daniel Maddern
684b645546 Match Torch NVFP4 scale precision 2026-08-15 01:57:25 +07:00
Daniel Maddern
0fda84502a Add Vortex NVFP4 scale kernel 2026-08-15 01:54:02 +07:00
Daniel Maddern
03dbe1456d Prototype Vortex NVFP4 quantizer seam 2026-08-15 01:47:03 +07:00
Daniel Maddern
d6eabb150e Add NVFP4 linear profiler 2026-08-15 01:37:42 +07:00
Daniel Maddern
7cc03f3e57 Add H3 block profiler plan 2026-08-15 01:32:18 +07:00
Daniel Maddern
3d0c093168 Add Sol-Attn backend 2026-08-14 21:04:32 +07:00
Daniel Maddern
75ee9ba4ca Document backend implementation status 2026-08-14 20:39:20 +07:00
Daniel Maddern
c3cc04e98d Add approximate H3 cache modes 2026-08-14 20:38:48 +07:00
Daniel Maddern
fb257ef982 Add exact memory backend options 2026-08-14 20:34:41 +07:00
Daniel Maddern
eaf9324145 Add KJ Sage attention backends 2026-08-14 20:28:56 +07:00
Daniel Maddern
1c0883a54b Add persistent hot runtime service 2026-08-14 20:13:48 +07:00
Daniel Maddern
54dd649ccf Plan persistent hot runtime 2026-08-14 20:02:47 +07:00