daniel/h3-blackwell-runtime
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
h3-blackwell-runtime/tools
History
Daniel Maddern 807bd64a82 Add direct first/last-frame (fl2va) keyframe conditioning 
Wire full fl2va into the direct H3 runtime so first/last keyframes flow
through VAE encode -> Qwen vision tokens -> DiT cond segments:

- vae_encoder.py: direct encoder-only H3 video VAE (causal 3D convs,
  reflect spatial padding, causal temporal padding, single-frame tap
  truncation, tiling, FP32 moments + mean/std normalization).
- qwen3vl_vision.py: Qwen3-VL-32B visual tower (27 blocks, 2D rotary,
  deepstack mergers) ported to match the Comfy reference exactly
  (head_dim=72, no-bias proj, LayerNorm blocks, split-half apply_rope),
  plus Qwen image preprocess, mrope ids/freqs, DiT token tags, keyframe
  resize (first=stretch / last=center cover-crop matching Comfy
  common_upscale), and build_fl2va_presentation.
- qwen3vl_text.py: split-half apply_rope, _embed_rows/_run_layers,
  optional mrope position_ids + DeepStack injection at the first three
  decoder layers at visual positions.
- packing.py: H3PromptPacker builds [text | cond | audio | video] with
  tag-run text spans, cond rows (first/last cond_t anchors,
  VISUAL_COND_TIMESTEP=0.999 noise augmentation via CPU-seeded RNG),
  three-timestep row table (t_row*3 + modality_tag), and rope positions.
- runtime.py: load VAE encoder + vision tower; generate() accepts
  first_frame/last_frame, builds the fl2va presentation, encodes keyframes,
  and passes text_token_tags/cond_latents/frame_count/seed to the sampler.
- sampler.py: thread pack kwargs + seed.
- serve_hot_runtime.py / direct_t2v_preview.py: /generate and
  --first-frame/--last-frame accept image paths or base64.
2026-08-19 20:17:41 +07:00
..
benchmark_video_vae_decode.py Default video VAE decode to FP16 2026-08-14 19:50:36 +07:00
compare_adaln_dispatch.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
compare_attention_backends.py Add KJ Sage attention backends 2026-08-14 20:28:56 +07:00
compare_benchmark.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
compare_block0_adaln.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
compare_block0_adaln_gates.py Add H3 parity diagnostics 2026-08-12 21:11:02 +07:00
compare_block0_k_norm.py Add H3 parity diagnostics 2026-08-12 21:11:02 +07:00
compare_block0_mlp_projections.py Add H3 parity diagnostics 2026-08-12 21:11:02 +07:00
compare_block0_mlp_weight_layout.py Add H3 parity diagnostics 2026-08-12 21:11:02 +07:00
compare_block0_norm_variants.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
compare_block0_qkv.py Add H3 parity diagnostics 2026-08-12 21:11:02 +07:00
compare_curve_embedding.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
compare_direct_adaln_weight.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
compare_fl2va_free_run.py Add KJ Sage attention backends 2026-08-14 20:28:56 +07:00
compare_fl2va_steps.py Match Comfy audio sampler boundary 2026-08-13 21:11:41 +07:00
compare_frame_dirs.py Add frame directory comparison tool 2026-08-14 00:28:00 +07:00
compare_h3_assembled_input.py Match H3 assembled FL2VA input 2026-08-13 00:55:58 +07:00
compare_h3_loaded_patch_projection.py Match H3 assembled FL2VA input 2026-08-13 00:55:58 +07:00
compare_h3_patch_projections.py Match H3 assembled FL2VA input 2026-08-13 00:55:58 +07:00
compare_qwen0_loaded_modules.py Probe loaded Comfy Qwen projections 2026-08-12 22:43:43 +07:00
compare_qwen0_projections.py Capture Qwen projection parity 2026-08-12 22:12:19 +07:00
compare_qwen0_quantized_dispatch.py Match Comfy Qwen FP32 projections 2026-08-12 23:59:05 +07:00
compare_qwen_layer_trace.py Verify full Qwen layer trace 2026-08-13 00:05:32 +07:00
compare_refiner_trace.py Verify token refiner trace 2026-08-13 00:13:36 +07:00
compare_text_repeatability.py Add Qwen text repeatability check 2026-08-13 22:53:36 +07:00
compare_vae_decoder_clip.py Compare VAE decoder projection output 2026-08-14 00:34:36 +07:00
compare_vae_full_decode.py Add full VAE decode comparator 2026-08-14 01:12:12 +07:00
compare_vae_pixels.py Decode final sampled latent in VAE gate 2026-08-13 15:59:25 +07:00
compare_vae_temporal_assembly.py Trace VAE temporal blend metadata 2026-08-14 12:56:36 +07:00
compare_vae_tiled_clip.py Add VAE tiled clip comparator 2026-08-14 01:20:27 +07:00
decode_audio_latent.py Make runtime diagnostics opt in 2026-08-14 14:11:36 +07:00
decode_video_latent.py Default video VAE decode to FP16 2026-08-14 19:50:36 +07:00
direct_t2v_preview.py Add direct first/last-frame (fl2va) keyframe conditioning 2026-08-19 20:17:41 +07:00
Inspect-H3Nvfp4Checkpoint.ps1 Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
inspect_capture.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
inspect_checkpoint_tensors.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
inspect_comfy_db.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
inspect_fl2va_initial.py Unpack FL2VA sampler reference state 2026-08-12 14:46:43 +07:00
inspect_fl2va_refiner.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
inspect_safetensors.py Add safetensors shape inspector 2026-08-14 13:32:01 +07:00
inspect_sampler_reference.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
list_attention_backends.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
localize_block0_sublayers.py Add H3 parity diagnostics 2026-08-12 21:11:02 +07:00
localize_block_mismatch.py Add H3 parity diagnostics 2026-08-12 21:11:02 +07:00
localize_block_sublayers.py Add H3 parity diagnostics 2026-08-12 21:11:02 +07:00
patch_comfy_h3_adaln_dispatch.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
patch_comfy_h3_adaln_gate_capture.py Add H3 parity diagnostics 2026-08-12 21:11:02 +07:00
patch_comfy_h3_adaln_weights.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
patch_comfy_h3_block0_adaln.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
patch_comfy_h3_block0_adaln_capture.py Add H3 parity diagnostics 2026-08-12 21:11:02 +07:00
patch_comfy_h3_block0_qkv.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
patch_comfy_h3_block0_raw_qkv.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
patch_comfy_h3_block0_sublayers.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
patch_comfy_h3_block0_sublayers_minimal.py Add H3 parity diagnostics 2026-08-12 21:11:02 +07:00
patch_comfy_h3_block2_sublayers.py Add H3 parity diagnostics 2026-08-12 21:11:02 +07:00
patch_comfy_h3_block_capture.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
patch_comfy_h3_block_capture_minimal.py Add H3 parity diagnostics 2026-08-12 21:11:02 +07:00
patch_comfy_h3_capture.py Capture H3 denoiser calls per sampler step 2026-08-13 15:23:18 +07:00
patch_comfy_h3_curve_embedding.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
patch_comfy_h3_final_capture.py Match H3 final AdaLN bias 2026-08-13 12:40:44 +07:00
patch_comfy_h3_loaded_patch_projection_probe.py Match H3 assembled FL2VA input 2026-08-13 00:55:58 +07:00
patch_comfy_h3_mlp_capture.py Add H3 parity diagnostics 2026-08-12 21:11:02 +07:00
patch_comfy_h3_mlp_projections_minimal.py Add H3 parity diagnostics 2026-08-12 21:11:02 +07:00
patch_comfy_h3_mlp_weight_capture.py Add H3 parity diagnostics 2026-08-12 21:11:02 +07:00
patch_comfy_h3_patch_projection_probe.py Match H3 assembled FL2VA input 2026-08-13 00:55:58 +07:00
patch_comfy_h3_qkv_capture.py Add H3 parity diagnostics 2026-08-12 21:11:02 +07:00
patch_comfy_h3_refiner_trace.py Verify token refiner trace 2026-08-13 00:13:36 +07:00
patch_comfy_h3_temb_trace.py Trace H3 final AdaLN parity 2026-08-13 02:59:46 +07:00
patch_comfy_h3_text_capture.py Add H3 parity diagnostics 2026-08-12 21:11:02 +07:00
patch_comfy_qwen_layer0_projections.py Capture Qwen projection parity 2026-08-12 22:12:19 +07:00
patch_comfy_qwen_layer0_sublayers.py Capture Qwen projection parity 2026-08-12 22:12:19 +07:00
patch_comfy_qwen_layer_trace.py Add H3 parity diagnostics 2026-08-12 21:11:02 +07:00
patch_comfy_qwen_loaded_projection_probe.py Probe loaded Comfy Qwen projections 2026-08-12 22:43:43 +07:00
patch_comfy_qwen_output_capture.py Add H3 parity diagnostics 2026-08-12 21:11:02 +07:00
patch_comfy_qwen_quantized_tensor_probe.py Match Comfy Qwen FP32 projections 2026-08-12 23:59:05 +07:00
patch_comfy_sampler_capture.py Use configured H3 sampler capture directory 2026-08-13 14:07:28 +07:00
patch_comfy_wrapper_capture.py Capture Comfy denoised wrapper boundary 2026-08-13 16:23:08 +07:00
Probe-H3Nvfp4Linear.ps1 Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
profile_attention_path.py Add Sol native attention baseline 2026-08-15 03:35:59 +07:00
profile_h3_block.py Add Sol native attention baseline 2026-08-15 03:35:59 +07:00
profile_nvfp4_linear.py Prototype Vortex native NVFP4 quantizer 2026-08-15 02:22:37 +07:00
profile_nvfp4_pack.py Fix pack profiler imports 2026-08-15 02:57:21 +07:00
replay_comfy_history.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
report_attention_path.py Add Sol native attention baseline 2026-08-15 03:35:59 +07:00
report_h3_block.py Add Sol native attention baseline 2026-08-15 03:35:59 +07:00
run_qwen3vl_text.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
run_upstream_comfy_decode.sh Add Comfy upstream decode runner 2026-08-14 00:23:37 +07:00
serve_hot_runtime.py Add direct first/last-frame (fl2va) keyframe conditioning 2026-08-19 20:17:41 +07:00
smoke_h3_packing.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
smoke_h3_t2v_denoiser.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
smoke_h3_token_refiner.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
smoke_h3_vae_decoder.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
smoke_qwen3vl_text.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
submit_fl2va_reference.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
submit_h3_capture_workflow.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
submit_h3_cat_benchmark.py Add Comfy cat benchmark submitter 2026-08-14 14:20:45 +07:00
Summarize-H3Nvfp4Checkpoint.ps1 Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
sweep_adaln_linear_precision.py Initial direct H3 runtime 2026-08-12 14:12:42 +07:00
trace_block0_exact.py Match H3 DiT backbone trace 2026-08-13 01:30:32 +07:00
trace_qwen0_attention.py Add H3 parity diagnostics 2026-08-12 21:11:02 +07:00
validate_captured_denoiser.py Add H3 parity diagnostics 2026-08-12 21:11:02 +07:00