daniel/h3-blackwell-runtime
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
h3-blackwell-runtime/H3_PROMPT_GUIDE.md
Daniel Maddern bd92baeb46 Add distributed H3 execution and validation
2026-08-22 14:09:45 +07:00

6.7 KiB
Raw Blame History

H3 Prompt Guide

Project guidance for MiniMax H3 audiovisual prompts. This document combines MiniMax's official prompt format with observations from our direct-runtime and ComfyUI tests.

Core Principle

Treat an H3 prompt as an audiovisual timeline, not as a loose prose image description. For T2VA and keyframe modes, use these exact fields in this order:

integrated_multimodal_description: ...

overall_soundscape: ...

non_diegetic_music: ...
  • integrated_multimodal_description contains shots, visible actions, speakers, exact dialogue, singing, and synchronized diegetic events.
  • overall_soundscape contains ambience, Foley, and non-verbal human sounds such as breathing, laughter, and panting.
  • non_diegetic_music contains only music heard by the audience rather than the characters. Use N/A when no score is wanted.

T2VA begins directly with these fields. I2VA, FL2VA, and L2VA add the official picture-alignment instruction before them. Ref2VA uses a different six-section format and should follow the official reference guide.

Dialogue Syntax

Give every speaking character a stable speaker ID such as (S1) or (S2). Describe the speaker, voice, action, and delivery outside the dialogue block. Use quoted speech as the project default and state the language outside it:

The woman with a low, clear voice and measured pace (S1) says in English in a
flat, matter-of-fact delivery: "The meeting starts at three."

MiniMax's official guide recommends <d>[Language]...</d>. Do not use that markup by default in this project. In a matched 10-seed Sage2 sweep, tagged dialogue produced immediate first-100ms activity in 9/10 cases, while quoted dialogue produced it in 0/10. All ten quoted WAVs passed subjective listening. See AUDIO_BOUNDARY_INVESTIGATION.md.

Rules:

  • Preserve the dialogue wording and punctuation exactly.
  • Specify the language explicitly in prose, such as says in English.
  • Describe pitch, timbre, pace, volume, accent, and emotional restraint only when useful.
  • Keep the same speaker ID across shots.
  • Use a compound ID such as (S1,S2) only when speakers vocalize together.
  • Reserve <d>[Language]...</d> for controlled compatibility experiments until the upstream startup-audio defect is resolved.
  • Describe visible text explicitly without reusing dialogue syntax, for example a sign visibly reads MEETING ROOM.
  • For voiceover, use says in an off-screen voiceover and state that the visible character's lips remain completely closed.

Lip Sync And End State

Describe speech as a physical action and specify what happens when it ends:

Her mouth movements naturally synchronize with each spoken word. Immediately
after the final word, her lips meet and her jaw ceases speaking motion. She
remains silent through the final frame.

This is especially important in short clips. Without an explicit end state, H3 may continue mouth motion or add a non-verbal vocal reaction after the line.

For multiple speakers, state whose lips move and whose remain still during each line and pause.

Avoiding Unwanted Laughter

Laughter is a non-verbal human sound, so control it in overall_soundscape, not inside the dialogue block:

overall_soundscape: Quiet, dry indoor room tone with a faint ventilation hum.
No laughter, chuckling, giggling, sighing, gasping, audible breathing, filler
sounds, audience reaction, or other voices.

Also reinforce the visible end state in the shot description: closed lips, stopped jaw motion, neutral expression, and silence through the final frame.

These exclusions are project guidance, not a guaranteed negative-prompt mechanism. H3 jointly generates video and audio, so unwanted sounds remain probabilistic.

Avoid positive cues that can conflict with the exclusion. In our first dialogue test, She smiles slightly repeatedly became laughter across Sage2, PyTorch SDPA, and FlashAttention-4. Since the behavior survived attention changes, the prompt was the stronger common cause. Use observable neutral behavior instead:

She maintains a neutral, composed, closed-mouth expression.

Short Dialogue Tests

For a five-second diagnostic clip:

  • Use one visible speaker and one short sentence.
  • Use one static shot with no cuts.
  • Choose emotionally neutral wording.
  • Request a flat or matter-of-fact delivery.
  • Specify the post-dialogue closed-mouth state.
  • Permit only a simple, positive acoustic bed such as dry room tone.
  • Set non_diegetic_music: N/A.

Do not use overall_soundscape: N/A for a speaking test. The official guide reserves it for complete silence throughout the video.

Reusable Neutral T2VA Prompt

integrated_multimodal_description: [Shot 1] Live-action, cinematic, a static
medium close-up frames exactly one adult woman seated alone in an otherwise
empty, quiet cafe. Warm natural window light falls evenly across her face. She
maintains a neutral, composed, closed-mouth expression and looks steadily
toward the camera. The woman with a low, clear voice and slow, even speaking
pace (S1) physically speaks once in a flat, matter-of-fact delivery:
"The meeting starts at three." Her mouth movements naturally
synchronize with each spoken word. Immediately after the final word, her lips
meet and her jaw ceases speaking motion. She remains silent and maintains the
same neutral expression through the final frame. The camera remains completely
static with no cuts.

overall_soundscape: Quiet, dry indoor room tone with a faint ventilation hum.
No laughter, chuckling, giggling, smiling vocalization, sighing, gasping,
audible breathing, filler sounds, audience reaction, or other voices.

non_diegetic_music: N/A

Prompt Checklist

  • Correct task mode selected.
  • Exact field names and order preserved.
  • Every shot has observable visual and audible events.
  • Every speaker has a stable ID.
  • Dialogue uses quoted speech.
  • Spoken language, delivery, and voice are stated outside the quotation.
  • <d>[Language]...</d> is avoided unless explicitly testing official syntax.
  • Lip motion and post-speech closure are explicit.
  • Ambience and non-verbal sounds are in overall_soundscape.
  • Music is isolated in non_diegetic_music.
  • Script length fits the requested duration.
  • Positive emotional cues do not conflict with audio exclusions.

Official Sources