MiniMax-H3 Video Pipeline via ComfyUI APIs: A Reference Implementation
In this article
Developers building production multimodal pipelines now have a concrete reference implementation for driving MiniMax-H3 video and audio generation entirely through ComfyUI's HTTP and WebSocket APIs, without touching the graphical interface. The workflow demonstrates how to orchestrate node graph construction, model weight selection, joint latent decoding, and progress monitoring from a single Python control plane — the kind of headless automation that matters when embedding generation inside a larger system rather than clicking through a GUI. For teams already thinking about automated LLM prompt optimisation in production, this represents the analogous pattern applied to diffusion-based multimodal generation.
Hardware Preflight and VRAM-Gated Model Selection
The pipeline begins with a preflight() function that blocks execution before any model download if BF16 support is absent, explicitly rejecting T4 and K80 runtimes and requiring A100, L4, or H100 class hardware. From the detected VRAM, it selects one of three weight profiles:
| Profile | Minimum VRAM | Diffusion UNet (fl/flf2v) | Text Encoder | ComfyUI flags |
|---|---|---|---|---|
| quality | 70 GB | minimax_h3_fl2va_bf16.safetensors | qwen3vl_32b_minimax_h3_int8_convrot.safetensors | --normalvram |
| balanced | 38 GB | minimax_h3_fl2va_pruned_int8_convrot.safetensors | qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors | --normalvram --cache-none |
| squeeze | 20 GB | minimax_h3_fl2va_pruned_fp8_scaled.safetensors | qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors | --lowvram --cache-none --disable-smart-memory |
Anything below 20 GB VRAM terminates with an explicit error. The pipeline also warns if free disk space falls below 45 GB, since the first run loads roughly 37 GB of weights. The reference-image-conditioned (r2v) mode swaps to a separate UNet file — minimax_h3_ref2va_* — while sharing the same text encoder and VAE assets. Both video VAE (minimax_h3_video_vae_fp16.safetensors) and audio VAE (minimax_h3_audio_vae_fp32.safetensors) are fetched regardless of profile.
Graph Construction and Schema Validation
The H3Graph class builds the ComfyUI execution graph as a Python dictionary. Each call to node() increments an integer counter used as the node ID, stores the class_type and inputs, and cross-references declared inputs against the live /object_info endpoint via the Schema class. This schema-aware validation catches mismatches between the code and whatever node set the running ComfyUI version actually exposes; the pipeline requires ComfyUI version 0.30.0 or newer for the native nodes_minimax_h3.py to be present.
Frame count alignment is enforced through align_frames(), which snaps the requested duration upward to satisfy MiniMax-H3's 17k+5 grid constraint — valid frame counts are 5, 22, 39, and so on at 24 fps. Canvas dimensions are computed by h3_canvas(), which respects both the aspect ratio and a hard area cap of 768 × 1344 pixels. The default configuration targets 0.4 megapixels at 16:9 for a 5-second clip.
Three conditioning modes route through the same backbone: text-to-video (t2v), first/last-frame-conditioned (flf2v) through MiniMaxH3ImageToVideo, and reference-image-conditioned (r2v) through MiniMaxH3ReferenceToVideo with up to 9 reference slots discovered dynamically via autogrow(). The sampler tail is shared across all three: SamplerCustomAdvanced with BasicGuider, BasicScheduler, and KSamplerSelect, defaulting to 20 steps with the res_multistep sampler and simple scheduler, or 8 steps with euler/beta when the Turbo LoRA is active.
Execution, Progress Monitoring, and Output Collection
ComfyUI is launched as a background subprocess bound to 127.0.0.1:8188 with --disable-auto-launch and --preview-method none. The control script polls /system_stats every 2 seconds for up to 300 seconds until the server reports ready. The generated graph is submitted via POST /prompt with a UUID client ID, and progress is consumed over a WebSocket at ws://127.0.0.1:{PORT}/ws. The listener distinguishes executing, progress, and execution_error message types, printing per-node class names and per-step value/max counters inline.
After the WebSocket signals completion, output files are resolved through GET /history/{prompt_id} and then by scanning the output directory for .mp4, .webm, or .mkv files newer than the job start time. The completed workflow is written to /content/last_workflow_api.json for inspection or reuse. This approach to agentic pipeline construction through direct API interaction — bypassing UI state entirely — is increasingly the pattern for integrating generative models into automated systems.
The implementation treats the ComfyUI node graph as an intermediate representation, constructed and validated programmatically against a live schema rather than edited by hand. Combined with VRAM-tiered weight selection and frame-grid enforcement, the pattern is directly portable to any environment where MiniMax-H3's hardware floor can be met, and extensible to new conditioning modes as the node set evolves.