[aryan5v]

Purpose

Adds NVlabs' Open-H3-Regenerate-2K as a video-to-video pipeline. It takes a 768P MiniMax-H3 draft and regenerates it at 2560x1440 in three refine steps. It is served through the OpenAI video routes (POST /v1/videos/sync and POST /v1/videos).

The reference's dense BF16 forward (standard_fwd) is the parity target. Its Sol-Engine path (Sol-Attn, MXFP8, CUDA graphs) and its fast 24-layer VAE decoder are not ported.

Draft while we follow up on speed (NVFP4 decoder, FastH3 refine base) in stacked PRs.

Changes

• Pipeline: new OpenH3Regenerate2KPipeline in fastvideo/pipelines/basic/open_h3_regenerate_2k/. It reuses the MiniMax-H3 DiT, VAEs and community-LoRA path. The merged rank-576 refiner LoRA loads as a constructor-time adapter.
• Ported pieces:
  • the 3D-conv latent upscaler (fastvideo/models/upsamplers/minimax_h3_latent_upscaler.py);
  • the LMS Sharp curve AdaLN branches with their adaln_t_table;
  • the fixed-sigma refine stage, which starts from (1 - s0) * upscaled + s0 * noise;
  • the draft-video loader.
• Shared code touched:
  • LoRA merge precision: an FP32 adapter on a BF16 base is now merged in FP32 and rounded once. Parity needs this. BF16 adapters are unchanged.
  • Checkpoint layout: pinned-class lookups no longer read a root manifest, because the release keeps it under base_model/. The component loader also accepts base_model/<name> module paths.
• Registry and preset: open_h3_regenerate_2k, with no workload type (none exists for video-to-video, same as Kandinsky6 SR).
• OpenAI routes:
  • exactly one video_reference (the draft);
  • output fixed at 2560x1440, 124 frames, 24 fps;
  • data: video URLs stay refused, and local paths need server.media_roots.
• Audio: as in the reference, the transformer gets an encoded-silence stream, so the output MP4 is silent.
• Docs and examples: the example, serving YAML, docs/inference/open_h3_regenerate_2k.md, and a support-matrix row.

curl -sS -X POST http://localhost:8000/v1/videos/sync -H 'Content-Type: application/json' \
  -d '{"prompt": "A red fox trots through fresh snow", "video_reference": {"video_url": "https://example.com/draft.mp4"}}' \
  -o out.mp4

Test Plan

• CPU: pytest fastvideo/tests/pipelines/test_open_h3_regenerate_2k.py fastvideo/tests/entrypoints/test_openai_open_h3_regenerate_2k.py fastvideo/tests/layers
• GPU parity against the reference standard_fwd on one GB200, using the same draft, seed, prompt and noise.
• An opt-in GPU test for the upscaler: tests/local_tests/open_h3_regenerate_2k/test_upscaler_parity.py.
• pre-commit run --files <changed>.

Test Results

CPU tests and pre-commit pass.

	Fox draft (FastVideo H3 t2v, 768P)	Dance draft (576P)
Draft latent, upscaled latent, transformer input	bit-identical	bit-identical
One transformer forward, relative L2	1.1e-2	2.6e-2
Refined latent, relative L2	1.3e-2	3.7e-2
Decoded video, PSNR / SSIM	45.2 dB / 0.993	42.1 dB / 0.992

The transformer outputs are close but not bit-exact. The likely cause is the two different H3 implementations (diffusers' and FastVideo's), not the ported pieces.

Warm timings on one GB200:

• Reference: upscale 0.87 s, refine 71.6 s, decode 15.0 s, so 87.5 s total.
• FastVideo: refine 69.5 s, 107.5 s end to end, including the draft encode and the host copy.
• Attention: the refine forces cuDNN SDPA. PyTorch's default dispatch chose flash attention and took 192 s.

Licences

• MiniMax H3 Community License: base_model/, the refiner LoRA and the fast VAE.
• Their own licences:
  • the latent upscaler (MIT per the reference code, Apache-2.0 per the weights);
  • the LightX2V Turbo and LMS Sharp LoRAs.
• Not redistributed: adaln_t_table.bin, which must be fetched from Comfy-Org/MiniMax-H3.

Known gaps

• No Sol-Engine speed-ups and no fast VAE decoder.
• Not run: 1080p/4K drafts, and parity for the 2-step variant.
• No CI GPU or SSIM test.
• The committed upscaler parity test has not been run.

Checklist

• I ran pre-commit on the changed files and fixed all issues
• I added or updated tests for my changes
• I updated documentation if needed
• I verified SSIM regression tests pass (no SSIM reference added yet)

[mergify]

Merge Protections

🔴 1 of 1 protections blocking · waiting on 👀 reviews and 🤖 CI

	Protection	Waiting on
🔴	PR merge requirements	👀 reviews and 🤖 CI

🔴 PR merge requirements

Waiting for

• #approved-reviews-by>=1
• check-success=full-suite-passed

This rule is failing.

• #approved-reviews-by>=1
• check-success=full-suite-passed
• check-success=fastcheck-passed
• check-success~=pre-commit
• title~=(?i)^\[(feat|feature|bugfix|fix|refactor|perf|ci|doc|docs|misc|chore|kernel|new.?model|skill|skills|infra)\]

[aryan5v]

@greptileai

[greptile-apps]

Confidence Score: 5/5

[Medium impact] The PR appears safe to merge, with one non-blocking cookbook filtering issue.

Findings

1.  Model gets the wrong task ▶

Fix with agent prompt

### Issue 1
docs-site/lib/cookbook-pages.ts:136
The new `backendModels` mapping makes the directory use the page's first tag, `"Video to video"`, instead of the recipe's super-resolution task. `taskLabel()` does not recognize that tag and falls back to `"Text to video"`. Users filtering for `"Video upscale"` therefore cannot find this model. Add a matching task label or use a first tag that maps to `"Video upscale"`.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Adds Open-H3-Regenerate-2K to regenerate draft videos at 2560x1440 through the OpenAI video routes.

• One H3 draft becomes a silent 2K video.
• The OpenAI video routes accept one draft for 2K regeneration.
• The release loads from its nested checkpoint, and wider LoRAs merge with one final rounding.

All supplied previous threads are unnumbered and were rechecked without reposting them. The current code addresses the Sharp reload, merge precision, attention defaults, narration comments, cookbook entry, cache bypass, offload cleanup, and missing-page findings.

_{Reviews (3) · Last reviewed commit: "[docs]: give Open-H3-Regenerate-2K a rea..." · Reviewed by Greptile}

[greptile-apps]

Model gets the wrong task

The new backendModels mapping makes the directory use the page's first tag, "Video to video", instead of the recipe's super-resolution task. taskLabel() does not recognize that tag and falls back to "Text to video". Users filtering for "Video upscale" therefore cannot find this model. Add a matching task label or use a first tag that maps to "Video upscale".

Prompt To Fix With AI

This is a comment left during a code review.
Path: docs-site/lib/cookbook-pages.ts
Line: 136

Comment:
**Model gets the wrong task**

The new `backendModels` mapping makes the directory use the page's first tag, `"Video to video"`, instead of the recipe's super-resolution task. `taskLabel()` does not recognize that tag and falls back to `"Text to video"`. Users filtering for `"Video upscale"` therefore cannot find this model. Add a matching task label or use a first tag that maps to `"Video upscale"`.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.