Take a video, throw away everything except motion and camera geometry, and generate a completely different scene that moves the same way. That is the operation this episode builds, and it is the most interesting thing in the H3 toolchain right now.

The two workflows

# Workflow Role
01 Video depth extractor: VDA+++, DepthCrafter, dual route Extracts the geometric skeleton
02 OpenVDN DMD8 depth-guided generation Rebuilds the scene on that skeleton

Workflow 01 is the under-appreciated half. Everything downstream is only as good as the depth it receives.

Two depth extractors, two failure modes

DepthCrafter — strong on temporal coherence. Consecutive frames agree about where things are, which is what you need for camera motion. Weaker on fine geometry: thin structures get blurred together.

VDA+++ — stronger per-frame geometry, especially at edges and thin objects. Weaker temporally: the depth can flicker frame to frame, which shows up as shimmer in the output.

Choosing between them is choosing which artifact you can live with. For a slow camera move, DepthCrafter. For a static camera with complex geometry, VDA+++.

What "structure replay" preserves and what it discards

Preserved: camera movement, subject motion path, spatial relationships, timing.

Discarded: everything visual. Colour, material, lighting, texture, identity. That is the point — the output is a new world with the same choreography.

OpenVDN DMD8 as the engine

Eight-step distilled sampling is what makes this practical: each frame gets depth conditioning plus the motion prior, and the DMD8 variant keeps the cost per frame low enough that a sequence is feasible.

The failure that appears around 20 seconds: motion drift accumulates because each frame conditions on the depth sequence rather than on the previous rendered frame. The choreography slowly diverges from the source. For longer sequences, re-anchor on the source depth every few seconds.

Where you would use this

  • Previs: block a sequence before building the actual set.
  • Style transfer with temporal fidelity: keep the performance, change the world.
  • Music video: the choreography is the content, the look is the variable.

Files

Video walkthrough (Chinese): https://youtu.be/ETAVoT2e8pM
Bilibili: https://www.bilibili.com/video/BV1pPhU6XERp/

Notes

  • Two ComfyUI workflow exports; H3 weights plus one of the depth extractors must exist locally.
  • Tutorial material, not a product.
  • Independent AI-KSK release; not affiliated with the model authors.