Hook
A maps company just open-sourced one of the most interesting video models of the year. It does not just make the picture. It makes the sound at the same time, in the same network, in sync. And it hands you the weights under Apache 2.0.
The Story
The model is DreamX-Creator 1.0. It comes from the DreamX Team at AMap — yes, Alibaba’s navigation and maps division (AMAP-ML on GitHub). The paper landed on arXiv on August 31, and the weights and inference code dropped on September 3.
Here is what makes it stand out. Most AI video tools make a silent clip first. Then you bolt sound on afterward — a separate model, a separate step, and lip-sync that never quite lands. DreamX-Creator is native audio-video. You give it a first frame and a text prompt. It denoises the video and the audio together, so the footsteps hit when the foot hits, the voice matches the mouth, and the room tone belongs to the room.
The engine is a 7B joint generator — a video DiT and an audio DiT running side by side. For the first half of the network the two streams stay separate, each an expert in its own medium. Then they meet through a trick the team calls Gated Cross-Modal Attention: small learned gates decide, token by token and head by head, how much the sound should listen to the picture and the other way around. It is a clean way to let two very different signals share information without drowning each other out.
The base model tops out at a working resolution, so a second model does the polish: an Autoregressive 1-Step 2K Refiner (a 5B SR-DiT). It takes a bidirectional, multi-step teacher and distills it into a student that upscales to a crisp 2K in a single step. Fast, and sharp. The whole thing is built on top of the open Wan2.2 foundations — shared text encoder, video VAE and tokenizer — with a dedicated audio VAE (CreatorDACVAE) for the sound.
To get the sync tight, the team trains with reinforcement learning that scores three things at once: is the audio good, is the video good, and do they actually agree? That last reward — the cross-modal one — is the hard part, and it is where most stitched-together pipelines fall apart.
Why You Should Care
Sound is the half of video that AI kept skipping. A silent generated clip is a mood board. A clip that arrives with its own synced audio is a shot. For anyone building trailers, animatics, game cutscenes or short films, native A/V removes an entire painful stage of the pipeline.
And then there is the license. Apache 2.0 is about as permissive as it gets — commercial use, no MAU cliff, no regional carve-outs. That is a real gift next to the more restrictive community licenses on some rival 3D and video models. Because it sits on Wan2.2, it should also feel familiar to anyone already living in that ecosystem, and it is a natural fit for a ComfyUI graph.
The bigger picture is who shipped it. AMap is a maps company. It also has DreamX-World, an interactive world model, in the same family. A navigation team quietly turning into a serious open creative-AI lab tells you how wide the spatial-intelligence race has become.
Try It / Follow Them
The code and weights are live now. The main generator is single-GPU capable, with CPU-offload options for tighter VRAM.
Repo: github.com/AMAP-ML/DreamX-Creator
Weights: huggingface.co/GD-ML/DreamX-Creator
Paper: arXiv 2608.31106
The team: AMAP-ML on GitHub
Two modules, two folders. Run audio_video_generation for the joint clip, then feed the output to video_refiner for the 2K pass. Each has its own requirements file and README.
IK3D Lab Take
This is the release we get quietly excited about. No launch hype, no waitlist — just a strong idea (generate sound and picture as one thing) shipped as open weights under the friendliest license around. The Gated Cross-Modal Attention design is elegant, and building on Wan2.2 means it plugs straight into workflows people already run at home.
The honest caveat: the paper reports competitive results but no hard benchmark tables yet, and native ComfyUI nodes aren’t confirmed — for now it is inference scripts, not a one-click graph. But the trajectory is clear. The silent-clip era is ending, and the model that ends it came from the people who map the roads. We’ll be running it.
Generating synchronized audio and 2K video in one pass is a much more interesting approach than bolting sound on afterward. The Apache 2.0 release should make DreamX-Creator 1.0 especially useful for researchers testing genuinely multimodal video generation.
The single-pass audio-visual generation at 2K is the standout here—especially with the sound and picture kept synchronized by the same network. Releasing it under Apache 2.0 could make DreamX-Creator 1.0 unusually practical for experimentation and production.
Open-sourcing a 2K audio-visual model under Apache 2.0 is a major move, especially with sound and imagery generated in sync through one network. I’m curious to see how DreamX-Creator 1.0 handles longer scenes and whether the audio consistency holds beyond short clips.
The joint generation of 2K video and synchronized sound in a single network is the standout here—especially under Apache 2.0. It will be interesting to see whether the open weights hold up on complex scenes where timing, motion, and audio cues all need to align.
The unified audio-visual generation at 2K is the standout here—especially with the sound and picture produced in sync by the same network. Releasing the weights under Apache 2.0 could make DreamX-Creator 1.0 a particularly useful foundation for open video experiments.
The unified audio-visual generation at 2K is especially intriguing—producing synchronized sound and video in one pass could simplify workflows far beyond adding audio after rendering. Releasing it under Apache 2.0 also makes DreamX-Creator 1.0 unusually practical for experimentation and commercial projects.
Generating synchronized audio and video in one pass at 2K is a much more interesting approach than stitching sound on afterward. The Apache 2.0 release from Alibaba’s Maps team could make DreamX-Creator 1.0 especially useful for researchers who want to experiment with truly multimodal video generation.
The joint generation of 2K video and synchronized sound in a single network is the most compelling part here, especially under Apache 2.0. It’s surprising to see this coming from Alibaba’s Maps team rather than a conventional media AI lab.
The combination of synchronized audio and 2K video in a single network is the standout here, especially with the weights released under Apache 2.0. It’s surprising to see this level of multimodal video generation coming from Alibaba’s Maps team.
Generating synchronized audio and 2K visuals in one pass is a much more ambitious approach than bolting sound on afterward. Apache 2.0 licensing could make DreamX-Creator especially useful for researchers experimenting with genuinely multimodal video generation.
The single-pass generation of 2K visuals and synchronized audio is the standout here, especially with the weights released under Apache 2.0. It’s fascinating to see Alibaba’s Maps team pushing multimodal video in such a practical, open-source direction.
Generating 2K video and synchronized audio in one pass is a genuinely interesting approach, especially with the weights released under Apache 2.0. I’m curious how DreamX-Creator 1.0 handles long-form temporal consistency and whether the joint generation actually avoids the dubbing and timing artifacts common in separate pipelines.
Generating synchronized 2K video and audio in one pass is a much more interesting approach than bolting a separate sound model onto the pipeline. The Apache 2.0 release is especially notable here—it should make it easier to test whether the claimed audiovisual coherence holds up beyond the demos.
Generating 2K video and synchronized sound in a single pass is a genuinely interesting step beyond the usual silent video models. Releasing the weights under Apache 2.0 also makes DreamX-Creator 1.0 especially compelling for researchers and developers who want to experiment with joint audio-visual generation.
The single-pass generation of 2K video and synchronized sound is the standout here—especially with the weights released under Apache 2.0. It will be interesting to see whether the shared audio-visual representation holds up on longer, more complex scenes.
Generating synchronized audio and 2K video in one pass is a much more interesting leap than another image-to-video model. The Apache 2.0 release is especially notable because it gives researchers a chance to study whether shared audio-visual generation improves timing and coherence in practice.