Every world model demo so far has sold you the same fantasy: feed it one photo, fly a camera through the scene, watch a 3D world unfold. Beautiful — until you want it to rain. Holo-World, out of USTC and Li Auto, is the first I’ve seen that treats weather as a control knob you can turn independently — change the sky to a downpour, a snowstorm or fog while every building, road and tree stays exactly where it was. Camera, objects, weather: three dials, one image.
The Story
If you’ve followed the Lab, you know the world-model gold rush: Genie 3, NVIDIA Cosmos, Lyra, Tripo’s Project Eden — all chasing the dream of an explorable world conjured from a single image or prompt. They got very good at camera control and increasingly good at object motion. Weather, though, was always baked in. The world you generated had the lighting and atmosphere of the photo you fed it, and that was that. Want the same street at golden hour and in a blizzard? Generate two worlds and pray they match.
Holo-World’s bet is that these three axes shouldn’t fight each other. Its core trick is a Unified Scene Adapter that splits the problem into separate parameter subspaces: one job is preserving the world (the static geometry — backgrounds, terrain, buildings, captured via rendered background plates and geometry buffers), and a second job is weather transfer (rain, snow, fog, the particle effects and lighting shifts that ride on top). Because those live in different parameter spaces, you can crank the weather without the model hallucinating a different street underneath.
The second piece is what they call Scene-Weather Decomposed CFG — a twist on classifier-free guidance that steers the scene residual and the weather residual separately. In plain terms: you can dial the storm harder without the rest of the image melting into over-saturated mush, which is exactly what happens when you push guidance on a normal video model. The third piece, and the unglamorous hero of the whole thing, is HoloStateData: 15,000+ training samples spanning real footage, simulation, and video-to-video pairs, plus a 150-sample benchmark built specifically to score two things — did the world stay put, and did the weather actually change?
Against video-to-video weather baselines and recent controllable world models — VerseCrafter, Gen3C, Uni3C — the team reports it holds camera and object control while beating them on weather-state generation. The honest caveat: it’s a research drop, the affiliations (Li Auto is in the author list) tell you a chunk of the data is driving footage, and “832×480, a few seconds” is still the resolution reality of open world models. This isn’t a polished app. It’s a capability.
Why You Should Care
Decoupling weather from geometry is a bigger deal than it sounds, because it’s the difference between a pretty clip and a controllable shot. Think about what this unlocks for the people the Lab is built for:
- Previz & virtual production: establish a location once, then deliver it dawn, dusk, rain and snow as separate passes that share the exact same set.
- Game environment art: a weather system that doesn’t require re-authoring the level — same world, swappable atmosphere.
- Architecture & arch-viz: show a client the building on a clear day and in a moody storm without re-rendering the geometry twice.
- Autonomous-driving sim (the obvious Li Auto motive): stress-test the same road in conditions you can’t safely film.
It’s also a philosophical marker. World models are slowly being broken into composable controls instead of one monolithic “generate a world” button. Camera was first. Objects came next. Weather is the third dial to get its own slider — and once a property is decomposable, it’s editable, re-mixable, and pipeline-friendly. That’s the direction every tool the Lab loves is heading.
Try It / Follow Them
- Project page (videos & comparisons): xiangchenyin.github.io/Holo-World
- Code: github.com/XiangchenYin/Holo-World
- Paper: arXiv 2606.20083
Start on the project page — the weather-transfer comparison reels are the fastest way to feel why decomposition matters. If you run your own pipelines, watch the repo: the interesting reusable idea here is the decomposed CFG, which is the kind of thing that quietly shows up in a ComfyUI node six weeks later.
IK3D Lab Take
We’ve covered a dozen world models that can build a place. Holo-World is one of the first that lets you change the weather over it without breaking it — and that small-sounding sentence is exactly the kind of control that turns a tech demo into a tool. It’s low-res, research-grade, and clearly born out of self-driving data. But the idea is the headline: weather is now a knob, not a constraint. The moment someone wires this into a real-time engine and a clean UI, the establishing shot you generated this morning gets four seasons by this afternoon. We’ll be watching the repo.
