
Date:2026-07-13
Who this is for: Graduate students, researcher on body intelligence and robots, multimodal generation and reinforcement learning direction readers, technical readers with engineering background
search date:2026-07-13
Abstract
World Models is expanding from the "environmental dynamics model" in reinforcement learning to infrastructure for the physical World, robotics and interactive video environments. The early World Models, PlaNet, Dreamer series proved that the Agent can imagine the future and train strategies in the submersible space; 2024 Genie, Gamegenen Promote the route to "control environment from video learning"; Genie 3, V-JEPA 2, Cosmos, DreamGen, Qwen-RobotWorld, OSCAR from 2025-2026 And related reviews, the focus of the research is to long-term consistency, action condition control, physical interpretability, robot strategy assessment and virtual real migration.
If an agent takes a particular action, how will the world change? Visual realism answers only part of that question. Do the predicted changes respect conservation laws and causal relationships? Can the model predict them fast enough for closed-loop control, and can it help real robots learn with less trial and error?
Contents
- research background
- core science questions
- A reading guide to the past year’s papers
- Key papers by research question
- Comparing the methods
- How to interpret the experiments and benchmarks
- Reproducibility and data problems
- Limitations and open questions
- Research questions for graduate students
- Summary
- References
research background
"The World model" is not a new concept. HA and Schmidhuber's World Models in 2018 compress visual observations into latent variables, predict the future with a circular model, and let the controller train strategies in "dreams". The PlaNet and Dreamer series further make the modeling reinforcement learning route from pixel to submersible space planning: first learn environmental dynamics, then rollout inside the model, and finally convert the imagination trajectory into an action strategy.
The concept has expanded significantly in the past two years: the video generation model has allowed researchers to rethink "whether video prediction can be a world simulation"; the robot Foundation model needs a lot of interactive data, and the real collection is expensive, slow and at risk; LLM/VLM It can handle language and high-level semantics, but it is still unstable to contact, occlusion, inertia, spatial constraints and long-term cause and effect. Therefore, World Model is becoming a key complement of "how language intelligence falls into the physical world".

core science questions
first, Predict what is to understand the world. Pixel-level prediction can produce beautiful pictures, but it may not be conserved or cause and effect; it means that space prediction is more efficient, but it is difficult to directly check the details. Researchers need to distinguish three things: visual realistic, dynamic dynamics, and useful tasks.
second, How to enter the model. The action of traditional RL is defined by Simulator; real robots have high-dimensional joints, end effectors, camera perspective, language instructions and human demonstrations. In 2026, the discussion about the World Action Model is essentially asking: How is the world model connected to an executable action?
third, How long-term consistency is assessed. The interactive world model is more difficult to generate than normal video, because every step error is inherited by the next step. The Genie 3 official page emphasizes real-time interaction, minutes consistency, and one-minute visual memory; these capabilities are critical, but independent reproduction is still limited.
fourth, Does simulation really help real robots?. If the synthesis trajectory is only visually diverse, the strategy may learn wrong contact dynamics; if the simulation is too conservative, it cannot cover the real long tail. The value of DreamGen, Cosmos, and OSCAR is to start linking "generating video" with downstream policy benefits.
A reading guide to the past year’s papers
interactive video world model is the hottest main line. GameGen proves that the diffusion model can be used in real-time Next-Frame Simulation in DOOM; Genie uses unmovement tag video, Latent Action and controlled environment to generate a paradigm; Genie 3 The official introduction pushes the goal to the text prompt to generate a dynamic world that can be navigated in real time and is consistent for a long time. The 2026 review summarizes the difficulties into action-controlled, long-term memory and real-time response.
World model and robot planning is the second main line. V-JEPA 2 uses large-scale video self-supervised learning, and then uses a small amount of DROID robot video to train the Action condition model. Lifting Embodied World Models focuses on the problem that high-dimensional actions are difficult to search, and uses high-level Waypoint as a low-dimensional interface. The World Action Models tutorial organizes Observation-Space, State-Space, Imagine-then-execute, Joint Video-Action Modeling etc. paradigm.
Physical AI Platform is the third main line. NVIDIA Cosmos defines the World Foundation Model as a general-purpose platform for post-trainable training; the Cosmos-Predict2.5 report Text2World, Image2World, Video2World are unified to physical AI scenes. Although the GR00T N1 is closer to the VLA policy model, the dual system architecture of "visual language understanding + real-time action diffusion" represents the trend of the combination of high-level reasoning and low-level control.
Robot Synthetic Data and Virtual Assessment is the fourth main line. DreamGen generates Synthetic Neural Trajectories and restores pseudo-movement training strategies; OSCAR uses Skeleton-Conditioned World Action Model to unify different Embodiment Qwen-RobotWorld uses language as the action interface, covering operations, driving, navigation and human-to-robot transfer. Its ranking and data details should be verified again before publication.
Key papers by research question
1. Modeling reinforcement learning tradition
The common paradigms of World Models, PlaNet, and DreamerV3 are: compression observation, learning dynamics, predicting the future in the submersible space, and using prediction training strategies. The advantage is that the goals are clear, and the task return and sample efficiency can be seen; the limitation is that the environment is controlled, and the open World diversity is much lower than that of Internet video and real robots.
2. Generated interactive environment
Genie’s central component is a latent action model. Without ground-truth action labels, it must discover which changes in a video can be controlled. GameNGen works in a more constrained setting. It collects trajectories from an RL agent playing DOOM, then trains a diffusion model to predict the next frame. Genie 3 illustrates the ambition of a general-purpose world model. In research writing, distinguish what an official demo shows from what a technical report documents or an independent reproduction confirms.
3. Indicates the predicted route
V-JEPA 2 stands for JEPA's idea: predict abstract representation instead of each pixel, avoid spending computing power on irrelevant details. It connects large-scale video understanding, physical reasoning benchmark and the robot image-goal planning. The open problem is interpretability: when planning fails, it is difficult to judge whether it is a representation, an action condition, a search, or a controller error.
4. Synthesized Data and World Action Model
Cosmos, DreamGen, OSCAR, and Qwen-RobotWorld are all dealing with "How the World Model Becomes a Robot Research Tool". The key is not to generate good-looking videos, but to generate trusted Rollout, recoverable actions, and cross-EMBOdiment generalization, and bring benefits in real strategy training or evaluation.
Comparing the methods
| Route | Typical work | prediction object | working condition | Main use | Main risk |
|---|---|---|---|---|---|
| Modeling RL | World Models、PlaNet、DreamerV3 | Submersible state, REWARD, future observations | Simulator action | Efficient sample RL, submersible space planning | The open world generalizes the weak |
| Interactive video generation | Genie、GameNGen、Genie 3 | next frame or video latent variable | Latent action, user action, text | Interactive environment, agent training | Physics is not conserved, long-term drift |
| signify | V-JEPA 2 | Video Characterization, Future Embedding | Post training with a small number of robots | Video understanding, robot planning | Error is not intuitive |
| Physical AI Platform | Cosmos、Cosmos-Predict2.5 | Text/Image/Video-to-World | Text, Image, Video Conditions | synthetic data, policy evaluation | Data licensing, computing power, sim-to-real deviation |
| World action model | DreamGen、OSCAR、Qwen-RobotWorld | Future Video + Action Representation | Language, Skeleton, Waypoint, Robot State | Strategy training, cross-EMBODiment evaluation | Action authenticity, evaluation correlation |
How to interpret the experiments and benchmarks
Don't just watch video quality indicators. FVD, CLIPScore, and artificial preferences can indicate visual quality, but World Model should look at action response, object durability, post-occlusion recovery, causality intervention, physical constraints, long-term consistency, and downstream policy benefits.
The value of WorldModelBench is to examine the video generation model as a world model, and add the dimension of instruction compliance and physical compliance. DreamGen Bench attempts to verify the correlation between the quality of video generation and the success rate of robot policies. Datasets such as DROID, Open X-Embodiment, and RoboMIND are important foundations, but camera calibration, control frequency, hardware differences, contact dynamics and failure recovery will still cause deviations.
For closed source systems such as Genie 3, there should be three layers of evidence: official presentations, technical reports or papers, and third-party reproducible experiments. The official blog can explain the direction and ability boundary, but cannot directly replace the open benchmark.
Reproducibility and data problems
World Model's data problems are more complex than LLM. Video data involves copyright, privacy, location and platform licenses; robot data also involves hardware configuration, calibration parameters, controllers, remote operation quality and failure trajectory. It is not enough to write "use several hours of robot video" in the paper, and researchers also need to know the acquisition frequency, action space, camera arrangement, task distribution and filtering rules.
For graduate students, the more realistic reproduction route is not to retrain the general world model, but to choose a small environment or an open robot data set, a fixed action interface, and compare the differences in control gain between pixel prediction, representation prediction and state prediction.
Limitations and open questions
First, whether the video model really understands physics is still controversial, especially in contact, conservation, liquid, occlusion and multi-agent interaction. Second, the mechanism of robot strategy benefiting from synthetic video is still unclear: did you learn dynamics, or did you get visual augmentation and task priors? Third, the closed source demonstration is driving up expectations, but the community lacks the interactive Benchmark of uniform, open, and cherry-picking.
Research questions for graduate students
- Action Response Evaluation: Test whether different actions produce a differentiated and physically reasonable future under the same initial observation.
- Cover and object durability: Construct a long-term interactive scene to test whether the model remembers the object and state change from the field of view.
- World Model and VLA interface: Study how Rollout is a Verifier, Planner or Data Augmenter as a VLA policy.
- Synthetic trajectory credibility: Compare the effect of Latent Action, Inverse Dynamics, and Real Action Labels on the quality of Synthetic Trajectory.
- Reproduction of small computing power: Based on the DROID subset, RoboMIND subset or simple simulator, the system compares the control gains of different predicted targets.
Summary
The intersection of World Models and Embodied AI is to let AI not only describe the World, but also internally simulate "what will happen after action". Progress 2025-2026 shows that video generation, robot strategy, simulation platform and VLA control are converging. However, the open evaluation is insufficient, the closed source demonstration is difficult to reproduce, the physical consistency is difficult to quantify, and the synthetic data income is unstable. The next step is to put the World Model into a verifiable perception-predicting-planning-action closed-loop check.
References
Search date: 2026-07-13. The following will give priority to the papers, official project pages, arXiv and official instructions of the institution; the fast-changing model open status, benchmark ranking, data scale and code repository link, and it is recommended to manually verify before publishing.
- David Ha, Jürgen Schmidhuber. World Models. arXiv, 2018. https://arxiv.org/abs/1803.10122
- Danijar Hafner et al. Learning Latent Dynamics for Planning from Pixels. arXiv, 2018. https://arxiv.org/abs/1811.04551
- Danijar Hafner et al. Mastering Diverse Domains through World Models. arXiv, 2023. https://arxiv.org/abs/2301.04104
- Jake Bruce et al. Genie: Generative Interactive Environments. arXiv, 2024. https://arxiv.org/abs/2402.15391
- Dani Valevski et al. Diffusion Models Are Real-Time Game Engines. arXiv, 2024. https://arxiv.org/abs/2408.14837
- Google DeepMind. Genie 2: A large-scale foundation world model. Official blog, 2024. https://deepmind.google/blog/genie-2-a-large-scale-foundation-world-model/
- Google DeepMind. Genie 3: A new frontier for world models. Official blog, 2025. https://deepmind.google/blog/genie-3-a-new-frontier-for-world-models/
- Mido Assran et al. V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning. arXiv, 2025. https://arxiv.org/abs/2506.09985
- Meta AI. Introducing the V-JEPA 2 world model and new benchmarks for physical reasoning. Official blog, 2025. https://ai.meta.com/blog/v-jepa-2-world-model-benchmarks/
- NVIDIA et al. Cosmos World Foundation Model Platform for Physical AI. arXiv, 2025. https://arxiv.org/abs/2501.03575
- NVIDIA et al. GR00T N1: An Open Foundation Model for Generalist Humanoid Robots. arXiv, 2025. https://arxiv.org/abs/2503.14734
- Joel Jang et al. DreamGen: Unlocking Generalization in Robot Learning through Neural Trajectories. arXiv, 2025. https://arxiv.org/abs/2505.12705
- Dacheng Li et al. WorldModelBench: Judging Video Generation Models As World Models. arXiv, 2025. https://arxiv.org/abs/2502.20694
- Jiuming Liu et al. Towards Interactive Video World Modeling. arXiv, 2026. https://arxiv.org/abs/2606.01164
- Xiaoxiong Zhang et al. From World Models to World Action Models. arXiv, 2026. https://arxiv.org/abs/2607.00836
- Alex N. Wang et al. Lifting Embodied World Models for Planning and Control. arXiv, 2026. https://arxiv.org/abs/2604.26182
- Jie Zhang et al. Qwen-RobotWorld Technical Report. arXiv, 2026. https://arxiv.org/abs/2606.17030
- Zhuoyuan Wu, Jun Gao. OSCAR: Omni-Embodiment Skeleton-Conditioned World Action Model for Robotics. arXiv, 2026. https://arxiv.org/abs/2606.04463
- Google DeepMind. Gemini Robotics brings AI into the physical world. Official blog, 2025. https://deepmind.google/blog/gemini-robotics-brings-ai-into-the-physical-world/
- Alexander Khazatsky et al. DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset. arXiv, 2024. https://arxiv.org/abs/2403.12945
- Kun Wu et al. RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulation. arXiv, 2024. https://arxiv.org/abs/2412.13877