Video recording and physical intervention

Large-model research controversy: from the paper conclusion to the boundary of evidence

Abstract: The prediction is correct, what kind of evidence is still missing?

A model generates realistic dominoes video, and the direction of the fall is reasonable. It may learn how to spread the contact, or it may follow the sequel to the common picture, and it may also be a candidate, and then another model will pick out a more reasonable one. Just watching the final film, these explanations are difficult to distinguish.

This article discusses a narrow question: under what conditions can video prediction become the evidence that the model has removable mechanics? The conclusion is that the picture quality, the prediction accuracy under a given condition, and the stable response after the intervention need to be measured separately. Three studies provide different levels of diagnosis, and it is not enough to directly convert the name "world model" into a proof of causal understanding. The "understanding" here adopts a verifiable behavioral standard, does not discuss consciousness, and does not require the model to write the Newtonian equation.

From the sequel to the picture to the change of the world

Video prediction is to generate subsequent frames based on existing frames. The dynamic model describes how the state changes with time, action and environmental parameters. The "world model" can refer to these two types of models, or it can refer to models that only predict compression characterization; using the same name does not mean that they have accepted the same test.

Taking the small ball as a hypothetical example: the model sees the ball moving to the right, predicting that it will continue to the right, only that this time is correct. Change the ball from blue to red and keep the speed unchanged, and the prediction should be basically unchanged; change the speed to a larger and keep the color unchanged, and the displacement should change accordingly. The former checks the stability of the irrelevant appearance, and the latter checks the sensitivity to the relevant variables. Both tests are missing, and both may leave shortcuts to explain.

"Intervention" is to actively change a certain factor and try to keep other conditions consistent; "counterfactual" further asks what will happen if the same event adopts another intervention. Replacing Prompt to "heavier ball" does not automatically constitute a strict counterfact: the model may even change the material, background and initial speed. Only when the intervention object and the common initial state can be traced, the paired comparison has a clear meaning.

Another easily overlooked condition is observability. A still photo does not necessarily contain speed, and balls with the same appearance may have different densities. When entering missing information that determines the future, the only correct continuation may not exist at all. At this time, the distribution of reasonable results should be checked, or the model should be allowed to request more observations, and the only video in the guessing should not be regarded as a mastery of the law.

Three explanations, three different stress tests are required

The first explanation is that the model mainly relies on the statistical matching of appearance and common trajectories. This does not mean copying the training set frame by frame; interpolation and reorganization can also generate videos that have never been seen before. Its risk is that irrelevant changes in appearance may alter dynamic predictions.

The second explanation is that the model forms some kind of transferable state relationship. It does not have to explicitly save the variable names of quality and speed, but should be consistent in a set of pre-declared interventions, and can handle no parameter combinations. Passing the test will enhance this explanation, but still cannot uniquely determine the internal implementation.

The third explanation is that the system has the ability to generate reasonable candidates, but the default sampling and screening is not good enough. The performance increases after joining the search, which may indicate that the available candidates existed originally, and may also contain information provided by the external reward model. It is not mutually exclusive with the second explanation, and it must be clear whether the research object is a single generator or the entire system.

Controlled experiments tell us that generalization must be disassembled

The research of Kang et al. put video generation into a controlled two-dimensional physical environment, distinguishing new combinations within and outside the training range and already seen. The diffusion model is re-trained, but the video compressor uses a fixed pre-trained VAE; therefore, it cannot be called the entire system at all without pre-training. The prediction within the scope of the research report still has obvious errors outside the scope, and combined generalization can benefit from extended data. 1

The value of this design is that the physical parameters and test division can be controlled; its boundaries are also clear: it studies the specified architecture, data scope and tasks, not to prove that all video models can never be extrapolated. The "case-based" interpretation of the paper comes from behavior diagnosis, not directly reading the internal retrieval algorithm. 1

First ask exactly what makes a test example new. Saying that a particular video never appeared in training rules out only an exact duplicate. A new combination of familiar objects is different again from a speed outside the training range. Calling all three “unseen scenes” can overstate the generalization result. Equally, failure outside the training range does not erase useful relationships learned within it.

The real video is closer to the world, but the rating is still the agent

Physics-IQ uses real physical scenes to test subsequent videos, give a single frame model a frame, and a context for a model that supports multiple frames. It combines the spatial, temporal coincidence and pixel error of the moving area, and normalizes the difference between the two real shots. The paper pointed out that visual realisticness is not equivalent to these physical predictors. 2

This score is not "understanding how many percentages of physics" nor the pass rate of checking the laws of physics one by one. The author clearly acknowledges that the indicator is an agent, and the lens movement and mirror cutting are also punished. The amount of information of a single frame and multiple frames is different, so the total score of the model cannot be directly interpreted as the causal advantage of the architecture. 2

This suggests a way to read a paper: read the input information, the output target and the scorer together. If the model predicts a reasonable position but the time is misplaced, the spatial indicator may be good; if the model changes the lens without authorization, the pixel alignment will fail. The researcher needs to fail to dismantle in the state estimation, motion prediction or output presentation, and cannot let a comprehensive score bear all the explanations.

AT Inference Time Improvement, who has changed?

WMReward in 2026 uses the V-JEPA 2's characterization prediction signal to score or guide denoising for generation candidates. Best-of-N refers to generating multiple candidates and then selecting. Its PhysicsIQ experiment uses 16 candidates to search settings, and the score is improved under several conditions after adding this method; the generator does not need to be retrained, but the system increases the reward model and inference cost. 3

The paper also reports a trade-off between physical consistency and fidelity to the prompt in text-to-video generation. The authors acknowledge that their reward model still struggles with properties such as weight and friction. I used the accessible v2 preprint. A challenge result does not establish conference acceptance, and the normalized scores in the paper do not measure the percentage of physical laws the videos obey. 3

The most useful inference here is not that "physical knowledge has been completely hidden in the generator", but that the default output cannot fully represent the reachable performance of the system after searching. If the chip selector brings an independent prior, the improvement should be attributed to the combination system. If you only observe the last video, you cannot distinguish the three contributions of the generator better understand physics, more sampling or the referee will recognize the error.

source of evidence controlled object Supported Observation can not be proved
PhyWorld research[1] Synthesis parameters, training distribution Generalization type affects results The upper limit of the extrapolation of all models
Physics-IQ[2] Continuation and rating of real video Realistic and predictive performance need to be measured separately Scores directly represent knowledge of causality
WMReward[3] Rewards and Reasoning Strategies System output can be improved by search The generator masters all the rules alone

The starting point, training data, reasoning budget and indicators of the three articles are different. The controlled speed error cannot be ranked on the same list as the comprehensive score of the real video; the search system cannot be compared with a single sampling and mixing cost. They are complementary diagnosis, not two factions that overturn each other.

The next step in the interpretation

Observe the different processes of forecasting and intervention testing

Figure: The original test process is shown, the prediction after the upper row inspection is observed, and the lower row inspection is reset and the response after the intervention. The arrows only indicate the evaluation sequence; the domino is a concept icon, not a simulated data. The two candidates on the lower right represent the candidates to be compared, and do not claim that any paper has passed these interventions.

Below is a research design that has not yet been performed. The goal is not to make another "look better" display, but to create a competitive interpretation on the same model gives different prediction conditions.

First, establish a pair of intervention sets. Save images and parameters in a simple environment with a recordable state. Color and light are used as appearance factors, and the initial velocity and friction are used as dynamic factors, and the independent changes are changed and cross-combined. Training and testing are separated according to the parameter area and combination to prevent the same trajectory with only changing the color and crossing the division. If the color change brings a large trajectory change, but the frictional friction is almost unresponsive, the statistical shortcut explanation will be supported; if the response direction and amplitude are reasonable, the evidence for the transfer of relationship can be accumulated.

Second, the fixed candidate pool comparison selector. All random seeds and candidates are saved for the same initial conditions, and randomly selected, external rewards and environmental truth are scored respectively. True value selection is only used as the diagnostic upper limit, and cannot be disguised as a deployment result. If the true value can pick a good candidate and the reward model cannot, the bottleneck is closer to the choice; if there is no good candidate in the pool, it is necessary to expand the budget or change the generator, and the probability cannot be asserted only by limited failure to assert that its probability is zero.

Third, do the after-reset action fork. Let the system predict the follow-up of "removing the barrier" and "reserving the barrier" from the same state, and then compare it with the actual execution of the environment. The control interface must record the timing of the action, the action object and the execution result. The test that only changes the text description and cannot confirm whether the environment is executed, and cannot answer the action cause and effect problem.

These tests should also pre-agreed the error caliber, test time and failure processing: the object is drawn, the parsing fails, and the refusal is generated must be counted separately and cannot be discarded quietly; Multi-frame input and single-frame input are reported separately, and the number of sampling times is saved with the total calculation budget, to avoid misjudging the measurement error, more information or more calculations as an improved understanding.

The current conclusion and still open issues

Existing evidence is sufficient to oppose a strong inference: the generation of realistic videos has already obtained a universal model of causality. Evidence also does not support the opposite absolute judgment: video prediction can only be remembered, and neither scale nor reasoning strategy can be useful. A more secure conclusion is to bind capabilities to specific states, scope of interventions, information conditions and budgets.

What is really worth asking is not whether the name is worthy of the "world", but the layer of failure. Whether it can identify the status, whether it can respond to the intervention, whether it can be consistent for a long time, and whether the information is insufficient is different. If these problems are disassembled, the graduate student may propose an explanation that can be denied by the experiment, rather than selecting edges between the successful example and the failed clip.

Summary

Video prediction provides a training signal for learning dynamics, and also provides a window to observe model behavior; it itself does not automatically complete causal verification. When reading a world model paper, first confirm what it has changed, what is fixed, and what is measured, and then judge whether the evidence supports a reasonable picture, an accurate prediction or a transferable intervention response. This article does not claim to fully cover the research as of the search date, and did not perform model reproduction.

References

Data search and deadline: 2026-10-31. Nearly 12 months were searched for supplementing new evidence, while retaining early studies needed to explain the problem.

  1. Bingyi Kang、Yang Yue、Rui Lu、Zhijie Lin、Yang Zhao、Kaixin Wang、Gao Huang、Jiashi Feng. How Far Is Video Generation from World Model: A Physical Law Perspective. ICML 2025; Official publication page, the method and results verification of this paper are used arXiv V2 full text(2025-06-22).
  2. Saman Motamed、Laura Culp、Kevin Swersky、Priyank Jaini、Robert Geirhos. Do generative video models understand physical principles?. 2025, arXiv V3 full text(2025-02-27). Retrieve the clues of the official page of WACV 2026, but the page and PDF access are limited, and the experimental description in this paper is strictly limited to the read v3.
  3. Jianhao Yuan、Xiaofeng Zhang、Felix Friedrich、Nicolas Beltran-Velez、Melissa Hall、Reyhane Askari-Hemmat、Xiaochuang Han、Nicolas Ballas、Michal Drozdzal、Adriana Romero-Soriano. Inference-time Physics Alignment of Video Generative Models with Latent World Models. 2026, arXiv V2 full text(2026-02-27), according to the pre-printed edition, the meeting for the meeting is not confirmed.

Apps by the author

The following is the author's own product promotion. In addition to writing a technical blog, I also independently developed two daily gadgets, welcome to understand according to your needs:

  • Daily Expense Journal: Record the daily income and expenditure, arrange the budget and savings target, so that each account can be traced. View on the App Store.
  • One Small Action: Choose small actions according to the current time and energy, record the feelings after completion, and leave a daily review. View on the App Store.

More technical articles and personal works, welcome to visit Author personal website. These works have no endorsing relationship with the papers described in this article.