MBZUAI method teaches video models to predict object collisions

Key Points

1. MBZUAI researchers built KineMask, giving video generators sharper handling of physical collisions between objects
2. Current generative video models often make objects vanish, pass through each other or move illogically
3. The capability underpins world models that robots need to predict motion outcomes before acting

The latest:

A new method developed at Mohamed bin Zayed University of Artificial Intelligence in Abu Dhabi, called KineMask, lets video generation models work out what happens when objects hit each other. Instead of being handed a full motion path, the model receives a single image plus an initial direction and speed for one object, and must infer the collision itself.

Details:

• The problem: Today’s video generators produce convincing-looking clips but frequently break physics, according to the researchers. Objects disappear, drift in illogical directions, or pass through one another on impact. That failure becomes disqualifying if video models are to serve as world models for robotics and embodied AI, where predicting the outcome of a motion is decisive.
• How it differs: Earlier motion-control systems are drag-based: the user specifies the entire trajectory in advance. KineMask supplies only a starting direction and velocity for a chosen object and leaves the rest to the model, so the collision outcome is generated rather than prescribed.
• The architecture: The team adds a ControlNet-style control mechanism on top of a pretrained diffusion video model, CogVideoX-5B. Velocity of the moving object is encoded as a colored mask laid over the object’s pixels, with three channels carrying motion along the three spatial axes.
• Two-stage training: In the first stage the model sees the velocity mask across the whole video. In the second, that future information is progressively removed, forcing the model to predict motion from the launch point alone rather than reading it off the input.
• Simple-motion scores: On a simple motion benchmark, the full two-stage system reached an FVD of 160.3, compared with 288.8 when relying on the second stage only. FVD measures generated video quality, and lower scores are better.
• Collision scores: On a harder test involving object-to-object collisions, the base CogVideoX model scored 807.3. KineMask trained on simple motion cut that to 301.1, and a version trained directly on collision data reached 250.7.
• Human preference: In a survey of 30 participants comparing KineMask against CogVideoX, Wan, TORA, MotionI2V and Force Prompting on real-world scenes, participants preferred KineMask in 51% to 88% of cases on motion accuracy, 64% to 86% on interaction realism, and 59% to 77% on overall physical consistency.
• The authors: The paper was presented at ECCV 2026 by David Romero, Ariana Bermudez, Vyacheslav Yablochnikov, Hao Li, Fabio Pizzati and Ivan Laptev.

Background:

Generative video models are increasingly pitched as world models: systems that simulate how an environment will evolve so an agent can test an action before performing it. That use depends on physical plausibility rather than visual plausibility, which is where current models have fallen short.

Between the lines:

The gap between 807.3 and 250.7 on collision scenes points to how little of the physics was coming from the base model itself. The design choice is the tell: by withholding the trajectory and stripping future velocity information during the second training stage, the researchers test prediction rather than reproduction — the capability a robot would actually need.

What’s next

Watch whether the two-stage training approach is applied to larger video backbones beyond CogVideoX-5B, and whether collision-trained variants hold up on scenes involving multiple interacting objects.