Relational Physics Tests for Video Models

Physical reasoning remains a meaningful challenge for video generative models, yet accurately measuring it has proven difficult. Most existing approaches rely on subjective plausibility judgments, trajectory matching against a single reference video, or absolute metric quantities such as mass, scale, and camera calibration — none of which transfers cleanly to AI-generated content. This has left a gap: even as video generators grow more visually convincing, there is no robust way to test whether they understand the physical laws governing the world they depict.

To address this, a team of researchers—Varun Varma Thozhiyoor, Shivam Tripathi, and Venkatesh Babu Radhakrishnan (from the Vision & AI Lab at the Indian Institute of Science), together with Anand Bhattad (from Johns Hopkins University)—introduced Principia, a physics benchmark that sidesteps the calibration problem by evaluating relational consistency between paired objects rather than the absolute motion of any single one. The key idea is that when two objects in the same scene obey the same physical law, their motions must satisfy predictable relationships that hold regardless of frame rate, camera intrinsics, or metric scale. For example, two blocks of different masses released simultaneously from the same inclined plane must arrive at the bottom at the same time, because gravitational acceleration on a ramp is mass-independent. A model that violates this invariant has failed at physics, independent of how visually convincing the individual trajectories appear.

The benchmark spans eight Newtonian phenomena—gravity, restitution, friction, rotational inertia, projectile motion, momentum, pendulum, and mass-spring oscillation—covering translational, rotational, collisional, and oscillatory dynamics. The real-world dataset comprises over 500 paired-object scenes collected under strict experimental controls. All object tracking is performed in pixel space using SAM3, so the relational consistency score depends only on ratios and equalities of pixel-space quantities, with no need to recover any physical parameters.

Beyond the real-world recordings, the team also built Principia-Synth, a synthetic testbed in Isaac Sim that generates both physically correct and deliberately anti-physics videos — enabling evaluation of vision-language models (VLMs) on their ability to detect relational violations, something real-world recordings alone cannot support.

Across thousands of generations from six state-of-the-art video generators, no model exceeds a consistency score of 0.42 on Principia while all score around 0.8 on VBench, a widely used visual quality benchmark. Visual quality and physical consistency appear largely independent. VLMs also perform poorly as physics detectors, with most performing near chance level on the relational violation task and even the best-performing model reaching only 67% accuracy.

This research work has been accepted at the Neural Information Processing Systems Conference (NeurIPS 2026).

Paper: https://arxiv.org/abs/2609.04200

Project page: PrincipiaBench.github.io

MENU