Binding vision to physics based simulation: The case study of a bouncing ball

Brief description

A dynamic scene and, therefore, its visual observations are invariably determined by the laws of physics. We demonstrate an illustrative case where physical explanation, as a vision prior, is not a commodity but a necessity. By considering the problem of ball motion estimation we show how physics-based simulation in conjunction with visual processes can lead to the reduction of the visual input required to infer physical attributes of the observed world. Even further, we show that the proposed methodology manages to reveal certain physical attributes that are impossible to extract by other means. A series of synthetic experiments as well as experiments with image sequences of an actual ball support the validity of the proposed approach. The use of generic tools and the top-down nature of the proposed approach make it general enough to be a likely candidate for handling even more complex problems in larger contexts.

Single view estimation of 3D trajectories

Estimation of the 3D trajectory of a ball from single camera 2D observations and the assumption of a given physical world.

Seeing through walls with a single camera

Single view estimation of the 3D trajectory of a ball from partial 2D observations and the assumption of a given physical world. Left: the actual view showing 2D observations and back-projections of the estimated and the ground truth 3D ball trajectories, Right: 3D plot of estimated and ground truth 3D trajectory.

Inferring angular velocity

A ball is thrown with a high back-spin towards a table. By incorporating physics based simulation, we infer the ball’s 3D trajectory and its linear and angular velocities from a single camera (camera 7). The proposed method identifies that a back-spin is the cause of the reduction of the outgoing angle of the bounce. The green ellipses are projections of an equator of the ball and the arrows represent the direction of the estimated angular speed.


Relevant publications

The electronic versions of the above publications can be downloaded from my publications page.