How Does a Robot Learn to Catch a Ball?
When someone throws a ball at you, it is pretty easy to catch it. You don't calculate the trajectory of the ball, the movement of your body, and where your hand needs to be at the precise moment the ball reaches you. No child learns calculus before learning how to catch a ball. We learn something much more useful: an intuitive understanding of the physical world. A robot has to learn some version of that too. It needs to understand where objects are, how they move, what might happen if it takes an action, and how that action could change both the robot and the environment around it. This is the basic idea behind a World Model (WM). A simple way to think about a World Model is that it tries to answer: If I do this, what happens next? Last week, I attended Actuate 26 . World Models and the related problem of robotics training data came up repeatedly. Image created by an LLM :) Large Language Models (LLMs) had one enormous advantage: the training data already existed. Trillions ...