How Does a Robot Learn to Catch a Ball?

When someone throws a ball at you, it is pretty easy to catch it. You don't calculate the trajectory of the ball, the movement of your body, and where your hand needs to be at the precise moment the ball reaches you. No child learns calculus before learning how to catch a ball.

We learn something much more useful: an intuitive understanding of the physical world.

A robot has to learn some version of that too. It needs to understand where objects are, how they move, what might happen if it takes an action, and how that action could change both the robot and the environment around it.

This is the basic idea behind a World Model (WM). A simple way to think about a World Model is that it tries to answer:

If I do this, what happens next?

Last week, I attended Actuate 26. World Models and the related problem of robotics training data came up repeatedly. 

Image created by an LLM :)

Large Language Models (LLMs) had one enormous advantage: the training data already existed. Trillions of words had been written before ChatGPT came along. Robotics does not have an equivalent dataset of physical actions and their consequences.

So how do robots get the data?

There seem to be five main ways:

1. Existing machines. Collect sensor and operating data from machines already doing useful work. This works particularly well for task-specific robots.

2. Simulation. Build virtual environments and let robots generate experience inside them.

3. Video. Train models using videos of humans interacting with the physical world.

4. Instrument humans. Equip people with cameras, sensors, gloves, or other devices and record them performing tasks.

5. Teleoperation. Have humans remotely control robots and record what the robot sees and what actions the human takes.

Startups are using one or more of these approaches.

There does not seem to be a consensus yet on which one wins. Each has different trade-offs around cost, compute, fidelity, scalability, and time.

Robotics has become a broad field. A useful way to think about it is as a spectrum. At one end are task-specific robots. An industrial robotic arm might perform the same operation thousands of times on a factory line. At the other end is the ambition behind general-purpose robots: one machine capable of learning and performing many different tasks.

In between is probably where much of the progress will happen.

• One robot.
• Several tasks.
• Then dozens of tasks within a particular environment.
• Then many environments.

And eventually, perhaps, something approaching a truly general-purpose robot. This is where Robot Foundation Models (RFMs) come in.

There are interesting parallels between LLMs and RFMs. Both are pretrained on large amounts of data. Both try to generalize beyond a single task. And both try to replace many specialized models with a more general model.

But RFMs have a much harder job. When an LLM makes a mistake, the error usually remains in the information domain. When a robot makes a mistake, the consequence can be physical.

So robotics is not only a data problem. It is also a reliability and safety problem. Words live inside computers. Robots live in the physical world. They have to deal with gravity, friction, contact, uncertainty, objects they have never seen before, and humans who tend not to stand exactly where the training data says they should.

There are a few terms worth separating.

A Robot Foundation Model (RFM) describes a broadly reusable robotics model.

A Vision-Language-Action model (VLA) takes inputs such as images and language and generates robot actions.

A World Model (WM) tries to predict how the physical world will change.

Since this is a rapidly evolving field, the definitions are evolving as well. For example, Nvidia researchers are increasingly using another term: World-Action Model (WAM). A WAM combines elements of a World Model and an action model (VLA). It tries to predict how the physical world will change while also determining what action the robot should take.

The terminology will probably keep changing as the technology evolves.

These technologies can work together, but they are solving somewhat different problems.

So where do we go from here?

My view is that we are still quite far from truly general-purpose robots.

Some problems can be solved with money: more compute, more data, and more people collecting it. Other problems require iteration: deploying robots, watching them fail, learning from those failures, and improving the technology over time.

This is why I think we will see many more task-specific robots before truly general-purpose ones. They solve a clearly defined customer problem, and the customer knows what that problem is worth.

The more likely path is not:

task-specific robot -> general-purpose robot

but:

one task -> several tasks -> many tasks in a constrained environment   -> many environments -> general purpose

The interesting question may therefore not be:

When will we have general-purpose robots?

It may be:

How quickly can we expand the number of tasks and environments a robot can handle economically?

---

See my notes from Actuate 24 here.