Embodied Reasoning The 1 Skill Robots Have Been Missing

Embodied Reasoning Is Manufactured, Not Discovered

By Bobby Carlton

Embodied reasoning is the term you’re going to hear a lot over the next year, so let’s get ahead of it in plain language. It’s the ability to understand how physical work actually happens. Not recognizing a wrench in a photo. Understanding what a hand does with the wrench. Where the grip lands, when contact starts, how the tool moves through the turn, when it lets go, and what order all of it happens in.

Humans get this for free. We’ve been watching hands do work since before we could talk, and by the time we take our first job we’ve logged tens of thousands of hours of physical observation without ever thinking of it as training. Robots get none of that for free. And that gap is the quiet reason the smartest AI models on the planet still fumble tasks a first-week warehouse temp handles without thinking.

Here’s the root of it. A language model got to read the entire internet. Trillions of words, sitting there, already written. Robots have no equivalent. There is no internet of physical skill. Nobody ever wrote down the ten thousand tiny motions inside “assemble the bracket,” because no human ever needed them written down. The knowledge lives in hands, and hands don’t publish.

So the robotics industry is doing something genuinely strange when you step back and look at it. It’s building that missing internet by hand, one captured hour of human work at a time. And this month gave us the clearest look yet at where the whole effort is headed.

The Transitions Are Where the Skill Lives

Perceptron AI just launched a tool called Egocentric, the first in what they’re calling a series of embodied reasoning offerings. What it does sounds narrow and is actually a big deal. You feed it raw first-person video, the view from a camera on someone’s head or chest while they work. It hands back structure: a 21-point skeleton of each hand tracked through every single frame, persistent identity for which hand is which, the exact moments contact begins and the object releases, and clean boundaries around where each sub-task starts and stops.

The reason continuous tracking matters is simple once you see it, and it’s the single most important idea in this post. Manipulation doesn’t live in the snapshots. It lives in the transitions between them.

Picture how most video analysis works today. A system samples frames, maybe one per second, and asks a vision model what it sees. Frame one: a hand near a connector. Frame two: a hand holding a connector. Looks fine on paper. But everything that constitutes the actual skill happened between those two frames. The approach angle. The fingers spreading to match the part. The grip choice, two fingers or four. The moment of contact and the tiny correction right after, when the part shifted and the hand adjusted without the person even noticing they did it. Sample your way through a video and you’ve recorded that a task happened while missing everything about how. Train a robot on that and you get a machine that nails the demo and flubs the job.

The benchmark results back the idea up. On WGO (What’s Going On)-Bench, the standard test for this kind of video understanding, Perceptron reports a 0.280 end-to-end score where the best pipelines built on Google’s Gemini robotics models scored 0.158. That’s a 77 percent jump, and in benchmark terms that’s not a tweak winning. That’s a different way of looking at video paying off.

The Economics Underneath All of This

Now the part that explains why this is an industry story and not just a product launch. The bottleneck in robot learning was never recording demonstrations. Cameras are cheap and everywhere. The bottleneck is annotation, turning raw footage into labeled structure a model can learn from. Every grasp, hold, move, and release has to be marked, and until now that meant human annotators at roughly $50 an hour working through footage frame by frame. That price is why labeled physical-interaction data has been the scarcest resource in robotics. Perceptron claims its system does the work 10 to 15 times cheaper. If that holds in practice, mountains of footage that were too expensive to process just became usable.

And the mountains are real. The industry has been quietly stockpiling first-person work video for years. Meta’s Ego4D dataset kicked the field off with about 3,700 hours of daily-life footage from hundreds of participants around the world. Apple captured over 800 hours through Vision Pro with every finger joint tracked at recording time, across nearly 200 tabletop tasks like folding laundry and tying shoelaces. On the industrial side, one open-source release last year put out 100,000 hours of footage from factory workers across real production sites, over a billion frames of hands doing actual manufacturing work. One founder in the space calls the goal “the internet for physical AI,” and the biggest collections are now pushing toward a million hours.

There’s even early evidence this scales the way language did. NVIDIA research found a log-linear scaling law for egocentric data: every doubling of human first-person video hours produces a predictable gain in downstream robot task success. If that pattern holds, it’s the same drumbeat that carried language models from autocomplete to writing code. More data, predictably better results, repeat.

Stockpiles of raw video plus tooling that can finally structure it cheaply. That’s the moment we just entered.

The Full Supply Chain of Physical Skill

Zoom out and the picture organizing all of this comes into focus. Embodied reasoning gets manufactured through three channels, and serious robotics programs are now running all three at once.

Teleoperation is the ground truth. A trained operator drives the actual robot through the actual task, and every joint angle and gripper force gets recorded. It’s the most direct data there is, because it’s captured on the exact body the model will control. It’s also the most expensive per hour, which is why quality matters so much. Egocentric video is human skill at scale. It’s cheaper to capture and there’s vastly more of it, but until recently it was locked footage, too costly to label. That’s the lock that’s now breaking. Simulation is the multiplier. Take demonstrations from either source, rebuild the task virtually, and generate thousands of variations. Different lighting, different part positions, the dropped-part edge case that happens once a month on a real line but can happen ten thousand times tonight in sim.

None of the three is sufficient alone. Teleop is precise but scarce. Egocentric is abundant but comes from human bodies, not robot ones. Synthetic is unlimited but needs real seeds to stay honest. The programs that win are the ones that run the full chain and, more importantly, run it with quality control at every step, because a bad label or a sloppy demonstration doesn’t just waste an hour. It teaches the robot something wrong.

FS Studio and Teleoperation Data

That supply chain is where we live, and not in theory. FS Studio runs teleoperation data collection at production scale right now for one of the most recognizable technology companies in the world, inside their own robotics program, staffing trained operators on humanoid platforms daily and running the pipeline that turns thousands of raw demonstrations into clean, model-ready training data. We’ve learned the hard lessons about operator skill, capture quality, and what separates data a model learns from and data that quietly poisons a training run.

The launches making headlines this month validate what we see on the floor every day: the race in robotics isn’t just about better robots. It’s about who can manufacture better understanding of physical work, at scale, without letting the quality slip.

If you’re watching the humanoid wave and wondering what will separate the robots that hold a job from the ones that just look good on camera, this is the answer. It comes down to what they were fed. Embodied reasoning is built frame by frame, from data somebody had to capture, structure, and multiply.

The robots are getting the bodies. Somebody has to give them the hands. If your team is building in this space and the data layer is the thing slowing you down, that’s exactly the conversation we want to have. Reach out.