Brain Waves May Teach Physical AI What Video Misses

If robots need human judgment as much as human motion, should brain waves physical AI data become part of the training stack?

That question is now live inside Encord’s warehouse in San Leandro, California, where a robotic trainer named Andrew Ceja pulls blocks from a Jenga tower while wearing a headset that tracks what he sees and measures his brain waves, according to TechCrunch. The trial, run with German neuroscience startup Zander Labs, is testing whether neural signals can add a missing layer to robot training data: intent, effort, surprise, and error recognition that cameras may not capture.

This is the sharp edge of the physical AI problem. Chatbots had the internet. Robots don’t have an internet-scale archive of humans gripping slippery mugs, plugging ethernet cables into servers, stacking poker chips, or deciding when a Jenga tower is about to fall.

“The data simply does not exist,” said Vineeth Velmurugan, Encord’s head of robot learning.

Why could brain waves physical AI data change how robots learn physical tasks?

The core problem is simple: video shows what happened. Robots need to learn why a human chose one action over another.

A single clip of a person pulling a Jenga block can show the hand, the block, and the collapse. It may not show the moment the person sensed instability, hesitated, changed grip, or realized the move was wrong. Those invisible judgments matter for physical AI, especially when the task involves manipulation rather than recognition.

Encord is betting that training models for humanoid and warehouse robotics will be constrained less by model architecture and more by scarce real-world data. That’s why the company is moving from managing training data to manufacturing it.

The brain wave layer is not a mind-reading system. In the TechCrunch account, Zander Labs is measuring brain activity to infer states such as error, intent, and surprise. Lucas Gehrke, a Zander neuroscientist supervising the work, said brain activity during a task can help model builders understand when they may need to deploy their highest-effort models.

XOOMAR analysis: that makes brain waves most useful as a timing signal, not a replacement for cameras or robot sensors. If the human trainer’s neural data suggests “something is wrong” before a visible failure, the model may get a richer label than video alone can provide.

What data do frontier physical AI models need beyond YouTube-style video?

The physical AI data gap starts with fidelity.

General video can teach a model what the world looks like. It is weaker at teaching how the world behaves under contact: how hard to grip, when to slow down, how a wrist rotates, whether a cable is aligned with a port, or whether a cup is about to slosh.

Encord’s San Leandro facility shows what richer data collection looks like. Pilots use leader-follower rigs, paired robotic arms where a human directly controls one arm and another mimics the movement. TechCrunch saw trainers creating data for tasks including pouring coffee from a pot into mugs and stacking poker chips.

The company also collects egocentric video, footage from workers wearing cameras, from factories around the globe. In San Leandro, it experiments with other modalities, including brain waves and forearm sensors that detect electrical signals in muscles.

Dense annotation is another piece. Encord labels physical actions with descriptions such as:

“right hand tightens bolt”

Velmurugan estimates this kind of dense annotation is worth 100 times as much as “junky ego data” for training specific tasks, while costing 20 times more to produce. That ratio is the business case, at least on paper.

That cost problem is why physical AI can’t simply copy the LLM playbook. Frontier labs scraped huge text corpora cheaply. Physical training data has to be staged, captured, labeled, and checked.

For more context on how physical AI is pulling attention toward industrial use cases, see XOOMAR’s coverage of Kalanick’s Atoms physical AI push.

How would brain waves become training data for robots and embodied AI systems?

The workflow is not magic. A human performs a task while wearing a neural sensing headset. The system time-syncs that brain signal with video, task labels, and other sensor data.

In Encord’s trial, Ceja wears a headset with a camera that tracks what he sees and sensors that measure his brain waves while he plays Jenga. Encord’s goal is to build an initial brain wave-tagged dataset, run it through customer robotics models, and evaluate whether it improves performance before deciding whether to scale.

That last part matters. This is a trial, not proof.

Brain wave data may help identify moments when the human trainer detects an error or changes strategy. If a model sees the hand movement and also gets a time-aligned signal that the human registered surprise or error, it may learn the task with a more precise supervisory signal.

But the brain wave layer only works if it improves model performance enough to justify extra collection complexity. Encord already has expensive data pipelines: multiple tasks, human pilots, robotic rigs, annotation, and experimental sensors. Brain waves add another layer that has to earn its place.

Velmurugan called this the “bleeding edge” of solving the robotics data bottleneck. That’s the right framing. The technology is early, and the useful question is not whether neural data sounds futuristic. It’s whether it beats cheaper signals.

What does a brain-guided training session look like inside Encord’s warehouse?

The most concrete example is Ceja’s Jenga task.

He pulls wooden blocks from a tottering tower while the headset records his viewpoint and brain waves. The task is useful because it requires fine manipulation, sequencing, and judgment under instability. The system can capture what he does, what he sees, and potentially when his brain activity reflects effort or an error signal.

Nearby, Encord pilots run other manipulation tasks. Sofia Infante operates robotic arms to plug and unplug ethernet cables from the back of a server, a task TechCrunch describes as attractive to data center operators if robots can reach the needed precision. The limitation is blunt: pincers are less dextrous than human fingers and lack the degrees of freedom people take for granted.

Storage racks at the facility held fake flowers in vases, books, plastic vegetables, kitty litter trays and scoops, bags, and bundles of wires. These are not random props. They are the materials of household and industrial manipulation training.

“Every humanoid company has asked us for these pieces,” Velmurugan said.

XOOMAR analysis: the warehouse looks less like a robot demo lab and more like a factory for edge cases. Coffee sloshes. Poker chips slide. Cables resist alignment. Jenga towers fall. Each failure can become training data if it is captured with enough context.

That is also why plain video is not enough, even in a world dominated by short-form clips. Our coverage of vertical video’s grip on phone viewing shows how much visual data people produce, but robotics data needs more than a camera pointed at the world.

Which questions around neural training data are still unanswered?

The TechCrunch report does not say how Encord or Zander Labs will handle neural data governance if the trial expands. That leaves decision-makers with practical questions before brain waves physical AI datasets move beyond experimentation.

Purpose: Will brain wave readings be used only to improve task labeling, or could they be reused for other model development?

Access: Who can inspect raw neural recordings, Encord, Zander, robotics customers, or only a restricted internal team?

Retention: How long is the data kept after the model evaluation is complete?

Deletion: Can a pilot ask for brain wave-linked recordings to be removed from future training sets?

Proof: What performance gain would justify collecting this data at all?

Those questions are not side issues. They determine whether brain wave-tagged datasets become a credible tool or a trust problem. The source supports the existence of a trial, not a settled operating model.

Will brain waves become a breakthrough for physical AI or stay a specialist signal?

Brain wave data is promising, but it will not replace cameras, robot telemetry, forearm sensors, simulation, or human demonstrations. Its best role is narrower: an extra supervisory layer for moments when the human’s judgment is hard to see.

That points to high-precision manipulation first. Encord’s own examples fit that pattern: Jenga, ethernet cables, pouring coffee, stacking poker chips, and fine-tuning datasets around specific skills.

The bigger constraint is economics. Velmurugan says physical AI may need a dataset something like five times the size of YouTube’s video corpus to break through. If that estimate is directionally right, the winners will be the teams that can manufacture high-value data without drowning in cost.

The watch item is Encord’s trial result. If brain wave-tagged data measurably improves customer robotics models, it becomes a serious input for frontier physical AI. If not, it remains a fascinating signal that loses to cheaper cameras, labels, and robot-operated data.

Either way, the lesson is already clear: the next phase of physical AI won’t be trained only on visible motion. It will depend on capturing the hidden parts of human expertise, then proving those signals are worth paying for.

Why It Matters

• Robots lack the internet-scale training data that helped chatbots improve quickly.
• Brain wave signals could help physical AI learn the human judgment behind difficult manipulation tasks.
• If successful, this approach could reshape how warehouse and humanoid robots are trained.