Teleoperation May Really Be in Danger
In May this year, Jim Fan, NVIDIA’s head of robotics, declared during a public talk that VLA and teleoperation were dead.
At the time, most people’s first reaction was: the AI community, never short of hot takes, has come up with yet another one.
But unexpectedly, just a few months later, that statement—which sounded somewhat exaggerated at the time—has actually begun to become reality.
Just recently, X2Robot released its new dexterous manipulation system, TwinDEX. With zero real-robot teleoperation data during the post-training stage, the robot successfully performed a range of delicate operations in chemical laboratory scenarios, including unscrewing bottle caps and injecting with a syringe.
Previously, teaching robots such delicate operations really did require real-robot teleoperation data to provide “hands-on” instruction.
But TwinDEX has eliminated that data requirement entirely, replacing it with just a few hundred pieces of embodiment-free data, containing virtually no robot-specific information.
How did it do this?
Teleoperation May Really Be in Danger
As the saying goes, a good demo deserves careful examination.
Among the demos released for TwinDEX, the most complete is a continuous-shot chemical experiment:
From opening bottles and taking samples, to operating droppers and test tubes, directing liquid with a glass rod, and shaking and observing samples, the entire process is executed autonomously by the policy.
The experiment contains 24 subtasks, spans three types of tools, and involves multiple instances of bimanual coordination and tool switching. Every step requires millimeter-level positioning and stable force control.
Looking more closely, TwinDEX’s three fingers demonstrate several capabilities that have traditionally been among the hardest challenges in dexterous manipulation.
The first is fine manipulation, which demands extremely high precision.
When opening the toolbox, the robot must align its left and right index fingers with two narrow latches, insert them accurately, push them downward, and then pinch the handle with its index finger and thumb to open the lid.
The difficulty lies in whether the robot’s fingers can accurately enter a space with very tight tolerances and complete the operation.
Syringe injection is a similar task.
The index and middle fingers stabilize the barrel, while the thumb aligns with the plunger and applies forward force. The robot must keep the syringe from shifting while precisely controlling the direction and force of injection.
As embodied intelligence gradually enters laboratories, factories, and homes, these small-tolerance, high-precision, high-contact operations will increasingly become basic skills for robots.
The second capability is three-finger coordination.
For example, in the broom-and-dustpan task, the three fingers envelop the handle, allowing different fingers to share the load of supporting and controlling it.
Compared with using only two fingers, the third finger provides an additional contact point and another way to stabilize an object.
This extra support is particularly important in tasks involving tool use and grasping long, narrow objects.
The third category is compliant manipulation and in-hand adjustment, which more closely resembles how a human hand operates.
When unscrewing a bottle cap, TwinDEX pinches the cap with its index finger and thumb, then rotates it primarily through the lateral movement of the fingers themselves, requiring almost no large wrist rotation.
When turning pages, meanwhile, the robot first rubs the top page out with its thumb, then pinches it, transfers it between both hands, places it, and turns it over.
After watching all of this, my most immediate impression was:
When dealing with these kinds of actions in the past, people would often instinctively think of structurally more complex five-finger dexterous hands, along with massive amounts of real-robot teleoperation data.
But the demos above point to something important: many delicate operations previously thought to require five fingers can actually be completed with just three.
To some extent, these three fingers are already more capable than many five-finger dexterous hands.
However, everything discussed above is only one side of TwinDEX—the side onstage as the robot’s end effector.
Before the robot actually gets to work, TwinDEX has another identity: a wearable, embodiment-free data collection system designed to match the execution side.
Unlike many solutions that use two different sets of hardware for data collection and robot execution, TwinDEX directly adopts a homologous design for collection and execution.
Put simply, these three fingers are not only responsible for doing the work; the same structure is also used to collect the data. This is what might be called “what you collect is what you get.”
Actions performed by the operator on the collection side can be mapped more directly to the robot’s execution side. There is no need to transfer the data across an entirely different hardware configuration or perform complicated data migration.
That is precisely why the delicate operations described above, which place extremely high demands on fingertip position and force, can be executed more reliably by the robot.
At the same time, compared with directly teleoperating a cumbersome robot, operators can perform actions more naturally and quickly. The data is also easier and less expensive to collect.
The experimental results verified both of these points.
In terms of collection efficiency, TwinDEX can produce approximately 5.3 times as many valid trajectories per unit of time as traditional real-robot teleoperation.
Given the same operator and the same amount of time, TwinDEX can accumulate a usable training dataset much more quickly.
As for data utilization efficiency, X2Robot found that:
As the amount of data increases, policies trained on embodiment-free data and real-robot teleoperation data continue to improve at the same rate, eventually converging to nearly identical performance.
In other words, in this set of experiments, embodiment-free data could almost completely replace real-robot teleoperation data during model training.
So, as a data collection tool, TwinDEX solves more than just the problem of “how to collect data faster.”
More importantly, while improving collection efficiency, it also minimizes the loss of precision when the data is ultimately transferred to the robot.
And both of these points lead to the same underlying question:
How can embodiment-free data genuinely become data that robots can use?
TwinDEX: Aligning Embodied Data in Advance
Strictly speaking, TwinDEX is not merely a three-finger dexterous hand.
It is more like a dexterous manipulation system composed of data collection hardware, a robot end effector, a data processing pipeline, and a model training workflow.
Nor can data migration happen naturally simply because the data collection and robot execution hardware look the same.
What truly matters about TwinDEX is that it starts with the goal of completing post-training using only embodiment-free data and eliminating real-robot teleoperation data, then designs the entire system around that objective.
From the configuration of the dexterous hand and the way the collection device moves, to visual observations, time synchronization, data processing, and finally model training, everything revolves around the same question:
How can the collected data be made as close as possible, from the very beginning, to the data the robot actually needs?
With this in mind, let’s look at TwinDEX’s most visible hardware components.
Judging from the division of functions in the demos, the thumb and index finger handle much of the opposition, pinching, twisting, and fine manipulation, while the third finger provides additional enveloping, support, and stability.
The overall design of TwinDEX can be summarized through three characteristics: dexterity, consistency, and scalability.
The first is the dexterity discussed repeatedly above.
TwinDEX did not choose a common two-finger gripper such as UMI, nor did it simply replicate the human hand all the way up to five fingers. Instead, it strikes a balance between dexterity, stability, and engineering complexity.
As everyone knows, the more fingers there are and the more closely the structure resembles a human hand, the higher the theoretical upper limit for dexterous manipulation.
But this also means more joints, actuators, and sensors, along with the calibration, control, and maintenance costs that come with them.
Conversely, although two-finger grippers are simple enough, they struggle to cover delicate operations such as twisting, in-hand adjustment, and multi-point contact.
Therefore, after testing elementary operations including basic grasping, stationary twisting, tool use, and in-hand manipulation, and comparing multiple candidate configurations, X2Robot ultimately found that:
Three fingers with nine degrees of freedom are currently the sweet spot. They can cover most of the dexterous capabilities genuinely required by current tasks while keeping complexity relatively manageable.
The second—and the most important design principle behind TwinDEX—is consistency.
Here, consistency means that TwinDEX’s data collection and final execution systems remain as consistent as possible in terms of movement patterns, contact patterns, visual observations, collection precision, and time synchronization.
This is also the key to TwinDEX’s ability to reduce the need for real-robot teleoperation data.
In the past, much embodiment-free data was easy to collect, but transferring it to a robot often revealed a problem: the collection and execution systems were not the same embodiment.
The way a human hand moves and the way a collection device records that movement cannot necessarily be reproduced exactly by the actual robot. Kinematic differences and precision losses typically arise along the way.
As a result, although such data is easy to scale, it is difficult to use directly to train the target robot. In many cases, another batch of real-robot teleoperation data is still needed for embodiment alignment and post-training.
TwinDEX’s consistency-focused design is intended to solve precisely this problem.
Rather than waiting until data collection is complete and then finding a way to adapt the data to the robot, it makes the collection and execution systems as homologous as possible at the hardware-design stage.
In other words, it moves the “embodiment alignment” that previously occurred during data processing to a point before the data is even generated.
This minimizes the loss incurred when embodiment-free data is transferred to the robot, thereby reducing the need for additional real-robot teleoperation data.
That is why, for the tasks covered in this experiment, TwinDEX could complete post-training directly with a few hundred pieces of embodiment-free data, without adding any real-robot teleoperation demonstrations from the target robot, and achieve the delicate operations described above.
The third characteristic is scalability.
TwinDEX’s data collection side is a wearable three-finger exoskeleton.
Its greatest advantage is that data collection does not require occupying a real robot or taking place at a fixed workstation.
Once operators put on the device, they can directly perform actions such as twisting, pressing, and using tools.
Compared with teleoperating a robot through a controller, this approach is also closer to natural human operation: the hand can directly contact objects and receive genuine force feedback, without needing to adapt to spatial mapping or communication delays.
More importantly, data collection is no longer tied one-to-one to the number of robots. Multiple operators can collect data simultaneously in different locations, then consolidate it into the same processing and training pipeline.
During collection, the system synchronously records multimodal information such as visual observations, joint states, and wrist poses. Through calibration, time synchronization, and standardization, data from different operators and locations is organized into a unified training space.
However, once the scale expands, new problems inevitably emerge.
Different devices and operators will inevitably introduce jitter, drift, and positioning errors during collection.
X2Robot therefore also made corresponding adjustments to the model architecture and training process, enabling the model to tolerate a certain range of collection errors and ultimately learn policies that the robot can execute in a closed loop.
At this point, TwinDEX’s three design principles become clear:
Dexterity determines what it can collect and what it can do; consistency determines whether the collected embodiment-free data can be transferred smoothly to the robot; and scalability determines whether the collection approach can truly scale.
The Tip of the Embodied Data Pyramid Is Being Flattened
The real significance of TwinDEX, then, is not merely that it “collects data faster.” It further demonstrates something important: the highest-quality data, most closely aligned with robot actions, does not necessarily have to be produced by real robots.
Over the past two years, real-robot teleoperation data has stood at the top of the embodied-data pyramid.
Its quality is the highest because visual observations, joint states, and end-effector trajectories are naturally aligned. But the cost is also the greatest—
Every data sample requires occupying a real robot.
At the same time, robots must be deployed and maintained, and operators must learn how to teleoperate them. Once the number of robots becomes the upper limit on data production, true scaling becomes extremely difficult.
This is part of the context behind Jim Fan’s claim that teleoperation is dead.
For this reason, the entire embodied-AI community has spent the past two years exploring new data collection strategies, including UMI, gloves/exoskeletons, and egocentric first-person-view approaches.
Their central idea is the same: use more scalable embodiment-free data to reduce reliance on real-robot data.
But this has always presented a dilemma:
The freer the collection method, the easier the data is to scale. Yet the farther the collection side is from the real robot, the more difficult the subsequent mapping and transfer become.
In particular, differences between human hands and robots in structure, degrees of freedom, and contact patterns continually introduce motion-retargeting challenges and precision losses.
What TwinDEX has done is find a new sweet spot between scalability and the richness of the physical information preserved in the data.
It does not completely discard the constraints imposed by the robot embodiment. Instead, it “copies” the action structures the robot truly needs onto the data collection side in advance.
As a result, for the tasks already demonstrated, just a few hundred pieces of embodiment-free data were enough to perform part of the alignment work that would traditionally have required real-robot teleoperation data during post-training.
Of course, objectively speaking, embodied-data recipes are still far from converged. Given the need for broader task coverage and greater data diversity, we cannot yet simply declare that “teleoperation is dead.”
But at least one point is becoming increasingly clear:
The strong link between high-quality data and real-robot teleoperation is beginning to loosen.
If embodiment-free data can gradually approach the training value of real-robot data during post-training, the next question worth exploring is a Scaling Law for embodiment-free data.
But even before scaling up embodiment-free data and truly validating that Scaling Law, X2Robot has already offered its answer at this particular point in time:
The era in which every new task must begin with real-robot teleoperation data is coming to an end.
Project homepage: https://x2robot.com/pages/twindex