Skip to content
Main Site News Console

Magic Atom's Embodied Agent Framework Tops MolmoSpaces Global Leaderboard

· 量子位
国内AI

Pushing VLAs from “Demo-Ready” to “Production-Ready”

Recently, on the latest leaderboard of MolmoSpaces—a leading global zero-shot evaluation benchmark for embodied AI launched by the Allen Institute for AI (AI2)—Magic Atom’s EchoPolicy-0, a VLM Agent framework designed for general-purpose robotic manipulation, ranked #1 globally on the MolmoSpaces Combined benchmark with an average robotic manipulation success rate of 75.4%, covering all four v1 tasks.

On the same leaderboard and under the same category, Magic Atom’s in-house embodied AI agent framework, MagicBrain, with a parameter size of just 3.4B, secured 4th place globally on the MolmoSpaces Combined leaderboard with an average robotic manipulation success rate of 72.4%. Notably, its grasping success rate reached 94.4%, far exceeding the industry average.

Image 3

The MolmoSpaces benchmark is currently one of the most authoritative zero-shot evaluation benchmarks in embodied AI, spanning manipulation, navigation, and longer-horizon embodied tasks. The MolmoSpaces ecosystem encompasses over 230,000 indoor environments and approximately 130,000 object assets, providing a rich diversity of scenarios for evaluation. Robots must navigate complex conditions beyond fixed demo setups, searching for targets across diverse backgrounds, geometries, and object distributions, and completing tasks grounded in local physical constraints.

Furthermore, MolmoSpaces covers both movable rigid bodies and articulated interactive objects. A robot must not only accurately recognize targets within images, but also alter object states through realistic physics-simulated contact, which imbues the manipulation tasks with substantial physical complexity.

The MolmoSpaces benchmark explicitly defines initial scenes, robot and camera configurations, task instructions, and success criteria. Its comprehensive evaluation checklist, standardized scoring system, and public results help transform fragmented demonstrations into quantifiable, benchmarkable data. The four tasks evaluated by Magic Atom’s EchoPolicy-0 and MagicBrain frameworks span free-form object grasping, articulated object interaction, and multi-stage complex manipulation, providing a multi-dimensional assessment of an embodied AI agent framework’s system-level capabilities rather than evaluating an isolated metric.

Since 2026, embodied AI has been accelerating its transition from conceptual demos to real-world production lines. The center of gravity in technological competition has shifted upstream to the robot’s “brain”—embodied foundation models. With the rapid advancement of robot motion control, robot bodies are becoming more agile and durable, and the practical application of foundation models is evolving swiftly alongside expanding physical capabilities. Consequently, industry research is pivoting toward a new focus: enabling robots to accomplish more complex tasks and rapidly master new ones. The MolmoSpaces benchmark similarly places a heavy emphasis on the generalization capabilities of embodied foundation models.

The head of MagicLab Studio, Magic Atom’s R&D laboratory, stated: “When it comes to the technical roadmap for the embodied brain, we strongly advocate for a hierarchical architecture. We identified instruction-following as the core bottleneck currently facing models in general-purpose tasks. EchoPolicy ensures instruction-following via an agent framework and trajectory scoring mechanism, whereas MagicBrain optimizes it through end-to-end visual guidance. Both approaches have achieved standout results.”

EchoPolicy-0: Enabling Robots to Share a Unified Thinking and Acting Mechanism Across Tasks

To complete a manipulation task, a robot must continuously chain together linguistic goals, visual perception, and physical actions. For instance, after grasping an object, it needs to verify whether it has achieved a stable grip; before opening a cabinet door, it must locate the interactive handle; and if an action falls short of expectations, it must dynamically adjust subsequent plans based on real-time feedback.

Although these actions differ in form, they inherently share common underlying challenges that embodied robots must resolve across tasks: Where is the target? What is the current state? Which capability should be invoked? Did the action achieve its expected outcome? And how should the system recover from failure?

Addressing these cross-task shared challenges, Magic Atom’s MagicLab Studio integrated vision-language models (VLMs), geometric manipulation tools, and general-purpose action policies into a unified VLM Agent framework: EchoPolicy-0. This establishes a full-lifecycle closed loop of “observe environment → understand goal → select capability → execute action → observe & verify.” Different tasks reuse this central mechanism, while concrete contacts and motions are delegated to low-level actuators.

The R&D lead for the EchoPolicy-0 framework stated: “EchoPolicy-0’s generality lies in its ability to uniformly invoke, compose, and verify capabilities. The current version has successfully completed validation across four manipulation categories on the Franka single-arm robot platform.”

In utilizing Vision-Language-Action (VLA) models, MagicLab Studio introduced a VLM goal-guided multi-candidate trajectory selection mechanism: the action model first generates multiple candidate motion trajectories projected onto the image plane; next, the trajectory best matching the task objective specified by the VLM is selected; and finally, continuous verification is performed post-execution. This allows high-level visual understanding to actively participate in low-level action selection rather than remaining confined to task decomposition.

Image 4

The EchoPolicy-0 framework completed a full evaluation of 3,915 episodes across four manipulation tasks on MolmoSpaces, achieving an equally weighted average success rate of 75.4%. As of October 13, 2026, it firmly holds the #1 spot globally on the MolmoSpaces Combined leaderboard.

MagicLab Studio regards EchoPolicy-0’s performance on MolmoSpaces as a system-level validation of its unified framework, leveraging episode-by-episode evaluation data to guide subsequent technical iterations. EchoPolicy-0 demonstrates a viable pathway toward general-purpose robot manipulation: dispatching diverse manipulation capabilities via a unified agent framework—where the VLM Agent governs semantic goals and task states, geometric tools provide verifiable spatial interactions, general action policies generate candidate solutions, VLM goal-guided trajectory selection constrains action directions, and continuous verification loops integrate the entire process end-to-end.

For real-world deployment of embodied models, EchoPolicy-0 addresses a critical challenge: how to ensure the model selects the correct solution from generated actions. Without modifying model inputs, this approach filters out the most optimal trajectory based on visual goals from multiple candidate action sets and maintains continuous verification post-execution. For downstream customers, the value of this approach far exceeds a simple boost in point-in-time success rates: adding new task categories requires no ground-up rebuild—new capabilities simply plug into the same overarching framework seamlessly.

MagicBrain: Empowering Models to Focus on What Matters, Driving Embodied Manipulation Toward “Production-Ready”

The embodied AI agent framework MagicBrain also originates from Magic Atom’s MagicLab Studio. Across the four MolmoSpaces manipulation tasks—pick, pick-and-place, open, and close—MagicBrain completed a full evaluation of 3,915 episodes, ranking 4th globally in average success rate across all four v1 tasks. Notably, it claimed #1 globally in both Pick-v1 and Pick & Place-v1 individual task leaderboards, achieving a grasping success rate of 94.4%.

As one of the core model architectures in embodied AI, Vision-Language-Action (VLA) models have long faced an industry-wide bottleneck in language following. Having a model execute an instruction like “put the apple in the basket” requires mastering two critical steps: first, understanding the semantic concept of “apple,” and second, pinpointing exactly which object constitutes the “apple” at the pixel level.

During R&D testing, MagicLab Studio discovered that the vast majority of grasping failures were not caused by unstable motion execution, but rather by grasping the wrong target or placing it in the wrong location. In other words, VLA learning at the action level is already relatively mature; the true weakness lies in grounding noun phrases to specific image regions, which happens to be the area where training signals are sparsest. When a robot intends to grasp an “apple” but mistakenly grabs a neighboring “pear,” the failure lies not in the gripper, but in visual localization.

In practice, VLAs follow visual cues significantly better than pure linguistic instructions—that is, their ability to “understand visual input” outperforms their ability to “interpret verbal commands.” To this end, MagicBrain adopted an In-Context Visual Guidance technical path: the VLA focuses solely on action execution, while target identification is offloaded to the VLM Agent, which excels at image interpretation.

Image 5

In the future, this visual guidance technique can be expanded to broader scenarios, such as human-robot interaction—highlighting “what to look at” in a frame is by no means limited to grasping tasks. MagicBrain introduces a more efficient division-of-labor paradigm: language defines “what to do,” pixels delineate “which one and where,” delegating each type of information to the model best suited for it. This approach does not require re-architecting model backbones or stacking massive parameters; it achieves a quantum leap in efficiency simply by optimizing the information delivery pipeline.

“Building upon the VLA’s language comprehension, we further guide it to ‘clearly see’ and ‘lock onto’ key targets,” said the head of MagicLab Studio. This roadmap breaks away from the conventional brute-force approach of scaling parameters for performance gains; the model evaluated here features a parameter size of just 3.4B.

This also implies that any team with fundamental data engineering capabilities can reproduce this approach at relatively low cost, achieving substantial performance gains without blindly inflating model scale. As embodied AI moves toward scaled deployment, a decoupled, collaborative division-of-labor mindset may prove far more pragmatic than pursuing a “single all-powerful foundation model.”

On the industrial deployment front, Magic Atom’s breakthrough directly addresses an industry pain point: whether VLAs can be “affordable and practically usable.” Grasping and placing represent the most frequent fundamental actions as embodied AI enters real-world settings like industrial sorting, warehousing, logistics, and domestic services. Elevating the grasping success rate from 70% to 94.4% signals that VLAs are crossing the tipping point from “demo-ready” to “production-ready.” For the entire industry, every fraction of an improvement in base success rates significantly broadens the commercial viability of embodied systems.

Magic Atom’s Design Philosophy for the “Embodied Brain”

“Securing the global #1 spot on MolmoSpaces is certainly significant, but what Magic Atom cares about even more is the foundational question behind the benchmark: how should an embodied robot’s brain truly be built, and where do its boundaries lie?” said the head of MagicLab Studio.

Magic Atom firmly believes that embodied AI should follow a hierarchical, collaborative evolutionary path: high-level models handle macro-level comprehension, task planning, and autonomous error recovery, while low-level models translate task intent into high-precision physical actions.

High-level models must support visual comprehension, task breakdown, planning and decision-making, error correction, and memory management. While current frontier foundation models provide a solid baseline, they still require deep adaptation to spatial topology, physical laws, and continuous interaction mechanisms in embodied contexts. The Studio head noted that in the near term, the industry should concentrate on building a foundational execution harness for models—tightly coupling context management, tool dispatching, execution feedback, and evaluation mechanisms, while exploring feedback-driven iterative optimization (including recursive self-improvement / RSI). In the long run, models must undergo pre-training and post-training grounded in real production line data to steadily elevate native comprehension and decision-making capabilities in embodied environments.

The head pointed out that models responsible for action generation and low-level control can leverage VLAs, WAMs, or other hybrid architectures. Architecture selection should closely align with practical business goals, prioritizing instruction-following fidelity, action precision, execution efficiency, and generalization capability.

While hierarchical architecture has become a broad consensus across the industry, a crucial bottleneck persists: how high-level task intent can be accurately perceived and stably executed by low-level models.

Experiments by MagicLab Studio indicate that existing action models still exhibit issues like target recognition drift and spatial constraint misinterpretation when facing out-of-distribution (OOD) scenarios. To tackle this, the team explored two complementary solutions:

The first is the leaderboard-topping EchoPolicy-0 approach, which ensures high alignment between low-level execution and high-level instructions through agent scheduling, candidate trajectory selection, and closed-loop verification. The high-level model breaks down the task into concrete sub-goals; the system comprehensively evaluates multiple candidate action trajectories generated by the low-level model to select the one best aligned with the instruction; post-execution, the system evaluates progress against fresh observations to autonomously decide whether to proceed, retry, re-plan, or safely fall back, invoking geometric tools as needed to assist manipulation.

The core advantage of this route is that it requires no secondary training of the low-level model. By optimizing scheduling, filtering, and verification logic alone, it enables rapid technical iteration and significantly boosts task success rates on complex tasks for existing models.

The second is the technical route adopted by MagicBrain, which emphasizes strengthening target alignment during action generation through explicit visual guidance and targeted post-training.

The visual guidance mechanism between high and low levels offers immense engineering value. MagicBrain injects visual prompts (markers)—representing interactive targets, placement zones, and short-term trajectories parsed by the upper layer—directly into the low-level model’s observation frames. This establishes an end-to-end mapping between abstract task intent and specific pixel coordinates in spatial imagery. Without altering the VLA’s network architecture, dedicated post-training enables the model to autonomously bind visual markers with manipulation semantics and physical actions, eliminating the ambiguity inherent in relying solely on natural language instructions.

These two solutions target two core phases—“action generation” and “execution, selection, and verification”—exhibiting natural complementarity: visual guidance empowers low-level models to generate highly aligned action candidates, while agent scheduling focuses on global oversight, precise selection, continuous verification, and fault-tolerant recovery.

The outstanding performances of EchoPolicy-0 and MagicBrain strongly validate the effectiveness of this dual-track approach.

Looking ahead, Magic Atom stated: “In the next phase, we will further deepen the technical convergence of both approaches, establishing a full-lifecycle feedback loop spanning from task understanding to physical execution. Concurrently, we will immerse ourselves in industrial production environments, continually refining task success rates, operational stability, and overall deployment costs, while establishing matching data and model platforms. We will distill validated capabilities into reusable skill modules, standard APIs, and industry solutions, systematically accumulating data on successful runs, failure edge cases, and human interventions. This will build a virtuous flywheel of ‘deployment → data collection → training → evaluation → re-deployment,’ accelerating the translation of engineering expertise into self-evolving foundation model capabilities.”