Skip to content
Main Site News Console

ToyBlocks: A Controlled World for Robot Learning and Evaluation

· Hugging Face Translated
教程模型卡

💻 Code | 📊 Data | 🤖 Models | 📖 Documentation

Image 9: Isometric robot illustration of ToyBlocks

Robots are learning to fold laundry, sort parts, and manipulate unfamiliar objects. Yet these demonstrations are often quite narrow, leaving it unclear what the robot has actually learned or why it fails.

To study complex systems, scientists often start with small, controlled environments—such as a petri dish or an ecosphere—where they can manipulate conditions and observe the effects. Robotics similarly needs environments that enable this kind of exploration: setups where researchers can vary training data or model designs, run repeatable tests, and trace the root causes of failure.

Today, we are introducing a framework for building such environments, which we call FoundationWorlds. Each FoundationWorld pairs a controlled simulation environment—designed to study robots in a specific context—with tools for automatically generating tasks and training demonstrations, alongside metrics to evaluate robot behavior over time. Together, they provide a research testbed to explore questions such as how models improve with additional training data and how best to combine high-level reasoning with precise physical control.

Video 1

Our first FoundationWorld is ToyBlocks: an open-source simulation framework built around a robotic arm and tabletop blocks. While the setup is deliberately kept simple, the robot must still reason about how the blocks interact and how to arrange them when pushing, flipping, and stacking.

ToyBlocks can automatically generate demonstration data—records of the robot completing tasks in simulation—without requiring time-consuming human teleoperation (where a person remotely controls the robot). Researchers can easily modify these training experiences. For example, a team can generate more pushing or flipping demonstrations, and then evaluate performance on tasks held out from training.

With ToyBlocks as our first FoundationWorld, our goal is to progressively scale this approach to increasingly complex environments.

How ToyBlocks Works

Building on our earlier work in open robot learning (including MolmoAct 2, MolmoBot, and MolmoSpaces), ToyBlocks offers researchers a way to investigate robotics problems within a deliberately constrained domain.

As a FoundationWorld, ToyBlocks is designed around four key requirements: (1) realistic physics and appearance; (2) automated, large-scale task and demonstration generation; (3) challenges that combine reasoning with dexterity; and (4) broad coverage of possible tasks within the specific domain.

Open-ended domains require modeling a vast variety of objects and interactions, making it impractical to satisfy all of these requirements simultaneously. In contrast, the narrow scope of ToyBlocks allows it to fulfill all four.

Within ToyBlocks’ compact setting, researchers can focus on exploring how robots learn to perform any plausible task, with high confidence that the behavior of the simulated arm and blocks closely matches their physical counterparts.

Image 10: Physical robot gripper over wooden blocks in the ToyBlocks setup

Physical robot gripper over wooden blocks in the ToyBlocks setup.

ToyBlocks is not just about grasping and