A Vision-Native Harness for Multimodal Models Is Here!
Recently, Kaiming He’s team introduced a vision-native harness in their latest paper, VISTA: A Visual Harness for Reasoning in an Interactive World—
VISTA.
It enables existing multimodal models to directly observe environments, store raw frames, and review, zoom in, and inspect fine details at any time during reasoning—all without requiring training from scratch.

Unlike traditional approaches that abstract environments into text or code, VISTA fully preserves raw visual information. Based on the problem at hand, the model can bring previously observed frames back into context to assist in current judgment and reasoning.
With this approach, visual experience is transformed into contextual information that the model can retrieve and cross-examine at any time, equipping multimodal models with native scaffolding built around visual memory.
In benchmark evaluations, Claude Opus 5.0 equipped with VISTA cleared all 25 public games in ARC-AGI-3, achieving a perfect score of 100 on relative human action efficiency and using 57.4% fewer game actions than the first-time human baseline.
Switching to GPT-5.6 Sol also achieved a 100% completion rate, with a high score of 99.

Furthermore, the team validated the effectiveness of this approach in browser games, mazes, and path-connecting puzzles, and designated embodied tasks closer to the physical world as future research directions.
How exactly is this solution implemented?
Vision-Native Scaffolding
From ResNet and Mask R-CNN to MAE, visual perception and representation learning have always been a central focus of Kaiming He’s research.
This time, VISTA tackles a new challenge: how can a model accumulate, preserve, and efficiently utilize visual experience throughout continuous interaction?
The answer is a Harness.
Last November, Anthropic used long-horizon programming tasks as an example to demonstrate how a harness helps models make sustained progress across multiple context windows.

Specifically, the harness tracks to-do items via a task checklist and records completed work using progress files and code logs.
This way, even when the model moves into a new context window, it can read these state records to clearly understand past progress and upcoming plans.
As a result, complex tasks can be executed step by step, verified iteratively, and seamlessly resumed across context windows.

It is fair to say that this harness mechanism—which primarily relies on text and code to maintain task state—has largely driven the recent surge in agent capabilities.
In real-world scenarios, however, relying solely on text memory is clearly not enough.
Once placed in a real visual environment, the model must not only remember object coordinates and orientations, but also accurately comprehend the subtle dynamic changes occurring in the environment before and after every action.
More critically, when the model first observes a frame, it often cannot anticipate which details will play a decisive role in subsequent steps.
Such rich visual information is difficult to exhaustively capture upfront in a text note, and there is no guarantee that it will remain complete and actionable when referenced later.
In benchmarks like ARC-AGI-3, a common existing approach is to first convert frames into textual grids of numbers, and then prompt the model to write code to simulate environment rules and deduce action plans.
Note: ARC-AGI-3 is an interactive visual reasoning benchmark that does not provide game rules or objectives upfront; agents must uncover winning strategies through autonomous observation and trial-and-error interactions.
However, this approach has limitations: as environmental complexity grows, object appearances, spatial topology, and dynamic transitions become exceedingly difficult to translate losslessly into text and code. Moreover, as interaction rounds stretch on, earlier visual frames are gradually squeezed out of the context window or replaced by heavily compressed text summaries.
Given this, VISTA focuses on solving a core question:
Without retraining the base model, how can an agent retrieve and verify historical visual information whenever reasoning demands it?
VISTA’s solution is: preserve every frame returned by the environment intact outside the context window (including animation transition frames during action execution), and selectively retrieve them whenever the model needs them.
In the ARC-AGI-3 evaluation, the system supports both cross-temporal multi-frame comparisons and localized cropping and zooming to verify fine-grained details such as orientation markers on tiles.

Under this architecture, text notes serve to capture the model’s current, stage-by-stage understanding, while raw frames act as objective ground truth that can be repeatedly re-examined.
The research team defines this mechanism as “explicit attention” over interaction history—the model can autonomously decide which visual data to bring back into context based on its current reasoning needs.
To achieve this, the underlying system must fully archive the raw image stream and provide an agile toolchain for retrieval and inspection.
Concrete Implementation
To give the model the ability to persistently store and review visual experience, VISTA establishes three core components:
Visual Observation, Lossless Visual Memory, and Active Visual Inspection. Working in concert, they handle frame input representation, historical retention, and on-demand retrieval, respectively.

1. Visual Observation: Provides the model with high-fidelity environmental frames.
In ARC-AGI-3 experiments, VISTA upscales the officially provided 64×64 frames to 512×512 PNG images, preserving color, shape, and spatial topological relationships so the model can perceive physical features directly.
2. Lossless Visual Memory: Structures and stores the frames seen by the model.
After each action execution, VISTA losslessly archives all images returned by the environment (including intermediate animation frames during action rollouts) and indexes them using round numbers and frame sequences.
Rather than rushing to judge which features are worth recording early on, the model first retains all visual experience as underlying assets, ready to be retrieved at any time.
3. Active Visual Inspection: Enables the model to review and delve into historical frames on demand.
Using the inspect tool, the model can both retrieve corresponding historical images for a specific round and crop/zoom into localized patches to inspect pixel-level details.
If it needs to clarify the exact state transition triggered by an action, the model can also pull frames across multiple rounds to perform differential comparisons.
This mechanism functions like equipping the multimodal model with a searchable “visual archive.” The model can ground itself in its current view while autonomously retracing its historical trajectory, providing empirical evidence for subsequent decisions.
In terms of runtime workflow, the three components follow strict temporal coordination:
At the start of each round, the model first parses the current frame and legal action space, evaluates the system state using prior experience, and formulates a hypothesis for the next action.
If current perceptual confidence is insufficient, the model can activate its toolchain to inspect historical crops, zoom in on localized regions, or even read discrete pixel values to substantiate its hypothesis.

Once the chain of evidence is established, the model does not rush into blind actions; instead, it first makes a forward prediction of the action’s outcome.
After execution, it compares and verifies the actual state feedback against its prediction, dynamically updating its mental model of the underlying rules.
Through this closed loop, the agent simultaneously advances environment exploration and knowledge evolution throughout continuous interaction.
To ensure contextual continuity across long-horizon tasks, VISTA supplements this with two structured text memos:
GUIDE.md: Preserves general rules and strategic patterns reusable across levels.WORKING.md: Records specific state, real-time progress, and upcoming execution plans for the current task.
When the context window approaches its capacity limit, the model generates a handover summary before smoothly transitioning to a new window to continue execution. Existing text records, action logs, and underlying visual assets are all seamlessly carried over.
Notably, while VISTA achieves a lossless, complete record of visual history, it avoids brute-forcing everything into the context window.
In the full configuration, after each round of execution, the framework injects only the final frame into the active context by default. Animation transition frames are hosted uniformly in external storage, waiting for targeted invocation commands.
This decoupled design prevents an influx of image frames from consuming precious context capacity while preserving the integrity of visual information without loss.
More importantly, VISTA introduces no new model weight training.
Scene understanding, reasoning decisions, and path planning rely entirely on existing multimodal foundation models, while the harness focuses on tool orchestration, experience management, and evidence delivery.
In essence, VISTA does not remodel the architecture of the neural network itself; instead, it revolutionizes the engineering methodology by which multimodal agents capture, retain, and orchestrate visual experience.
Experimental Evaluation
Beyond ARC-AGI-3, the team comprehensively evaluated VISTA across three additional benchmark suites—GameWorld, AI GameStore, and BabyVision—spanning various task modalities such as web games, maze navigation, and topological path-connecting puzzles.

Using GPT-5.6 Sol across all setups, compared to the official baselines:
- Across the 170 tasks in GameWorld, VISTA significantly boosted the completion rate from 40.0% to 63.3%;
- Across the 10 games in AI GameStore, the aggregate benchmark score surged from 47.3 to 140.3 (normalized to 100 for human median);
- Across 39 selected static maze and topological path-connecting puzzles in BabyVision, accuracy improved from 41.0% to 63.2%. Even in purely static tasks, localized zoom-ins and pixel comparisons enabled the model to substantially rectify its reasoning logic.
The empirical results demonstrate that losslessly archiving and selectively retrieving historical images can unleash the latent visual reasoning potential of existing large models.
With only minimal adaptation, the same VISTA paradigm seamlessly transfers to vastly different game mechanics and puzzle reasoning tasks, exhibiting strong engineering generalization.
The team stated that future research will focus on embodied AI tasks in the physical domain: investigating how models can efficiently retain, verify, and refine their spatiotemporal visual memories in highly dynamic, non-static environments to make more accurate physical decisions.
About the Authors
The paper’s co-first authors are Qiushi Han, Keya Hu, and Linlu Qiu. Corresponding and contributing authors include Cathy Wu and Kaiming He.
Qiushi Han (Josh Han) is currently a Ph.D. student at the MIT Operations Research Center, advised by Cathy Wu.

Keya Hu is currently a Ph.D. student in Electrical Engineering and Computer Science (EECS) at MIT, co-advised by Kaiming He and Jacob Andreas.

Keya Hu received her undergraduate degree from the ACM Class at Shanghai Jiao Tong University. Her research focuses on the intersection of vision and language, aiming to build general-purpose agents with high sample efficiency and strong generalization capabilities.
Linlu Qiu is currently a Ph.D. student in MIT EECS and the Computer Science and Artificial Intelligence Laboratory (CSAIL), advised by Yoon Kim and Jacob Andreas.

Her research focuses on natural language processing and machine learning. She previously conducted research at Google Research and Meta FAIR.
Cathy Wu is currently an Associate Professor in the Department of Civil and Environmental Engineering and the Institute for Data, Systems, and Society (IDSS) at MIT, with long-standing expertise in applying machine learning and reinforcement learning to optimize complex transportation systems.

She received her B.S. and M.Eng. degrees from MIT and her Ph.D. from UC Berkeley.
Kaiming He is currently a tenured Associate Professor in MIT EECS and the primary author of landmark computer vision breakthroughs including ResNet, Mask R-CNN, and MAE.

Kaiming He received his undergraduate degree from Tsinghua University and his Ph.D. from The Chinese University of Hong Kong. He previously worked for many years at Microsoft Research Asia (MSRA) and Meta FAIR before officially joining the MIT faculty in 2024.