Skip to content
Main Site News Console

Google Open-Sources New RSI Breakthrough, Led by Chinese Team

· 量子位
国内AI

Harness-Level RSI

Google has achieved a new milestone in RSI (Recursive Self-Improvement) and officially open-sourced the project.

Dubbed Harness², the project enables large language models to autonomously adjust and optimize runtime extension code, system prompts, toolchains, and skill libraries on the fly.

This is RSI applied directly at the harness (test scaffolding) level.

Its real-world benchmark performance is remarkably striking.

Equipped with Harness² capabilities, GLM-5 saw its score on the office workflow benchmark WorkBuddy-Bench surge by 11.5 points—

Surpassing vanilla Gemini 3.5 Flash in a single stroke.

For every model plugged into this framework, benchmark scores saw substantial gains across the board.

Image 3: image.png

The project is now fully open-source, with built-in adaptation and support for OpenCode and Codex.

Why “Self-Modification” Usually Backfires

Previously, the industry explored a variety of Recursive Self-Improvement (RSI) paradigms to make agents smarter.

The conceptual logic sounds ideal: allow AI to autonomously write code, refine prompts, and optimize the overall system. Yet, once deployed in real-world, open-ended environments, almost every such attempt hits a wall.

At its core, this boils down to two critical pain points:

The absence of ground truth.

In laboratory benchmarks, unit tests and ground truth provide straightforward feedback. But in authentic office and enterprise production environments—whether drafting a business proposal or reviewing a legal contract—live settings cannot offer instant, objective evaluations.

When an AI iterates on a prompt version by itself, assessing whether the update genuinely yields performance gains or simply introduces hallucinations and drift becomes extraordinarily difficult.

A severe vulnerability to single-task overfitting.

Systems often aggressively overhaul their underlying harnesses to fit a single target task. However, the moment the task shifts to another domain (such as writing a web scraper), those accumulated rules turn into dead weight.

If a modified harness is forcibly reused in subsequent tasks, the system can easily spiral into noise, causing performance to degrade rather than improve.

To tackle these two fatal bottlenecks, the Google team introduced Harness² alongside a dual-level recursive architecture.

Dual-Level Recursion: Experiment on the Left, Take Notes on the Right

The “squared” in Harness² signifies two mutually nested recursive loops: Task-level recursion and Harnessing-level recursion.

Level One, Task-level recursion: Running controlled experiments without ground truth.

Image 4: image.png

When encountering a new task, the system first generates a reference trajectory based on the initial harness. At this stage, the primary focus is not whether the outcome is flawless, but rather uncovering where defects and bottlenecks emerge along the execution chain.

Next, the system triggers a Propose–Probe–Compose mechanism:

  • Propose: A dedicated editor role puts forward two sets of modifications targeting the prior trajectory. One set focuses on “invariant fixes” designed to resolve cross-domain, common failure modes (e.g., erratic planning, tool-calling infinite loops, context window explosion); the other set focuses on “domain-specific patches” that target the specific business context, domain knowledge, and constraints of the active task.
  • Probe: The two candidate harnesses are deployed back into the original task in isolated runs. Even without an external judge, the system extracts execution traces and runs Behavioral Contrast against the reference trajectory to verify whether the modifications genuinely unblock stuck steps without introducing new behavioral regressions.
  • Compose: The composer integrates all behavioral execution evidence, filters out modifications that have been empirically verified to work, and merges them into the final execution harness.

Moreover, this mechanism supports parallel recursion by concurrently fanning out across $k$ candidate trajectories, enabling optimal composition across a much broader search space.

Level Two, Harnessing-level recursion: Crystallizing harness engineering experience into meta-capabilities.

Image 5: image.png

Once a task concludes and deliverables are completed, the lifecycle of most systems comes to an end. This is precisely where Harness² shines.

The system forwards all complete debugging logs—including reference trajectories, candidate modifications, contrastive evidence, and composition decisions—to an Experience Reflector.

Upon retrospective analysis, the reflector generates an EXPERIENCE.md document, capturing the modification rationale, activation conditions, and downstream impacts, and permanently deposits them into the Harness Improvement Library.

When new tasks arrive down the line:

The execution runtime still receives a clean, unpolluted baseline harness; however, the “meta-brain” responsible for harness tuning is armed with an extensive playbook of historical lessons. During the proposal and composition phases, the system can instantly query verified components and anti-patterns from similar historical scenarios.

This does not merely polish the harness for the task at hand; it elevates the overarching capability to iterate on future harnesses—transforming one-off temporary patches into sustainably reusable cognitive assets.

Benchmark Results

Beyond the theoretical framework, the research team conducted rigorous evaluations across three representative professional benchmarks:

  • WorkBuddy-Bench: Encompassing complex office workflows across three domains: Code, Web, and Office suites;
  • JobBench: Long-horizon, end-to-end complex professional job tasks;
  • LAB: Harvey’s challenging open-source legal agent benchmark.

The base models evaluated included Gemini 3.5 Flash and GLM-5, paired with popular open-source harness frameworks OpenCode and Codex.

Empirical results show that with just a single step of Harness² iteration ($k=1$) on WorkBuddy-Bench:

  • GLM-5’s overall score on OpenCode jumped from 64.2 to 75.7, an 11.5-point boost; on Codex, it increased from 64.4 to 75.3.
  • Gemini 3.5 Flash rose from 72.4 to 77.5 on OpenCode, and from 74.5 to 77.5 on Codex.

Image 6: image.png

The data reveals that the harness-optimized GLM-5 (75.7 / 75.3) completely surpassed vanilla Gemini 3.5 Flash without harness optimization (72.4 / 74.5).

On the end-to-end professional task benchmark JobBench, single-step self-improvement delivered comprehensive gains as well: Gemini 3.5 Flash gained 4.5 points, and GLM-5 gained 3.6 points; on the legal benchmark LAB, GLM-5’s partial pass rate surged by 11.8 percentage points.

Scaling the parallel exploration budget to $k=5$ (i.e., running 5 concurrent candidate variants) pushed performance even further: WorkBuddy-Bench gained up to an additional 3.9 points, JobBench improved by up to an additional 4.2 points, and the LAB pass rate expanded by an extra 6.7 percentage points.

Image 7: image.png

In cross-domain experience transfer tests, the team transferred the experience library compiled from JobBench tasks directly into the WorkBuddy-Bench environment. Without performing any additional probing runs, the system realized immediate gains of +0.6 points in Code, +0.5 points in Web, and +2.9 points in Office workflows.

Image 8: image.png

For some time, industry progress in AI capabilities has relied predominantly on the “foundation model scaling” paradigm, with the prevailing expectation that next-generation models would automatically resolve downstream deployment bottlenecks.

Real-world production environments, however, suggest that as training costs for base models soar and marginal returns diminish, long-tail challenges in production scenarios remain ubiquitous. Even as model context windows expand, critical details can still slip through the cracks during intricate multi-file cross-referencing; even as benchmark scores climb, a model may still lack the engineering discipline to proactively inspect and validate file formats after drafting a report.

These engineering challenges cannot be conquered purely through pre-trained weights alone. Instead, they demand a runtime cognitive engineering mechanism capable of self-perception, autonomous self-patching, and continuous evolution during execution.

Historically, harness construction has heavily depended on human engineers manually drafting prompts, configuring toolchains, and maintaining execution logic—an approach characterized by high maintenance overhead and inevitable edge-case blind spots. The findings of Harness² demonstrate that returning the initiative of rule adaptation and tuning directly to the agent itself is emerging as a practical, viable path forward.

Paper Authors

The paper features 9 authors, with a notable presence of Chinese scholars. Core contributors include:

Ruiyao Xu, first author and corresponding author.
Ph.D. student in the Department of Statistics and Data Science at Northwestern University, advised by Kaize Ding. His research focuses on large language models, graph machine learning, anomaly detection, and AI agent systems. This paper represents work led during his internship at Google.

Yanfei Chen, Senior Research Engineer at Google Cloud AI Research.

Image 9: image.png

He received his bachelor’s degree from Peking University, double-majoring in Electrical Engineering and Economics, and earned his Ph.D. in Industrial Engineering from the University of Pittsburgh in 2018. Prior to Google, he worked as a research engineer at the machine learning company Qeexo. Joining Google in 2020, he has contributed to core projects such as Vertex AI Search and Gemini Enterprise Deep Research. His primary research interests encompass LLM agents, retrieval-augmented generation (RAG), and tool learning.

Zhongying CuiZhu (Zoey CuiZhu), Research Engineer at Google Cloud AI Research.

Image 10: image.png

She received her Ph.D. in Electrical Engineering from Stanford University in 2019. Her primary research areas include generalist agents, verifiable agents, and agent evaluation frameworks.

Reference Link:
[1] https://arxiv.org/html/2610.14224v1