Skip to content
Main Site News Console

MIT Star Project Astra Under Fire: Accused of Repackaging Qwen Weights and Exaggerated Claims

· 量子位
国内AI

spaCy Creator Slams Viral MIT Paper: Did Astra Really Edit Model Weights Directly?

Jay, reporting from the Office

Can Astra directly edit model weights? That viral MIT paper is now facing serious pushback from an industry veteran.

Just days ago, Astra took the AI world by storm with an astonishing claim: modifying the weights of another AI model directly—without training and without computing gradients.

The MIT team claimed that by tweaking just 0.016% of the weights in Qwen3.5-0.8B, Astra propelled its math benchmark score from 7.9% to a staggering 48.1%—more than a sixfold improvement.

Netizens even marveled that AI had finally begun to decipher the “mind” of another AI.

However, the narrative quickly took a turn.

A seasoned industry insider published a detailed post raising serious doubts:

This research is deeply misleading.

Image 3

That critic is Matthew Honnibal, creator of spaCy and co-founder of the AI company Explosion.

The Magician’s Trick

So what’s actually going on here?

Honnibal used an apt metaphor to call out what he sees as sleight of hand in the MIT paper:

Stage magicians often use multiple methods to perform what looks like the same trick.

The first time, they might use a gimmick. The next time, they show the prop is ordinary, but secretly palm the object. The third time, they might rely on a psychological force.

Watching the routine, you assume there is just one method behind it all, so each variation seems to disprove your theory of how it works.

Honnibal argues that the way Astra’s experiments were presented closely mirrors this kind of carefully choreographed magic show.

Looking at the paper itself, it actually demonstrates three distinctly different capabilities:

  1. Reading weights: Identifying what task a tiny neural network performs purely by inspecting its raw weight values.
  2. Writing weights: Directly generating or modifying the parameters of a small network from scratch without environmental feedback, enabling it to play mini-games like Breakout.
  3. Patching an LLM: Significantly boosting Qwen’s math benchmark performance by tweaking a subset of its weights.

Yet, the abstract leaves readers with the overarching impression that Astra understands and modifies another model “solely by observing raw weights.”

A casual read easily gives the false impression that identifying a toy network’s function and patching an LLM’s math capabilities stem from the exact same “mind-reading” trick.

But is that really what happened?

After meticulously examining the paper’s eight experiments, Honnibal discovered that the experimental setups used to achieve these three tasks were fundamentally different.

In other words, it was a classic bait-and-switch.

Two Entirely Different Approaches

So what really lies under the hood across these different setups?

In experiments like identifying the task of a small network or fixing a stalled racing-car policy, Astra indeed operated strictly by “inspecting raw weights.”

That’s because these experiments imposed strict constraints:

  • Raw weight floats were fed directly into Astra’s context window;
  • Astra was barred from executing Python code;
  • Astra was not allowed to run the target network to observe its runtime behavior.

Under these rigid constraints, the models Astra analyzed were minuscule, spanning merely 17,000 to 172,000 parameters. While certainly an intriguing technical feat, it remains worlds away from genuinely comprehending a modern LLM.

Image 4

However, when it came to the headline-grabbing, most sensational experiment—repairing Qwen’s math capabilities—the methodology was entirely different.

When patching the 800-million-parameter Qwen model, Astra did not read raw weights directly.

Instead, Astra was given full access to write and execute Python code, with up to 50 forward passes allowed. It ran the model over a batch of practice problems, analyzed its actual performance, and iteratively adjusted the weights.

Mechanistically, these are two completely distinct approaches.

As Honnibal pointed out, conflating these two divergent setups creates a misleading illusion: that the “mind-reading” trick validated on toy models somehow scales seamlessly to an 800M-parameter LLM.

In reality, when dealing with an actual language model, Astra acted more like an “external coach”—relying on standard engineering workflows involving coding, execution, and empirical observation.

The Real Value

Does this mean MIT’s research has no value?

Absolutely not.

Even stripped of the sensationalized “telepathy” framing, Astra’s actual capabilities remain genuinely impressive.

Honnibal acknowledged that prompting an AI model to conceive and execute workflows that previously required a human ML engineer—writing scripts to inspect and patch network weights—is in itself a significant milestone.

AI models from a year or two ago would have failed at all of these tasks.

It demonstrates that state-of-the-art LLMs possess mature tool-use and complex reasoning capabilities, making them capable of acting as “AI engineers” that diagnose and patch other models. That alone is a noteworthy engineering achievement.

The real issue lies in how the results were framed: conflating two entirely separate technical paradigms into a unified, mythical “superpower” capable of directly reading and rewriting an AI’s brain.

In many ways, it reflects the rampant hype cycle currently permeating the AI industry.

Source: https://honnibal.dev/blog/astra-read-write-misleading