Editing Qwen weights to reduce reasoning loops, DeepSeek’s super-agent prototype, and GDPx benchmarks.
Nov 3 · 30 min read
Amid the deepening bear market casting a shadow over the AI/tech industry (and sparking plenty of gallows humor)—with Anthropic bonds rated “junk” and a whirlwind of headlines swirling around OpenAI’s fundraising—we continue to witness jaw-dropping capability breakthroughs across the community, including today.
To set the stage for today’s blockbuster paper, the best starting point is looking back just 8 months ago to Karpathy’s autoresearch, which foreshadowed a whole summer of anxieties surrounding recursive self-improvement (RSI):

At an abstract level, that demo showed that you could take a sufficiently intelligent model and have it autonomously train other models without human expertise or intervention by running a battery of experiments across a compute cluster. However, the process remains notoriously messy and complex—writing PyTorch/Jax code, burning compute, staring at loss curves, interpreting eval metrics, tuning kernels and data, and so on…
Now, last month’s podcast guest Alex Zhang has co-authored a massive breakthrough finding: what happens if you simply have a smart model directly read and edit the weights of a dumber model?
In machine learning terms, this is the equivalent of bypassing source code entirely to inspect the binary 0s and 1s of a (tiny) compiled executable, and being able to: 1) figure out what they do; 2) oneshot-fix bugs from failed execution traces, or iterate over 50 steps toward a desired improvement (though this might be slightly overblown); and 3) one-shot generate the weights of an entire network from scratch, even outperforming some independently trained models.
If you firmly believe that LLMs with sufficient parameter scale can fit and emulate anything (indeed, LLMs are getting remarkably better at compute as they scale), then theoretically the emergence of this capability was inevitable; yet watching it materialize out of nowhere is still mind-bending, as evidenced by [an independent replication](https://x.co