Skip to content
Main Site News Console

RSI Powers Chinese Open Source to Global #1: 1.3B OXR Redefines OCR

· 量子位
国内AI

Applying RSI (Recursive Self-Improvement) to OCR: The Results Are Stunning

Recently, an open-source model with just 1.3B parameters took the document parsing leaderboard by storm.

It claimed the #1 spot across all three major benchmarks: OmniDocBench, olmOCR-Bench, and DocParsingBench.

Benchmark scores aside, let’s first look at how it performs in practice.

Old-school handwritten advanced calculus notes converted to Markdown with a single click:

Image 3

It can even fully reconstruct intricate chemical molecular diagrams:

Image 4

Its performance is remarkably robust, essentially parsing “anything and everything.”

Image 5

Even more surprising is its processing speed.

On a single RTX PRO 6000, parsing 100 pages of documents end-to-end takes just 101 seconds.

Image 6

The model quickly gained traction in the global developer community, amassing over 1.1K GitHub stars and trending on X (Twitter)‘s Daily News.

Image 7

All of these breakthroughs stem from a core mechanism currently taking the AI world by storm—

RSI (Recursive Self-Improvement).

Powered by this self-evolving capability, this new breed of OCR is called OXR (Optical “Everything” Recognition).

Ranked #1 Across Three Global Benchmarks

On the surface, document parsing might not seem like a high-barrier task, but in reality, it remains an unsolved engineering challenge.

When faced with an ordinary PDF, traditional OCR often falls short: two-column layouts get scrambled, handwritten formulas turn into garbled text, and cross-page complex tables break down entirely. As for chemical structural formulas, many tools simply ignore them as random noise or background lines.

If such low-quality extraction is fed into a RAG system, the entire tool-calling pipeline will fail, no matter how capable the downstream model’s reasoning is.

Now, this bottleneck limiting AI from acquiring high-quality context has finally seen a breakthrough.

  • OmniDocBench: Ranked #1 with a score of 97.18 (open-sourced by Shanghai AI Laboratory, covering 10 types of complex documents).
  • olmOCR-Bench: Ranked #1 with a score of 87.84 (released by renowned AI institute Ai2, featuring over 7,000 unit-test-grade benchmark questions).
  • DocParsingBench: Secured #1 with a score of 87.76 (covering 5 major industries and 14 real-world business scenarios).

Image 8

The open-source document parsing model OXR topped all three authoritative, world-class benchmarks.

Furthermore, its full evaluation pipeline and code have been completely open-sourced for 100% reproducibility.

Hands-on testing confirms it significantly outperforms existing solutions. Legacy tools typically just convert images into rough text strings, leaving structural organization to downstream LLMs to guess; OXR, on the other hand, delivers all-in-one parsing for 13 distinct document elements:

Including tables, formulas, code blocks, images, chemical formulas, footnotes, headers, footers, stamps/seals, and more.

Here is how it performs in real-world scenarios:

Parsing magazines: outputs structured Markdown preserving full semantic hierarchy and reading order:

Image 9

Parsing mathematical formulas: directly converts them into standard LaTeX code:

Image 10

Parsing academic papers: two-column layouts cause zero interference, producing high-quality results without requiring LLMs to call auxiliary tools:

Image 11

It can even accurately parse complex math exam papers:

Image 12

Equally exciting is its open-source nature.

Unlike LLMs with tens or hundreds of billions of parameters, open-sourcing small, domain-specific models offers much greater utility for everyday developers. Even 27B models pose high local deployment barriers, whereas OXR has only 1.3B parameters.

This means it can be fully deployed and run efficiently on a standard consumer GPU (like an RTX 3060).

The global developer community has been effusive in its praise:

Image 13

Image 14

So, how does a lightweight model with just 1.3B parameters achieve state-of-the-art accuracy?

The secret lies in the mechanism introduced during its post-training stage: RSI (Recursive Self-Improvement).

Demystifying RSI Training

For a long time, the industry has relied on a familiar playbook: achieving high recognition accuracy meant stacking tens of billions of parameters alongside massive human-annotated datasets.

But in document parsing, annotated data is an endless black hole. Layouts are wildly diverse, with intricate formulas, code snippets, and charts. Relying purely on human annotation is not only exorbitantly expensive, but also prone to human error. In the real world, endless long-tail corner cases are virtually impossible to exhaustively cover.

The R&D team took a counter-intuitive angle: document parsing possesses a core, often-overlooked property—every input has a highly objective, deterministic ground truth.

Character spelling, multi-row/multi-column cell alignment, and mathematical symbol integrity can all be measured programmatically and objectively.

If the criteria for right and wrong are clear and objective, why can’t the system form an autonomous closed loop?

Based on this premise, the team integrated a Recursive Self-Improvement (RSI) training paradigm tailored for document parsing during OXR’s post-training phase: rather than relying on newly added manual annotations, the AI system autonomously detects errors, attributes root causes, generates the next round of training data, and iterates on itself.

Image 15

The RSI pipeline consists of three main stages:

1. Prediction & Classification

OXR first performs batch parsing across massive volumes of unlabeled real-world documents.

Next, a general vision-language model (General VLM) steps in. Evaluating the original image, parsed output, and predefined metrics, it categorizes samples into three groups:

  • Correct: Fully accurate; passed directly.
  • Wrong: Explicitly identified as incorrect; forwarded to the in-depth analysis pipeline.
  • Uncertain: Unstable prediction confidence or edge distributions; also routed to the analysis pipeline.

Introducing the Uncertain category is a crucial design choice. If forced into a binary black-and-white split, the evaluator model’s uncertainty would be hardcoded as erroneous pseudo-labels, contaminating subsequent training data. The Uncertain bucket effectively isolates volatile samples.

Furthermore, the system performs multiple rollout samplings of OXR. If a document yields high variance across rollouts, it is flagged with low confidence and assigned to the Uncertain pool, even if a correct result occasionally surfaces.

Finally, while preserving semantic content, the system introduces slight visual perturbations (e.g., downscaling, lossy compression, slight blurring, or translation) to high-variance samples. If these perturbations significantly widen output discrepancies, it indicates hyper-sensitivity to subtle visual variations. This perturbation pattern is then logged as a characteristic “pathology” for targeted reinforcement in the next phase.

2. Attribution & Training Planning

After filtering out erroneous and edge cases, the system must pinpoint the root causes of failure.

In this stage, the General VLM acts as a diagnostician, analyzing specific failure points: whether it is a multi-row/column cell alignment failure or cursive handwriting misrecognition.

The natural-language explanations generated during diagnosis are passed into SigLIP2 to extract feature vectors, which are then grouped into distinct Reason Clusters via independent clustering algorithms. For instance, while financial balance sheets and chemistry lab reports differ vastly in content, if clustering shows both failed due to rare multi-row/multi-column structures, this structural flaw is abstracted into a shared training target.

3. Data Generation

Once reason clusters are identified, the synthetic data pipeline receives clear, targeted objectives.

OXR’s approach uses reason clusters as structural challenge inputs and the real sample pool as the baseline for layout design and visual style. For each synthetic batch, layout characteristics are extracted from real samples as anchors, and specific layout challenges are blended in. Visual vulnerabilities exposed during perturbation testing are also systematically injected into the synthesis pipeline, compelling the model to build robustness during training.

Strictly validated synthetic data is piped directly back into training the next-generation OXR. Once an iteration finishes, prior weaknesses are resolved; a new round of real-world testing then reveals deeper failure modes, driving the system into the next cycle.

Through this, RSI achieves a genuinely positive self-evolution loop. The data distribution becomes increasingly comprehensive with each iteration. This is the underlying reason OXR achieves such high “intelligence density” with just 1.3B parameters: when data quality is sufficiently precise and refined, models don’t need brute-force parameter expansion for rote memorization.

A Lightweight, Pluggable Architecture

Beyond high accuracy, OXR’s inference speed is equally impressive: end-to-end parsing of a 100-page document on a single RTX PRO 6000 takes only 101.0 seconds.

This performance doesn’t just stem from a “small parameter count”—it is driven by a ground-up architectural redesign of document parsing.

Looking back at the industry’s recent technical evolution:

  1. Traditional Pipeline Approach: Splits layout detection, text recognition, and reading order into separate small models processed sequentially. The pipeline is cumbersome and costly to maintain.
  2. End-to-End Pure VLM Approach: Treats the task as a direct image-to-sequence mapping, using a single vision-language model to generate the full page’s text and layout. While architecturally unified, autoregressive generation of long sequences is computationally heavy, resulting in extremely slow inference.
  3. PaddleOCR-VL Approach: Combines Layout Analysis (LA) and Reading Order (RO) into an upstream small model, followed by a downstream Recognition model for a two-stage decoupled architecture. This significantly boosted layout detection speed and became a representative industry paradigm. However, on the annotation side, layout annotations had to be tightly bound to reading order, drastically inflating data processing overhead.

After analyzing the trade-offs, the OXR team proposed a restructured architecture: an independent lightweight model handles Layout Analysis (LA), while the primary OXR model takes charge of both Content Recognition and Reading Order (RO) inference.

That is: Layout Analysis (LA) -> Recognition + Reading Order (RO).

This division of labor delivers notable engineering advantages:

First, it virtually eliminates the performance penalty of calculating reading order. Throughout the pipeline, reading order inference consumes far less compute than text recognition. By folding reading order into the main model, OXR introduces virtually no extra latency. Instead, it leverages the main model’s inherent semantic understanding (Content-Aware) to markedly boost reading order accuracy. Meanwhile, the upstream Layout Analysis (LA) model is freed from ordering constraints, allowing it to remain a streamlined, high-throughput lightweight object detector that improves overall concurrency.

Second, the layout analysis module is completely decoupled and pluggable. Different business scenarios have vastly different layout requirements: For example, in mixed text-and-image regions, some use cases require extracting the area as a single large image, whereas knowledge base workflows require pulling out the text as body content; for densely arranged multi-panel comics, slicing criteria differ drastically across domains.

Image 16

Under OXR’s architecture, Layout Analysis acts as an independent component that can be customized and swapped out with minimal effort:

Image 17

For niche layout scenarios such as ancient book digitization or medical invoices, users can seamlessly plug in custom-trained layout detectors while continuing to leverage OXR’s robust backend content recognition and structured generation capabilities.

Out-of-the-Box Ecosystem

To simplify developer integration, an official free demo API is currently available with no registration or API keys required.

Provided server capacity permits, this open demo endpoint will remain available to the community.

In real-world writing and knowledge management workflows, comparing this parsing interface against traditional tools reveals a marked leap in layout fidelity across complex PDFs:

Image 18

For users planning on-premises deployment, OXR is already compatible with mainstream inference frameworks. There’s no need to manually recompile underlying inference engines—services can be launched with a single command via official Docker images or the CLI.

Currently, OXR’s model weights, source code, and full benchmark reproduction pipelines are completely open-sourced:

Image 19

For advanced format support (such as Word, PPT, and Excel processing) and deep customization capabilities, the team also offers an enterprise-grade document engine called SoMark:

Image 20

One More Thing

Why is document parsing still an area worth deep investment today?

As AI agents rapidly reshape productivity workflows, the deepest, most critical proprietary business knowledge accumulated across industries over past decades still primarily lives in inconsistent, intricately formatted PDFs, scans, and spreadsheets.

If the input side is plagued with garbled characters, misaligned lines, and fragmented tables, even the most powerful downstream LLM will suffer from hallucinations and broken logic.

From traditional OCR to next-gen OXR, the mission of this technology has quietly evolved: it serves as the structural bridge linking human physical documents to LLM understanding, designed to losslessly transform complex, unstructured visual content into high-quality digital assets fit for agent comprehension and consumption.

In an era overflowing with new tech narratives, solving real-world long-tail engineering problems often lays a far sturdier foundation. As underlying document data parsing solutions mature, the foundational puzzle pieces of the Agent era will rapidly fall into place.