Skip to content
Main Site News Console

Constant-Size KV Cache! A New Open-Source OCR Benchmark: Unlimited Pages, 96% Cache Reduction

· 量子位
国内AI

The OCR 3.0 Era Has Arrived

Baidu’s OCR technology has taken another giant leap forward: Wenxin Unlimited OCR 2 is officially released and open-sourced!

This time, it takes on the ultimate challenge: reading entire epic book series in a single breath.

From The Sorcerer’s Stone to The Deathly Hallows, all 7 volumes of the Harry Potter series were recognized and transcribed continuously—breezing through over 3,600 pages seamlessly from start to finish.

Switching to the entire The Three-Body Problem trilogy—spanning over 1,200 pages and hundreds of thousands of words—it accomplished the same feat effortlessly, without a single pause or interruption throughout.

Yet, continuously processing such a massive volume of pages requires overcoming a formidable barrier: GPU memory (VRAM).

The longer the document, the more intermediate context (KV cache) must be retained during inference. Think of it as a “cheat sheet” the model jots down while reading—the more pages, the thicker the cheat sheet gets. Not only does this consume immense VRAM, but flipping back and searching through it becomes exceptionally slow.

In the official technical report’s single-sequence benchmark on hundreds of pages, Wenxin Unlimited OCR 2 demonstrated astonishing numbers:

Compared to standard decoding—which keeps the visual cache of all pages resident in GPU memory—its peak KV cache footprint plunged from 47 GB to just 1.77 GB, a massive reduction of roughly 96%.

What does 47 GB mean in perspective? It exceeds the total system RAM of many mainstream laptops. More crucially, that is just the KV cache alone, not even factoring in model weights and other memory overhead.

In fact, the first-generation Unlimited OCR had already built massive popularity in the open-source community thanks to its outstanding continuous multi-page parsing capabilities: racking up over 10 million downloads and more than 28,600 GitHub stars. From community-built online demos to high-performance inference integration with SGLang, the surrounding ecosystem has been steadily expanding.

With the second generation, research and development shifted focus to a core objective: dramatically lightening the computational and memory burden while handling significantly longer documents.

Image 3

To achieve this, Wenxin Unlimited OCR 2 introduces Sparse-Reference Sliding Window Attention (SR-SWA), enabling the model to dynamically pinpoint only the relevant pages and local visual details on demand, eliminating redundant computation.

Paired with its continuous decoding mechanism, the model processes ultra-long documents smoothly without single-step decoding overhead escalating as document length grows.

Wenxin Unlimited OCR 2 is now officially open-sourced on Hugging Face and GitHub.

Open-Source Links:


Core Tech Breakdown: How SR-SWA Cuts Computation, and Chunked Decoding Saves Memory

How does Wenxin Unlimited OCR 2 manage to power through millions of words across 3,600+ pages of Harry Potter in one go?

Its core inspiration actually comes from the way humans transcribe books.

When someone copies a manuscript by hand, the workflow is typically: read a passage, copy a passage, keeping eyes primarily focused on the current page being written. Only when cross-referencing context is necessary do they flip back to specific relevant pages.

Human intuition here is clear: no one copying The Analects of Confucius spreads every single page of the entire book across their desk all at once.

However, the first-generation Unlimited OCR operated quite differently: although R-SWA drastically curtailed the KV cache during inference, all input visual features still had to participate in attention computation whenever looking back.

As a result, longer documents introduced heaps of redundant information into the calculation—much of which had zero relevance to the text currently being recognized, acting as sheer computational deadweight.

Image 4
△ SR-SWA draws inspiration from the localized focus humans use when transcribing books

Following this logic, Wenxin Unlimited OCR 2 introduces Sparse-Reference Sliding Window Attention (SR-SWA).

Its core mechanic relies on a “two-tier filtering” process: first selecting which pages to reference, then extracting genuinely relevant local details from those selected pages, shutting out all irrelevant computation.

Structurally, Wenxin Unlimited OCR 2 inherits the foundational design framework of its predecessor.

Image 5
△ Wenxin Unlimited OCR 2 Overall Architecture

Input page images first pass through DeepEncoder, where they are encoded into visual tokens digestible by the model—effectively generating index cards packed with textual, visual, and layout features. The language model then relies on these tokens to generate text recognition outputs step-by-step.

The key breakthrough happens inside the language model itself: the original Reference Sliding Window Attention (R-SWA) has fully evolved into SR-SWA.

The bottom-left of the architecture diagram illustrates SR-SWA’s two-tier filtering pipeline:

  1. Tier 1: Page Gate — Controls “which pages to look at”
    Based on the model’s current parsing progress, it restricts candidate visual features strictly to highly relevant pages, filtering out unrelated ones. In the book-copying analogy, it’s like flipping back only to the few related pages rather than scanning the entire book indiscriminately.

  2. Tier 2: Token Sieve — Discerns “which lines to look at within the page”
    Narrowing down pages alone isn’t enough. Even within relevant pages, there’s no need to involve every single visual token in computation. Token Sieve performs a secondary fine-grained pass, picking out the visual information blocks that best match the current context, precisely capturing long-range visual dependencies.

The right side of the diagram details this process: the model first calculates relevance weights based on compact representations of each block, then uses a Top-K strategy to select a fixed number of high-scoring blocks. Once selected, the model digs deep into fine-grained visual details rather than relying on coarse summaries.

Image 6
△ Details of Token Sieve

These two filtering layers work in synergy: quickly locking onto key pages within massive documents while drastically slashing the volume of visual information processed at each step, significantly accelerating long-document parsing efficiency.

The selected visual information then works alongside the Local Window shown in the architecture diagram during generation. This window maintains a fixed size, always holding the most recent context and sliding continuously as generation progresses to ensure seamless text output.

Because both the number of visual blocks and the local window size are strictly bounded, the information load required for each inference step remains capped within a steady, predictable range. This fundamentally resolves the computational bottleneck where “thicker documents mean increasingly sluggish generation.”


If SR-SWA solves the problem of “computing less” at each step, deciding “how much to store” in GPU memory addresses the other critical pain point.

Each encoded document page turns into a set of visual tokens. Because the model must reference these features during generation, they are stored in the KV cache for later reuse.

Loading an entire document at once and keeping all page visual caches resident in GPU memory inevitably causes memory consumption to explode linearly with page count. Worse still, even when earlier pages are no longer referenced, they continue occupying memory unless explicitly freed—much like leaving piles of discarded drafts on your desk while copying a manuscript; no matter how large the desk, it will eventually overflow.

Therefore, “computing less” must be complemented by “storing less.”

To solve this, Wenxin Unlimited OCR 2 introduces the Unlimited-Horizon Decoding Paradigm, achieving precise, controllable cache management through chunked streaming.

Image 7
△ Chunked Decoding Diagram

This mechanism partitions long documents into manageable page chunks processed sequentially. A 1-page overlap is maintained between adjacent chunks—where the last page of the preceding chunk serves as the first page of the subsequent one, acting as a natural “bookmark.”

When transitioning to the next chunk, the model directly reuses the visual KV cache of this overlapping page and retains the text cache from the end of the previous output, guaranteeing a seamless contextual transition. Meanwhile, visual caches from earlier chunks that are no longer needed are freed immediately, reclaiming GPU memory on the fly.

In this way, resident visual cache in VRAM is strictly bounded to the current individual chunk. Once a chunk is completed, only essential bridging information is preserved before moving to the next, completely shedding the crushing burden of holding the entire book’s cache throughout.

SR-SWA slashes per-step computational overhead, while the Unlimited-Horizon Decoding paradigm caps peak GPU memory usage. Together, they enable the model to effortlessly conquer continuous parsing across massive, marathon-length documents.


Reading the 884-Page The Lord of the Rings in 42 Minutes, Cache Dropped to 1.77 GB

Beyond architectural discussions, how does it perform in practice? The team fed Wenxin Unlimited OCR 2 an 884-page English edition of The Lord of the Rings.

Image 8

In about 42 minutes and 56 seconds, the entire epic was fully parsed. 884 input pages yielded exactly 884 output pages, averaging roughly 2.9 seconds per page—about the time it takes to sip a cup of coffee or watch a TV episode, the vast expanse of Middle-earth was completely digitized.

For a text volume this large, does recognition accuracy degrade midway through?

The team sampled and verified the text from pages 600 and 750: the output matched the original source text verbatim. Delivering flawless precision even at page 750 confirms that the model remains rock-solid and stable throughout ultra-long inference chains, showing no signs of hallucination or drift.

Image 9

Throughout the test, the system accurately identified and resolved 466 cross-page merges. For instance, between pages 873 and 874 near the end of the book, long sentences split abruptly by physical page breaks were stitched back together seamlessly in the final output.

Image 10

While the live test validated real-world engineering readiness for end-to-end book parsing, the technical report further quantifies key metrics across parsing efficiency, cache control, and accuracy.

On MarathonDocBench, a multi-hundred-page evaluation benchmark, the team compared standard full decoding against Unlimited-Horizon Decoding. In single-sequence inference tests, introducing chunked processing with boundary cache reuse slashed peak KV cache from 47 GB to 1.77 GB—a reduction of ~96% (Note: 1.77 GB accounts only for KV cache; static memory like model weights is calculated separately).

Beyond single-stream optimization, concurrent multi-task performance was equally impressive. In a 16-way concurrency test, the new architecture boosted end-to-end system throughput from 360 tokens/s to 1,315 tokens/s—3.65× that of standard decoding mode.

Image 11
△ Decoding Comparison: Single-Sequence Peak KV Cache vs. 16-Way Concurrency Throughput

This is a game-changer for enterprise scenarios handling bulk financial reports and technical engineering manuals: lower VRAM requirements mean the same hardware can sustain higher concurrent throughput, multiplying productivity per unit of time.

For end users, the experience is radically streamlined: simply upload a complete multi-hundred-page document. Chunk orchestration and boundary stitching are handled autonomously by the model’s underlying architecture, eliminating tedious manual slicing and post-processing alignment.

While pursuing peak performance and storage optimization, recognition quality remains uncompromising.

On MPDocBench, which evaluates multi-page complex document parsing, Wenxin Unlimited OCR 2 achieved an overall score of 84.59.

To assess ultra-long documents spanning hundreds of pages, the research team established a brand-new stress benchmark: MarathonDocBench. This dataset aggregates 100 lengthy documents ranging from 103 to 300 pages, packed with intricate multi-page paragraphs and sprawling cross-page tables that heavily test a model’s contextual continuity over long spans.

In this rigorous benchmark, Wenxin Unlimited OCR 2 earned a composite score of 77.06. Across both authoritative benchmarks, it ranked first among all models evaluated in the report.

Image 12
△ Technical Report Results: Multi-Page Document Parsing Evaluation

In terms of decoding speed, SR-SWA’s single-step decoding speed reached 7.0× that of Full Attention and 5.6× that of previous-generation R-SWA.

The performance curve clearly illustrates the difference: traditional approaches see single-step decoding latency surge steeply as document length increases, whereas SR-SWA’s latency curve remains virtually flat.

This means that no matter how thick the book is, the time the model spends generating each token remains practically constant—truly achieving “boundless depth without slowdown.”

Image 13
△ Single-Step Decoding Latency Across Different Document Lengths

Compared to the previous generation, Wenxin Unlimited OCR 2 maintains its high standard on standard single-page parsing, but its true disruptive leap emerges as document scale grows: continuous page processing capacity jumps by orders of magnitude, cross-page logical resolution becomes significantly sharper, and the overall cost of long-document inference is dramatically reduced.

Image 14

Whether it’s hundreds of pages of corporate annual audit reports, dense industrial specification manuals, or voluminous scanned archives, users can hand them directly to the model for parsing without complex pre-splitting or alignment cleanup.

Image 15


The OCR 3.0 Era Has Arrived

From the earliest single-receipt recognition to continuous, end-to-end parsing of entire books, the document understanding paradigm is undergoing a fundamental transformation.

In the technical report, the team categorizes the evolution of OCR into three pivotal eras: OCR 1.0, 2.0, and 3.0:

  • OCR 1.0 Era: Built on modular pipelines that step through text detection, recognition, layout analysis, and formatting reconstruction sequentially. While decoupled, this approach inevitably compounds errors down the pipeline.
  • OCR 2.0 Era: Vision-language multimodal foundation models took center stage, directly parsing input images via end-to-end architectures and eliminating information loss between legacy modules. The first-generation Unlimited OCR was a hallmark of this era, expanding the parsing frontier to dozens of pages.

Image 16
△ AI Generated

  • OCR 3.0 Era: As requirements scale to hundreds or thousands of pages, systems must not only maintain airtight cross-page semantic coherence, but also overcome the dual bottlenecks of compute explosion and memory bloat.

Harnessing on-demand localized attention (SR-SWA) and the Unlimited-Horizon Decoding paradigm, Wenxin Unlimited OCR 2 ensures compute does not blow up with page count, and memory does not bottleneck as length expands.

The team officially defines this as OCR 3.0: setting a new technical benchmark with continuous, unsegmented, and boundless full-book parsing.

Interestingly, the naming of this technical endeavor also carries an homage to anime/geek culture.

“Unlimited OCR Works” is a tribute to “Unlimited Blade Works” from the classic work Fate/stay night.

Image 17
△ Fate/stay night Official Anime Website Homepage

In the lore of the original work, “Unlimited Blade Works” is an innate bounded field that unfolds an infinite landscape of the mind. Following the release of the first-generation model, Hugging Face community developer Snarcy was inspired to pen a geek-romantic OCR spin on the iconic chant.

In the technical report for Wenxin Unlimited OCR 2, the research team formally incorporated this community tribute into the introduction, complete with an attributed acknowledgment.

Image 18

And so, on the opening pages of a technical paper examining cutting-edge attention mechanisms and GPU memory bounds, readers are greeted with this solemn, poetic declaration:

Pixels are my body, and tokens are my blood.
(身化像素,血铸符元。)