ModelBest Open-Sources MiniCPM-V 4.7
Jin Lei reporting from the Office
QbitAI
Wait, a 35B MoE running butter-smooth on a smartphone?
You heard that right.
Completely offline, we uploaded a classic data visualization chart to an iPhone 17 Pro Max running the on-device model—the famous marching map of Napoleon’s Russian campaign:

Then, we asked it directly:
What is the lowest temperature in Celsius recorded on the chart, and on what date did it occur? Please also explain the conversion process.

In less than a minute, the locally running model accurately pinpointed the -30° annotation along the route, identified the corresponding date as December 6, 1812, and converted it using 1 °Ré = 1.25 °C to arrive at the precise answer: -37.5 °C.
What’s more, generation speed on this on-device model can get even faster.
For example, when presented with the same photo of a curbside parking sign and the same question regarding parking duration, the model on the left is the unoptimized 35B model, while the right shows the optimized version:

Comparing the two, the unoptimized model generated tokens at roughly 2 tokens/s. After deep optimization, the generation speed surged to 8–12 tokens/s. Even without an internet connection, responses stream out rapidly.
Reportedly, when running on an iPhone 17 Pro Max with only ~3GB of RAM usage, this 35B model achieves a decoding speed of over 13 tokens per second.
The smoothness is genuinely impressive.
So, what is the story behind this model that runs so effortlessly on mobile devices?
It is MiniCPM-V 4.7, officially open-sourced by ModelBest and OpenBMB. This release includes two size variants:
- 35B-A3B: Features 35B total parameters with only 3B activated per token inference—making it the first MoE model in the MiniCPM-V series. It requires only ~3GB of memory to run smoothly across iOS and Android devices.
- 1B: With an actual size of 1.3B parameters, this variant focuses on an ultra-compact footprint while standing in the top tier among models of comparable size across OCR, mathematical reasoning, and video understanding.

Beyond raw speed, how does MiniCPM-V 4.7 perform overall in the realm of on-device multimodal LLMs?
In general visual understanding, 35B-A3B scored 91.72 on MMBench and 80.67 on MMStar. In mathematical reasoning, it reached 80.22 on MMMU and 88.00 on MathVista. In video understanding benchmarks, it secured 76.40 on Video-MME and 68.90 on LongVideoBench.
Notably, in the OmniDocBench v1.6 document understanding benchmark, 35B-A3B scored 94.24 points, delivering a clear lead of more than 12 points over comparable models that typically hover around 80 points.

Even more remarkable is its performance with 16x visual compression enabled: MMBench scores only dipped slightly from 91.72 to 90.87, and OCRBench dropped from 91.20 to 90.40. It significantly slashes on-device compute overhead while sacrificing almost zero accuracy.
Furthermore, in text-only capabilities, MiniCPM-V 4.7 achieved 90.21 on CMMLU (a comprehensive Chinese commonsense benchmark) and 84.65 on τ2-Bench for tool-use evaluations, leading comparable benchmark leaderboards as well.

MiniCPM-V 4.7 is reported to be the industry’s first open-source MoE model capable of running smoothly on edge devices while maintaining lossless, high-tier performance.
Now, let’s explore its capabilities across more real-world scenarios.
Fitting 35B in Your Pocket: Fully Usable Offline
Let’s start with a problem rooted in everyday life.
We provided the model with a household electricity survey sheet detailing the power wattage and usage duration for an air conditioner, refrigerator, electric kettle, TV, washing machine, and standby devices, alongside the monthly electric bill—totaling 420 kWh and 126 RMB.
The test consisted of three sub-tasks:
- Estimate and rank the monthly electricity consumption of each appliance;
- Calculate the proportion of total consumption accounted for by these appliances, and evaluate whether unlisted appliances could account for the remaining deficit;
- Compare three energy-saving strategies (running the AC 1 hour less per day, watching TV 2 hours less per day, and unplugging standby devices completely for 8 hours per day) to determine the most effective approach.

From the reasoning breakdown, 35B-A3B first converted watts to kilowatts and calculated monthly consumption item by item based on daily usage hours. The AC ranked first at 144 kWh per month. The refrigerator, based on its energy label rating of 280 kWh annually, was converted to roughly 23.33 kWh per month. These were followed by the kettle, TV, standby devices, and washing machine.
The model calculated that these six appliances consumed roughly 213.63 kWh in total, accounting for 64.09 RMB—only about 50.9% of the overall bill, leaving a gap of 206.37 kWh. Among the three energy-saving strategies, the model correctly identified that “running the AC 1 hour less per day” yielded the highest savings, cutting 24 kWh and saving 7.2 RMB per month.
Even in a strictly offline environment, 35B-A3B delivered a rigorous, accurate response to this multi-step chart reasoning challenge.
Having seen the 35B version, let’s look at the real-world performance of the 1B variant. This time, we asked the 1B model to convert an image of an academic paper page directly into Markdown:
Please transcribe this image into markdown.

In under 30 seconds, the model faithfully reconstructed heading hierarchies, body paragraphs, and table structures. Formulas were properly transcribed into LaTeX formatting, and the rendered layout closely matched the original literature.
For users who regularly handle paper screenshots and organize PDF materials, such tasks traditionally relied on cloud-hosted multimodal models. Now, a compact 1.3B model can handle them rapidly right on a mobile device.
Beyond smartphones, the model performs just as seamlessly on tablets like the iPad.
For example, when running the Napoleon campaign chart Q&A on an M4-powered iPad, the model still returned the exact conclusion—“December 6, 1812, -37.5 °C”—in seconds.

In addition to on-device deployments, we conducted multiple rounds of testing on MiniCPM-V 4.7 via API.
The first test evaluated structured receipt accounting. We provided a supermarket receipt containing six routine grocery items (ranging from whole milk and whole-wheat bread to facial tissues), complete with member discounts and coupon deductions (with a final payment of 78.40 RMB). The model was instructed to organize it into a standardized bookkeeping table and verify whether the monetary calculations were accurate.

The results showed that 35B-A3B accurately extracted the store name, transaction date, and the unit price, quantity, and billed amount for all six items without a single OCR error.
During the price verification step, even though the receipt explicitly listed a subtotal of 86.40 RMB, the model proactively summed the individual line items to double-check. Once verified, it deducted the 8 RMB discount to arrive at the final payable amount of 78.40 RMB, demonstrating complete logical consistency.
The second test focused on parking signage, but with far more intricate regulatory logic.
The test scenario was set as follows: Tuesday, vehicle holds a resident permit, has not parked on the street earlier that day, with a planned parking window from 14:10 to 15:40. The model was tasked with evaluating parking duration, potential street cleaning violations, time limits, required fees, and overall compliance.

35B-A3B correctly answered all five evaluation points: calculating a total parking duration of 90 minutes; verifying that the 14:10 start time neatly avoided the 12:00–14:00 street-sweeping restriction; ensuring it stayed within the 2-hour maximum stay; and noting that parking fees were waived due to the resident permit.
In its final summary, the model methodically mapped out rule boundaries, even explicitly noting that “because this is the vehicle’s first parking instance of the day, the ‘30-minute interval required for re-entry’ restriction does not apply,” ultimately determining that the parking plan was fully compliant.
How Does a 35B Model Run Smoothly on Just 3GB of RAM?
With the real-world benchmarks established, a key engineering question emerges:
How can a 35B parameter model run stably on a phone with only 12GB of physical RAM while sustaining high inference throughput?
35B-A3B utilizes a Mixture-of-Experts (MoE) architecture. Although its total parameter count reaches 35B, only around 3B parameters are activated during the generation of each token. However, deploying MoE models on edge devices poses unique engineering hurdles: which expert networks need to be activated at each layer can only be determined after inference computation on the preceding layer finishes.
Meanwhile, mobile RAM cannot hold all expert weights at once. The vast majority of parameters must reside in flash memory (NAND Flash) and be read on demand.
Previously, the industry largely relied on two paths: one was “dynamic loading on demand,” which allows the model to run, but frequent I/O wait cycles make decoding speeds too slow for real-time interaction; the other was “locking a static subset throughout the conversation,” which curtails I/O read operations but heavily restricts expert diversity, inevitably degrading inference quality.
MiniCPM-V 4.7 aims to preserve both execution speed and full-precision accuracy. In real-world engineering deployment, it also resolves a major developer pain point—enabling the model to master new downstream business tasks without suffering catastrophic forgetting of its general capabilities.
In terms of decoding acceleration:
Lookahead Expert Scheduling (LES) performs end-to-end co-optimization across expert prefetching, compute pipeline scheduling, and weight storage access: clearly defining when to transfer, execution timing, and data granularity.
The R&D team discovered an essential property: input features between adjacent layers in large models exhibit exceptionally high similarity. Consequently, while the current layer executes matrix operations, its routing features can be fed in advance into the next layer’s routing module. This enables early prediction of the expert subset most likely to be invoked by the subsequent layer, allowing an asynchronous background thread to prefetch the corresponding weights from flash storage into RAM.
By the time the next layer initiates forward computation, the majority of required expert weights are already cached in memory. Empirical benchmarks show that this approach accurately predicts and prepares over 80% of required expert weights in advance—all without incurring any additional training overhead.
The compute pipeline has also been redesigned: experts that are routed and already prefetched are computed first, while other required experts load concurrently. This maximizes overlap between compute and I/O, substantially reducing processor idle time spent waiting for data.
A core architectural advantage of this design is that the pre-scheduling mechanism handles only data movement and execution sequencing. The final combination of experts activated for each token strictly adheres to native routing decisions. Any expert missed by the prediction simply undergoes regular supplementary loading and compute phases, never sacrificing inference accuracy for single-dimensional speed gains.
Another layer of optimization stems from model weight quantization and compression. Through a layer-by-layer mixed-precision quantization strategy, the team significantly shrank expert weight footprints with virtually no performance degradation, sharply lowering data transfer volume per load. Concurrently, the physical storage layout of the weights was restructured to eliminate fragmented addressing and achieve high-efficiency sequential continuous reads.
Compressing data throughput and streamlining read/write paths, combined with predictive lookahead and duplex concurrency mechanisms, comprehensively alleviates hardware bottlenecks in memory capacity and bus bandwidth on edge devices.
On an iPad Pro (M4 chip) equipped with 16GB of RAM, Lookahead Expert Scheduling delivers up to a 5x boost in decoding throughput.
Additionally, the team trained a dedicated DSpark speculative draft model and seamlessly integrated it into mainstream edge inference frameworks such as llama.cpp. The draft model speculatively generates multiple candidate tokens ahead of time, which the primary model then batches and verifies, accelerating generation even further. In benchmark evaluations on an M5 Mac platform with 16GB of unified memory, decoding throughput saw an average additional improvement of 17%.
Overhead on the visual encoding side was similarly targeted. MiniCPM-V 4.7 adopts the intra-ViT early feature compression technique from LLaVA-UHD v4, pre-aggregating visual tokens within the visual encoder to drastically cut computational load across subsequent layers. Developers can flexibly toggle between 4x and 16x compression ratios; benchmarks indicate that even under 16x high compression, model accuracy remains virtually lossless.
The deep integration of multiple optimization techniques achieves what ModelBest describes as “speed on top of speed” for on-device AI.
Beyond inference latency, developers commonly encounter “catastrophic forgetting” during domain fine-tuning: tailoring models to downstream tasks often causes a noticeable decline in their baseline general intelligence.
To address this, ModelBest introduced Self-Replay Fine-Tuning. Its core logic operates as follows: prior to fine-tuning, the base model is prompted with a benchmark test set covering its core native capabilities to generate high-confidence, high-quality responses. These “self-generated replay samples” are then mixed into the downstream task dataset at a calibrated ratio. As the model assimilates new domain knowledge, it continuously reviews existing general knowledge, significantly curbing generalization degradation.
Because these “review corpora” originate directly from the model’s own output distribution, they harmonize naturally with its native expression patterns. The entire pipeline seamlessly reuses the standard SFT workflow without requiring architectural modifications or custom, complex loss functions. More importantly, the replay dataset only needs to be constructed once and can be reused across multiple fine-tuning runs and tasks.
Compared to compute-heavy paradigms like Online RL or On-Policy Distillation—which require frequent dynamic sampling during each training run—Self-Replay Fine-Tuning reduces the computational and engineering cost of domain adaptation down to standard Supervised Fine-Tuning (SFT) levels, all while robustly preserving general model performance.
When LLMs No Longer Demand High-End Hardware
Returning to the broader consumer electronics and mobile computing landscape:
In 2026, on-device AI is entering the phase of massive commercial deployment. According to IDC projections, shipments of next-generation AI smartphones in China will reach 147 million units this year, marking a 31.6% year-over-year increase and pushing market penetration past the 53% milestone for the first time. This means that out of every two new smartphones sold, one will feature deep on-device AI capabilities. ModelBest CEO Li Dahai also noted that 2026 marks the pivotal inaugural year for the large-scale rollout of on-device AI.
For smart hardware OEMs, however, the hardware Bill of Materials (BOM) cost remains an unavoidable practical constraint.
With memory prices experiencing ongoing volatility and upward pressure, 12GB has gradually become the baseline threshold for AI smartphones, 16GB has become standard across flagship lineups, and select premium devices are pushing into 24GB territory. Advancing model capability has traditionally relied on stacking memory capacity, yet soaring memory costs significantly hinder the widespread democratization of LLMs across full product tiers.
Consequently, powering models with larger parameter counts and stronger performance within a smaller memory footprint holds immense appeal for hardware manufacturers. On an iPhone 17 Pro Max equipped with 12GB of RAM, 35B-A3B runs smoothly while consuming only about 3GB of memory, leaving ample operating headroom for the operating system and other demanding foreground applications.
In the long run, the competitive landscape of on-device AI is shifting. While the initial phase focused primarily on raw silicon lithography and peak TOPS, going forward, the efficient synergy among on-device foundation models, underlying algorithms, and inference runtime software/hardware will serve as the true watershed defining user experience.
On the ecosystem front, MiniCPM-V 4.7 offers extensive multi-platform compatibility: providing full coverage for iOS, Android, and HarmonyOS, with day-one adaptation completed across mainstream hardware architectures including Rockchip, Houmo AI, Intel, AMD, and Arm. For inference engines, it provides comprehensive support for SGLang, vLLM, llama.cpp, and Ollama. On the fine-tuning and quantization toolchain side, it supports SWIFT and LLaMA-Factory while offering quantization formats including GGUF, AWQ, GPTQ, and FP8.
Underpinning this technical trajectory is the “Law of Large Model Density,” jointly formulated by ModelBest and Tsinghua University: the capability density of large models roughly doubles every 3.5 months, while the effective parameter count required to reach a given intelligence tier is dropping rapidly. As of September 2026, cumulative global downloads for the open-source MiniCPM family have officially surpassed 50 million.
Carrying top-tier, high-performance multimodal foundation models right in your pocket—and interacting with them smoothly—is becoming an everyday reality.