OpenBMB

OpenBMB (Open Lab for Big Model Base) aims to build foundation models and systems towards AGI.
Connect with us:

[OpenBMB @OpenBMB]

🚀 Introducing MiniCPM-V 4.7, our new open-source MLLM series!
Advancing the Performance–Efficiency Pareto Frontier.
It brings strong multimodal capabilities to iOS and Android devices with extreme efficiency.
Two new models:
🔹 MiniCPM-V 4.7 35B-A3B — Bring SOTA performance to local devices with newly introduced LES (Lookahead Expert Scheduling) technique, a new milestone of on-device multimodal AI.
🔹 MiniCPM-V 4.7 1B — Lightweight, efficient multimodal AI. Easier to deploy.

⚡ What's new?
SOTA performance: Competitive results across 34 benchmarks spanning multimodal understanding, STEM reasoning, OCR and long-video comprehension.
Fast on-device inference: LES predicts and preloads expert weights to reduce inference stalls, boosting 5× speedup on iPad Pro M4 (16GB) with byte-for-byte identical outputs. It enables MiniCPM-V 4.7 35B-A3B to achieve 17+ tokens/s on iPad Pro M4 (16GB) and 13+ tokens/s on iPhone 17 Pro Max, while costing only 3GB memory 🤯
Efficient visual processing: 4× and 16× hybrid visual token compression. Even at 16× compression, 35B-A3B scores 90.4 on OCRBench, 5.2 points ahead of Qwen3.6 35B-A3B.
Easier customization: We introduce Self-Replay Fine-Tuning, a simple way to customize new target capabilities at low cost while maintaining general performance.
Broad hardware support: Day-0 adaptations for Apple, Intel, AMD, Arm and Rockchip, with many more platforms supported through FlagOS!

Download, deploy, and build something we haven't imagined yet. 👀👇
🔗 GitHub: https://t.co/msjm2QlrvX
🤗 Hugging Face: https://t.co/W95QKGhymw

[OpenBMB @OpenBMB]

@PaulGugAI You absolutely deserve the recognition! 🥹💙 Seeing passionate developers like you push the boundaries of what's possible on limited hardware is exactly what inspires us. Thank you for being part of the MiniCPM community. Keep building amazing things! 🚀��

[OpenBMB @OpenBMB]

@jumanzi_dev Exactly! 🙌 LES helps reduce expert-loading stalls without changing the actual expert routing. Long multi-turn conversations are a great stress test, especially with growing KV cache demands. We'd love to see more community benchmarks on this! 🚀💙

[OpenBMB @OpenBMB]

📱 A 35B MoE model running on a phone at over 20 tokens/s.

Meet Lookahead Expert Scheduling (LES), the inference technology behind MiniCPM-V 4.7’s efficient on-device performance.

🧠 The challenge: 35B total parameters, 3B activated per token, and limited device memory. Keeping experts in flash storage makes the model fit, but loading their weights can leave computation waiting.

LES co-optimizes expert loading, compute scheduling, and weight access:

⚡ Expert prefetch and compute scheduling: Cross-layer routing prediction prefetches experts the next layer is likely to need. Experts selected by the actual router that are already loaded compute first, while the remaining required experts load in parallel. Prediction only guides prefetch and scheduling: every required expert is still loaded and used. No additional training is needed.

💾 Weight access optimization: Repack expert weights into contiguous storage to reduce scattered reads, and apply precise layer-wise mixed-precision quantization to reduce weight storage and transfer costs while preserving model capability.

We also adapted DSpark speculative sampling to work alongside LES in the on-device inference pipeline.

In our iPhone 18 Pro Max tests, the full system reaches 20.51 tokens/s, up from a 3.38 tokens/s on-demand loading baseline: 6.07× faster.
🔗 https://t.co/kFD3X4RN0P

The original model architecture and expert routing stay intact. Smarter scheduling makes larger models practical on-device.

👉 MiniCPM-V GitHub: https://t.co/msjm2QlrvX
🤗 Model info: https://t.co/W95QKGhymw
💻 Technical Details: https://t.co/6Ul7qXCgjs

#MiniCPM #OnDeviceAI #OpenSource

[OpenBMB @OpenBMB]

35B multimodal AI. Running locally on an iPhone. Now at over 20 tokens/sec. 🚀

MiniCPM-V 4.7 achieves 20.51 tokens/sec average decoding speed on iPhone 18 Pro Max (12GB) — a 6.07× speedup over the same device’s unoptimized SSD expert-loading baseline.

⚡ Behind the speed: Lookahead Expert Scheduling (LES), optimized expert loading, fine-grained quantization, and DSpark speculative decoding.

📱 Watch the new on-device demo below. Powerful multimodal AI, right in your pocket.

Try it 👇
🔗 GitHub: https://t.co/msjm2QkTGp
🤗 Hugging Face: https://t.co/W95QKGh0wY
LES: https://t.co/6Ul7qXBItU

#MiniCPM #OpenBMB

[OpenBMB @OpenBMB]

@Vanshdhawan07 @mustafaergisi Nice breakdown! 🧠🙌 Quantization plays a big role in making small models practical on everyday devices. And as you pointed out, weights are only part of the memory story—KV cache and image processing matter too! 📱🚀

[OpenBMB @OpenBMB]

🚀 MiniCPM-V 4.7 1B is bringing open-source multimodal AI to everyday iPhones!

Developer @mustafaergisi shared that the model is already running in OnDeviceLLM on everyday iPhones, beyond the 12GB Pro Max. 📱

What does that look like in practice?
🧾 Read receipts and ask about the total
📃 Pull information from flyers
🖼️ Understand screenshots
✈️ Run offline, even in Airplane Mode
🔒 Keep image processing on the device

This is a compelling example of what small multimodal models can make possible: useful visual understanding on the phone you already carry.

For developers exploring on-device AI, MiniCPM-V 4.7 1B offers a concrete example of an open-source model finding its way into a real app. 🛠️

Great to see the community turning MiniCPM-V into everyday experiences. 🙌

🚀 What will you build with MiniCPM-V 4.7? We welcome your apps, demos, and real-world use cases. Share your project and tag @OpenBMB!

👉 Check out the app: https://t.co/4IT6T8D2tj
🔗 MiniCPM-V GitHub: https://t.co/msjm2QlrvX
🤗 Model info: https://t.co/W95QKGhymw
#MiniCPM #OnDeviceAI #OpenSource

[OpenBMB @OpenBMB]

@achaosxyz @mustafaergisi Great question! 👀 Balancing visual understanding with low latency is a key challenge for on-device multimodal AI. OCR accuracy, complex layouts, and efficient inference all matter. It's exciting to see developers pushing these boundaries! 🚀💙

[OpenBMB @OpenBMB]

@Yuan1197783 @mustafaergisi Fair point! 🙌 Dense tables can be challenging for smaller vision models, especially with complex layouts and tiny text. We're always looking to improve, and we'd love to see some challenging examples from the community! 💙

[OpenBMB @OpenBMB]

@EmbedLearner @mustafaergisi Great question! 📱🚀 MiniCPM-V 4.7 1B is open-source, and llama.cpp is one option worth exploring for Android deployment. We'd love to see more community demos on Android! 💙

[OpenBMB @OpenBMB]

Meet StaffDeck 1.0.0-preview — an open-source platform for building, executing, and operating AI digital employees.

The new release brings a modular Harness v3, a CLI that connects agent workflows to your terminal, and async execution with traceable results — making it easier to move from agent setup to reliable, repeatable work.

⚙️ Modular Harness v3 to compose and extend agent capabilities
⌨️ CLI + API to bring digital employees into developer workflows
🔄 Async execution with traceable task status, results, and resumable SOPs
🤝 Human-in-the-loop to hand off tasks and resume execution

Build your own digital employees and put them to work.

🔗 GitHub: https://t.co/OipKO48f82
📖 Installation: https://t.co/wvR2CKHwVF
💻 CLI Guide: https://t.co/bl7UTbitGf

[OpenBMB @OpenBMB]

Congratulations again to all the winners, and thank you to everyone who contributed projects, code, and lessons learned!

Building with MiniCPM? Share your setup, model version, and real-world results with the OpenBMB community. Your experiment could help another developer take their next step.

S1 is complete. The exploration continues. See you next season!

👉 GitHub: https://t.co/FxKaFMW2cu
🤗 Hugging Face: https://t.co/fW2ghL9C2z
👀 Docs: https://t.co/Jz40tYMGdX

#MiniCPM #OpenBMB #OpenSource #EdgeAI

[OpenBMB @OpenBMB]

Small models, real-world impact. The MiniCPM Spark Program S1 challenge has wrapped up with 16 award-winning projects from developers across China, Spain, Italy, Australia, and the US! 🎉

Congratulations to our three Special Award winners, who will each receive a 1-year Typeless membership! 🏆

🏆 Special Awards
• Dubedo @dubedostudio
• ESP32-S3 @emwstudio
• Personal Admin Agent Evaluation @PaulGugAI

More projects were recognized with Spark Case Awards and Blazing Spark Case Awards. Head to our Discord to see the full list of winners! 🔥
https://t.co/d13hUMNzqb

From practical applications to deployment tools and training research, these projects show what thoughtful engineering can do: choosing the right inputs, connecting components, pushing limits, and sharing what actually works.

Congrats to all the winners, and thanks to everyone who shared their projects, code, and learnings!

Here are 5 projects that caught our eye. 🧵👇

[OpenBMB @OpenBMB]

Training experiments others can build on 🧪

Neko-OPSD explores self-distillation with MiniCPM5-2B, sharing training data, methods, scripts, and 12,400 evaluation responses across five experimental runs.

The goal: make the experiments easier to inspect, reproduce, and build on.

🔗 Github : https://t.co/l6R69ZIB9z

[OpenBMB @OpenBMB]

Thanks for the clear breakdown! 🙌
Yes — MiniCPM-V 4.7 35B-A3B is the first open-source MoE vision-language model that runs smoothly on-device without compromising performance. With Lookahead Expert Scheduling (LES), fine-grained layer-wise quantization, and DSpark speculative decoding, it achieves 13+ tokens/s on an iPhone 17 Pro Max while using only about 3 GB of memory.
update: https://t.co/kFD3X4Rfbh
MiniCPM-V 4.7 achieves 20.51 tokens/sec average decoding speed on iPhone 18 Pro Max (12GB) — a 6.07× speedup over the same device’s unoptimized SSD expert-loading baseline.

[OpenBMB @OpenBMB]

35B multimodal AI. Running locally on an iPhone. Now at over 20 tokens/sec. 🚀

MiniCPM-V 4.7 achieves 20.51 tokens/sec average decoding speed on iPhone 18 Pro Max (12GB) — a 6.07× speedup over the same device’s unoptimized SSD expert-loading baseline.

⚡ Behind the speed: Lookahead Expert Scheduling (LES), optimized expert loading, fine-grained quantization, and DSpark speculative decoding.

📱 Watch the new on-device demo below. Powerful multimodal AI, right in your pocket.

Try it 👇
🔗 GitHub: https://t.co/msjm2QkTGp
🤗 Hugging Face: https://t.co/W95QKGh0wY
LES: https://t.co/6Ul7qXBItU

#MiniCPM #OpenBMB

[Jurly @jurlycat]

A 35B vision-language model running locally on a phone with about 3GB of memory.

OpenBMB reports MiniCPM-V 4.7 reaching 13+ tokens/sec on an iPhone 17 Pro Max.

It’s a MoE with roughly 3B active parameters per token. Expert weights stream from storage as needed.

Lookahead Expert Scheduling predicts which experts the next layer will need and preloads them while the current layer computes. If a prediction misses, the required experts still get loaded and evaluated.

That works alongside layer-wise quantization and DSpark speculative decoding.

Open-source, with on-device deployment support for iOS, Android and HarmonyOS.

The clip below is their iPad M4 demo, recorded directly on the device.

[OpenBMB @OpenBMB]

@mustafaergisi Really like your project and appreciate your support! 🙌 Love seeing this kind of excitement around on-device AI. More coming soon 👀

[OpenBMB @OpenBMB]

@LIJaByXa @aiwithsally Thanks for your interest! 🙌 We’re excited to see more ways to run MiniCPM-V 4.7 on Android. Stay tuned for updates!

[OpenBMB @OpenBMB]

From data and charts to full reports, running locally. Great to see @aiwithsally putting MiniCPM-V 4.7 to work in an automated reporting workflow. Thanks for sharing the demo!

[Sally Stockholm @aiwithsally]

I built an automated report generator that runs entirely on my own machine.

Input: structured data + charts.
Output: a full long-form report — earnings interpretation, weekly summaries, whatever you need.

No API. No cloud. No monthly bill. ->

Here's MiniCPM-V 4.7 35B-A3B doing exactly that 👇

It read the charts AND the dense tables in one pass, then streamed out analyst-grade markdown: scorecard, segment breakdown, margin drivers, cash flow, even a bull/bear case.

The part that got me: it didn't just OCR numbers. It connected them — input costs → margin pressure → guidance risk. That's the jump from "text extraction" to "actual analysis."

Why this works locally:
→ 35B total params, only ~3B active (MoE)
→ Efficient visual encoding keeps multi-chart inputs cheap
→ Structured markdown output — goes straight into your doc pipeline

Try it
GitHub: https://t.co/WrzyrPH0ub
Hugging Face: https://t.co/AsTjfpbUnn