OpenBMB

⚡ Making MiniCPM5-2B even more lightweight for local inference! A community-built EXL3 4-bit quantization of MiniCPM5-2B brings the quantized model weights down to just 1.61 GB. ✨ Highlights: ⚡ 4.0 bpw EXL3 quantization for a smaller model footprint 🚀 ~68–70 tokens/s verified on an NVIDIA Tesla T4 💾 Just 1.61 GB for the quantized model weights 🛠️ Supports local inference with ExLlamaV3 and TabbyAPI A great community contribution showing how MiniCPM5-2B can be optimized for more lightweight and accessible local AI deployments. 🤗 Model: http://huggingface.co/ewin-reg/MiniCPM5-2B-EXL3-Quantized 🤗 Base model: http://huggingface.co/openbmb/MiniCPM5-2B

Post-training pipelines now use on-policy distillation (OPD) to hand a student the teacher's full next-token distribution at every prefix it visits: the student samples its own rollouts, and Qwen3, MiMo, GLM-5, DeepSeek-V4 and Kimi K3 all pair it with SFT and RL. Yet work on OPD has almost all stood on the algorithm side, treating the training set as given—so how much of OPD's gain does the data account for? Introducing One-Shot OPD, from @TsinghuaNLP (OpenBMB member) with the University of Chinese Academy of Sciences, Northeastern University, UIUC and Johns Hopkins University. It cuts the training set to one query, and the answer is that OPD is data-overfed but algorithm-starved. 1️⃣ One query, hundreds of steps. On math it goes from 59.1 to 68.5 by step 300, against 69.8 for full-data OPD—87% of its gain. It holds across code, instruction following and agentic tool use, and across Qwen, Llama and OLMo; a query the student never solves works about as well as one it always solves. 2️⃣ States, not questions. A prefix is a state where the teacher gives a target distribution, so 64 rollouts per step already yield tens of thousands of supervised positions. One query reaches 71.5% state coverage of what full-data OPD visits; 16 diverse queries reach 98.9% and match full data—extra queries buy new states, not new questions. 3️⃣Alignment slows, not the data. The student keeps improving, but each update absorbs less of what is left, which is why a run takes hundreds of steps rather than tens. This decline hardly depends on training-set size: on 1, 4, 16 and all 17k queries, alignment slowed at a similar pace. What limits a run is not how much data it gets, but how fast the student absorbs it. 📄 Paper: http://huggingface.co/papers/2609.04172 💻 Code: http://github.com/Thinking-Space/One-Shot-OPD #AI #THUNLP #OpenBMB #LLM #PostTraining #Distillation #OpenSource

🎙️ Real-time interpretation, powered locally by VoxCPM2. Developer @HenryZ30734018 built VoxWeft, an open-source simultaneous interpretation system for Apple Silicon. It uses an MLX implementation of VoxCPM2 to turn live speech into translated speech on-device, keeping audio private and responsive. ✨ Highlights: ⚡ VoxCPM2 streams first audio in ~170 ms on an M5 MacBook 🌍 Generates speech across 30 languages, supporting direct language-pair interpretation without a pivot 🗣️ Clones a target voice from ~5 seconds of reference audio for a consistent interpreted voice 💻 Runs in 4-bit quantization on MLX, making low-latency local speech generation practical on Apple Silicon VoxWeft shows how VoxCPM2 can become the speech layer of a full real-time application: not just producing audio, but enabling private, multilingual interaction that stays on-device. Try VoxCPM2 and see what you can build with it! 🔗GitHub:http://github.com/HenryZ838978/VoxWeft 🔗Devlog:http://github.com/HenryZ838978/VoxWeft/blob/main/docs/DEVLOG.md 🤗 VoxCPM2:http://huggingface.co/openbmb/VoxCPM2

🚀 FIT-GGUF brings controllable-size mixed-precision quantization to MiniCPM5-2B Developer @Scorp1o_117 used FIT-GGUF to build MiniCPM5-2B GGUF variants around specific size and quality targets. Instead of choosing a fixed quantization preset, you can set a target file size or fidelity tier, and FIT-GGUF automatically decides how much precision to allocate to different tensors—then predicts, generates, and verifies the final GGUF. ✨ What’s included 🧠 Tensor-level mixed-precision quantization 📦 Four MiniCPM5-2B builds from ~1.14 GiB to ~1.46 GiB 🎯 Quality / Balanced / Compact / Mini presets 📊 KL Divergence and Same-top evaluation ✅ Generated file sizes matched the predicted targets A nice example of how MiniCPM5-2B can be tuned for different memory and deployment constraints, without being locked into a single Q4/Q5-style quantization preset. Check out FIT-GGUF and try building a MiniCPM5-2B variant that fits your own device budget. 🤗Model: http://huggingface.co/SC117/MiniCPM5-2B-abliterated-FIT-GGUF/tree/main http://huggingface.co/openbmb/MiniCPM5-2B

Thanks so much for sharing this! Really cool to see MiniCPM5-2B being used in a practical multi-agent workflow like this — especially with the workers actually handling matching, short payments, duplicate references, and disputes through tool calls. Appreciate all the work you put into testing and documenting this. Such a nice case for the community 🙌

MiniCPM-o 4.5 is now in SGLang Omni v0.1.7. More flexibility for developers to run and build with the model.

This is awesome! Running the whole perception stack with a Pi Zero feeding MiniCPM-o4.5 in real time is a seriously cool use case. Love seeing the real-time interactions with the personal robot!

A reward of “3” can mean two completely different things. Everyone thinks a response is mediocre — or half the people love it while the other half hate it. Most Reward Models cannot tell the difference. Introducing Diffusion Reward Models (DRM): instead of collapsing human preference into a single score, DRM learns the full reward distribution, preserving disagreement, uncertainty, and multiple plausible judgments. ✨ Paper:https://arxiv.org/abs/2609.33803 🤗 Models: https://huggingface.co/Teburile/DRM 💻 GitHub: https://github.com/thunlp/DRM Why it matters: Human disagreement is structured, not just noise. On datasets with repeated annotations, judgments often form separated or polarized patterns. More importantly, as human disagreement increases, DRM’s learned reward distribution becomes increasingly multimodal. The distribution is useful, not just descriptive. DRM can use distributional uncertainty to identify unstable reward decisions, and distribution-aware ranking improves Best-of-N selection beyond simply taking the mean reward. Reward Models get their own test-time scaling. Instead of only spending more compute on generating more responses, DRM can keep the response fixed and sample its reward distribution more times. More reward samples give a more reliable estimate — a new scaling axis unavailable to deterministic scalar RMs. And the gains survive RLHF. When used as the training-time reward, DRM improves downstream policy performance over scalar reward baselines, showing that the benefit is not limited to offline RM benchmarks. The takeaway: Reward modeling may lose something important when it compresses every human judgment into one number. “Everyone thinks this is average” and “people strongly disagree about this” should not look identical to a Reward Model. DRM makes that difference visible — and usable.

Great to see MiniCPM-OCR and Jev being explored in community demos! 💙 Thanks for sharing this! 🙌

🚀 PilotDeck 2.0 is here! Meet the open-source AI Agent Harness built to get complex work done. Developed by Tsinghua THUNLP, @modelbest_ai, @OpenBMB , and AI9Stars, PilotDeck is a model-agnostic Agent operating system that turns natural language instructions into completed tasks — with transparent data flows and the freedom to choose your models. What’s new in 2.0? 📄 End-to-end office work: Create, edit, and verify Word, Excel, and PowerPoint files — not just generate drafts. 🔄 More reliable execution: complex, long-running tasks going through interruptions. Stress-tested with 2,000 interaction messages. 🛠️ Easy to get started: One-click desktop installation and a streamlined setup flow to connect your model and create your workspace. 🔓 Open by design: Fully open-source, model-agnostic, with data flows visible to users. On Workbuddy Bench, PilotDeck scored 79.1 with Qwen3.6-35B-A3B, outperforming WorkBuddy (74.7) and Claude Code (77.3). Build your own AI workspace for the Agent era. 🔗 GitHub: http://github.com/OpenBMB/PilotDeck Web: https://pilotdeck.openbmb.cn

📱 A 35B MoE model running on a phone at over 20 tokens/s. Meet Lookahead Expert Scheduling (LES), the inference technology behind MiniCPM-V 4.7’s efficient on-device performance. 🧠 The challenge: 35B total parameters, 3B activated per token, and limited device memory. Keeping experts in flash storage makes the model fit, but loading their weights can leave computation waiting. LES co-optimizes expert loading, compute scheduling, and weight access: ⚡ Expert prefetch and compute scheduling: Cross-layer routing prediction prefetches experts the next layer is likely to need. Experts selected by the actual router that are already loaded compute first, while the remaining required experts load in parallel. Prediction only guides prefetch and scheduling: every required expert is still loaded and used. No additional training is needed. 💾 Weight access optimization: Repack expert weights into contiguous storage to reduce scattered reads, and apply precise layer-wise mixed-precision quantization to reduce weight storage and transfer costs while preserving model capability. We also adapted DSpark speculative sampling to work alongside LES in the on-device inference pipeline. In our iPhone 18 Pro Max tests, the full system reaches 20.51 tokens/s, up from a 3.38 tokens/s on-demand loading baseline: 6.07× faster. 🔗 https://x.com/OpenBMB/status/2112529266537639962?s=20 The original model architecture and expert routing stay intact. Smarter scheduling makes larger models practical on-device. 👉 MiniCPM-V GitHub: http://github.com/OpenBMB/MiniCPM-V 🤗 Model info: http://huggingface.co/collections/openbmb/minicpm-v-47 💻 Technical Details: https://github.com/tc-mb/llama.cpp-omni/blob/master/docs/les/lookahead-expert-scheduling.md #MiniCPM #OnDeviceAI #OpenSource

🚀 Introducing MiniCPM-V 4.7, our new open-source MLLM series! Advancing the Performance–Efficiency Pareto Frontier. It brings strong multimodal capabilities to iOS and Android devices with extreme efficiency. Two new models: 🔹 MiniCPM-V 4.7 35B-A3B — Bring SOTA performance to local devices with newly introduced LES (Lookahead Expert Scheduling) technique, a new milestone of on-device multimodal AI. 🔹 MiniCPM-V 4.7 1B — Lightweight, efficient multimodal AI. Easier to deploy. ⚡ What's new? SOTA performance: Competitive results across 34 benchmarks spanning multimodal understanding, STEM reasoning, OCR and long-video comprehension. Fast on-device inference: LES predicts and preloads expert weights to reduce inference stalls, boosting 5× speedup on iPad Pro M4 (16GB) with byte-for-byte identical outputs. It enables MiniCPM-V 4.7 35B-A3B to achieve 17+ tokens/s on iPad Pro M4 (16GB) and 13+ tokens/s on iPhone 17 Pro Max, while costing only 3GB memory 🤯 Efficient visual processing: 4× and 16× hybrid visual token compression. Even at 16× compression, 35B-A3B scores 90.4 on OCRBench, 5.2 points ahead of Qwen3.6 35B-A3B. Easier customization: We introduce Self-Replay Fine-Tuning, a simple way to customize new target capabilities at low cost while maintaining general performance. Broad hardware support: Day-0 adaptations for Apple, Intel, AMD, Arm and Rockchip, with many more platforms supported through FlagOS! Download, deploy, and build something we haven't imagined yet. 👀👇 🔗 GitHub: http://github.com/OpenBMB/MiniCPM-V 🤗 Hugging Face: http://huggingface.co/collections/openbmb/minicpm-v-47