MiniCPM5-2B Operational

MiniCPM5-2B is an on-device language foundation model jointly open-sourced by ModelBest, the OpenBMB community, and Tsinghua University. With approximately 2.5 billion parameters, it supports a 128K context and achieves an average score of 53.9 across 34 benchmarks, surpassing many 4B-class models and reaching the open-source SOTA level for 2B models. The model brings Agent capabilities such as tool calling, deep search, and code generation to the edge, and can run on devices with 6GB of memory. It also open-sources its training data, the Meshy RL framework, and the JustRL II algorithm, making it a powerful small model for on-device use.

On-device conversational Q&A: Run the language model directly on local devices to handle everyday questions, writing, summarization, and other tasks without uploading data to the cloud.
Long-context understanding: Supports a 128K context, enabling local processing of meeting minutes, contracts, and long documents with accurate extraction of key information.
Code generation and repair: From code completion to repository-level fixes (SWE-bench Verified 46.4), it can serve as a local coding assistant.
Tool calling: Natively supports function calling and can invoke external tools as needed for queries, calculations, database interactions, and other operations.
Deep search: Provides search Agent capabilities, autonomously planning multi-step retrieval and synthesizing answers.
Hybrid reasoning: Use the enable_thinking switch to move between fast responses and deep-thinking modes, balancing efficiency with difficult problem solving.
Local batch processing: Privacy-sensitive tasks such as resume screening, exam grading, and contract analysis can be processed in parallel on the device at low cost.

Follow WeChat and reply “开源” to join the AI open-source project community
Standard Dense Transformer architecture: The model uses the standard LlamaForCausalLM structure with 42 layers and GQA to reduce KV Cache overhead. It natively supports a 131,072-token context. Because the architecture is fully standard, mainstream frameworks such as vLLM, llama.cpp, and Ollama can load it natively without custom kernels.
End-to-end Agent capability training: Agent capabilities are planned from the pre-training stage. First, 200B-scale Agent Midtraining injects priors for task decomposition and tool use. Then, 500,000 SFT trajectories covering tool calling, web search, code Agents, and other scenarios teach the model to execute tasks correctly. Finally, Agent RL alignment ensures stable policies during multi-turn interactions, forming a complete capability chain that both runs reliably and works well.
Meshy reinforcement learning framework: Meshy removes the central controller and Ray dependency from traditional RL training. It models training, inference, and reward calculation as peer node services, using data readiness in TransferQueue to drive the actual control flow. It can flexibly switch among synchronous, asynchronous, and fully asynchronous modes, making large-scale RL training easier to scale and lighter to deploy.
JustRL II reward learning algorithm: Targeting two major challenges in long-chain-of-thought RL for on-device models, namely reward signals steering the model off course and coarse credit assignment in GRPO, JustRL II uses a three-stage pipeline on the data side to calibrate training samples. On the algorithm side, it introduces a Critic into GRPO for token-level credit assignment, accurately distinguishing the contribution of each step in a long reasoning chain.
UltraX function-calling-based data refinement: UltraX abandons the traditional approach of having a large model rewrite web text end to end. Instead, it trains a lightweight 0.6B model to predict structured editing operations, which are then applied line by line by a deterministic executor. This avoids the semantic drift and structural damage caused by rewriting, while making the entire process traceable because editing operations can be saved and audited. In practice, training on 16B tokens of refined data exceeds the results of full training on the original 20B-token corpus.
UltraData tiered data governance system: Code data is governed progressively from L0 to L3. RL data passes through a three-stage pipeline that removes unverifiable samples, unreliable rewards, and mismatched difficulty levels. High-quality data signals compensate for the capacity disadvantage of a small model, providing the key engineering foundation for outperforming 4B models with fewer parameters.

Confirm hardware requirements: The original BF16 weights require approximately 4.7GiB, while the approximately 1.2GB GGUF quantized version can run on phones and standard laptops with 6GB of memory.
Obtain the model weights: Download the required version from Hugging Face or the ModelScope community, choosing among BF16, GGUF, MLX, GPTQ, SFT, and other formats as needed.
Install an inference framework: Use vLLM for cloud or high-concurrency scenarios (natively supported in ≥0.21.0; single-card TP=1 is sufficient). For lightweight local deployment, use llama.cpp, Ollama, or SGLang. CPU environments can use ModelBest’s self-developed Arclight framework.
Load and run the model: Load it directly using the standard Llama architecture without any custom kernels.
Switch thinking modes: Use the enable_thinking switch in the chat template to control fast responses or deep reasoning, selecting a mode based on task difficulty.
Integrate tool calling/Agents: Combine the model’s native function calling and deep search capabilities with the deployment Cookbook and Agent Skills to build local Agent workflows.

Open-source SOTA at the 2B level: An average score of 53.9 across 34 benchmarks, surpassing all tested 4B-class models and achieving more with less.
On-device general-purpose Agent foundation: Natively integrates core Agent capabilities such as tool calling, deep search, and code generation. It ranks first with a score of 20 on the Agent leaderboard, ahead of larger models such as Granite 4.2 8B and Qwen3.5 9B.
Exceptional deployment compatibility: The standard Llama architecture and availability in multiple formats allow vLLM, Ollama, and llama.cpp to load it directly without modification. The INT4 version requires only 1.2GB and runs on devices with 6GB of memory.
Native 128K long context: Its 130K-token context provides the foundation for processing long documents such as meeting minutes and contracts locally on edge devices.
Fully open source: Beyond the weights, the training data (UltraData series), RL framework (Meshy), RL algorithm (JustRL II), and training recipes are also available, enabling complete reproduction and further innovation.
Efficient inference: Each task uses approximately 21K output tokens on average, consuming fewer tokens than comparable models for faster, more power-efficient on-device operation.
Broad chip ecosystem: Day0 support covers 9 domestic chip platforms, including AMD, Intel, MediaTek, Qualcomm, Ascend, Hygon, Kunlunxin, and Moore Threads.

GitHub repository: https://github.com/OpenBMB/MiniCPM
Hugging Face model collection: https://huggingface.co/collections/openbmb/minicpm5

Privacy-sensitive document processing: Handle Q&A and summarization for meeting minutes, contracts, and internal emails locally, keeping data on the device throughout.
Recruiting and human resources: Screen resumes in batches locally, extract candidate information, and compare it in a structured format without uploading resumes to the cloud.
On-device coding assistant: Complete code, provide debugging suggestions, and even fix repository-level defects offline while protecting code assets.
Local deep search: Automatically plan multi-step web searches and synthesize answers for research, price comparison, and information gathering.
Education and grading: Grade exams, review assignments, and analyze mistakes in parallel batches locally, reducing costs for schools and educational institutions.