Xiaomi-OCR-0 Operational

Xiaomi-OCR-0 is Xiaomi's open-source 0.8B lightweight multimodal document understanding model, trained on Qwen3.5-0.8B. It integrates document parsing, key information extraction, and OCR visual question answering. It ranks first on Real5-OmniDocBench with 95.24 points, second on OmniDocBench v1.6 with 96.83 points, and surpasses comparable models with an average score of 83.2 across five OCR-VQA tasks. Xiaomi-OCR-0's data engine uses expert-consensus annotation and rendering verification, with supporting MCP and Agent Skill toolchains. It has been deployed in Xiaomi phones through “Super XiaoAi.”

Document parsing: Recognizes and reconstructs body text, complex tables, and mathematical formulas in documents, outputting structured Markdown/LaTeX results in natural reading order.
Key information extraction (KIE): For semi-structured documents such as invoices and cards, precisely locates specified fields and outputs structured data in JSON format.
OCR visual question answering: Goes beyond text transcription to support cross-element retrieval, numerical calculation, and semantic reasoning over documents and charts.

Follow WeChat and reply “Open Source” to join the AI open-source project community
Heterogeneous expert consensus annotation (ETC): Eight OCR expert models with different architectures, data, and strategies independently predict document regions. Pairwise agreement is calculated separately for text (NED), tables (TEDS), and formulas (CDM). The three-member combination with the highest consensus is selected, and its median is used as a pseudo-label. Only high-confidence samples with consensus ≥0.9 enter the training set directly, filtering out single-model bias at the source.
Render-Guided Refine-and-Judge: For regions where experts disagree, Qwen3.5-122B-A10B re-renders candidate parsing results as images and compares them with the original regions at the pixel and structural levels to identify subtle defects such as row or column misalignment and missing symbols. Corrections are iterated for up to 3 rounds. Cases that still fail to converge enter a second-stage “diagnose-refine-verify” process with a complete verification history, and are accepted only after returning to the expert pool to reach consensus again.
Sample mining and targeted synthesis: The training mix is determined jointly by three signals: expert consensus, model self-consistency, and K-Means semantic clustering. These signals are used to construct a broadly covered mixed set for CPT and a hard-example set for RL. Synthetic data follows two paths, “coverage-driven” and “failure-driven,” and a recipe is adopted only after its effectiveness is validated on a frozen validation set.
Progressive three-stage training: First, Q-Mask text anchoring pretraining establishes fine-grained alignment between textual content and spatial positions. Next, continual pretraining for multitask OCR expands coverage of full-page parsing, region recognition, VQA, and other tasks under the same autoregressive format. Finally, DAPO-style Mix-RL reinforcement learning on mined hard examples completes training, routing rewards by task type and using verifiable rewards to overcome limits in complex layouts and multi-step reasoning.

Get the code: Clone the GitHub repository SeerRay-Lab/Xiaomi-OCR-0 and install the dependencies listed in requirements.
Download the weights: Pull the 0.8B model weight files from Hugging Face at SeerRay-Lab/Xiaomi-OCR-0.
Quick start: Run the project's built-in script to launch the interactive browser Demo, then upload document images to try parsing, extraction, and question answering online.
Configure Agent integration: Connect the repository's MCP service or Agent Skill to your agent to automate the workflow of “recognize screen/document—extract fields—perform the next step.”
Deploy a local service: Use SGLang or vLLM to deploy the model as a local inference service for API calls from business systems.
Add layout detection: With a two-stage pipeline, first use PP-DocLayoutV3 to detect text, table, and formula regions, then send each region to Xiaomi-OCR-0 for recognition and assemble the page-level result.
Write instructions: Construct prompts in a consistent image–instruction–response format, specifying the parsing, KIE, or VQA task type.

Exceptional parameter efficiency: With only 0.8B parameters, it scores 96.83 on OmniDocBench v1.6. Specialized 0.8B–1.2B models comprehensively surpass general-purpose models such as GPT-5.2 and Qwen3-VL-235B.
Leading robustness on real-world captures: It ranks first on Real5-OmniDocBench with 95.24 points and leads across all five challenging conditions: scanning, curved paper, screen recapture, uneven lighting, and angled photography.
Three capabilities in one: Document parsing, key information extraction, and OCR visual question answering are unified in a single model. Tasks can be switched through a shared instruction interface without combining multiple specialized models.
Data engine instead of parameter scaling: 170 million samples are automatically produced through expert-consensus voting, render-guided verification, and failure-driven synthesis, trading data quality for model scale.
Positive transfer through curriculum learning: Understanding supervision feeds back into parsing after parsing matures, improving it by up to +0.64 points and delivering a dual capability boost in a lightweight model.

Project website: https://huggingface.co/spaces/xiaomi-research/Xiaomi-OCR-0
GitHub repository: https://github.com/xiaomi-research/Xiaomi-OCR-0
Hugging Face model repository: https://huggingface.co/xiaomi-research/Xiaomi-OCR-0
arXiv technical paper: https://arxiv.org/pdf/2609.36136

Instant management of daily-life credentials: Recognizes time-sensitive information such as food delivery pickup codes, parcel collection codes, and queue numbers, then “pins” it for one-touch access when needed.
Smart expense tracking and bill organization: Extracts amounts, merchants, times, and categories from WeChat/Alipay bill details with one tap, automatically generating expense records.
Structured entry of identity cards and receipts: Extracts fields from ID cards, bank cards, invoices, passports, and other documents as JSON through photography or screen recognition, replacing manual entry and verification.
Smart schedule generation: Extracts times and locations from group chat notifications and event posters, automatically generating schedule cards and adding them to the calendar.
Travel information linkage: Extracts itinerary details such as flight numbers from chats, automatically checks real-time departure and arrival updates, and sends reminders.