OpenBMB/MiniCPM-OCR

最近记录的来源活动: 2026年11月2日 15:28

SkillsMP 收录数据更新: 2026年11月9日 14:22

已收集 skills: 1

GitHub 星标: 846
GitHub Forks: 62

用 AI 助手安装

复制这段 Prompt，发给你正在使用的 AI 助手。

Follow https://skillsmp.com/skill-install/prompt.md to install Agent Skills from https://github.com/OpenBMB/MiniCPM-OCR.

仓库 README

仓库 README 来源：README.md

查看 README 源文件

展开完整 README

<div align="center"> <img src="./assets/minicpm-ocr/release-hero.svg" alt="MiniCPM-OCR: fast, accurate document parsing into text, tables, and formulas, with a lightning arrow connecting documents to structured results; a decision model helps identify regions that need review (illustration)." width="100%"> <a href="https://github.com/OpenBMB/MiniCPM"><img src="./assets/minicpm-ocr/minicpm-logo.jpg" width="300" alt="MiniCPM logo"></a> <a href="./docs/minicpm_ocr_1dot0_en.md">MiniCPM-OCR 1.0 Guide</a>&ensp;<a href="https://huggingface.co/openbmb/MiniCPM-OCR-1.0" title="Hugging Face"><img src="./assets/minicpm-ocr/huggingface_logo-noborder.svg" width="19" height="18" alt="Hugging Face"></a>&ensp;<a href="https://modelscope.cn/models/OpenBMB/MiniCPM-OCR-1.0" title="ModelScope"><img src="./assets/minicpm-ocr/modelscope_logo_padded.svg" width="25" height="14" alt="ModelScope"></a> &emsp; <a href="./docs/minicpm_ocr_jev_en.md">MiniCPM-OCR Jev Guide</a>&ensp;<a href="https://huggingface.co/openbmb/MiniCPM-OCR-Jev" title="Hugging Face"><img src="./assets/minicpm-ocr/huggingface_logo-noborder.svg" width="19" height="18" alt="Hugging Face"></a>&ensp;<a href="https://modelscope.cn/models/OpenBMB/MiniCPM-OCR-Jev" title="ModelScope"><img src="./assets/minicpm-ocr/modelscope_logo_padded.svg" width="25" height="14" alt="ModelScope"></a> Demo (<a href="https://platform.modelbest.cn/console/playground/document-parse">CN</a> / <a href="https://platform.modelbest.ai/console/playground/document-parse">Global</a>)  ·  <a href="#quick-start">Get Started</a>  ·  <a href="#model-weights">Weights</a>  ·  <a href="https://platform.modelbest.ai/console/docs/api/document-parse">Free API</a>  ·  <a href="#skill">Skill</a>  ·  <a href="./docs/minicpm_ocr_tutorial_en.md">Local Deployment</a>  |  <a href="./README_zh.md">中文</a> Join the Community via <a href="https://applink.feishu.cn/client/chat/chatter/add_by_link?link_token=c35j072a-9abe-4ba6-ba1d-3ab340f9ba8c">Feishu / Lark</a>  ·  <a href="https://discord.gg/JE8KQqtdn">Discord</a> </div> <a id="overview"></a> **MiniCPM-OCR** is a family of models for efficient document parsing and text recognition. **MiniCPM-OCR 1.0** parses Chinese and English documents and PDFs, balancing quality and efficiency across challenging inputs such as scans, photographs, curved pages, and recaptured documents. **MiniCPM-OCR Jev** evaluates OCR results at the region level, distinguishing problematic regions from reliable ones. It works with different parsing models to help users locate content that needs review. ## 📢 News - **[2026.11.05]** The official MiniCPM-OCR 1.0 [Docker image](https://hub.docker.com/repository/docker/openbmbofficial/minicpm-ocr/general) is now available, bundling model weights, the SDK, and vLLM for end-to-end image and PDF parsing. See [Docker deployment](#docker-deployment) to get started. - 🔥🔥🔥 **[2026.11.02] MiniCPM-OCR 1.0 now runs on iOS!** Parse documents entirely on your device, even offline. [Try it now](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/README.md#mobile-demo). - **[2026.11.09]** [pip SDK 1.0.5](https://pypi.org/project/minicpm-ocr/1.0.5/): Parse several inputs in one run, with one result per file. `--split-images` makes each standalone image its own document. - **[2026.11.02]** [pip SDK 1.0.4](https://pypi.org/project/minicpm-ocr/1.0.4/): JSON output now uses the same format as the online document parsing API. Page orientation is corrected automatically by default, and GIF images are now supported. - **[2026.10.30]** The Web Demo is now open source, with support for self-hosting and switching between Chinese and English. See the [Web Demo deployment tutorial](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/webdemo_deploy.md). - **[2026.10.29]** [SDK 1.0.3](https://pypi.org/project/minicpm-ocr/1.0.3/): Markdown output preserves text recognized in image regions and includes links to the corresponding cropped images. The [Skill](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/README.md#skill) now includes an Ollama deployment guide. - 🚀🚀🚀 **[2026.10.27] MiniCPM-OCR 1.0 and MiniCPM-OCR Jev are now open source!** Get started with the [model weights](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/README.md#model-weights), the [pip SDK](https://pypi.org/project/minicpm-ocr/), the [Skill](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/README.md#skill), and a [free API](https://github.com/OpenBMB/MiniCPM-OCR/blob/main/README.md#api). ## MiniCPM-OCR 1.0 **MiniCPM-OCR 1.0** (1.3B) is the first OCR-specific model in the [MiniCPM-V/o](https://github.com/openbmb/MiniCPM-V) family. It focuses on text-centric document understanding, covering text, tables, formulas, charts, seals, and structured information extraction. Its key features include: - 🏆 **SOTA document parsing**: MiniCPM-OCR 1.0 achieves a **SOTA score of 93.68 on Real5-OmniDocBench**, a benchmark for real-world document parsing, and demonstrates leading parsing capabilities on OmniDocBench v1.6, Wild-OmniDocBench, and olmOCR-Bench. It supports Chinese and English documents and PDFs, handling both standard documents and challenging capture conditions, including scans, curved pages, photographs, skewed pages, and recaptured documents. - ⚡ **Efficient inference.** With DSpark decoding acceleration ([**DSpark-BF16**](https://huggingface.co/openbmb/MiniCPM-OCR-1.0-DSpark), [**DSpark-GGUF**](https://huggingface.co/openbmb/MiniCPM-OCR-1.0-DSpark-gguf)), MiniCPM-OCR 1.0 improves **end-to-end processing efficiency by 55%**. Its throughput is approximately **1.49×** that of PaddleOCR-VL-1.6 and **3.62×** that of MinerU2.5 Pro, reducing the time needed for large-scale document parsing. - 🔍 **General recognition and information extraction**: MiniCPM-OCR 1.0 achieves a **SOTA score of 54.7 on OCRBench v2**, demonstrating strong general OCR capabilities. It **also supports key information extraction (KIE)**, extracting relevant document content for downstream information management and automated processing. - 🛠️ **Easy deployment and development**: MiniCPM-OCR 1.0 supports **vLLM, SGLang, llama.cpp, and Ollama from day one**, with multiple chip platforms and quantized versions available at release. It **supports further fine-tuning** and provides a **[pip SDK](https://pypi.org/project/minicpm-ocr/)** and a **[full-featured API](#api)** for application integration and customization. You can also **[get started quickly with SKILL.md](#skill)**. Explore parsing results with the online Demo ([CN](https://platform.modelbest.cn/console/playground/document-parse) / [Global](https://platform.modelbest.ai/console/playground/document-parse)). ### SOTA Performance, Higher Throughput Among the open-source OCR models shown below, MiniCPM-OCR 1.0 achieves 1.2 pages/s and the highest overall performance on OmniDocBench v1.6, Real5-OmniDocBench, Wild-OmniDocBench, and olmOCR-Bench. <a href="./assets/minicpm-ocr/efficiency-performance-en.png"><img src="./assets/minicpm-ocr/efficiency-performance-en.png" alt="Throughput and parsing quality of open-source OCR models. The horizontal axis shows throughput in pages per second on a logarithmic scale; the vertical axis shows the equally weighted average score across four document parsing benchmarks." width="90%"></a> See the [reproduction guide](./docs/benchmark_reproduction_en.md) for test settings and reproduction details, and the [model guide throughput evaluation](./docs/minicpm_ocr_1dot0_en.md#throughput) for each model's throughput. <a id="ocr-benchmark"></a> ### Benchmark Comparisons <a href="./assets/minicpm-ocr/benchmark-radar.png"><img src="./assets/minicpm-ocr/benchmark-radar.png" alt="Radar chart comparing MiniCPM-OCR 1.0 with other OCR models across six benchmarks; some values remain to be confirmed." width="80%"></a> <details> <summary>View complete MiniCPM-OCR 1.0 evaluation results</summary> <a href="./assets/minicpm-ocr/ocr-benchmarks.png"><img src="./assets/minicpm-ocr/ocr-benchmarks.png" alt="Detailed OCR model results on OmniDocBench v1.6, Real5, Wild, olmOCR-Bench, CHAOS-Bench, and OCRBench v2." width="100%"></a> </details> See the [MiniCPM-OCR 1.0 guide](./docs/minicpm_ocr_1dot0_en.md#evaluation) for per-benchmark bar charts and throughput details. ### Examples <a href="https://youtu.be/lJOzZytWoK8"><img src="./assets/minicpm-ocr/main_video_en_play.PNG" alt="MiniCPM-OCR 1.0 and Jev promo video. Click to watch the demo on YouTube." width="80%"></a> Left to right: original document, detected layout, and rendered Markdown. #### English Documents <a href="./assets/minicpm-ocr/minicpm-english-examples-3-column.gif"><img src="./assets/minicpm-ocr/minicpm-english-examples-3-column.gif" alt="English document animation: original, detected layout, and rendered Markdown." width="100%"></a> <details> <summary>More English Examples</summary> Original documents and parsed results, side by side. <a href="./assets/minicpm-ocr/minicpm-ocr-english-cases.png"><img src="./assets/minicpm-ocr/minicpm-ocr-english-cases.png" alt="English examples comparing inputs and results: handwriting, historical and scanned documents, newspapers, financial statements, and key information extraction." width="100%"></a> </details> #### Chinese Documents <a href="./assets/minicpm-ocr/minicpm-chinese-examples-3-column.gif"><img src="./assets/minicpm-ocr/minicpm-chinese-examples-3-column.gif" alt="Chinese document animation: original, detected layout, and rendered Markdown." width="100%"></a> <details> <summary>More Chinese Examples</summary> Original documents and parsed results, side by side. <a href="./assets/minicpm-ocr/minicpm-ocr-chinese-cases.png"><img src="./assets/minicpm-ocr/minicpm-ocr-chinese-cases.png" alt="Chinese examples comparing inputs and results: handwritten notes, complex formulas, newspapers, financial statements, stamped documents, illustrations, and key information extraction." width="100%"></a> </details> ## MiniCPM-OCR Jev **MiniCPM-OCR Jev** is an **OCR correctness decision model** for text, formulas, and tables. Given an image and its corresponding OCR result, it returns a **quality score from 0 to 1** to help select reliable results and locate content that needs review. - 🛡️ **Strong performance in a small model**: With only 2B parameters, MiniCPM-OCR Jev achieves an AUROC (×100) of 93.6 on [JevOCRBench](https://huggingface.co/datasets/openbmb/JevOCRBench), **outperforming JEV-35B-A3B (90.1)** and approaching Qwen3.5-397B-A17B (93.8). - 🧩 **Consistent generalization**: Across parsing results from MiniCPM-OCR 1.0, GLM-OCR, PaddleOCR-VL-1.6, and Xiaomi-OCR-0, its AUROC (×100) ranges from **91.9 to 93.4**. <a id="jev-benchmark"></a> <a id="评测表现"></a> ### Evaluation Results The table below reports overall and per-category AUROC (×100) on JevOCRBench. Higher values indicate better discrimination between correct and incorrect parsing results. <a href="./assets/minicpm-ocr/jev-benchmark-score-table.png"><img src="./assets/minicpm-ocr/jev-benchmark-score-table.png" alt="Overall, text, formula, and table AUROC results on JevOCRBench, highlighting MiniCPM-OCR Jev." width="100%"></a> <a id="判别力跨ocr模型泛化"></a> <a id="使用场景"></a> <a id="用法一块级直通省掉大部分人工核验"></a> <a id="用法二多专家聚合扩展-ocr-的能力边界"></a> ### Review-Free Accuracy Accuracy at different review-free rates across three document types. <a href="./assets/minicpm-ocr/review-free-en.png"><img src="./assets/minicpm-ocr/review-free-en.png" alt="Accuracy versus review-free rate for legal documents, financial receipts, and complex documents." width="100%"></a> Set the threshold only after checking it on a small batch from the real application. ### Decision Examples Jev scores for text, formula, and table recognition. <a href="./assets/minicpm-ocr/jev-case-en.png"><img src="./assets/minicpm-ocr/jev-case-en.png" alt="Original images, two OCR candidates, and their Jev scores for text, formulas, and tables." width="100%"></a> See the [MiniCPM-OCR Jev model guide](./docs/minicpm_ocr_jev_en.md) for more on usage, technical specifications, evaluation results, and efficiency results. <a id="quick-start"></a> ## Using MiniCPM-OCR Choose how to integrate MiniCPM-OCR 1.0 and MiniCPM-OCR Jev: ### How Document Parsing Works Full-page parsing runs as a **two-stage pipeline** orchestrated by the [`minicpm-ocr` SDK](https://pypi.org/project/minicpm-ocr/). The two stages use separate models, run in separate environments, and may run on different machines: ```mermaid flowchart LR DOC["PDF / image"] --> S1 subgraph SDKENV ["minicpm-ocr SDK (its own environment)"] S1["Stage 1 · Layout detection PP-DocLayoutV3 layout model (PaddlePaddle / PaddleX) regions, reading order, cropping"] ASM["Assemble output Markdown / JSON in reading order"] end subgraph SRVENV ["Recognition service (separate environment)"] S2["Stage 2 · Region recognition MiniCPM-OCR 1.0 vLLM / SGLang / llama.cpp / Ollama"] end S1 -- "each cropped region, over HTTP, with a prompt for its region type" --> S2 S2 -- "text, tables, formulas, charts, seals" --> ASM ``` - **Stage 1 — Layout detection (run locally by the SDK).** The PP-DocLayoutV3 layout model detects document regions — text, tables, formulas, charts, seals — and analyzes reading order; the SDK then crops the regions. It runs inside the SDK environment and requires PaddlePaddle / PaddleX. - **Stage 2 — Region recognition (inference service).** The SDK sends each cropped region over HTTP to a MiniCPM-OCR 1.0 inference service — served by vLLM, SGLang, llama.cpp, or Ollama — with a prompt matching the region type (`Text Recognition:`, `Table Recognition:`, `Formula Recognition:`, …). The SDK assembles the recognized content into the final Markdown / JSON in reading order. A full local deployment therefore prepares **two environments side by side**: the SDK environment (pip package `minicpm-ocr`, plus PaddlePaddle / PaddleX for the layout model) and the inference-service environment of the chosen backend. The [Skill](#skill) walks through the complete setup — backend choice, environment installation, model downloads, and verification — and the [SDK tutorial](./docs/minicpm_ocr_tutorial_en.md) covers the same steps for manual deployment. For region prompts and per-stage details, see the [document parsing workflow](./docs/minicpm_ocr_1dot0_en.md#document-parsing-workflow). <a id="使用-agent-skill-部署推荐"></a> <a id="skill"></a> ### Skill Ask a coding agent to follow the MiniCPM-OCR 1.0 [skill](./docs/SKILL_en.md) to set up the environment, download models, and parse your first document. Send the following instruction to the agent: ```text Read and execute https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/SKILL_en.md ``` Ask a coding agent to follow the MiniCPM-OCR Jev [skill](./docs/SKILL_jev_en.md) to deploy, start, and verify the scoring service locally — either from the Docker image or with the SDK, using vLLM, SGLang, llama.cpp, or Ollama as the inference backend. Send the following instruction to the agent: ```text Read and execute https://github.com/OpenBMB/MiniCPM-OCR/blob/main/docs/SKILL_jev_en.md ``` If you have already cloned this repository, you can use `./docs/SKILL_en.md` and `./docs/SKILL_jev_en.md` directly. <a id="docker-deployment"></a> ### Docker Deployment Deploy MiniCPM-OCR 1.0 with the official [Docker image](https://hub.docker.com/repository/docker/openbmbofficial/minicpm-ocr/general), which bundles model weights, the SDK, vLLM, and the document parsing backend. The Docker Hub page provides startup instructions for both the OCR model API and the end-to-end document parsing API, supporting image and PDF parsing into Markdown / JSON. Requires Linux x86-64, an NVIDIA GPU, a driver compatible with CUDA 12.9, and Docker configured with NVIDIA Container Toolkit. Follow the image documentation to select and start the service you need. <a id="api"></a> ### Free API Use the Free API to access MiniCPM-OCR 1.0 recognition and MiniCPM-OCR Jev quality scoring. [MiniCPM-OCR 1.0 API docs](https://platform.modelbest.ai/console/docs/api/document-parse) &emsp;·&emsp; [MiniCPM-OCR Jev API docs](https://platform.modelbest.ai/console/docs/api/ocr-jev) <a id="mobile-demo"></a> ### MiniCPM-V Demo — iOS, Android & HarmonyOS MiniCPM-OCR 1.0 parses documents entirely on your device, even offline. This document recognition feature is currently available on iOS only. [On-device demo tutorial](https://github.com/OpenBMB/MiniCPM-V-Apps/blob/main/README.md) <a id="手动部署"></a> <a id="本地部署"></a> ### Local Deployment Run the models on your own hardware: MiniCPM-OCR 1.0 parses PDFs and images into Markdown or JSON, while MiniCPM-OCR Jev scores the parsing results. [MiniCPM-OCR 1.0 deployment tutorial](./docs/minicpm_ocr_tutorial_en.md) · [MiniCPM-OCR Jev deployment tutorial](./docs/minicpm_ocr_jev_tutorial_en.md) · [Web demo deployment tutorial](./docs/webdemo_deploy.md) <a id="模型微调"></a> ### Fine-Tuning | Model | Fine-tuning tutorial | | :--- | :--- | | MiniCPM-OCR 1.0 | [LLaMA-Factory](./docs/llamafactory_minicpm_ocr_french_en.md) · [Swift](./docs/swift_minicpm_ocr_french_en.md) | | MiniCPM-OCR Jev | [Swift](./docs/finetune_jev_en.md) | <a id="resources"></a> <a id="model-weights"></a> ## Model Weights <table border="1" cellspacing="0" cellpadding="8"> <tr> <td nowrap>Model</td> <td width="360">Description</td> <td nowrap width="170">Main Model Weights</td> <td nowrap width="150">DSpark Heads</td> </tr> <tr> <td nowrap>MiniCPM‑OCR 1.0</td> <td width="360">Efficient document parsing and text recognition for Chinese and English documents and PDFs, balancing quality and speed across scans, photographs, curved pages, and recaptured documents.</td> <td nowrap><a href="https://huggingface.co/openbmb/MiniCPM-OCR-1.0">🤗</a>  <a href="https://modelscope.cn/models/OpenBMB/MiniCPM-OCR-1.0"><img src="./assets/minicpm-ocr/modelscope_logo.png" width="20" alt="ModelScope"></a> BF16 <a href="https://huggingface.co/openbmb/MiniCPM-OCR-1.0-gguf">🤗</a>  <a href="https://modelscope.cn/models/OpenBMB/MiniCPM-OCR-1.0-gguf"><img src="./assets/minicpm-ocr/modelscope_logo.png" width="20" alt="ModelScope"></a> GGUF <a href="https://huggingface.co/openbmb/MiniCPM-OCR-1.0-AWQ">🤗</a>  <a href="https://modelscope.cn/models/OpenBMB/MiniCPM-OCR-1.0-AWQ"><img src="./assets/minicpm-ocr/modelscope_logo.png" width="20" alt="ModelScope"></a> AWQ <a href="https://huggingface.co/openbmb/MiniCPM-OCR-1.0-GPTQ">🤗</a>  <a href="https://modelscope.cn/models/OpenBMB/MiniCPM-OCR-1.0-GPTQ"><img src="./assets/minicpm-ocr/modelscope_logo.png" width="20" alt="ModelScope"></a> GPTQ</td> <td nowrap><a href="https://huggingface.co/openbmb/MiniCPM-OCR-1.0-DSpark">🤗</a>  <a href="https://modelscope.cn/models/OpenBMB/MiniCPM-OCR-1.0-DSpark"><img src="./assets/minicpm-ocr/modelscope_logo.png" width="20" alt="ModelScope"></a> BF16 <a href="https://huggingface.co/openbmb/MiniCPM-OCR-1.0-DSpark-gguf">🤗</a>  <a href="https://modelscope.cn/models/OpenBMB/MiniCPM-OCR-1.0-DSpark-gguf"><img src="./assets/minicpm-ocr/modelscope_logo.png" width="20" alt="ModelScope"></a> GGUF</td> </tr> <tr> <td nowrap>MiniCPM‑OCR Jev</td> <td width="360">An OCR correctness decision model for text, formulas, and tables. Given an image and its OCR result, it returns a quality score from 0 to 1.</td> <td align="center" valign="middle" colspan="2"><a href="https://huggingface.co/openbmb/MiniCPM-OCR-Jev">🤗</a>  <a href="https://modelscope.cn/models/OpenBMB/MiniCPM-OCR-Jev"><img src="./assets/minicpm-ocr/modelscope_logo.png" width="20" alt="ModelScope"></a> BF16&emsp;<a href="https://huggingface.co/openbmb/MiniCPM-OCR-Jev-gguf">🤗</a>  <a href="https://modelscope.cn/models/OpenBMB/MiniCPM-OCR-Jev-gguf"><img src="./assets/minicpm-ocr/modelscope_logo.png" width="20" alt="ModelScope"></a> GGUF <a href="https://huggingface.co/openbmb/MiniCPM-OCR-Jev-AWQ">🤗</a>  <a href="https://modelscope.cn/models/OpenBMB/MiniCPM-OCR-Jev-AWQ"><img src="./assets/minicpm-ocr/modelscope_logo.png" width="20" alt="ModelScope"></a> AWQ&emsp;<a href="https://huggingface.co/openbmb/MiniCPM-OCR-Jev-GPTQ">🤗</a>  <a href="https://modelscope.cn/models/OpenBMB/MiniCPM-OCR-Jev-GPTQ"><img src="./assets/minicpm-ocr/modelscope_logo.png" width="20" alt="ModelScope"></a> GPTQ</td> </tr> </table> ## 💡 FAQ and Troubleshooting See the [FAQ](./docs/faq_en.md) for common questions about installation, model downloads, GPU memory, input formats, and unexpected outputs. ## 📄 License The entire project is released under the [Apache License 2.0](./LICENSE), including code, documentation, and model weights for MiniCPM-OCR 1.0 and MiniCPM-OCR Jev.

正在加载仓库 README…

这个仓库中的 skills

职业分类待补全

skill

职业分类

描述

更新

minicpmvocr-sdk

未分类

安装、部署和验证 MiniCPM-OCR 1.0 文档解析 SDK 与 vLLM、SGLang、llama.cpp 或 Ollama 推理服务，包括 DSpark、PDF/图片转 Markdown、本地 Web Demo 和安装排错。Jev 打分服务使用 SKILL_jev.md。

2026-11-02

已展示 1 / 1 个已收集 Skill。