MiniCPM-OCR Uyasebenza

I-MiniCPM-OCR iwuchungechunge lwamamodeli e-OCR olushicilelwe futhi lwaba umthombo ovulekile yi-Mianbi Intelligence. I-MiniCPM-OCR 1.0 (1.3B) iyimodeli ekhethekile yokuhlaziya amadokhumenti, ethole i-SOTA ku-Real5-OmniDocBench naku-OCRBench v2, kanti ukusheshiswa kokukhipha nge-DSpark kwenza umthamo wayo ufinyelele kumakhasi angu-1.217 ngomzuzwana, ngokushesha nangokunembile; i-MiniCPM-OCR Jev (2B) iyimodeli yokuqala embonini ehlola imiphumela yokuhlaziya, ikhiphe amaphuzu ekhwalithi futhi ihlunge ngokuzenzakalelayo imiphumela engadingi ukuqinisekiswa komuntu, ngaleyo ndlela yehlise izindleko zabasebenzi. I-MiniCPM-OCR ifanele izimo ezibucayi zobumfihlo ezifana nezezimali, intela nokubalwa kwezimali, ezomthetho nezempilo, futhi isekela ukusebenza kwasendaweni emishinini kanye nokulungiswa okwengeziwe kwemodeli.

Ukuhlaziya nokuhlela amadokhumenti: Ibuyisela amarisidi, izinkontileka, imibiko, amarekhodi ezokwelapha namanye amadokhumenti esakhiweni sawo sekhasi, ikhiphe i-Markdown noma idatha ehlelwe ngezinkambu.
Ukuqaphela izakhiwo eziyinkimbinkimbi: Isekela izimo eziyisikhombisa ezivamile, okuhlanganisa amaphephandaba anamakholomu amaningi, amathebula ezimali abanzi, amanothi abhalwe ngesandla, amafomula e-matrix ayinkimbinkimbi kanye namadokhumenti avalwe izitembu.
Ukulungisa ngokuzenzakalela izithombe: Ithola futhi ilungise ngokuzenzakalela amadokhumenti ajikile noma aphendukile, ngaphandle kokuwabeka kahle ngesandla.
Ukuhlanganisa okuqukethwe kwamakhasi amaningi: Ihlanganisa ulwazi olusakazeke kumakhasi amaningi kumadokhumenti amade, ukuze lusetshenziswe amasistimu ebhizinisi.
Ukuhlola ikhwalithi yokuhlaziya (Jev): Ikhipha amaphuzu ekhwalithi angu-0–1 emiphumeleni yokuhlaziya yanoma iyiphi isistimu ye-OCR, ibonise ukuthi umphumela ungethenjwa yini.
Ukuhlukanisa ukuqinisekiswa komuntu: Ihlunga ngokuzenzakalela ngokomkhawulo wamaphuzu; imiphumela enamaphuzu aphezulu ingena kusizindalwazi ngaphandle kokuhlolwa, kanti enamaphuzu aphansi idluliselwa kumuntu, kunciphe kakhulu umsebenzi wokuqinisekisa.
Ukukhetha umphumela ongcono phakathi kwamamodeli: Ngemva kokuhlaziya ngamamodeli amaningi, i-Jev ihlola amaphuzu endaweni ngayinye bese ihlanganisa imiphumela engcono kakhulu, iphakamise amaphuzu aphelele afinyelele ku-0.25.
Ukukhipha ulwazi olubalulekile (KIE): Ikhipha ulwazi lwezinkambu ezibalulekile ezimweni ezinjengamadokhumenti avalwe izitembu namarisidi amancane.

Landela i-WeChat bese uphendula ngokuthi “开源” ukuze ujoyine iqembu lokuxhumana lamaphrojekthi e-AI omthombo ovulekile
Isisekelo esinamandla: Iqeqeshwe ngemodeli ye-multimodal MiniCPM-V 4.6, iphinde isebenzise idatha enkulu, esezingeni eliphezulu yokuqeqeshwa kwangaphambili kwamadokhumenti eqoqwe uchungechunge lwe-MiniCPM-V/o. Lokhu kwenza imodeli ibe namakhono aqinile okuqonda izithombe namadokhumenti ngaphambi kokuqeqeshwa okukhethekile kwe-OCR, ukuze ukuqeqeshwa kugxile ekunembeni kokubona nasekukhipheni okuhlelekile.
Isakhiwo esihlakaniphile: Isebenzisa indlela esebenza kahle yokuhlukanisa izithombe i-LLaVA-UHDv4, egwema ukukhula komthamo wokubala ngokuhambisana nokwanda kokulungiswa kwesithombe; ihlanganisa imiklamo emibili ye-Automatic Upright ne-Adaptive Upsize.
Ukuhlanganiswa kwedatha nokufunda okuqiniswayo: Iqoqa idatha eyisisekelo ezimweni eziningi zangempela, iyandise ngezinga elikhulu nge-Agentic synthesis, igcine izici zangempela kanye nokuhlukahluka; bese yenza ukufunda okuqiniswayo ngezinga elikhulu ezimweni zangempela eziyinkimbinkimbi ukuze ithuthukise ikhwalithi yokuhlaziya amasampula angavamile.
Ukusheshiswa kokukhipha nge-DSpark: Imodyuli encane yokuqala iphakamisa ama-token amaningi angaba khona ngesikhathi esisodwa, imodeli eyinhloko iwahlola ngokufanayo, esikhundleni sokuwakhiqiza i-token ngayinye; ukusebenza kusukela ekuqaleni kuya ekugcineni kuthuthuka ngo-55% ngaphandle kokunciphisa ikhwalithi.
Indlela ye-Jev yokuhlola ngokushesha: Isebenzisa umklamo we-System-1, ibala i-prefill ngokudlula okukodwa ngaphandle kokukhipha okuzenzakalelayo, ngakho ukubambezeleka kuphansi kakhulu futhi ingasetshenziswa njengesango lokuhlola ikhwalithi elivamile; iqeqeshwe ngamalebula okunemba ezingeni lamabhulokhi avela ekulebheni kwabantu kwangempela, okwenza amaphuzu asetshenziswe ngqo njengomkhawulo webhizinisi.

Ukusebenzisa nokwenza inference endaweni yangakini: Landa izisindo zemodeli nge-Hugging Face CLI (openbmb/MiniCPM-OCR-1.0 noma openbmb/MiniCPM-OCR-Jev).
Faka uhlaka lwe-inference (ngosuku lokuqala isekela i-vLLM, SGLang, llama.cpp ne-Ollama; khetha eyodwa ngokwesidingo. Indlela evamile yokuqalisa amamodeli ochungechunge lwe-OpenBMB ithi vllm serve openbmb/<igama lemodeli> --port 8000).
Khetha uhlobo lwe-quantization ngokwehardware; kunokusekelwa kwama-chip amaningi namafomethi amaningi e-quantization, kanti izindawo eziseceleni noma ezinamandla amancane zingakhetha uhlobo lwe-GGUF/INT4.
Faka isithombe sedokhumenti, bese imodeli ilungisa ngokuzenzakalela ukujika, ihlaziye ukwakheka kwekhasi futhi yenze ukuhlaziya okuhlelekile, ikhiphe i-Markdown noma imiphumela ehlelwe ngezinkambu.

Ukunemba kwe-SOTA okubili: amaphuzu angu-93.68 ku-Real5-OmniDocBench namaphuzu angu-54.7 ku-OCRBench v2, okuhola imodeli elandelayo ngamaphuzu angu-7.7; kumamodeli amakhulu angu-14 aqhathaniswe kumazinga ayisisekelo angu-6, iphume kuqala kwezingu-4.
Ukusebenza okuhola phambili: Ukusheshiswa kokukhipha nge-DSpark kuthuthukisa ukusebenza kusukela ekuqaleni kuya ekugcineni ngo-55%, umthamo ufinyelele kumakhasi angu-1.217 ngomzuzwana, okuyimodeli kuphela eshadini lokuqhathanisa ukusebenza nomphumela ethi “ishesha kakhulu futhi inembe kakhulu”.
Ilula kakhulu: Ngamapharamitha angu-1.3B kuphela idlula amamodeli amakhulu, ivumelana nemishini eseceleni nezindawo ezinamandla alinganiselwe, futhi ingasebenza endaweni yangakini ukuze kuhlangatshezwane nezidingo zobumfihlo.
Imiphumela ethenjwayo nenokuhlukaniseka: I-Jev iyimodeli yokuqala embonini yokuhlola ikhwalithi ye-OCR, i-AUROC yayo engu-93.6 idlula amamodeli okuhlukanisa ezinga le-35B, okwenza kube khona ukubikezela okokuqala kokuthi “iliphi ikhasi elidinga ukuqinisekiswa komuntu”.
Ukuhlolwa kwekhwalithi okulungele ukusetshenziswa: I-Jev ayiboshiwe kusistimu ethile ye-OCR; ku-GLM-OCR, PaddleOCR-VL nakwezinye izinhlelo eziningi, i-AUROC ihlala ku-91.9~93.4.

Indawo yokugcina ye-GitHub: https://github.com/OpenBMB/MiniCPM-OCR
Indawo yamamodeli ye-HuggingFace: https://huggingface.co/openbmb/MiniCPM-OCR-1.0
https://huggingface.co/openbmb/MiniCPM-OCR-Jev

Ukucutshungulwa kwamarisidi ezimali: Ukuqaphela nokufaka ngokuzenzakalela ngendlela ehlelekile izitatimende zebhange, amasheke namarisidi okubuyiswa kwemali; ngemva kokuhlolwa kwekhwalithi yi-Jev, amarisidi anamaphuzu aphezulu angena kusizindalwazi ngqo, kunciphe ukufakwa kwedatha ngesandla emafasiteleni ebhange.
Ukuhlolwa kwezinkontileka zomthetho: Amafemu omthetho neminyango yezomthetho ahlaziya izinkontileka ngobuningi; ukusebenza endaweni yangakini kuqinisekisa ukuthi amadokhumenti awaphumi endaweni, kanti nge-Jev amakhasi afinyelela ku-85% angadlula ngaphandle kokuhlolwa, kunciphe kakhulu isikhathi sokuqinisekisa.
Ukwenza amarekhodi ezokwelapha abe yidijithali: Izibhedlela ziguqula amarekhodi ezokwelapha ephepha nemibiko yokuhlolwa ibe idatha ye-elekthronikhi ehlelekile; ukusetshenziswa okuyimfihlo kuhlangabezana nezidingo zokuhambisana nobumfihlo beziguli.
Ukuphathwa kwamarekhodi kahulumeni: Ukuqaphela amadokhumenti kahulumeni, omazisi namadokhumenti anezitembu, kanye nokukhipha ulwazi olubalulekile (KIE), kuhlanganise nokukhipha ngokuzenzakalela izinkambu ezibalulekile ezimbozwe izitembu.