WeVisOne-VL Uyasebenza

I-WeVisOne-VL iyimodeli eyisisekelo yokubona nolimi ekhishwe njengomthombo ovulekile yiqembu le-Tencent WeChat Vision, esekelwe ku-Qwen3-VL, futhi itholakala ngezinguqulo ezimbili: i-2B ne-4B. I-WeVisOne-VL isebenzisa indlela ebumbene yokubikezela ama-token ahlelekile ukuze imodeli eyodwa yenze ngesikhathi esisodwa imisebenzi eyisikhombisa: ukuthola izinto, ukuthola into ekhonjwe yincazelo, ukubala, ukukhomba, i-OCR, ukuhlukanisa into ekhonjwe yincazelo, kanye nokususa ingemuva okuqondiswa iziqondiso. I-decoder yamaphikseli enama-parameter ayizigidi ezingu-9.6 ikhipha ngokuqondile imaski noma ukususa ingemuva okuningiliziwe okufana nezinwele, ngaphandle kwesidingo se-SAM yangaphandle. Idatha yokuqeqesha ye-WeVisOne-VL idlula ezithombeni eziyizigidi ezingu-100 nakumabhokisi ezinto angaba yizigidi eziyizinkulungwane ezingu-3.6. Ngemva kokulungiswa okuqondiswa yizifundo ngezigaba ezintathu, okuhlanganiswe nokufunda okuqinisa, idlula amamodeli afanayo kuma-benchmark angaphezu kuka-30.

Ukuthola izinto: ingakhipha ngokuqondile wonke amabhokisi ezinto esithombeni ngokulandela imiyalo efana nokuthi “thola zonke izinja”.
Ukuthola into ekhonjwe yincazelo: ingathola into ethile esithombeni ngokulandela incazelo yolimi lwemvelo efana nokuthi “umuntu ogqoke okubomvu”.
Ukubala: iphendula ngokuthi zingaki izinto zesigaba esithile ezisesithombeni.
Ukukhomba: iyaqonda izincazelo eziqondisayo ezifana nokuthi “leyo nkomishi” futhi ithole into ekhonjwayo.
I-OCR: ifunda umbhalo osesithombeni futhi imake indawo yawo.
Ukuhlukanisa into ekhonjwe yincazelo: idweba uhlaka lwento ngephikseli ngayinye bese ikhipha imaski kanambambili.
Ukususa ingemuva okuqondiswa iziqondiso: ikhipha isiteshi se-alpha esiqhubekayo, igcine ngokwemvelo imininingwane engabonakali ngokuphelele efana nezinwele, futhi ivumele ukushintshwa kwengemuva ngokuqondile.

Landela i-WeChat bese uphendula “开源” ukuze ujoyine iqembu lezingxoxo zamaphrojekthi e-AI anomthombo ovulekile
Okukhishwayo okuhlelekile lapho i-coordinate iyi-token: imodeli ihlukanisa ama-coordinate abe ama-token akhethekile angu-1001. Ibhokisi elilodwa lisebenzisa ama-token angu-4 ncamashi, iphuzu elilodwa lisebenzisa ama-token angu-2. Amabhokisi okuthola izinto, izindawo zombhalo namamaski wamaphikseli konke kuvezwa kusetshenziswa i-vocabulary efanayo, bese kukhiqizwa ngendlela ebumbene yokubikezela i-next-token ehlelekile. Ukushintsha umsebenzi kusho ukushintsha kuphela isiqalo se-prompt; uma into ingekho, imodeli ikhipha i-None ngokuqondile.
I-decoder yamaphikseli eyakhelwe ngaphakathi ithatha indawo ye-SAM cascade: ngaphandle kwesisekelo se-Qwen3-VL, imodeli ixhunywe kumodule encane enama-parameter ayizigidi ezingu-9.6 kuphela. I-adapter yezici ezibukwayo ithatha izici zezingqimba ezine ze-ViT ngaphambi kokuhlanganiswa kwangaphambili kwesisekelo, izicwengisise nge-deep convolution ukuze yakhe i-feature pyramid eyabiwe. Isimo esifihlekile se-token ngayinye, sihlanganiswe ne-box prior enekhodi ye-Fourier, sakha i-instance query. Le query icwengisiswa ngokuphindaphindiwe ngamabhulokhi angu-6 e-deformable cross-attention. Ekugcineni, i-dense head esekelwe ku-instance iqala ibikezele imaski nge-dynamic convolution kernels, bese ibuyisa i-alpha matte enesinqumo esigcwele ezingeni lezinwele kusukela ku-S/4 kuya ku-S/2 kuya ku-S/1, kusuka kokumahhadlahhadla kuya kokunemininingwane. Yonke le nqubo ayincikile kumamodeli angaphandle okuhlukanisa afana ne-SAM.
Ukuqeqeshwa okuyizifundo kanye nokufunda okuqinisa kwemisebenzi exubile: ngemva kokuqeqeshwa kwangaphambilini, imodeli idlula ngokulandelana ku-SFT yezigaba ezintathu: ukuvumelanisa umsebenzi, ukulungiswa kwezinqumo eziphezulu nezimo eziminyene, kanye nokufunda kweleveli yamaphikseli. Ekugcineni, isebenzisa ukufunda okuqinisa kwe-GRPO kwemisebenzi exubile ngamasampula angu-178K akhethwe ngokungaqiniseki. Amabhokisi anembe kakhulu futhi ama-hallucination ancipha. Amaphutha amabili agcizelelwa kakhulu: “ukudweba ibhokisi noma kungekho nto esithombeni” kanye “nokweqa ibhokisi lapho kukhona into”. Lokhu kuthuthukisa kanyekanye ikhono lokwenqaba nokunemba kokuthola izinto lapho zikhona.
Injini yedatha eqhutshwa yi-knowledge graph: i-knowledge graph yezigaba nezincazelo ezikhombayo isetshenziselwa ukuqoqa idatha eqondiswe ezintweni. I-SAM3 kanye ne-DINO detector yangaphakathi kukhiqiza amabhokisi angaba ngamakhandidethi, i-Qwen3-VL iqinisekisa ukuthi ayiqiniso yini, i-SAM2 iyawanweba abe amamaski, kanti i-BiRefNet ne-MatAnyone2 zikhiqiza ama-alpha matte angaba ngamakhandidethi. Isampula yamukelwa kuphela uma idlula uchungechunge lokuhlolwa olufaka i-IoU, izinga lokumboza nokunemba kwangaphambili. Ezinye zihlolwa ngesandla. Amaphutha emodeli abuyiselwa emzuliswaneni olandelayo wokukhetha. Sekubhalwe amalebula ezithombeni ezingaphezu kwezigidi ezingu-100 nakumabhokisi ezinto angaba yizigidi eziyizinkulungwane ezingu-3.6, kwakhiwa umjikelezo wokuqamba amalebula ozithuthukisayo.

Thola ikhodi: klona i-repository ye-Tencent/WeVisOne-VL ku-GitHub bese ufaka indawo edingekayo ngokulandela i-README.
Lungisa imodeli: landa izisindo ze-WeVisOne-VL-2B noma ze-WeVisOne-VL-4B ku-Hugging Face.
Yakha i-task prompt: bhala isidingo ku-prompt ngokomsebenzi, njengokuthi “Locate all the instances of …” ekutholeni izinto, incazelo yento ekutholeni into ekhonjwe yincazelo, noma isicelo sokukhipha i-alpha ekususeni ingemuva. Ungadlulisa namalebula akhona usebenzisa ama-tag ahlelekile afana no-, kanye no-.
Shayela i-inference: sebenzisa i-interface ye-model.predict() ehlinzekwa yi-repository, okuyiyona esetshenziswa kudemo esemthethweni, ukuze uthumele isithombe kanye ne-prompt kumodeli.
Hlaziya okukhiphayo: imodeli ikhiqiza umphumela nge-token ngayinye. Ama-coordinate abuyiselwa kuma-coordinate ajwayelekile kusuka kuma-token akhethekile athi <0>–<1000>. I- ivula i-decoder yamaphikseli ukuze ikhiphe imaski noma i-alpha; uma into ingekho, kubuyiselwa i-None.
Sebenzisa i-post-processing: buyisela amabhokisi, imaski noma umphumela wokususa ingemuva kusayizi wesithombe sokuqala. Lokhu kungasetshenziswa ekusesheni, ekubhaleni amalebula, ekushintsheni ingemuva nakweminye imisebenzi elandelayo.

Ukwenqaba okuthembekile: uma kungekho nto esithombeni, iyakwazi ukusho ukuthi “ayikho” futhi ayiphoqi impendulo, ngaleyo ndlela yehlise kakhulu ama-hallucination esigabeni sokubona.
Imodeli eyodwa, imisebenzi eminingi: ukuthola izinto, ukuthola into ekhonjwe yincazelo, ukubala, ukukhomba, i-OCR, ukuhlukanisa nokususa ingemuva konke kusebenzisa imodeli efanayo. Ukushintsha umsebenzi kudinga ukushintsha i-prompt kuphela, kunciphise izindleko zokufaka nokugcina amamodeli amaningi.
Ayidingi i-SAM yangaphandle: i-decoder yamaphikseli eyakhelwe ngaphakathi enama-parameter ayizigidi ezingu-9.6 ikhipha ngokuqondile imaski nokususa ingemuva okuningiliziwe okufana nezinwele, ithathe indawo ye-cascade ethi “detector+SAM+imodeli yokususa ingemuva” futhi yenze uchungechunge lwe-inference lube lufushane.
Ukunemba nokuhlukanisa okunemininingwane ngesikhathi esisodwa: amalebula amakhulu ezithombe eziyizigidi ezingu-100 namabhokisi ayizigidi eziyizinkulungwane ezingu-3.6, kuhlanganiswe nokuqeqeshwa okuyizifundo nokufunda okuqinisa kwemisebenzi exubile, kwenza idlule ama-VLM ajwayelekile namamodeli akhethekile kuma-benchmark angaphezu kuka-30. I-MAD yokususa ingemuva icishe yehliswe ngesigamu.
Umthombo ovulekile futhi iyafakeka: izisindo, ikhodi kanye namathuluzi e-2B ne-4B kuvuliwe ngokuphelele ngaphansi kwelayisensi ye-Apache 2.0. Ingahlolwa kuwebhu noma isebenze endaweni, futhi ilungele amaqembu amancane.

Iwebhusayithi esemthethweni yephrojekthi: https://tencent.github.io/WeVisOne-VL/
I-repository ye-GitHub: https://github.com/Tencent/WeVisOne-VL
Iqoqo lamamodeli e-HuggingFace: https://huggingface.co/collections/tencent/wevisone-vl
Iphepha lobuchwepheshe ku-arXiv: https://arxiv.org/pdf/2610.31329

Ukuhlolwa kokuqukethwe nokuphepha: ngenxa yekhono layo lokwenqaba elithi “iyakwazi ukusho ukuthi ayikho”, ithola ngokunembile izinto ezingavunyelwe esithombeni. Uma ingazitholi, ibuyisa ngokusobala ukuthi akukho lutho, inciphise ukweqiwa nokubikwa okungamanga futhi ihambisane nezidingo zokuhlola okuqukethwe kwe-WeChat.
Ukususa ingemuva kwemikhiqizo ye-e-commerce: faka isithombe somkhiqizo bese ikhipha ngokuqondile i-alpha matte enemininingwane yezinwele, yenze izithombe ezinesizinda esimhlophe namaphosta anezizinda ezishintshiwe ngokuchofoza okukodwa, ithathe indawo yokulungiswa ngesandla kanye ne-pipeline yamamodeli amaningi.
Ukuhlelwa kwamadokhumenti namarisidi: i-OCR kanye nezindawo ezihambisanayo ziguqula ama-invoice, izinkontileka namafomu abe yidatha ehlelekile enama-coordinate, kuvumele ukufakwa kwedatha nokubuyisaniswa kwama-akhawunti okuzenzakalelayo.
Ukubhala amalebula edatha nokuthuthukisa ukusesha: sebenzisa ukuthola into ekhonjwe yincazelo nokuhlukanisa ukuze ukhiqize ngobuningi idatha enamalebula anamabhokisi namamaski, noma ubuyise indawo enembile yento lapho usesha ngesithombe usebenzisa umbhalo, esikhundleni sesithombe sonke, ukuze kusekelwe ukuqeqeshwa kwamamodeli nokusesha okulandelayo.
Ukubona kwamarobhothi nezimoto ezizishayelayo: ukufakwa ohlangothini kwenza ukubona indawo nokuphatha izinto. Indlela yokwenqaba igwema i-hallucination “yokuthatha umoya njengento evimbayo” futhi ithuthukise ukuphepha komsebenzi.