VDN-MiniMax-H3 Uyasebenza

I-VDN-MiniMax-H3 iyisixazululo sokusheshisa ukukhiqizwa kwamavidiyo esivulwe umphakathi yiqembu le-OpenVDN, esakhiwe phezu kwe-MiniMax-H3. Imodeli isebenzisa isakhiwo sokunaka esixubile: i-Softmax yewindi elislayidayo lendawo igcina imininingwane emihle, ukunaka komugqa kwebanga elide kusingatha umongo webanga elide, bese izinyathelo ezi-8 zokufundisa kabusha zinciphisa kakhulu izinyathelo zokususa umsindo. Kuma-NVIDIA B200 angu-8, imodeli idinga imizuzwana engu-11.23 kuphela ukukhiqiza ividiyo engu-768p ethatha cishe imizuzwana engu-14.4, kuyilapho ikhwalithi yesithombe icishe ingalahleki uma iqhathaniswa ne-Dense H3 yasekuqaleni enezinyathelo ezingu-50.

Isakhiwo sokunaka esixubile: Sihlukanisa ukunaka kwevidiyo kube igatsha lendawo le-Softmax negatsha lomugqa lebanga elide, sihlanganise imininingwane enembile nomongo webanga elide.
Ukukhiqiza okusheshayo ngezinyathelo eziyi-8: Ngokusebenzisa i-DMD2 distillation, inani lezinyathelo zokususa umsindo lehliswa lisuka kwezingama-50 laya kweziyi-8, kunciphise kakhulu isikhathi sokukhiqiza.
Ikhwalithi yesithombe ecishe ingalahleki: Ngenkathi kukhiqizwa ividiyo yemizuzwana engu-14.4 ngemizuzwana engu-11.23, ikhwalithi ebonakalayo isondela kakhulu kwisisekelo sokuqala se-Dense H3 sezinyathelo ezingama-50.
Ukuj адапtha ukuqeqeshwa kwezigaba ezintathu: Kusetshenziswa ukuqondanisa kwesendlalelo ngesendlalelo, ukuj адапtha kwegatsha kusukela ekuqaleni kuye ekugcineni, kanye nokuj адапtha okuhlangene kwe-LoRA, kugwenywe ukuphazamiseka kwemodeli eqeqeshwe ngaphambilini okubangelwa ukuqaliswa okungahleliwe.
Ukufana okukhethelwe amagatsha: Amagatsha e-Softmax nomugqa abelwa kuma-GPU ahlukene, kwehliswe ukubambezeleka ngo-13.3% uma kuqhathaniswa nokufana okujwayelekile kwe-Ulysses.

Landela i-WeChat, uphendule ngokuthi “开源”, bese ujoyina iqembu lezingxoxo lamaphrojekthi e-AI anomthombo ovulekile
Isakhiwo sokunaka esixubile: I-VDN ihlukanisa ukunaka kwevidiyo kube amagatsha amabili ahambisanayo, i-Softmax yasendaweni kanye ne-linear ekude; elokuqala ligcina imininingwane enembile, kanti elesibili licubungula umongo webanga elide ngobunkimbinkimbi be-linear, ngaleyo ndlela lithathe indawo yokunaka okugcwele okunobunkimbinkimbi besikwele.
Igama le-Softmax yasendaweni: Kusetshenziswa iwindi elislayidayo eliya kuzo zombili iziqondiso. Amafreyimu angu-5 ahlanganiswa njengeqembu ukuze kugxilwe emabhulokini aseduze, bese kufakwa ifreyimu yokuqala neyokugcina njengamaphuzu okubambelela emingceleni. Lokhu kwandisa ukuminyana kokunaka ngo-3.57% kuphela, kodwa kugcina ukuvumelana kwesikhathi emhlabeni wonke.
I-Video Delta Attention: Umthetho wendabuko we-Delta osebenza nge-token ngayinye uthuthukiselwa ekuxazululeni okuhlangene ngefreyimu ngayinye. Ngezibalo ezijwayelekile, wonke ama-token esikhala anquma ngokuhlanganyela isimo esisha; lokhu akukhulisi ubukhulu ngokwemvelo futhi kungakwazi ukuzivumelanisa nokuhlobana kwama-token, ngaphandle kwesidingo sokukala ngokuqinile usayizi wefreyimu.
Ukufakwa kombhalo ngamagatsha amabili: Embhalweni wegatsha le-Softmax, ifreyimu ngayinye ingawubona umbhalo wonke emhlabeni wonke; kanti igatsha le-linear liwubhala kanye kuphela esimweni esiya kuzo zombili iziqondiso ngesikhathi sokuqalisa, ngaleyo ndlela ligweme ukubala okuphindwayo.
Indlela yokuhlanganisa enamasango: Imiphumela yamagatsha amabili inamazinga ahlukene. Ngalinye lifakelwe isango le-Sigmoid elihlobene nokuqukethwe kanye ne-projection ezimele yokukhipha, bese kuhlanganiswa imiphumela. Isango le-Softmax ligcinwa limisiwe ezigabeni ezimbili zokuqala zokuqeqesha, ukuvimbela i-optimizer ekwehliseni ukulahlekelwa ngokunciphisa igatsha lokuqala.
Ukujwayelaniswa kokuqeqeshwa ngezigaba ezintathu: Okokuqala kuqondaniswa igatsha le-linear ngokwesendlalelo, bese wonke amamojuli axubile ethuthukiswa ngokuhlanganyela kusukela ekugcineni kuya ekugcineni. Ekugcineni, kwenziwa i-fine-tuning ngokuhlanganyela kwe-QKVO LoRA negatsha le-linear. Umgogodla oqeqeshwe kusengaphambili ugcinwa umisiwe kuyo yonke inqubo, ukuze kuqinisekiswe ukuthi ikhwalithi yokukhiqiza ayonakali.

Ukulungiselela indawo: Ngemva kokwenza i-clone yekhosombe le-GitHub, dala indawo ye-Python 3.12, bese ufaka i-PyTorch 2.13.0 kanye nez依依 zephrojekthi.
Ukulanda imodeli: Sebenzisa i-ModelScope ukulanda izisindo ze-OpenVDN/vdn-minimax-h3 uzigcine kufolda yendawo ethi ckpts.
Ukuqalisa isibonelo esisemthethweni: Sebenzisa i-scripts/inference/8nfe_tuned_fp8.sh ukuze ukhiqize ividiyo yesibonelo ngekhadi elilodwa.
Ukuqagela ngamakhadi amaningi: Khetha iskripthi esithi 8nfe_tuned_fp8_ulysses_h200.sh noma b200.sh ngokuya ngehadiwe, bese usiqalisa kumakhadi angu-8.
Ukwenza ngokwezifiso iziqondiso: Sebenzisa i-encode_prompt.py ukubiza i-Qwen3-VL-32B ukuze ifake umbhalo kufayela lezici le-.pt, bese ulidlulisela ku-infer.py ukuze ukhiqize ividiyo.

Ukukhiqiza ngesivinini esikhulu: Ukudistila ngezinyathelo eziyi-8 kuhlanganiswe nokunaka okuxubile, kukhiqiza ividiyo engu-768p ecishe ibe yimizuzwana engu-14.4 ngemizuzwana engu-11.23 kuphela kuma-B200 angu-8, kushesha ngokuphindwe izikhathi ezingu-74.5 kunemodeli yokuqala.
Ikhwalithi yesithombe ecishe ingalahleki: Imininingwane ebonakalayo, ukuvumelana kwento eyinhloko kanye nekhono lokulandela imiyalelo kusondelene nesisekelo se-Dense H3 sezinyathelo ezingu-50, futhi kuphakeme kakhulu kune-FastH3 yezinyathelo ezi-4.
Isakhiwo sokunaka okuxubile: I-Softmax yasendaweni igcina imininingwane yesikhathi eseseduze, kuyilapho ukunaka komugqa kwebanga elide kucubungula umongo webanga elide ngobunkimbinkimbi bomugqa, kuhlanganisa ikhwalithi nokusebenza kahle.
Ukuxazulula okuhlanganyelwe kwezinga lohlaka (VDA): Kuthuthukisa ukubuyekezwa kwe-token ngayinye kube ukuthuthukisa okuhlanganyelwe kohlaka lonke, okungakhulisi ngokwemvelo futhi okujwayelanisa nobudlelwano bama-token, ngaphandle kwesidingo sokukala okuphoqelelwe.
Ukujwayelanisa okuzinzile ngezigaba: Ngokuqeqeshwa kwezigaba ezintathu okubandakanya ukuqondanisa kwesendlalelo ngesendlalelo, ukujwayelanisa kwegatsha kusukela ekuqaleni kuye ekugcineni kanye nokulungisa okuhlanganyelwe kwe-LoRA, kugwenywa ukuphazamiseka kwemodeli eqeqeshwe kusengaphambili okubangelwa ukuqaliswa okungahleliwe.
Ukufana okukhethekile kwegatsha: Amagatsha e-Softmax ne-VDA abelwa kuma-GPU ahlukene ukuze asebenze, kunciphisa ukubambezeleka ngo-13.3% ngaphezu kokufana okujwayelekile kwe-Ulysses.

Iwebhusayithi esemthethweni yephrojekthi: https://openvdn.github.io/
Indawo yokugcina ku-GitHub: https://github.com/OpenVDN/vdn-minimax-h3
Indawo yamamodeli ku-HuggingFace: https://huggingface.co/OpenVDN/vdn-minimax-h3

Ukudala amavidiyo asebenzisanayo ngesikhathi sangempela: Ukwakhiwa okusheshayo ngezinyathelo eziyi-8 kuvumela abadali ukuthi balungise imiyalelo ngesikhathi sangempela futhi babuke imiphumela ngokushesha, kunciphise kakhulu isikhathi sokuphindaphinda imibono.
Ukukhiqizwa ngobuningi kwezinto zokukhangisa nezokumaketha: Izinkampani ze-e-commerce nemikhiqizo zingakha ngokushesha izinguqulo eziningi zamavidiyo okubonisa imikhiqizo namaphosta ashukumisayo, zihlangabezane nezidingo ezivamile zezinto zokukhangisa ngezikhathi zokukhushulwa kwamaholide nokunye.
Ukubikezela amafilimu namabhodi ezithombe ashukumisayo: Abaqondisi namathimba okukhiqiza bangakha ngokushesha izithombe ezinyakazayo zamabhodi ezithombe ngaphambi kokuqoshwa正式, ukuze baqinisekise ulimi lwekhamera, ukuhlelwa kwezigcawu nesigqi sokuxoxa indaba.
Ukukhiqizwa kokuqukethwe okushukumisayo kwemidlalo: Ukukhiqiza ngesikhathi sangempela izithombe ezinyakazayo zama-NPC, izigcawu ezidlulayo zemvelo neziqephu zezindaba ezenzelwe abadlali emidlalweni yomhlaba ovulekile noma ezinkundleni zokuxhumana ezingokoqobo.
Ukukhiqizwa kwezimboni kwamavidiyo amafushane ezinkundleni zokuxhumana: Izikhungo ze-MCN nabadali abazimele bokuqukethwe bangakhiqiza ngezindleko eziphansi nangomthamo omkhulu amavidiyo amafushane afana nezindaba ezimfushane nama-Vlog abalingiswa abangokoqobo.