H3-World Uyasebenza

I-H3-World iyimodeli yomhlaba elawulwa yizenzo, eyethulwe yi-Tencent ngokubambisana ne-National University of Singapore kanye ne-Hong Kong Polytechnic University, futhi ilungiswe kahle kusetshenziswa ijeneretha yamavidiyo i-MiniMax-H3 engu-33B. Imodeli ihumusha okokufaka kwekhibhodi kube imiyalelo yesiNgisi, ikufake ngemzila wombhalo oqeqeshwe kusengaphambili, bese isebenzisa ukuqondiswa kokunaka kwesikhathi ukuze ixhumanise umyalo ngamunye nohlaka lwevidiyo oluhambisanayo, ngaleyo ndlela inikeze ukulawulwa okunembile okuhambisana nesikhathi. I-H3-World isebenzisa amavidiyo emidlalo angu-8000 kuphela, iqeqeshe kahle i-LoRA ye-rank-32 ku-0.199% yamapharamitha, futhi ivumela imodeli ukuthi ikhiqize izithombe ezilawulekayo ngesikhathi sangempela ngokusekelwe kokokufaka kwekhibhodi. Iphinde ijwayele ezindaweni engakaze izibone, okuyisinyathelo esibalulekile esiya emhlabeni okhiqizwayo osebenzisanayo.

Ukulawula ividiyo ngezenzo zekhibhodi: Kuguqula ukunyakaza komlingiswa nokusebenza kwekhamera okufakwa ngekhibhodi kube izinguquko ezihambisanayo esithombeni sevidiyo, kwakhe ukulingisa komhlaba osebenzisanayo lapho ukucindezela ukhiye kulawula khona isenzo.
Ukulawula okunembile kwesikhathi: Izenzo ezihlukene zingasetshenziswa ngezikhathi ezihlukene zevidiyo efanayo, kulawulwe ngokunembile ukuthi isenzo senzeka nini.
Ukujwayela ezindaweni eziningi: Kusukela ekuqeqeshweni ngezithombe zemidlalo, imodeli ijwayela ezindaweni ezihlukahlukene zokubuka, njengamadolobha akhanyiswe izibani ze-neon, ama-oasis asogwadule namabandla ezinganekwane, kuyilapho igcina okuqukethwe kwendawo kuzinzile.
Ukujwayela kwezenzo ezihlanganisiwe: Ihlanganisa izingxenye eziyisisekelo zezenzo zomlingiswa noma zekhamera ezibonwe ekuqeqeshweni ukuze yakhe amapheya wezenzo angakaze avele ngaphambilini, inikeze ukulawulwa okujwayela izinhlanganisela ezintsha.
Ukukhiqizwa kwesikhathi eside: Isekela ukukhiqizwa kwamavidiyo amade, aqhubekayo futhi alawulekayo, lapho imiyalelo yezenzo iqhubeka iqondisa khona ukuvela kwesithombe.

Landela i-WeChat bese uphendula ngokuthi “开源” ukuze ujoyine iqembu lokuxoxa ngamaphrojekthi e-AI avulekile
Ukumelwa kwezenzo ngolimi: Ukusebenza kwekhibhodi kohlaka ngalunye kuhunyushwa kube umyalo omfushane wesiNgisi, njengokuthi “strafe left”, ukuze izenzo zime ngendlela yolimi lwemvelo.
Ukusebenzisa kabusha umzila wombhalo oqeqeshwe kusengaphambili: Imiyalelo ifakwa ngemzila yokuqala yokufaka umbhalo ye-MiniMax-H3, kusetshenziswe ngokuqondile ukumelwa kwencazelo okufundwe ngesikhathi sokuqeqeshwa kwevidiyo. Ngalokho imodeli iyayiqonda ngokwemvelo incazelo yezenzo ngaphandle kokufunda ikhodi yezenzo kusukela ekuqaleni.
Ukuqondiswa kokunaka kwesikhathi: Imaski yokunaka eqondisiwe ixhumanisa umyalo ngamunye namafreyimu afanele eziguquko ezifihlekile zevidiyo, ikhawulele umthelela womyalo esikhathini esihlelwe kusengaphambili futhi igweme ukuvuza kokulawula phakathi kwezenzo eziseduze. Lokhu kuyisihluthulelo sokulawula okunembile kwesikhathi.
Ukulungiswa okuncane kwe-LoRA: Kuqeqeshwa kuphela i-LoRA ye-rank-32 kumaphrojekthi wokunaka komuntu siqu e-H3. Kusetshenziswa u-0.199% wamapharamitha aqeqeshwayo, amavidiyo emidlalo angu-8000 nezinyathelo zokuthuthukisa ezingu-10000 ukuze kuqedwe ukujwayelanisa, kwehliswe kakhulu izindleko zokuguqula ijeneretha yamavidiyo ibe yimodeli yomhlaba.

Indlela yokuqala: Ukuyizama ku-inthanethi
Indlela yesibili: Ukuyifaka endaweni yakini
Vula i-Space yedemo esebenzisanayo ku-Hugging Face (hugging-apps/h3-world-action-demo); ungazama ukulawula okuqhutshwa yikhibhodi ngaphandle kokulanda izisindo ezingu-135GB.
Layisha noma khetha isithombe sohlaka lokuqala, bese ufaka i-prompt echaza indawo.
Faka imiyalelo yokulawula usebenzisa izenzo ezisethiwe noma okhiye bekhibhodi, okuhlanganisa ukunyakaza komlingiswa nokusebenza kwekhamera.
Imodeli ikhiqiza ngesikhathi sangempela isiqeshana sevidiyo esilawulekayo esingaba imizuzwana engu-5.2, esingu-832×480.
Ukulungiselela imvelo: Dala imvelo ye-conda ene-Python 3.10 + CUDA 12.8, bese ufaka i-PyTorch 2.10 kanye nokuncika okukwi-requirements.txt.
Ukuthola ikhodi: Klona inqolobane esemthethweni ye-H3-World, bese uklona i-DiffSynth-Studio uye kufolda ebekiwe, ushintshele ku-commit ebekiwe bese ufaka i-patch esemthethweni yokunaka. Lokhu kuyadingeka, kungenjalo i-LoRA ngeke isebenze.
Ukulanda izisindo: Ku-Hugging Face, landa izisindo eziyisisekelo ze-MiniMax-H3, ezingaba ngu-135GB, kanye ne-H3-World LoRA (step-10000.safetensors), ukubeke kumafolda ahambisanayo enqolobaneni.
Ukumisa inqolobane: Sebenzisa i-env.sh ukuze uqondise izinqolobane ze-Hugging Face, Torch ne-Triton ngaphakathi kwenqolobane.
Ukuqhuba inference: Nikeza isithombe sohlaka lokuqala, isimo esimile sencazelo kanye nokulandelana kwezenzo okuhlelwe ngokwesikhathi, lapho ingxenye ngayinye ihambisana nomyalo owodwa womlingiswa noma wekhamera.
Umphumela wokukhiqiza: Imodeli ikhiqiza izinguquko ezifihlekile zevidiyo ezihambisana neshejuli yezenzo bese izikhipha ngemva kokuziguqula zibe yividiyo; inganwetshwa ngokuzenzakalela ukuze kukhiqizwe isikhathi eside.

Idizayini enhle engenamamojuli angeziwe: Isebenzisa kabusha umzila wombhalo oqeqeshwe kusengaphambili, iguqule inkinga yokulawula izenzo ibe yinkinga yokuqonda ulimi imodeli esivele inekhono kuyo.
Izindleko zokuqeqesha eziphansi kakhulu: Ngamavidiyo emidlalo angu-8000 kuphela, izinyathelo zokuthuthukisa ezingu-10000 kanye ne-LoRA ye-rank-32 eqeqesha kahle u-0.199% wamapharamitha, ivula amandla okulawula asebenzisanayo emodelini engu-33B.
Ikhono lokulawula isikhathi ngokunembile: Ngokusebenzisa ukuqondiswa kokunaka kwesikhathi, izenzo ezahlukene zingenziwa ngokweshejuli efanayo yevidiyo, kuxazululwe inkinga yokuthi i-prompt yomhlaba wonke ayikwazi ukulawula ngokunembile isikhathi sesenzo.
Ingcono kakhulu kunezindlela zendabuko zokufaka izimo: Uma iqhathaniswa nezindlela eziqondile zezenzo ezifana ne-additive-bias ne-FiLM, indlela esekelwe olimini ikhiqiza ukunyakaza okuhambisanayo komlingiswa kanye nekhamera, ngaphandle kokuphazamisa okuqukethwe kwendawo.
Ikhono elinamandla lokujwayela izinhlanganisela: Ihlanganisa izingxenye eziyisisekelo zezenzo ezibonwe ekuqeqeshweni ukuze yakhe amapheya wezenzo angakaze avele, iphinde ijwayele ezindaweni zokubuka ezingaphandle kokusatshalaliswa kokuqeqeshwa.
Ukusebenza okuvamile: Imodeli eyodwa ingaphatha umuntu obuka kumuntu wokuqala noma wesithathu, ukunyakaza komlingiswa nokunyakaza kwekhamera kanye nezinye izinhlobo zokulawula, ngaphandle kokuqeqesha umsebenzi ngamunye ngokwehlukana.

Iwebhusayithi yephrojekthi: https://danzer1xxxxchan.github.io/H3-World/
Inqolobane ye-GitHub: https://github.com/Danzer1xxxxChan/H3-World
Inqolobane yemodeli ye-HuggingFace: https://modelscope.cn/models/DANNY621/H3-World
Iphepha lobuchwepheshe: https://modelscope.cn/papers/2609.01560

Ukuthuthukiswa kwe-prototype yemidlalo: Umklami angafaka isithombe sokuqala bese “edlala” indawo ebonakalayo ngesikhathi sangempela, ahlole ngokushesha umqondo wezinga kanye nolimi lwekhamera ngaphandle kokwakha kuqala injini yomdlalo ephelele.
Indawo yokuhlola ama-AI Agent: Ihlinzeka ama-agent anomzimba ngendawo ebonakalayo esebenzisanayo; i-agent ihlola umhlaba ngezenzo zekhibhodi futhi ithole impendulo yokubuka, kwehliswe izindleko zokuqeqeshwa emhlabeni wangempela.
Ukubuka kuqala kwefilimu nokugqwayiza: Umqondisi ulawula ikhamera ebonakalayo ngekhibhodi, njengokuyizungezisa, ukuyisusa nokuyiphakamisa noma ukuyiyehlisa, bese ekhiqiza ukubuka kuqala kwama-storyboard ngesikhathi sangempela, kuncishiswe kakhulu isikhathi sokukhiqiza i-previz.
Ukuhlolwa kwamasu okusebenza kwamarobhothi: Imiyalelo yokunyakaza yengalo yerobhothi noma ye-drone iguqulelwa olimini lwezenzo, bese kuhlolwa ukulandelana kwesikhathi kwezenzo endaweni yokulingisa ekhiqiziwe.
Ukudalwa kokuqukethwe okugxilisa umsebenzisi: Ngokuhlanganisa i-VR nokusebenzisana kwemidlalo, kuba ithuluzi le-UGC elivumela abasebenzisi ukuthi baqhube ukuvela komhlaba obonakalayo owenziwe ngezifiso ngokusebenzisa okhiye bemvelo, ukuze wonke umuntu akwazi ukukhiqiza “ividiyo edlalekayo”.