DMAD Uyasebenza

I-DMAD (Distribution Matching as Adversarial Distillation) iwuhlaka lokucindezela olunezinyathelo ezimbalwa, olukhishwe njengomthombo ovulekile ngokubambisana yi-ByteDance ne-Texas A&M University, olusheshisa ukukhiqizwa ngamamodeli okusabalalisa. I-DMAD iqeqesha umhlukanisi wezigaba ezimbili ukuthi afunde ngokuqondile umehluko phakathi kokusatshalaliswa okuqondiwe nokusatshalaliswa komfundi, isuse isidingo semodeli yokusabalalisa eyisizayo, futhi inciphise imodeli kathisha yezinyathelo ezingu-50 ibe imodeli yomfundi yezinyathelo ezingu-4. Manje isekela i-ImageNet, i-SDXL, i-Wan2.1 kanye ne-MiniMax-H3-33B; i-8GB VRAM ingakhiqiza ividiyo enomsindo engaba imizuzwana engu-5 enobukhulu obungu-1344×768.

Ukukhiqizwa kwezithombe ngezinyathelo ezimbalwa: imodeli ye-EDM ye-ImageNet-64 incishiselwa ekusampuleni kwesinyathelo esingu-1, i-FID yehle ifike ku-1.04, kanti i-SDXL idinga izinyathelo ezingu-4 kuphela ekukhiqizeni umbhalo ube isithombe.
Ukukhiqizwa kwevidiyo ngombhalo ngezinyathelo ezimbalwa: i-Wan2.1-T2V-14B incishiselwa ekubeni imodeli yomfundi yezinyathelo ezingu-4, ithole i-VBench engu-85.15.
Ukukhiqizwa okuhlangene komsindo nevidiyo: imodeli kathisha yezinyathelo ezingu-50 ye-MiniMax-H3-33B incishiselwa ezinyathelweni ezingu-4, ikhiphe ividiyo engu-1344×768 nomsindo we-stereo wendabuko ongu-32kHz.
Ukwesekwa kwehadiwe yabathengi: kunikezwa ukugeleza komsebenzi kwe-ComfyUI okune-8GB VRAM kanye nokucubungula ngomugqa womyalo onokusetshenziswa okuphansi kwe-VRAM; ikhadi lemifanekiso elingu-8GB lingakhiqiza ividiyo enomsindo engaba imizuzwana engu-5, kanti elingu-12GB lingakhiqiza engaba imizuzwana engu-15.
Umthombo ovulekile ngokuphelele wokuqeqesha nokucubungula: kukhishwa indlela yokucindezela, ikhodi yokuqeqesha, imibhalo yokucubungula kanye nezisindo ze-LoRA zamamodeli amaningi, kusekelwa kokubili umugqa womyalo ne-ComfyUI.

Landela i-WeChat bese uphendula ngokuthi “umthombo ovulekile” ukuze ujoyine iqembu lokuxhumana lamaphrojekthi e-AI omthombo ovulekile
Ukufanisa ukusabalaliswa njengokucindezela okuphikisanayo: I-DMAD iqeqesha umhlukanisi wezigaba ezimbili ukuze ahlukanise amasampuli okuqondiwe namasampuli omfundi. I-logit yomhlukanisi ongcono kakhulu ilingana nesilinganiso samalogarithmu obuningi bala masampuli amabili; ukuthola i-gradient kokufakiwe kwawo kunikeza umehluko wama-score, ngaleyo ndlela kunikeze ngokuqondile indlela yokulungisa yomfundi futhi kususe imodeli eyisizayo yokulinganisa ama-score.
Isisekelo esabiwe nomhlukanisi onamakhanda amabili: umhlukanisi usebenzisa isisekelo esisodwa sezici esabiwe namakhanda amabili okuhlukanisa. Ikhanda likathisha lihlukanisa amasampuli kathisha namasampuli omfundi, kanti ikhanda ledatha yangempela lihlukanisa amasampuli angempela namasampuli omfundi; womabili aqeqeshwa kusetshenziswa i-binary cross-entropy. Umfundi ubuyekezwa ngokulahleka komugqa okunesisindo kwama-logit amabili; ngesikhathi sokubuyekeza amapharamitha omhlukanisi ayamiswa, kodwa i-gradient iyaqhubeka ibuyela emuva.
Ukukala okunamandla kwezikhawu zomsindo: umehluko wokusabalaliswa phakathi kwamasampuli kathisha nedatha yangempela uyashintsha kuye ngezinga lomsindo. I-DMAD isebenzisa umehluko omaphakathi wama-logit ekhanda ledatha yangempela phakathi kwezinhlobo ezimbili zamasampuli njengophawu lwesipiliyoni, ilandele izikhawu zomsindo ezahlukene ngesilinganiso esinyakazayo esincipha ngo-0.99. Izikhawu ezinomehluko omncane zithola isisindo esiphezulu sikathisha, bese kuthi ngemva kokujwayelekile zibambe iqhaza ekulahlekeni komfundi.
Ukusheshisa ukuqeqeshwa ngamasampuli kathisha angaxhunyiwe ku-inthanethi: uthisha uhlinzeka kuphela ngamasampuli akhiqizwe futhi agcinwe kusengaphambili, ngakho akudingeki abizwe ku-inthanethi ngesikhathi sokuqeqesha. Ngama-H200 angu-4 kanye nokucushwa kwe-MiniMax-H3 rank-128 LoRA, isikhathi sokubuyekeza umfundi kanye nokubuyekeza okuhlobene komhlukanisi sehla sisuka kumasekhondi angu-818.1 saya kumasekhondi angu-233.3, okwenza ukuqeqeshwa kusheshe cishe izikhathi ezingu-3.51.

Clone indawo yokugcina: sebenzisa i-git clone https://github.com/Yzmblog/DMAD.git bese ungena kufolda yephrojekthi.
Dala indawo: sebenzisa i-conda ukudala indawo ye-Python 3.12 bese uyivula.
Faka okudingekayo: landela izinguqulo ezinqunyiwe ezisemthethweni ufake i-PyTorch 2.11.0 (cu126), i-torchvision ne-torchaudio, bese usebenzisa i-pip install -r requirements.txt kanye ne-pip install -U modelscope-hub.
Lungisa ihadiwe: qinisekisa ukuthi unekhadi lemifanekiso le-NVIDIA elisekela i-CUDA, futhi ubeke eceleni cishe u-64GB wememori yesistimu kanye no-170GB wesikhala sediski.
Landa imodeli eyisisekelo: sebenzisa i-modelscope download MiniMax/MiniMax-H3 ukulanda imodeli eyisisekelo ye-MiniMax-H3.
Landa izisindo zokucindezela: landa i-LoRA yezinyathelo ezingu-4 ye-DMAD ku-ModelScope.
Ukucubungula ngomugqa womyalo: sebenzisa i-inference.py, ucacise indlela yemodeli, i-LoRA, ifayela leziyalezo, --steps 4, --height 768 --width 1344, bese ukhetha --low-vram 8 noma --low-vram 12 kuye nge-VRAM etholakalayo.
Buka okukhiphayo: ifolda yokukhiphayo iqukethe i-0000.mp4, i-txt yeziyalezo kanye ne-settings.json; ngemva kokuphumelela ungayishintsha ifayela leziyalezo ufake elakho.

Ukusheshisa okukhulu kokucubungula: imodeli kathisha yezinyathelo ezingu-50 incishiselwa imodeli yomfundi yezinyathelo ezingu-4, kuncishiswe ukuphindaphindwa kokususa umsindo ngaphezu kuka-92%, kuyilapho kugcinwa ikhwalithi yokukhiqiza eseduze neka-thisha.
Ukusebenza kahle okuphezulu kokuqeqesha: umhlukanisi ufunda ngokuqondile umehluko wokusabalaliswa, ususe imodeli yokusabalalisa eyisizayo; kuma-H200 angu-4, ukubuyekezwa kokuqeqesha kushesha cishe izikhathi ezingu-3.51.
Ikhwalithi yokukhiqiza ehamba phambili: ukukhiqizwa kwesinyathelo esisodwa kwe-ImageNet-64 kufinyelela ku-FID engu-1.04, kungcono kunezindlela ezifana ne-DMD2 ne-SiDA; ekuhlolweni kwabantu, izinga lokuthandwa ngokujwayelekile uma kuqhathaniswa ne-DMD2 lingu-79.1%, kanti uma kuqhathaniswa ne-rCM lingu-84.6%.
Ukumboza izimo eziningi zamamodali: uhlaka olulodwa lusekela izithombe, ividiyo ekhiqizwa ngombhalo kanye nokukhiqizwa okuhlangene komsindo nevidiyo.

Iwebhusayithi yephrojekthi: https://yzmblog.github.io/projects/DMAD/
Indawo yokugcina ye-GitHub: https://github.com/Yzmblog/DMAD
Ilabhulali yamamodeli ye-HuggingFace: https://huggingface.co/ZhengmingYu/DMAD
Iphepha lobuchwepheshe le-arXiv: https://arxiv.org/pdf/2610.02188

Ukudala okuqukethwe kwamavidiyo amafushane: khiqiza ividiyo yemizuzwana engu-5–15 enomsindo wendabuko ngezinyathelo ezingu-4; abadali bangakhiqiza ngokushesha izinto zemidiya yomphakathi ngekhadi lemifanekiso labathengi elingu-8GB, banciphise kakhulu izindleko zokukhiqiza.
Ukukhiqizwa kwezinto zokukhangisa nezokumaketha: amabhrendi angakhiqiza ngobuningi amavidiyo okubonisa imikhiqizo nezigcawu ngokusekelwe eziyalweni, ahlanganise nokukhipha umsindo osezingeni eliphezulu ukuze anciphise isikhathi kusukela emcabangweni wokukhangisa kuya kumkhiqizo ophelile.
Ukubuka kuqala kwamafilimu nokuklama imiqondo (Previs): abaqondisi namaqembu obuciko basebenzisa ukucindezela ngezinyathelo ezimbalwa ukukhiqiza ngokushesha ukubukwa kuqala kwezigcawu, izinkomba zesimo kanye nemidwebo yokuqala yamabhodi ezigcawu, kusheshiswe ukuphindaphindwa kwemibono yesigaba sokuqala.
Ukukhiqizwa kwempahla yemidlalo namaanimethi: khiqiza ama-animethi abalingiswa, izithombe zemiqondo yezigcawu nezinto zangemuva ezinyakazayo, usize izitudiyo zemidlalo ezizimele namaqembu amancane opopayi ukunciphisa izindleko zokukhiqiza impahla.