AuK Uyasebenza

I-AuK iyimodeli eyisisekelo yokukhiqiza nokuhlela inkulumo enamapharamitha ayizigidi eziyizinkulungwane ezingu-1.5, ekhishwe njenge-open source yi-Tencent Hunyuan. Isebenzisa ukwakheka kwe-flow-matching diffusion esikhundleni se-autoregressive. Imodeli ingenza imisebenzi engu-16 yenkulumo ngesixhumi esisodwa semiyalo yolimi lwemvelo, okuhlanganisa i-zero-shot/instruct TTS, ukuhlela okuqukethwe kwenkulumo, ukulungisa izici ze-acoustic, ukuhlela imisindo ehambisana nolimi, ukunciphisa umsindo nokuhlukanisa izwi lomuntu nokunye. Imodeli inikeza inguqulo esheshayo i-AuK-Flash esebenzisa inference yezinyathelo ezi-4, enesivinini esikhuphuka cishe ngo-4.5x.

Zero-shot TTS: Sebenzisa imizuzwana embalwa yomsindo wereferensi ukukopisha umbala wezwi nokuhlanganisa inkulumo entsha.
Instruct TTS: Chaza izici zezwi ngombhalo kuphela ukuze kukhiqizwe inkulumo ehambisanayo, ngaphandle komsindo wereferensi.
Ukuhlela okuqukethwe kwenkulumo: Shintsha, faka noma susa amagama ekurekhodweni ngaphandle kokushintsha umbala wezwi noma iphimbo.
Ukuhlela amagama ezingoma: Gcina umculo nezici zokucula, uguqule kuphela amagama engoma.
Ukuhlela iphimbo/isivinini/ivolumu: Lungisa ngokunembile izici ze-acoustic usebenzisa ama-semitone, isivinini esiphindwe kabili nama-decibel.
Ukuhlela imizwa nombala wezwi: Shintsha umuzwa ngaphandle kokushintsha amagama, noma uguqule umbala wezwi ugcine okuqukethwe.
Ukuhlela isisho nokunciphisa umsindo: Susa isisho sendawo, unciphise umsindo nokunanela ukuze kuthuthukiswe ukucaca.
Ukuhlela imisindo ehambisana nolimi: Faka noma ususe ukuphefumula, ukuhleka nokukhwehlela, uguqule phakathi kwenkulumo evamile nokuhleba.
Ukuhlukanisa inkulumo: Hlukanisa umsindo wabantu abaningi ngokulandelana kokukhuluma, noma ukhiphe isikhulumi esithile ngokwencazelo yokuqukethwe.
Ukuhlukanisa umculo: Khipha izwi engomeni, ususe okukhapha umculo.

Landela i-WeChat bese uphendula ngokuthi “开源” ukuze ujoyine iqembu lezingxoxo lamaphrojekthi e-AI open source
Indlela iyonke: I-AuK isebenzisa ukwakheka kwe-flow-matching diffusion ukukhiqiza ukumelwa okufihlekile kwendawo yomsindo, okulungele ngokwemvelo imisebenzi “yokuhlela”. Isebenzisa ukumelwa okufihlekile komsindo wokuqala njengesiqalo sokususa umsindo, iguqule kuphela izingxenye ezidingwa umyalo, ngaleyo ndlela igcine kangcono umbala wezwi, umculo nokuqukethwe okungahlelwanga. Lokhu kuyisisekelo sobuchwepheshe esenza ikwazi ukumboza umndeni wemisebenzi yokuhlela.
Ukwakhiwa kwepayiphi enezingxenye ezintathu: Uhlelo lunezingxenye ezintathu: imodeli enkulu ye-multimodal ifunda imiyalo yolimi lwemvelo nomsindo wereferensi, bese ikhiqiza izimo zencazelo; i-audio VAE ifaka i-waveform engu-24kHz endaweni ye-acoustic latent engu-50Hz; i-Mixed Transformer eyinhloko yakhiwe izingqimba ezingu-10 ze-dual-stream MMDiT kanye nezingqimba ezingu-20 ze-single-stream DiT, inobubanzi bemodeli obungu-1536 namapharamitha ayizigidi eziyizinkulungwane ezingu-1.5.
Ukuqeqeshwa kokuqondanisa ngemiyalo ngezinga elikhulu: Idatha yokuqeqesha iqukethe cishe amapheya ayizigidi eziyizinkulungwane ezingu-3.03 “umyalo—umsindo”, anokuqondisa okusebenzayo okungamahora ayizigidi ezingu-1.95. Okubalulekile ukuthi yonke imisebenzi ihlanganiswe ibe yifomethi efanayo ethi “umyalo + umsindo wokufaka → umsindo wokukhipha”, okwenza imodeli ijwayele imisebenzi engu-16 ngesixhumi esisodwa.
Ukuqeqeshwa kwangemva nokuLungiselelwa ngokuthandwa ngabantu: Ngemva kokuqeqeshwa okuyisisekelo, imodeli idlula ekuthuthukisweni ngokuthandwa ngabantu nasekuqeqeshweni kokuqinisa (RL post-training), okuthuthukisa ubunjalo benkulumo ekhiqiziwe, ukulandela imiyalo nekhwalithi yokulalela.

Qinisekisa indawo yehardware: Lungiselela i-NVIDIA GPU enenkumbulo yevidiyo ecishe ibe ngu-24GB, njenge-RTX 3090/4090.
Clone ikhosombe lekhodi: Clone ikhosombe le-Tencent-Hunyuan/AuK ku-GitHub ulifake endaweni yangakini.
Faka indawo yokuncika: Lungisa indawo ye-Python ngokwezidingo zekhosombe bese ufaka amaphakheji adingekayo.
Landa izisindo zemodeli: Landa izisindo ze-AuK namafayela e-VAE ahambisanayo ku-Hugging Face noma ku-ModelScope.
Landa imodeli yezimo zencazelo: Landa futhi ulungise ngokwehlukana imodeli enkulu ye-multimodal i-Qwen2.5-Omni-3B ukuze ihumushe imiyalo nomsindo wereferensi.
Lungisa izinto zokufaka: Lungisa umbhalo ohlosiwe, umsindo wereferensi noma ifayela lomsindo lokuqala elizohlelwa.
Bhala umyalo wolimi lwemvelo: Bhala umyalo ngokwezidingo zomsebenzi, njengokuchaza izici zezwi, ukucacisa umusho ozoshintshwa noma ukulungisa iphimbo/isivinini.
Qalisa i-inference: Faka imiyalo nezinto nge-command line (CLI), wenze ukukhiqiza noma ukuhlela bese ulinda umphumela.
Lalela futhi ulungise: Hlola umsindo ophumayo; uma unganelisi, lungisa amagama omyalo, inani lezinyathelo zokusampula noma amandla e-CFG bese ukhiqiza futhi.

Isixhumi esisodwa, ukumbozwa okuphelele kwemisebenzi: Ngemiyalo yolimi lwemvelo, imodeli ingenza imisebenzi engu-16 ye-TTS, ukuhlela, ukuthuthukisa nokuhlukanisa, ngaphandle kokudinga ukufaka imodeli ehlukile yomsebenzi ngamunye.
Ikhono lokuhlela elingavamile embonini: Isekela ukushintsha amagama, ukufaka imisho, ukulungisa iphimbo nokushintsha imizwa, kuyilapho igcina umbala wezwi nephimbo lokuqala ngangokunokwenzeka.
I-Instruct TTS ayidingi umsindo wereferensi: Incazelo ebhaliwe yezici zezwi ingakhiqiza inkulumo, yehlise umgoqo wokusebenzisa.
Ukwakhiwa kwe-flow-matching kugcina ukwethembeka: Isebenzisa ukukhiqiza kwe-diffusion esikhundleni sokubikezela i-token ngayinye; lapho ihlela, isusa umsindo iqala ngomsindo wokuqala, ngakho izingxenye ezingaguqulwanga zicishe zingalahlekelwa lutho.

Iwebhusayithi yephrojekthi: https://auk-project.github.io/
Ikhosombe le-GitHub: https://github.com/Tencent-Hunyuan/AuK
Ikhosombe lemodeli ku-HuggingFace: https://huggingface.co/tencent/AuK
Iphepha lobuchwepheshe ku-arXiv: https://arxiv.org/pdf/2609.08936

Ukulungisa izingxoxo zamafilimu nemidlalo emifushane: Uma umlingisi ephimisele kabi igama noma umbhalo ushintshwa kancane, shintsha ngokuqondile amagama ambalwa ekurekhodweni ngaphandle kokuphinda kuqoshwe, ugcine umuzwa wokulingisa wokuqala.
Ukukhiqizwa kwamaphodikasti nezingxoxo: Susa amaphutha okukhuluma namazwi angenasidingo, lungisa isivinini nesigqi, unciphise umsindo ongemuva nokunanela, uguqule izinto ezisikiwe zibe umkhiqizo wokugcina ngokushesha.
Izincwadi ezilalelwayo nokukhiqizwa kwezwi le-AI: Sebenzisa i-Instruct TTS ukukhiqiza imibala yezwi ehlukene ngokuchazwa kwabalingiswa; umxoxi ofanayo angalungiswa kamuva ngokwemizwa nangephimbo, kwehliswe izindleko zokukhiqiza abalingiswa abaningi.
Ukudala umculo nokwenza kabusha: Bhala kabusha amagama engoma eculwa kabusha ugcine umculo nezici zokucula zokuqala, noma ukhiphe izwi engomeni ukuze usetshenziswe ku-remix.
Ukuqeqeshwa kolimi nezinhlelo zokufinyeleleka: Susa isisho sendawo, lungisa isivinini nevolumu kubasebenzisi abanesisho noma abanenkinga yokukhuluma, ukhiqize okuqukethwe kwenkulumo okucacile nokuqondakala kalula.