YuE2 Uyasebenza

I-YuE2 iyimodeli yokukhiqiza umculo ekhishwe njenge-open source yi-Hong Kong University of Science and Technology kanye nethimba le-M-A-P. Imodeli ingakhiqiza ishidi lomculo le-ABC elihlelekayo (umculo oyinhloko namachords) ngokusekelwe kumazwi engoma nesitayela, bese iliguqula libe yingoma ephelele engu-48kHz, okwenza inqubo yokudala icace futhi ilawuleke. Imodeli isekela ukudala kusukela ekuqaleni, ukucula kabusha ngaphandle kwamasampula okuqeqesha, kanye nokuhlela ishidi lomculo ngengxoxo ne-Agent. Ku-WildSongBench, i-SongBench Avg ifinyelela ku-6.9632, idlule amamodeli ezentengiso afana ne-Suno v5.

Ukudala okuhleliwe ngokwezimpawu: faka amazwi engoma nesaziso sesitayela, imodeli iqale ikhiqize ishidi le-ABC elihlelekayo elinomculo oyinhloko namachords, bese iliguqula libe yingoma ephelele enezwi nomculo ohambisanayo.
Ukucula kabusha ngaphandle kwamasampula okuqeqesha: sebenzisa i-SheetSage2 ukuguqula ukuqoshwa kwengoma yoqobo kube ishidi lomculo, bese uyihumusha kabusha ngesitayela noma ngamazwi amasha ngaphandle kokulungisa imodeli ngokukhethekile.
Ukuhlela ishidi lomculo ngengxoxo ne-Agent: xoxa ne-Agent mayelana neshidi lomculo, uguqule ama-harmony, umculo oyinhloko, ijubane noma isakhiwo sengoma, bese uyikhiqiza kabusha inguqulo entsha.
Ukuguqula umsindo ube ishidi lomculo (SheetSage2): guqula noma yikuphi ukuqoshwa kube ishidi lomculo elingahlolwa nelihlelekayo, okuyisango lemisebenzi yokucula kabusha nokuhlela.
Ukuqonda umculo (MERT2): inikeza izethulo zomculo ezingeni lengoma yonke, isekele imisebenzi elandelayo efana nokuguqula umsindo ube ishidi lomculo, futhi ithole i-SOTA emisebenzini engu-14 kwengu-15 ku-MARBLE benchmark.

Landela i-WeChat bese uphendula ngokuthi “开源” ukuze ujoyine iqembu lokuxoxa ngamaphrojekthi e-AI open source
Isendlalelo esiphakathi sokuhlela ngezimpawu: Umklamo oyinhloko we-YuE2 ufaka ishidi lomculo elisobala elinezimpawu (notation ye-ABC, equkethe umculo oyinhloko namachords) phakathi kwezimo zombhalo/umsindo kanye ne-waveform yokugcina. Ishidi lomculo lisebenza njengesixhumi esibonakalayo “sebhokisi elivulekile”; umuntu noma i-Agent angalifunda futhi alihlele, kanti lona linquma isakhiwo sokukhiqizwa komsindo okulandelayo, liguqule ukudala kusukela ekukhetheni okungaqondakali kube inqubo yokuqamba elawulekayo.
Isakhiwo esixubile sochwepheshe be-AR–NAR: Imodeli isebenzisa i-backbone eyodwa ye-Mixture-of-Transformers. Uchwepheshe we-AR usebenzisa ukunaka okuyimbangela ukuze abikezele ngokulandelana ama-token eshidi lomculo nama-token encazelo, aqinisekise ukuhleleka komculo nokuhambisana kwamazwi; uchwepheshe we-NAR usebenzisa ukunaka okubheka izinhlangothi zombili ukubikezela i-acoustic latent nge-flow matching (25 Hz × 64), aqinisekise ikhwalithi yomsindo. Lezi zinhlobo ezimbili zochwepheshe zabelana ngezendlalelo zokuzinaka ezixubile kanye ne-FFN, ziphathe ukuhlela nokuguqula zibe umsindo ndawonye ngaphakathi kwe-checkpoint eyodwa.
Uhlelo lokukhiqiza umsindo lwe-VAE: I-acoustic latent ekhishwa uchwepheshe we-NAR ibuyiselwa ibe umsindo we-stereo ongu-48 kHz yi-VAE decoder, ngaphandle kokulahleka okubangelwa quantization kuyo yonke inqubo. Ngesikhathi sokuqeqesha, amamojula angaxhunyiwe akha okuqondiwe: i-SheetSage2 ikhipha okuqondiwe kweshidi lomculo emsindweni, i-MERT2+CVQ ikhiqiza okuqondiwe kwama-token encazelo, kanti i-VAE encoder ikhiqiza okuqondiwe kwe-latent.
Izindlela ezintathu zisebenzisa izisindo ezifanayo: ukudala, ukucula kabusha nokuhlela kusebenzisa i-checkpoint efanayo yokukhiqiza. Umehluko kuphela umthombo weshidi lomculo: lizikhiqizwa yimodeli, liguqulwe yi-SheetSage2, noma lihlelwe umuntu/i-Agent; ekuculeni kabusha kusetshenziswa imodi “yomculo oyinhloko kuphela” ukuze umculo ohambisanayo uzivumelanise ngokukhululekile nesitayela esisha.
I-API yokucubungula ngezigaba: Ukucabanga kuhlukaniswe ngezigaba ezine: plan() → generate_semantic() → synthesize() → decode(). Ishidi lomculo, ama-token encazelo, i-acoustic latent kanye nemingcele yokukhiqiza kugcinwa kudiski. Umsebenzisi angangenelela kunoma yisiphi isigaba: akhiphe ishidi lomculo alihlele bese elikhiqiza kabusha, noma asebenzise kabusha i-plan esivele yenziwe ashintshe i-decoder kuphela, kuhlanganiswe ukuguquguquka nokuphindaphindeka kwemiphumela.

Ukulungiselela imvelo: kudingeka uhlelo lwe-Linux, i-Python 3.12 kanye ne-NVIDIA GPU esekela i-BF16. Izisindo zemodeli zizodawunilodwa ngokuzenzakalela ku-Hugging Face lapho ziqala ukusetshenziswa.
Ukufaka nokusebenzisa: ngemva kokwenza i-git clone yenqolobane, dala imvelo ebonakalayo bese usebenzisa pip install . ukuze uqede ukufaka.
Ukukhiqiza ingoma yokuqala ngokushesha: sebenzisa python examples/generate.py --output outputs/first-song ukuze ukhiqize ingoma ngesibonelo esakhelwe ngaphakathi. Ifolda yokuphuma izoqukatha umsindo (flac), ishidi lomculo nazo zonke izinto eziphakathi.
Ukusebenzisa i-Python API: layisha uhlelo nge-YuE2Pipeline.from_pretrained("m-a-p/YuE2-3B"), bese unikeza isitayela namazwi engoma ukuze ukhiqize; ipharamitha ye-cot ilawula imodi: "full" ikhiqiza ishidi lomculo elihlelekayo (okuzenzakalelayo), "melody" ihlela umculo oyinhloko kuphela, kanti "off" ikhiqiza ngokuqondile.
Ukucula kabusha ingoma: qala usebenzise i-SheetSage2 ukuguqula ukuqoshwa kwengoma yoqobo kube ishidi le-ABC lomculo oyinhloko, bese unikeza amazwi amasha noma isitayela esihlosiwe nge-cot="melody" ukuze ukhiqize kabusha.
Ukuhlela umsebenzi okhona: shayela i-pipe.plan() ukuze ukhiphe ishidi lomculo, umuntu noma i-Agent ahlele ifayela le-ABC, bese usebenzisa --abc-file edited.abc --cot full ukuze ukhiqize inguqulo entsha.
Ukusebenzisa i-Agent skill: ngenisa i-skills/yue2-music/ esenqolobaneni ku-Agent esekela i-SKILL.md, bese usebenzisa ingxoxo yemvelo ukubhala izingoma, ukuguqula umsindo ube ishidi, ukuhlela ishidi nokuqhathanisa izinguqulo.

Ikhwalithi eqhudelana namamodeli ezentengiso: Ku-WildSongBench, i-SongBench Avg ifinyelela ku-6.9632 (best-of-8), idlule imikhiqizo yezentengiso efana ne-Suno v5 ne-Mureka 9, iphakamisa izinga lokukhiqizwa komculo nge-open source.
Ukudala okulawulekayo okusekelwe ebhokisini elivulekile: Umqondo wokuqala we-“symbolic planning” uveza umculo oyinhloko namachords njengeshidi le-ABC, ukuze umuntu ne-Agent bakwazi ukufunda nokuhlela umsebenzi ngaphambi kokuwuguqula ube umsindo.
Izindlela ezintathu ezisebenzisa izisindo ezifanayo: ukudala, ukucula kabusha ngaphandle kwamasampula okuqeqesha nokuhlela ngengxoxo kusebenzisa i-checkpoint efanayo. Ukucula kabusha akudingi ukulungiswa okukhethekile futhi kufinyelela ku-0.647 CLEWS mAP ekugcineni komthombo wengoma.
I-open source ephelele yasendaweni: Ikhodi ilandela i-Apache 2.0, izisindo ezingu-3.6B ziyatholakala ukuze zilandwe, futhi ingasebenza ungaxhunyiwe ku-inthanethi ngekhadi elilodwa elino-24GB memory; idatha ayishiyi ikhompyutha yendawo.
Uhlelo olugcwele lwe-ecosystem: I-SheetSage2 yokuguqula umsindo ube ishidi, imodeli yokuqonda umculo i-MERT2, iqoqo lokuhlola i-WildSongBench kanye ne-yue2-music Agent skill nakho kukhishwa njenge-open source, kuhlanganiswe lonke uchungechunge “lokuguqula ishidi → ukudala → ukuhlela → ukuhlola”.
Ukungenelela ngezigaba: I-API enezigaba ezine plan() → generate_semantic() → synthesize() → decode() igcina ishidi lomculo, ama-token encazelo ne-latent kudiski, futhi isekele ukusebenzisa kabusha izinhlelo noma ukushintsha i-decoder phakathi nenqubo.

Iwebhusayithi yephrojekthi：https://map-yue2.github.io/
Inqolobane ye-GitHub：https://github.com/multimodal-art-projection/YuE

Ukudala umculo komuntu siqu: Abantu abangakwazi ukuqamba noma ukuhlela umculo bangafaka amazwi engoma nesitayela ukuze bakhiqize ingoma, bese belungisa ishidi lomculo ukuze balawule ngokuningiliziwe indlela umculo oyinhloko ohamba ngayo, baguqule “ugqozi” lube umsebenzi ophelele.
Ukucula kabusha nokudala kabusha: Sebenzisa i-SheetSage2 ukuguqula ingoma yoqobo ibe ishidi lomculo, bese uyiguqula ngokushesha ibe yisitayela esisha esifana ne-jazz noma i-rock, noma ushintshe amazwi ukuze udale inguqulo yezilimi eziningi, ehambisana nezidingo zokudala kwabalandeli namavidiyo amafushane.
Idemo yomculo wamafilimu, imidlalo noma izikhangiso: Abakhiqizi bangakhiqiza ngokushesha umculo oyisibonelo onezwi noma ohambisanayo kuphela, bahlole ngezindleko eziphansi ngaphambi kokunquma ukuqasha umkhiqizi wangempela, banciphise kakhulu isikhathi sokuxhumana sokuqala.
Imfundo yomculo nokuhlaziywa kwe-harmony: Othisha bangayisebenzisa ukukhiqiza izingoma eziyizibonelo ezinama-chord progression nesakhiwo sengoma esithile, kanti abafundi bangafunda amasu okuqamba beqhathanisa neshidi le-ABC, benze umjikelezo wokufunda “ozwakalayo, obonakalayo futhi ohlelekayo”.