MAGI-2-preview – Imodeli yokukhiqiza amavidiyo e-multimodal evulekile ye-Sand.ai Uyasebenza

I-MAGI-2-preview iyimodeli yokuqala emhlabeni yokukhiqiza amavidiyo e-multimodal ye-MoE enamapharamitha ayizigidigidi eziyikhulu, evulwe yi-Sand.ai, enamapharamitha aphelele angu-114B kodwa evula angu-6B kuphela. Imodeli iqhubeka nokusebenzisa isakhiwo se-single-stream, ihlanganisa umbhalo, ividiyo nomsindo ku-Transformer eyodwa ukuze ikwenze ukumodela okuhlangene, ngaleyo ndlela ikhiqize i-multimodal ngokwemvelo. Ihlele endaweni yesithupha ohlwini lwe-AA lokukhiqiza amavidiyo. Ikhodi yemodeli nezisindo zokuqeqeshwa kwangaphambilini sekukhishiwe, okuqinisekisa indlela entsha nephumelelayo ye-Scaling emkhakheni wokukhiqiza amavidiyo.

Ukukhiqiza amavidiyo e-multimodal: Isekela ukukhiqiza kusuka embhalweni, esithombeni, nasevidiyweni kuya kwenye ividiyo, futhi ihlanganisa ngokwemvelo ukuqonda umsindo.
MoE sparse activation: Kumapharamitha aphelele angu-114B, kuvulwa angu-6B kuphela, okwehlisa kakhulu izindleko ze-inference.
Isakhiwo esihlanganisiwe se-single-stream: Ishiya indlela evamile yokuhlanganisa nge-cross-attention, futhi kusukela kungqimba lokuqala imodela isithombe nomsindo ngokuhlanganyela.
I-Scaling esebenza kahle: Yakha kabusha ukuxhumana kwe-MoE nokuzinza kokuqeqeshwa kuzo izimo zokukhiqiza amavidiyo, ixazulule ibhodlela lokuxhumana phakathi kwamakhadi lapho kusetshenziswa ama-sequence amade kakhulu.

Landela i-WeChat bese uphendula ngokuthi “开源” ukuze ujoyine iqembu lokuxhumana lamaphrojekthi e-AI avulekile.
Isakhiwo se-MoE Mixed Experts: Umthamo ophelele wemodeli ufinyelela ku-114B, kodwa i-forward propagation eyodwa ivula amapharamitha angu-6B kuphela, ihlukanise umthamo nokubalwa.
I-Single-stream Transformer: Ama-token ombhalo, evidiyo nawomsindo angena ku-Transformer eyodwa kusukela kungqimba lokufaka. Nge-self-attention, ashintshana ngokuqondile ngolwazi lwezindlela ezahlukene, okwenza kube lula ukubamba imininingwane yokuvumelana kwesithombe nomsindo uma kuqhathaniswa nesakhiwo sokuhlanganisa imibhoshongo eminingi.
Ukuthuthukiswa kokuxhumana okusabalalisiwe: Ngokucabangela ama-sequence amade kakhulu evidiyo, kwakhiwe kabusha indlela yokuhambisa ama-token kanye neqhinga lokuxhumana ku-expert parallelism, kwehliswe izindleko zokudluliswa kwedatha phakathi kwama-GPU.
Isixazululo sokuzinza kokuqeqeshwa: Izindlela ezintathu zokuqinisekisa ukuzinza, okuhlanganisa dynamic routing, ukulinganisa umthwalo wochwepheshe, nokuqeqeshwa okuhlangene kwedatha ye-multimodal.

Ukulungiselela indawo: Kloni i-GitHub repository SandAI-org/MAGI-2-preview, bese ulungisa iz依依 ngokuya nge-README.
Ukulanda izisindo: Thola izisindo zokuqeqeshwa kwangaphambilini ku-GitHub Releases.
Ukuqalisa i-inference: Shintsha ifayela lokumisa namapharamitha eskripthi. Isekela izindlela ezintathu: t2v / i2v / v2v. Sebenzisa ifomethi ye-run.sh ye-MAGI-1 njengereferensi.
Izidingo zehadiwe: Ukucushwa okuqondile kusalinde ukubuyekezwa kwemibhalo esemthethweni. Kunconywa ukubhekisa kusixazululo se-MAGI-1 se-24B esinama-H100 amaningi noma se-4.5B esinekhadi elilodwa eline-24GB VRAM.

Isemthombo ovulekile futhi ingasetshenziswa kwezentengiso: Imodeli isebenzisa ilayisense le-Apache 2.0, kanti izisindo zokuqeqeshwa kwangaphambilini nekhodi ye-inference kuvuleke ngokuphelele.
I-MoE yokuqala yezigidigidi eziyikhulu: Imodeli yokuqala emhlabeni yokukhiqiza amavidiyo ye-MoE esezingeni lamapharamitha ayizigidigidi eziyikhulu evulwe ngempumelelo, eqinisekisa indlela entsha ye-Scaling emkhakheni wamavidiyo.
Umthamo omkhulu ngezindleko eziphansi: Amapharamitha angu-6B avuliwe angashayelisa imodeli engu-114B, okwenza izindleko ze-inference zibe ngaphansi kakhulu kunemodeli eminyene enosayizi ofanayo.
I-multimodal yemvelo: Isakhiwo se-single-stream sihlanganisa ngokujulile umsindo nevidiyo ezingeni eliyisisekelo lem model, sigweme ukubambezeleka nokulahleka kwekhwalithi okubangelwa ukuvumelanisa kwesikhathi sakamuva.

Iwebhusayithi yephrojekthi: https://sand.ai/blog/magi-2-preview
I-GitHub repository: https://github.com/SandAI-org/MAGI-2-preview
I-HuggingFace model hub: https://huggingface.co/sand-ai/MAGI-2-preview

Izikhangiso zamavidiyo amade namadrama amafushane: Umthamo omkhulu we-MoE usekela ukumodelwa kwezindaba eziyinkimbinkimbi futhi ungakhiqiza amavidiyo ezindaba ahambisanayo athatha imizuzu.
Ukuhlolwa kwangaphambili kwamafilimu nemikhiqizo ye-multimodal: Ikhono lokuqonda umsindo ngokwemvelo lenza ukuvumelaniswa nokuhlelwa kwezwi, imisindo ekhethekile nesithombe kube khona.
Ucwaningo lwezemfundo lwamavidiyo e-AI: Izisindo ze-MoE zamavidiyo ezingeni lamapharamitha ayizigidigidi eziyikhulu, ezivulekile, zinikeza umphakathi wezemfundo isisekelo se-Scaling esingaphindwa.
Ukusetshenziswa kwangasese ezinkampanini: Imodeli esekelwe kulayisense le-Apache 2.0 isekela izimboni ezibucayi, njengezezimali nezempilo, ekwakheni amandla okukhiqiza amavidiyo ngaphakathi kwazo.
Ukukhiqizwa kwemidiya esakazwayo ngesikhathi sangempela: I-sparse activation yehlisa izindleko ze-inference, futhi ingasekela ukukhiqizwa kwamavidiyo ngesikhathi sangempela okunokubambezeleka okuphansi kanye nezimo zokusakaza bukhoma.