Nemotron 3.5 Transcribe Uyasebenza

I-Nemotron 3.5 Transcribe iyimodeli enkulu evulekile yenkulumo eyethulwe yi-NVIDIA, esebenzisa ukwakheka kwe-encoder yenkulumo ne-MoE, enamapharamitha asebenzayo angaba yizigidi eziyizinkulungwane ezingu-4.3, isekela izilimi ezingu-40, futhi ingacubungula okurekhodiwe okufika ehoreni elingu-1 okunabantu abafika kwabangu-8 abakhulumayo. Isekela ukuloba kwezwi nezwi, ukuhlukanisa izikhulumi, ukuhumusha nezinye izici. I-Nemotron 3.5 Transcribe ivumela ukusebenzisa imiyalelo yolimi lwemvelo ukwenza ngendlela oyifisayo ifomethi yokuphumayo, futhi ingashintshwa ukuze ivumelane nezizinda ezikhethekile ngokuyiqeqesha kabusha. Ukusetshenziswa kwezohwebo kumahhala.

Ukuloba ngezilimi eziningi: Isekela izilimi ezingu-40, ingaguqula okurekhodiwe kube umbhalo, futhi ibone ulimi olusetshenziswa enkulumweni.
Ukuhlukanisa izikhulumi: Iyakwazi ukuhlukanisa izikhulumi ezingafika kweziyisi-8, imake ukuthi umusho ngamunye ungowasiphi isikhulumi, futhi ilungele imihlangano nezingxoxo zabantu abaningi.
Ukucubungula umsindo omude: Ingacubungula okurekhodiwe okufika cishe ehoreni elingu-1 ngesikhathi esisodwa.
Ukuloba okuqondiswa imiyalelo: Sebenzisa ulimi lwemvelo ukucacisa ifomethi yokuphumayo, njengombhalo wezwi nezwi, ukuhlela ube ulimi olubhaliwe, ukususa amagama agcwalisa inkulumo nokulungisa amagama obuchwepheshe, ngaphandle kokuqeqesha kabusha.
Ukuhumusha inkulumo: Iyakwazi ukuhumusha phakathi kwezilimi ezingu-31 nesiNgisi.
Izitembu zesikhathi ezingeni lamagama: Ibuyisa indawo yesikhathi yomsindo ehambisana negama ngalinye, okwenza kube lula ukulalela futhi useshe.
Ukushunwa kwesizinda: Ngokusekelwe ku-NeMo Speech toolkit, ingashintshwa ngokushesha ukuze isetshenziswe ezimweni zobungcweti ngedatha encane ku-GPU eyodwa.

Landela i-WeChat bese uphendula ngokuthi “开源” ukuze ujoyine iqembu lezingxoxo zamaphrojekthi e-AI avulekile.
Imodeli ye-Speech-Augmented Language Model (SALM): Imodeli inezingxenye ezintathu. I-Transformer speech encoder ikhipha izici zomsindo; imodeli efriziwe yokuhlukanisa izikhulumi i-Nemotron 3 inikeza ulwazi lomsebenzi wezikhulumi; zombili zixhunywa kumodeli yolimi enkulu i-Nemotron 3.5 Lightning 30B-A3B, eyinhlanganisela ye-Mamba-Transformer MoE. Amapharamitha esewonke angaba yizigidi eziyizinkulungwane ezingu-34, kodwa cishe izigidi eziyizinkulungwane ezingu-4.3 kuphela ezisebenzayo ngesikhathi sokucabanga, okulinganisa amandla nokusebenza kahle.
I-encoder yochwepheshe ehambisanayo: Isignali yomsebenzi wezikhulumi ephuma kumodeli efriziwe yokuhlukanisa ifakwa ezethulweni ze-ASR encoder, okwenza imodeli ikwazi ukuloba okuphawulwe izikhulumi ngaphansi kwesakhiwo esisodwa se-LLM ngaphandle kokushintshwa, futhi iqeqeshwe ngokukhipha okulandelanayo.
Ukuhlanganiswa okugadiwe okulinganiselayo: Idatha yokuqeqesha isampulwa ngokwenani lama-token empendulo, hhayi ngenani lamasampula. Isampula ngayinye ye-ASR ingasetshenziswa nezinhlobo eziningi zemiyalelo yomsebenzi, ngakho amakhono amasha angangezwa ngesilinganiso sesampula esingaphansi kuka-1%. Ukuqeqesha kuhlukaniswe kwaba ukushunwa okugadiwe kanye nesigaba esifushane sokufunda ngokuqinisa, esithuthukisa ngokuqondile ukunemba kokuloba.
Ukukhipha okuqagelayo kwe-MTP: Imodeli ithuthukiswa ngesikhathi sokuqeqesha ukuze ibikezele ama-token amaningi (MTP), ngakho izendlalelo zayo ze-MTP zingasebenza ngokuqondile njengemodeli esalungiselela i-vLLM speculative decoding. Kuphinde kunikezwe imodeli ehlukile ye-DFlash2 njengenketho. Ngokuhlanganyela, lezi zingafinyelela ekucubunguleni imizuzwana engu-2238 yomsindo ngomzuzwana ku-RTX PRO 6000 Blackwell GPU eyodwa.
I-adapter yezitembu zesikhathi ye-CTC: Ngemva kokukhipha, i-adapter ye-CTC elula iqondanisa umbhalo okhiqiziwe nomsindo, ikhiqize izitembu zesikhathi ezingeni lamagama ezinelebula zezikhulumi zomsindo omude nezingxoxo ezigqagqene kakhulu, ngaphandle kokwengeza umthwalo kumodeli enkulu.

Lungisa ihardware: Inguqulo ye-BF16 idinga cishe u-80GB wememori ye-GPU; i-FP8/NVFP4 noma i-GGUF ingasetshenziswa ukwehlisa izidingo.
Thola imodeli: Sesha i-NVIDIA-Nemotron-3.5-Transcribe-30B-A4B ku-Hugging Face noma ku-Magic Model, bese ulanda izisindo ngaphansi kwelayisensi ye-OpenMDW-1.1.
Faka indawo yokusebenza: Khetha enye yezindlela ezintathu zokuyisebenzisa: i-Nemotron Transcribe NIM (container esemthethweni), i-vLLM (isekela ukusheshiswa kwe-MTP), noma i-NeMo-Speech.cpp (ukusebenza kwendawo nge-GGUF).
Lungisa umsindo: Layisha ifayela lomsindo elifika ehoreni elingu-1, bese uliguqula libe yifomethi edingwa yifreyimu oyisebenzisayo.
Bhala imiyalelo: Chaza umsebenzi ngolimi lwemvelo, njengokuthi “Gcina amagama angempela nezwi nezwi bese umaka izikhulumi” noma “Susa amagama agcwalisa inkulumo bese uhumushela esiNgisini”.
Qalisa ukucabanga: Shayela i-API uthumele umsindo nomyalezo; imodeli izobuyisa umbhalo olotshiwe, futhi uma kudingeka ifake amalebula ezikhulumi nezitembu zesikhathi ezingeni lamagama.
(Okuzikhethela) Zama ngokushesha: Uma ungafuni ukuyifaka, ungazama kuqala i-nvidia/nemotron-transcribe ku-Hugging Face online Space.
(Okuzikhethela) Ukushunwa kwesizinda: Sebenzisa i-NeMo Speech toolkit ukulungisa idatha enamalebula; nge-GPU eyodwa, ukulungisa indlela yokuphimisa, amagama obuchwepheshe namafomethi akho kuthatha cishe amahora angu-0.5–2.3.

Ukuphumayo okuqondiswa imiyalelo: Umyalelo owodwa wolimi lwemvelo ungashintsha phakathi kombhalo wezwi nezwi, umbhalo ohleliwe, ukuhumusha nezinye izinhlobo, ngaphandle kokuqeqesha kabusha noma ukulungisa amamodeli amaningi.
Ukunemba okuphezulu kokuloba: I-WER emaphakathi ohlwini lwe-English Open ASR ingu-4.71% kuphela, kanti ohlwini lwezilimi eziningi ingu-4.62%; zombili zingamamodeli anesisindo esivulekile ahamba phambili.
Ukuhlukanisa izikhulumi okuhamba phambili: Emisebenzini yokuloba okuphawulwe izikhulumi enenkulumo yabantu abaningi egqagqene, iqale ukudlula amasistimu akhethekile axhunyiwe, futhi isekele izimo ezifinyelela ehoreni elingu-1 nabantu abangu-8.
Ukulungisa amaphutha ngomongo okuqinile: Ngemva kokunikeza uhlu lwamagama obuchwepheshe, amagama abantu nezinye izinhlangano, izinga lamaphutha ezinhlangano ezibiziwe lingehla izikhathi eziyisi-6, ngakho amagama obuchwepheshe awasadingi ukuqagelwa.
Izitembu zesikhathi ezingeni lamagama ezinelebula zezikhulumi: Ukuqondanisa kwe-CTC okulula kwenza imiphumela yokuloba ikwazi ukulalelwa futhi iseshwe, kusekele ngokuqondile ukubuyekezwa kwezifundo nokusesha emihlanganweni.
Ukucabanga okusebenza kahle: Ukukhipha okuqagelayo kwe-MTP kwasekuqaleni kuhlanganiswe nemodeli esalungiselela ye-DFlash2, okwenza i-RTX PRO 6000 eyodwa icubungule imizuzwana engu-2238 yomsindo ngomzuzwana.

Ilabhulali yamamodeli ye-Hugging Face：https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B
Iphepha lobuchwepheshe：https://research.nvidia.com/labs/nemotron/files/NVIDIA-Nemotron-3.5-Transcribe-Technical-Report.pdf

Amaminithi emihlangano nokukhipha izinqumo: Ihlukanisa izikhulumi ngokuzenzakalelayo, igcine amagama ezingxoxo njengoba enjalo, futhi ikhiphe iziphetho zomhlangano nemisebenzi okufanele yenziwe ngokwemiyalelo.
Izingxoxo zabezindaba nokuhlela izindaba: Igcina amazwi oqobo alowo oxoxwayo izwi nezwi futhi imake izikhulumi, iqinisekise ukunemba kwezingcaphuno, kuyilapho ingasusa amagama agcwalisa inkulumo uma kudingeka.
Ukubuyekezwa kwezifundo nokuqeqeshwa: Ikhiqiza umbhalo onempawu zesikhathi ezingeni lamagama, ivumele abafundi baseshe amagama ayisihluthulelo bese begxumela esikhathini uthisha achaza ngaso lelo phuzu lolwazi.
Amarekhodi ezokwelapha nezindawo zobungcweti: Ihlanganisa uhlu lwamagama obuchwepheshe nokushunwa kwesizinda ukuze iguqule izingxoxo phakathi kodokotela neziguli zibe imibhalo yezokwelapha ngefomethi ejwayelekile, yehlise amaphutha amagama obuchwepheshe.
Ukwenziwa kwasendaweni kokuqukethwe ngezilimi eziningi: Ihumusha ngokuqondile okurekhodiwe ngezilimi ezingu-40 kube umbhalo wesiNgisi, ukuze kusetshenziswe emihlanganweni yamazwe ngamazwe, ezingxoxweni noma ekwenzeni imibhalo engezansi yamavidiyo.