SeedRealtime – Imodeli yomdabu ye-full-duplex yomsindo nevidiyo eyethulwe yi-ByteDance Uyasebenza

I-SeedRealtime iyimodeli yomdabu ye-full-duplex yomsindo nevidiyo eyethulwe ithimba le-ByteDance Seed. Ihlanganisa umsindo, ividiyo nombhalo ngokwemvelo ngaphansi kwesakhiwo esihlanganisiwe, ukuze ikwazi ukuxhumana kwesikhathi sangempela kwe-multimodal ngokubuka, ukulalela nokukhuluma ngesikhathi esisodwa. Imodeli inezinguquko ezintathu ezisemqoka: ukuqonda okuhlangene komsindo nevidiyo, ukuxhumana ngokuqhubekayo kanye nokuhamba kwengxoxo okushelelayo. Ukuhlolwa kokusebenza kusukela ekuqaleni kuze kube sekupheleni kubonisa ukuthi izinkinga zokuhamba kwengxoxo zehliswe ngesigamu uma kuqhathaniswa namamodeli axhunywe ngezigaba. Manje isitholakala kubo bonke abasebenzisi be-Doubao App, yaba umkhiqizo wokuqala we-AI womsindo nevidiyo we-full-duplex osetshenziswe ngezinga elikhulu embonini.

Ukuqonda okuhlangene komsindo nevidiyo: Ihlanganisa ngokujulile umsindo, isithombe kanye nolwazi lokulandelana kwesikhathi ngokwemvelo. Ingasebenzisa isimo esibonakalayo ukuqeda ukungaqondakali kwamagama aphinyiselwa ngokufanayo, iqonde ngokunembile izinkomba ezihlobene nesikhathi, futhi ihlanganise lokho okubukwayo, okuzwakalayo nokukhulunywayo kube ukuqonda okukodwa.
Ikhono lokuxhumana ngokuqhubekayo: Iyakwazi ukubona njalo imvelo nokuziveza ngokwayo. Ingakhumbuza umsebenzisi lapho isimo esikrinini sishintsha, futhi ibize amathuluzi ukuze iwahlanganise ekuchazeni kwayo, iguqule impendulo engenzi lutho ibe ukubambisana okusebenzayo.
Ukuhamba kwengxoxo okushelelayo: Ibona ngesikhathi sangempela isimo sengxoxo yomsebenzisi kanye nesigqi sokuxhumana, ingene engxoxweni, ime kancane futhi iphendule ngokwemvelo ngesikhathi esifanele. Inamandla okumelana nokuphazamiseka, futhi ingahlukanisa izingxoxo zabanye abantu nomsindo wangemuva.
Ukubona izigcawu ezinabantu abaningi: Ezindaweni ezinabantu abaningi nezinoshintsho, ibona ubuso, ihlukanise imithombo yemisindo futhi iqonde okuqukethwe ngesikhathi esisodwa, iqhubeke ihlobanisa izwi lomuntu ngamunye nobunikazi bakhe.

Ukumodela okuhlanganisiwe kusukela ekuqaleni kuze kube sekugcineni: Isebenzisa uhlaka oluhlanganisiwe lokumodela umsindo nevidiyo kusukela ekuqaleni kuze kube sekugcineni, luhlanganise ukubona, ukuqonda, ukwenza izinqumo nokuziveza kumodeli eyodwa esebenza ngesikhathi esisodwa. Lokhu kugwema ukulahleka kolwazi nokunqwabelana kokubambezeleka kwezigaba eziningi zesistimu exhunyiwe ye-ASR→VLM→TTS.
Ukuxhumana komdabu kwe-full-duplex: Ayincikile emithethweni yangaphandle ye-VAD ukuze inqume ukushintshana kwezikhathi zokukhuluma. Imodeli ngokwayo yenza izinqumo ngokuqhubekayo mayelana nesimo sengxoxo nesikhathi esifanele, isebenzisa ukugeleza kolwazi lwe-multimodal, futhi ifinyelele ekuxhumaneni kwangempela “ukukhuluma ulalele” esikhundleni semodi ye-half-duplex yokubuza nokuphendula.
Ukuvumelanisa kwesikhathi somsindo nevidiyo: Ihlanganisa umsindo, isithombe nolwazi lokulandelana kwesikhathi kumodeli eyodwa. Isebenzisa isimo samanje, ukushukuma komzimba, ukubuka kwamehlo nezenzo zangaphambilini ukuze iqonde inhloso yomsebenzisi, iqede ukungaqondakali kwezinhlobo ezahlukene zemidiya futhi ihlaziye izinkomba.
Ukunciphisa ukubambezeleka kwezobunjiniyela: Ngokufaka umsindo nevidiyo ngezingxenye kanye nokukhiqiza okusakazwayo, ihlanganisa nokulinganisa okusebenzayo nokuthuthukisa ukucabanga ukuze iqhubeke inciphisa ukubambezeleka kusukela “ekuzweni kuya ekuqondeni kuya ekuphenduleni”.

Buyekeza i-Doubao App: Buyekeza i-Doubao App ibe yinguqulo yakamuva.
Ngena ocingweni lwezwi: Vula uhlelo, bese uchofoza inkinobho ethi “Shaya ucingo” kunoma yiliphi iwindi lengxoxo.
Shintshela kumodi yevidiyo: Ngemva kokungena kusixhumi esibonakalayo socingo, vumela imvume yekhamera nemakrofoni, bese ushintshela ocingweni lwevidiyo.
Qala ukuxhumana kwesikhathi sangempela: Bonisa isithombe bese ukhuluma ngokwemvelo; imodeli izobona ukugeleza komsindo nevidiyo ngesikhathi esisodwa futhi iphendule ngesikhathi sangempela.
Sebenzisa ukubuka okuqhubekayo: Unganikeza umyalo wokubuka ngokuqhubekayo; imodeli izokukhumbuza lapho into eqondisiwe ivela noma isimo sesithombe sishintsha.

Isakhiwo esihlanganisiwe kusukela ekuqaleni kuze kube sekugcineni: Isebenzisa imodeli eyodwa ehlanganisa umsindo, ividiyo nombhalo ngokwemvelo, iqede ukulahleka kolwazi nokunqwabelana kokubambezeleka okubangelwa amamojula amaningi axhunywe ohlelweni olunezigaba.
Ukuxhumana komdabu kwe-full-duplex: Ayincikile emithethweni yangaphandle ye-VAD; imodeli izinqumela ngokuqhubekayo isikhathi sengxoxo isebenzisa ukugeleza kolwazi lwe-multimodal, futhi ifinyelele “ekukhulumeni ulalele” kwangempela esikhundleni sokubuza nokuphendula.
Ukuqonda okuhlangene komsindo nevidiyo: Ihlanganisa ngokujulile umsindo, isithombe kanye nolwazi lokulandelana kwesikhathi, futhi ingasebenzisa isimo esibonakalayo ukuqeda ukungaqondakali kwamagama aphinyiselwa ngokufanayo nokuchaza ngokunembile izinkomba ze-multimodal ezifana nokuthi “lokhu” kanye “laphaya”.
Ukubona imvelo ngokuqhubekayo: Iyakwazi ukubuka isithombe ngokuqhubekayo, futhi ikhumbuze umsebenzisi lapho isimo esikrinini sishintsha noma into eqondisiwe ivela, iguqule ukuxhumana kusukela empendulweni engenzi lutho kube ukubambisana okusebenzayo.
Isigqi esibushelelezi esimelana nokuphazamiseka: Ibona ngesikhathi sangempela isimo sengxoxo yomsebenzisi, ihlukanise ngokunembile izingxoxo nabanye nemibuzo esemthethweni ezindaweni ezinomsindo, futhi inciphise ukuphazamiseka nokusebenza okungalindelekile ngesigamu uma kuqhathaniswa namamodeli axhunywe ngezigaba.

Iwebhusayithi esemthethweni: https://seed.bytedance.com/en/seedrealtime

Umqondisi ohlakaniphile: Emnyuziyamu, ibuka ngokuqhubekayo izithombe zemibukiso, bese ichaza umlando nezici zobuciko lapho into eqondisiwe ivela.
Ukuzilolonga kolimi lwangaphandle: Isebenzisa isithombe ukulungisa ukuphimisela ngesikhathi sangempela futhi inikeze izibonelo zokwakha imisho, ihlale igxile kumfundi oqondisiwe nasezindaweni ezinomsindo.
Ukuqondisa ekusebenziseni imishini: Lapho kusetshenziswa imishini eyinkimbinkimbi, ibona izinguquko zesimo ezibonakalayo ngesikhathi sangempela, ilungise amaphutha futhi inikeze iziphakamiso zokulungisa.
Umsizi wolwazi lokuhamba: Esikhumulweni sezindiza esinomsindo, ibona ulwazi lwezindiza esikrinini esikhulu, iphendule ngesikhathi sokufika futhi inikeze izinkomba zomzila.
Usizo lwemfundo yezingane: Ngesikhathi ifunda nengane, ibuka okuqukethwe esikrinini ngesikhathi sangempela, ilungise ukuphimisela futhi ingaphazanyiswa amazwi angemuva.