Sophon-Realtime v1 Uyasebenza

I-Sophon-Realtime v1 iyimodeli yezwi ye-full-duplex ekhishwe njenge-open source yi-AI降临派 (Adventists AI). Iyakwazi ukulalela ngesikhathi esifanayo ibone ukuthi ubani okufanele akhulume, iphazanyiswe noma nini, iqonde iphimbo, ukuhleka nokububula, bese iphendula ngezwi elinemizwa. I-Sophon-Realtime v1 isekelwe kumapharamitha angu-32B, ithole amaphuzu angu-82.8 ekuhlolweni kwezwi i-VoiceBench, yaba kweyokuqala phakathi kwamamodeli e-full-duplex e-open source. I-Sophon-Realtime v1 inezinguqulo ezintathu: i-Tag (enokuzwakalisa okuqinile), i-Flash (enempendulo esheshayo) ne-E2E (ukubuka kuqala kocwaningo). Ingafakwa nge-GPU eyodwa engu-80GB, isebenzisa ilayisensi i-Apache-2.0 futhi ingasetshenziselwa ezentengiselwano.

Ingxoxo ye-full-duplex: Iyaqhubeka nokulalela ngesikhathi ukhuluma, bese iqala ukuphendula ngokushesha lapho ume; uma uphazamisa, iyama khona manjalo iphendule umbuzo omusha.
Ukuqonda iphimbo: Iyakwazi ukubona ukungabaza, ukugcizelela, ukuhleka, ukububula, inkulumo ephikisanayo nezinye izimpawu ezingezona ezamagama, hhayi okuqukethwe kwamagama kuphela.
Ukukhuluma ngephimbo: Lapho iphendula, izwi layo lihleka, libubule, licacise umphimbo futhi liphefumule ngokwemvelo, kunokuba lifunde amagama nje.
Ukuqonda ngezindlela eziningi: Yinike amashadi noma izithombe-skrini, izifunde bese izichaza ngomlomo.
Ukubiza amathuluzi: Isekela ukubiza amathuluzi ngezinyathelo eziningi (i-Full-Duplex-Bench v3 Pass@1 ifinyelela ku-48%), ngakho iyakwazi “ukwenza umsebenzi”.
Imodi ye-Live: Ihambisana nephrothokholi ye-OpenAI GPT-Live futhi ingasebenza ngokuhlanganyela ne-backend yamamodeli amakhulu.
Ukufakwa okusheshayo: Umyalo owodwa ne-GPU eyodwa engu-80GB kwanele ukuyiqalisa; inguqulo ye-Flash ithumela iphakethe lokuqala ngemizuzwana engu-0.31 kuphela.
Ingaculwa: Ngokusebenzisa i-ms-swift ne-LoRA, ungenza i-fine-tuning yobuntu, indlela yokukhuluma, ulwazi lomkhakha nemikhuba yokubiza amathuluzi.

Ukulungiselela imvelo: Qinisekisa ukuthi uhlelo luyi-Linux + CUDA + Python 3.12, bese usebenzisa i-pip install sophon-realtime.
Ukuqalisa isevisi: Sebenzisa i-sophon-realtime serve. Ekuqaleni izodawuniloda ngokuzenzakalela izisindo ze-Tag ezingaba ngu-42GB; ngemva kwalokho isevisi ilalela ku-port 8000.
Ukubiza i-interface: Xhumana nemodeli nge-ws://HOST:8000/v1/realtime (full-duplex) noma nge-POST /v1/chat/completions (kungena izwi, kuphume umbhalo+izwi).
Ukuhlola ingxoxo: Sebenzisa i-python client_example.py --wav 我的问题.wav --lang zh --voice vivian_zh ukuthumela umsindo bese ulalela impendulo.
Ukuxhuma kwimodi ye-Live (uyazikhethela): Qalisa i-cd sophon-realtime/live && BUILD=tag PORT=8000 LIVE_PORT=8780 bash start_live.sh eduze kwe-engine, bese usebenzisa nephrothokholi ye-OpenAI GPT-Live ukuxhuma ne-backend model yakho enkulu.
Ukwenza i-fine-tuning (uyazikhethela): Sebenzisa i-ms-swift ne-LoRA ukuqeqesha ku-GPU engu-80GB. Lungisa idatha yengxoxo yefomethi ye-jsonl (umsindo ubhalwe , impendulo yengezwe omaka abafana no-[laughter]), bese usebenzisa umyalo we-swift sft ukushintsha ubuntu nolwazi lomkhakha.

Eyokuqala ku-full-duplex ye-open source: Ithole amaphuzu angu-82.8 ku-VoiceBench, ilingane ne-GPT-4o-mini-Audio, futhi yeqa amanye amamodeli e-full-duplex e-open source afana ne-MiniCPM-o 4.5 (76.1).
Ukuqonda ngezwi okunamandla kakhulu: Ku-Big Bench Audio enemibuzo engu-1,000 yokuqonda ngezwi, iphendule kahle ku-81.8%, yeqa kakhulu imodeli ye-open source ebihamba phambili ngaphambilini, i-Raon SpeechChat (58%).
Iqonda iphimbo: Imodeli yokuzwa i-DuplexJev ikleliswe kuqala ngokuhlanganyela ezilinganisweni ezingezona ezamagama ze-LISTEN, StressTest ne-MSPB (48.6), ngaphambi kwe-Gemini 3.8 Flash ne-Qwen3-Omni.
Ikwazi ukukhuluma ngephimbo: Ku-NVV-SuperBench, inguqulo ye-Tag ithole u-0.86 ngesiShayina no-0.80 ngesiNgisi; ukuhleka, ukububula nokucacisa umphimbo kuzwakala kungokwemvelo kakhulu.
Iyakwazi ukwenza umsebenzi: I-Full-Duplex-Bench v3 tool-calling Pass@1 ifinyelela ku-48%, iphakeme kune-Grok (43%); lapho kufanele iphendule khona, iphendula ngo-100%.
Iyashesha futhi iphatha abasebenzisi abaningi: Inguqulo ye-Flash ithumela iphakethe lokuqala ngemizuzwana engu-0.31 futhi i-GPU eyodwa isekela izikhathi ezingu-40 ezifanayo; inguqulo ye-Tag ithatha cishe imizuzwana engu-0.6 futhi isekela izikhathi ezingu-16 ezifanayo.

Indawo yokugcina ye-GitHub: https://github.com/adventists-ai/sophon-realtime

Insizakalo yamakhasimende ehlakaniphile: Inguqulo ye-Flash ithumela iphakethe lokuqala ngemizuzwana engu-0.31 futhi i-GPU eyodwa isekela izikhathi ezingu-40 ezifanayo. Iyakwazi ukuqonda imizwa yekhasimende futhi ibize amathuluzi ngezinyathelo eziningi ukuze iqhube izinsizakalo, ngakho ilungele izikhungo ezinkulu zezingcingo.
Ukuphelezela kwe-AI nezinhlamvu ezibonakalayo: Inguqulo ye-Tag inamandla amakhulu kakhulu okuveza iphimbo, iyahleka futhi ibubule, ngakho ilungele ukuphelezela ngokomzwelo, izithandani ezibonakalayo nama-NPC emidlalweni.
Umsizi wezwi emotweni: I-full-duplex ivumela ukuphazamisa noma nini nokuphendula ngokubambezeleka okuncane, ngaphandle kwegama lokuyivusa. Iqonda iphimbo futhi ingasiza ukuhlola isimo somshayeli.
Amadivayisi e-AI: Ubuchwepheshe beqembu sebuvele busetshenziswe emikhiqizweni yehardware engaphezu kuka-300. Ilayisensi i-Apache-2.0 ivumela ukusetshenziswa kwezentengiselwano, ngakho ilungele ukuhlanganiswa ezipikheni, kumathoyizi, kumarobhothi nakwamanye amadivayisi.
Abasebenzeli bezwi bezinkampani: Imodi ye-Live ihambisana nephrothokholi ye-GPT-Live. I-Sophon iphethe “ukulalela nokukhuluma”, kanti imibuzo nokwenziwa kwemisebenzi kudluliselwa ku-backend model enkulu, okulungele izinsizakalo ezenzelwe amabhange, uhulumeni, izindiza nezinye izimboni.