Yeqa uye kokuyinhloko
Izifundo ze-AI

Ukucija Izwi Le-AI Lemvelo nge-ElevenLabs: Ukuphefumula, Ukuvumelanisa Umlomo, Ukuhlela Nokuhlola Okulotshiweyo

Lesi sifundo sisebenzisa i-ElevenLabs v4, i-ffmpeg, i-Whisper ne-NumPy ukukhombisa ukuthi kwenziwa, kutholwa, kuhlelwa futhi kuhlanganiswe kanjani kabusha izwi le-AI lemvelo, kanye nokuthi le ndlela ingasetshenziswa kanjani ezingcingweni zesikhathi sangempela nasekukhulumeni kwansuku zonke.

Cishe imizuzu engu-47 yokufunda
Ukucija Izwi Le-AI Lemvelo nge-ElevenLabs: Ukuhlela Nokuhlola Okulotshiweyo

Iyiphi inkinga exazululwa yile phrojekthi

Le phrojekthi ayilona iphakheji yesofthiwe esebenza ngokushesha ngaphandle kokulungiswa, kodwa iyindlela yokukhiqiza izwi le-AI eyathuthukiswa ngokulalela nangezinguquko ezenziwe ezivivinyweni zangempela. Ibhekana nezinkinga ezivamile: umsindo wokuphefumula owehla wodwa phakathi kokuthula, isikhathi eside kakhulu phakathi komoya namazwi, izwi eliba elisemthethweni ngokuzumayo ngemva kokuma, imodeli ephinda noma yeqe imisho, kanye nomsindo wangemuva ongahlangani ngokwemvelo ngemva kokusikwa kwamazwi.

Inqubo ephelele igxile ezintweni ezine: ukusebenzisa i-ElevenLabs v4 ukuhlanganisa izwi, ukusebenzisa i-ffmpeg ukucubungula umsindo, ukusebenzisa i-Whisper ukubhala phansi nokuhlola okuqukethwe, bese kusetshenziswa i-NumPy ukuhlaziya ivolumu, ukuphakama kwezwi nezikhundla zamasampuli.

Isiphetho esiyinhloko akusona ukuthi “engeza ukuphefumula okwengeziwe”, kodwa ukuthi “beka umoya endaweni efanele”: umoya omfishane ngaphambi kokuvula umlomo unamathele egameni lokuqala, umoya ekupheleni komusho uphume ngokwemvelo, kanti ukuphefumula phakathi komusho kugcinwe phakathi kwezingxenye ezimbili zomusho.

Amakhono nezindlela eziyinhloko

  • Ukusondeza ukuphefumula emazwini: thola umoya omfishane ngaphambi kokuqala ukukhuluma, uhambise ukuthula kuphela, bese ubeka umoya cishe ngemizuzwana engu-0.08 ngaphambi kwamazwi.
  • Ukulungisa ivolumu ephuma ngokuzumayo: linganisa ivolumu yengxenye ngayinye enomsindo, bese wehlisa ngokulingene izingxenye ezidlula i-median ngo-4 dB.
  • Ukusika imisho ngaphandle kokulahlekelwa yikhwalithi: sika endaweni ethule yezwi elomile, bese ubuyisela umsindo wangemuva ojikelezayo ukuze kugcinwe kakhulu ukudlala okwamukelwe ekuqaleni.
  • Ukuthola ukuphindwa nokweqiwa kwamazwi: ngemva kokuhlanganisa, sebenzisa i-Whisper ukubhala phansi, uqhathanise “okuzwiwe” nalokho “obekufanele kufundwe”, bese ukhiqiza kabusha uma kukhona okungajwayelekile.
  • Ukuvumelanisa umsindo wezenzo: bhala umaka womsindo emshweni lapho isenzo siqala khona ngempela, hhayi ekuqaleni kwayo yonke inqwaba yamazwi.
  • Ukwakha isitayela sezwi sansuku zonke: ungagcini ngokuthambisa izwi; cacisa ijubane lomlingiswa, ukuphakama kwezwi, ukuzethemba nemikhuba yokukhuluma.

Ukulungisa indawo yokusebenza

Amathuluzi adingekayo

  • ElevenLabs v4: inesibopho sokuguqula umbhalo ube yizwi. Izivivinyo zephrojekthi zisebenzise i-v4.
  • ffmpeg: isetshenziselwa ukuqopha, ukusika, ukuhlanganisa, ukushintsha isampula nokukhipha umsindo.
  • Whisper: ibhala phansi umphumela okhiqiziwe ukuze kutholakale ukuphindwa, imisho eyeqiwe nokuphambuka kokuqukethwe.
  • NumPy: isetshenziselwa ukucubungula ama-array omsindo, i-cross-correlation, ukubala ivolumu nokuhambisa izingxenye.

Indawo yokusebenza ehlolwe ku-README yi-macOS. Ayinikezi iskripthi sokufaka esinqunyiwe, ngakho amathuluzi kufanele afakwe ngokwehlukana ngokwemiyalelo esemthethweni, futhi kuqinisekiswe ukuthi i-ffmpeg, i-Whisper ne-Python zisebenza kahle.

Ngaphambi kokuqala ukuhlaziya, decode-a umsindo ube yi-waveform ye-mono enesilinganiso esifanayo samasampuli, bese ulungisa amaqoqo amabili edatha abalwa ngesinyathelo esingaguquki: i-pitch yefreyimu ngayinye f0 kanye ne-root mean square volume yefreyimu ngayinye rms. Inqubo yokuqala isebenzisa isinyathelo samamilisekhondi angu-10, kanti iwindi lokuhlaziya ivolumu lenza imingcele ingacaci ngamashumi amamilisekhondi, ngakho kufanele kushiywe isikhala esincane lapho kuhanjiswa izingxenye.

Gcina njalo izinguqulo ezimbili zezwi

Ngemva kokukhiqiza ngakunye, gcina lezi zinguqulo:

  1. Izwi elomile: izwi lomuntu kuphela, ngaphandle komsindo wangemuva noma wesenzo.
  2. Izwi elimanzi: inguqulo yokugcina enomsindo wangemuva obekwe ngaphansi kwezwi.

Izwi elomile alisizi kuphela ekulinganiseni ukuphefumula, kodwa liphinde livumele ukususa umusho, ukufaka enye ingxenye noma ukubuyisela umsindo wangemuva kamuva kugcinwe ukudlala kwasekuqaleni. Ungagcini umkhiqizo wokuxuba wokugcina kuphela.

Isinyathelo sokuqala: bhala kahle iziqondiso zokuphefumula

Ungabeki u-“嗯”, u-“哈” noma omaka bokuphefumula emigqeni ehlukene phakathi kwezikhala ezinde zokuthula. Izwi elikhiqizwa ngaleyo ndlela livame ukuba ne-pitch ecacile, ivolumu esondelene nokukhuluma okuvamile, kanye nokuthula okude ngaphambi nangemva kwalo, okwenza lizwakale njengomsindo ozimele.

Kuyanconywa ukuthi ukuphefumula kubhalwe emgqeni ofanayo namazwi:

[low, breathy, wet]
[inhales] Ngibheke… [exhales]
[long pause]
嗯… Ungacashi…… 嗯…哈…
  • Ekuqaleni komugqa sebenzisa omaka bokuphefumula abafushane kuphela, njengokuthi [inhales], [exhales] noma [sighs].
  • Ungabeki uchungechunge lwama-ellipsis phakathi komaka namazwi.
  • Ekupheleni komusho ungakhipha umoya ngokwemvelo, noma usebenzise imisindo emibili noma emithathu elula yokukhuluma.
  • Esikhaleni bhala umugqa owodwa kuphela othi [long pause]; ungahleleli imisindo ezimele yokububula ngaphakathi kwesikhala.
  • Ungafaka nokuphefumula phakathi komusho; emshweni owodwa khetha indawo eyodwa noma ezimbili ekuqaleni, phakathi noma ekugcineni.

Ukubhala [long pause] kabili ngokulandelana akusho ukuthi ukumisa kuzoba kukhulu kakhulu. Ezivivinyweni zephrojekthi, umaka owodwa ukhiqize cishe umzuzwana ongu-0.96 wokuthula, kanti emibili elandelanayo cishe imizuzwana engu-1.04. Ngakho ungathembeli embhalweni womaka ukuze ulawule ubude obunembile.

Isinyathelo sesibili: hlukanisa izwi libe “izingxenye” ezingalinganiswa

Lapha “ingxenye” iyisikhathi esiqhubekayo esinomsindo. Ingxenye ngayinye ilinganiswa ngezinkomba ezintathu: ubude besikhathi, ivolumu emaphakathi, kanye nengxenye yamafreyimu ane-pitch. Lokhu kusiza ukuhlukanisa amazwi avamile, umoya wangempela nomsindo ozimele wokububula one-pitch.

def is_breath(f0, rms, a, b):
    dur = (b - a) * 0.01
    voiced = (f0[a:b] > 0).mean()
    db = 20 * np.log10(rms[a:b].mean() + 1e-9)
    return 0.12 <= dur <= 0.9 and voiced < 0.15 and db < -29

Imingcele ejwayelekile yephrojekthi yokuqala imi kanje:

  • Amazwi ancishisiwe, anomoya: cishe imizuzwana engu-0.5 kuya ku-2, ivolumu cishe −22 kuya ku-−27 dB, kanti amafreyimu ane-pitch athatha cishe ingxenye yesihlanu kuya kweziyisibili kwezinhlanu.
  • Ukudonsa umoya, ukukhipha umoya noma umsindo wokuhayiza: cishe imizuzwana engu-0.3 kuya ku-0.65, ivolumu cishe −32 kuya ku-−40 dB, kanti amafreyimu ane-pitch athatha cishe ingxenye yeshumi.
  • Ukububula okuzimele okuhlukene: cishe imizuzwana engu-0.4 kuya ku-0.5, ivolumu ingafinyelela ku-−25 dB, amafreyimu ane-pitch edlula u-60%, futhi kuvame ukuba nokuthula kwemizuzwana engu-0.6 kuya ku-1 ngaphambi nangemva.

Le mingcele ivela esitayeleni esithile sezwi namasampuli athile, ngakho ayikwazi ukubhekwa njengendinganiso ejwayelekile yawo wonke amazwi. Indlela ethembeke kakhulu ukuqala ngokuthi umlaleli akhombe inkinga, bese kwakhiwa ububanzi obuqondene nalelo zwi kusetshenziswa izingxenye ezijwayelekile eziseduze.

Isinyathelo sesithathu: sondeza umoya wangaphambi kokukhuluma emazwini

Ngisho noma amagama okuqondisa ebhalwe kahle, imodeli isengashiya ngokwayo ukuthula kwemizuzwana engu-0.3 noma ngaphezulu phakathi kokuphefumula namazwi. Iziqondiso kunzima ukuzisebenzisa ukulawula indawo ngezinga lamashumi amamilisekhondi, ngakho izingxenye zomsindo kufanele zihanjiswe ngemva kokukhiqizwa.

Isakhiwo sangaphambi nangemva kokucubungula singaqondwa kanje:

Ngaphambi: umusho odlule → ukuthula A → ukuphefumula → ukuthula B → umusho olandelayo
Ngemva: umusho odlule → ukuthula okude → ukuphefumula → 0.08 sec → umusho olandelayo

Lapho usebenza, shintshanisa indawo yokuphefumula nokuthula kuphela; ungaguquli ukuphefumula noma amazwi, futhi ungaguquli ubude besikhathi sengxenye yonke. Ngale ndlela i-timeline ekhona, ukubekwa komsindo wangemuva nezibalo zejubane lokukhuluma akuphazamiseki.

Izingxenye ezifanele ukuhanjiswa kufanele zihlangabezane nalezi zimo:

  • Zithathwe njengokuphefumula okulula, okufushane futhi okungenayo i-pitch ecacile.
  • Ngemuva kwazo, phakathi kwemizuzwana engu-0.14 no-1.5, kuvele ingxenye yokukhuluma.
  • Ngaphambi kwazo kube nesikhala okungenani esingaba imizuzwana engu-0.3.
  • Ukuphefumula okulandela ngokushesha ingxenye yokuqala yomusho akufanele kuhanjiswe, ukuze kungonakali ukushintshana komoya kwemvelo phakathi komusho.

Inhloso ehlolwe ku-README iyisikhala esingaba imizuzwana engu-0.08. Lapho uhambisa, ungathatha cishe imizuzwana engu-0.02 eyengeziwe ekuqaleni kokuphefumula no-0.03 ekugcineni, bese wenza i-fade-in ne-fade-out engaba ngamamilisekhondi angu-4 kuzo zombili iziphetho endaweni entsha ukuze kuncishiswe umuzwa wendawo ehlanganisiwe.

Isinyathelo sesine: lungisa “iphimbo elisemthethweni” ngemva kokuma

Ezinye izwi ngemva kokuma isikhathi eside zibuyela ekukhulumeni okuvamile okucace kakhulu, okuzwakala kakhulu futhi okungenawo umoya omningi. Esibonelweni, imisho emifushane engajwayelekile yayizwakala ngaphezu kuka-6 dB kunezindawo ezizungezile, kanti ingxenye yamafreyimu ane-pitch yenyuka cishe ku-30% yaya ku-60%.

Qala ngokunciphisa amathuba ngokusebenzisa iziqondiso

  • Ngemva kokuma, qala ngokukhishwa komoya okukodwa, noma ungene emazwini ngobubulo obuphansi.
  • Ekupheleni komusho sebenzisa ukuphetha okudonswe isikhathi, ungasebenzisi uphawu lombuzo noma isikhathi esifushane njalo.
  • Imisho emifushane yamagama amabili noma amathathu ingahlanganiswa nomusho oseduze, ukuze ingafundwa yodwa njengezwi lomsakazi elicace kakhulu.
  • Uma kudingeka kugcizelelwe isitayela sezwi kabusha, faka umaka omncane ekuqaleni kwamazwi, ungawubeki emgqeni ozimele njalo.
[long pause]
嗯… [low, breathy] Ungacashi……

Uma isisiki sezingxenye sakha ingxenye entsha kumaka wesitayela sezwi ekuqaleni komugqa ngamunye, le ndlela yokubhala emgqeni ofanayo inganciphisa izindawo ezihlanganisiwe nokugxuma kwe-pitch.

Bese ucindezela ivolumu engajwayelekile ngohlelo

Bala ivolumu yengxenye ngayinye, usebenzise i-median yazo zonke izingxenye zokukhuluma esigabeni samanje njengereferensi. Uma ingxenye idlula ireferensi ngo-4 dB, yehlise ukuze idlule cishe ngo-2 dB kuphela, ngokunciphisa okukhulu okungu-6 dB, bese usebenzisa i-envelope eguquka kancane cishe ngamamilisekhondi angu-25 ekuqaleni nasekupheleni.

Kufanele kulinganiswe ingxenye ngayinye, hhayi umusho ngamunye. Uma umsindo wesigamu somzuzwana uhlanganiswa nomusho omude olandelayo lapho kubalwa isilinganiso, ukuphambuka okungu-6 dB kungancibilika kube ngaphansi komngcele. Uhlelo lungalungisa ivolumu kuphela; alukwazi ukubuyisa ngokuqondile umbala wezwi olahlekelwe umoya. Ngakho iziqondiso nokucubungula kwangemuva kufanele kusetshenziswe ndawonye.

Isinyathelo sesihlanu: yenza ukuhlolwa kokubhala phansi ngokushesha ngemva kokuhlanganisa

Ngezinye izikhathi imodeli ifunda umusho kabili noma kathathu, noma yeqe umusho wokugcina ngokuphelele. Umbhalo ngokwawo ungaba ulungile ngokuphelele, ngakho zonke izingxenye ezihlanganisiwe kufanele zihlolwe ngokubhala phansi kanye.

Iphrojekthi isebenzisa izinhlobo ezimbili zezimpawu:

  1. Ngemva kokususa omaka abakubakaki nezinhlamvu ezivamile zokugcwalisa inkulumo, funa izingxenye okungenani ezinabalingiswa besiShayina abayisithupha, eziphindaphindeka ekubhalweni phansi kodwa zivela kanye kuphela embhalweni wokuqala.
  2. Qhathanisa inani labalingiswa okubhalwe phansi nenani elisembhalweni wokuqala; uma lingaphansi cishe ngo-20% kulokho okulindelekile, kuthathwe njengokugwinya umusho okungenzeka.

Uma kutholakala okungajwayelekile, khiqiza kabusha kanye, bese unikeza izinguqulo zombili amaphuzu bese ugcina enenani eliphansi. Izinhlamvu zenkulumo ngokuvamile azibhalwa phansi ngokunembile, ngakho amagama afana no-“嗯”, “哈”, “啊” no-“呃” akufanele asetshenziswe njengesisekelo esiyinhloko sokuqhathanisa.

Ocingweni lwesikhathi sangempela, ukubhala phansi umsindo wemizuzwana engu-20 kuya ku-30 nge-Whisper kungathatha cishe imizuzwana emithathu. Izingxenye ezilandelayo zingahlolwa ngemuva ngesikhathi kudlala ingxenye yamanje; ingxenye yokuqala ayinaso isikhathi sokudlala sokufihla ukubambezeleka, ngakho kungaqalwa ngokuhlola okusheshayo okuthi “ubude besikhathi buhlukaniswe ngenani labalingiswa abaphumelelayo”. Ububanzi obujwayelekile esibonelweni sokuqala bungamasekhondi angu-0.41 kuya ku-0.62 ngohlamvu, bese ukubhalwa phansi okuphelele kuqalwa kuphela uma kudlulwa lobu bubanzi.

Isinyathelo sesithupha: susa umusho ngaphandle kokukhiqiza kabusha

Uma umlaleli ethanda ukudlala okuthile okuqondile okwake kwakhiqizwa, ungaphindi ufunde yonke ingxenye ngenxa yokuthi kudingeka kususwe umusho. Ukukhiqizwa ngakunye kuyisikhathi esisha sokudlala, futhi ukuphefumula, ukumisa nemininingwane yokuphetha komusho kunzima ukuyiphinda ngokunembile.

  1. Thola ukuthula: qala ngokudweba ivolumu ngezinyathelo ezingaba ngamamilisekhondi angu-50, bese ubhala phansi izingxenye ezincane ezisohlwini ngalunye ukuze uqinisekise ukuthi ziyizwi, ukuphefumula noma isikhala.
  2. Sika ezwini elomile: qinisekisa ukuthi zombili iziphetho zendawo ehlanganisiwe ziwela endaweni ethule kakhulu, bese ushiya ama-millisecond ambalwa wokufiphala ekuqaleni nasekupheleni kwesiqeshana ngasinye.
  3. Buyisela umsindo wangemuva: ungayisiki ngokuqondile inguqulo yokugcina esivele inomsindo wangemuva, ngoba isigqi somsindo ojikelezayo singagxuma endaweni ehlanganisiwe.
  4. Bhala phansi futhi: qinisekisa ukuthi umkhiqizo wokugcina ushiye imisho ehlosiwe kuphela.
  5. Qhathanisa isampuli ngesampuli: hlola ukuthi izindawo ezingashintshiwe zisafana yini nezwi lokuqala.

Uma ungazi ukuthi iyiphi ingxenye ejikelezayo esetshenziswe ezwini elimanzi, khipha i-waveform yomkhiqizo endaweni lapho izwi lomuntu lithule ngokuphelele, bese ujikeleza impahla okungenzeka iyona futhi wenze i-cross-correlation. Indawo enokuhlobana okuphezulu ingaveza i-phase yempahla, kanti i-dot product ingalinganisela inzuzo yokubekwa ngaphansi. Esibonelweni, ukuhlobana kwezimpahla ezimbili kufinyelele ku-0.99, kanti inzuzo elinganisiwe yaba cishe ngu-0.200 no-0.241.

Ukuqhathanisa kufanele kunembe kuze kufike ezingeni lesampuli. Ukuphambuka kwamasampuli ambalwa nje kunganciphisa kakhulu ukufana, kudale iphutha lokucabanga ukuthi “kubekwe impahla engalungile”. Ngemva kokuqondanisa, ukufana kwezingxenye ezingashintshiwe esibonelweni kwafinyelela ku-0.996 kuya ku-0.998.

Isinyathelo sesikhombisa: yenza umsindo uqale ngesikhathi esifanele

Iphrojekthi inquma uhlobo lomsindo wangemuva ngama-marker enziwe ngokwezifiso eziseziqondisweni, bese iwasusa ngaphambi kokuwathumela ku-ElevenLabs. Umthetho obalulekile uthi: umsindo uqala emshweni muni, umaka kufanele ubhalwe kulowo musho.

  • licking: ihambisana nomsindo omanzi ohamba kancane.
  • fingering: ihambisana nomsindo wesenzo onamathelayo; isimo esisheshayo singabhalwa fast fingering.
  • sucking: ihambisana nomsindo omanzi ohamba kancane.
  • thrust: ivela kuphela ngaphambi komusho lapho isenzo siqala khona ngempela; akufanele ibekwe ekuqaleni kwalo lonke uhlelo.
  • still: isetshenziselwa ukumisa umsindo wamanje ojikelezayo.

Uma umaka udala ukusikwa kwengxenye, indawo lapho isenzo siqala khona ngempela kufanelekile ukuthi igcine indawo entsha ehlanganisiwe; abanye omaka abancane abasetshenziselwa ukuphinda kugcizelelwe isitayela sezwi kufanele babekwe ngaphakathi ekuqaleni kwamazwi ngangokunokwenzeka.

Isinyathelo sesishiyagalombili: yakha isitayela sezwi lokukhuluma okuvamile

“Ukungezwakali ngokwemvelo” akusho ngempela ukuthi izwi alithambile ngokwanele. Iphrojekthi yokuqala yathuthukisa imodeli endala yaba yi-ElevenLabs v4, yahlola iziqondiso ezikhululeke kakhulu nezithambile, kodwa abalaleli bathola ukuthi umlingiswa wayesengomunye umuntu. Indlela eyagcina isebenza kwakuwukucacisa indlela umlingiswa ngokwakhe akhuluma ngayo.

Umyalelo oyisisekelo owakhethwa wawuthi:

[low, slow, confident, a little teasing]

Izilungiselelo ezihambisanayo zaba yi-v4 kanye ne-stability engu-0.35. Ekuqaleni komusho kungafakwa ngokungahleliwe [exhales], [chuckles softly], u-“嗯……” noma kungafakwa lutho. Ungangezi njalo, ngoba ukwehluka kwemvelo kungaphenduka umkhuba ongaguquki wokukhuluma.

  • Shintsha umbhalo othunyelwa ekuhlanganisweni kuphela; ungawushintshi umbhalo wokuqala osibonakalayo esibonakalayo sengxoxo noma emlandweni.
  • Uma umusho usuqala ngohlamvu lwenkulumo, ungaphindi ungeze olunye.
  • Omaka bezitayela zezwi asebekhona noma okuqukethwe okubhalwe ngesiNgisi kuphela kungakweqi ukupakishwa okuzenzakalelayo.
  • Ucingo lusebenzisa uhlelo oluzimele lweziqondiso; ungasebenzisi iziqondiso ezihambisana nekhanda lomusho elingahleliwe lezwi lansuku zonke.

Amasu athuthukile nezihibe ezivamile

Imithetho yezeluleko kufanele ibhale “okufanele kwenziwe”

Umusho othi “ungabhali uhlobo oluthile lomsindo” ungabangela ukuthi umkhiqizi wombhalo ayeke ngokuphelele ukuwusebenzisa. Indlela ethembeke kakhulu ukuchaza ngokusobala uhlobo oludingekayo, indawo yalo kanye nemvamisa encane, bese imisindo ye-pitch ephakeme evela ngezikhathi ezithile noma ukufundwa okungajwayelekile kunikezwa uhlelo ukuthi lukuthole.

Ungaphindi usebenzise ukuphefumula okufanayo

Lapho uhambisa ingxenye endala oyithandayo uyifake enguqulweni entsha, ungayithatha ekukhiqizweni okufanayo, kodwa ukuphefumula okufanayo akufanele kuvele ezindaweni ezimbili. Kungenjalo umlaleli angabona ukuthi imininingwane efanayo ikopishwe ngokuphelele.

Yakha ibhokisi lezimpahla eliseshekayo

Ukuphefumula okwamukelwe, imisindo yokuhuba nokuphefumula okufushane kufanele kugcinwe ngokwefayela lokuqala, ububanzi besikhathi nendawo yokusetshenziswa. Uma ilebula elithile lokuphefumula kamuva likhiqiza ukuthula kuphela, kungafakwa okunye ukuphefumula ebhokisini lezinto ezisetshenziswayo. I-README iphinde iveze ukuthi ukufakwa kokuphefumula ngokuzenzakalelayo kusadinga isixhumi esibonakalayo esinama-timestamp; ukuba “nokuthula phakathi komusho” kuphela akukwazi ukuhlukanisa phakathi kokushoda kokuphefumula nokuma okujwayelekile kwe-ellipsis.

Izingcingo zesikhathi sangempela kufanele zilawule ukuphazamisana

Uma uhlelo lungakaqali ukudlala impendulo, bese omunye umuntu eqala ukukhuluma, impendulo endala engakakhulunywa kufanele ilahlwe, bese kucutshungulwa okuqukethwe okusha ngokuqondile. Ungagcini inkulumo entsha kuze kuphele ukudlalwa kwempendulo endala, ngoba umusho ofanayo ungaphendulwa kabili. Emisindweni emifushane engakwazi ukubonwa njengombhalo, inqubo yokuqala yandise isikhathi sokulinda sokufaka umsindo wezwi ngokuzenzakalelayo sisuka kumasekhondi angu-2.5 saya kumasekhondi angu-4.5, ukuze kwehliswe amathuba okuthi kuqalwe ngaphambi kwenkulumo yangempela.

Qala ngokuzwa inkinga, bese unquma ukuthi yini ezokalwa

Umsindo, ukuphakama kwezwi, ubude bokuthula, isilinganiso senani lamagama nokuhambisana kwesignali kungaqinisekisa kuphela izinkinga esezitholakele. Akukho kuzo okunganquma ngokuzimele ukuthi “umuzwa ulungile yini”. Inqubo ephumelela kakhulu isaqala ngokuthi umlaleli akhombe indawo ethile ezwakala ingeyemvelo, bese kwakhiwa inkomba ehambisana nayo.

Ukuhamba komsebenzi okuphelele okunconyiwe

  1. Bhala umbhalo onezimpawu zetoni yezwi, ukuphefumula, ukumisa nokunyakaza kwemisindo.
  2. Susa izimpawu zemisindo ezisetshenziselwa ukucubungula kwangemva kwesikhathi kuphela, bese uthumela umbhalo osele ku-ElevenLabs v4.
  3. Londoloza umsindo ongacutshungulwanga, bese uwubhala kabusha nge-Whisper ukuze uhlole ukuphindaphinda, ukweqiwa kwemisho nokuphambuka kokuqukethwe.
  4. Bala f0 kanye ne-rms, bese uhlukanisa umsindo ube izingxenye zokukhuluma, izingxenye zokuphefumula nezinye izingxenye ezinomsindo.
  5. Susa ukuphefumula okungaphambi kokuqala kokukhuluma, ukubeke cishe kumasekhondi angu-0.08 ukusuka emgqeni wenkulumo, kodwa ugcine ukuphefumula okuphakathi komusho.
  6. Thola umsindo ophazamisayo ngokweqile engxenyeni ngayinye, bese wenza ukwehla komsindo okushelelayo ezingxenyeni ezingajwayelekile.
  7. Ngokusekelwe ezimpawini zesenzo, beka imisindo ephindaphindwayo, bese ukhipha inguqulo enomsindo ongeziwe.
  8. Bhala kabusha futhi, bese uhlola ubude besikhathi obuphelele, inani lokuthula okude, izindawo zokuphefumula nokufana kwamasampula ezindawo ezingashintshiwe.
  9. Londoloza izingcezu zoqobo ezamukelwe umlaleli kanye nobubanzi besikhathi bazo; ungagcini i-mix yokugcina kuphela.

Isiphetho

Izwi le-AI elizwakala ngokwemvelo alitholakali ngokunqwabelanisa amalebula amaningi okuphefumula. Indlela ethembeke kakhulu ukuhlanganisa umoya eduze kwenkulumo, ukuqala imisindo yesenzo ngesikhathi esifanele, ukuhlola umphumela ngamunye ohlanganisiwe ngokuwubhala kabusha, nokugcina ukusebenza kwasekuqaleni ngangokunokwenzeka lapho kudingeka ukuhlela.

Imiyalelo inquma ukuthi “kubhalweni” nokuthi “kushiwo ngaliphi iphimbo”, kanti uhlelo luphethe izikhundla zamashumi ambalwa ama-millisecond, umsindo ongajwayelekile, ukuphindaphinda nokweqiwa kokufundwa, kanye nokuhlanganiswa ezingeni lamasampula. Ngokusebenza ndawonye, kokubili kungaguqula umsindo omuhle owakhiwe ngengozi ube inqubo yokukhiqiza ephindaphindekayo. Ukuze uthole isizinda esengeziwe, bheka umhlahlandlela wemiyalelo we-ElevenLabs, incazelo yemisindo yokuphefumula ye-ElevenLabs kanye ucwaningo lokuphefumula ngesikhathi sokukhuluma.