Indlela yokwakha umuntu wedijithali we-AI endaweni yangakini

Ufuna ukwakha umuntu wedijithali we-AI ekhompyutheni yakho okwazi ukulalela, ukukhuluma, ukuvumelanisa ukunyakaza kwezindebe, nokuxoxa nawe, ngaphandle kokusebenzisa amathuluzi e-AI noma ukukhokha izindleko ezibizayo zokubhalisa? Ngokuxhumanisa ukuqaphela inkulumo, imodeli enkulu esebenza endaweni, ukwenziwa kwezwi, nokuvumelanisa ukunyakaza kwezindebe, ungathola uhlelo lokuxhumana ngesikhathi sangempela olusebenza ekhompyutheni yakho.

Lesi sihloko siqala ngokulungiselela i-hardware, bese sikuhola ekufakweni okuphelele endaweni ye-Windows + WSL2. Umphumela wokugcina uthi: ikhasi lewebhu libonisa umuntu wedijithali owenziwe ngokwezifiso; sixoxa nomuntu wedijithali we-AI ngemakrofoni, bese umuntu wedijithali we-AI ephendula ngezwi olilingise lona.

1. Ukuhamba komsebenzi kohlelo lonke

Lolu hlelo aluqhutshwa yimodeli eyodwa kuphela, kodwa lufaka izingxenye eziningi:

Lapha, i-llama.cpp inesibopho sokusebenzisa imodeli yolimi yendawo; i-faster-whisper iguqula inkulumo yomsebenzisi ibe umbhalo; i-Qwen3-TTS ikhiqiza izwi lokuphendula ngokusekelwe kumsindo oyisibonelo; kanti i-LiveTalking ne-Wav2Lip ziguqula umsindo ube ukunyakaza kwezindebe okuhambisanayo.

2. Izidingo ze-hardware nesabelomali se-video memory

Ukucushwa okuncane:

• Ikhadi le-NVIDIA: 8GB ye-video memory

• Imemori: 16GB

• Isikhala sediski esitholakalayo: okungenani 60GB

• Uhlelo: Windows 10/11, olusekela i-WSL2

Ukucushwa okunconyiwe:

• Ikhadi le-NVIDIA: 16GB ye-video memory noma ngaphezulu

• Imemori: 32GB noma ngaphezulu

• Kuyanconywa ukusebenzisa i-SSD ukugcina amamodeli namafayela e-WSL

Ngokusebenzisa imodeli ye-Qwen3 14B quantized njengesibonelo, i-video memory izokwabiwa cishe kanje:

• Ingxenye｜Ukusetshenziswa kwe-video memory okuyisibonelo

• Qwen3-14B Q4_K_M｜Cishe 9GB

• faster-whisper large-v3｜Cishe 3GB

• Qwen3-TTS 1.7B｜Cishe 4GB

• Wav2Lip 256｜Cishe 1.3GB

• Isamba｜Cishe 17GB

Ikhadi eline-24GB ye-video memory lizosebenzisa le nhlanganisela kalula. Nge-16GB ye-video memory, ungasebenzisa imodeli ye-8B futhi wehlise ubude bomongo; cishe nge-8GB, kunconywa ukuqala ngemodeli ye-4B nemodeli ephakathi yokuqaphela inkulumo.

3. Lungisa inethiwekhi nememori ye-WSL2

1.  Qala ngokwakha ifayela le-.wslconfig kufolda yomsebenzisi ku-Windows:

C:\Users\igama_lomsebenzisi\.wslconfig

2. Faka okuqukethwe okulandelayo:

[wsl2]
memory=32GB
networkingMode=mirrored

[experimental]
hostAddressLoopback=true

I-memory ingashintshwa kuye ngememori yakho yangempela; ngokuvamile ukwaba uhhafu kuya ezingxenyeni ezintathu kwezine zememori ebonakalayo kwanele. I-hostAddressLoopback=true ibaluleke kakhulu ekuxhumaneni kwe-WebRTC phakathi kwe-Windows ne-WSL kule khompyutha. Uma ingekho, ukuxhumana kwe-WebRTC okulandelayo kuzohluleka njalo ngaphandle kokubonisa isizathu esicacile; isiphequluli sizobonisa iphutha lokuhlaziya i-JSON, kanti uhlu lwabancintisana be-ICE luzoba nalutho.

3. Ngemva kokulungisa, sebenzisa lokhu ku-PowerShell:

wsl --shutdown

Ukucushwa kuzosebenza kuphela ngemva kokungena kabusha ku-WSL.

4. Qalisa imodeli enkulu yolimi yendawo ku-Windows

1. Lungisa i-llama.cpp

Landa i-llama.cpp efanele i-Windows, uyikhiphele kufolda engaguquki, isibonelo:

E:\llama.cpp

Ikheli lokulanda lenguqulo ezinzile: https://pan.quark.cn/s/0073852e4042

2. Khetha imodeli ngokwe-video memory

• Ububanzi be-video memory｜Imodeli enconyiwe｜Ikheli lokulanda

• Ngaphezu kuka-16GB｜Qwen3-14B-Instruct Q4_K_M｜https://pan.quark.cn/s/f48c2144ac71

• 8–16GB｜Qwen3-8B GGUF Q4_K_M｜https://pan.quark.cn/s/78b8643f1be3

• Ngaphansi kuka-8GB｜Qwen3-4B GGUF｜https://pan.quark.cn/s/fd588898ccbf

Faka ifayela le-.gguf elilandiwe lapha:

E:\llama.cpp\models

3. Qalisa isevisi ye-inference

Ku-Windows Command Prompt, sebenzisa:

cd /d E:\llama.cpp
llama-server -m E:\llama.cpp\models\Qwen3-14B-Instruct-Q4_K_M.gguf ^
  -np 1 -c 8192 -fa on --temp 1.0 --top-p 0.95 ^
  --host 0.0.0.0 --port 8090

Imisebenzi yamapharamitha ambalwa:

• host 0.0.0.0: ivumela i-WSL ukuthi ifinyelele isevisi yemodeli eseceleni kwe-Windows;

• c 8192: isetha ubude bomongo, obanele ezingxoxweni ezimfushane;

• temp 1.0: kwenza izimpendulo zingabi lukhuni kakhulu;

• port 8090: icacisa imbobo yesevisi.

Leli windi lomyalo kufanele lihlale lisebenza. Uma kubikwa ukuthi imbobo isetshenziswa kwenye indawo kodwa ukuhlola okuvamile kungayitholi inqubo, ungabheka ukuthi i-Hyper-V iyibekele yini le mbobo:

netsh interface ipv4 show excludedportrange protocol=tcp

Uma kutholakala ukungqubuzana, indlela elula ukushintsha imbobo kusevisi yemodeli nasemibhalweni eyibizayo ngesikhathi esisodwa.

5. Yenza isithombe esiyisisekelo somuntu wedijithali nevidiyo yakhe

1. Qala ngokuklama isithombe somuntu wedijithali osithandayo

Isithombe somuntu wedijithali akufanele sigxile ebuhleni kuphela; kufanele sishiye nendawo yokwakheka kwekhasi lewebhu nokukhiqizwa kokunyakaza kwezindebe. Ukwakheka okuthembekile kungaba:

• Sebenzisa isilinganiso sesithombe esingu-16:9;

• Umuntu abe engxenyeni yesithathu engakwesokudla;

• Shiya isikhala esikhulu esimnyama kwesokunxele ukuze kubekwe ibhola lokuxhumana nemibhalo engezansi;

• Izindebe zivaleke ngokwemvelo, kungabonakali amazinyo;

• Izandla akufanele zivale umlomo, isilevu noma uhlaka lobuso;

• Sebenzisa ukukhanya okuseceleni okumnyama kancane ukuze kuncishiswe ukubukeka okungajwayelekile okubangelwa ukufiphala okuncane endaweni yezindebe.

Ungasebenzisa lo mbono we-prompt, ushintshe umuntu, izingubo nendawo ngokwakho:

Isithombe esiseceleni sowesifazane omdala ohlala kwesokudla sesithombe, ukukhanya okuphansi okunomuzwa we-cinema, ukukhanya okuthambile okuchaza uhlaka, 
isikhala esikhulu esimnyama kwesokunxele, isimo sobuso semvelo, izindebe ezivaliwe, ikhamera enganyakazi, 
ukuthungwa kwesikhumba kwangempela, ukwakheka okuvundlile okungu-16:9.

Kuma-negative prompt ungafaka: umlomo ovulekile, amazinyo abonakalayo, umuntu ophakathi, isizinda esikhanyayo nesiminyene, isandla esivale umlomo, umbhalo, i-watermark, isikhumba sepulasitiki, isitayela sekhathuni.

2. Sebenzisa i-ComfyUI + Wan2.2 ukukhiqiza ividiyo yomuntu wedijithali yemizuzwana engu-5

Landa futhi ufake i-ComfyUI, uyivule bese ukhetha i-workflow ye-Wan2.2 yokuguqula isithombe sibe yividiyo, usebenzise isithombe esiyisisekelo njengohlaka lokuqala. Ekusetshenzisweni kokuqala, kunconywa la mapharamitha alondolozayo:

• Ipharamitha｜Inani elinconyiwe

• Ukulungiswa｜Landela isilinganiso sesithombe esiyisisekelo; isibonelo sesifundo sangempela ngu-704 × 896

• Ubude｜Imizuzwana engu-5

• Izinga lohlaka｜25 fps

• Prompt Enhance｜Vala

Ikheli lokulanda i-ComfyUI: https://pan.quark.cn/s/bb9d21f1e724

I-prompt kufanele ichaze ukunyakaza kuphela, ungaphindi uchaze ukubukeka komuntu. Isibonelo:

Umuntu uhlale endaweni efanayo, uphefumula kancane kuphela, acwayize kancane kanye, anyakazise ikhanda kancane ngokwemvelo bese ebuyela esimweni sokuqala, 
izindebe zihlale zivaliwe, ikhamera enganyakazi, akukho kusondeza, ukuphenduka noma ukunyakaza kwekhamera.

Khipha ifayela uliqambe ngokuthi: idle.mp4

Njengoba ukunyakaza kwezindebe ngesikhathi sokukhuluma kuzokhiqizwa bukhoma yi-Wav2Lip, akudingeki wenze amanye amavidiyo amaningi afana nokuthi “uyakhuluma” noma “ulalele”.

Nanka amaphutha avamile:

• Ungavuli ukuthuthukiswa kwe-prompt; vala i-prompt_enhance, ngoba imodeli ingase iphinde idizayine umuntu noma indawo;

• Isilinganiso sokukhishwa kufanele sifane nesesithombe sokufaka. Uma usebenzisa isithombe esivundlile esingu-16:9, shintshela kusayizi ovundlile osekelwa yi-workflow ukuze ugweme ukwelulwa noma ukusikwa komuntu;

• I-prompt mayibhale ukunyakaza kuphela; ukuphinda uchaze ukubukeka kungabangela umuntu ashintshe indawo noma ukubukeka;

• Ividiyo yemizuzwana engu-5 yanele ukwenza i-loop yokulinda; ukwelula ividiyo ngaphandle kwesizathu kwandisa amathuba okuphindaphindeka noma ukuguquka komumo.

6. Qopha izwi eliyisibonelo (ukulingisa izwi)

I-Qwen3-TTS izolingisa umbala wezwi nendlela yokukhuluma ngokusekelwe kumsindo oyisibonelo. Kunconywa ukuqopha umsindo wemizuzwana engu-5–15, onezidingo ezilandelayo:

• Kukhulume umuntu oyedwa kuphela;

• Indawo ithule, ingabi nawo umculo noma i-echo ecacile;

• Ijubane lokukhuluma libe lemvelo, imizwa ifane nesitayela ofuna ukusilingisa;

• Gcina nombhalo oqondile onembile walokho okushiwo.

Ngemva kokuqopha, khipha ifayela lomsindo ( wav noma mp3), ulibeke kufolda i-AI ekwideskithophu, uliqambe ngokuthi: ref.wav. Bhala phansi ngokoqobo wonke amagama ashiwo ekurekhodweni, ngoba uzowafaka ekucushweni kamuva. Qaphela ukuthi nemizwa iyalingiswa, ngakho ngesikhathi uqopha ungasebenzisa iphimbo olifunayo.

Ukuze uthole umsindo oyisibonelo, landa “iphakheji yokufaka ngokuchofoza kanye”. Le mibhalo ifaka indawo yengxoxo yezwi, iqalise izinsiza ezihlukahlukene futhi icubungule ukuvumelanisa komsindo nesithombe. Njengoba imibhalo ingase ishintshwe izinguqulo zephrojekthi, kungcono uhlole ngokushesha amakheli okulanda, izimbobo namagama amamodeli ngaphambi kokuyisebenzisa. Amafayela afaka:

• Ifayela｜Umsebenzi

• install-voice.sh｜Ukufaka ingxenye yengxoxo yezwi ngokuchofoza kanye

• start-voice.sh｜Ukuqalisa / ukumisa / ukubuka izingodo zesevisi yezwi

• start-livetalking.sh｜Ukuqalisa / ukumisa isevisi yokunyakaza kwezindebe

• avatar-sync.js｜Ukudlulisa umsindo + ungqimba lwangemuva lomuntu wedijithali

• wav2lip256.pth｜Izisindo zemodeli yokunyakaza kwezindebe

• wav2lip256_avatar1.tar.gz｜Isibonelo esisemthethweni somuntu wedijithali (sokuqinisekisa uchungechunge lokusebenza)

Ikheli lokulanda iphakheji yokufaka ngokuchofoza kanye: https://pan.quark.cn/s/d4fc5bfc88eb

7. Hlela amafayela

Ekugcineni, faka ifayela levidiyo yomuntu wedijithali yemizuzwana engu-5, i-idle.mp4, kanye nefayela lomsindo ref.wav ndawonye kufolda i-AI ekwideskithophu. Isakhiwo sokugcina sefolda:

C:\Users\igama_lomsebenzisi\Desktop\AI\
├── install-voice.sh
├── start-voice.sh
├── start-livetalking.sh
├── avatar-sync.js
├── wav2lip256.pth
├── wav2lip256_avatar1.tar.gz
├── idle.mp4
└── ref.wav

8. Faka i-WSL2 (Ubuntu 24.04 nokusekelwa kwe-GPU)

Vula i-PowerShell njengomlawuli:

wsl --update
wsl --install -d Ubuntu-24.04

Ngemva kokuqeda ukufaka, qalisa kabusha ikhompyutha ukuze ifayela .wslconfig lesinyathelo sokuqala lisebenze. Ngemva kokuqalisa kabusha, vula i-Ubuntu terminal, setha igama lomsebenzisi nephasiwedi, bese uqinisekisa ukuthi i-GPU idluliselwa kahle:

nvidia-smi

Uma ubona igama lekhadi le-GPU nolwazi lomshayeli, kusho ukuthi ukudluliselwa kwe-GPU kusebenza kahle.

Ungaphindi ufake abashayeli be-GPU ngaphakathi kwe-WSL. Umshayeli we-GPU ufakwa ku-Windows kuphela; i-WSL isebenzisa i-CUDA edluliswa yisistimu yokusingatha. Ukuphinda ufake umshayeli we-GPU ngaphakathi kwe-WSL kuzobhala phezu kwe libcuda.so, bese i-GPU ingasatholakali.

9. Kopisha iphakheji yokufaka uyifake ku-WSL

Ku-Ubuntu terminal, shintsha indlela ukuze ifake igama lakho lomsebenzisi le-Windows:

#Kopisha (shintsha indlela ufake igama lakho lomsebenzisi):
mkdir -p ~/setup
cp -r "/mnt/c/Users/igama_lomsebenzisi/Desktop/AI/." ~/setup/
cd ~/setup
ls -la

#Guqula ukuphela kwemigqa:
sudo apt update
sudo apt install -y dos2unix
dos2unix *.sh

#Engeza izimvume zokuqalisa:
chmod +x *.sh

• Kungani kufanele kuguqulwe ukuphela kwemigqa: Amafayela ombhalo e-Windows avame ukusebenzisa i-CRLF ekupheleni kwemigqa; uma kungaguqulwanga, kungavela amaphutha afana ne-bash^M noma i-bad interpreter, futhi abantu abaningi babambeka lapha.

• Ukusebenzisa amafayela amaningi e-Python ngokuqondile ku-/mnt/c kuvame ukuhamba kancane, futhi kungaholela ezinkingeni zezimvume. Ukuwakopisha kumkhombandlela wasekhaya we-Linux kuzinza kakhulu.

10. Faka futhi uhlole imojuli yengxoxo yezwi

1. Sebenzisa iskripthi sokufaka

cd ~/setup
./install-voice.sh

Inqubo yokufaka izokwakha indawo ye-Python futhi ilungiselele ukutholwa komsebenzi wezwi, i-faster-whisper, i-Qwen3-TTS nezingxenye ze-frontend. Kuye ngesivinini senethiwekhi nediski, kungathatha imizuzu eyishumi nambili noma ngaphezulu.

2. Hlanganisa ifomethi yomsindo oyireferensi

ffmpeg -i ~/setup/ref.wav -ac 1 -ar 16000 -c:a pcm_s16le ~/s2s/ref.wav

#Qinisekisa ifomethi: kufanele kube pcm_s16le / 16000 / isiteshi esingu-1:
ffprobe -v error \
  -show_entries stream=sample_rate,channels,codec_name \
  -show_entries format=duration \
  -of default=nw=1 ~/s2s/ref.wav

Imiphumela yokuhlola kufanele ihlangabezane nalokhu:

• Ukubhalwa kwekhodi: pcm_s16le

• Izinga lokusampula: 16000

• Isiteshi: mono

3. Bhala umbhalo ogcwele womsindo oyireferensi

Shintsha umbhalo ongezansi uwenze okuqukethwe okushiwo ngempela ekurekhodweni:

sed -i 's|^REF_TEXT=.*|REF_TEXT="Faka lapha umbhalo ogcwele ofana ncamashi nokurekhodiwe"|' ~/setup/start-voice.sh
grep -n "^REF_TEXT=" ~/setup/start-voice.sh

Uma umbhalo ogcwele unamagama ashiyiwe, amaphutha okupela noma umehluko omkhulu wezimpawu zokuloba, umphumela wokukopisha izwi uzokwehla.

4. Ukuqalisa kokuqala

cd ~/setup
./start-voice.sh

Bese uvula lokhu kusiphequluli se-Windows:

http://localhost:7860/

Vumela isiphequluli sisebenzise imakrofoni, bese uchofoza inkinobho yokusebenzisana ekhasini ukuze uhlole. Ekuqaleni kunconywa ukuthi ungene kuzilungiselelo, wehlisele i-NOISE GATE kwesokunxele kakhulu noma uyivale ngokuphelele; uma kungenjalo, ngisho nokukhuluma okuvamile kungahlungwa njengomsindo wangemuva.

11. Faka i-LiveTalking (ukuvumelanisa ukunyakaza kwezindebe) ne-Wav2Lip

Vula enye i-terminal ye Ubuntu, ungene ku-WSL, bese usebenzisa lokhu ngokulandelana:

mkdir -p ~/livetalking
cd ~/livetalking
git clone https://github.com/lipku/LiveTalking.git
cd LiveTalking

uv venv --python 3.10 .venv-lt
source .venv-lt/bin/activate

uv pip install torch==2.5.0 torchvision==0.20.0 torchaudio==2.5.0 \
  --index-url https://download.pytorch.org/whl/cu124

uv pip install -r requirements.txt
sudo apt install -y ffmpeg libgl1 libglib2.0-0

Uma i-uv ingakafakwa ohlelweni, qala ulandele imiyalelo esemthethweni yokufaka i-uv, bese uphinda usebenzise imiyalo engenhla.

Beka imodeli yokunyakaza kwezindebe ne-avatar yedijithali ezenzakalelayo:

cd ~/livetalking/LiveTalking

# Qaphela: kufanele iqanjwe kabusha ibe: wav2lip.pth
cp ~/setup/wav2lip256.pth models/wav2lip.pth

# Isithombe esiyisibonelo esisemthethweni
tar -xzf ~/setup/wav2lip256_avatar1.tar.gz -C data/avatars/

# Qinisekisa
ls -lh models/ && ls data/avatars/

Qaphela: nakuba ifayela lomthombo libizwa nge-wav2lip256.pth, ngemva kokulikopisha kufanele libizwe nge-wav2lip.pth, ngoba iphrojekthi ifuna izisindo isebenzisa leli gama lefayela ngokuzenzakalelayo.

Qala uhlole i-avatar yedijithali eyisibonelo esemthethweni:

cd ~/livetalking/LiveTalking
source .venv-lt/bin/activate

python app.py --transport webrtc --model wav2lip \
  --avatar_id wav2lip256_avatar1 \
  --stun 'stun:stun.l.google.com:19302'

Vula kusiphequluli:

http://localhost:8010/index.html

Ngemva kokuxhuma ngempumelelo, thumela umbhalo wesiShayina. Uma kuvela ngesikhathi esisodwa umsindo, ukunyakaza komuntu nokuvumelana kwezindebe, lesi sigaba sesisebenza ngokuphelele.

I-inferfps ne-finalfps kumalogi e-terminal kungasetshenziswa ukubheka ukusebenza kwesikhathi sangempela. Ngokuvamile kufunwa ukuthi zombili zifinyelele cishe ku-25 noma ngaphezulu; uma i-inferfps iphansi kakhulu, kuzodingeka wehlise umthwalo wemodeli noma usebenzise ukucushwa okulula.

Izifundiso eziku-inthanethi zivame ukuncoma i MuseTalk. Nakuba ikhwalithi yesithombe ingcono, izidingo zesithombe sale avatar ye-AI aziphezulu kakhulu, kanti ukusetshenziswa kwememori yevidiyo kwayo kungaba cishe ngu-1.3GB kuphela. Ezithombeni ezinokukhanya okuncane, ubuso obuseceleni noma umuntu ongathathi ingxenye enkulu yesithombe, singathola umphumela owamukelekayo ngezindleko eziphansi; i MuseTalk yona isebenzisa imemori yevidiyo engu-12G. Ngaphezu kwalokho, ukufakwa kwendawo kuphela akudingi i-stun, kodwa ungathumeli iyunithi yezinhlamvu engenalutho, ngoba i-WebRTC ingase yehluleke ukuqalisa ngenxa yefomethi engalungile yekheli le-STUN.

12. Guqula ividiyo ibe yi-avatar yakho yedijithali

Ngesikhathi i-LiveTalking isebenza, vula:

http://localhost:8010/avatar.html

Thumela umsebenzi wokukhiqiza ngale ndlela:

• I-algorithm: Wav2Lip

• I-Avatar ID: wav2lip256_myavatar (gcina isiqalo se wav2lip256_)

• Ifayela levidiyo: idle.mp4 ekhiqizwe ngaphambilini

Linda kuze kube isimo siba yi-COMPLETED, bese uqala i-avatar yakho:

cd ~/setup
AVATAR_ID=wav2lip256_myavatar ./start-livetalking.sh

Uma ufuna ukuthi iqale ngokuzenzakalelayo ngokuzayo, shintsha i-Avatar ID ekuskripthini:

sed -i 's/wav2lip256_lingdu/wav2lip256_myavatar/' ~/setup/start-livetalking.sh

Uma ngemva kokukhiqiza kuvela ukuklebhuka okusobala eduze komlomo, qala uhlole ukuthi ividiyo yokulinda ayinazo yini izandla, izinwele noma ikhola evame ukuvala umlomo nesilevu, kunokuba ushintshe imodeli ngokushesha.

13. Qalisa lonke uhlelo ngokulandelana okufanele

Lonke lolu hlelo ludinga amafasitela amathathu e-terminal.

Iwindi lokuqala: imodeli enkulu yendawo ye-Windows

cd /d E:\llama.cpp
llama-server -m E:\llama.cpp\models\Qwen3-14B-Instruct-Q4_K_M.gguf ^
  -np 1 -c 8192 -fa on --temp 1.0 --top-p 0.95 ^
  --host 0.0.0.0 --port 8090

 Iwindi lesibili: isevisi yokunyakaza kwezindebe ye-WSL

cd ~/setup
./start-livetalking.sh

Iwindi lesithathu: isevisi yezwi ye-WSL

cd ~/setup
./start-voice.sh

Ekugcineni vula i-http://localhost:7860/, futhi uma kudingeka cindezela u-Ctrl + F5 ukuze uphoqelele ukuvuselela. Chofoza kuqala ekhasini ukuze i-avatar yedijithali ixhume, bese uchofoza inkinobho yokusebenzisana ngezwi ukuze uqale ukukhuluma.

Lapho uvala amasevisi, sebenzisa:

./start-voice.sh stop
./start-livetalking.sh stop

I-llama.cpp ku-Windows ingamiswa ngokuqondile ngokucindezela u-Ctrl + C.

14. Lungisa ubuntu bemvelo obunokubambezeleka okuphansi

Ku-Settings → INSTRUCTIONS ekhasini lewebhu, faka ubuntu. Ungagcini ngokunqwabelanisa izichasiso ezifana nokuthi “umnene, unomdlandla, uyazwelana”; kungcono futhi unikeze izibonelo ezimbalwa zezingxoxo ezimfushane ukuze imodeli ifunde isigqi sokuphendula.

Ungasebenzisa imikhawulo efana nale:

Ungumngane obonakalayo wemvelo nonobungane. Khuluma ngendlela yansuku zonke, uphendule ngemisho emibili kuya kwemithathu kuphela ngesikhathi.
Ungasebenzisi i-Markdown, izinhlu, izinombolo, izimpawu zemizwa noma izincazelo zezenzo zasesiteji.
Ungawuphindi umbuzo womsebenzisi; phendula ngokuqondile. Uma ungazi kahle ngolwazi oluthile, yisho ngobuqotho.

Isibonelo:
Umsebenzisi: Namuhla ngikhathele kancane.
Umsizi: Khona-ke ungaziphoqi ukuthi usebenze ngamandla aphelele okwamanje. Phuza amanzi, uphumule imizuzu eyishumi, bese sibhekana nakho kancane kancane.

Izimpendulo ezimfushane azibukeki nje njengengxoxo yangempela kuphela, kodwa futhi zinciphisa kakhulu isikhathi sokulinda ukuze kuqedwe ukuhlanganiswa kwezwi. Ukuvala i-Markdown nezimpawu zemizwa kwenzelwa ukuvimbela i-TTS ekufundeni izimpawu noma omaka befomethi. I-/no_think ivala ukukhishwa kokucabanga okude okungadingekile, iphinde inciphise ukubambezeleka kwemisho yokuqala.

15. Nciphisa ukuphazamisa nokunqanyulwa ngephutha

Into evame ukuba nzima kakhulu ukuyilungisa ohlelweni lwezwi lwesikhathi sangempela akuyona imodeli, kodwa ukuthi “umsebenzisi uqedile nini ukukhuluma”. Enkulumweni yesiShayina kuvame ukuba nokuma phakathi kwemisho; uma ukuhlola ukuthula kufushane kakhulu, uhlelo luzophazamisa njalo.

Ku-~/setup/start-voice.sh ungazama le mingcele:

MIN_SILENCE_MS=1200 
VAD_THRESH=0.6
MIN_SPEECH_MS=500
SPEECH_PAD_MS=300

Incazelo yemingcele:

• MIN_SILENCE_MS: isikhathi sokuthula ngaphambi kokunquma ukuthi umusho uphelile; qala ngo-1200ms, bese wehlisela ku-900ms uma impendulo ibonakala ihamba kancane;

• VAD_THRESH: umkhawulo womsebenzi wezwi; endaweni enomsindo ungawukhuphula uye ku-0.7;

• MIN_SPEECH_MS: ihlunga imisindo emifushane kakhulu;

• SPEECH_PAD_MS: igcina isikhala esincane ngaphambi nangemva kwezwi ukuze kuncishiswe ukugwinywa kwamagama.

Bheka amalogi:

tail -f ~/s2s/logs/s2s.log | grep -E "soft-ended|discard"

Uma ukuziphatha kwamalogi kungashintshi nhlobo ngemva kokuguqula imingcele, ngokuvamile akusho ukuthi imingcele ayisebenzi, kodwa ukuthi isevisi ayiqaliswanga kabusha ngempela.

16. Yini okufanele uyenze uma imemori yevidiyo inganele

Ngokwezinzuzo ezinkulu kuya kwezincane, zama lokhu ngokulandelana:

• Shintsha i-Qwen3-14B ibe yi-Qwen3-8B, okungonga cishe u-3GB;

• Shintsha ibe yi-Qwen3-4B, okungonga cishe u-6GB;

• Shintsha imodeli yokuqaphela izwi ibe yi-`medium`, okungonga cishe u-1.5GB;

• Yehlisa umongo we-llama.cpp usuke ku-`-c 8192` uye ku-`-c 4096`, okungonga cishe enye i-1GB;

• Gcina i-Wav2Lip; ungayishintshi ibe yimodeli esindayo yokunyakaza kwezindebe lapho imemori yevidiyo incane.

Ingxoxo yezwi ngokuvamile iba nemisho emibili noma emithathu kuphela emzuliswaneni ngamunye. Umehluko “ekuzwakaleni” phakathi kwe-14B ne-8B ungase ungabi mkhulu kunomehluko wesipiliyoni obangelwa isikhathi sokulinda. Uma imemori yevidiyo incane, beka phambili ukuzinza nokubambezeleka okuphansi.

17. Ukubheka ngokushesha izinkinga ezivamile

• Uphawu｜Isizathu nendlela yokululungisa

• bad interpreter noma kuvela ^M｜Iskripthi sisebenzisa imigqa emisha ye-Windows, sebenzisa `dos2unix *.sh

• I-WebRTC ayikwazi ukuxhuma njalo｜Hlola ukuthi `.wslconfig` ine-`hostAddressLoopback=true`, bese usebenzisa `wsl --shutdown`

• Ikhasi lewebhu libonisa ukuthi ukuhlaziywa kwe-JSON kwehlulekile｜Ngokuvamile iseva yangemuva ibuyise iphutha elingu-500; hlola iphutha langempela kutheminali yokuqalisa

• malformed/uri:invalid scheme｜Ikheli le-STUN alinalutho noma alifomethwanga kahle, faka ikheli elisebenzayo le-`stun:`

• Ikhasi lewebhu liyavuleka kodwa akukho ukusabela lapho ukhuluma｜Vala noma wehlise i-NOISE GATE, uqinisekise ukuthi isiphequluli sinemvume yemakrofoni

• Umuntu wedijithali uhlala ephazamisa inkulumo｜Khuphula i-`MIN_SILENCE_MS`, kunconywa ukuqala ku-1200ms

• Izimpendulo azihambisani nemibuzo｜Qala ngokulungisa ukuhlukaniswa kwamazwi okungalungile, futhi uqinisekise ukuthi azikho izinqubo zokubhala bukhoma eziphikisanayo ezisebenza ngesikhathi esisodwa

• Umehluko womsindo mkhulu njalo｜Hlola ukuthi kusetshenziswa yini imodeli ye-TTS Base, futhi uqinisekise umsindo oyisithenjwa nombhalo oqondile

• Umsindo uzwakala sengathi uguquliwe iphimbo｜Hlola izilungiselelo zesilinganiso sokusampula ku-`avatar-sync.js`; ngokuvamile kufanele kuhambisane nomsindo ongu-16000Hz

• Ukubopha imbobo kwehlulekile kodwa akukho nqubo etholakalayo｜Hlola ububanzi bezimbobo ezigcinelwe i-Hyper-V, bese usebenzisa enye imbobo engagciniwe

• I-GPU igcwele kodwa akukho nqubekelaphambili isikhathi eside｜Kungenzeka ukuthi imemori yevidiyo ibuyela kumemori yesistimu; yehlisa imodeli, noma hlola isu lokubuyela kumemori yesistimu ye-CUDA ku-NVIDIA Control Panel

• Ayikho imakrofoni lapho kusetshenziswa i-IP yenethiwekhi yendawo｜Iziphequluli ngokuvamile zidinga i-localhost noma i-HTTPS ukuze zivumele ukufinyelela kumakrofoni

• Isevisi iyahluleka ngemva kokuvuka kwekhompyutha｜Sebenzisa `wsl --shutdown`, bese uqala kabusha izinsizakalo ezintathu

• Ukuqalisa futhi kubonisa ukuthi imbobo iyasetshenziswa｜Qala ngokusebenzisa imiyalo emibili ye-`stop`, uqinisekise ukuthi izinqubo ezindala ziphelile ngaphambi kokuqalisa kabusha

十八、Ongakukhetha: yenza iskripthi sokuqalisa ngokuchofoza okukodwa

Ngemva kokuqinisekisa ukuthi izinsizakalo ezintathu zisebenza ngokuzimela nangokuzinza, ungakha i-start-ai-avatar.bat entsha ku-Windows:

@echo off
start "llama-server" cmd /k "cd /d E:\llama.cpp && llama-server -m E:\llama.cpp\models\Qwen3-14B-Instruct-Q4_K_M.gguf -np 1 -c 8192 -fa on --temp 1.0 --host 0.0.0.0 --port 8090"
timeout /t 25

start "livetalking" wsl -d Ubuntu-24.04 -- bash -lc "cd ~/setup && ./start-livetalking.sh"
timeout /t 30

start "voice" wsl -d Ubuntu-24.04 -- bash -lc "cd ~/setup && ./start-voice.sh"
timeout /t 10

start http://localhost:7860/

Isikhathi sokulinda kufanele silungiswe ngokwamandla ekhompyutha yakho. Ekufakweni kokuqala, unganciki ngokuqondile kuskripthi se-batch, ngoba lapho isinyathelo esithile sihluleka kuba nzima ukubona ukuthi iyiphi isevisi ebangele inkinga.

十九、Izinkinga zokuphepha zokufinyelela ngomakhalekhukhwini nokuvula ukufinyelela ku-inthanethi yangaphandle

Imakrofoni yesiphequluli iyimvume ebucayi. Lapho uvula ikhasi usebenzisa i-IP evamile yenethiwekhi yendawo, isiphequluli singase sinqabe imvume ngoba ikhasi alikho kumongo ovikelekile. Ngokobuchwepheshe ungasebenzisa amathuluzi afana ne-Cloudflare Tunnel ukuze uguqule ikhasi lendawo libe ikheli le-HTTPS, isibonelo:

cloudflared tunnel --url http://localhost:7860

Kodwa lokhu kusho nokuthi izinsizakalo zendawo zingafinyelelwa yi-inthanethi. Ungasabalalisi esidlangalaleni ikheli lesikhashana elingenakho ukuqinisekiswa kobunikazi, futhi ungavezi ngokuqondile isevisi enemisindo eyimfihlo, amarekhodi ezingxoxo nezithombe zekhamera. Uma kudingeka ukusetshenziswa ukude, kufanele okungenani wengeze ukulawulwa kokufinyelela, bese uvala i-tunnel ngokushesha ngemva kokuyisebenzisa.

二十、Indlela yokwahlulela ukuthi ukufakwa kuphumelele ngempela

Ungasebenzisi kuphela ukuthi “ikhasi lewebhu liyavuleka” njengendinganiso yokuqeda. Kunconywa ukwamukela ngokulandelana okulandelayo:

• 1. Imbobo engu-8090 ye-llama.cpp ikwazi ukukhiqiza izimpendulo zesiShayina ngokujwayelekile;

• 2. Ikhasi lewebhu lezwi liyakwazi ukubona okokufaka kwemakrofoni bese libuyisa izwi elihlanganisiwe;

• 3. Umuntu oyisibonelo we-LiveTalking uyakwazi ukukhuluma futhi akhiqize ukunyakaza kwezindebe okuhambisanayo;

• 4. Ividiyo yokulinda oyenze ngokwezifiso ingaguqulwa ngempumelelo ibe yi-Avatar;

• 5. Ngemva kokuxhumanisa izinsizakalo ezintathu, ukulinda umusho wokuqala kuba cishe imizuzwana engu-1–2, futhi ezingxoxweni eziqhubekayo akukho ukuphazamisa inkulumo njalo;

• 6. Lapho isebenza isikhathi eside, imemori yevidiyo ayikhuphuki ngokuqhubekayo, futhi imiyalo yokuqalisa kabusha neyokumisa izinsizakalo isebenza kahle.

Ukubambezeleka kwempendulo eyodwa ngokuvamile kuvela ekuhlanganisweni kwezigaba eziningi: ukubona inkulumo, i-token yokuqala yemodeli yolimi, ukuhlanganisa izwi, ukucubungula ukunyakaza kwezindebe, kanye nokulinda ibhafa yezwi eliphelele. Ukuze uthuthukise ulwazi lokusebenzisa, qala ngokusebenzisa amalogi ukuze uthole ukuthi yisiphi isigaba esihamba kancane kakhulu, kunokushintsha wonke amamodeli ngokungacabangi.

Isiphetho:

Into ethakazelisa ngempela ngale ndlela akukhona nje “ukwenza isithombe sikhulume”, kodwa ukuxhumanisa amamojuli ambalwa e-AI asebenza endaweni ukuze kwakheke uchungechunge oluphelele lokuxhumana ngesikhathi sangempela. Ingase ibe umngane obonakalayo, futhi ingaguqulwa ibe umchazi wasemnyuziyamu, umsizi wezwi wedeskithophu, umuntu wedijithali osakazayo noma isibonelo sokuqala somlingiswa.

Ekufakweni kokuqala, isu elizinzile kakhulu ukuqinisekisa isigaba ngasinye: qala ngemodeli yolimi, bese uhlola izwi, ulandele ngokuhlola ukunyakaza kwezindebe, bese kuphela ushintshela kumuntu owenzile ngokwezifiso. Uma isinyathelo ngasinye sinendinganiso ecacile yempumelelo, ngisho nesistimu eyinkimbinkimbi ingahlukaniswa ibe iqoqo lezinkinga ezincane ezingatholwa futhi ziphindwe kalula.

Okokugcina, khumbula futhi: isithombe nezwi lomuntu wedijithali kungase kuthinte amalungelo omuntu. Ngaphambi kokukusebenzisa ekuboniseni esidlangalaleni noma kwezentengiso, qiniseka ukuthi uthola imvume yomnikazi wamalungelo ezinto ezisetshenzisiwe, futhi umchazele ngokucacile ukuthi lezi zinto zisetshenziswa ohlelweni lwe-AI ukuze kuxhunyanwe futhi kuxoxwe nabantu.