Yakha futhi Ulawule Amamodeli Ezinhlayiyana ze-GPU nge-WonderSnap

Yini i-WonderSnap?

I-WonderSnap iwumuzwa we-3D osekelwe esipheqululini oguqula amamodeli akhiqizwa ngokwenqubo abe ngamafu akhazimulayo ezinhlayiya ezifika ku-250,000 eziqhutshwa yi-GPU. Ungaqhwiza iminwe ukuze ubize izinhlayiya, uzihlele zibe izimpawu zezindawo noma izakhiwo zomzimba, uzungezise amamodeli ngesandla, futhi uvule imihlangano ibe ukubukwa okuhlukanisiwe okunelebula.

Lolu hlelo luhlanganisa ukulandelela izandla kudivayisi, ukuqaphela izimpawu zezandla okwenziwe ngokwezifiso, amamodeli akhiqizwa ngokwenqubo, kanye nomdwebi we-WebGL2 obhalwe ngesandla. Alusebenzisi i-Three.js, injini yemidlalo, noma uhlaka lwe-frontend. Ividiyo yekhamera yewebhu ihlala emshinini wakho futhi ayithunyelwa kuseva.

Ikhamera yewebhu iyazikhethela. Konke ukusebenzisana okubalulekile kuyatholakala nangezilawuli ezisesikrinini, ikhibhodi, negundane.

Izici Eziyinhloko

• Ukudweba izinhlayiya nge-GPU: I-physics yezinhlayiya isebenza nge-WebGL2 transform feedback futhi isekela izinhlayiya ezifika ku-250,000.
• Ukulandelela izandla kudivayisi: I-MediaPipe Hand Landmarker ilandelela izandla ezifika kwezimbili ezinamaphuzu ayisikhombisa okuma ngesandla ngasinye.
• Ukuqaphela izimpawu: Abahlukanisi abenziwe ngokwezifiso baqaphela ukuqhwiza, izibhakela, izandla ezivulekile, ukukhomba, ukuncinza, izimpawu zokuthula, ukuzungezisa isandla, nokusondeza ngezandla zombili.
• Amamodeli akhiqizwa ngokwenqubo angu-33: Iqoqo lihlanganisa izimpawu zezindawo, ukwakheka komzimba, ibhayoloji, izinjini, izimoto, nemishini.
• Izingxenye ezisebenzisanayo: Izingxenye ezinamagama zingakhanya, zihlukane zibe ukubukwa okuhlukanisiwe, zibonise amalebula, futhi zikhishwe ukuze zihlolwe eduze.
• Ukwakheka komzimba okugqwayizayo: Inhliziyo ishaya izikhathi ezingu-72 ngomzuzu, kanti amaphaphu aphefumula ngomjikelezo wemizuzwana engu-4.5.
• Amathuluzi okufunda: Imodi yemibuzo ikucela ukuthi ukhombe izingxenye, kanti izici zencazelo nezwi ziyazichaza.
• Ukulawula ngezwi: I-Chrome ne-Edge zingacubungula imiyalo efana nokuthi show me the heart, zoom in, kanye ne-quiz.
• Ukusikwa nokurekhoda: Ungaveza izingxenye ezikhanyayo ezisikiwe futhi ulondoloze indawo njengevidiyo ye-WebM.

Izidingo

Ngaphambi kokufaka i-WonderSnap, qiniseka ukuthi unalokhu okulandelayo:

• Node.js 18 noma entsha.
• I-Chrome noma i-Edge ukuze usebenzise isiphequluli esibhalwe phansi.
• Ikhamera yewebhu uma ufuna izilawuli zezimpawu.
• Isiphequluli nedivayisi esekela i-WebGL2.

Ikhamera kufanele ifinyelelwe ku-localhost noma ekhelini le-HTTPS. Ungaliqalisi uhlelo ngokuchofoza kabili ku-index.html; sebenzisa iseva yendawo efakiwe esikhundleni salokho.

Faka bese Uqalisa i-WonderSnap

1. Klona inqolobane

git clone https://github.com/AkbarSheikh-debug/wondersnap.git
cd wondersnap

2. Faka okuncikile

npm install

Lesi sinyathelo siphinde senze isikhathi sokusebenza sokulandelela izandla sitholakale ku-node_modules. Uma i-MediaPipe ingalokothi ilayishe, qinisekisa ukuthi ukufakwa kuqedwe ngempumelelo.

3. Qalisa iseva yendawo

npm start

Vula http://localhost:5173 ku-Chrome noma ku-Edge. Khetha Start with camera ukuze uvule ukulandelela izandla, noma ukhethe Continue without camera ukuze usebenzise igundane, ikhibhodi, nezilawuli ezisesikrinini.

4. Sebenzisa enye imbobo uma kudingeka

Uma imbobo engu-5173 isivele isetshenziswa, nikeza umyalo wokuqalisa enye imbobo:

npm start -- 8080

Ungabe usuvula http://localhost:8080.

Indlela Eyisisekelo Yokusebenzisa Izimpawu

Ngemva kokunikeza imvume yekhamera, beka isandla sakho lapho ikhamera yewebhu ingasibona kahle khona. Ukusebenzisana okujwayelekile kulandela lezi zinyathelo:

1. Qhwiza iminwe ukuze ubize izinhlayiya. Ukuqhwiza futhi kuqeda imodeli yamanje.
2. Yenza isibhakela ukuze wakhe isimangaliso, isitho, imoto, injini, noma umshini okhethiwe.
3. Phakamisa, jikisela, noma tshekisa isandla sakho ukuze uzungezise futhi utshekise imodeli eyakhiwe.
4. Vula isandla sakho ukuze uhambise isimangaliso uye kwimodeli elandelayo. Ezithweni, ezinjini, nasezimotweni, ukuvula isandla sakho kulawula ukuthi izingxenye ziqhelelana kangakanani; ukusivala kuzibuyisela ndawonye.
5. Khomba ingxenye bese ubamba umunwe wakho phezu kwayo ukuze uyikhethe. Ingxenye iyakhanya futhi ikhadi lolwazi lichaza indima yayo.
6. Qinisa iminwe ukuze udonsele ingxenye ekhethiwe kuwe. Qinisa futhi ukuze uyibuyisele.
7. Sebenzisa izandla zombili bese ushintsha ibanga phakathi kwazo ukuze usondeze noma uhlehlise.
8. Yenza uphawu lokuthula ukuze uye kumodeli elandelayo.

Ukusebenzisa Ikhibhodi Negundane

Uma ungenayo ikhamera yewebhu, noma ufuna ukulawula okunembe kakhudlwana, sebenzisa ikhibhodi negundane. Chofoza ingxenye ukuze uyikhethe bese uhudula indawo ukuze uyizungezise.

• Space: lingisa ukuqhwiza iminwe.
• F, O, kanye no-V: qalisa izenzo zesibhakela esivaliwe, isandla esivulekile, kanye nophawu lokuthula.
• Left Arrow kanye ne-Right Arrow: khetha imodeli yangaphambilini noma elandelayo.
• E, Up Arrow, Down Arrow, noma isondo legundane: lungisa ukubukwa kwezingxenye ezihlukanisiwe.
• +, -, kanye no-0: shintsha noma setha kabusha ukusondeza. Ungasebenzisa futhi u-Control kanye nesondo legundane.
• C: vula noma ucime ikhamera.
• L: bonisa noma fihla amalebula ezingxenye.
• R: vula noma ucime ukuzungeza okuzenzakalelayo.
• G: vula noma ucime ukuzungeza okulawulwa ngesandla.
• X: vula ingxenye ebonisa ngaphakathi. Sebenzisa u-, no-. ukuhambisa indiza yokusika kancane.
• Q: qala imodi yemibuzo.
• M: sebenzisa imiyalo yezwi nezici zokufundelwa phezulu.
• K: rekhoda ividiyo.
• D: dlala idemo eyakhelwe ngaphakathi.
• I: chaza ingxenye ekhethiwe.
• Esc: susa ukukhethwa kwengxenye yamanje.
• H: bonisa usizo.

Hlola Iqoqo Lamamodeli

I-WonderSnap inamamodeli enqubo angu-33 athathwe ezilinganisweni zangempela futhi ahlelwe ngamaqembu ayisithupha:

• Izimangaliso: Izimpawu zomhlaba eziyishumi nanye, okuhlanganisa i-Eiffel Tower, i-Great Pyramid, i-Taj Mahal, i-Big Ben, kanye ne-Sydney Opera House.
• Ukwakheka Komzimba: Amamodeli ayishumi, okuhlanganisa ubuchopho, inhliziyo eshayayo, amaphaphu aphefumulayo, uhlaka lwamathambo, kanye nomzimba womuntu onezingqimba.
• Isayensi Yezinto Eziphilayo: I-DNA ene-double helix engavuleka kanye neseli lesilwane.
• Izinjini: Injini ye-Inline-4, i-HEMI V8 ene-supercharger, injini yendiza ye-turbofan, kanye nezinjini ze-radial ezinamasilinda ayisishiyagalolunye.
• Izimoto: Imoto yezemidlalo, isithuthuthu, indiza yabagibeli, kanye ne-Saturn V enokuhlukaniswa kwezigaba.
• Imishini: Iwashi lesandla elisebenza ngomshini kanye nephakethe lebhethri le-EV elinamaseli, ama-busbar, ukupholisa, kanye nesistimu yokuphatha ibhethri.

Sebenzisa Izici Zezemfundo

Hlola izingxenye ezibhalwe amalebula

Khomba noma uchofoze ingxenye ukuze uyikhethe. Ingxenye ekhethiwe iyakhanya, bese ikhadi layo lichaza umsebenzi wayo. Vula amalebula ngo-L ukuze ubonise imigqa exhumanisa amagama nezingxenye ngazinye.

Qala umbuzo

Cindezela u-Q ukuze ungene kwimodi yemibuzo. I-WonderSnap iveza imiyalo efana nokuthi Thola: Hippocampus. Khomba noma uchofoze ingxenye eceliwe. Umbuzo uqukethe imibuzo emihlanu futhi ukhiqiza amaphuzu.

Lawula indawo ngelizwi

Cindezela u-M ukuze ufinyelele imiyalo yezwi nokufundelwa phezulu. Izibonelo ezisekelwayo zihlanganisa ngibonise inhliziyo, yivule, ikuphi i-atrium yangakwesokudla, sondeza, kanye nokuthi umbuzo. Ukuguqulwa kwenkulumo kube umsindo kubhalwe ukuthi kusebenza ku-Chrome naku-Edge.

Dala ingxenye ebonisa ngaphakathi

Cindezela u-X ukuze uvule ukubuka okusikiwe. Indiza yokusika ingalandela isandla sakho futhi iveze ingxenye esikiwe ekhanyayo. Ngaphandle kokufaka isenzo sesandla, sebenzisa u-, no-. ukuhambisa indiza kancane kancane.

Rekhoda umphumela

Cindezela u-K ukuze urekhode indawo bese uyigcina njengevidiyo ye-WebM.

Lungiselela i-WonderSnap Ngezinketho ze-URL

Izinketho zochungechunge lwemibuzo zikuvumela ukuthi uguqule ukuziphatha kokuqala futhi ulungise ukunikezwa ngaphandle kokuhlela amafayela omthombo.

• ?n=250000: setha inani lezinhlayiya.
• ?model=12: qala ngenombolo yemodeli ethile.
• ?autostart=camera: yeqa isikrini sokuqala bese ucela imodi yekhamera.
• ?autostart=nocamera: yeqa isikrini sokuqala bese usebenza ngaphandle kwekhamera.
• ?trails=0: cisha imizila yezinhlayiya.
• ?dpr=1: phoqa isilinganiso samaphikseli sedivayisi sibe ngu-1, okungasiza kuma-GPU ahamba kancane.

Izinketho zingahlanganiswa. Ngokwesibonelo, leli kheli liqala ngaphandle kwekhamera, lisebenzisa izinhlayiya ezingu-100,000, liphoqe isilinganiso samaphikseli sedivayisi sibe ngu-1, futhi licime imizila:

http://localhost:5173/?autostart=nocamera&n=100000&dpr=1&trails=0

Amathiphu Athuthukile

Thuthukisa ukusebenza kwehadiwe ehamba kancane

Inani lezinhlayiya nesilinganiso samaphikseli sedivayisi kuthinta ngokuqondile umthwalo wokunikeza. Qala ngokucushwa okunconywe ku-README kwezinga eliphansi lamafreyimu:

http://localhost:5173/?n=100000&dpr=1

Uma kudingeka, engeza futhi u-trails=0 ukuze ucime imizila yezinhlayiya.

Sebenzisa izixhumanisi zokuqalisa ezinqunyiwe

Emibonisweni noma ekuhlolweni okuphindaphindekayo, hlanganisa u-model nemodi ye-autostart. Lokhu kugwema ukuzulazula mathupha esikrinini sokuqala nasohlwini lwamamodeli njalo.

Qonda ukwakheka kohlelo

Umjikelezo omkhulu wokunikeza, ukukhetha, ukusondeza, ukubukwa kwezingxenye ezihlukanisiwe, amalebula, imibuzo, ukubuka okusikiwe, nokuziphatha kwedemo kutholakala ku-src/app.js. Ukuhlanganiswa kwe-webcam ne-MediaPipe kutholakala ku-src/hands.js, kanti ukulawulwa ngezwi, ukusekelwa kokufundelwa phezulu, nokuqopha kwenziwa ku-src/features.js.

Uhla lwemibhalo src/gl/ luqukethe ama-shader e-WebGL2 kanye nenjini yokunikeza ye-transform-feedback. Umqondo wezenzo zesandla, umshini wezimo, abalawuli, kanye nokulingana kwe-CPU kwefiziksi yezinhlayiya kutholakala ku-src/logic/. Izinsiza zejiyomethri ezenziwa ngenqubo zingaphansi kwe-src/lib/, kanti izincazelo zamamodeli zihlelwe ku-src/models/.

Qalisa Iqoqo Lokuhlola

I-WonderSnap inezivivinyo ezingu-36 zokuphela-kuya-ekupheleni nezeyunithi ezakhiwe nge-Playwright. Lezi zivivinyo zisebenzisa uhlelo lwangempela ngezandla zokwenziwa newashi elinqunyiwe.

Faka isiphequluli se-Chromium se-Playwright kanye:

npx playwright install chromium

Bese uqalisa izivivinyo:

npm test

I-suite ihlanganisa ukugeleza okuphelele kokuthinta, yonke imodeli, ukubukwa kwezitho ezihlukanisiwe, izilawuli zekhibhodi nesondo, izakhiwo ezisabela kusayizi wesikrini, izithombe ezinyakazayo zokushaya kwenhliziyo nokuphefumula, ukusondeza ngezandla zombili, ukukhetha izingxenye, ukusebenzisana kokuncinza, imibuzo, imiyalo yezwi, imodi yokubuka ngaphakathi, ukuqopha, ukuziphatha kokusebenzisa ikhamera njengenye indlela, izihlukanisi zokuma, izinguquko zesimo, ubuqiniso bamamodeli akhiqizwa ngenqubo, kanye ne-physics ye-GPU. I-shader ye-GPU iphinde iqhathaniswe nozakwabo we-CPU ukuze kufinyelelwe cishe ku-1e-7 kuzo zonke izindlela.

Ukuxazulula Izinkinga

• Ikhamera ayiqali: Vula uhlelo nge-http://localhost:5173 esikhundleni sokuvula i-index.html ngokuqondile, bese uvumela imvume yekhamera lapho ucelwa.
• Ukulandelela izandla akulayishi: Sebenzisa i-npm install. Isikhathi sokusebenza sokulandelela sinikezwa sisuka ku-node_modules.
• Izinga lamafreyimu liphansi: Zama i-http://localhost:5173/?n=100000&dpr=1.
• I-port 5173 isetshenziswa olunye uhlelo: Sebenzisa i-npm start -- 8080 bese uvula i-http://localhost:8080.

Isiphetho

I-WonderSnap ikhombisa ukuthi ukulandelela izandla kudivayisi, izihlukanisi zangokwezifiso zokuthinta, ukumodela okukhiqizwa ngenqubo, kanye ne-physics yezinhlayiya ye-GPU kungakha kanjani isipiliyoni semfundo esisebenzisanayo ngokuphelele esipheqululini. Qala ngokusetha okujwayelekile kwasendaweni, funda ukusebenzisana kokuchofoza, isibhakela, isandla esivulekile, ukukhomba, nokuncinza, bese uhlola ukulungisa i-URL, ukulawula ngezwi, imibuzo, izigaba ezinqamulelayo, ukuqopha, kanye nesuite yokuhlola ezenzakalelayo.