Isifundo se-Jev GUI Delegate: Vumela i-Codex Yabe Imisebenzi ye-Windows ne-Chrome GUI

Uhlolojikelele lwephrojekthi

I-Jev GUI Delegate iyisibonelo sokuqala sokudlulisela imisebenzi ye-Windows ne-Chrome GUI ku-Codex. Imodeli eyinhloko inikeza inhloso yomsebenzi nemibandela yokwamukelwa, kuyilapho isilawuli sasendaweni sibheka isibonisi, siqondanise izilawuli, senze izenzo, futhi sihlole umphumela wangempela esinyathelweni ngasinye. Isilawuli sibiza i-TypeSafe Jev kuphela lapho sibhekene nokukhetha okungachazwa ngomqondo okungenakusingathwa ngemithetho enqunyiwe.

Inkinga eyinhloko le phrojekthi efuna ukuyixazulula yile: uma imodeli eyinhloko isebenzisa iwebhu noma uhlelo lwedeskithophu isinyathelo ngesinyathelo, kufanele iphindaphinde ifunde isibonisi, ikhethe izilawuli, futhi icubungule impendulo yamathuluzi, okudala umjikelezo wokusebenza obizayo. I-Jev GUI Delegate idlulisela isenzo esingamukelwa ngokucacile kusilawuli sasendaweni ukuze sisisebenzise ngokuqhubekayo, kuyilapho igcina ukuhlolwa kwesimo, ukubuyiselwa emaphutheni kanye nekhono lomsebenzisi lokuthatha ulawulo.

Le phrojekthi iyisibonelo sokuqala esiklanywe umuntu, sakhiwa ngosizo lwe-AI futhi esiqhubeka sihlolwa. Inkontileka yomsebenzi, ukuhlelwa kwamathuluzi, indlela yokubuyisela, ukuhlolwa kwezimvume, izivivinyo nokuphindaphindwa kwempendulo kungumnikelo wephrojekthi; imodeli ye-Jev, i-TypeSafe SDK, i-Codex kanye ne-plugin yesiphequluli yimikhiqizo yangaphandle.

Ukuhamba komsebenzi

1. Imodeli eyinhloko ye-Codex ithumela inhloso nemibandela yokwamukelwa.

2. Umsebenzi uhlelwa ube yinkontileka yomsebenzi yasendaweni ehlolwayo.

3. Isilawuli sibheka isibonisi samanje bese sizama ukuqondanisa izilawuli ngendlela enqunyiwe.

4. Uma singakwazi ukunquma ngokuqondile, isilawuli sibiza i-Jev ukuze sikhethe ngokomqondo.

5. Imisebenzi ye-Chrome yenziwa ngeseshini yesiphequluli esemthethweni ehambisanayo ehlinzekwa umsingathi, kanti imisebenzi ye-Windows yenziwa nge-UI Automation.

6. Isilawuli siqinisekisa umphumela wangempela, sigcine inqubekelaphambili, bese sinquma ukuthi siqhubeke, sibuyisele, simise okwesikhashana, noma sicele ulwazi olwengeziwe kumodeli eyinhloko.

Le nqubo ayikopishi izilungiselelo zesiphequluli somuntu siqu futhi ayikhiphi ama-Cookie. Indlela ye-Chrome incike ku-API yeseshini yesiphequluli esemthethweni ehambisanayo ehlinzekwa umsingathi we-Codex; indawo yokugcina yona ayikwazi ukudala noma ukufaka esikhundleni sale seshini.

Imisebenzi eyinhloko

Izindlela ezimbili zokusebenzisa i-GUI

• Chrome: Iphinda isebenzise iseshini yesiphequluli esemthethweni ehlinzekwa umsingathi.

• Windows: Isebenzisa i-Windows UI Automation ukusebenzisa izilawuli zedeskithophu ezifinyelelekayo.

Inkontileka yomsebenzi ecacile

Inkontileka yomsebenzi ichaza ngokusobala inhloso, izenzo ezivunyelwe, ububanzi bokusebenza, okokufaka okudingekayo, izinyathelo zokwenza kanye nemibandela yempumelelo. Ngakho isilawuli singahlola izimvume nobubanzi ngaphambi kokwenza, bese sisebenzisa imibandela yempumelelo ukuze sinqume ukuthi umsebenzi uqedwe ngempela yini.

Ukubiza i-Jev lapho kudingeka

Izilawuli ezinganqunywa ngemithetho yasendaweni ziqondaniswa ikhodi ngokuqondile; i-Jev ibizwa kuphela lapho kudingeka ukuqonda incazelo yombhalo noma yomuntu ozokhethwa. Iphrojekthi ibhala ngokuhlukile izimo lapho kungabizwanga modeli, futhi ayizibali ngephutha njengokucabanga kwe-Jev okuphumelele.

Ukwenza ngokuqhubekayo nokubuyisela

Isilawuli singagcina inqubekelaphambili yokwenza futhi siphathe ukuntuleka kokokufaka, amaphutha okuzikhethela ayingxenye, ukumisa okwesikhashana, ukukhansela kanye nokuthatha ulawulo ngumsebenzisi. Uma kungaqinisekwa ukuthi isenzo sokuthumela siphumelele yini, ngeke siphinde sithumele ngokuzenzakalelayo, ukuze sinciphise ingozi yokuthumela kabili.

Ukuhlolwa kwemiphumela nokuphepha

Uhlelo luqinisekisa ubuwena, ukugunyazwa nobubanzi ngaphambi kokusebenza, bese luhlola inani langempela noma isimo sekhasi ngemva kokusebenza. Ukusetshenziswa kwemodeli nezizathu zokwehluleka kubhalwa ngokwehlukana, ukuze kuhlukaniswe izinkinga zezindleko nezinkinga zokwenza.

Indawo yokusebenza nezimfuneko zokuqala

Iphrojekthi ihloselwe lezi zindawo:

• Windows x64

• Python 3.12

• Node.js

• Ideskithophu esebenzisanayo yokusebenzisa i-GUI yomdabu

• Uma kusetshenziswa indlela ye-Chrome, kudingeka futhi kube khona i-API yeseshini yesiphequluli esemthethweni ehambisanayo ehlinzekwa umsingathi we-Codex

Kuphakanyiswa ukuthi iphrojekthi ifakwe ku-C:\jev\jev-bridge. Eminye imiyalo yakudala nezibonelo ze-Codex skill zisebenzisa le ndawo ngokuzenzakalelayo. Uma kukhethwa olunye uhla lwemibhalo, izibonelo nezilungiselelo ezifanele kufanele zishintshwe ngokufanayo.

Ukufaka iphrojekthi

Isinyathelo sokuqala: Klona indawo yokugcina bese udala indawo ebonakalayo

git clone https://github.com/YUTA-fywoo/jev-gui-delegate.git C:\jev\jev-bridge
cd C:\jev\jev-bridge
py -3.12 -m venv .venv

Isinyathelo sesibili: Faka okuncikile okukhiyiwe

.\.venv\Scripts\python.exe -m pip install --require-hashes -r requirements.lock

--require-hashes ifaka okuncikile ngokusekelwe kuma-hashi akufayela lokukhiya, okusiza ukuqinisekisa ukuthi okuqukethwe okufakiwe kuyahambisana nalokho okubhalwe kuphrojekthi. Uhlu lwemibhalo lokuncika kwabanye alushicilelwa kanye nendawo yokugcina; kufanele lufakwe eziteshini zokuqala zokukhishwa.

Isinyathelo sesithathu: Lungisa isikhathi sokusebenza

.\.venv\Scripts\python.exe scripts/setup-runtime.py

Isinyathelo sesine: Sebenzisa izivivinyo ezingaxhunyiwe ku-inthanethi

.\.venv\Scripts\python.exe -m unittest discover -s gui_delegate/tests -p "test_*.py"

Lezi zivivinyo zihlanganisa isilawuli, inqubo yokubuyisela kanye nezimo zemingcele. Ukuhlolwa kokubuyela emuva okungaxhunyiwe ku-inthanethi kusebenzisa idatha yokwenziwa nabalingisi abalawulwayo, akufani nokusebenzisa i-GUI endaweni yangempela, futhi akwenzi sengathi yizicelo zangempela ze-Jev. Ukuze uthole imiphumela ephelele, bheka idokhumenti yokuqinisekisa ukukhishwa esendaweni yokugcina.

Ukulungisa ukhiye we-Jev

Ukhiye unganikezwa okuguquguqukayo kwemvelo kwasendaweni TYPESAFE_API_KEY noma i-Windows Credential Manager. Iphrojekthi iphakamisa ukusebenzisa iwindi lokufaka lendawo elivulwa yi-set-key.cmd ukuze uwulungise.

set-key.cmd

Ungawufaki ukhiye endaweni yokugcina ye-Git, enkontilekeni yomsebenzi noma kokuqukethwe kwengxoxo. I-chrome-runtime.json ekhiqizwa endaweni yomfaki ishiyiwe ngaphandle, futhi indawo yokugcina ayiphethe izindlela zemvelo zomshini wombhali wokuqala.

Ukubhalisa ku-Codex

Uma iphrojekthi kufanele ixhunywe ku-Codex, sebenzisa le mibhalo yokufaka ohlwini lwemibhalo lwephrojekthi:

install.cmd
gui-install.cmd

Isifaki sizoshintsha izilungiselelo ze-MCP, skill kanye nemizila ezingeni lomsebenzisi, sigcine amakhophi ayisipele futhi sihlole ukungqubuzana. Uma uhla lwemibhalo olunegama elifanayo selukhona emshinini, qala uhlole injongo yalo; isifaki ngeke simboze ngokuqondile iphrojekthi esetshenziswa umshini njengamanje.

I-Codex skill yomuntu siqu efakekayo itholakala ku-gui_delegate/skill/SKILL.md. Uma iphrojekthi ingafakwanga ohlwini lwemibhalo oluzenzakalelayo, kufanele kubhekwe nezibonelo ezikhuluma ngezindlela ezikulo.

Ukusetshenziswa okuyisisekelo nokuqinisekisa

Hlola ukulungela kwasendaweni

Ngemva kokufaka ungasebenzisa:

health.cmd

health.cmd ihlola kuphela ukuthi izingxenye zikulungele yini ukusebenza; ayisho ukuthi sekwenziwe ukuqinisekiswa kwangempela kwe-Jev, futhi ayisho ukuthi umshini wamanje usuphumelele ukuhlolwa kwesimo sangempela se-GUI.

Yenza ukuqinisekiswa kwesikhathi sangempela

Izinqubo zokuqinisekisa zesikhathi sangempela ezinikezwa iphrojekthi, ezifana ne-verify.cmd, zibiza isevisi ye-Jev:

verify.cmd

Ngaphambi kokusebenzisa, qinisekisa ukuthi ukhiye ulungiswe kahle, futhi uhlukanise imiphumela yokubuyela emuva engaxhunyiwe ku-inthanethi nemiphumela yesevisi yesikhathi sangempela. I-README ayithathi inkulumo ethi “umshini uqobo uqinisekisiwe” evela endaweni yokuthuthukisa yakudala njengokuphetha okungadluliselwa ngokuqondile kubo bonke abafaki.

Yakha umsebenzi ofanele ukudluliselwa

Lesi sibonelo sokuqala sifanele imisebenzi ye-GUI enemingcele ecacile, izinyathelo ezingakhawulelwa kanye nemiphumela ehlolwayo. Lapho uletha umsebenzi kusilawuli, kufanele okungenani ucacise okulandelayo:

• Inhloso: Isiphi isimo sokugcina okufanele sifinyelelwe.

• Ububanzi: Imaphi amakhasi, amawindi noma izinto lapho kuvunyelwe ukusebenza khona.

• Izenzo ezivunyelwe: Iziphi izinhlobo zokuxhumana isilawuli esingazenza.

• Okokufaka: Idatha edingekayo nevunyelwe ukusetshenziswa ekwenzeni umsebenzi.

• Izinyathelo: Izigaba zemisebenzi ezingasetshenziswa ngokulandelana.

• Imibandela yempumelelo: Inani langempela noma isimo sekhasi okufanele kubonakale ngemva kokuqedwa.

Ngokwesibonelo, umsebenzi ungacela isilawuli ukuthi sifune isilawuli esiqondiswe kuso ngaphakathi kobubanzi bekhasi obushiwo, sigcwalise okokufaka okugunyaziwe, bese siqinisekisa umphumela ngesimo sekhasi. Izinkambu ezithile zenkontileka nemodeli yomphumela kufanele kusekelwe ekusetshenzisweni okukwi-gui_delegate/schema.py; akufanele kucatshangwe ifomethi engachazwanga yi-README.

Indlela yokufunda ikhodi yomthombo

Uma kudingeka ukuqonda indlela yokwenza noma ukuthuthukisa kabusha, funda ngale ndlela elandelayo:

1. gui_delegate/schema.py: Qonda inkontileka yomsebenzi, isimo nemodeli yomphumela.

2. gui_delegate/controller.py: Bheka ukuthi isilawuli sihlela kanjani umjikelezo wokwenza.

3. gui_delegate/drivers.py: Qonda abashayeli nokubhekwa kwesibonisi okuhlelekile.

4. gui_delegate/security.py: Bheka ukuhlolwa kobubanzi, ukugunyazwa nokokufaka.

5. gui_delegate/chrome_gateway.mjs: Qonda indawo yokungena yeseshini yesiphequluli esemthethweni.

6. jev_client.py: Bheka ukugoqa kwe-Jev SDK, ukuhlolwa kwenguqulo nokubhalwa kokusetshenziswa.

7. gui_delegate/tests: Sebenzisa izivivinyo ukuqonda umqondo wokubuyisela nokuziphatha kwemingcele.

Iziphakamiso zokusetshenziswa okuthuthukile

Yenza ukuqondanisa okunqunyiwe kube yinto yokuqala

Uma isilawuli singabonakala ngezici ezizinzile noma ngolwazi oluhlelekile, kufanele kuqala siqondaniswe ikhodi yasendaweni. I-Jev kufanele ibizwe kuphela lapho kudingeka ukwahlulela ngokomqondo; lokhu kuhambisana nokwakhiwa kwephrojekthi futhi kwenza kube lula ukubhala ngokunembile ukusetshenziswa kwemodeli.

Hlela umphumela obonakalayo esinyathelweni ngasinye

Ungachazi izenzo ezinjengokuthi “chofoza inkinobho” kuphela; kufanele uchaze nenani, isimo sesilawuli noma ushintsho lwekhasi olulindelekile ngemva kokuchofoza. Iphrojekthi igcizelela ukuqinisekisa umphumela wangempela, hhayi ukuthatha impumelelo yocingo lwethuluzi njengempumelelo yomsebenzi.

Phatha ngokucophelela izenzo zokuthumela

Uma umphumela ungacaci ngemva kokuthumela, ungacabangi ukuthi kwehlulekile bese uphinda ngokuzenzakalelayo. Le phrojekthi igwema ngokusobala ukuphinda ithumele ngokuzenzakalelayo lapho umphumela ungaziwa. Esimweni esinjalo, kungcono ukumisa, ugcine inqubekelaphambili, bese uvumela umsebenzisi noma imodeli eyinhloko iqinisekise.

Hlukanisa okokufaka okushodayo namaphutha okukhetha

Uma okokufaka okudingekayo kushoda, kufanele kwengezwe bese kuqhubekwe; kuphela uma kunekhethi eliyingxenye elingelona elomqondo lapho kufanele kulungiswe lelo khethi. Lokhu kugwema ukuphinda kuqalwe umsebenzi kusukela ekuqaleni futhi kugcina inqubekelaphambili esivele yaqinisekiswa.

Qinisekisa kabusha ngemva kokuthuthukisa inguqulo ye-Jev

I-settings.json yamanje ibeka i-jev-1.13.0 eyasetshenziswa ekuhlolweni komlando. Uma inguqulo eseceleni kweseva ishintsha, kufanele kuqinisekiswe kabusha futhi kubuyekezwe izilungiselelo; imiphumela yokuhlola yenguqulo endala ayikwazi ukuthathwa njengesiqinisekiso sekhwalithi yenguqulo entsha. Isu lesibonelo nalo alisho ukuthi umshini wamanje usuqedile ukulinganiswa.

Qonda kahle ukuhlolwa kokuqhathanisa

I-README ibhala ukuhlolwa kwemisebenzi yokwenziwa kwemijikelezo emibili kwangomhla ka-23 Septhemba 2026. Umjikelezo ngamunye wawunemisebenzi efanayo engu-4 nezenzo ezingu-24, futhi womabili amaqembu aqeda u-4/4.

• Kumjikelezo wokuqala, ama-token e-Astra ehla ku-2,266,693 aya ku-236,295, ukwehla ngo-89.6%, kanti izinga lomsebenzi kusukela ekuqaleni kuye ekugcineni laba izikhathi ezingu-3.05 kune-baseline.

• Kumjikelezo wesibili, ama-token e-Astra ehla ku-2,256,091 aya ku-192,752, ukwehla ngo-91.5%, kanti izinga lomsebenzi kusukela ekuqaleni kuye ekugcineni laba izikhathi ezingu-3.53 kune-baseline.

Ama-token e-Astra lapha afaka okokufaka okugcinwe kunqolobane, kanti ukusetshenziswa okwengeziwe kwe-Jev kubhalwa ngokwehlukana. Le mijikelezo emibili iphinda imisebenzi emine efanayo, ngakho ayimeleli izimo ezizimele eziyisishiyagalombili; lezi zinombolo nazo azisona isilinganiso sokonga enkokhelweni futhi azikwazi ukufakazela ukuthi zonke izinhlelo zokusebenza, yonke imisebenzi noma zonke izindawo zokufaka zizothola umphumela ofanayo. Imiphumela yama-token angagciniwe kunqolobane yenguqulo yokuqala enemisebenzi emithathu ayizange ibe ngcono; bheka incazelo yokuhlolwa ukuze uthole ububanzi obuningiliziwe.

Imikhawulo eyaziwayo

• Lesi sibonelo sokuqala sincike olwazini lwe-DOM noma lokufinyeleleka olungafundeka.

• I-Canvas, ukuqonda izithombe, ideskithophu evikelekile nezinye izilawuli eziphakathi kwamafreyimu zingase zingacutshungulwa ngokuqondile.

• Imisebenzi umsingathi ongayivulanga ingase idinge ukuthi imodeli eyinhloko iyiphathe.

• Indlela ye-Chrome ayikwazi ukusebenza ngokuzimela ngaphandle kweseshini esemthethweni ehambisanayo yomsingathi.

• Amarekhodi okuqinisekisa omshini wakudala awasho ukuthi indawo entsha yokufaka isiphumelele ukwamukelwa.

• Amalogi okusebenza, izilungiselelo zesiphequluli, okhiye, izinto zomsebenzi womuntu siqu, isimo sokufakwa emshinini namaphakheji okubuyisela akushicilelwanga.

Isifinyezo

I-Jev GUI Delegate ihlukanisa imisebenzi ye-GUI engamukelwa emjikelezweni wesinyathelo ngesinyathelo wemodeli eyinhloko, bese isilawuli sasendaweni senza ukuqondanisa okunqunyiwe, ukukhetha ngokomqondo lapho kudingeka, ukuqinisekisa isimo nokubuyisela emaphutheni. Ngesikhathi sokufaka, kufanele kuqedwe kuqala okuncikile okukhiyeziwe, ukulungiswa kwesikhathi sokusebenza nezivivinyo ezingaxhunyiwe ku-inthanethi, bese kulungiswa ukhiye futhi kuxhunywe ku-Codex. Ekusetshenzisweni kwangempela, ububanzi bomsebenzi obucacile, imibandela yempumelelo ebonakalayo, isu lokubuyisela ngokucophelela ekuthumeleni kanye nokuqinisekisa kabusha ngemva kwezinguquko zenguqulo yizinto ezibalulekile zokuthola imiphumela ethembekile.

Ukuze ufunde kabanzi ngezingxenye ezisetshenziswa phezulu, vakashela imibhalo ye-TypeSafe, i-MCP Python SDK kanye ne-pywinauto.