I-E-Bench – ibhentshimakhi yokuhlola ama-agent ehlakaniphile eyethulwe yi-Tencent Hunyuan nabanye Uyasebenza

I-E-Bench iyibhentshimakhi yokuhlola ukusetshenziswa kwamathuluzi ngezinyathelo eziningi, eyethulwe ithimba le-Tencent Hunyuan ngokubambisana ne-Tsinghua University AIR kanye ne-Southeast University. Isebenzisa imikhiqizo emithathu yangempela, i-Honor of Kings, i-QQ Music kanye ne-Tencent Meeting, njengezibonelo ukwakha indawo ebonakalayo yokwenziwa ngokuphelele, equkethe imisebenzi engu-323 yokushintsha isimo, amathebula esizindalwazi angu-41 kanye nedatha engaphezu kuka-7.6 万. Ngokuklama ukungalingani okubili kwegebe lolwazi kanye negebe lamathuluzi, iphoqa i-agent ukuthi iqoqe ngokuzimela ulwazi olufihliwe futhi ihlele izingcingo zamathuluzi ezinyathelo eziningi. Ekugcineni, isebenzisa umehluko oqinisekile wesimo sesizindalwazi ukuze ikale, iveze imingcele yamakhono ama-agent e-AI ezimweni zemikhiqizo yangempela.

Ukwakhiwa kwezindawo zemikhiqizo zokwenziwa ngokuphelele: Kusetshenziswa ubuchwepheshe bokugcwalisa izizindalwazi obuqondiswa igrafu, kwakhiwa imihlaba emithathu ebonakalayo yemikhiqizo engenawo amarekhodi ahlukanisiwe futhi ehambisana kuwo wonke amathebula.
Ukukhiqizwa kwemisebenzi enamagebe amabili: Imodeli ebeka umbuzo inokufinyelela okuphelele kusizindalwazi, kanti imodeli ehlolwayo ingasebenzisana kuphela ngamathuluzi ebhizinisi anemikhawulo, kwakheka ukungalingani kolwazi namathuluzi.
Ukuhlolwa kwesimo okuqinisekile: Umehluko wesimo sesizindalwazi ngaphambi nangemva kokusebenza usetshenziswa njengendlela yokwahlulela eyodwa; ukudlulisa kubalwa kuphela uma kufanelana ngokuqondile, kugwenywe amaphuzu ayingxenye nokuhluka komahluleli.
Ukuhlaziywa okuhlangene kwezindleko nokusebenza: Izindleko ze-API zomsebenzi ngamunye zirekhodwa ngesikhathi esisodwa, kwakhiwe i-Pareto frontier yokunemba nezindleko ukuze kuhlolwe inani lemodeli uma kuqhathaniswa nezindleko.
Ukwandiswa kokusebenzisa ikhodi: Inguqulo ye-E-Bench-Code ivula ithuluzi le-exec_code, elihlukanisa ubunzima bokuhlela kanye nobokuthola ulwazi.

Ukugcwaliswa kwesizindalwazi okuqondiswa igrafu: Kusukela ku-schema yesizindalwazi esihlobene, kwakhiwa igrafu yokuncika ezingeni lamathebula. Amathebula ayisisekelo namathebula ancike kuwo agcwaliswa ngokulandelana kwe-topology, kusetshenziswa i-LLM enamandla njengomhlanganisi olawulwayo ovunyelwe ukubhala idatha kuphela ngethuluzi lokufaka. Amathebula ancikayo kufanele abuze isizindalwazi samanje ukuze athole izinhlangano ezingakhethwa. Ekugcineni, iskripthi esiqinisekile sihlola futhi silungise amaphutha encazelo njengokulandelana kwesikhathi nokubalwa okuhlanganisiwe, siqinisekise ukuthi indawo iphelele futhi iyazivumelanisa.
Ukungalingani phakathi komkhiqizi nomxazululi: I-Agent ebeka imibuzo inamalungelo akhethekile e-query_sql ne-exec_code, ikwazi ukubuka sonke isizindalwazi futhi yenze izibalo eziyinkimbinkimbi. I-Agent ehlolwayo ingabiza kuphela amathuluzi ezinga lebhizinisi, kwakheka igebe lolwazi (isimo somhlaba wonke sifihliwe) kanye negebe lamathuluzi (amathuluzi okubala angaphakathi asusiwe), okuphoqa imodeli ukuthi ihlole indawo ngokuzimela ngezingcingo zamathuluzi eziningi nezihambisanayo.
Ukuguqulwa kwenhloso nokushintshwa kwemithetho: Amanani athile athathwe kusizindalwazi ashintshwa abe yimithetho ewakhiqizayo. Isibonelo, kuthiwe “engeza zonke izingoma ezisusiwe ohlwini lwezintandokazi” esikhundleni sokunikeza ngokuqondile ama-ID ezingoma, ukuze kuqinisekiswe ukuthi umsebenzi awukwazi ukuqedwa ngaphandle kokusebenzisana nendawo.
Ukukhiqizwa kwemisebenzi ngokuqinisekisa okuphambanayo: Umsebenzi ngamunye ukhiqizwa ngomjikelezo owenziwe njengomkhiqizo othi buka idatha → nquma umgomo → shintsha isimo → fingqa inhloso. Uqinisekiswa ngokuphambana ngamamodeli amathathu ahlukene e-Strong Agent; wamukelwa kuphela uma okungenani amabili ewuphinda ngendlela efanayo, kuqinisekiswe ukuthi umsebenzi uyaxazululeka futhi amalebula anembile.

Ukulungiselela indawo: Thola indawo ebonakalayo yokwenziwa ngokuphelele ye-E-Bench ezinsizeni ezihambisana nephepha locwaningo, bese ulayisha amathempulethi ezizindalwazi nemisebenzi yokuhlola engu-323 ehlanganisa izizinda ezintathu zemikhiqizo, i-Honor of Kings, i-QQ Music kanye ne-Tencent Meeting.
Ukukhetha inguqulo: Khetha i-E-Bench eyisisekelo noma i-E-Bench-Code ngokuya ngenhloso yocwaningo.
Ukuxhuma imodeli: Xhuma i-LLM Agent ehlolwayo kuhlaka lokuhlola, uyilungiselele ukuthi isebenzisane nendawo ngokungaqondile ngamathuluzi esizinda anemikhawulo kuphela, njengokusesha izingoma, ukudala imihlangano nokwengeza abangani, futhi uyivimbele ekubuziseni ngokuqondile isizindalwazi esingaphansi.
Ukulungisa imisebenzi: Setha inani lezikhathi umsebenzi ngamunye ozosebenza ngazo ngokuzimela kanye nesimo sokuqala sekhophi entsha yesizindalwazi, bese uvula ukulandelela izindleko zezingcingo ze-API ukuze imiphumela iphinde ikhiqizwe.
Ukwenza ukuhlola: Qalisa ukuhlola okuzenzakalelayo. I-Agent ithola umyalelo wolimi lwemvelo, bese isebenzisana nendawo ebonakalayo futhi ishintshe isimo ngezingcingo zamathuluzi ezinyathelo eziningi. Uhlelo luqopha ngokuzenzakalelayo wonke umlando wokusebenza kanye nokusetshenziswa kwama-Token.
Ukuhlaziya imiphumela: Ngemva kokuphela kokuhlola, uhlelo lukhipha ukunemba kwe-Pass@1 ne-Pass³ ngokusekelwe kumehluko oqinisekile wesimo sesizindalwazi, futhi lukhiqize umbiko ohlangene wokuhlaziywa kwezindleko nokusebenza ukuze kusizwe ekuhloleni inani lemodeli uma kuqhathaniswa nezindleko.

Ukuphazamiseka okulingana nesikali sangempela: Isizindalwazi ngasinye sigcwaliswa ngemigqa engaphezu kuka-7.6 万 namayunithi edatha angaphezu kuka-60 万. Izinhlangano eziqondiwe zizungezwe amarekhodi amaningi acishe afane, ngakho imodeli ayikwazi ukuqagela ngokuthembela ekusweleni kwedatha endaweni.
Ukuhlukaniswa nokusetshenziswa kabusha kwendawo nemisebenzi: Indawo yomkhiqizo eyodwa ezivumelanayo ingasekela ukukhiqizwa kwemisebenzi ecishe ibe yikhulu, ngezindleko eziphansi kakhulu ezengeziwe; ungqimba lwendawo luyalawuleka, kanti ungqimba lwemisebenzi lunganwetshwa.
Ukuhlola okuzinzile nokuphindaphindekayo: Indawo yokwenziwa ngokuphelele ayithintwa ukuvela kwezinsizakalo eziku-inthanethi, kanti ukukala ngomehluko oqinisekile kususa ukuchema kokwahlulela okusekelwe encazelweni.
Ukuhlukaniswa kobunzima okucacile: Inguqulo eyisisekelo ihlola ukuhlela izinyathelo eziningi nokuqoqwa kolwazi, kanti inguqulo ye-Code iphinde ihlole ikhono lokudlulisela ngaphandle imisebenzi edinga ukubalwa okuningi ngekhodi.

Iphepha lobuchwepheshe le-arXiv: https://arxiv.org/pdf/2607.23722

Ukuhlolwa kwamakhono e-agent ye-AI: Ihlinzeka nge-LLM Agent ngokuhlolwa okujwayelekile nokulinganisekayo kokusebenzisa amathuluzi ngezinyathelo eziningi, okuhlanganisa ukuthola ulwazi, ukushintsha isimo kanye nokuhlela izinhlangano ezahlukene.
Ukukhetha amamodeli nezinqumo zokwenza umkhiqizo: Ngamagrafu e-Pareto ezindleko nokusebenza, isiza amabhizinisi ukuthi enze ukulinganisa phakathi kokunemba nezindleko ze-API, futhi abone amamodeli anenani elihle uma kuqhathaniswa nezindleko.
Ukuhlanganiswa kwedatha yokuqeqesha ama-agent ehlakaniphile: Isebenza njengomthombo wedatha yokuqeqesha olawulekayo nonwebekayo, osetshenziselwa ukuthuthukisa ngokuqhubekayo amakhono e-Agent ekubizeni amathuluzi ngezinyathelo eziningi nasekuthembekeni.
Ucwaningo lwama-agent athuthukiswe ngekhodi: I-E-Bench-Code isekela ucwaningo ngomngcele wenzuzo yokusebenzisa ikhodi ezinhlotsheni ezahlukene zemisebenzi, njengemisebenzi edinga ukubalwa okuningi uma iqhathaniswa nemisebenzi yokucabanga nokwenza izinqumo.
Ukwandiswa kwamabhentshimakhi ezizindeni ezahlukene: Indlela ingadluliselwa ezinhlelweni eziningi zangemuva zemikhiqizo nasezimweni ze-CLI, iqhubekisele phambili ukusungulwa kwamazinga okuhlola ama-agent ajwayelekile.