PAST-Bench Uyasebenza

i-PAST-Bench wuhlolo oluyisisekelo olwakhiwe ithimba likaMengdi Wang eNyuvesi yasePrinceton, olusetshenziselwa ukuhlola ikhono le-Personal AI Agent lokuzithuthukisa ngokuphindaphindiwe. I-PAST-Bench iqhathanisa ukusebenza emisebenzini ngaphansi kwezimo ezinememori nezingenayo imemori, ukuze inqume ukuthi ulwazi olugcinwe yi-Agent phakathi kwezikhathi ezahlukene luyakuthuthukisa ngempela yini ukuziphatha kwayo okulandelayo. I-PAST-Bench ihlanganisa amakhono amane: imemori, ukuphinda kusetshenziswe izinqubo, ukuqoqwa kolwazi, nokubuyekeza; inezimo ezingu-26 kanye nemijikelezo yemisebenzi engu-204.

Ukuhlola ukuzithuthukisa okuphindaphindwayo: ukuhlola ukuthi ulwazi olugcinwe yi-AI Agent yomuntu siqu phakathi kwezikhathi ezahlukene (inkumbulo, amakhono, umlando wemisebenzi) luyakuthuthukisa yini ngempela ukuziphatha okulandelayo.
Ukuhlukanisa umphumela wokuqoqwa kolwazi: ukuhlukanisa ukuthi “ukwanda kwamaphuzu kubangelwa ukuqoqwa kolwazi” noma ezinye izici ezifana nokuthi “imodeli eyisisekelo iba namandla/ushintsho ku-Prompt/ubunzima bomsebenzi”.
Ukuxilonga amakhono ngezindlela ezine: kuhlanganisa inkumbulo, ukusetshenziswa kabusha kwezinqubo, ukuqoqwa kolwazi, nokubuyekezwa kwesimo, ukuze kutholakale ukuthi yiluphi uhlobo lwekhono lwe-Agent olunenkinga.
Ukuhlaziya imbangela yendlela yokusebenza: akubhekwa amaphuzu okugcina kuphela, kodwa kulandelwa nendlela ephelele ethi “ukugcina→ukubuyisa→ukusebenzisa”, ukuze kunqunywe ukuthi ukuthuthuka kusekelwa yini indlela yangempela.

Ukwakhiwa kokulandelana kwemindeni yemisebenzi: I-Agent yenza imijikelezo eminingi yezingxoxo yomndeni womsebenzi ofanayo ngokulandelana. Izingxoxo zokuqala zinikeza amathuba okudala nokugcina ulwazi, kanti izingxoxo zakamuva zihlola ukuthi lolo lwazi lusetshenziswe kahle yini.
Ukuhlolwa okulawulwayo okubhanqiwe: Kumjikelezo ngamunye wengxoxo yakamuva, kwakhiwa inguqulo yokulawula lapho amandla okuphikelela evaliwe, kuyilapho zonke ezinye izimo zifana ngokuphelele. Umehluko wokuzithuthukisa u-Δ utholakala ngokuthi “amaphuzu anenkumbulo – amaphuzu angenankumbulo”.
Ukulandelela ubufakazi bendlela yokusebenza: Ngesikhathi sokusebenza, kuqoshwa idatha ye-telemetry efana nokubhalwa kwenkumbulo, ukubizwa kwamakhono, ukubuyiswa kwezingxoxo, nokubuyekezwa kwesimo. Kube sekubalwa i-Mechanism-Evidence Score ukuze kuqinisekiswe ukuthi ukuthuthuka kulandela indlela elindelekile.
Ukuhlolwa kwemikhiqizo ephikelelwayo: Kuhlolwa okuqukethwe i-Agent ekulondolozile ngempela, okuhlanganisa okufakiwe kwenkumbulo, amafayela amakhono, nezinkomba zezingxoxo, bese kuhlaziywa ukwakheka kwakho, ukuhambisana nesikhathi, kanye nokusebenziseka kabusha.

Ukulungiselela imvelo: Klona inqolobane ye-PAST-Bench ku-GitHub bese ufaka okuncikile okuyisisekelo nama-adaptha e-Agent.
Ukumisa imodeli: Lungiselela i-API Key yemodeli oyisebenzisayo kokuguquguqukayo kwemvelo.
Ukwakha i-sandbox: Sebenzisa i-past-bench build-image --kind sandbox ukuze wakhe isithombe se-Docker sandbox esidingekayo ekuhloleni.
Ukuqinisekisa ngokushesha: Sebenzisa i-past-bench evolve ukuze usebenzise umndeni womsebenzi owodwa, bese wengeza ipharamitha ye- --compare-no-persistence ukuze wenze i-Smoke Test.
Ukuhlola okuphelele: Sebenzisa yonke imindeni yemisebenzi engu-26 ukuze wenze ukuhlola; umndeni ngamunye wemisebenzi uzisebenzisela ngokuzenzakalelayo izivivinyo ezimbili zokuqhathanisa, “ngokuphikelela” kanye “ngaphandle kokuphikelela”.
Ukuhlaziya imiphumela: Buka i-sequence_comparison.json ohlwini lwe- --trace-dir ukuze uthole amanani e-Δ namaphuzu obufakazi bendlela yokusebenza.
Ukuhlanganisa i-Agent yangokwezifiso: Uma udinga ukuhlola i-Agent yakho, sebenzisa i-adaptha ohlwini lwe-agents/ futhi uqinisekise ukuthi isekela iswishi ye- --compare-no-persistence.

Ikhono langempela lokunikeza imbangela: I-PAST-Bench ikwazi ukuhlukanisa ngokunembile ukuthi ukuthuthuka kubangelwa ukuqongelela kokuhlangenwe nakho, ukuqina kwemodeli uqobo, izinguquko ku-Prompt, noma ukwenziwa lula komsebenzi.
Idizayini yokuhlola eqhathanisayo ehambisanayo: Umndeni ngamunye wemisebenzi unenguqulo eqhathanisayo evala ukuqhubeka kwenkumbulo, kuyilapho zonke ezinye izimo zihlala zifana ngokuphelele, okwenza kuqhathaniswe ngqo imbangela phakathi kokuba nenkumbulo nokungabi nayo.
Ukuxilonga ezingeni lomshini: Kwethulwa i-Mechanism-Evidence Score ukuze kuhlolwe ukuthi i-Agent iphendule kahle yini, kanti futhi kulandelwa yonke indlela ethi “ukugcina → ukubuyisa → ukusebenzisa”, kuhlukaniswe phakathi kokuthi “uphendule kahle ngenhlanhla” nokuthi “uphinde wasebenzisa ngempela ulwazi olutholakele”.
Ukuhlukaniswa kwamakhono abe yizilinganiso ezine: Ukuzithuthukisa okungacacile kuhlukaniswa kube amakhono amane athile: inkumbulo, ukuphinda kusetshenziswe kwenqubo, ukuqoqwa kolwazi, nokubuyekezwa kwesimo, ukuze kubonakale kahle izindawo ezibuthaka.
Ukuthuthukisa okuqhutshwa ukuxilonga: Ngokusekelwe emaphutheni esikhathi sokusebenza avezwe yile benchmark, kwadaleka ngokuqondile uhlaka lwe-Hermes+, okufakazela ukuthi le benchmark ayilona nje ithuluzi lokuhlola, kodwa futhi iyinjini yokuxilonga yokuthuthukisa ukwakheka kwama-Agent.

Inqolobane ye-GitHub：https://github.com/Gen-Verse/PAST-Bench
Iphepha lobuchwepheshe le-arXiv：https://arxiv.org/pdf/2608.04003

Ucwaningo lwezemfundo: Ihlinzeka ngendawo ejwayelekile, ephindaphindekayo yokuhlola ngobuningi ukuzithuthukisa okuphindaphindayo kwama-Agent, futhi isekele ukuqhathaniswa okunobulungiswa kwezakhiwo ezahlukene.
Ukuthuthukiswa kwezinhlaka: Ngokuhlukanisa amakhono abe yizici ezine, ikhomba ngokunembile ubuthakathaka bezinhlaka zama-Agent futhi iqondise ukuthuthukiswa kwezakhiwo, njengokuzalwa kwe-Hermes+.
Ukukhetha amamodeli: Ngaphansi kohlaka olungaguquki, iqhathanisa indlela amamodeli ayisisekelo ahlukene aphinda asebenzise ngayo okuhlangenwe nakho phakathi kwezikhathi zengxoxo, futhi ihlonza ukuthambekela kwamakhono emodeli ngayinye.
Ukuqinisekisa ukugcinwa okuqhubekayo: Ihlola imiphumela yangempela yezakhiwo zememori namasu okusesha ahlukene, igweme ukugcinwa okungasebenzi lapho “okugcinwe kungatholakali” noma “okugcinwe kungasetshenziswa”.
Ukuphindaphinda umkhiqizo: Ihlukanisa ukuthi ukwanda kwamaphuzu kwenguqulo entsha kuvela “ekuqongeleleni okuhlangenwe nakho phakathi kwezikhathi zengxoxo” noma “ekuthuthukeni kwemodeli eyisisekelo”, iqinisekise ukuthi umsebenzi wokugcinwa okuqhubekayo udala inani langempela.