Hlukanisa Imiphumela ye-CLI Endaweni nge-Verdict

Lokho Okwenziwa yi-Verdict

I-Verdict iwukubuka kuqala kocwaningo lwesendlalelo semiphumela esiqondakalayo sama-agent okubhala ikhodi, ama-runner e-CI, nezinye izinhlelo ezisebenzisa amathuluzi. Esikhundleni sokuthatha ikhodi yokuphuma kwenqubo njengobufakazi obuphelele, ihlola umyalo, ikhodi yokuphuma, okukhiphayo okujwayelekile, namaphutha ajwayelekile ndawonye.

Imodeli ibuyisa ilebula elilodwa kwamane: success, failure, blocked, noma unknown. Lo mehluko uwusizo ngoba umyalo ungaphuma ngekhodi engu-zero ungazange wenze izivivinyo, ubuyise impendulo ye-HTTP 500, ulinde i-worker, noma ukhiqize okukhiphayo okungaphelele. Ngokufanayo, ikhodi yokuphuma engeyona i-zero ingabonisa isithiyo sendawo kunokwehluleka kwe-assertion.

Okubalulekile: I-Verdict iyisihlukanisi esinikeza iseluleko. Ayifakazeli ukuthi umsebenzi uqediwe noma ukuthi indawo ishintshwe ngendlela ebihlosiwe. Gcina ukuhlola okunqunywa kusengaphambili nokuqinisekiswa kwesimo okugunyaziwe njengomthombo weqiniso.

Izici Eziyinhloko

• Ukuqagela kwasendaweni: Okokufaka kucutshungulwa ku-CPU yendawo ngaphandle kwezingcingo zenethiwekhi noma i-telemetry.
• Imodeli encane: Imodeli e-quantized icishe ibe ngu-197 KB.
• Isikhathi sokusebenza esingenancika kokunye: Ukuqagela kusebenzisa i-Python standard library futhi akudingi amafayela e-tokenizer, i-GPU, noma okunye ukuncika kwe-Python ngaphandle kwephakheji uqobo.
• Ukuhlukanisa ngokwencazelo: Imodeli ibheka umyalo, ikhodi yokuphuma, i-stdout, ne-stderr esikhundleni sokuthembela kukhodi yokuphuma kuphela.
• Ukuqagela okusheshayo: I-repository ibika isikhathi esimaphakathi sokuqagela esingu-0.54 ms endaweni yayo ebhaliwe yokuhlola ye-Windows 11, AMD64, ne-Python 3.12.
• Izixhumanisi ze-Python ne-CLI: Sebenzisa imodeli kuhlelo lokusebenza noma ngokuqondile ku-shell pipeline.
• Umboniso wesiphequluli: Ikhasi lephrojekthi lisebenzisa izisindo ezifanayo e-quantized esipheqululini ngaphandle kokulayisha amalogi.

I-Verdict isebenzisa ama-3–5-gram ezinhlamvu nama-1–3-gram amagama ahashiwe kanye ne-multiclass logistic regression. Okokufaka kwayo umbhalo we-UTF-8, lapho ukusakaza ngakunye kokukhiphayo kukhawulelwe kuzinhlamvu ezingu-12,288 ngokusikwa okucacile kwasekuqaleni nasekupheleni.

Qonda Amalebula Amane

• success: Okukhiphayo kuqukethe ubufakazi obuhle bokuthi umsebenzi uqediwe.
• failure: Umsebenzi usebenze futhi wabika ukwehluleka okuphathekayo.
• blocked: Izimvume, ukuncika, inethiwekhi, imikhawulo ye-quota, noma indawo kuvimbele ukusebenza.
• unknown: Ubufakazi abukho, abuphelele, buyeqiwe, bunqunyelwe ku-dry run, noma abunqumi ngenye indlela.

Amathuba abuyisiwe avela kumodeli yomugqa ye-multiclass. Akufanele ahunyushwe njengokuzethemba okukalwe kahle ezinqumweni ezibalulekile zokuphepha.

Faka i-Verdict

Clone-a i-repository, ngena kufolda yayo, bese ufaka iphakheji nge-pip:

git clone https://github.com/Balanov/verdict.git
cd verdict
python -m pip install .

Lokhu kufaka iphakheji ye-Python ethi verdict_tiny kanye nesixhumi somugqa womyalo esithi verdict.

Qalisa Ukuqagela Kwakho Kokuqala kwe-Python

Dala i-instance ye-Verdict bese uyinika ubufakazi obufanayo obabonwe yi-tool runner yakho. Lesi sibonelo simele inqubo ye-pytest ephumelele kodwa engazange isebenzise noma yiziphi izivivinyo:

from verdict_tiny import Verdict

model = Verdict()
result = model.predict(
    command="pytest tests/",
    exit_code=0,
    stdout="collected 0 items\nno tests ran in 0.01s",
)
print(result)

Okukhiphayo okubhalwe phansi kwemodeli kuhlukanisa lobu bufakazi njenge-unknown, naphezu kwekhodi yokuphuma engu-zero:

{
  "label": "unknown",
  "confidence": 0.999981,
  "probabilities": {
    "success": 0.000019,
    "failure": 0.0,
    "blocked": 0.0,
    "unknown": 0.999981
  },
  "advisory": true
}

I-label ingumphumela okhethiwe. Into ethi probabilities iveza amaphuzu ekilasi ngalinye, kanti i-advisory igcizelela ukuthi ukuqagela akubona ubufakazi obugunyaziwe.

Sebenzisa i-Verdict Kumugqa Womyalo

Ungadlulisela okukhiphayo komyalo ku-Verdict ngenkathi unikeza umyalo wokuqala nekhodi yawo yokuphuma:

printf 'collected 0 items\nno tests ran\n' | verdict \
  --command 'pytest tests/' --exit-code 0

Ngokokukhiphayo osekuvele kugcinwe kumafayela, sebenzisa --stdout ne---stderr:

verdict \
  --command 'pytest tests/' \
  --exit-code 0 \
  --stdout output.log \
  --stderr error.log

Ungalayisha futhi izisindo endaweni ethile:

verdict \
  --command 'pytest tests/' \
  --exit-code 0 \
  --stdout output.log \
  --stderr error.log \
  --weights /path/to/verdict-q8.bin

Hlanganisa i-Verdict Ngokuphepha

Ukuhlanganisa okusebenzayo kufanele kuphathe i-Verdict njengesignali yesinqumo eyengeziwe, hhayi njengejaji lokugcina. I-coding agent noma i-CI runner ingasebenzisa ilebula ukunquma ukuthi kufanele ihlole obunye ubufakazi, iphinde izame umsebenzi, noma icele ukubuyekezwa komuntu.

1. Qalisa umyalo wokuqala futhi ugcine ikhodi yawo yokuphuma, i-stdout, ne-stderr.
2. Dlulisela bonke ubufakazi obutholakalayo ku-Verdict.
3. Hlola kokubili ilebula eqageliwe namathuba amakilasi.
4. Sebenzisa imingcele eqondene nohlelo lokusebenza ngaphambi kokuthatha izinyathelo ezinemiphumela ebalulekile.
5. Qinisekisa ukuqedwa ngokuhlola okunqunywa kusengaphambili, njengamanani ezivivinyo, amafayela akhiqiziwe, isimo se-API, noma amarekhodi esizindalwazi.

Isibonelo, umphumela othi unknown kufanele ngokuvamile uqalise uphenyo olwengeziwe kunokwenza isimangalo sokuqedwa. Umphumela othi blocked ungase wenze kube nengqondo ukuhlola iziqinisekiso, ukuncika, ukuxhumana, izimvume, noma ama-quota ngaphambi kokuzama futhi. Umphumela othi success usadinga ukuqinisekiswa okugunyaziwe lapho ukunemba kubalulekile.

Amathiphu Athuthukile

Gcina Umongo Ophelele Wokwenziwa

Faka umyalo wangempela, ikhodi yokuphuma, i-stdout, kanye ne-stderr noma nini lapho kungenzeka. Ukushiya imifudlana ingafakwanga kungasusa ubufakazi obuhlukanisa ukwehluleka okuqondile kusivimbeli sendawo yokusebenza noma ekuqalisweni okungacacile.

Cabangela Ukusikwa Kokufakiwe

Umfudlana ngamunye wokuphumayo ukhawulelwe ezinhlamvini ezingu-12,288 kusetshenziswa ukusikwa kwengxenye yokuqala neyokugcina. Lapho amalogi emakhulu kakhulu, qinisekisa ukuthi imilayezo ebalulekile ivela eduze kwesiqalo noma ekugcineni, noma wenze olunye usesho olunqunywe ngokucacile kumalogi aphelele.

Setha Imikhawulo Yesinqumo Eqaphile

Amathuba awawona ukuzethemba okulinganiselwe ngomqondo obalulekile kwezokuphepha. Sebenzisa imikhawulo efanele ukuhamba komsebenzi wakho, bese udlulisela amacala angaqinisekile noma anemiphumela emikhulu ekuhlolweni okunqunywe ngokucacile noma ekubuyekezweni umuntu.

Sebenzisa Izisindo Ezenziwe Ngokwezifiso noma Ezilandiwe

Izisindo ezikhishwe ezilinganiselwe zigcinwe ku-weights/verdict-q8.bin futhi ziyatholakala naku-Hugging Face. I-CLI yamukela enye indlela yesisindo nge---weights.

Zama Umboniso Wesiphequluli

Ikhasi lephrojekthi ye-Verdict lisebenzisa izisindo ezifanayo ezilinganiselwe esipheqululini. Ngokwemibhalo yephrojekthi, amalogi awalayishwa.

Phinda Ukhiqize Ukuqeqeshwa Nokuhlola

Faka izengezo zokuqeqesha nezokuthuthukisa, khiqiza kabusha iqoqo ledatha lokwenziwa, qeqesha imodeli, khipha izisindo ze-int8, hlola i-artifact, bese usebenzisa izivivinyo:

python -m pip install -e '.[train,dev]'
python training/train.py
pytest -q

I-benchmark yomphakathi isebenzisa izibonelo zokwenziwa ezingu-192 ezivela emindenini engu-48 ye-CLI ebingekho ekuqeqeshweni. Ukuqeqeshwa kusebenzisa izibonelo zokwenziwa ezingu-1,440 ezivela emindenini ehlukene engu-60. Imodeli elinganiselwe ebikiwe ithole ukunemba okungu-93.2% kanye ne-macro-F1 engu-93.2%, uma kuqhathaniswa nokunemba okungu-47.9% kanye ne-macro-F1 engu-31.9% kwesisekelo sekhodi yokuphuma.

Iqoqo ledatha lenziwe ngokwenziwa futhi ligxile esiNgisini. Lezi zilinganiso zihlola izifanekiso ezigcinelwe ukuhlolwa, hhayi ukwethembeka emikhondweni yama-agent akhiqizwa ekukhiqizweni, futhi imiphumela yokubambezeleka ayizona izithembiso kuwo wonke ama-hardware.

Yazi Imingcele Yephrojekthi

IVerdict incane ngamabomu. Ihlukanisa ubufakazi obubonwe ekuphumeni kwe-CLI, kodwa ayinqumi ukuthi umsebenzi obanzi uqediwe yini. Futhi ayiqinisekisi ukunemba kwekhodi, ukuphepha, noma ukuba yiqiniso.

Ngakho-ke iphrojekthi isetshenziswa kangcono njengengqimba yencazelo phakathi kokwenziwa kwenqubo eluhlaza kanye nesenzo esilandelayo se-agent. Ingakhomba ubufakazi obuphuthelwa yikhodi yokuphuma, kodwa kufanele igcwalise, ingathathi indawo yezivivinyo, ukuhlolwa kwesimo, imithetho yokuqinisekisa, kanye nokwahlulela komuntu.

Isiphetho

IVerdict inikeza indlela encane, esheshayo, esebenza endaweni yangakini yokuhumusha imiphumela ye-CLI ngokucophelela kunokuhlola okusekelwe kwikhodi yokuphuma kuphela. Yifake njengephakheji ye-Python, unikeze umongo ophelele wokwenziwa, bese usebenzisa amalebula ayo amane ukuqondisa izenzo ezilandelayo. Ukuze kube nokuzishintsha okuthembekile, hlanganisa njalo umphumela wayo oyiseluleko nokuqinisekisa okunqunywe ngokucacile nokugunyaziwe.