Yakha Abahlukanisi Abasheshayo Bendawo Abangenazo Izibonelo noJeff

Yini uJeff?

UJeff uwumndeni wamamodeli amancane e-Qwen3.5 ne-Gemma 4 aqeqeshwe kabusha, aklanyelwe ukuhlukanisa ngokushesha ngaphandle kwezibonelo. Uhlinzeka ngesimo ngolimi olujwayelekile, uchaze izinketho ezitholakalayo, bese uthola amathuba alinganisiwe enketho ngayinye ngokusebenzisa ukudlulisela phambili okukodwa.

Ngokungafani nemodeli evamile ekhiqizayo, uJeff akakhiqizi umbhalo wohlelo lwakho okufanele luwuhlaziye. Uklanyelwe ukwenza izinqumo eziqondile emisebenzini efana nokudlulisela izicelo zosekelo, ukuthola inhloso, ukulinganisa okuqukethwe, imiyalo yezwi, neminyakazo yemidlalo. Izigaba akudingeki ukuba zivele kudatha yakhe yokuqeqesha, inqobo nje uma uzichaza ngokucacile.

UJeff usebenzisa ifomethi yesicelo efanayo nekaJev, kodwa uyiphrojekthi ezimele futhi akahlangene no-TypeSafe futhi akagunyazwanga yiyo. Ikhodi yakhe yokuqeqesha yakhela phezu kwendlela yomthombo ovulekile ye-AutoJev.

Izici ezibalulekile

• Ukuqagela okusheshayo kwasendaweni: UJeff-Qwen3.5-0.8B uthatha cishe u-22 ms ngesinqumo ngasinye ku-RTX PRO 6000 no-28 ms ku-Apple M4 Max kusetshenziswa i-MLX ebhentshimakhi ebikiwe.
• Imiphumela elinganisiwe: Isinqumo ngasinye sihlanganisa amathuba, inketho ekhethiwe, nokuzethemba.
• Izigaba ezingadingi izibonelo: Izinketho zichazwa ngesikhathi sesicelo, kunokuba zikhawulelwe kusethi yamalebula engaguquki.
• Izinhlobo ezintathu zemibuzo: Sebenzisa imibuzo ethi choice, noul, noma score.
• Ukuhlanganisa izicelo: Imibuzo eminingana ezimele ngesimo esifanayo ingaphendulwa ndawonye.
• Izinhlelo zasendaweni: Sebenzisa i-PyTorch kuma-GPU noma kuma-CPU e-NVIDIA, noma usebenzise i-MLX nemodeli ye-Qwen ku-Apple silicon.
• Ukuzivumelanisa nesizinda: Qeqesha kabusha i-checkpoint kaJeff lapho ukusebenza ngaphandle kwezibonelo kunganele.

Khetha imodeli

UJeff uhlinzeka ngama-checkpoint amathathu emodeli ku-Hugging Face:

• Jeff-Qwen3.5-0.8B: imodeli enamapharamitha ayizigidi eziyizinkulungwane ezingu-0.8 enesisindo esingu-1.7 GB se-16-bit. Iyona enconywayo yokukhetha izinketho ngokushesha.
• Jeff-Qwen3.5-2B: imodeli enamapharamitha ayizigidi eziyizinkulungwane ezingu-2 enesisindo esingu-4.2 GB se-16-bit. Ithola amaphuzu aphezulu ngokujwayelekile kuphaneli yebhentshimakhi ebikiwe, kodwa ihamba kancane futhi iqaphe kakhulu.
• Jeff-Gemma4-E2B: imodeli enamapharamitha asebenzayo ayizigidi eziyizinkulungwane ezingu-2, amapharamitha agciniwe ayizigidi eziyizinkulungwane ezingu-4.6, nesisindo esingu-9.3 GB se-16-bit. I-backend ye-MLX yephrojekthi isekela amamodeli e-Qwen kuphela.

Ekusetshenzisweni kokuqala, qala ngo-Jeff-Qwen3.5-0.8B. Shintshela kwenye i-checkpoint kuphela ngemva kokuyihlola ngezibonelo ezimele uhlelo lwakho.

Faka futhi ulande uJeff

Indawo yokugcina isebenzisa i-uv ukuvumelanisa okuncikile nokusebenzisa imiyalo. Usuka kufolda yephrojekthi, faka okuncikile okuzenzakalelayo bese ulanda i-checkpoint engu-0.8B:

uv sync
uv run hf download mstrasser/Jeff-Qwen3.5-0.8B --local-dir checkpoints/jeff-0.8b

I-checkpoint izobekwa ku-checkpoints/jeff-0.8b. Ungashintsha indawo yokugcina ye-Hugging Face nefolda yendawo uma ufuna ukusebenzisa enye imodeli eshicilelwe.

Sebenzisa nge-PyTorch

Kwi-GPU noma kwi-CPU ye-NVIDIA, qala iseva nge-backend ye-PyTorch ezenzakalelayo:

JEFF_CHECKPOINT=checkpoints/jeff-0.8b PORT=8765 uv run jeff-serve

Sebenzisa ku-Apple silicon nge-MLX

Kwi-Mac ehambisanayo, faka okuncikile okwengeziwe kwe-Mac bese ukhetha i-backend ye-MLX. Ukusebenzisa i-MLX kusebenzela ama-checkpoint e-Qwen:

uv sync --extra mac
JEFF_BACKEND=mlx JEFF_CHECKPOINT=checkpoints/jeff-0.8b PORT=8765 uv run jeff-serve

Yomibili imiyalo yenza isevisi itholakale endaweni ku-port 8765.

Thumela isicelo sakho sokuqala sokuhlukanisa

Isicelo siqukethe i-state echaza isimo kanye nento ethi questions echaza isinqumo esisodwa noma eziningi. Isibonelo esilandelayo sicela uJeff adlulisele umlayezo wekhasimende eqenjini elifanele futhi ngokuzimela alinganise ukuthi ikhasimende lithukuthele yini:

curl -s localhost:8765/v1/systemone \
  -H 'content-type: application/json' \
  -d '{
    "model": "jeff-latest",
    "state": "Refund request: the customer says the parcel arrived crushed and wants their money back.",
    "questions": {
      "route": {
        "type": "choice",
        "instructions": "Which team should handle this?",
        "criteria": {
          "1": "Refunds and payments",
          "2": "Damaged or lost parcels",
          "3": "Account and login problems"
        }
      },
      "angry": {
        "type": "noul",
        "instructions": "Is the customer angry?"
      }
    }
  }'

UJeff uhlola yomibili imibuzo ngesicelo esisodwa. Impendulo ngayinye inikeza amathuba, inketho ekhethiwe, nenani lokuzethemba. Uhlelo lwakho lungasebenzisa impendulo ekhethiwe ngokuqondile noma lusebenzise umkhawulo walo wokuzethemba.

Qonda izinhlobo zemibuzo

Choice

Umbuzo we-choice ukhetha inketho eyodwa kwezifika ku-255. Faka izihlonzi ezimfushane ku-criteria bese usebenzisa amanani ahambisanayo ukuchaza incazelo yezinketho.

"route": {
  "type": "choice",
  "instructions": "Which team should handle this?",
  "criteria": {
    "1": "Refunds and payments",
    "2": "Damaged or lost parcels",
    "3": "Account and login problems"
  }
}

Noul

Umbuzo we-noul umele isinqumo sika-yebo-noma-cha esibuyiselwa njengamathuba. Uwusizo ekuhloleni ukuthi umlayezo uthukuthele, uyaphuthuma, awuphephile, noma uyahambisana yini.

"angry": {
  "type": "noul",
  "instructions": "Ingabe ikhasimende lithukuthele?"
}

Amaphuzu

Umbuzo we-score ubeka impendulo esikalini osichazayo. Wusebenzise lapho uhlelo ludinga ukuhlolwa okunamazinga esikhundleni selebula lesigaba.

Bhala imiyalelo yesinqumo ephumelelayo

Indlela umyalo obhalwe ngayo inomthelela omkhulu kuma-classifier amancane. Le mikhuba elandelayo ibaluleke kakhulu lapho usebenzisa uJeff.

1. Cabanga ngekhodi bese wenza isinqumo ngoJeff. Bala imithetho enqunyiwe, izimo zesikhathi esizayo, nemiphumela yezinyathelo eziningi kuhlelo lwakho. Nikeza uJeff isimo esiwumphumela ukuze enze isinqumo sokugcina.
2. Chaza imiphumela ngokuqondile. Esikhundleni sokucela imodeli ukuthi ibikezele ukuthi imoto izofika emijikeni emibili, sho ukuthi umnyakazo othile uzoholela ekushayweni yimoto.
3. Gcina izincazelo zezinketho zifana. Izinketho kufanele zisebenzise isakhiwo esifanayo nemininingwane eqhathanisekayo. Amagama angahambisani angachema isinqumo.
4. Sebenzisa okhiye bezinketho abafushane. Khetha {"1": "Engagement letter"} esikhundleni sesihlonzi sangaphakathi eside. Okhiye abade bandisa izindleko zokucubungula ngaphandle kokuthuthukisa incazelo yenketho.
5. Hlanganisa imibuzo ezimele. Uma izinqumo eziningana zabelana ngesimo esifanayo, zithumele ngesicelo esisodwa esikhundleni sokuphinda okokufaka.
6. Hlola idatha emelela okwangempela. Imiphumela yokuqhathanisa iwusizo njengomongo, kodwa ayiqinisekisi ukusebenza kokugeleza komsebenzi okuthile.

UJeff uyi-classifier, akayena umhleli wezinhlelo. Usebenza kahle kakhulu lapho ikhodi yakho ilungisa amaqiniso afanele bese imodeli ikhetha phakathi kwezinye izindlela ezichazwe ngokucacile.

Ukusebenza nemikhawulo

Kuma-benchmark omphakathi amahlanu, uJeff-Qwen3.5-0.8B uthole u-79.1, uJeff-Qwen3.5-2B wathola u-83.1, kanti uJeff-Gemma4-E2B wathola u-81.6 ekuhlolweni kwephrojekthi. Umphumela we-Qwen engu-2B wawusondele esibalweni esiphelele esishicilelwe sikaJev esingu-83.0, nakuba izibalo zikaJev ne-AutoJev zikalwe kumasampula ahlukene avela kuma-benchmark afanayo.

Imiphumela iyahluka kakhulu kuye ngomsebenzi. UJeff usebenza kahle ekuhlolweni okuningi kokuhlukanisa nokuxhumanisa, okuhlanganisa i-Financial PhraseBank ne-RAGTruth, kodwa usalele emuva kwezinhlelo ezinkulu kuma-benchmark adinga ukucabanga okujulile njenge-BBH, JudgeBench, kanye nesigaba esinzima se-JevBench.

Izikhathi ezimaphakathi zokucabanga ezibikiwe zokokufaka okunamathokheni acishe abe ngu-200 zazingu-22 ms ku-RTX PRO 6000, 28 ms ku-M4 Max ene-MLX, kanye no-463 ms ku-CPU enemicu engu-32 yemodeli engu-0.8B. Lezi zinombolo zincike kakhulu ku-hardware futhi kufanele ziphinde zikalwe ekusetshenzisweni kwakho.

Zama i-game harness

Indawo yokugcina ikhodi ihlanganisa izivivinyo ze-Doom, Frogger, ne-Pac-Man ezibonisa izinqumo mayelana neminyakazo esemthethweni echazwa ngokuguquguqukayo. Faka okuncikile kwemidlalo ongakukhetha ngaphambi kokuyisebenzisa:

uv sync --extra games

Njengoba iseva kaJeff isivele isebenza, ungaqalisa ukuhlolwa kwe-Doom ne-Frogger okunikeziwe:

uv run python -m jeff.games --game doom --player jeff --criteria situation --url http://127.0.0.1:8765 --video --out runs/games/doom.json
uv run python -m jeff.games --game frogger --player jeff --criteria outcomes --url http://127.0.0.1:8765 --out runs/games/frogger.json

Ungasebenzisa futhi umdlali we-Pac-Man osekelwe emithethweni ukuze uqhathanise:

uv run python -m jeff.games --game pacman --player rule --out runs/games/pacman-rule.json

Lezi zivivinyo ziphinde zibonise umkhawulo obalulekile: amazinga kuma-benchmark awabikezeli ngaso sonke isikhathi ukusebenza emidlalweni esebenzisanayo. Emiphumeleni eshicilelwe, imodeli kaJeff engu-0.8B yadlala imidlalo kangcono kunemodeli kaJeff engu-2B ezimweni eziningana, nakuba imodeli engu-2B yayinamaphuzu aphezulu e-benchmark ehlanganisiwe.

Qeqesha uJeff ngokwezidingo zesizinda sakho

Uma ukunemba kwe-zero-shot kunganele, qala nge-checkpoint kaJeff bese uqeqesha ngezibonelo ezivela ohlelweni lwakho. I-README ibika ukuthi ukuqeqeshwa okwengeziwe kokuzulazula ngezwi, kusetshenziswa cishe izibonelo eziyi-11,000 eziqondene nohlelo, kwakhuphula ukunemba kwedatha yokuhlola ebigcinwe eceleni kusuka ku-31.7% kuya ku-95.8% cishe ngesigamu sehora ku-GPU eyodwa.

autojev-train --initial-checkpoint <jeff> --epochs 1 ...

Indawo yokugcina ikhodi iphinde iveze ukugeleza komsebenzi wayo obanzi wokuqeqesha. Inqubo evamile yakha ingxube yedatha, iqhube ukuqeqesha, bese ihlola i-checkpoint ekhethiwe:

uv run autojev-mix ...
scripts/train.sh RUN data/mix/public.jsonl data/mix 5e-6 40 Qwen/Qwen3.5-0.8B <revision> --epochs 1
uv run autojev-evaluate --data data/panel.jsonl --local --checkpoint checkpoints/RUN/selected --output runs/eval/RUN.json

Iresiphi eshicilelwe isebenzisa ukuqeqeshwa okuphelele kwezisindo isikhathi esisodwa, ama-batch angu-256, i-cross-entropy phezu kwezinhlamvu zezinketho, kanye ne-temperature elungisiwe yokulinganisa amathuba. Ama-checkpoint akhethwa kusetshenziswa idatha yokuthuthukisa esikhundleni sephaneli ye-benchmark.

Ukuze uthole ukugeleza okuphelele kwasendaweni kwedatha yokwenziwa, ukuhlunga ukuvuza, ukuhlolwa kwama-learning rate, kanye nokugeleza komsebenzi kwedeshibhodi, bheka scripts/train_all.sh. Iphrojekthi ibhala imithombo yokuqeqesha namalayisensi yayo ku-docs/data-sources.md.

Amasu okusebenzisa athuthukile

• Linganisa ukulinganiswa kwedatha yakho. Imodeli evamile elinganisiwe ingase isadinge imingcele eqondene nohlelo ukuze kunqunywe ukuzidlulisela ezingeni eliphezulu noma ukuziphatha kokubuyela kwesinye isixazululo.
• Faka isixazululo esiphephile sokubuyela emuva. Uma ukuzethemba kuphansi kakhulu, dlulisela isicelo kumuntu, emthethweni onqunywe kusengaphambili, noma kumodeli enamandla kakhulu.
• Gcina izimo zifushane. Nikeza amaqiniso adingekayo esinqumweni futhi ugweme umbhalo ongahlobene ongaphazamisa imodeli encane.
• Hlola ukuhlukahluka kwamagama. Qhathanisa izindlela eziningana ezingaguquguquki zokubhala imiyalelo nemigomo ngaphambi kokulungisa isikimu sakho sokukhiqiza.
• Khetha amamodeli ngokuhlola. Sebenzisa imodeli engu-0.8B ekukhetheni izinketho lapho isikhathi sokuphendula sibalulekile, kodwa hlola zonke izindawo zokuhlola ezifanele usebenzisa isethi yakho yokuqinisekisa.
• Qaphela imikhawulo yezindlela zokuxhumana. Ngokwe-README, uJeff usekela umbhalo wesiNgisi kuphela.

Isiphetho

UJeff unikeza indlela ephathekayo yokwengeza izinqumo ezisheshayo, zasendaweni, ezisekelwe emathubeni kuhlelo lokusebenza ngaphandle kokukhiqiza noma ukuhlaziya umbhalo okhululekile. Qala ngo-Jeff-Qwen3.5-0.8B, chaza inketho ngayinye ngokucacile, hlanganisa imibuzo ezimele ibe yisicelo esisodwa, futhi uhlole ngezibonelo zangempela zohlelo lokusebenza. Uma ukuziphatha kwe-zero-shot kunganele, lungisa kahle indawo yokuhlola ekhona kaJeff usebenzisa idatha eqondene nesizinda.