Yakha ama-API Ezinqumo Asekelwe Ekucabangeni ngoJeeves

Yini i-Jeeves?

I-Jeeves iyi-classifier yesitayela se-Jev eyakha uchungechunge lokucabanga ngaphambi kokwenza isinqumo esihlelekile. Yakhiwe nge-Qwen3.5-9B enama-adapter e-LoRA, i-pointer head, amathuba alinganisiwe, kanye ne-block-4 diffusion drafter. I-repository iqukethe ikhodi yemodeli ne-drafter, i-pipeline yokuqeqesha, amathuluzi okuhlola, kanye nedatha yokuqeqesha, yokuthuthukisa, neyokuhlola.

Le phrojekthi yakhelwe izinhlelo ezidinga okungaphezu kokukhiqizwa kombhalo okukhululekile. Uma inikezwa isimo esabiwe kanye nombuzo owodwa noma eminingi, i-Jeeves ibuyisa izimpendulo ezihlelekile namathuba nge-API ehambisana ne-Jev. Imibuzo eminingi ingacutshungulwa esicelweni esisodwa.

Izici eziyinhloko

• Izinhlobo ezintathu zezinqumo: imibuzo ye-binary noul, ye-categorical choice, kanye ne-ordinal score.
• Ukucabanga ngaphambi kwezinqumo: imodeli ingakha uchungechunge lokucabanga bese isebenzisa i-pointer head yayo ukukala izinketho ezitholakalayo.
• Okuphumayo okulinganisiwe: amathuba okugcina ezinketho asebenzisa izinga lokushisa elilinganiswe kusethi yokuthuthukisa.
• I-API ehambisana ne-Jev: amaklayenti e-Jev akhona angathumela ifomethi yesicelo eyinhloko, kuyilapho izinketho eziqondene ne-Jeeves zilawula ukucabanga.
• i-SDK ye-Python: iphakheji efakiwe ye-sdk/ ingena esikhundleni se-typesafe-sdk ye-Jev ngaphandle kwezinguquko ezinkulu.
• Ukwakhiwa nge-diffusion: i-drafter ye-block-4 isheshisa ukukhiqizwa kochungechunge lokucabanga futhi isekela ama-layer e-Gated DeltaNet e-Qwen3.5.
• Ikhodi yokuqeqesha: i-repository iqukethe izingxenye ze-SFT, CISPO, calibration, evaluation, export, kanye nokuqeqeshwa kwe-drafter.

Ngokucabanga kwe-greedy okukhawulelwe kuma-token angu-2,560, i-README ibika ukunemba okungu-0.889 kusethi yayo yokuhlola egcinelwe eceleni nengaphandle kwesizinda, kanye no-0.935 ezintweni ezingu-231 zomphakathi ze-JevBench. Nokho, imiphumela yayo yolwazi isalele emuva kwe-Jev ku-MMLU naku-MMLU-Pro, ngakho ukufaneleka kwebenchmark kuncike emsebenzini.

Okudingekayo

Udinga i-Python 3.12 kanye ne-CUDA GPU. I-kernel ye-FP8 yokusebenzisa imodeli idinga i-Hopper GPU. Sebenzisa imiyalo elandelayo usuka empandeni ye-repository.

Faka okuncikile

pip install -r requirements.txt

Landa ama-weight akhishiwe

Sebenzisa i-Hugging Face CLI ukulanda i-checkpoint eshicilelwe ye-9B kanye ne-block-4 drafter yayo:

hf download PostHog/jeeves --local-dir jeeves-weights

Imodeli ekhishiwe iyatholakala ku-Hugging Face.

Qalisa iseva ye-API

python -m inference.serve --model jeeves-weights --drafter jeeves-weights/drafter_k4.safetensors --port 8009

Lokhu kuqalisa isevisi ehambisana ne-Jev ku-port 8009. Iklayenti le-Python elifakiwe lisebenzisa http://127.0.0.1:8009 ngokuzenzakalela, alidingi ukhiye we-API, futhi lilinda imizuzwana efika kwengu-120 ukuze lithole impendulo.

Thumela isicelo sakho sokuqala sesinqumo

I-endpoint ye-/v1/systemone yamukela i-state, imephu ye-questions, kanye nezilawuli zokuzikhethela zokuqagela. Isicelo esilandelayo sicela i-Jeeves iqondise udaba lwekhasimende, inqume ukuthi ludinga ukukhushulelwa yini, futhi ilinganise ukukhungatheka kwekhasimende.

curl -s localhost:8009/v1/systemone \
  -H 'content-type: application/json' \
  -d '{
    "state": "Izicathulo zifike sekudlule amasonto amabili futhi zingosayizi ongalungile. Ngaphezu kwalokho ngibona izindleko ezimbili ekhadini lami.",
    "questions": {
      "department": {
        "type": "choice",
        "instructions": "Iliphi iqembu okufanele liphathe lokhu?",
        "criteria": {
          "returns": "Ukushintshanisa, ukubuyiselwa kwemali, izinto ezingalungile noma ezilimele",
          "shipping": "Isimo sokulethwa, ukubambezeleka, amaphakheji alahlekile",
          "billing": "Izindleko, ama-invoice, izinkinga zokukhokha"
        }
      },
      "escalate": {
        "type": "noul",
        "instructions": "Ingabe lokhu kudinga ukunakwa komuntu ngokuphuthumayo?"
      },
      "frustration": {
        "type": "score",
        "instructions": "Ikhasimende likhungatheke kangakanani?",
        "criteria": ["Uzolile", "Ukhathele", "Uthukuthele kakhulu"]
      }
    },
    "options": {"max_think": 512}
  }'

Qonda izinhlobo zemibuzo

• choice: ikhetha kwezinye izindlela eziqanjwe amagama futhi ibuyisa ukhiye okhethiwe, amathuba, kanye nokuzethemba.
• noul: imele isinqumo sohlobo lukayebo/cha futhi ibuyisa inani le-noul eliyinombolo.
• score: ihlola uhlu oluhlelekile lwamalebula futhi ibuyisa i-score eyinombolo, i-legend, amathuba, kanye nokuzethemba.

Impendulo iphinde ibe nolwazi lokusetshenziswa, oluhlanganisa izibalo zama-token okokufaka, okukhiphayo, nokucabanga, kanye nesikhathi esiphelele sokuphendula. Imibuzo esesicelweni esisodwa ingacabanga ngokuhambisana.

Sebenzisa i-Jeeves usuka ku-Python

Faka i-SDK efakiwe

pip install ./sdk

Dala imibuzo enohlobo

from jeeves_sdk import Choice, Noul, Score, TypeSafeClient

with TypeSafeClient() as client:
    result = client.system_one(
        state="I was charged twice. Please help.",
        questions={
            "billing": Noul(
                instructions="Is this about billing?"
            ),
            "tone": Choice(
                instructions="What is the tone?",
                criteria={"calm": None, "angry": None},
            ),
            "urgency": Score(
                instructions="How urgent is this?",
                criteria=["can wait", "this week", "today"],
            ),
        },
        max_think=768,
        return_reasoning=True,
    )

    print(result.nouls["billing"].noul)
    print(result.choices["tone"].choice)
    print(result.scores["urgency"].score)
    print(result.reasoning["tone"].text)

TypeSafeClient ihlanganisa amanani abuyisiwe ngokohlobo lombuzo. Ukusetha return_reasoning=True kwenza nombhalo wokucabanga wombuzo ngamunye utholakale. I-README ixwayisa ngokuthi ngesikhathi sokuqeqeshwa akuzange kusetshenziswe umvuzo wokuvumelana kolimi, ngakho-ke amaketanga okucabanga awahlali elula ukuwahumusha.

Xhuma kuseva ehlukile

Iklayenti liqondisa ku-http://127.0.0.1:8009 ngokuzenzakalela. Setha i-environment variable JEEVES_BASE_URL lapho iseva isebenza kwelinye ikheli.

export JEEVES_BASE_URL=http://your-server:8009

Lawula ukucabanga nokubambezeleka

Into yokuzikhethela ethi options iveza izilawuli ezine zokucabanga. Amafulegi ahambisanayo eseva angasungula okuzenzakalelayo okusebenza kuyo yonke iseva.

• think: okuzenzakalelayo ngu-true. Setha ku-false ukuze uphendule ngokuqondile ususelwa ku-prompt ngaphandle kokukhiqiza iketanga lokucabanga.
• max_think: okuzenzakalelayo ngu-2560. Inqamula iketanga lokucabanga ngalinye enanini lamathokheni elicacisiwe ngaphambi kokuba imodeli iphendule.
• nothink_threshold: okuzenzakalelayo ngu-null. Lapho kulungiselelwe, uJeeves weqa ukucabanga uma ukuzethemba kwakhe kokungacabangi kufinyelela noma kudlula le threshold.
• return_reasoning: okuzenzakalelayo ngu-false. Kwenze kusebenze ukuze ufake umbhalo wokucabanga empendulweni.

Khetha iphrofayela yokubambezeleka

Ekuhlolweni kokuthuthukisa kwemibuzo engu-325 okuku-README, ukucabanga okugcwele kufinyelele ukunemba okungu-0.825 ngamathokheni okucabanga aphakathi angu-1,138 kanye nokubambezeleka okumaphakathi kwemizuzwana engu-3.3. Ukusebenzisa i-max_think engu-768 ne-nothink_threshold engu-0.9 kukhiqize ukunemba okungu-0.806, amathokheni okucabanga aphakathi angu-344, kanye nokubambezeleka okumaphakathi kwemizuzwana engu-2.0. Ukucisha ukucabanga kukhiqize ukunemba okungu-0.775 cishe ngemizuzwana engu-0.3.

Ukucushwa okusebenzayo okulinganiselayo yilokhu:

{
  "options": {
    "max_think": 768,
    "nothink_threshold": 0.9,
    "return_reasoning": false
  }
}

Ukucabanga okugcwele kunokubambezeleka okude kakhulu: i-p90 ebikiwe yimizuzwana engu-17.1. Sebenzisa amaketanga amafushane noma ukucabanga okusekelwe ekuzethembeni lapho isikhathi sokuphendula sibalulekile.

Indlela uJeeves enza ngayo izinqumo

UJeeves ufometha isimo, imiyalelo yombuzo, nezinketho zempendulo esebenzisa amathokheni angavamile avela ku-Qwen tokenizer. Ube esekhiqiza iketanga lokucabanga eliqala ngomaka wokucabanga. Ngemva kokucabanga, umbuzo nezinketho ziphindwa ngaphambi komaka wesinqumo.

I-pointer head ibala i-scaled dot product phakathi kwe-query projection emaka isinqumo nama-key projection emngceleni wenketho ngayinye. I-softmax iguqula lawo mamaki abe amathuba, bese izinga lokushisa elilinganiswe kusethi yokuthuthukisa liwalinganisa. Ngokusho kwama-ablation aku-README, ukufaka amalebula avamile esikhundleni samathokheni akhethekile e-prompt noma ukushiya umbuzo ophindwayo wangemva kokucabanga kwehlise ukusebenza.

I-drafter ye-diffusion

I-drafter yokuzikhethela isheshisa ukukhiqizwa kokucabanga nge-speculative decoding. Ivumelanisa indlela ye-Orthrus diffusion nezendlalelo ze-Gated DeltaNet ze-Qwen3.5 ngokuvumela amathokheni e-mask ukuthi asebenzise i-cross-attention kuma-key nama-value angemva kwe-convolution.

Ukuphuma okubikiwe kombuzo owodwa kukhuphuka kusukela kumathokheni eketanga angu-109 ngomzuzwana nge-plain graphed greedy decoding kuye kumathokheni angu-176 ngomzuzwana nge-block 4. I-block 8 ifinyelela kumathokheni angu-193 ngomzuzwana, kodwa i-block 4 isalokhu iyona ezenzakalelayo ngoba ihlala ingabizi uma imibuzo eminingana ihlanganiswa nge-batch. Imibuzo eyisishiyagalombili ehlanganiswe nge-block 4 ifinyelela cishe kumathokheni eketanga angu-960 ngomzuzwana esewonke.

Sebenzisa i-checkpoint oyiqeqeshile

UJeeves usebenzisa i-LoRA ngesikhathi sokuqeqeshwa. Ngaphambi kokusebenzisa i-checkpoint eqeqeshiwe, yihlanganise ibe imodeli ezimele bese unikeza ifayela le-drafter elihambisanayo:

python export.py runs/cispo/final --out runs/fused
python -m inference.serve \
  --model runs/fused \
  --drafter runs/drafter_k4/drafter.safetensors \
  --port 8009

Phinda ukhiqize i-pipeline yokuqeqeshwa

Lungisa amasethi edatha

Izikripthi zokulungiselela zilanda amasethi edatha asesidlangalaleni kwa-Hugging Face ngezinguqulo ezibekwe ku-prep/public.py. Isethi ngayinye yedatha yomthombo igcinwa ngaphansi kwelayisensi yayo.

python -m prep.prep

Qalisa ukuqeqeshwa nokuhlola

I-pipeline yokuphindaphinda ebhaliwe isebenzisa ama-GPU ayisishiyagalombili. Yenza i-supervised fine-tuning, ukuqeqeshwa kwe-CISPO, ukuhlolwa kwe-test ne-JevBench, ukuthunyelwa kwemodeli, ukukhiqizwa kwamaketanga, kanye nokuqeqeshwa kwe-drafter ye-block-4.

torchrun --nproc_per_node 8 train.py sft --run-dir runs/sft

torchrun --nproc_per_node 8 train.py cispo \
  --run-dir runs/cispo \
  --init runs/sft/final

torchrun --nproc_per_node 8 test.py runs/cispo/final
torchrun --nproc_per_node 8 jevbench.py runs/cispo/final
python export.py runs/cispo/final --out runs/fused

torchrun --nproc_per_node 8 -m drafter.gen --model runs/fused

torchrun --nproc_per_node 8 train.py drafter \
  --model runs/fused \
  --block 4 \
  --run-dir runs/drafter_k4

Inqubo eshicilelwe iqala ngokuqeqesha ama-epoch amabili e-SFT emibuzweni engu-19,126 evela kumadathasethi omphakathi angu-12 kanye nedatha yokwenziwa yezinqubomgomo. Bese isebenzisa i-CISPO ngokuphindaphinda okuyisishiyagalombili ngombuzo ngamunye, izinga lokushisa elingu-1, kanye nomkhawulo wokucabanga wamathokheni angu-2,560. Ukuqeqeshwa kwamiswa esinyathelweni sama-402 ohlelweni olwaluhlelelwe izinyathelo ezingu-624, ngoba izinyathelo zakamuva zenza i-pointer head ibe bukhali ngokweqile echibini lokufunda ngokuqinisa elaligcwele. Ekugcineni, kwalinganiswa izinga lokushisa elilodwa lokulinganisa kusethi yokuthuthukisa.

Amathiphu okusebenzisa athuthukile

1. Hlanganisa izinqumo ezihlobene. Faka imibuzo eminingana ngaphansi kwesimo esisodwa esabiwe ukuze iseva ikwazi ukucubungula amaketanga ayo okucabanga ngokuhambisana.
2. Sebenzisa izimpendulo eziqondile ezimweni ezilula. Lungiselela i-nothink_threshold ukuze ugcine ukucabanga kwenzelwa izinqumo lapho ukuzethemba kokungacabangi kungaphansi komkhawulo owukhethile.
3. Khawulela ukucabanga ezinsizeni ezizwela ukubambezeleka. Yehlisa i-max_think kunokuba ukhubaze ukucabanga ngokuphelele uma ufuna ibhalansi phakathi kwejubane nokunemba.
4. Buyisa ukucabanga ngokukhetha. Gcina i-return_reasoning ikhutshaziwe ngaphandle uma uhlelo ludinga lowo mbhalo, ikakhulukazi ngoba amaketanga awaqinisekisiwe ukuthi azohunyusheka kalula.
5. Hlola esizindeni sakho. I-Jeeves isebenza kahle kusethi yokuhlola yephrojekthi kanye namazinga omphakathi e-JevBench, kodwa isalela emuva kwe-Jev ekuhlolweni kolwazi okubikiwe kwe-MMLU ne-MMLU-Pro.
6. Humusha iziqhathaniso ngokucophelela. Ngaphandle kwe-JevBench, iziqhathaniso ezishicilelwe ze-Kev ne-Jev zisebenzisa izinto ezihlukene ezivela emithonjeni efanayo eyisisekelo.

Isiphetho

I-Jeeves ihlanganisa imiphumela ehlelekile yesitayela se-Jev nokucabanga ngesikhathi sokuhlola, amathuba alinganisiwe, kanye nokukhiqizwa kwamaketanga okusheshisiwe. Qala ngezisindo ezikhishiwe kanye ne-HTTP endpoint ehambisana ne-Jev, sebenzisa i-Python SDK yekhodi yohlelo lokusebenza enohlobo olucacisiwe, bese ulungisa i-max_think ne-nothink_threshold ukuze ulinganise ukunemba nokubambezeleka. Ukuze wenze ukulungisa okujulile, indawo yokugcina inikeza ipayipi eliphelele le-SFT, CISPO, lokulinganisa, lokuthekelisa, kanye ne-drafter.