Yeqa uye kokuyinhloko
Izifundo ze-AI

Yakha Izinqumo Ezithembekile Nezinhlobo Ze-LLM usebenzisa i-AnyJev

Funda ukufaka i-AnyJev, ukuchaza imibuzo yokukhetha, ye-Boolean, neyokulinganisa, nokuthola izinqumo ezihlelekile ku-LLM evulekile ngaphandle kokukhiqiza noma ukwenza i-fine-tuning. Lesi sifundo siphinde sichaze i-zero-label L0 inference, i-L1 calibration, ama-head e-L2 closed-form, ukunikezwa kwamaqoqo, ukuphathwa kwama-artifact, nemikhawulo yokusabalalisa.

Cishe imizuzu engu-25 yokufunda
Yakha Izinqumo Ezithembekile Ze-LLM usebenzisa i-AnyJev

Okwenziwa yi-AnyJev

I-AnyJev iguqula imodeli yolimi evulekile ibe imodeli yezinqumo yesitayela se-Jev. Esikhundleni sokucela imodeli ukuthi ikhiqize impendulo bese sihlaziya umbhalo wayo, i-AnyJev ifunda ukusatshalaliswa kwamathokheni alandelayo kwemodeli kusukela ku-prefill bese ibuyisa isinqumo esinohlobo namathuba aso.

Lokhu kuwusizo ekugelezeni komsebenzi okufana nokuqondisa isicelo, ukulinganisa ukuthi isenzo siyingozi yini, noma ukukala ukuqedwa komsebenzi. Umphumela ngamunye ugcina izinga lesinqumo elisetshenzisiwe, okwenza ikhodi elandela kamuva ikwazi ukuhlukanisa umphumela ongakalinganiswa noma onelebula eziro kulowo okhiqizwe yi-head eqeqeshiwe.

I-AnyJev ibhekana nezinkinga ezimbili ezisebenzayo zama-logit aluhlaza: izibikezelo zingashintsha lapho kushintshwa ukuhleleka kwezinketho, futhi amanani okuzethemba aluhlaza kungenzeka angalinganiswa kahle ngokwanele ukuze kusetshenziswe okuzenzakalelayo okusekelwe emikhawulweni. Esivivinyweni se-Qwen3-8B BANKING77 esiku-README, i-L0 yehlisa izinga lokuphenduka kokukhethwa ngenxa yokuhleleka kwezinketho lisuka ku-0.230 laya ku-0.073 ngaphandle kwamalebula. I-L1 yehlisa iphutha elilindelekile lokulinganisa lisuka ku-0.240 laya ku-0.095 kusetshenziswa izibonelo ezinalebula.

Imiphumela ye-AnyJev yokuzinza kokuhleleka, ukulinganisa, ukunemba, nokumbozwa

Amazinga Ezinqumo

I-AnyJev inikeza amazinga amaningana anezidingo ezihlukene zedatha nokusebenzisa:

  • raw: Isebenzisa i-softmax ekhawulelwe phezu kwamathokheni amalebula. Ayilungisi ukuchema kwendawo futhi ayilinganisi ukuzethemba.
  • L0: Ayidingi malebula. Ihlola ukuzungeziswa kwezinketho okuyiziyingi, ikhiphe isilinganiso sokuchema kwendawo, bese ihlukanisa nge-prior yelebula elinganisiwe.
  • L1: Isebenzisa cishe amalebula angu-100–500 ngombuzo ngamunye ukuze ifake i-temperature scaling phezu kwe-L0. Ithuthukisa ukulinganiswa kodwa ayishintshi ukuhleleka kwezikhundla.
  • L2: Isebenzisa cishe amalebula angu-100–300 ngombuzo ngamunye ukuze ixazulule i-shrunk LDA noma i-ridge head evela esimweni esifihlekile esiphakathi. Idinga imodeli yasendaweni futhi iqondene nemodeli nombuzo kokubili.

I-L0 idinga ama-prefill amaningi ekukhetheni okunezinketho eziningi ngoba ihlola ukuzungeziswa. I-L2 yona isebenzisa i-prompt eyodwa futhi ime ku-block ephakathi enqunyiwe, okwenza ishibhe kune-full forward pass ezivivinyweni ze-Qwen3 ezibikiwe.

Izici Eziyinhloko

  • Imibuzo enohlobo ethi choice, noul, kanye score.
  • Akukho ukukhiqizwa kombhalo noma ukuhlaziya impendulo.
  • Ukulungiswa kwe-L0 okungenamalebula kwemiphumela yendawo yezinketho kanye ne-prior yelebula.
  • Ukulinganiswa kwe-L1 temperature ngamalebula angamakhulu ambalwa.
  • Ama-head e-L2 efomu elivaliwe afakwa ngemizuzwana ngaphandle kwama-gradient noma ukulungiswa kahle kwemodeli.
  • Ukukhethwa okuzenzakalelayo kwe-L2, L1, noma L0 nge-level="auto".
  • Ukuqoqwa kwamalebula kancane kancane nge-observe.
  • Izinqumo zeqoqo zezimo eziningi nombuzo owodwa.
  • Ama-artifact amancane e-JSON angagcinwa futhi alayishwe ukuze asetshenziswe kamuva.

Ukufakwa Nokulungiselela

Faka i-backend ye-Hugging Face

Faka i-AnyJev nokuhlanganiswa kwayo ne-Hugging Face:

pip install "anyjev[hf]"

Inguqulo yamanje isebenzisa wonke amazinga ezinqumo nge-backend ye-anyjev.backends.hf esekelwe ku-Transformers. Ukusekelwa kwe-vLLM ne-SGLang kuse-roadmap futhi akutholakali kule nguqulo.

Dala i-decider

Ngenisa i-API eyinhloko bese uqala i-HFBackend ngemodeli evulekile:

from anyjev import Decider, Question
from anyjev.backends.hf import HFBackend

decider = Decider(HFBackend("Qwen/Qwen3-8B"))

Ama-head e-L2 ahlinzekiwe ahlanganisa amamodeli amahlanu e-Qwen3: 1.7B, 4B, 8B, 30B-A3B, kanye 32B. Ama-head e-L2 ahlala eqondene nemodeli eyisisekelo nombuzo, ngakho i-head efakwe imodeli eyodwa noma umbuzo owodwa ayikwazi ukusetshenziswa ngokuzenzakalelayo kwenye.

Chaza Imibuzo Enohlobo

Umbuzo uchaza uhlobo lomphumela, izimpendulo ezivumelekile, kanye negama elizinzile. Ungahlola izinhlobo eziningi zemibuzo usebenzisa isimo esifanayo sohlelo.

route = Question.choice(
    "Which team should handle this?",
    ["billing", "technical", "sales", "other"],
    name="route",
)

risky = Question.noul(
    "Is this tool call destructive or irreversible?",
    name="risky",
)

done = Question.score(
    "How complete is the task?",
    bins=5,
    name="done",
)

Sebenzisa i-choice esinqumweni sezigaba, i-noul esinqumweni seqiniso-noma-amanga, kanye ne-score esikolweni sezinombolo esihlukaniswe ngamabhini. Ukufundwa kwamathokheni ezinhlamvu okwamanje kusekela izinketho ezingafika kwezi-26.

Yenza Isinqumo Esiyisisekelo Se-L0

I-L0 iyona ezenzakalelayo futhi ayidingi izibonelo ezinalebula. Yakha isimo esiqukethe ulwazi oludingwa yimibuzo, bese ubiza i-decide:

state = {
    "conversation": [
        {"role": "user", "content": "I was charged twice."}
    ],
    "tool_call": {
        "name": "refund_payment",
        "arguments": {"payment_id": "pay_123"},
    },
}

result = decider.decide(state, [route, risky, done])

print(result["route"].distribution)
print(result["risky"].p_true)
print(result["done"].value)
print(result.level)

Ukusatshalaliswa komzila okuyisibonelo kungase kuhlanganise inketho ngayinye enikeziwe nethuba layo. Izinqumo ze-Boolean ziveza i-p_true, kanti izinqumo zamaphuzu ziveza i-value. Umphumela wonke ubika i-L0 lapho kungekho-artifact yezinga eliphakeme etholakalayo.

Amathuba kufanele ahunyushwe ngokwezinga lawo. I-L0 ithuthukisa ukuzinza futhi ilungise ukuchema kwelebula okulinganisiwe, kodwa ayenzi ukungaqiniseki kulinganiswe ngokuphelele.

Engeza Ukulinganiswa Kwe-L1

Uma unezimo ezinalebula ezingaba ngu-100–500 zombuzo, biza i-calibrate ukuze ufake i-temperature ye-L1:

decider.calibrate(risky, states, labels)

I-L1 ilinganisa amathuba akhiqizwe phezu kwe-L0. Ayishintshi ukuthi iyiphi impendulo esesikhundleni sokuqala, kodwa ingenza imikhawulo yokuzethemba ibe nencazelo engcono.

Faka i-Head Ye-L2 Yefomu Elivaliwe

Ukuze uthole ukunemba okuphezulu, lungisa i-L2 head eqondene nombuzo usebenzisa cishe izibonelo ezilebulwe ezingu-100–300:

decider.fit_head(route, states, labels)

decider.save_artifacts("qwen3-8b.json")

Ukulungisa kwenza ukudlula okukodwa kwemodeli phezu kwezibonelo bese kuxazulula i-head ngendlela evaliwe. Akusebenzisi ama-gradient futhi akuguquli izisindo zemodeli yolimi. Ngokwephrojekthi, i-head ivamise ukuba cishe ngu-100 KB futhi ingaxazululwa ngemizuzwana kumamodeli amancane e-Qwen3 asekelwayo.

Layisha ama-artifact agciniwe kunqubo ezosebenza kamuva, bese ucela ukukhethwa kwezinga okuzenzakalelayo:

decider.load_artifacts("qwen3-8b.json")

result = decider.decide(state, [route], level="auto")
print(result["route"].level)

Nge-level="auto", i-AnyJev isebenzisa i-L2 uma i-head ehambisanayo ingakwazi ukuqondisa umbuzo. Uma kungenjalo, ibuyela ku-L1 uma i-calibration ikhona, bese iya ku-L0.

Qoqa Amalebula Kancane Kancane

I-AnyJev ingamukela amalebula njengoba efika emgqeni wokubuyekezwa, emiphumeleni ebonwe, noma komunye umthombo wempendulo:

decider.observe(route, state, correct_label)

Iluphu ezenzakalelayo yephrojekthi ixazulula i-head kuma-observation angu-30 bese iyihlola kabusha kuma-milestone angu-60, 120, nakwamanye alandelayo. Lokhu kusekela umjikelezo wokusetshenziswa lapho umbuzo omusha uqala ku-L0 bese uthuthela ku-L2 ngemva kokuqoqwa kwempendulo eyanele.

Inference Yamaqoqo

Uma usebenzisa umbuzo owodwa ezimweni eziningi, sebenzisa i-batch API esikhundleni sokubiza i-decide ngokuphindaphindiwe:

results = decider.decide_batch(states, route)

Lena i-interface ehloswe ukucubungula okokufaka okuningi ngombuzo owodwa ohlotshiwe.

Zama I-Demo Epakishiwe

Kusuka ku-checkout ye-repository, sebenzisa i-demo yokwenziwa ngaphandle kokulanda imodeli:

python -m demo.jev_mode --backend fake

Ukuze ulingise umjikelezo wokusetshenziswa, engeza inketho ye-lifecycle:

python -m demo.jev_mode --backend fake --lifecycle

Susa i---backend fake ukuze usebenzise i-demo yangempela ye-Qwen3 enama-head ahlinzekiwe.

Amathiphu Athuthukile Okusetshenziswa

Gcina ubunikazi bombuzo buzinzile

I-L2 ilungiswa ngombuzo nangemodeli ngayinye. Iqoqo lezinketho elisha lidinga amalebula amasha ne-head entsha. Ukuphinda ubhale umbuzo ofanayo noma ukushintsha ukuhleleka kwezinketho ezifanayo kungaqondisa ku-head ekhona.

Sebenzisa ithrafikhi engenawo amalebula ukuze ujwayelanise amagama

Emibuzweni ebhalwe kabusha, i-AnyJev ingaphinda igxilise isilinganiso sezici se-head futhi ilungise isikali isebenzisa cishe izicelo ezingu-30 ezingenawo amalebula. Lokhu kujwayelanisa kusebenza kumagama ombuzo nasekuhlelekeni kwezinketho; akuboni ushintsho ezimweni zohlelo ngokwalo.

Qapha ukushintsha kwedatha yangempela

Ngenxa yokuthi ukushintsha kokusatshalaliswa kwezimo akubonakali endleleni yokuphinda kugxilwe kuyo, gcina ingxenye yokuhlola elebulwe ngezikhathi ezithile futhi uhlole ukusebenza ngezikhathi ezithile. Ungakuthathi ukujwayelanisa amagama njengokuthatha indawo yokuqapha okukhiqizwayo.

Lawula izenzo ngezinga lesinqumo

Isinqumo ngasinye siphatha izinga laso. Amasistimu alandelayo angalihlola ngaphambi kokwenza isenzo esibucayi, futhi iphrojekthi isekela nokuphoqelela amazinga nge-require=. Lokhu kuvimbela ukuhamba komsebenzi okwenzelwe izinqumo ezikalibreyithiwe noma ze-L2 ukuthi kwenze isenzo buthule sisebenzisa umphumela wokubuyela emuva.

Khetha ama-threshold kuphela ngemva kokuqinisekisa

Inzuzo enkulu ye-calibration yikhono lokuthumela izimo ezinokuzethemba okuphezulu ku-automation kuyilapho izimo ezingaqinisekile zidluliselwa ekuhlolweni komuntu. Khetha ama-threshold kudatha emele iqiniso elebulwe futhi ulinganise iphutha nokumbozwa okuwumphumela, kunokucabanga ukuthi amathuba abonisiwe athembekile ngokuzenzakalelayo.

Imikhawulo Ebalulekile

  • Ama-head e-L2 awadluliseli komunye umbuzo noma komunye umodeli oyisisekelo.
  • Okwamanje ukukhishwa kwephrojekthi kufaka kuphela ama-head e-Qwen3.
  • I-L2 idinga ukufinyelela kuma-hidden states, okwamanje okunikezwa nge-Transformers backend.
  • I-calibration ayikwazi ukwenza imodeli ixazulule umsebenzi engakwazi ukuwuphendula ngokuyisisekelo.
  • I-L0 inganciphisa ukunemba uma ilebula eyodwa ibusa kakhulu i-batch prior.
  • Ukufundwa kwezinhlamvu kwamanje kusekela izinketho ezingekho ngaphezu kuka-26.
  • Isilinganiso esibikiwe se-5% risk coverage sisekelwe ezibonelweni ezingu-300 ngakho sinokuguquguquka okuphezulu.
  • Ukunemba kwezinqumo ezithayiphiwe okushicilelwe kukala ukuvumelana ne-teacher LLM, hhayi iqiniso eliyisisekelo elizimele elisunguliwe.
  • Izinqumo ezibikiwe zahlolwa zodwa, hhayi ngaphakathi kwe-agent loop ephelele.

Isiphetho

I-AnyJev inikeza inqubekelaphambili esebenzisekayo kusukela ezinqumweni ezithayiphiwe ezingenawo amalebula kuya emathubeni akalibreyithiwe nama-head asebenzayo e-hidden-state. Qala nge-L0, qoqa amalebula angempela, engeza i-L1 lapho i-calibration iyona nto ebalulekile, bese ulungisa i-L2 lapho udinga indlela yesinqumo eqondene nombuzo enembe kakhulu. Gcina izinga elibikiwe, qinisekisa ama-threshold, futhi uqaphe amasampula alebulwe njengoba idatha yokukhiqiza ishintsha.

Ukuze uthole imininingwane yokusebenzisa nezinhlelo zamanje, bheka i-AnyJev repository, isivumelwano samazinga ayo, imibhalo yama-benchmark, kanye ne-roadmap.