Yakha Izinqumo Ezisheshayo, Ezinohlobo Olucacile ze-LLM Yasendaweni nge-Rizzo Flow

Yini i-Rizzo Flow?

I-Rizzo Flow iwuhlelo oluwumthombo ovulekile, olusebenza kuqala endaweni, oluguqula umbhalo ongahlelekile noma isimo se-JSON sibe izinqumo ezinohlobo namathuba. Esikhundleni sokucela imodeli yolimi ukuthi ikhiqize umbhalo noma i-JSON ithokheni ngethokheni, ifunda amathuba emodeli ezinhlamvini zezimpendulo ezinqunyelwe ngemva kokudlula phambili.

Lo mklamo usekela izinqumo ezifana nalezi:

• Impendulo ye-boolean enethuba le-true.
• Ukukhetha phakathi kwezinketho eziqanjwe, nethuba lenketho ngayinye.
• Isikolo kuwo wonke amazinga ahlelekile erubhrikhi.
• Isilinganiso senani esisekelwe kumaphoyinti ayizikhonkwane amele.

I-Rizzo Flow isebenza ku-hardware yakho nge-llama.cpp. Ingasebenzisa i-Apple Metal, i-NVIDIA CUDA, i-Vulkan, i-AMD ROCm, i-Intel SYCL, noma ukusebenza kwe-CPU. Iphinde inikeze isixhumi esibonakalayo se-HTTP esihambisana ne-Jev, esivumela izinhlelo ezihambisanayo ukuthi ziqondise ku-URL yasendaweni esikhundleni sensizakalo esingathwa kwenye indawo.

I-Rizzo Flow yiphrojekthi ezimele. Iphindaphinda iphethini yesixhumi esibonakalayo esingemuva kwe-Jev, hhayi ukwakheka kobunikazi noma ukuqeqeshwa kwe-Jev. Amathuba ayo awalinganiswanga ngaphandle kokuthi uwalinganise kudatha yakho emele.

Indlela izinqumo zamathokheni aziro ezisebenza ngayo

Ngesicelo ngasinye, i-Rizzo Flow ibeka isimo ekuqaleni kwe-prompt bese isicubungula kanye. Imibuzo ibe isahlukaniswa isuka kunqolobane yesimo eyabiwe. Impendulo ngayinye engaba khona imelelwa ngohlamvu olukhulu, bese uhlelo lufunda kuphela ama-logit ezinhlamvu ezivunyelwe.

1. Isimo siguqulwa sibe umbhalo futhi sigcwaliswe kusengaphambili kunqolobane ye-KV yemodeli.
2. Umbuzo ngamunye umelelwa njengenkinga yokukhetha okuningi enezithiyo.
3. Imibuzo eyabelana ngesimo esifanayo ihlolwa ngama-micro-batch.
4. Ama-logit ezimpendulo ezivunyelwe aguqulwa abe amathuba kusetshenziswa i-softmax.
5. Ikhodi ye-Python ibuyisa idatha ye-boolean, ukukhetha, isikolo, noma inani eqinisekiswe yi-schema.

Akukho loop yokukhipha, umbhalo osampuliwe, ukuhlaziya okukhiphayo, noma ukulungiswa kwe-JSON. Nokho, amathokheni akhiqiziwe aziro awasho ukuthi ukubala kuncane: isimo nama-prompt emibuzo kusadinga ukucatshungulwa kwemodeli.

Izici ezibalulekile

• Ukusebenza kwasendaweni ngokuphelele: ukucatshungulwa kwemodeli kwenzeka emshinini wakho.
• Imiphumela enohlobo: izinhlelo zithola amanani ahlelekile esikhundleni sombhalo okhiqiziwe.
• Ukusatshalaliswa kwamathuba: imiphumela yokukhetha nesikolo iveza amathuba esikhundleni sempendulo ye-argmax kuphela.
• Izisekelo ezine zomdabu: boolean, choice, score, ne-numeric.
• Ukuzigodla ngokuzikhethela: i-API yomdabu ingabika ubufakazi obunganele, ukungaqiniseki, noma umphumela wenani ongaphandle kobubanzi.
• Ukucubungula amaqoqo anesimo esabiwe: imibuzo eminingi esicelweni esisodwa iphinde isebenzise inqolobane ye-KV yesimo.
• Amaphoyinti okugcina ahambisana ne-Jev: amaklayenti akhona angasebenzisa i-/v1/systemone ne-/v1/models.
• Imodeli enomongo omude: i-Spark-X2.5 isekela umongo womdabu ongafika kumathokheni angu-1,048,576, nakuba i-Rizzo Flow isetha ngokuzenzakalela amathokheni angu-8,192 umbuzo ngamunye.
• Amathuluzi asendaweni: iseva ihlanganisa i-playground, imibhalo ye-OpenAPI esebenzisanayo, nomboniso we-Snake.

Izidingo zangaphambili

Ngaphambi kokufaka i-Rizzo Flow, qinisekisa ukuthi unalezi:

• i-Python 3.11 noma entsha.
• i-Git.
• uv yokuphatha okuncikile nendawo yokusebenza.
• Isikhala sediski esanele semodeli nesikhathi sokusebenza okukhethiwe.

Ukulanda okuzenzakalelayo kwemodeli ye-Spark-X2.5-4B Q8_0 kungaba cishe ngu-4.4 GB. Ukulanda kwesikhathi sokusebenza kuyahluka kuye ngeplathifomu, kusuka cishe ku-11 MB ku-Mac kuya cishe ku-570 MB kwiphakheji ye-CUDA.

Faka bese uqala iseva

Clone-isha i-repository, vumelanisa okuncikile kwayo okuqondiswe ezinguqulweni ezithile, landa isikhathi sokusebenza nemodeli okuzenzakalelayo, bese uqala insizakalo:

git clone https://github.com/Rizzo-AI-Academy/rizzo-flow
cd rizzo-flow
uv sync --locked
uv run rizzo download
uv run rizzo serve

Umyalo wokulanda ukhetha iphakheji esemthethweni eyakhiwe kusengaphambili ye-llama.cpp yomshini wamanje, uqinisekise i-checksum ye-SHA-256, bese ulanda i-Spark-X2.5-4B Q8_0. Ukulanda okuphazamisekile kungaqhubeka lapho kumiswe khona.

Ukulayishwa kwemodeli kuthatha cishe imizuzwana eyishumi ngokwemibhalo yephrojekthi. Uma insizakalo isilungile, vula:

• http://127.0.0.1:8017/playground ukuze uthole i-playground ebonakalayo.
• http://127.0.0.1:8017/docs ukuze uthole imibhalo ye-OpenAPI esebenzisanayo.
• http://127.0.0.1:8017/snake ukuze uthole umboniso we-Snake.

I-playground ihlanganisa izibonelo ezilungiselelwe, umakhi wemibuzo, abahleli be-JSON eluhlaza bawo womabili ama-API, amabha amathuba, imininingwane yesikhathi, nemiyalo ye-cURL ehambisanayo. Ayenzi izingcingo zangaphandle futhi ingashintshwa phakathi kwesiNgisi nesiNtaliyane.

Sebenzisa imodeli encane

Ukuze ulande okokuqala ngokushesha, faka imodeli engu-1.7B:

uv run rizzo download --size 1.7b
uv run rizzo serve --size 1.7b

Ifayela le-1.7B Q8_0 lilinganiselwa ku-1.8 GB futhi lisebenza cishe ngokushesha ngokuphindwe kabili, kodwa i-README ixwayisa ngokuthi linembe kancane kakhulu. Livame futhi ukukhetha inketho yobufakazi obunganele lapho ukuhoxa kunikwe amandla, ngakho lihlole ngokucophelela emsebenzini wakho.

Yenza isinqumo sakho sokuqala

Ukuhlola okusheshayo kwe-API kusebenzisa i-endpoint POST /v1/systemone ehambisana ne-Jev. Isicelo esilandelayo sibuza ukuthi umlayezo wosekelo ubonisa yini ukuphuthuma:

curl http://127.0.0.1:8017/v1/systemone \
  -H 'Content-Type: application/json' \
  -d '{
    "state": "Help! My payouts have been failing for 3 days.",
    "model": "rizzo-latest",
    "questions": {
      "is_urgent": {
        "type": "noul",
        "instructions": "Does this convey urgency?"
      }
    }
  }'

Umphumela we-noul uyis_probability_ yokuthi impendulo inguyebo, emelwe inombolo esukela ku-zero iye ku-one. Impendulo ibika isihlonzi sangempela semodeli yasendaweni, ngisho nalapho isicelo sisebenzisa i-rizzo-latest noma elinye igama elilula.

Buza imibuzo eminingana ngesicelo esisodwa

IRizzo Flow yakhelwe ukuhlola imibuzo eminingi ngokumelene nesimo esifanayo. Ukuyihlanganisa esicelweni esisodwa kuvumela imibuzo ukuthi yabelane nge-KV cache yesimo:

curl http://127.0.0.1:8017/v1/systemone \
  -H 'Content-Type: application/json' \
  -d '{
    "state": "Help! My payouts have been failing for 3 days.",
    "model": "rizzo-latest",
    "questions": {
      "is_urgent": {
        "type": "noul",
        "instructions": "Does this convey urgency?"
      },
      "department": {
        "type": "choice",
        "instructions": "Which team should handle this?",
        "criteria": {
          "billing": "Payments, invoicing, refunds",
          "technical": "Bugs and outages",
          "sales": null
        }
      },
      "frustration": {
        "type": "score",
        "instructions": "How frustrated is the customer?",
        "criteria": ["Calm", "Frustrated", "Very angry"]
      }
    }
  }'

Impendulo iqukethe amathuba okuthi impendulo ibe nguyebo ku-noul, amathuba azo zonke izinketho ze-choice, kanye nesikolo esinesisindo samathuba nencazelo yaso. Inani le-usage.output_tokens lihlala lingu-zero.

Sebenzisa i-API yezinqumo yomdabu

I-endpoint yomdabu ethi POST /v1/decisions iveza zonke izici ze-Rizzo Flow, kuhlanganise nemibuzo yezinombolo nokuhoxa. Izinhlobo zayo ezine zemibuzo yilezi:

• boolean: ibuyisa inani elinohlobo oluqondile kanye namathuba okuba iqiniso.
• choice: ibuyisa inketho ekhethiwe nokusatshalaliswa okuphelele kwazo zonke izinketho.
• score: ibuyisa izikolo ezinesisindo samathuba nezijwayelekile kuwo wonke amazinga ahleliwe.
• numeric: ibuyisa isilinganiso, i-median, ukusabalala, kanye namathuba okuba ngaphansi noma ngaphezu kobubanzi.

Linganisa inani lezinombolo usebenzisa ama-anchor

Umbuzo wezinombolo uchaza ama-anchor amele amanani akhuphukayo. Lesi sibonelo sicela imodeli ukuthi ifunde iphesenti lokugcwala elibikiwe:

curl http://127.0.0.1:8017/v1/decisions \
  -H 'Content-Type: application/json' \
  -d '{
    "state": {"measurement": 75, "unit": "percent"},
    "questions": {
      "fill": {
        "type": "numeric",
        "instructions": "Read the reported fill percentage.",
        "unit": "percent",
        "anchors": [
          {"value": 0, "description": "Empty"},
          {"value": 50, "description": "Half full"},
          {"value": 75, "description": "Three quarters full"},
          {"value": 100, "description": "Completely full"}
        ]
      }
    }
  }'

Ama-anchor amele amanani, awawona amabanga ezibalo. Isilinganiso esibikiwe sihlala siphakathi kwe-anchor ephansi nephezulu, kanti ama-quantile achaza ukusatshalaliswa kwamathuba okuhlukene kulawo ma-anchor.

Qonda ukuhoxa

Imibuzo yomdabu ivumela ukuhoxa ngokuzenzakalelayo. IRizzo Flow yengeza inketho yangaphakathi yobufakazi obunganele, kanti imibuzo yezinombolo ihlanganisa namathuba okuba ngaphansi noma ngaphezu kobubanzi. Kuya ngenketho ekhethiwe nenqubomgomo, inani eliyinhloko lingaba null, futhi isimo singabika insufficient_evidence, out_of_range, noma uncertain.

Ifomethi ehambisana ne-Jev ayisebenzisi ukuhoxa. Umphumela wayo woyebo/cha ubalwa phezu kwezinketho ezimbili kuphela. Uma usebenzisa imodeli encane ye-1.7B nge-API yomdabu, cabanga ukusetha allow_abstain ku-false, njengoba kunconywe iphrojekthi, bese uqinisekisa umthelela wayo kudatha yakho.

Qalisa izinqumo ngaphandle kweseva

Kuma-script, izivivinyo, noma ukuhlola okukodwa, dlulisela ifayela lesicelo ngqo ku-CLI:

uv run rizzo decide examples/ticket.json

Ungakwazi futhi ukwenza kusebenze i-virtual environment bese ususa isiqalo esithi uv run:

source .venv/bin/activate
rizzo decide examples/ticket.json

Ku-PowerShell, yenze isebenze kanje:

.venv\Scripts\activate

Khetha imodeli, i-quantization, nedivayisi

Ukucushwa okuzenzakalelayo kusebenzisa i-Spark-X2.5-4B Q8_0. Amanye ama-quantization abhaliwe yi-Q4_K_M ne-BF16:

• 4B Q8_0: cishe 4.4 GB futhi iwukucushwa okuzenzakalelayo.
• 4B Q4_K_M: cishe 2.6 GB.
• 4B BF16: cishe 8.2 GB.
• 1.7B Q8_0: cishe 1.8 GB.
• 1.7B Q4_K_M: cishe 1.1 GB.
• 1.7B BF16: cishe 3.4 GB.

Hlola amadivayisi abonakala ku-runtime ngaphambi kokuqala iseva:

uv run rizzo devices

Ngemuva kwalokho ungakhetha ngokusobala umndeni wedivayisi:

uv run rizzo serve --device cuda
uv run rizzo serve --device vulkan
uv run rizzo serve --device metal
uv run rizzo serve --device cpu

Imindeni yamadivayisi ebekiwe iyizimfuneko, hhayi iziphakamiso, ngakho i-Rizzo Flow ayishintsheli buthule emndenini we-GPU oshiwo ngokusobala iye ku-CPU. Amaphakheji engeziwe esikhathi sokusebenza angalandwa ngokwehlukana:

uv run rizzo download --only runtime --runtime rocm
uv run rizzo download --only runtime --runtime sycl
uv run rizzo download --only runtime --runtime cpu

Ukucushwa okuthuthukile namathiphu asebenzayo

Hlanganisa imibuzo ehlobene

Beka yonke imibuzo emayelana nesimo esifanayo esicelweni esisodwa. Lokhu kuyisisekelo sokwakheka kwe-Rizzo Flow: isimo sigcwaliswa kanye, bese izijobelelo zemibuzo zihlolwa ngamaqoqo amancane. Usayizi ozenzakalelayo weqoqo elincane lemibuzo ngu-4 futhi ungashintshwa nge---batch-size.

uv run rizzo serve --batch-size 8

Amaqoqo amakhulu awabi ngcono ngokuzenzakalelayo. Qhathanisa ukubambezeleka nokusetshenziswa kwememori emshinini okuhloswe ngawo.

Yandisa umongo ngokucophelela

Nakuba i-Spark-X2.5 inomongo wama-token ayisigidi owakhelwe ngaphakathi, iseva isebenzisa ama-token angu-8,192 kumbuzo ngamunye ngokuzenzakalelayo. Nyusa umkhawulo nge---ctx:

uv run rizzo serve --ctx 32768

I-cache ye-KV yabelwa ekuqaleni. Kumodeli ye-4B, i-README ilinganisela cishe ku-144 KiB ngetokeni ngayinye, noma cishe ku-1.4 GiB emkhawulweni ozenzakalelayo kanye no-4.8 GiB kuma-token angu-32,000. Okokufaka okwedlula umkhawulo omisiwe kuyanqatshwa kunokuba kunqunywe. Ngemva kwama-token acishe abe ngu-60,000, umkhawulo wesimo ongu-256 KB okurephozithoriyeni ku-schema.py nawo kufanele unyuswe.

Vikela iziphetho ezihambisanayo

Setha i-RIZZO_API_KEY ngaphambi kokuqala iseva ukuze kudingeke ukuqinisekiswa kwe-Bearer eziphethweni ezihambisana ne-Jev:

export RIZZO_API_KEY="replace-with-a-secret"
uv run rizzo serve

Ku-Windows PowerShell:

$env:RIZZO_API_KEY = "replace-with-a-secret"
uv run rizzo serve

Ukwehluleka kokuqinisekisa kubuyisa i-HTTP 401. Idatha yesicelo engavumelekile ingabuyisa i-HTTP 422.

Qondisa kabusha iklayenti elihambisanayo

Ikliyenti elakhelwe i-TypeSafe API esingathwe lingakhomba isevisi yasendaweni ngokushintsha i-base URL yalo:

export TYPESAFE_BASE_URL=http://127.0.0.1:8017

Iphrojekthi ithi lokhu kusethwa kokuguquguquka kwemvelo kwakhelwe ama-SDK asemthethweni, kodwa akukahlolwa ngawo. Isixhumi esibonakalayo siyahambisana, kodwa imodeli yasendaweni engaphansi akuyona i-Jev.

Phatha ukuzethemba namathuba ngendlela efanele

Inani lokuzethemba le-API ehambisanayo lichaza ukuma kokusatshalaliswa kwezinketho. Akulona ithuba eliqinisekisiwe lokuthi impendulo ilungile. Ngokufanayo, amathuba aluhlaza emodeli angaba nokuzethemba okweqile noma angalinganiswa ngokungalungile.

Qinisekisa izinqumo kudathasethi emelelayo enamalebula futhi uzilinganise endaweni yangempela yokusetshenziswa lapho kudingeka. Iseva yamukela ifayela lokulinganisa nge---calibration:

uv run rizzo serve --calibration fit.json

Ukulinganisa kuxhumene nefayela lemodeli, isikhathi sokusebenza, i-quantization, kanye ne-backend yehadiwe esetshenziswe ekukulinganiseni. I-CUDA, i-Vulkan, ne-Metal zingase zisondeze izinombolo ngezindlela ezahlukene, futhi i-quantization ingashintsha amathuba abuyiswayo.

Hlonipha umkhawulo wezikhala zezimpendulo

Umuntu ngamunye okhethwayo ufaniswa nohlamvu olukhulu olulodwa, okudala izikhala zezimpendulo ezifika ku-26 kumbuzo ngamunye. Izinketho zangaphakathi zokungaphenduli nezobubanzi nazo zisebenzisa izikhala. Ngenxa yalokho, ukukhetha kusekela izinketho ezijwayelekile ezifika ku-26 ngaphandle kokungaphenduli noma ezingu-25 ngokungaphenduli. Imibuzo yezinombolo inama-anchor ambalwa atholakalayo ngoba izinketho ezingaphansi kobubanzi, ezingaphezu kobubanzi, kanye nokunganele kobufakazi ngokuzikhethela nazo zisebenzisa izikhala.

Sebenzisa ifayela lemodeli langokwezifiso noma i-llama.cpp build

Qala iseva ngefayela elithile le-GGUF usebenzisa i---model:

uv run rizzo serve --model /path/to/model.gguf

Ukuze usebenzise ukufakwa kwe-llama.cpp okwenziwe ngokwezifiso, khomba i-RIZZO_LLAMA_DIR kufolda equkethe i-libllama. I-README idinga i-llama.cpp commit 161755f ngoba ama-bindings axhumene ne-header yaleyo nguqulo.

Hlola umbukiso we-Snake

Ikhasi lendawo le-Snake libonisa ukuthi izinqumo ezithayiphiwe zingalawula kanjani uhlelo olusebenzisanayo. Konke ukunyakaza kuthumela isicelo esisodwa se-POST /v1/decisions esiqukethe incazelo yebhodi nombuzo wokukhetha obala ukunyakaza okusemthethweni. Ikhasi libonisa amathuba ezimpendulo, ama-logit, izikhathi, kanye nerekhodi lezinqumo ngaphandle kokukhiqiza umbhalo.

Lo mbukiso uphinde ubonise isifundo esibalulekile sokumodela: indlela okokufaka okumelwa ngayo ibalulekile. I-README ibika ukuthi imodeli ye-4B isebenza kangcono kakhulu ngezinzwa ezibalwe ngokunyakaza ngakunye kunokuba isebenzise igridi ye-ASCII kuphela. Lokho kubonwa kuvela emidlalweni embalwa engahlelekile futhi akufanele kuthathwe njenge-benchmark.

Ukuhlolwa kokusebenza

Sebenzisa i-GET /health ukuhlola umsuka wemodeli nama-hashi wamafayela. Ama-schema ezicelo nezimpendulo ayatholakala futhi ku-request.schema.json naku-response.schema.json. Ukuze uthole ukuthunyelwa okuphindaphindekayo, gcina imodeli, i-quantization, isikhathi sokusebenza, i-backend, ukucushwa komongo, kanye nokulinganisa kungashintshi.

I-README ibika cishe ama-millisecond angu-50 ngesinqumo esifushane nge-Spark-X2.5-4B Q8_0 ku-RTX 5060 Ti, kodwa lokhu kuncike kuhadiwe nomthwalo womsebenzi. Imibhalo efanayo iphawula ukuthi ukucushwa kwe-Apple Silicon, AMD, Intel, Linux NVIDIA, ne-CPU akukatholi ukuhlolwa okulinganayo konke, ngakho hlola umshini wakho ngaphambi kokubeka okulindelwe mayelana nokubambezeleka.

Isiphetho

I-Rizzo Flow inikeza isixhumi esibonakalayo sendawo esisebenzayo sokuguqula isimo esingahlelekile sibe izinqumo ezinohlobo nezinokwenzeka ngaphandle kokukhiqiza umbhalo. Qala ngendawo yokuhlola, hlanganisa imibuzo ehlobene ibe isicelo esisodwa, bese usebenzisa i-API yomdabu lapho udinga izilinganiso zezinombolo noma ukugwema ukuphendula. Ngaphambi kokuyisebenzisa ekukhiqizeni, hlola ukunemba, ukubambezeleka, ukulinganiswa, ukuncishiswa kokunemba, nokuziphatha kwe-backend kudatha emele uhlelo lwakho lwangempela.