Sebenzisa Amamodeli e-AI Asheshayo Endaweni ku-Apple Silicon nge-Splash

Yini i-Splash?

I-Splash iyinjini yokuqagela esebenza endaweni, eyakhelwe ngokukhethekile i-Apple silicon. Ihlinzeka ngamamodeli olimi alungiselelwe ama-coding agent nezinhlelo ezisekela ama-API e-OpenAI noma e-Anthropic. Iseva isebenza endaweni ku-Mac eyodwa futhi iveza i-API yayo ku-127.0.0.1:8000.

Ngokungafani nenjini yokuqagela esetshenziswa ngezinhloso eziningi, i-Splash yenza ngokukhethekile ama-kernel ayo, imodeli esalungiselelwe i-speculative decoding, ukuhlelwa kwezisindo, nohlelo lwenkumbulo kumodeli ngayinye esekelwayo. Amaphakheji amamodeli afika elungele ukusebenza, ngakho awudingi i-Xcode, amathuluzi omhlanganisi, noma ukulungisa okuqondene nemodeli.

Izici Ezibalulekile

• Ukuqagela kwasendaweni ku-Apple silicon: I-Splash isebenzisa amamodeli ngokuqondile ku-Mac esekelwayo.
• Ukuhambisana ne-OpenAI: Ihlinzeka nge-Chat Completions ku-/v1/chat/completions kanye ne-Responses ku-/v1/responses.
• Ukuhambisana ne-Anthropic: Izicelo ze-Anthropic Messages zitholakala ku-/v1/messages.
• Ukusekelwa kwezicelo okuthuthukile: Ama-API asekela ukusakaza, izingcingo zamathuluzi, okukhiphayo kwe-JSON Schema, izithombe, nama-PDF afakwe ngaphakathi.
• Ukuhlanganiswa nama-coding agent: I-Splash ihlanganisa imiyalo ye-OpenCode, Claude, Codex, ne-Hermes.
• Ukusheshiswa okuqondene nemodeli: Iphakheji yemodeli ngayinye ihlanganisa i-DFlash 2 draft ye-speculative decoding nama-kernel e-Metal ahlanganiswe kusengaphambili futhi alungiselelwe ubukhulu obuqondile bemodeli.
• Ukuhlelwa kwenkumbulo okuzenzakalelayo: I-Splash ibala umthamo womongo, umthamo we-KV-cache, nemikhawulo ye-batch ekuqaleni, ngokusekelwe enkumbulweni etholakalayo ku-Metal.
• Ukuhlolwa kwe-prompt: Ama-endpoint e-/tokenize ne-/apply-template abuyisa ama-ID amathokheni noma ama-prompt ahunyushiwe ngaphandle kokuqalisa ukuqagela.

Izidingo Zesistimu

Ngaphambi kokufaka i-Splash, qinisekisa ukuthi isistimu yakho ihlangabezana nazo zonke lezi zidingo:

• Iprosesa ye-Apple M3 noma entsha
• i-macOS 26.4 noma entsha
• I-Homebrew
• Okungenani u-36 GB wememori ehlanganisiwe
• Kunconywa u-48 GB noma ngaphezulu wememori ehlanganisiwe

Umthamo womongo osebenzisekayo uncike enkumbulweni etholakalayo. Nakuba amamodeli asekelwayo enewindi lomongo lendabuko elifika ku-256K, iwindi eligcwele lingase lingangeni kuwo wonke ama-Mac.

Faka i-Splash

Faka ukukhishwa okupakishiwe nge-Inco Homebrew tap:

brew install incoai/tap/splash

Ayikho ifayela lokumisa okufanele ulidale. I-Splash isebenzisa amafulegi omugqa womyalo neziguquguquko zemvelo kuzilungiselelo ozikhethela zona.

Qalisa Iseva Yemodeli

Hlinzeka ngephakheji elungiselelwe ye-Qwen3.8-27B ngalo myalo:

splash serve --model incoai/Qwen3.8-27B-Splash

Ekusebenzeni kokuqala, i-Splash ilanda futhi iqinisekise iphakheji yemodeli, ihlole inkumbulo etholakalayo, idale uhlelo lwenkumbulo, bese iqalisa iseva ku-127.0.0.1:8000. Iphakheji ye-Qwen3.8-27B ingukulandwa okungu-17.4 GB futhi ihlanganisa imodeli eqondiswe ku-4-bit kanye ne-DFlash 2 draft yayo.

Linda kuze kube yilapho itheminali iphrinta Ready. Gcina leyo theminali ivuliwe ngenkathi usebenzisa imodeli. Ngemuva kwalokho ungavula http://127.0.0.1:8000 esipheqululini ukuze ufinyelele ikhasi lengxoxo.

Cindezela u-Ctrl+C etheminali yeseva uma ufuna ukumisa i-Splash.

Sebenzisa Enye Imodeli

I-Splash iphinde ishicilele iphakheji engu-20.9 GB equkethe imodeli ye-Qwen3.6-35B-A3B ye-4-bit kanye ne-DFlash 2 draft yayo:

splash serve --model incoai/Qwen3.6-35B-A3B-Splash

Inketho ethi --model ingakhomba enye inqolobane ye-owner/repo equkethe iphakheji ye-Splash evumelekile. Ama-checkpoint e-MLX nama-Transformers angenalutho awahambisani. Enqolobaneni eyimfihlo ye-Hugging Face, nikeza i-HF_TOKEN efanele.

Xhuma I-Coding Agent

Ngemva kokuba iseva ibike ukuthi isilungile, vula enye itheminali bese uqala enye yama-coding agent afakiwe asekelwayo:

splash opencode
splash claude
splash codex
splash hermes

Qalisa kuphela umyalo we-agent ofuna ukuwusebenzisa. Inqubo yokuqala yeseva kufanele ihlale isebenza ngenkathi i-agent ithumela izicelo.

Shayela I-OpenAI Chat Completions API

Izinhlelo zingashayela i-Splash njengeseva yasendaweni ehambisana ne-OpenAI. Isicelo esilandelayo sicela imodeli enikeziwe incazelo yomusho owodwa:

curl http://127.0.0.1:8000/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "incoai/Qwen3.8-27B-Splash",
    "messages": [
      {
        "role": "user",
        "content": "Chaza i-speculative decoding ngomusho owodwa."
      }
    ]
  }'

Impahla ethi model uyazikhethela. Uma uyifaka, inani layo kufanele lifane nephakheji esetshenziswa njengamanje.

Khetha I-API Endpoint Efanele

• Sebenzisa i-/v1/chat/completions kumaklayenti e-OpenAI Chat Completions.
• Sebenzisa i-/v1/responses kumaklayenti e-OpenAI Responses.
• Sebenzisa i-/v1/messages kumaklayenti e-Anthropic Messages.
• Sebenzisa i-/tokenize ukuze uthole ama-ID amathokheni ngaphandle kokuqalisa imodeli.
• Sebenzisa i-/apply-template ukuze uhlole i-prompt ehunyushiwe ngaphandle kokuqalisa ukuqagela.

Lawula Ukucabanga

Ukucabanga kuvulwa ngokuzenzakalelayo. Ukuvala, engeza "reasoning_effort": "none" emzimbeni wesicelo. I-Qwen3.8-27B iphinde yamukele i-low, i-medium, ne-xhigh.

{
  "model": "incoai/Qwen3.8-27B-Splash",
  "reasoning_effort": "none",
  "messages": [
    {
      "role": "user",
      "content": "Fingqa i-speculative decoding."
    }
  ]
}

Ukucisha ukucabanga kungaba wusizo uma umsebenzi ungadingi inqubo yokucabanga enwetshiwe. Emisebenzini ye-Qwen3.8-27B ezuzayo ngokucabanga, khetha izinga lomzamo elisekelwayo ngokuya ngomthwalo womsebenzi.

Lungisa Imikhawulo Yememori Nomongo

I-Splash inquma ngokuzenzakalelayo izilungiselelo ezizenzakalelayo ezifanele, kodwa i-splash serve inikeza amafulegi ongawakhetha ukuze ulawule ngokunembile:

• --max-memory 28G ibeka umkhawulo ophezulu ekwabelweni kwe-Metal.
• --max-context 100K ikhawulela iwindi lomongo, kuze kufike kumkhawulo wemvelo wemodeli ongu-256K.
• --max-image-pixels ilawula inani eliphezulu lamaphikseli ngemva kokushintshwa kosayizi kwesithombe ngasinye. Inani elizenzakalelayo ngu-4,194,304.
• --allowed-host ivumela elinye igama le-HTTP Host, njengalelo elisetshenziswa yiproksi. Le nketho ingaphindwa.
• --no-webui ikhubaza ikhasi lengxoxo lesiphequluli.

Isibonelo, umyalo olandelayo ubeka ukwabiwa kwe-Metal ku-28 GB futhi ukhawulele umongo ku-100K:

splash serve \
  --model incoai/Qwen3.8-27B-Splash \
  --max-memory 28G \
  --max-context 100K

Uma imodeli ingakwazi ukungena kwimemori etholakalayo, i-Splash iphrinta ukuhlukaniswa kwesabelomali sememori bese iyama esikhundleni sokuqala iseva engasebenzi. Ukwehlisa izidingo zomongo noma zememori kungasiza, kodwa izisindo zemodeli nesimo esidingekayo sokusebenza kusamele kungene.

Vikela i-API Yasendaweni

Ukuqinisekiswa kukhutshaziwe ngokuzenzakalelayo. Ungadinga ukhiye ngokusetha i-SPLASH_API_KEY ku-shell eqala iseva:

export SPLASH_API_KEY='replace-with-your-key'
splash serve --model incoai/Qwen3.8-27B-Splash

Setha leyo variable yemvelo efanayo ku-shell esetshenziselwa ukwethula i-coding agent ephethwe yi-Splash:

export SPLASH_API_KEY='replace-with-your-key'
splash opencode

Amaklayenti e-API angathumela ukhiye njengethokheni ye-bearer noma ngesihloko se-x-api-key. Ungaphinda unikeze ukhiye ngokuqondile ngenketho yeseva ethi --api-key. Ama-probe ezempilo nokulungela ahlala evulekile emphakathini.

Amathiphu Athuthukile

Hlela Ngokuzungeza I-Cache Yemodeli

Amaphakheji alandiwe agcinwa ku-cache ye-Hugging Face. Ukusebenzisa i-brew upgrade splash kugcina la maphekheji, izixhumanisi zamamodeli, namaseshini ama-agent, ngakho ukuthuthukisa injini akudingi ukuwasusa.

Sebenzisa Uhlu Olugunyaziwe Lwama-Host Nge-Proksi

Iseva ibophezela endaweni ethi 127.0.0.1:8000. Uma iproksi yasendaweni ithumela inani elihlukile le-HTTP Host, engeza lelo gama lomethuleli nge---allowed-host. Phinda ifulegi uma kufanele kwamukelwe amagama amaningi abamethuleli.

Khubaza Isixhumi Esibonakalayo Sewebhu Ukuze Usebenzise I-API Kuphela

Uma i-Splash izosebenzela izinhlelo zokusebenza noma ama-agent kuphela, yiqale nge---no-webui. Lokhu kuvala ikhasi lengxoxo kuyilapho kushiya ama-endpoint e-API asekelwayo etholakala.

Hlola Ama-Prompt Ngaphambi Kokwenza I-Inference

Sebenzisa i-/tokenize uma udinga ama-ID amathokheni, noma i-/apply-template uma udinga ukuhlola i-prompt yokugcina e-renderiwe. Njengoba lawa ma-endpoint engayisebenzisi imodeli, awusizo ekulungiseni usayizi womongo nokufomethwa kwe-prompt.

Hlola Ukudluliselwa Kwe-Cache Ku-SSD

Ukuqaliswa kokuhlola ku-pull request 3 kungadlulisela i-KV-cache nezimo ze-GDN ku-SSD. Yakha lelo gatsha bese uqala iseva nge---max-cache-disk 8G ukuze uyinike amandla. Lokhu kungagcina iziqalo ezingasetshenziswa kabusha lapho i-RAM ilinganiselwe futhi kunciphise umsebenzi ophindaphindwayo we-prefill.

Isici sokudlulisela ku-SSD siyahlolwa futhi asiyona ingxenye yenqubo evamile yamaphakheji echazwe ekuqaleni okusheshayo.

Ukuqonda Ukusebenza Kwe-Splash

I-Splash ithola isivinini ngokwenza iningi lesistimu ye-inference likhetheke kumodeli ngayinye. I-Speculative decoding iyindlela evamile yokwenza i-decode, hhayi imodi ongayikhetha. I-DFlash 2 draft eqondene nemodeli iphakamisa amabhulokhi amathokheni, bese imodeli eqondisiwe iwaqinisekisa ngokuhambisana.

Ama-kernel e-Metal ahlanganisiwe ahlanganiselwa ama-shape aqondile emodeli futhi afunde izisindo ezipakishwe ngokukhethekile, ezimephwe zisuka kudiski ngaphandle kwekhophi eyengeziwe. Ekuqaleni, i-Splash iphinde ibale imikhawulo yomongo, ye-cache, neyebhetshi yomshini wamanje.

Ngokwezilinganiso zephrojekthi ku-M5 Pro ene-GPU enamacore angu-16 nememori engu-48 GB, i-Qwen3.8-27B yafinyelela kumathokheni angu-74 ngomzuzwana ekwenzeni i-decode ngama-prompt amafushane futhi yabuyisa ithokheni yayo yokuqala ngo-282 ms lapho idlala kabusha umongo ogcinwe ku-cache ongu-32K. Imiphumela incike kwimodeli, i-prompt, ukuhambisana kwemisebenzi, ihardware, nezimo zememori. I-posi yokuqalisa ye-Splash ichaza indlela yokulinganisa nokuqhathanisa.

Isiphetho

I-Splash inikeza indlela elula yokusebenzisa amamodeli akhethiwe, alungiselelwe kakhulu ku-Apple silicon yakamuva. Yifake nge-Homebrew, uqale iphakheji yemodeli ye-Splash ehambisanayo, bese uxhuma ngesiphequluli, nge-coding agent esekelwayo, noma nge-API ehambisana ne-OpenAI noma i-Anthropic. Izilungiselelo ezizenzakalelayo zenza ukusetha kokuqala kube lula, kuyilapho izinketho zememori, zomongo, zokuphepha, zama-host, nezokuxhumana zisekela ukuthunyelwa okulawulwa kakhulu.