DeepInfer Uyasebenza

I-DeepInfer iwuhlaka lokucabanga olusezingeni lokukhiqiza olukhishwe yi-DeepSeek njengomthombo ovulekile, olususelwa ku-vLLM, olusekela i-V4.1 Flash, futhi olusebenza ku-NVIDIA SM100 naku-Huawei Ascend 950. I-DeepInfer isebenzisa ukuthunyelwa okuhlukanisa i-PD, isebenzisa iqoqo lenqolobane ye-3FS ukuze ifinyelele ku-96% kokushayelwa kwemininingwane yokufaka, inyuse ukusebenza kwe-Prefill izikhathi ezingu-25, kuyilapho i-SWA Bounded Replay inciphisa umsebenzi wokubala we-Prefill ngesigamu, futhi i-DSpark speculative decoding inyusa futhi throughput ngo-6%; iqoqo elilodwa lokukhiqiza licubungula ama-Token ayizigidigidi ezingu-155 ngosuku. I-deepseek-recipe ekhishwe nayo iwumkhiqizo wokufaka amakhodi nokukhipha amakhodi kwama-Token osezingeni lokukhiqiza, ophethe ukuhlanganisa amafomethi amathathu e-API: Chat Completions, Responses ne-Messages, kanye nokuvikela ekufakweni okunonya, i-tokenization ehambisanayo nokukhishwa okuqhubekayo, ukuze ukusebenza kwesixhumi esibonakalayo samamodeli azisingathwa kufane ngokuphelele ne-API esemthethweni.

Ukucabanga okusatshalalisiwe: kusekela ukusatshalaliswa kwedatha yokunaka, ukusatshalaliswa komongo kanye nokusatshalaliswa kochwepheshe be-MoE, kusekele izinsizakalo ezinkulu zokukhiqiza ezicubungula ama-Token ayizigidigidi ezingu-155 ngosuku eqoqweni elilodwa.
Ukuhambisana kwezinkundla zehadiwe: kusekela kanyekanye i-NVIDIA SM100 GPU ne-Huawei Ascend 950 NPU, kuhlanganise izimiso ezimbili ezinkulu zamandla okubala.
Ukucubungula umongo omude kahle: i-SWA Bounded Replay inciphisa izindleko zokubala kabusha uma inqolobane ingatholakali zibe ama-Token angu-128 angaguquki, kungakhathaliseki ubude bomongo nenani lezendlalelo.
Ukusheshisa nge-speculative decoding: ihlanganisa i-DSpark eyakhiwe ngaphakathi kanye nokuhlela okuqinisekisa ukuzethemba, ukuze imodeli iqhubekisele phambili ama-Token amaningi ngesikhathi esisodwa futhi inyuse kakhulu i-throughput yokukhiqiza.
Ukuhlela okulinganisa umthwalo: kusabalalisa ochwepheshe nezicelo ngokulinganayo, lapho ikhadi elimatasa kakhulu ekukhiqizeni lidlula umthwalo omaphakathi ngo-9% kuphela.
Ukuphinda kusetshenziswe inqolobane: isebenzisa iqoqo lokugcina le-3FS ukuze ifinyelele ku-96% kokushayelwa kwenqolobane yokufaka, futhi inyuse i-Prefill throughput cishe izikhathi ezingu-25.

Landela i-WeChat bese uphendula ngokuthi “开源” ukuze ujoyine iqembu lezingxoxo lamaphrojekthi e-AI omthombo ovulekile
Ukuthunyelwa okuhlukanisa i-PD: ukucabanga kuhlukaniswe kube izigaba ezimbili, i-Prefill ne-Decode: i-Prefill icubungula amashumi ezinkulungwane zama-Token wokufaka ngesikhathi esisodwa futhi idinga ukubala okuningi; i-Decode ikhiqiza i-Token eyodwa ngesikhathi, ivinjelwa umkhawulokudonsa wememori yevidiyo. I-DeepInfer ithumela lezi zigaba ezimbili kuhadiwe ehlukene: i-Prefill isebenzisa amaseva ajwayelekile anamakhadi angu-8, i-Decode isebenzisa ama-supernode anamakhadi angu-64–128, kanti inani lemishini yalezi zinhlobo ezimbili lihlelwa cishe ngo-1:1, ukuze ngayinye isebenze ngokwezici ezihamba phambili zehadiwe yayo.
Uhlelo lwenqolobane yesiqalo: esimweni sama-Agent, konke ukubizwa kwethuluzi kuthumela kabusha umlando ophelele, kanti ingxenye enkulu yawo isivele ibaliwe. Uhlelo lugcina kuphela isiqalo esifana ngokuphelele kusukela ku-Token yokuqala; ingxenye etholakele isebenzisa kabusha i-KV Cache ngokuqondile, kubalwe ingxenye entsha kuphela. Inqolobane igcinwa eqoqweni elizimele lokugcina elisatshalalisiwe le-3FS, ibhalwe ngamabhulokhi ama-Token angu-128; ithrafikhi yama-Agent igcinwa izinsuku ezingu-3, kanti ithrafikhi yezingxoxo igcinwa izinsuku ezingu-7.
SWA Bounded Replay: ezinye izendlalelo ze-V4.1 Flash zisebenzisa ukunaka kwewindi elislayidayo, lapho i-Token ngayinye inaka ama-Token angu-128 akamuva kuphela. Le KV Cache ayibhalwa kudiski eqinile, kodwa ihlala kwimemori yemishini yokubala kuphela. Uma ingatholakali, indlela endala yayidinga ukubala kabusha ngokunembile ngokusho “kwenani lezendlalelo × ubude bewindi”; i-V4.1 yona ibala kabusha ama-Token angu-128 okugcina kuphela, futhi ukunaka kwenzeka ngaphakathi kwawo, ngaleyo ndlela kwakheka inqolobane yewindi eseduze enezindleko zokubala kabusha ezingaguquki.
I-DSpark speculative decoding nokuhlela okungavumelani ngesikhathi: imojuli encane yokusalela iqagela ama-Token angu-5 alandelanayo kuqala, bese imodeli eqondiwe iqinisekisa wonke ngesikhathi esisodwa; okuqagelwe kahle kuyasetshenziswa ngokuqondile, kanti lapho kuqala khona iphutha imodeli enkulu iyalungisa, ukuze isinyathelo esisodwa sokubala siqhubekisele phambili ama-Token amaningi. Ikhanda lokuzethemba lilinganisela amathuba okwamukelwa kokuqagela ngakunye; okungenzeka kancane akusaqinisekiswa, futhi amandla okubala anikezwa ezinye izicelo. Uma kuqhathaniswa nendlela eqinisekisa ama-Token angu-5 njalo, i-throughput yokukhiqiza inyuka ngo-6%. Ngesikhathi esifanayo, i-CPU ihlela isinyathelo N+1 ngokuhambisana nokubala kwesinyathelo N kwe-GPU, iqede isikhathi sokungenzi lutho kwe-GPU phakathi kwezikhawu zokuhlela.

Qinisekisa izimfuneko zemvelo: uhlelo lwe-Linux, i-GCC ≥ 13.0, i-Python 3.12; inkundla ye-NVIDIA idinga i-CUDA 13, kanti inkundla ye-Ascend idinga i-CANN 9.2.
Clone indawo yokugcina: git clone https://github.com/deepseek-ai/DeepInfer bese ungena enkombeni yayo.
Dala indawo ehlanzekile: dala futhi uvule indawo ebonakalayo engenalutho ye-Python 3.12 ukuze ugweme ukungqubuzana kokuncika.
Khetha inkundla yehadiwe bese uyakha: sebenzisa export DEEPINFER_ACCELERATOR=nvidia (sebenzisa ascend ku-Ascend), bese usebenzisa ./build.sh ukwakha kusukela kukhodi yomthombo. Qaphela ukuthi le variable yemvelo kufanele iqhubeke isebenza endaweni ozosebenzisa kuyo kamuva.
Lungisa izisindo zemodeli: ngokuzenzakalelayo, ukusebenza kokuqala kuzolanda ngokuzenzakalelayo i-deepseek-ai/DeepSeek-V4.1-Flash-1017 ku-Hugging Face, noma usebenzise export DEEPINFER_HF_MODEL_DIR=/path/to/DeepSeek-V4.1-Flash-1017 ukukhomba ikhophi yendawo.
Qalisa isibonelo sokucabanga okungaxhunyiwe ku-inthanethi: sebenzisa python examples/offline_inference.py ukuqala ukucabanga; bheka inkomba ye-examples endaweni yokugcina ukuze uthole ezinye izindlela zokusebenzisa.

Ukusebenza okuphezulu kakhulu okuqinisekiswe ekukhiqizeni: isivele isebenza enkonzweni enkulu yokukhiqiza ye-V4.1 Flash, icubungula ama-Token ayizigidigidi ezingu-155 ngosuku eqoqweni elilodwa, ngokubambezeleka kwe-Token yokuqala okungu-400ms kuphela kanye nesivinini sokukhiqiza esingu-230 tokens/s.
Ukusebenza okuphezulu kakhulu kokuphinda kusetshenziswe inqolobane: u-96% wokufaka ushaya i-prefix cache, futhi i-Prefill throughput ngekhadi ngalinye ikhuphuka isuka ku-60,000 ingabalwa inqolobane iye ku-1,500,000 tokens/s, umehluko wezikhathi ezingu-25.
Ukwehla okukhulu kwezindleko zomongo omude: i-SWA Bounded Replay igcina inani lokubala kabusha uma inqolobane ingatholakali kuma-Token angu-128 okugcina, inciphise cishe ngesigamu ukubala okuphelele kwe-Prefill, kanti izindleko azincikile ebudeni bomongo.
Ukusebenza kokukhiqiza okuhamba phambili: i-DSpark speculative decoding, ukuhlela ngokuzethemba nokuhlela okungavumelani ngesikhathi kuvumela isinyathelo esisodwa ukuthi siqhubekisele phambili ama-Token amaningi. I-throughput yokukhiqiza ikhuphuka ngo-6% uma iqhathaniswa nohlelo lokuqinisekisa olungaguquki, futhi kukhiqizwa ama-Token angu-14K ngekhadi ngomzuzwana.
Ikhono elihle lokulinganisa umthwalo: amakhophi ochwepheshe be-MoE ahlelwa kabusha ngamandla njalo ezinyathelweni ezingu-10000; ikhadi elimatasa kakhulu ekukhiqizeni lidlula umthwalo omaphakathi ngo-9% kuphela, kanti lonke uhlelo lufinyelela ku-80% wesimo esilinganiselwe esifanele.
Ukuhambisana nezinkundla ezimbili zehadiwe: kusekela kanyekanye i-NVIDIA SM100 GPU ne-Huawei Ascend 950 NPU, futhi kungelinye lamafreyimu ambalwa kakhulu okucabanga asezingeni lokukhiqiza ahlanganisa lezi zimiso ezimbili zamandla okubala.

Indawo yokugcina ye-GitHub: https://github.com/deepseek-ai/DeepInfer
https://github.com/deepseek-ai/deepseek-recipe

Ukuthunyelwa kwangasese kwamabhizinisi: amabhizinisi athumela i-V4.1 Flash esikhungweni sawo sedatha, asebenzise i-deepseek-recipe ukuvumelanisa ukuziphatha ne-API esemthethweni, ukuze idatha ingaphumi endaweni futhi ukucabanga kuhambisane nemithetho.
Ukuvumelanisa namandla okubala akhiqizwa ekhaya: ukusekela kwendabuko kwe-Huawei Ascend 950 NPU kuhlangabezana nezidingo zezimo zikahulumeni, zamabhizinisi nezemali zokunciphisa ukuncika ku-NVIDIA.
Izinsizakalo ezinkulu zama-Agent: izinga lokushayelwa kwe-prefix cache elingu-96% linciphisa kakhulu ukubala okuphindwayo kokubizwa kwamathuluzi amaningi, lilungele amapulatifomu ama-Agent anokuncintisana okuphezulu nezindleko eziphansi zokusebenza.
Ukucubungula amadokhumenti anomongo omude: i-SWA Bounded Replay inciphisa cishe ngesigamu ukubala kwe-Prefill kombhalo omude, ilungele ukuhlaziya izinkontileka, ukuqonda amakhosombe ekhodi nezinye izimo zokufaka okude kakhulu.
Insizakalo yokucabanga ezakhelwe yona enezindleko eziphansi: i-throughput ephezulu ngekhadi kanye nesivinini sokukhiqiza esingu-230–300 tokens/s kuyenza ifanele izinkampani eziqalayo ezifuna ukushintsha ama-API akhokhelwa ngokusetshenziswa, futhi ukwakha ngezinga elikhulu kunciphisa izindleko.