APUS-OpenJev-v1 Uyasebenza

I-APUS-OpenJev-v1 iwuchungechunge lwamamodeli amakhulu e-System 1 okwenza izinqumo eceleni kwedivayisi, olukhishwe njenge-open source yi-APUS AI Lab futhi olufana ne-Jev evaliwe ye-TypeSafe AI. Itholakala ngezisindo ze-4B, 9B kanye ne-35B-A3B, futhi ingakhetha ngokushesha isenzo esilandelayo emisebenzini efana nokuzenzakalela kwesiphequluli, ukugcwalisa amafomu kanye nokuqondisa izinqubo. Imodeli yethula okokuqala ukujula kokubala okuguquguqukayo kwe-effort="low/high": i-low isebenzisa izendlalelo ezingu-16 kuphela futhi ihlanganisa i-candidate-aware projection, okuletha isivinini esikhulu nezindleko eziphansi; i-high isebenzisa ukujula okuphelele ukuze inikeze ukunemba okuphezulu.

Ukwenza izinqumo ngezenzo ezikhethiwe: Ngokusekelwe emsebenzini, kumongo wekhasi nasezincazelweni zezenzo ezikhethiwe, ikhipha isenzo sebhizinisi esilandelayo.
Ukuzenzakalela kwesiphequluli: Ibona ulwazi lwe-DOM olufana nezinkinobho, izinkambu zokufaka nezinto zokuzulazula, isize i-Agent ekwenzeni imisebenzi emakhasini ewebhu.
Ukuqondisa imithetho yebhizinisi: Ifanele izimo zokwenza izinqumo ezifana nokwabela amathikithi esevisi yamakhasimende, ukugunyaza izinqubo nokuthumela amafomu.
Ukulawula ukubala okuguquguqukayo: Isebenzisa i-effort="low/high" ukushintsha phakathi kokuphuma okuphezulu nokusebenza kwekhwalithi ephezulu.
Ukukhiqiza umbhalo wamafomu: Ngaphandle kwezinqumo ezinjengezinketho eziningi, ingakhiqiza namagama okusesha, umbhalo wokufaka nokunye okuqukethwe okuvulekile.
Ukufakwa kwasendaweni kwezinga lama-millisecond: Isekela i-GPU eyodwa kanye nokusetshenziswa kwesevisi nge-vLLM, kunciphisa ukubambezeleka nezindleko zezingcingo ze-API zefu.
Ukukhetha osayizi bamamodeli amaningi: Ihlinzeka ngezinguqulo ze-4B, 9B kanye ne-35B-A3B, ezibhekiswe eceleni kwedivayisi okulula, ekusetshenzisweni okujwayelekile kokukhiqiza nasezidingweni zokusebenza okuphezulu ngokulandelana.

Landela i-WeChat uphendule ngokuthi “开源” ukuze ujoyine iqembu lokuxhumana lamaphrojekthi e-AI open source
Isixhumi esibonakalayo sezinqumo solimi lwendabuko: Imodeli ithatha inhloso yomsebenzi, umongo nezincazelo zezenzo ezikhethiwe njengokufaka ngolimi lwemvelo, bese ibopha isenzo ngasinye esikhethiwe ne-Token emfushane efana no-A, B noma C. Ngesikhathi sokucabanga, imodeli ibala kuphela amathuba alawa ma-Token asemthethweni, bese uhlelo olusingathayo luwaguqulela ezenzweni zebhizinisi ezithile.
Ukujula kokubala okuguquguqukayo kwe-effort="low/high": Iqoqo elifanayo lezisindo zemodeli linezindawo ezimbili zokwenza izinqumo: i-effort="low" isebenzisa kuphela izendlalelo zokuqala ezingu-16 ze-Transformer, ihlanganiswe ne-candidate-aware output projection, ukuze iqedele izenzo ezivamile eziphindaphindwayo ngamandla amancane okubala; i-effort="high" yenza ukucabanga okuphelele kwezendlalelo ezingu-32, okulungele izenzo ezinobungozi obukhulu ezifana nezinkokhelo nokuqinisekiswa kwezimvume. Lokhu kwenza uhlelo lwe-Agent lwabele izinsiza zokubala ngokuguquguqukayo kuye ngokubaluleka kwesenzo nezinga lokuzethemba.
Ukuqeqeshwa kwangemva kokuqeqesha okuhlangene phakathi kokujula nokuzifundisa: Ukuze indawo yokuphuma engajulile iqhubeke nokwenza izinqumo ezithembekile ngemva kokunciphisa ukubala ngesigamu, imodeli isebenzisa ukugadwa ngamalebula angempela ezindleleni ezingajulile nezijulile ngesikhathi esisodwa, iphinde isebenzise ukusatshalaliswa kwamathuba kokuphumayo okujulile “njengamalebula athambile”, ikudlulisele endleleni engajulile nge-KL divergence. Ukuze umthamo onganele wendlela engajulile ungadonsi phansi imodeli ejulile, ukuqeqeshwa kuphinde kusebenzise ukuvimba ama-gradient ekuphumeni okujulile, okwenza ikwazi ukuzilungiselela futhi isebenze njengothisha wendlela engajulile.
I-candidate-aware output projection: Njalo lapho imodeli enkulu yendabuko ikhiqiza i-Token eyodwa, kufanele icubungule cishe amagama angu-248,000 esichazamazwini, kuyilapho izinqumo ezihlelekile zivame ukuba nezenzo ezimbalwa kuphela. Ngokukhipha ngokuqondile imigqa yezisindo zokuphumayo ehambisana nama-Token akhethiwe, ukubala kwe-matrix kuncishiswa kusukela kusichazamazwi sonke kuya kuma-candidate angu-K. Lokhu kugcina umqondo wamathuba amakhandidethi ulingana, kuyilapho kunciphisa kakhulu ukubala kwe-output head kanye nezindleko zokufinyelela inkumbulo yevidiyo.

Lungisa imvelo: Faka i-PyTorch esekela i-CUDA, i-Transformers kanye ne-huggingface_hub, futhi uqinisekise ukuthi idivayisi noma iseva inememori eyanele ye-GPU.
Landa imodeli: Sebenzisa i-hf download apus-ailab/APUS-OpenJev-v1 --include "9B-3000/*" --local-dir ./APUS-OpenJev-v1 ukuze ulande ifolda yemodeli edingekayo.
Ngena kufolda yemodeli: Shintshela kufolda engaphansi yemodeli elandiwe ukuze usebenzise ngokuqondile ingxenye ye-openjet_runtime yokucabanga efakiwe.
Layisha imodeli: Sebenzisa i-OpenJet.from_pretrained(".", device="cuda:0") ukuze ulayishe imodeli ku-GPU ethile.
Yakha isicelo sesinqumo: Faka inhloso yomsebenzi, umongo wamanje kanye nezenzo ezikhethiwe ezinkambini ezintathu: instructions, state kanye ne-candidates.
Yenza isinqumo esisheshayo: Shayela i-model.decide(request_data, effort="low") ukuze uthole izenzo ezivamile eziphindaphindwayo ngezindleko eziphansi zokubala.
Yenza isinqumo esijulile: Shayela i-model.decide(request_data, effort="high") ukuze uthole umphumela ophelele wokucabanga ofanele izenzo ezibalulekile ezifana nokukhokha nokuthumela.
Khiqiza umbhalo wokuhamba komsebenzi: Ezimo ezidinga amagama okusesha noma okuqukethwe kwamafomu, shayela i-model.generate_text() ukuze ukhiqize umbhalo othile.
Sebenzisa isevisi yokukhiqiza: Qalisa i-python deployment/serve_vllm.py --port 8000 ukuze uqale isixhumi esibonakalayo esihambisana ne-OpenAI nge-vLLM.

Ukunemba okuphezulu kwezinqumo: Kusilinganiso esisemthethweni esigcinwe sinemibuzo engu-80, inguqulo ye-9B ifinyelela ku-85%, ngaphezu kuka-82.5% we-Jev commercial API.
Isivinini sokuphendula esikhulu kakhulu: Emzileni othile wokufakwa kwe-vLLM endaweni, ukubambezeleka kwe-P50 kwe-9B kwehle kwaba ngu-25.58ms kuphela, kanti i-4B yaba ngu-18ms.
Izinsiza zokubala zingalungiswa ngokuguquguqukayo: Imodi yokuqala ye-effort="low/high" ivumela ukushintsha ngokukhululekile phakathi kwesivinini nokunemba kuye ngobungozi besenzo.
Izindleko eziphansi ezenzweni eziphindaphindwayo: I-effort="low" isebenzisa izendlalelo ezingu-16 kuphela futhi isebenzisa i-candidate-aware output projection, okulungele izinqumo eziphindaphindwayo ezifana nokuskrola nokuchofoza okulandelayo.
Izenzo ezibalulekile zithembeke kakhulu: I-effort="high" yenza ukujula okuphelele kwenethiwekhi, okulungele izimo ezinobungozi obukhulu ezifana nokuqinisekiswa kwenkokhelo nokugunyazwa kwezimvume.
Ivimbela amaphutha efomethi yokuphumayo: Isebenzisa isixhumi esibonakalayo sezinqumo solimi lwendabuko, ivalele ukuphumayo eqoqweni lamakhandidethi elichazwe kusengaphambili futhi iqede imibono eyiphutha yefomethi ye-JSON.
Ikhono eliqinile lokuthuthela emisebenzini yebhizinisi: Imodeli efanayo ingashintshwa ngemibhalo yolimi lwemvelo ukuze ivumelane nemisebenzi ehlukene efana nokusebenza kwesiphequluli, ukwaba izicelo zamakhasimende kanye nokubuyekeza imithetho.

Ilabhulali yamamodeli ye-HuggingFace: https://huggingface.co/apus-ailab/APUS-OpenJev-v1

Ukuzenzakalela kwesiphequluli: Ngokusekelwe ku-DOM yekhasi lewebhu, ezinkinobheni nasezinkambini zokufaka, inquma ngokushesha isenzo esilandelayo esifana nokuchofoza, ukuskrola noma ukuya kwenye indawo.
Ukuqondisa izinqubo zebhizinisi: Ngesikhathi sokugunyaza, ukucubungula amathikithi noma ukudlulisa amafomu, ikhetha ngokuzenzakalelayo indlela yokucubungula efana nokuthumela, ukubuyisela noma ukudlulisela komunye umuntu ngokusekelwe emithethweni yebhizinisi.
Ukwabela amathikithi esevisi yamakhasimende: Ihlanganisa umbuzo womsebenzisi, ulwazi lomlando neminyango engase icubungule, bese inquma ngesikhathi sangempela ithimba lesevisi noma isenzo esifanele kakhulu.
Ukubuyekezwa kwezezimali nokuhambisana nemithetho: Isebenzisa ukwahlulela okujulile kwe-effort="high" ezenzweni ezinobungozi obukhulu, ibone ukuthengiselana okungavamile, ubungozi bezimvume noma ukugeleza kwedatha okwephula imithetho.
Umsizi ohlakaniphile oseceleni kwedivayisi: Ifaka imodeli elula kumadivayisi endawo noma kumaseva ayimfihlo ukuze inikeze imisebenzi ye-Agent ephindaphindwayo amandla ezinqumo anokubambezeleka okuncane futhi avikele ubumfihlo.