QwenGyre Uyasebenza

I-QwenGyre iwuhlaka lwe-online reinforcement learning lwama-agent omsebenzi omude kakhulu olwenziwe yi-Qwen yase-Ali, oluqondiswe emisebenzini ethatha cishe ama-token angu-700,000 ngeseshini futhi eqhubeka amahora amaningana. I-QwenGyre isebenzisa ukuhlela okuguquguqukayo ukushintsha ama-GPU angasebenzi phakathi kokusampula nokuqeqesha ngaphandle kokuphazamisa umsebenzi, isebenzisa ukucubungula ama-trajectory ukugcina umongo wangempela, ihlole imiphumela eyingxenye, futhi isuse izimpinda zama-prefix abelwayo ukuze ilawule izindleko. Ku-Qwen3.8-2.4T, ngezinyathelo ezingu-48, inyuse izinga lokuphasa le-NL2RepoBench lisuka ku-52.48% laya ku-58.54%, yashesha ngo-1.78× kune-Async end-to-end.

I-online reinforcement learning yama-agent omsebenzi omude kakhulu: yenza ukuqeqeshwa kwe-RL ngokuqondile emisebenzini lapho ukwenziwa okukodwa kuthatha amahora amaningana futhi kuqoqwe cishe ama-token ayisigidi.
Ukuhlela ama-GPU okuguquguqukayo: ishintsha ama-GPU angasebenzi phakathi kokusampula nokuqeqesha ngesikhathi sangempela ngaphandle kokuphazamisa umsebenzi.
Ukuqondisa kabusha izicelo nokuthuthwa kwe-KV cache: lapho kushintshwa izindima zama-GPU, i-agent iyaqhubeka nokusebenza, indawo yokusebenza nesimo samathuluzi kuyagcinwa, kanti i-KV cache ithuthwa nge-RDMA ukuze kuncishiswe ukubalwa okuphindaphindiwe.
Ukuqeqesha okusakazwayo nokwengezwa kwama-node amasha phakathi nomsebenzi: i-core cell igcina amapharamitha agunyaziwe, kanti ama-satellite cell angalanda i-snapshot nganoma yisiphi isikhathi ajoyine ibhetshi eqhubekayo, athathe ama-micro-batch ngokwentuthuko.
Isihlahla sama-trajectory nokugcinwa komongo wangempela: isebenzisa i-Token-in Token-out ukurekhoda ama-input nama-output anembile ocingweni ngalunye, iwahlele abe yisihlahla sama-trajectory ngokusebenzisa ama-prefix abelwayo, ukuze i-output ihlale ihambisana nomongo eyakhiwe kuwo.
Ukulinganisa okuyingxenye (Partial Scoring): uma umsebenzi ongakaqedwa ngesikhathi esinqunyiwe ukhiqiza umphumela ongahlolwa, usalinganiswa ngokwemithetho yomsebenzi futhi usetshenziselwe ukuqeqesha.
Isabelomali sama-trajectory nokulinganisa ukulahleka: ukwenziwa ngakunye kugcina ama-trajectory angadluli ku-Jmax, kanti i-prefix eyabelwayo ibalwa kanye kuphela ekulahlekeni, ukuze imisebenzi enamagatsha amaningi ingabi nesisindo esidlulele.

Ukuhlela okuguquguqukayo (Elastic Scheduler): ihlukanisa ama-GPU abe ngamayunithi amaningana okubala afanayo; i-cell ngayinye ingasebenzisa i-SGLang ukuhlinzeka ngensizakalo yokusampula, noma isebenzise i-Megatron ukuthwala ikhophi ephelele yokuqeqesha. Isihleli silandelela “izinga lamanzi”, okungukuthi inani lemisebenzi engaqediwe. Uma umthamo osele wokusampula wanele ukusekela imisebenzi esaqhubeka futhi ukunikezwa kwedatha yokuqeqesha kwanele, ama-cell ashintshelwa ekuqeqesheni ngokulandelana. Ngesikhathi sokushintsha, i-proxy iqondisa kabusha izicelo zemodeli enjinini entsha yokwenza inference futhi izame futhi izicelo ezingakaqedwa; izinqubo zangemuva namaqembu okuxhumana akudingeki akhiwe kabusha. I-core cell iphethe izisindo ezigunyaziwe nesimo se-optimizer, kanti ama-satellite cell alanda ama-snapshot wamapharamitha nge-RDMA, ajoyine ukuqoqwa kwama-gradient kwebhetshi yamanje, bese i-core ihlanganisa futhi ibuyekeze konke. Lokhu kwenza ukuqeqesha kukhule kancane kancane njengoba idatha ilungela, kuyilapho ukusampula nokuqeqesha kudlulela kakhulu.
Ukucubungula ama-trajectory (Trajectory Processor): i-QwenGyre isebenzisa i-proxy ukurekhoda ama-token ID anembile e-input/output ngayinye yemodeli kanye nama-log probability enqubomgomo yokuziphatha. Iphinda isebenzise ama-prefix abelwayo, yakhe amagatsha ezindaweni zokuhlukana, futhi ihlele konke kube yisihlahla sama-trajectory, iqinisekise ukuthi i-output ngayinye ihambisana nomongo wayo wangempela ngesikhathi ikhiqizwa. Esigabeni sokwamukela ukuqeqeshwa, ikhetha ama-trajectory angeqi ku-Jmax ekwenzeni ngakunye ngokubaluleka kwendima; ama-token akhiqizwe yimodeli yenqubomgomo kuphela abamba iqhaza ekulahlekeni, kanti ithagethi eyabelwayo ibalwa kanye kuphela. Ukulahleka kuqala kulinganiswa ngokuthatha isilinganiso sama-token akhethiwe ngaphakathi kokwenziwa, bese kuthathwa isilinganiso phakathi kokwenziwa, ngaleyo ndlela isisindo sokwenziwa singanciki enanini lamagatsha aso. Ngokuhambisana ne-Partial Scoring, izingxenye eziqediwe zemisebenzi edlule isikhathi ziyahlolwa futhi zilinganiswe, ukuze kusetshenziswe kakhulu amasiginali asebenzayo ayivelakancane futhi abizayo emisebenzini emide kakhulu.

Okwamanje i-QwenGyre ishicilelwe ngendlela yephepha locwaningo; u-Ali akakakashicileli ikhodi noma iphakheji yokufaka.

Ukuqaliswa kokuqala kwe-online RL yomodeli osezingeni eliphezulu emisebenzini emide kakhulu: ku-Qwen3.8-2.4T, kuqede umsebenzi owodwa ongaba ama-token angu-700,000, kwathi ngezinyathelo ezingu-48 kwenyusa izinga lokuphasa le-NL2RepoBench lisuka ku-52.48% laya ku-58.54%.
Ukusebenza kahle kokuqeqesha kukhule kakhulu: ngaphansi kwesabelomali esifanayo sama-GPU, ukusheshisa kwe-end-to-end kufinyelela ku-1.78× uma kuqhathaniswa ne-Async, no-1.85× uma kuqhathaniswa ne-Colocate; isikhathi sokuqeqesha ngezinyathelo ezingu-48 sehle sisuka emahoreni angu-134.55 saya emahoreni angu-75.42.
Ukuhlela okuguquguqukayo okungaphazamisi umsebenzi: lapho ama-GPU eshintsha ngamandla phakathi kokusampula nokuqeqesha, ukwenziwa kwe-agent, indawo yokusebenza nesimo samathuluzi kugcinwa ngokuphelele; ukushintsha kuthatha imizuzwana embalwa kuphela, okunganakwa uma kuqhathaniswa nokwenziwa okuthatha amahora.
Ukusetshenziswa okuphezulu kwama-GPU: ukwabiwa kwezinsiza ezingeni le-cell elinemininingwane kuqeda ukulinda nokungenzi lutho komsila omude kwimodi ye-Colocate kanye nokuqina kokuhlukaniswa okumisiwe kwimodi ye-Async, ukuze ukuqeqesha kufihlwe ngokuphelele ukusampula.
Ukuqeqesha okuthembekile kumongo wangempela: indlela ye-TITO iqinisekisa ukuthi i-output ngayinye ihambisana nomongo wayo wangempela ngesikhathi ikhiqizwa, igweme amasiginali “omongo mbumbulu” adalwa ukucindezelwa komlando noma ukuhlanganiswa komongo wama-sub-agent.

Iphepha lobuchwepheshe le-arXiv: https://arxiv.org/pdf/2609.33848

Ukukhiqiza ikhosombe lekhodi eliphelele ngolimi lwemvelo: ukwakha ikhosombe lesoftware elinamaphakheji amaningi kusukela ekuqaleni ngokusekelwe ezicacisweni zolimi lwemvelo, lapho ukwenziwa okukodwa kuthatha cishe ama-token angu-700,000 futhi kuqhubeka amahora amaningana.
Ubunjiniyela besoftware besikhathi eside emakhosombeni akhona: ukuqedela ukuthuthukiswa kwemisebenzi, ukulungisa amaphutha, nokuguqulwa kwamafayela amaningi kumakhosombe akhona.
Ukuzenzakalela kwemisebenzi eminingi yetheminali: i-agent yenza amakhulu ezingcingo zamathuluzi ngokulandelana endaweni yomugqa womyalo, iqedele imisebenzi yokumisa nokuphatha amasistimu.
Ukulungisa kabusha nokuthutha amakhosombe amakhulu ekhodi: ukuthuthukisa isitaki sobuchwepheshe phakathi kwamakhosombe, ukulungisa kabusha okuphelele, kanye neminye imisebenzi yobunjiniyela edinga ukuhlela nokwenziwa okuqhubekayo amahora amaningana.
Imisebenzi yokubambisana kwama-agent amaningi: izimo zokwenziwa ngokwezigaba lapho i-agent eyinhloko inika ama-sub-agent imisebenzi; ukucubungula ama-trajectory kungasebenzisa isakhiwo samagatsha ukulawula izindleko zokuqeqesha zemisebenzi engaphansi.