Ukuqonda Nokusebenzisa Isakhiwo se-Recurrent Looped Transformer

Yini i-Recurrent Looped Transformer?

i-Recurrent Looped Transformer (RLT) iyisakhiwo socwaningo esiklanyelwe ukudlulisa ukubala okufihlekile kuyo yonke i-token ochungechungeni. I-causal encoder yakha inkumbulo yomhlaba wonke yama-key–value, kanti i-recurrent decoder ihlanganisa leyo nkumbulo nokunaka kwe-sliding-window kanye nempendulo evela ku-hidden state yayo yokugcina yangaphambilini.

Lokhu kwakha indlela yokubala yesikhathi enganwetshwa. Uma i-decoder inezingqimba L_D, ukucubungula ama-token angu-t kukhiqiza indlela ephindaphindayo edlula kuma-decoder block angu-tL_D, nakuba imodeli isebenzisa inani elinqunyiwe lama-block ku-token ngayinye.

Kule phrojekthi, “infinite temporal depth” kusho ukuthi indlela yokubala ingakhula ngokuhambisana nobude bochungechunge. Akusho ukuthi imodeli yenza ukubala okungenamkhawulo ngaphakathi kwe-token eyodwa.

Imigomo Eyinhloko Yokuklama

1. Ukucabanga okufihlekile ngokuhamba kwesikhathi

Okukhishwayo kokugcina kwe-decoder ku-token eyodwa kuba okokufaka ku-token elandelayo. Ngenxa yalokho, imodeli ingadlulisa isimo esifihlekile kuyo yonke i-prompt nempendulo ekhiqiziwe, esikhundleni sokuphatha umngcele we-prompt–impendulo njengendawo yokuqala kabusha.

2. Ukuklama ngokuhlanganyela kwemodeli nehadiwe

Lo mklamo ohlongozwayo uveza amathuba amaningi okuthuthukisa:

• Ukubala kwe-causal encoder ngokufanayo kuma-token aziwayo
• Ukuhlanganisa ama-batch ochungechunge oluzimele
• Ukuphinda kusetshenziswe inkumbulo ethathwe ku-encoder
• Ukugcina ama-activation checkpoint azungeze ukusetshenziswa kwe-recurrent decoder

Lawa ngamathuba okuklama nemigomo yocwaningo. Umbiko awusho ukuthi kukalwe ukusheshiswa okukhulu kwehadiwe.

3. Incazelo eyabiwe yokuqeqesha nokufunda kokuqinisa

Ukuqeqeshwa kwangaphambili, ukulungisa imodeli ngokufunda okugadiwe, ukusampula, kanye nokudlalwa kabusha kokufunda kokuqinisa kwenqubomgomo yamanje konke kusebenzisa i-state transition ephelele efanayo. Lesi simo sihlanganisa ukuphindaphindeka kwe-prompt kanye nama-key–value cache e-sliding-window e-decoder ezingqimbeni.

Indlela Isakhiwo Esisebenza Ngayo

Kwi-token x_t, i-causal encoder ikhiqiza ukumelwa e_t kanye nenkumbulo yomhlaba wonke etholwe ku-encoder M_≤t. Isimo se-decoder siqukethe kokubili okukhiphayo okuphindaphindayo kanye ne-layerwise sliding-window attention cache:

H_t = (s_t, C_t^D)
H_0 = (s_*, empty)

Ukushintsha kwesimo kungokulandelayo:

(s_t, C_t^D) = D_phi(
  Merge(e_t, s_(t-1)),
  M_≤t,
  C_(t-1)^D,
  t
)

Ukusatshalaliswa kwe-token elandelayo kubalwa kusetshenziswa okukhiphayo kokugcina okuphindaphindayo:

p(x_(t+1) | x_(1:t)) = softmax(W_o RMSNorm_o(s_t))

Izinhlobo ezintathu zomongo

• Umongo womhlaba wonke we-encoder: I-cross-attention ye-decoder ifunda inkumbulo ye-encoder kuphela ngendawo yamanje, igcina ukuziphatha kwe-causal.
• Inkumbulo yendawo ye-decoder: I-sliding-window attention ifunda ama-key–value entry akamuva e-decoder kanye ne-entry ye-token yamanje. Iwindi elinosayizi W lifaka i-token yamanje, lishiye okungenani okungu-W - 1 okungenile komlando ukuze kusetshenziswe ekuvuseleleni okulandelayo.
• Impendulo yesikhathi: Okukhiphayo kokugcina kwe-decoder kwangaphambilini kuhlanganiswa nokumelwa kwamanje kwe-encoder.

Okukhiphayo okuphindaphindayo kanye ne-cache ye-decoder akusethwa kabusha lapho uchungechunge ludlula lusuka ku-prompt luye kwimpendulo.

Ukumiswa kwemodeli okucacile

Umbiko uchaza ukumiswa okunezingqimba ezingama-48 ze-encoder nezingama-48 ze-decoder. Ama-weight e-attention ne-feed-forward ahambisanayo abiwe phakathi kwezigaba. Ama-decoder block enza futhi i-cross-attention phezu kwenkumbulo ye-encoder, ngakho inani elifanayo lezingqimba alisho inani elifanayo lemisebenzi yokubala ye-floating-point.

Ukufakwa Nokumiswa Kwendawo Yokugcina Ikhodi

I-README yethula umbiko wesakhiwo kunokuba ibe iphakheji yesofthiwe ebhalwe ngokuningiliziwe. Ayinikezi izidingo zama-dependency, umyalo wokufaka, iskripthi sokuqeqesha somphakathi, noma i-API esebenzisekayo. Noma kunjalo, ungakwazi ukwenza i-clone yendawo yokugcina ikhodi ukuze ufunde iphepha nezinto ezisekelayo:

git clone https://github.com/yifanzhang-pro/recurrent-looped-tranformer.git
cd recurrent-looped-tranformer

Qala nge-Recurrent_Looped_Transformer.pdf efakiwe. Ungavakashela futhi iwebhusayithi yephrojekthi bese ubuyekeza inothi lokungafani kwe-prefill–decode kernel.

Ngenxa yokuthi i-README ayibhali ngokusetshenziswa okusebenzisekayo, imiyalelo yokufaka iphakheji noma ikhodi yemodeli esebenza ngokushesha akufanele kuthathwe encazelweni yendawo yokugcina ikhodi.

Ukusetshenziswa Okuyisisekelo: Ukulandela Imodeli Yokusebenza ye-RLT

Indlela ewusizo kakhulu yokusebenzisa umbiko ukuwuthatha njengohlaka lokuqalisa. Le pseudocode elandelayo yomqondo ifingqa ukushintsha kwesimo okukodwa; ayiyona i-API enikezwe yindawo yokugcina ikhodi:

state = (initial_recurrent_state, empty_decoder_cache)

for each token x_t:
    e_t = causal_encoder_representation(x_t)
    memory = encoder_memory_through_position(t)

    merged = Merge(e_t, state.recurrent_output)
    state = recurrent_decoder(
        merged,
        memory,
        state.decoder_cache,
        position=t
    )

    next_token_distribution = output_head(state.recurrent_output)

Ukugcwaliswa kwangaphambili kwe-prompt

1. Qalisa i-causal encoder phezu kwama-token e-prompt aziwayo njenge-batch.
2. Phindaphinda kuwo wonke ama-token e-prompt usebenzisa i-decoder.
3. Yakha i-key–value cache ye-sliding-window ye-decoder phakathi nalokhu kuphindaphinda.
4. Gcina okukhiphayo kokugcina okuphindaphindayo kanye ne-cache ukuze kusetshenziswe ekukhiqizeni.

Ukusebenzisa i-encoder kuphela ngesikhathi sokugcwaliswa kwangaphambili kwe-prompt bekungeke kukhiqize kabusha isimo esiphelele se-RLT esichazwe embikweni.

Ukukhiqizwa kwe-autoregressive

1. Sebenzisa isimo esandulele ukubala ukusatshalaliswa kwethokheni elandelayo.
2. Khetha ithokheni kulokho kusatshalaliswa.
3. Encode ithokheni ekhethiwe ngokwandayo.
4. Dlulisa lelo thokheni ku-recurrent decoder kanye kuphela, ngendlela eqondile.
5. Buyekeza kokubili okukhiphayo kwe-recurrent kanye ne-sliding-window cache.

Le ndlela yokulandelanisa ivimbela ukuthi ithokheni ekhiqiziwe isetshenziswe kabili futhi igcina izincazelo ezifanayo zokuguquka kwesimo ezisetshenziswa ngesikhathi sokuqeqeshwa.

Ukuqeqeshwa kwangaphambili nokulungisa kahle okuqondiswa

Ngesikhathi sokuqeqeshwa kwangaphambili, i-encoder isebenza ngendlela ye-causal kanti i-recurrent decoder isebenzisa i-backpropagation through time ephelele. Wonke umgomo ovumelekile wethokheni elandelayo uqondiswa.

Ngesikhathi sokulungisa kahle okuqondiswa, isimo sibuyekezwa kuwo wonke amathokheni omongo, kodwa i-loss isetshenziswa kuphela emigomweni yomsizi. I-prompt isabamba iqhaza ekwakheni isimo se-recurrent noma amathokheni ayo engakaqondiswa.

Ukudlalwa Kabusha Kwe-Reinforcement Learning Yenqubomgomo Yamanje

Ukudlalwa kabusha kwenqubomgomo yamanje kufanele kwakhe kabusha umlando ophelele wokulandelana ngaphansi kwama-weight amanje emodeli. Lokhu kufaka okukhiphayo kwe-recurrent, inkumbulo ye-encoder, nawo wonke ama-sliding-window cache e-decoder.

1. Yakha kabusha izethulo ze-encoder nenkumbulo usebenzisa amapharamitha amanje.
2. Phinda udlale wonke amathokheni adlule ukuze wakhe kabusha isimo se-recurrent nama-cache e-decoder.
3. Hlola amathuba esenzo ngaphambi kokusebenzisa lelo thokheni lesenzo.
4. Sebenzisa isenzo kanye ukuze uqhubekele esimweni esilandelayo.

Ama-cache ancike kumapharamitha aba yizindala ngemva kokubuyekezwa kwama-weight, ngakho ukudlalwa kabusha okunembile kudinga ukuthi akhiwe kabusha. Ama-gradient aphelele adlula kokukhiphayo kwe-recurrent, kokufakiwe kwe-key–value kwe-decoder, nasenkumbulweni ye-encoder. Ukuhlukanisa noma iyiphi yalezi zingxenye kuyi-gradient approximation.

Ama-log-probability okuziphatha kufanele ahambisane nokusatshalaliswa okusetshenziswe ngempela ekukhetheni amasampula. I-importance sampling enembile idinga futhi ukumbozwa kwezindawo ezisekelwayo. Izincazelo ezifanayo zokusebenzisa zinciphisa ukungafani kwesakhiwo emingceleni ye-prompt, kodwa aziqinisekisi ukulingana kwezinombolo phakathi kwama-kernel noma i-off-policy objective engenakho ukuchema.

Imiphumela Yokuqala Yokwenziwa

I-README ibika ukuhlolwa okuzimele kobufakazi bomqondo kokulandelela isimo, kusetshenziswa uhlelo oluncane olunama-79,000 parameters cishe kanye nama-three random seeds. Amamodeli aqeqeshwe ngokulandelana kwemisebenzi engu-32 futhi ahlolwa emisebenzini efinyelela ku-128, ngezinhlelo zokuhlola ezingu-2,048 zomsebenzi ngamunye nobude ngamunye.

Umsebenzi we-parity

• Emisebenzini engu-32, i-RLT yafinyelela cishe ku-100% wokunemba kwesimo sokugcina.
• Emisebenzini engu-64, yafinyelela cishe ku-82%.
• Emisebenzini engu-128, yafinyelela ku-60.8%.
• Ukunemba okungahleliwe kwe-parity kungu-50%.

Umsebenzi woguquko lwezimo ezinhlanu

• Emisebenzini engu-32, i-RLT yafinyelela cishe ku-100% wokunemba kwesimo sokugcina.
• Emisebenzini engu-64, yafinyelela cishe ku-49%.
• Emisebenzini engu-128, yafinyelela ku-20.7%.
• Ukunemba okungahleliwe kwalo msebenzi kungu-20%.

Imiphumela ikhombisa ukuthi i-RLT yalingana nayo yomibili le misebenzi ebudeni bokuqeqeshwa, kodwa ukunemba kwayo kwehla emilandelwaneni emide. Amanani amakwe njengalinganisiwe afundwe esithombeni sokuqala, kanti amanani e-RLT emisebenzi engu-128 abhalwe ngezinombolo.

Izabelomali zamapharamitha nedatha zazilinganiswe, kodwa izabelomali zokusebenza kwamaphuzu antantayo zazingalinganisiwe. Lezi zivivinyo ziyimiboniso yokwenziwa, hhayi ubufakazi bokuthuthuka kokucabanga ngezinga elikhulu, ukusheshiswa kwehadiwe, noma ukukhuliswa kwe-reinforcement learning.

Amathiphu Athuthukile Okusebenzisa

• Phatha isimo esiphelele njengesingahlukaniseki: Londoloza kokubili s_t kanye ne-C_t^D. Ukubuyisela kuphela i-recurrent vector akukhiqizi kabusha uguquko olubhaliwe.
• Ungasethi kabusha emingceleni yempendulo: I-prompt recurrence ne-cache yayo ye-decoder kufanele kudlulele ngqo ekukhiqizweni kwempendulo.
• Chaza iwindi ngokunembile: I-sliding window yosayizi W ifaka ithokheni yamanje, ngakho kusele okufakiwe komlando okungu-W - 1 kuphela ukuze kusetshenziswe ekuvuseleleni okulandelayo.
• Yakha kabusha ama-cache ngemva kokubuyekezwa: Ukudlalwa kabusha kwenqubomgomo yamanje kudinga ama-cache akhiqizwe ngama-weight amanje, hhayi amanani agcinwe kunqubomgomo yangaphambilini.
• Gcina ukuhleleka kokukhetha amasampula nokudlalwa kabusha kuhambisana: Hlola isenzo esikhiqiziwe usesimweni esandulele, bese usisebenzisa kanye.
• Hlukanisa ama-gradient anembile kuma-approximation: Ukuhlukanisa isimo se-recurrent, ama-cache e-decoder, noma inkumbulo ye-encoder kushintsha indlela ye-gradient.
• Linganisa ukusetshenziswa kwekhompyutha ngokuhlukile: Izabelomali ezilinganayo zamapharamitha noma zedatha azisunguli ama-FLOPs alinganayo, ikakhulukazi ngoba izendlalelo ze-decoder zengeza i-cross-attention enkumbulweni ye-encoder.
• Qinisekisa ukuziphatha kwezinombolo: Izincazelo ezifanayo zokusebenzisa ezisezingeni eliphezulu azisusi ngokuzenzakalelayo umehluko phakathi kwama-kernel e-prefill nama-kernel e-decode.

Isiphetho

I-Recurrent Looped Transformer ihlanganisa inkumbulo ye-encoder ye-causal yomhlaba wonke, i-sliding-window attention yendawo ye-decoder, kanye nempendulo ye-recurrent esuka kuthokheni iye kwelinye. Umthetho wayo oyinhloko wokusebenzisa ukuhambisana: wonke amathokheni e-prompt nawezimpendulo kufanele abambe iqhaza ekuguqukeni kwesimo esiphelele okufanayo ngesikhathi sokuqeqeshwa kwangaphambili, sokulungisa kahle, sokukhiqiza, kanye nokudlalwa kabusha kwenqubomgomo yamanje.

Izivivinyo zokuqala ziphakamisa ukwenziwa jikelele kobude obuwusizo emisebenzini emincane yokulandelela isimo, kodwa ukusebenza kusalokhu kwehla ngaphezu kobude bokuqeqeshwa. Ngakho-ke le architecture kufanele ibhekwe njengesiphakamiso socwaningo nobufakazi bomqondo, hhayi njengohlelo lokucabanga olukhulu oluqinisekisiwe.