Understanding and Applying the Recurrent Looped Transformer Architecture

What Is the Recurrent Looped Transformer?

The Recurrent Looped Transformer (RLT) is a research architecture designed to carry latent computation across every token in a sequence. A causal encoder constructs global key–value memory, while a recurrent decoder combines that memory with sliding-window attention and feedback from its previous final hidden state.

This creates an extensible temporal computation path. If the decoder has L_D layers, processing t tokens produces a recurrent path through tL_D decoder blocks, even though the model executes only a fixed number of blocks for each token.

In this project, “infinite temporal depth” means that the computation path can grow with sequence length. It does not mean that the model performs infinite computation within a single token.

Key Design Principles

1. Latent reasoning across time

The final decoder output for one token becomes an input to the next token. Consequently, the model can propagate a latent state throughout the prompt and generated response instead of treating the prompt–response boundary as a reset point.

2. Model and hardware co-design

The proposed design exposes several possible optimization opportunities:

• Parallel causal-encoder computation over known tokens
• Batching across independent sequences
• Reuse of encoder-derived memory
• Activation checkpointing around recurrent decoder execution

These are design opportunities and research goals. The report does not claim measured large-scale hardware speedups.

3. Shared semantics for training and reinforcement learning

Pretraining, supervised fine-tuning, sampling, and current-policy reinforcement learning replay all use the same complete-state transition. This state includes prompt recurrence as well as the decoder's sliding-window key–value caches.

How the Architecture Works

For a token x_t, the causal encoder produces a representation e_t and encoder-derived global memory M_≤t. The decoder state contains both the recurrent output and the layerwise sliding-window attention cache:

H_t = (s_t, C_t^D)
H_0 = (s_*, empty)

The state transition is:

(s_t, C_t^D) = D_phi(
  Merge(e_t, s_(t-1)),
  M_≤t,
  C_(t-1)^D,
  t
)

The next-token distribution is computed from the final recurrent output:

p(x_(t+1) | x_(1:t)) = softmax(W_o RMSNorm_o(s_t))

Three forms of context

• Global encoder context: Decoder cross-attention reads encoder memory only through the current position, preserving causal behavior.
• Local decoder memory: Sliding-window attention reads recent decoder key–value entries and the current token's entry. A window of size W includes the current token, leaving at most W - 1 historical entries for the next update.
• Temporal feedback: The previous final decoder output is merged with the current encoder representation.

Neither the recurrent output nor the decoder cache resets when the sequence crosses from the prompt into the response.

Concrete model configuration

The report describes a configuration with 48 encoder layers and 48 decoder layers. Compatible attention and feed-forward weights are shared across stages. Decoder blocks additionally perform cross-attention over encoder memory, so identical layer counts do not imply identical floating-point operation counts.

Installation and Repository Setup

The README presents an architectural report rather than a documented software package. It does not provide dependency requirements, an installation command, a public training script, or a runnable API. You can still clone the repository to read the paper and supporting material:

git clone https://github.com/yifanzhang-pro/recurrent-looped-tranformer.git
cd recurrent-looped-tranformer

Start with the included Recurrent_Looped_Transformer.pdf. You can also visit the project website and review the prefill–decode kernel mismatch note.

Because the README does not document an executable implementation, package installation instructions or ready-to-run model code should not be inferred from the repository description.

Basic Usage: Following the RLT Execution Model

The most useful way to apply the report is as an implementation blueprint. The following conceptual pseudocode summarizes one state transition; it is not a repository-provided API:

state = (initial_recurrent_state, empty_decoder_cache)

for each token x_t:
    e_t = causal_encoder_representation(x_t)
    memory = encoder_memory_through_position(t)

    merged = Merge(e_t, state.recurrent_output)
    state = recurrent_decoder(
        merged,
        memory,
        state.decoder_cache,
        position=t
    )

    next_token_distribution = output_head(state.recurrent_output)

Prompt prefill

1. Run the causal encoder over the known prompt tokens as a batch.
2. Recur through every prompt token with the decoder.
3. Construct the decoder sliding-window key–value cache during this recurrence.
4. Retain the final recurrent output and cache for generation.

Running only the encoder during prompt prefill would not reproduce the complete RLT state described in the report.

Autoregressive generation

1. Use the preceding state to compute the distribution for the next token.
2. Sample a token from that distribution.
3. Incrementally encode the sampled token.
4. Consume that token through the recurrent decoder exactly once.
5. Update both the recurrent output and sliding-window cache.

This ordering avoids consuming a generated token twice and preserves the same state-transition semantics used during training.

Pretraining and supervised fine-tuning

During pretraining, the encoder runs causally and the recurrent decoder uses full backpropagation through time. Every valid next-token target is supervised.

During supervised fine-tuning, state is updated on all context tokens, but the loss is applied only to assistant targets. The prompt still participates in recurrent state construction even when its tokens are not supervised.

Current-Policy Reinforcement Learning Replay

Current-policy replay must reconstruct the complete sequence history under the current model weights. This includes recurrent outputs, encoder memory, and all decoder sliding-window caches.

1. Rebuild encoder representations and memory using the current parameters.
2. Replay every prior token to reconstruct the recurrent state and decoder caches.
3. Evaluate an action's probability before consuming that action token.
4. Consume the action once to advance to the next state.

Parameter-dependent caches become stale after a weight update, so exact replay requires rebuilding them. Full gradients pass through recurrent outputs, decoder key–value entries, and encoder memory. Detaching any of these components is a gradient approximation.

Behavior log-probabilities must correspond to the distribution actually used for sampling. Exact importance sampling additionally requires support coverage. Shared execution semantics reduce structural prompt-boundary mismatch, but they do not guarantee numerical parity between kernels or an unbiased off-policy objective.

Preliminary Synthetic Results

The README reports independent proof-of-concept state-tracking experiments using a small implementation with approximately 79,000 parameters and three random seeds. Models were trained on sequences of 32 operations and evaluated on as many as 128 operations, with 2,048 test programs for each task and length.

Parity task

• At 32 operations, RLT reached approximately 100% final-state accuracy.
• At 64 operations, it reached approximately 82%.
• At 128 operations, it reached 60.8%.
• Chance accuracy for parity is 50%.

Five-state transition task

• At 32 operations, RLT reached approximately 100% final-state accuracy.
• At 64 operations, it reached approximately 49%.
• At 128 operations, it reached 20.7%.
• Chance accuracy for this task is 20%.

The results show that RLT fit both tasks at the training length, but its accuracy declined on longer sequences. Values marked as approximate were read from the original figure, while the 128-operation RLT values were numerically labeled.

Parameter and data budgets were matched, but floating-point operation budgets were not. These experiments are synthetic demonstrations rather than evidence of large-scale reasoning improvements, hardware acceleration, or reinforcement learning scaling.

Advanced Implementation Tips

• Treat the complete state as indivisible: Preserve both s_t and C_t^D. Restoring only the recurrent vector does not reproduce the documented transition.
• Do not reset at response boundaries: Prompt recurrence and its decoder cache must flow directly into response generation.
• Define the window precisely: A sliding window of size W includes the current token, so only W - 1 historical entries remain available for the next update.
• Rebuild caches after updates: Current-policy replay requires caches generated by the current weights rather than cached values from an earlier policy.
• Keep sampling and replay order aligned: Evaluate a generated action from the preceding state, then consume it once.
• Distinguish exact gradients from approximations: Detaching recurrent state, decoder caches, or encoder memory changes the gradient path.
• Measure compute separately: Equal parameter or data budgets do not establish equal FLOPs, especially because decoder layers add encoder-memory cross-attention.
• Validate numerical behavior: Shared high-level execution semantics do not automatically eliminate differences between prefill and decode kernels.

Conclusion

Recurrent Looped Transformer combines global causal encoder memory, local decoder sliding-window attention, and token-to-token recurrent feedback. Its central implementation rule is consistency: every prompt and response token must participate in the same complete-state transition across pretraining, fine-tuning, generation, and current-policy replay.

The preliminary experiments suggest useful length generalization on small state-tracking tasks, but performance still declines beyond the training length. The architecture should therefore be viewed as a research proposal and proof of concept rather than a validated large-scale reasoning system.