Recurrent Memory Transformer
Summary
Recurrent Memory Transformer is a memory-augmented Transformer method that carries learned memory tokens between sequence segments.
Role In The Wiki
Use this page as the object card for the RMT method family. The source page carries evidence details, limitations, and agenda mapping.
Relation To Foundation TSFM Agenda
RMT is architecture background for compact recurrent state and long context. It is relevant when comparing ways to keep persistent latent state in a time-series model, but the source itself is not direct time-series or action-conditioned world-model evidence.
Evidence
Official Artifacts
- Official code: booydar/LM-RMT
- Newer official implementation: booydar/recurrent-memory-transformer
Overlap Notes
RMT is the explicit segment-memory-token branch. It differs from Titans because the memory is passed as learned tokens between chunks, not updated through a separate long-term neural memory module at test time. It differs from Mamba because the carried state is an explicit memory slot interface, not an implicit structured recurrent hidden state.
Direct descendants in this wiki are ARMT, which asks whether associative memory increases capacity and overwrite behavior, and RATE, which adapts the segment-memory contract to action trajectories.