LoopFormer: Elastic-Depth Looped Transformers for Latent Reasoning via Shortcut Modulation
Source
- Raw Markdown: paper_loopformer-2026.md
- PDF: paper_loopformer-2026.pdf
- Preprint: arXiv 2602.11451
- Project page: loopformer.github.io
Core Claim
LoopFormer trains looped Transformers on variable-length trajectories using shortcut consistency so representations remain useful under different compute budgets.
Relevance To This Wiki
It turns recurrent depth into an explicit elastic-depth interface, where loop count can be conditioned by the available budget rather than fixed at training and inference.
Limitations
The current evidence is language modeling and reasoning. Budget conditioning needs careful comparison against fixed-depth baselines under the same latency and memory targets.
Foundation TSFM Relevance
Adjacent to fixed-FLOPs hierarchy and dynamic compute, especially for systems that need controllable inference effort.
Links Into The Wiki
- LoopFormer
- Looped Transformers And Test-Time Memory
- Efficient Recurrent Sequence Models
- Time-Series Scaling And Efficiency
- Hierarchical Modeling with a Fixed FLOPs Budget
- Foundation Time-Series Model Research Agenda
Open Questions
- What matched-budget baseline should this source be compared against: unique-depth Transformer layers, recurrent state, explicit memory, or extra inference steps?
- Which claims transfer from token-sequence reasoning to multivariate time-series state tracking, event streams, or action-conditioned world models?
- Does shortcut consistency remain useful under hard serving latency budgets, or does trajectory training add overhead that only pays off offline?