nanoTabPFN_looped
Summary
nanoTabPFN_looped is the proof-of-concept looped tabular foundation model variant from Is One Layer Enough?. It reuses one Transformer block six times inside a simplified nanoTabPFN architecture, keeping six effective inference steps while reducing stored parameters relative to a six-layer baseline.
What It Is
The paper trains three controlled nanoTabPFN variants with the TabICL prior codebase:
nanoTabPFN_6l: six unique Transformer layers.nanoTabPFN_1l: one non-looped Transformer layer.nanoTabPFN_looped: one Transformer layer reused six times during training and inference.
The looped variant keeps the effective compute depth of the six-layer model but stores roughly the parameters of one Transformer block. In the paper’s appendix table, nanoTabPFN_looped has about 750K parameters versus about 3.7M parameters for nanoTabPFN_6l.
Evidence
The reported result is narrow but clean: nanoTabPFN_looped performs almost identically to nanoTabPFN_6l on the paper’s small-scale tabular benchmark setup, while the single non-looped layer underperforms. The paper also reports consistent multiclass and regression appendix evidence.
Limitations
This is not a released foundation-model family with broad production evidence. The experiment uses a small nanoTabPFN architecture, two main benchmark suites, and no ensembling. It does not prove that a looped single block scales to TabPFN(2.5), TabPFN-3, TabICL-scale systems, LimiX-scale systems, time-series forecasting, or action-conditioned world models.
Relation To Looped Depth
nanoTabPFN_looped is useful because it gives static-tabular evidence for repeated depth as an inference mechanism. It should be compared with looped language models and recurrent-depth systems under matched parameter memory, expected FLOPs, latency, and state-capacity constraints. For time-series work, it is an adjacent architecture signal, not direct evidence that repeated depth preserves temporal state, exogenous variables, or action history.
Official Artifacts
- Paper source: Is One Layer Enough?
- Official code: https://github.com/amirbalef/is_one_layer_enough