Revenge of Monosemanticity
Summary
A synthetic-regression study of how dense MLP neurons learn cluster-specific predictive directions without an explicit expert router. The name identifies a paper and its mechanism, not a released foundation-model checkpoint.
Mechanism And Evidence
- Ordinary and gated two-layer MLPs learn local predictive features on well-separated Gaussian mixture data.
- Monosemanticity is measured as alignment with known synthetic predictive directions among output-important neurons; it is not human-semantic interpretability.
- ReGLU/SwiGLU improve multi-cluster sample-efficiency curves, but equal hidden width does not match parameters or FLOPs.
- Clustering trained first-layer representations enables local Laplace/RFM predictors. Thus the finding concerns the limitation of a single global metric, not every use of kernels.
- Population-gradient-flow specialization and constrained-ERM sample complexity are separate results under different assumptions. The latter’s RFM lower bound is for an idealized population-AGOP comparator, not all empirical RFM algorithms.
Status And Artifacts
arXiv 2608.24007v1, 2026-08-25, by Amirhesam Abedsoltan, Enric Boix-Adsera, Fivos Kalogiannis, and Mikhail Belkin. As checked on 2026-09-09, it is a preprint with no verified peer-reviewed acceptance or official code/weights release. See the source page for provenance, numerical evidence, theorem qualifications, and retained artifacts.
Relation To Foundation TSFM Agenda
The source-level mapping treats this as adjacent evidence for regime-specific predictive representations and as a warning against equating one global feature metric with all useful learned structure. It does not demonstrate time-series state maintenance, conditional-compute savings, or action-conditioned world modeling.