Alex Open Research Wiki

Separable Sum of Gaussians (SSOG)

2 min read

Separable Sum of Gaussians (SSOG)

Summary

SSOG is an experimental Vision Transformer token mixer proposed by Raphael Pisoni. It replaces query-key dot-product scores with a small mixture of two-dimensional Gaussian fields per head. The fixed variant learns geometric routing fields; the steered variant predicts bounded query-conditioned changes to atom centers, widths, and mixture weights. Gaussian separability permits vertical and horizontal value-aggregation passes instead of materializing a dense score matrix.

Role In The Wiki

SSOG is an upstream architecture hypothesis for structured temporal mixing, not direct time-series or world-model evidence. Its useful question is whether a compact coordinate-aware prior plus bounded content steering can replace unrestricted pairwise matching when the data have meaningful geometry.

A time-series adaptation would need causal/asymmetric lag fields, irregular timestamp support, and separate temporal/channel/topology routing. It must be judged on preservation of rare regimes, delayed events, exogenous variables, action history, control inputs, and interventions, not only average forecast accuracy or symbolic FLOPs.

Evidence Boundary

  • The primary source is a 2026 author blog/X/code package, not a paper or peer-reviewed result.
  • The author reports large CIFAR-100 gains and small ImageNet-1k gains over a matched dot-product ViT, with fewer tabulated parameters/FLOPs.
  • The public code exposes the mechanism and passes same-resolution smoke tests, but does not include the ImageNet pipeline, checkpoints, raw logs, resolution-transfer utility, optimized kernels, or a detected license.
  • The displayed global mixture-softmax equation and the released per-atom-normalized factorized implementation are not generally the same operator.

Official Artifacts

Graph View

Backlinks