Alex Open Research Wiki

Sundial: A Family of Highly Capable Time Series Foundation Models

4 min read

  • Normal
  • Skimmed

Sundial: A Family of Highly Capable Time Series Foundation Models

Source

Core Claim

Sundial is a family of native continuous-valued time-series foundation models that uses TimeFlow Loss, a flow-matching objective, to generate multiple probable forecasts without discrete tokenization or a fixed parametric predictive distribution.

Key Contributions

  • Introduces TimeFlow Loss for autoregressive forecasting, where a small flow-matching network is conditioned on Transformer representations to sample future numeric patches.
  • Uses continuous patch tokenization, context-level re-normalization, a decoder-only Transformer, RoPE, Pre-LN, FlashAttention, KV cache support, and multi-patch prediction for flexible zero-shot forecasting.
  • Curates TimeBench, a pretraining corpus of about one trillion time points from mostly real-world sources plus a small synthetic component.
  • Reports zero-shot results on Time-Series-Library, GIFT-Eval, and FEV, covering point forecasting and probabilistic forecasting.
  • Frames generative forecasting as a way to reduce mode collapse from MSE-style objectives and to estimate arbitrary forecast statistics from sampled trajectories.

Benchmarked Models

ModelRole In PaperNotesOfficial Artifact
Sundial-Base-128MMain released benchmarked checkpointBase member of the Sundial family; the paper lists patch size 16, context length 2880, prediction lengths 16 and 720, 12 Transformer layers, hidden dimension 768, 12 attention heads, a 3-layer TimeFlow module, and 128M parameters.thuml/sundial-base-128m

Method Notes

Sundial treats multivariate time series through a univariate pretraining format rather than explicit cross-channel dynamics. Each variable is normalized and modeled as a sequence of continuous-value numeric patches, so the model’s main inductive bias is temporal continuity and autoregressive generation rather than channel interaction.

TimeFlow Loss conditions a flow-matching sampler on each lookback representation. At inference time, Sundial starts from Gaussian noise, follows a learned velocity field for a fixed number of steps, and repeats sampling to estimate medians, quantiles, and other forecast statistics.

For the knowledge base’s world-model frame, Sundial is a passive probabilistic forecasting model. It improves generative uncertainty modeling for future numeric observations, but it does not introduce explicit action, control input, intervention, or treatment channels.

Evidence And Results

  • On Time-Series-Library long-horizon forecasting, the paper reports Sundial ahead of the compared time-series foundation models on average MSE and MAE, with the family improving as parameter count grows.
  • On GIFT-Eval, Sundial reports the best MASE among the compared zero-shot and supervised models and the second-best CRPS behind TimesFM.
  • On FEV, Sundial reports second place among zero-shot pretrained models behind Chronos while being much faster at inference in the paper’s benchmark.
  • TimeFlow ablations report better average point-forecasting results than diffusion loss and MSE loss, and better CRPS than those objectives on most evaluated datasets.
  • Data-scaling experiments compare Sundial trained on 94B, 230B, and 1032B time points, arguing that the larger TimeBench scale improves zero-shot forecasting.

Limitations

  • The paper notes possible hallucinations in generated forecasts despite larger model capacity.
  • TimeBench is weighted toward middle- and low-frequency time series, so performance on very high-frequency data is not guaranteed.
  • The released approach uses a simple Gaussian-noise sampling procedure, leaving sampling strategy and post-processing as future work.
  • Sundial’s univariate pretraining format does not explicitly model multivariate channel correlations, covariates, actions, control inputs, or interventions.
  • Autoregressive rolling for long horizons can still lead to over-smooth or unreliable predictions.
  • Expectations vs. Realities sharpens the evaluation boundary: probabilistic samples should still be reported on an MSE–realism frontier, because improved marginal realism does not by itself establish conditional calibration, valid trajectories, or correct mode probability mass.

Foundation TSFM Relevance

Agenda slotVerdictEvidenceMissing pieces
Multi-modal future distributionspartially closesTimeFlow Loss samples multiple probable numeric futures and supports medians, quantiles, and other forecast statistics.Probabilistic output is still passive observation forecasting, not latent-state or intervention-conditioned futures.
Patch size, dynamic tokenization, and point-wise numeric embeddingsadjacentSundial uses continuous numeric patches and avoids discrete value tokenization or a fixed parametric predictive head.Patch size remains fixed and does not expose point-wise embeddings or event tokens.
Native multivariate encoding and high-channel scalingwarningThe paper uses a univariate pretraining format for broad transfer.Cross-channel correlations, covariates, actions, and interventions are not first-class.
Causal structure, counterfactuals, and controlinsufficient evidenceThe paper frames Sundial as a forecasting model.Needs action-conditioned inputs and counterfactual rollout evaluation.

Open Questions

  • How much of Sundial’s gain comes from TimeFlow Loss itself versus TimeBench scale and the engineering upgrades to the Transformer backbone?
  • Can the generative sampler be made more reliable for high-frequency or long-horizon forecasts without sacrificing inference speed?
  • Would explicit multivariate, covariate, action, control input, or intervention channels preserve Sundial’s zero-shot flexibility while making it more useful as a world-model component?

Graph View

Backlinks