Chronos
Forecasting & tabular · Mar 2024
Treats forecasting as language modelling. Values are scaled and rounded into a fixed vocabulary of bins, an off-the-shelf language model learns the resulting tokens, and sampling many continuations gives a probabilistic forecast.
The statistical view
A categorical distribution over the bin of the next value, given the bins before it: every step is a histogram forecast.
p(zₜ₊₁ | z₁ … zₜ), zₜ = bin(xₜ / s), s = mean |x| over the context
- Loss
- Cross-entropy over bins, exactly as for words. It ignores that bin 41 sits next to bin 42, yet still learns smooth distributions.
- Answering
- Sample 20 or more future paths, map bins back to values and rescale, then read the median and quantiles off the samples.
What does the number do next?
The same machinery pointed at numbers: series become tokens or patches, and a table of labelled rows becomes context for predicting new ones.
A demonstration of the objective, computed in your browser on illustrative data. It is not Chronos running here.
How it is trained
- 1Mean-scale each series: divide by the average absolute value of its context.
- 2Quantise the scaled values into 4,094 uniform bins, plus padding and end-of-sequence tokens.
- 3Train a T5 language model with cross-entropy on these token sequences.
- 4Forecast by sampling token paths, mapping bins back to values and rescaling.
- 5Summarise the sampled paths as quantiles for a prediction interval.
Line of descent
- Mar 2024Chronos — “Learning the language of time series” (Ansari et al.).
- Nov 2024Chronos-Bolt — Forecasts quantiles directly, up to 250 times faster.
- Oct 2025Chronos-2 — Adds covariates and multivariate series.
What data teams use it for
- Prediction intervals out of the box
- Zero-shot forecasting for many short series
- Inventory and staffing, where quantiles matter more than the mean
- Available in AutoGluon and Amazon SageMaker
Limits
- Bins cap precision: detail finer than a bin is rounded away
- The scaled range is bounded, so extreme values are clipped
- Sampling many paths is slow for the original models (Bolt fixes this)
- Univariate only in the original release
Classic algorithms underneath
The same ideas, small enough to train in a browser.
Same objective
TabPFN
Prior Labs · Jul 2022
A foundation model for spreadsheet-sized data. You don’t train it on your table: you pass your labelled rows as context and it predicts the new rows in one forward pass, approximating Bayesian inference.
TimesFM
Google Research · Oct 2023
Google’s zero-shot forecaster. It reads a series as a sequence of patches and predicts the next patch, the time-series version of next-token prediction, so it can forecast series it has never seen.