TimesFM
Forecasting & tabular · Oct 2023
Google’s zero-shot forecaster. It reads a series as a sequence of patches and predicts the next patch, the time-series version of next-token prediction, so it can forecast series it has never seen.
The statistical view
The next stretch of values given the history, one patch at a time. Each patch of 32 points plays the role a word plays for a language model.
ŷ_{t+1 … t+128} = f_θ(patch₁, …, patchₙ)
- Loss
- Mean squared error between the forecast patch and what actually happened, at every position of every training series.
- Answering
- A point forecast for the horizon, produced patch by patch and fed back in for longer horizons. Later versions add quantile forecasts.
What does the number do next?
The same machinery pointed at numbers: series become tokens or patches, and a table of labelled rows becomes context for predicting new ones.
A demonstration of the objective, computed in your browser on illustrative data. It is not TimesFM running here.
How it is trained
- 1Normalise the context window.
- 2Cut the series into input patches of 32 points; a small residual network turns each patch into one token.
- 3A decoder-only Transformer attends over the patches with causal masking.
- 4Each position predicts the next 128 points: output patches are longer than input patches.
- 5For longer horizons, feed the forecast back in as context.
Line of descent
- Oct 2023TimesFM paper — “A decoder-only foundation model for time-series forecasting”, ICML 2024.
- May 2024Open weights — The 200M checkpoint released on Hugging Face.
- Dec 2024TimesFM 2.0 — 500M parameters and a longer context.
What data teams use it for
- Baseline forecasts for thousands of series without training per series
- Demand, traffic and capacity planning
- Cold start: new products with short histories
- Available inside BigQuery for SQL forecasting
Limits
- Point forecasts first: uncertainty needs later versions or add-ons
- No external covariates in the original release
- Reads a limited window of history (512 points originally)
- Structural breaks it never saw in training, such as a new pricing policy, are invisible to it
Classic algorithms underneath
The same ideas, small enough to train in a browser.
Same objective
TabPFN
Prior Labs · Jul 2022
A foundation model for spreadsheet-sized data. You don’t train it on your table: you pass your labelled rows as context and it predicts the new rows in one forward pass, approximating Bayesian inference.
Chronos
Amazon · Mar 2024
Treats forecasting as language modelling. Values are scaled and rounded into a fixed vocabulary of bins, an off-the-shelf language model learns the resulting tokens, and sampling many continuations gives a probabilistic forecast.