AI Models
The models behind generative AI, read as statistics rather than products.
Five questions, every model
Strip away the branding and each model is a guess at a conditional distribution. The difference between them is what gets hidden and what has to be guessed.
- What comes next?
- p(xₜ | x<ₜ)
Cover everything after position t and predict it. Repeated over a corpus, this is maximum-likelihood estimation of the distribution of text.
- What is missing here?
- p(x_M | x∖M)
Hide a slice of the input and rebuild it from what remains on both sides. The point is not the guess but the representation learned along the way.
- Which of these belong together?
- p(match) ∝ exp(sim / τ)
Pull pairs that truly go together closer, push the rest apart. No labels needed beyond the pairing itself.
- What noise was added?
- xₜ = √ᾱₜ · x₀ + √(1 − ᾱₜ) · ε
Destroy the data with Gaussian noise in small steps, learn to undo one step, then run the whole chain backwards to sample something new.
- What does the number do next?
- p(y | x, context)
The same machinery pointed at numbers: series become tokens or patches, and a table of labelled rows becomes context for predicting new ones.
Thirteen years
Word vectors in 2013, masked language models in 2018, and by 2026 the same recipe pointed at banking events. Dot colour is the training objective.
- Word2Vec
- GPT
- BERT
- CLIP
- TabPFN
- Stable Diffusion
- Llama
- Claude
- TimesFM
- DeepSeek
- Gemini
- Sora
- Chronos
- PRAGMA
Next-token prediction
GPT
OpenAI · Jun 2018
The model family that made next-token prediction famous: a Transformer trained to guess the next piece of text, then tuned with human feedback to follow instructions.
Llama
Meta · Feb 2023
Meta’s open-weight language models. The same next-token objective as GPT, but the weights can be downloaded, inspected, fine-tuned and run on your own hardware.
Claude
Anthropic · Mar 2023
Anthropic’s family of language models: pre-trained on next-token prediction, then aligned with human feedback and Constitutional AI, where the model critiques answers against a written set of principles.
Gemini
Google DeepMind · Dec 2023
Google DeepMind’s natively multimodal models. Text, images, audio and video are turned into tokens in one sequence, so a single next-token model reasons across all of them.
DeepSeek
DeepSeek · Nov 2023
Open-weight models from a Hangzhou lab that matched frontier models at a fraction of the reported training compute, and showed that reinforcement learning with rule-checked rewards can teach a model to reason step by step.
Masked reconstruction
BERT
Google · Oct 2018
The model that showed reading in both directions pays off. Hide some words, predict them from the context on both sides, and the network learns representations that transfer to almost any language task.
PRAGMA
Revolut · Apr 2026
Revolut’s foundation model for banking. It reads a customer’s transactions, app activity and profile as one sequence, learns by filling in masked parts of it, and one backbone replaces task-specific models for credit scoring, fraud and more.
Contrastive learning
Word2Vec
Google · Jan 2013
The ancestor of modern embeddings. It learns one vector per word so that words used in similar contexts end up close together, famously making king − man + woman ≈ queen.
CLIP
OpenAI · Jan 2021
Learns one space for pictures and words by matching 400 million images to their captions. Because labels become sentences, it can classify images into categories it was never trained on.
Denoising diffusion
Stable Diffusion
Stability AI · Aug 2022
Generates images by learning to remove noise. Training adds noise to images step by step; the network learns to predict that noise, and generation runs the process backwards from pure static.
Sora
OpenAI · Feb 2024
OpenAI’s video model. It applies the diffusion recipe to video: compress a clip, cut it into spacetime patches, and train a Transformer to denoise all the patches together so frames stay consistent.
Forecasting & tabular
TabPFN
Prior Labs · Jul 2022
A foundation model for spreadsheet-sized data. You don’t train it on your table: you pass your labelled rows as context and it predicts the new rows in one forward pass, approximating Bayesian inference.
TimesFM
Google Research · Oct 2023
Google’s zero-shot forecaster. It reads a series as a sequence of patches and predicts the next patch, the time-series version of next-token prediction, so it can forecast series it has never seen.
Chronos
Amazon · Mar 2024
Treats forecasting as language modelling. Values are scaled and rounded into a fixed vocabulary of bins, an off-the-shelf language model learns the resulting tokens, and sampling many continuations gives a probabilistic forecast.
Logos are trademarks of their owners, shown only to identify each model’s developer.