DeepSeek
Next-token prediction · Nov 2023
Open-weight models from a Hangzhou lab that matched frontier models at a fraction of the reported training compute, and showed that reinforcement learning with rule-checked rewards can teach a model to reason step by step.
The statistical view
The next-token distribution, as in GPT. The R1 models then learn a policy that writes a chain of reasoning before the answer, rewarded only when the final answer checks out.
Aᵢ = (rᵢ − mean(r₁ … r_G)) / std(r₁ … r_G)
- Loss
- Pre-training: next-token cross-entropy plus multi-token prediction. Reasoning: group relative policy optimisation (GRPO), which scores each of G sampled answers against its own group (the formula above) instead of training a separate value model.
- Answering
- A long run of reasoning tokens, then the answer. More reasoning tokens means more computation spent per question.
What comes next?
Cover everything after position t and predict it. Repeated over a corpus, this is maximum-likelihood estimation of the distribution of text.
A demonstration of the objective, computed in your browser on illustrative data. It is not DeepSeek running here.
How it is trained
- 1Pre-train a mixture-of-experts model: a router picks 8 of 256 expert networks for each token, so 37B of the 671B parameters are active.
- 2Also predict tokens further ahead (multi-token prediction) for a denser training signal.
- 3R1-Zero: reinforcement learning on maths and code problems whose answers can be checked by rules, with no human-written reasoning.
- 4GRPO: sample a group of answers per question, reward each relative to the group average.
- 5R1: add a small cold-start set of reasoning examples and further fine-tuning for readability.
Line of descent
- Nov 2023DeepSeek Coder and LLM — First open-weight releases.
- May 2024DeepSeek-V2 — Mixture-of-experts with a compressed attention cache.
- Dec 2024DeepSeek-V3 — Reported 2.788M H800 GPU hours for the final training run.
- Jan 2025DeepSeek-R1 — Reasoning model under the MIT licence, with distilled 1.5B to 70B versions.
What data teams use it for
- Self-hosted reasoning for maths-heavy or code-heavy analysis
- Studying reasoning: the chain of thought is visible
- Low-cost batch text processing
- Small distilled models fine-tuned on R1’s outputs
Limits
- The reported training cost covers the final run only, not research and earlier runs
- The full model needs a multi-GPU server even though only 37B parameters run per token
- The hosted service filters politically sensitive topics
- Long reasoning makes answers slower and more expensive
Classic algorithms underneath
The same ideas, small enough to train in a browser.
Same objective
GPT
OpenAI · Jun 2018
The model family that made next-token prediction famous: a Transformer trained to guess the next piece of text, then tuned with human feedback to follow instructions.
Llama
Meta · Feb 2023
Meta’s open-weight language models. The same next-token objective as GPT, but the weights can be downloaded, inspected, fine-tuned and run on your own hardware.
Claude
Anthropic · Mar 2023
Anthropic’s family of language models: pre-trained on next-token prediction, then aligned with human feedback and Constitutional AI, where the model critiques answers against a written set of principles.