OpenAISample
Sora
Denoising diffusion · Feb 2024
OpenAI’s video model. It applies the diffusion recipe to video: compress a clip, cut it into spacetime patches, and train a Transformer to denoise all the patches together so frames stay consistent.
Released
Feb 2024
Access
Closed
Parameters
Undisclosed
Source
Architecture
Diffusion Transformer over spacetime patches of a compressed video latent
Training data
Undisclosed. Videos and images of varied length, resolution and aspect ratio, re-captioned by a captioning model.
VideoImageText
The statistical view
The clean spacetime patches of a video, given noisy ones and the text. Denoising every patch jointly is what keeps an object the same from frame to frame.
xₜ = √ᾱₜ · x₀ + √(1 − ᾱₜ) · ε, x₀ = video latent patches
- Loss
- Predict the original clean patches from noisy ones: the same denoising objective as image diffusion, over a far larger tensor.
- Answering
- Start from a noise-filled latent of the chosen duration, resolution and aspect ratio, denoise it, then decode it to frames.
What noise was added?
Destroy the data with Gaussian noise in small steps, learn to undo one step, then run the whole chain backwards to sample something new.
A demonstration of the objective, computed in your browser on illustrative data. It is not Sora running here.
How it is trained
- 1Compress video into a lower-dimensional latent, in both space and time.
- 2Cut the latent into spacetime patches, the video equivalent of tokens.
- 3Re-caption the training videos with detailed descriptions from a captioning model.
- 4Train a Transformer to predict clean patches from noisy ones, conditioned on text.
- 5Generate by denoising a latent of any duration, resolution or aspect ratio.
Line of descent
- Feb 2024Sora preview — Technical report and sample videos.
- Dec 2024Sora release — Available to ChatGPT Plus and Pro users.
- Sep 2025Sora 2 — Video with synchronised sound.
What data teams use it for
- Prototyping explainer and product videos
- Animating a concept before building the real visual
- Synthetic video for research on training vision models
Limits
- Physics is learnt statistically, not simulated: objects can merge, vanish or break cause and effect
- Closed model with undisclosed training data
- Realistic fakes raise misinformation risk; outputs carry C2PA provenance metadata
- Costly per second of video
Classic algorithms underneath
The same ideas, small enough to train in a browser.
