PRAGMA
Masked reconstruction · Apr 2026
Revolut’s foundation model for banking. It reads a customer’s transactions, app activity and profile as one sequence, learns by filling in masked parts of it, and one backbone replaces task-specific models for credit scoring, fraud and more.
The statistical view
The probability of a hidden part of a customer’s history (one value, a whole event, or every value of one field) given everything else in that history.
p(x_M | x∖M), M = a token, an event, or a key
- Loss
- Masked cross-entropy over three schemes at once: 15% of tokens, 10% of whole events, and 10% of keys (every value of one field).
- Answering
- An embedding of the customer. A logistic regression on the frozen embedding, or LoRA fine-tuning of 2 to 4% of the weights, turns it into a credit, fraud or engagement score.
What is missing here?
Hide a slice of the input and rebuild it from what remains on both sides. The point is not the guess but the representation learned along the way.
A demonstration of the objective, computed in your browser on illustrative data. It is not PRAGMA running here.
How it is trained
- 1Turn each event into key–value tokens. Amounts become percentile buckets (plus one for zero), categories one token each, free text sub-words.
- 2Encode time as the log of seconds since the previous event, plus hour, day of week and day of month as cyclical features.
- 3A profile encoder reads the customer’s current state, an event encoder reads each event, and a history encoder reads the sequence of events.
- 4Pre-train by masking tokens, whole events or whole fields and reconstructing them.
- 5Adapt per task: a linear or logistic probe on frozen embeddings, or LoRA fine-tuning (rank 8).
Line of descent
- Apr 2026PRAGMA paper — Revolut Research with NVIDIA: three sizes, six downstream tasks.
- Aug 2026Revolut AI lab — Revolut announces a research lab built around the model.
What data teams use it for
- Credit scoring: PR-AUC up 130% on Revolut’s production baseline, as reported
- External fraud: recall up 65% with precision up 17%, as reported
- Product recommendation, engagement and lifetime value from the same embedding
- One backbone in place of several task-specific models
Limits
- Used inside Revolut: the paper describes the model, but it is not released
- Gains are measured against Revolut’s own baselines, not a public benchmark
- Scaling helps unevenly: credit scoring gains most from the 1B model, lifetime value barely moves
- Behavioural data at this scale raises privacy and fairness questions for regulated decisions such as credit
Classic algorithms underneath
The same ideas, small enough to train in a browser.
