We visited Revolut's headquarters in Canary Wharf. The office was impressive, and from the balcony, you could see the JPMorgan Chase tower, which seemed fitting.
Anton Repushko introduced PRAGMA, Revolut’s core banking model. In short, PRAGMA creates a unique financial fingerprint for each user. It was trained on 1.5 years of raw data, including transactions, app activity, trading, and support chats. That’s over 40 billion events from 25 million users in more than 100 countries. All this data is turned into a dense vector, which is a few hundred numbers used as embeddings.
After the event, our LinkedIn post received a lot of attention. Most comments focused on two main topics: auditability and real-world impact. This article expands on that discussion and is based on two papers we read afterwards:
PRAGMA Foundation Mode from Revolut: https://arxiv.org/pdf/2604.08649
Modeling Financial Habits with Transformers from Nubank: https://arxiv.org/pdf/2507.23267
Revolut is our main focus since we saw it in action, but Nubank shows that the industry is moving forward with a different approach.
What PRAGMA actually is?
The model doesn’t create a summary you can read. Instead, it stores each user’s behaviours and patterns. For example, it might notice if someone pays rent on the 1st, tops up on weekends, buys ETH often, or travels to Spain every two months. All this information is stored as a vector of numbers, not as text. This is an important point for anyone familiar with AI to remember. That’s what encoder-only means. Typical large language models are decoders—they predict the next word and generate text. We’ll return to this difference as things get more complex.
The architecture is built for different types of banking data. Each event is a set of key-value pairs with a timestamp. PRAGMA uses a key-value-time tokenisation method. Keys are single tokens. Values are encoded by type: numbers are grouped by percentile with an extra group for zero, categories map to one token, and text is split into subwords using BPE (Byte Pair Encoding, a data compression and subword tokenisation method). Time is encoded both as log seconds since the last event and as calendar features.
The backbone has two branches. One encodes profile state, such as plan, region, and balance quantile, as well as lifelong events, such as the first top-up. The other encodes events. Outputs are fused by a history encoder. The model is pre-trained with masked modelling. During pre-training, random tokens from a user’s event history are hidden, and the model learns to reconstruct them. By repeating this across billions of events, the model learns the underlying patterns of financial behaviour.
Why encoder-only and not generative?
PRAGMA is an encoder-only model and works in both directions. Its goal is to create representations that can be used for different tasks, not to generate open-ended text. Masked modelling allows each token to use both past and future context, which helps the model fill in missing parts of event records and learn from complete histories. How is it possible? During pre-training, random tokens from a user’s event history are hidden (masked), and the model learns to reconstruct them. By repeating this across billions of events, the model learns the underlying patterns of financial behaviour.
The pre-training dataset is huge. Revolut’s data science team used 26 million user records from 111 countries, 24 billion events, and 207 billion tokens, covering 25 months from 2023 to 2025. The team balances recent data with historical changes and uses sequence packing and dynamic batching to train efficiently on long, irregular histories.
After pre-training, PRAGMA can be adapted in two ways. In the embedding probe setup, the main model stays the same and a small head is trained on top of the embeddings. In the LoRA fine-tuning setup, Low Rank Adaptation updates only a small part of the model’s parameters. This allows quick specialisation while keeping most of the model shared across tasks.
From fingerprint to products
PRAGMA acts as the foundation for many tasks. Revolut adds a small model on top of the same fingerprint. These simple models use embeddings for tasks such as fraud detection, credit scoring, lifetime value prediction, product recommendations, communication engagement, and identifying recurring transactions.
According to the Revolut paper, one architecture outperformed task-specific models, achieving 130% gains in credit and 67% in fraud in their synthetic tests. However, the slide included a disclaimer: these improvements are on synthetic data, not real production results.
The team also shared an interesting point about scaling. Bigger models don’t always give better results. For PRAGMA-M, 512 dimensions at 100 million parameters are enough, while 1024 dimensions for PRAGMA-L at 1 billion parameters don’t perform much better. Other speakers agreed: smaller models often do just as well on specific tasks and are much faster.
The two questions that matter for regulated payments
As payments product managers, we asked Anton two questions that matter most for regulated payments.
First, about auditability
Suppose the FCA asks, "Why was this customer declined? Why was that payment not blocked? Why was the credit limit cut?” So, how do you explain a decision based on a digital fingerprint made up of 500-dimensional vectors?
Anton explained that PRAGMA is currently used internally to help with decision-making. It does not conflict with current regulations, but the team knows this is still early. As mentioned earlier, there are other models for each service that provide explanations. The next step is to work with regulators to set standards and rules for foundation models in finance.
There is no public timeline yet for formal talks with regulators. We expect internal model risk governance to come first, followed by discussions with regulators. Frameworks like the EU AI Act, UK SS1/23, and NIST AI RMF are emerging, but there is still no banking-specific standard for logging, versioning, and explaining embeddings.
From an engineering standpoint, auditability for foundation models will likely require three layers:
1. Provenance logging for the backbone version and pre-training data snapshot
2. Embedding stability tests under concept drift
3. Post hoc explainers trained on top of frozen embeddings, plus counterfactual probes that show which event types moved the score
Second, what does this mean for users?
PRAGMA works best as a backbone. It gives a consistent user fingerprint that other models can use for different tasks. This helps reduce duplication across systems.
From what we see, PRAGMA mainly serves as an internal backbone. It is built for banking operations, credit, fraud, and risk, not as a feature that users interact with directly.
For users, Revolut built AIR, its AI tool. Right now, it’s just a basic chat. We expect that in a year, it will use PRAGMA embeddings. The real product challenge is not just understanding user behaviour, but connecting patterns to outcomes. For example, not just seeing that someone receives income monthly, but knowing that taxes will be paid on it, a certain amount stays each month, and giving advice on how to keep it.
Some companies build the data layer mainly for their own needs, not to improve the user experience, and are satisfied with lower processing costs. The real value comes when the fingerprint is used to give real advice, not just for scoring.
How Nubank approaches the same problem
Nubank’s nuFormer paper offers a good comparison. The team says financial institutions hold vast amounts of user data, but using it remains difficult due to the complexity and scale involved. Most production models still turn unstructured data into hand-made tabular features.
nuFormer suggests using transformer-based representation learning for transaction data, handling both text and structured details. The paper adapts the 'text is all you need' approach to transactions by representing key-value pairs as strings and training on next-token prediction. The model is causal, like GPT, and is chosen to capture long-term patterns and seasonal changes in spending. This is the decoder model that we referred to previously.
The paper also introduces an end-to-end fine-tuning method that combines user embeddings with existing tabular features. Experiments show better results for large-scale recommendation problems at Nubank, achieved solely through improved representation learning, not by adding new data sources.
PRAGMA uses multiple data sources and is encoder-only with a clear profile state. In contrast, nuFormer uses a text-focused decoder for recommendations. Both show that foundation models can beat task-specific baselines and reduce the need for hand-crafted features.
What product teams should take away?
1. The data layer is changing. We’re moving from hand-crafted features to learned fingerprints. This changes how we think about feature stores and model reuse. One backbone that encodes 24 billion events can replace many task-specific pipelines.
2. Governance is the main challenge. The technology is impressive, but in fintech, the real test is governance. To move to an AI-ready stack, you need to modernise legacy data rather than build on workarounds. Version your embeddings just like you version your models.
3. Auditability needs to be planned. If you want to use embeddings for credit or fraud, build explainability and logging into the system from the start. Regulators will expect it. Think about embedding probes with monotonic constraints for regulated outputs.
4. Bigger isn’t always better. Quality and speed matter more than parameter count, especially for real-time authorisation and fraud. PRAGMA-M with 100 million parameters and 512 dimensions is already a good fit for many tasks.
A final thought
Revolut is still making a name for itself as a disruptor. They are using their scale to challenge established players. The technology is impressive, but the industry advances through standards, not just new models.
PRAGMA and nuFormer offer two solid approaches to the same goal: a single financial fingerprint that can be reused across products. The next 12 to 18 months will be about proving that these fingerprints are stable as things change, can be audited by regulators, and are useful for users (not just for cutting internal costs).
We’ll keep following the team. We now understand what they’re doing and why. Next, we want to see how they handle auditability and production metrics.