eric

all opinions are mine

[Gautam Kedia @thegautam]

TL;DR: We built a transformer-based payments foundation model. It works.

For years, Stripe has been using machine learning models trained on discrete features (BIN, zip, payment method, etc.) to improve our products for users. And these feature-by-feature efforts have worked well: +15% conversion, -30% fraud.

But these models have limitations. We have to select (and therefore constrain) the features considered by the model. And each model requires task-specific training: for authorization, for fraud, for disputes, and so on.
Given the learning power of generalized transformer architectures, we wondered whether an LLM-style approach could work here. It wasn’t obvious that it would—payments is like language in some ways (structural patterns similar to syntax and semantics, temporally sequential) and extremely unlike language in others (fewer distinct ‘tokens’, contextual sparsity, fewer organizing principles akin to grammatical rules).

So we built a payments foundation model—a self-supervised network that learns dense, general-purpose vectors for every transaction, much like a language model embeds words. Trained on tens of billions of transactions, it distills each charge’s key signals into a single, versatile embedding.

You can think of the result as a vast distribution of payments in a high-dimensional vector space. The location of each embedding captures rich data, including how different elements relate to each other. Payments that share similarities naturally cluster together: transactions from the same card issuer are positioned closer together, those from the same bank even closer, and those sharing the same email address are nearly identical.

These rich embeddings make it significantly easier to spot nuanced, adversarial patterns of transactions; and to build more accurate classifiers based on both the features of an individual payment and its relationship to other payments in the sequence.

Take card-testing. Over the past couple of years traditional ML approaches (engineering new features, labeling emerging attack patterns, rapidly retraining our models) have reduced card testing for users on Stripe by 80%. But the most sophisticated card testers hide novel attack patterns in the volumes of the largest companies, so they’re hard to spot with these methods.

We built a classifier that ingests sequences of embeddings from the foundation model, and predicts if the traffic slice is under an attack. It leverages transformer architecture to detect subtle patterns across transaction sequences. And it does this all in real time so we can block attacks before they hit businesses.

This approach improved our detection rate for card-testing attacks on large users from 59% to 97% overnight.

This has an instant impact for our large users. But the real power of the foundation model is that these same embeddings can be applied across other tasks, like disputes or authorizations.

Perhaps even more fundamentally, it suggests that payments have semantic meaning. Just like words in a sentence, transactions possess complex sequential dependencies and latent feature interactions that simply can’t be captured by manual feature engineering.

Turns out attention was all payments needed!

[main @main_horse]

@miru_why I believe there is something wrong with their kernel -- it seems to 'steal' the result of the eager impl (memory reuse somehow?), allowing it to bypass the correctness check.

Here, I try executing impls in different order:
* torch, cuda
* cuda, torch

only the first order works!

[Sam Altman @sama]

no thank you but we will buy twitter for $9.74 billion if you want

[Katya Pavlushchenko @katlinegrey]

Great to see such a warm welcome of the new crew on the ISS! #SoyuzMS21 Love their suits!💛💙

[eric @0x6538]

My year on #Github2021 🐙 eegli

📬 Commits/Issues/PRs: 1008
🏝️ Repos contributed to: 11
⭐ New stars: 13
🔥 Hottest: eegli/pampa.place (+8)

Share yours 👉 https://t.co/0zEPTNxcR2 | Built by @jrieke w/ @streamlit @github

[eric @0x6538]

Wie schon beim letzten Mal: Das beste Argument für das #covidgesetz sind seine Gegnerinnen und Gegner. #srfarena

[Shahzod Boyhonov @specoolar]

Recursion.
Blender, Cycles animation.
Assets from https://t.co/07gDVXvXfX and https://t.co/LETmhauHE4

[Jamie Kyle is on 🦋 @buildsghost]

I am so sorry for this...

I wrote a JSON parser using @typescript's type system

https://t.co/0dxc9Q7A9D

[Stephen Thomas Erlewine @sterlewine]

Just noticed that the desktop version of Spotify now contains play counts for every track on an album. It's fascinating data even if it kind of strips away the mystique of an album as its own distinct work.

[Fabio @calandamountain]

Die Weichenheizung:
Ein unabdingbares Hilfsmittel im Kampf gegen Verspätungsminuten. ⏱️
⬇️⬇️⬇️

[XBOX @XBOX]

Xbox

[eric @0x6538]

Adobe brings Content-Aware Fill to video in After Effects https://t.co/dM4g56dkxI via @MichaelSteeber

[Max Howell @mxcl]

Google: 90% of our engineers use the software you wrote (Homebrew), but you can’t invert a binary tree on a whiteboard so fuck off.

[eric @0x6538]

@oliphox @uzh_news Schön, wie du dich gefreut hast, dass so viele gekommen sind! War ein super Vortrag!

[LEGO @LEGO_Group]

At 7,541 pieces, the new #LEGOStarWars UCS Millennium Falcon just might make the Kessel Run in less than 12 parsecs. 😯 @starwars

[Donald J. Trump @realDonaldTrump]

Who can figure out the true meaning of "covfefe" ??? Enjoy!

[Ed Balls @edballs]

Ed Balls

[eric @0x6538]

Shipped all the way from Portland, Oregon, to Zurich, Switzerland. My new #SurfaceBook is a beauty! @Toastmade

[ᑯᐹᒃ ᐅᒃᑯᖅ ᑕᑦᑐᐃᓂ @tattuinee]

.@FlySWISS flight 40 landing at @IqaluitAirport