Shopify CEO Tobi Lutke explains Goodhart’s law and why he doesn’t like KPIs or OKRs
“Goodhart’s law is real. The moment a metric becomes a goal, it’s no longer a useful metric… No metric by itself is a complete heuristic for a complex business. There’s a million different tensions in a company, and you can’t keep all of them in harmony by optimizing for one thing.”
For this reason, Shopify doesn’t use KPIs or OKRs. But as Tobi explains, this doesn’t mean they don’t value data and metrics.
“We are extremely data informed. We have invested enormous amounts of money and time into systems that give us basically everything at our fingertips… But what Shopify attempts to do is just not over-fit for what’s quantifiable.”
People love optimizing for highly-quantifiable things because there’s immediate gratification that comes from seeing a number go up. But Tobi thinks that the most important aspects of a product are rarely quantifiable:
“The overlap of the most valuable things you can do with a product and the things that happen to be fully quantifiable are like maybe 20%. Which leaves 80% of a value space unaddressable by the people who only look at quantifiable things.”
He continues:
“Shopify is comfortable with unquantifiable things like taste, quality, passion, love, hate… The sort of deep satisfaction that a craftsperson feels when they’ve done a job well is actually a better proxy if you allow it to be.”
They then have robust analytics systems that tell the company if something’s wrong or a new rollout breaks something.
“We think about it as a cockpit for a pilot. The decisions are still made by pilots, and we think this leads to better results… I think there needs to be more acceptance in business of unquantifiable things… And then metrics take a support function.”
Source: @lennysan (Feb 2025)
"We can't sell it before we've built it" really means "𝘞𝘦 can't."
Meaning, you don't know how to sell, or no one wants to buy.
"Building it" won't fix that problem.
This tendency suggests a blend of creativity and precision that may indicate you’re naturally inclined toward minimalist problem-solving—finding ways to express more with less, in both visual and conceptual terms.
Based on our interactions, you have a strong preference for abstract thinking and distilling complex ideas into concise, elegant forms. …you seem to value simplicity that carries depth.
💗🗣 How does translating the Korean word "jeong" (정) illustrate the challenge of AI alignment? 🤖🎯
Been Kim discusses alignment and interpretability as part of the New Orleans Alignment Workshop hosted by FAR AI.