Shopify CEO Tobi Lutke explains Goodhart’s law and why he doesn’t like KPIs or OKRs
“Goodhart’s law is real. The moment a metric becomes a goal, it’s no longer a useful metric… No metric by itself is a complete heuristic for a complex business. There’s a million different tensions in a company, and you can’t keep all of them in harmony by optimizing for one thing.”
For this reason, Shopify doesn’t use KPIs or OKRs. But as Tobi explains, this doesn’t mean they don’t value data and metrics.
“We are extremely data informed. We have invested enormous amounts of money and time into systems that give us basically everything at our fingertips… But what Shopify attempts to do is just not over-fit for what’s quantifiable.”
People love optimizing for highly-quantifiable things because there’s immediate gratification that comes from seeing a number go up. But Tobi thinks that the most important aspects of a product are rarely quantifiable:
“The overlap of the most valuable things you can do with a product and the things that happen to be fully quantifiable are like maybe 20%. Which leaves 80% of a value space unaddressable by the people who only look at quantifiable things.”
He continues:
“Shopify is comfortable with unquantifiable things like taste, quality, passion, love, hate… The sort of deep satisfaction that a craftsperson feels when they’ve done a job well is actually a better proxy if you allow it to be.”
They then have robust analytics systems that tell the company if something’s wrong or a new rollout breaks something.
“We think about it as a cockpit for a pilot. The decisions are still made by pilots, and we think this leads to better results… I think there needs to be more acceptance in business of unquantifiable things… And then metrics take a support function.”
Source: @lennysan (Feb 2025)
@yacineMTB The key is to leverage all that AI coding to not just add new features, but drove down tech debt and make the delivery and test platform more resilient. Probably 60% my team's AI output right now is in the latter bucket.
Ask Claude to document and describe the main flows in your app and output in a single page html + json data file.
Incredibly useful for humans and the JSON file is very useful for explaining the flow to the LLM when working on new features/bugfixes.
🚨 How the TanStack npm attack actually happened:
1. Attacker opened a normal-looking pull request (#7378) on the TanStack repo.
2. GitHub automatically ran CI tests on that PR.
3. Code inside the PR stole the workflow's GitHub Actions Cache write token during the test run.
4. The attacker used that token to plant poisoned files in the shared build cache. The PR could be closed afterwards. The poisoned cache stays.
5. The official release workflow later pulled from the cache, baked the malicious files into the build, and signed and published 84 malicious package versions to npm.
The Ghost in the Machine: How Player Pianos Sparked Protests, and What They Reveal About Our AI Future
In the early 1900s, the player piano was a sensation. These self-playing instruments used perforated paper rolls fed through pneumatic mechanisms to reproduce complex piano performances automatically.
By the 1910s to mid-1920s, they outsold ordinary pianos in many markets, filling American parlors, saloons, and theaters with ragtime, marches, and classical pieces.
Great artists like Sergei Rachmaninoff and Ignace Paderewski cut rolls, preserving their interpretations for generations.
It was automation that brought “live” music into every home, without the need for lessons or live performers.
Yet this marvel triggered intense resistance. Composers and musicians saw it as an existential threat. In his fiery 1906 essay “The Menace of Mechanical Music,” bandleader and composer John Philip Sousa warned that player pianos and phonographs would “substitute machinery for the human soul.”
He predicted the death of amateur music-making: children would stop learning instruments, families would stop gathering around the piano, and music would lose its emotional depth.
Sousa testified before Congress, helping drive the 1909 Copyright Act, which created compulsory licensing so composers could earn royalties from mechanical reproductions, a landmark victory born from protest.
As “talkies” and radio displaced theater orchestras in the late 1920s, the American Federation of Musicians (AFM) launched the Music Defense League in 1930.
Funded by a tax on members, the union spent hundreds of thousands of dollars (millions in today’s dollars) on a national advertising blitz.
Dramatic newspaper ads depicted sinister robots replacing human musicians, with slogans like “Is Art to Have a Tyrant?” and warnings that “canned music” would destroy jobs and degrade culture.
The campaign targeted not just records but all mechanical music, including player pianos in public spaces.
While there were no Luddite-style riots smashing machines (player pianos were mostly expensive home devices), the opposition was fierce: boycotts, lobbying, lawsuits, and cultural shaming of anyone who chose “the robot” over living performers.
The protests did not kill the player piano. Record sales, radio, and the Great Depression did that by the early 1930s.
But the episode left a lasting legacy: new copyright rules, heightened awareness of technology’s impact on artists, and a template for how workers respond to automation.
We are living through the same story with AI and robotics. Generative models now compose music, write screenplays, generate art, and even perform.
Musicians, writers, and visual artists are protesting in eerily familiar ways: lawsuits over unlicensed training data (the modern equivalent of the player-piano royalty fight), demands for “human-made” labels, strikes by Hollywood writers and actors, and public campaigns against “AI slop.”
Fears echo Sousa’s exactly: loss of soul, authenticity, jobs, and human connection. “The robot is coming” ads of 1930 could run unchanged today, just swap “canned music” for “AI-generated content.”
History’s lesson is nuanced.
The player piano did not end music; it briefly coexisted with live performance before giving way to richer ecosystems.
Rolls by legends now serve as priceless archives.
Protests forced legal compromises that protected creators while allowing innovation. Yet real displacement happened. Thousands of theater musicians lost steady work, and the cultural shift toward passive consumption was real.
Today’s AI moment carries higher stakes: it threatens not just one profession but broad swaths of cognitive and creative labor.
Robots and AI could augment surgeons, drivers, teachers, and artists, or render many obsolete. The player-piano saga shows that raw Luddism rarely wins,
We cannot stop technological progress, The music plays on. The question is: who, or what, plays it?
TODAY: Amazon is opening its entire logistics network—freight, distribution, fulfillment, and parcel shipping capabilities—to every business, of all types and sizes. 📦
Amazon has built one of the most reliable and efficient supply chains on Earth. Now, Amazon Supply Chain Services gives all businesses access to the same infrastructure that moves, stores, and ships goods for hundreds of thousands of Amazon sellers.
Healthcare, automotive, manufacturing, retail, and more. Businesses across industries can now tap into Amazon's logistics network. Learn more here. ⬇️
Andrej Karpathy: "90% of what AI twitter tells you to learn will be dead in 6 months"
Here are 10 things senior AI engineers stopped wasting time on:
1. AutoGen / AG2: moved to community maintenance, releases stalled. dead for production
2. CrewAI: demos well, breaks in production. engineers building real systems already moved off it
3. Autonomous agent pitches: the AutoGPT / BabyAGI wave is dead in product form. the industry settled on supervised, bounded, evaluated agents
4. Agent app stores / marketplaces: promised since 2023, zero enterprise traction
5. SWE-bench leaderboard chasing: researchers proved nearly every public benchmark can be gamed without solving the underlying task
6. Microsoft Semantic Kernel: unless you're locked into Microsoft enterprise stack, it's not where the ecosystem is heading
7. DSPy: philosophical merit, niche audience. not a general agent framework
8. Horizontal "build any agent" platforms: Google Agentspace, AWS Bedrock Agents, Copilot Studio. confusing, slow-shipping, the math still favors building yourself
9. Per-seat SaaS pricing for agent products: market moved to outcome-based. per-seat is already dead
10. The framework that went viral on HN this week: wait 6 months. if it still matters, it'll be obvious
what actually compounds instead:
- context engineering
- tool design
- orchestrator-subagent pattern
- eval discipline
- the harness mindset (harness > model, always)
- MCP as the protocol layer
be few steps ahead than your competitors and outperform this market till it became mass-opinion
study this.
I agree with this entire essay if you switch seniority for experience. Experience still an asset
Experience x humility x curiosity x AI = jet fuel
But no humility or curiosity and experience is a 0 and getting run the fuck over in the AI era
Claude Code's Head of Product: "The PM role is changing a lot. And it's changing really quickly.
The most important thing for building AI-native products is iterating quickly and finding a way to launch features every single week.
Putting less emphasis on making sure that you are aligning multi-quarter roadmaps with your partner teams, and more emphasis on, okay, how can we figure out the fastest way to get something out the door."
My tweet last week about Google's AI adoption drew a lot of pushback, to say the least.
Since then, Googlers from multiple orgs have reached out to me independently and anonymously. They've expressed fear of being doxxed, concern about what they saw as bullying of me, and general corroboration of my original tweet. I haven't verified each person's story, but the picture these Googlers paint is consistent across sources. It is more specific than what I originally wrote, and somewhat bleaker.
What they describe is a two-tier system. DeepMind engineers use Claude as a daily tool. Most of the rest of Google does not. When the question of equalizing access came up internally, the proposed response was to remove Claude for everyone — which DeepMind objected to so strongly that several engineers reportedly threatened to leave.
Non-DeepMind engineers get pushed onto internal Gemini variants behind router-style names that obscure which underlying model is actually serving a request. Multiple engineers describe regressions and reliability problems severe enough that some senior people have stopped using the tools. A senior manager on a major product line reportedly flagged attrition concerns over exactly this issue.
Googlers say leadership knows the gap is real. The response has been to mandate AI usage in OKRs and individual expectations, and to stand up an internal token-usage leaderboard. Unfortunately, managers have been told both that the leaderboard won't be used for performance reviews and, separately, that it absolutely will. And I hear other stories that Google's culture is not adapted properly yet for high-volume coding.
Addy Osmani's reply on behalf of Google said over 40,000 SWEs use agentic coding weekly. I don't doubt the number. But weekly use of a thin tool is precisely the box-checking I described in the original post. Volume of opens isn't adoption — and "weekly" is a low bar that includes a lot of people who tried it once and went back to writing code by hand.
The clearest thing I'm hearing is that Googlers do want to use high-quality agentic tools. They are asking repeatedly for better ones. But overall, this is not a picture of an engineering org that is fine.
My goal in the first tweet, and now, is always the same — get more people using AI and agentic coding. Nobody is as far ahead as they might look from the outside, and none of you are as far behind as you might be worried you are.
To all the Googlers who've reached out: thank you. You took a real risk and I appreciate you. Be safe. And good luck getting good models!
Also the fact that MICROSOFT which OWNS BILLIONS IN OPENAI EQUITY and is shoving its own AI tool, COPILOT (which sucks) in 78 different interfaces, is getting brutally PRODUCT MOGGED by a competitor IN ITS OWN PRODUCTS is just crazy.