Pep AI hit $100k total rev on May 9. Now we hit $200k total rev on June 10. $100,000 in revenue in 1 month is crazy! It keeps getting easier and easier. And the app only launched a little over 3 months ago.
How should companies measure ROI of AI?
Here's my working mental model. Tear it apart!
1) Below a certain investment level (determined by ELT or AI steering committee), ROI can be vibes-based through conversations with users. Goal here is to remove friction & empower people to play with the technology however they find helpful. It just has to lead to a high enough fidelity gut feeling to determine if a higher investment experiment is worth running.
2) Above a certain investment level, ROI has to be as high fidelity as possible. Every AI initiative is run like an experiment with friction minimized as much as possible. There’s a certain investment limit to experiments and investments can be revisited once experiments are complete. Here's how an experiment would be run & how (soft vs. hard) ROI would be calculated.
- Hypothesis: If recruiters use AI to screen resumes, then the time-to-hire will decrease and the interview-to-offer conversion rate will remain equal or improve.
- Independent Variable: The screening method used (AI-powered software versus traditional human resume review).
- Dependent Variables: Time spent screening (minutes per resume), candidate diversity metrics, and the hiring manager's satisfaction score of shortlisted candidates.
- Controlled Variables: The same job description, the same pool of raw applicant resumes, and the same evaluation criteria (rubric).
To ensure a fair test, you must use a randomized control design:
- Control Group: Group A consists of experienced human recruiters who screen 200 incoming resumes using your traditional manual process.
- Experimental Group: Group B uses the AI screening tool to parse and rank the exact same 200 resumes.
Experiment steps:
1) Time Tracking: Log the total hours Group A spends reading resumes versus the time it takes to configure and run Group B's AI tool.
2) Blinded Interview Review: Pass the top 10 candidates selected by the human process and the top 10 selected by the AI process to a hiring manager. Do not tell the manager which candidate came from which screening method.
3) Quality Metric: Have the hiring manager score each candidate's qualifications on a scale of 1–10 based on the interview.
4) Replication: Repeat this exact process across three different job openings (e.g., Sales, Engineering, and Marketing) to ensure the AI's effectiveness isn't limited to just one type of role.
Results & ROI:
Experiment proved successful if 2 conditions are met:
- Condition 1: Time Saved > 0
- Condition 2: AI Average Quality Score ≥ Human Average Quality Score
If not successful, run new experiment (i.e. how can we tweak the AI to deliver as high of an average quality score)
If successful, measure ROI.
In this example ROI would look like:
ROI % = (Annual Savings - Annual AI Cost / Annual AI cost) * 100
So if the company has 50 job roles per year, 9.5 hours are saved and the screening software costs $10,000, the ROI would be:
(475 hours saved * $58/hr - $10,000 AI tool/ $10,000 AI tool) * 100 = 174% ROI
And that ROI is realized (goes from soft savings to hard savings) either by slowing down the hiring of recruiters, firing recruiters, or revenue realized by getting new hires into seat faster.
What do you think? Right/wrong approach?
AI subscriptions are dead
Claude Fable 5 will only be on the Anthropic subscription until June 22nd. After that, you will need to pay for usage per token
This will be the start of a much larger trend
Frontier models will no longer be included in subs
You’ll pay a fee and it will only get you access to older, much cheaper models
If you want access to that dank AI sour diesel, you’re going to need to pay for every token you use. No more subsidies
And it make sense. The subsidies were just a Ponzi scheme
For those that don’t know, when you pay $200 a month for an AI sub, you get thousands of dollars of tokens
These AI companies actively lose tremendous amounts of money because of these subscriptions. GDPs of most countries every year are lost on your $200 Claude Max sub
The investor money is running dry. IPOs are coming because of this. And with IPOs need to come profitability
The golden age of paying $200 a month and being able to code on 40 Claude Code instances and getting a usage reset every 5 minutes are about to die
The party couldn’t continue ever. You can’t just leverage the entire global economy for years and expect nothing to break. Now it’s time to pay up
Means a few things:
1. Time to be responsible when it comes to which models you use. You don’t need Fable 5 for GPT 5.5 Xhigh for everything. Build the skill of knowing when to use cheap models
2. Local LLMS/hardware will come even more in demand. I’m currently running GLM on my Mac Studio. It’s great. Is it Fable? No. But it gets the job done for free on simple tasks. Learn about local LLMs
3. This is the beginning of the wealth gap expansion. Those that can afford to spend $10,000 a month on Fable 5 will build incredible products that eat up more and more of the economy. Those that can’t afford Fable 5 will have an insane disadvantage
4. The government will need to step in eventually. There will be too much civil unrest. I hope the answer isn’t free money. That won’t do anything. I hope the answer is education/access to AI resources for ALL. Universal Basic Opportunity
5. You need to seriously reconsider where your money goes every month. If you are complaining about AI prices and in the back of your mind you know your skill set is becoming quickly irrelevant, all while spending money every month on Netflix, Xbox Live, Paramount +, drugs, DoorDash, Uber, and other things that bring nothing positive to your life, you are simply doing it wrong. AI is an investment in yourself. It’s an investment in your relevance to the global economy. You need to make sure you make that investment
The pieces on the board are quickly moving around. The rules are changing. The battlefield is shifting. If you’re not strategizing accordingly, you’re cooked.
AI search is exploding, creating a massive blind spot for website traffic. While clicks are shifting, visibility shouldn't ✖️
GA is introducing automated AI Assistant traffic measurement. Track & trend human traffic from top chatbots directly in reports → https://t.co/9bwQ0yBLsY
i hooked my whoop to my work calendar to find which coworker gives me the most stress 🚨
thanks to fable, I reverse engineered whoop to pull per minute heart rate. nd matched spikes with cal events and attendees
I now have a leaderboard and I think about it daily.
few info masked for obvious reasons ;)
BREAKING NEWS: Anthropic's latest model will NOT help you if it thinks your ML research/ML engineering is interesting, and/or will secretly degrade its IQ so that the average engineer won't notice. We are already seeing Anthropic's latest model's moderation filters our GPU inference research and programming 😭
I am all for safety refusals, and TOS refusals are fine, but giving intentionally wrong/bad answers on perfectly legal things that "violate our TOS" and charging people for said token/usage is pretty brazenly fraudulent.
#Fibes se convierte esta semana en la capital mundial de la gobernanza de #Internet con la celebración del Foro de Políticas #ICANN86 🌐💻
📈 Más de un millar de expertos y profesionales analizan hasta este jueves los retos globales para garantizar una red segura y estable.
That’s a wrap on the "How It Works" sessions at #ICANN86! 💡
The sessions provided information on the technical coordination of the Internet's core identifiers, career development in the sector, and the implementation of Internationalized Domain Names (IDNs).
🙌 Big thanks to everyone who joined and contributed to these sessions!
If you've adopted AI at your company but haven't seen any tangible results, read this 1990 article: "The Dynamo and the Computer" by Paul David.
When electricity first arrived, factories that "adopted" it barely got faster. They just swapped the steam engine for an electric one and ran everything else exactly as before: same machine layout, same workflow, same management. Electricity in, no real gains out.
The most common mistake with any new technology is to drop it into the old organization and then declare the transformation done.
The real leap came decades later, when each machine got its own small motor. Suddenly machines no longer had to be lined up around one central drive shaft. They could be rearranged around the actual flow of work.
The productivity gains didn't come from electricity. They came from REDESIGNING THE ENTIRE FACTORY around it.
AI is the same. Bolting it onto your existing process gets you a faster steam engine. The payoff comes when you redesign the work itself.
(link to paper in comments)
Nobel Prize winner Demis Hassabis just accidentally revealed who survives the next 5 years and who doesn't.
"One person who understands AI will outperform an entire startup team"
Most founders heard that and thought: "Oh no, I need to learn prompt engineering"
Wrong.
That's not what "understands AI" means anymore.
It means: building workflows. Chaining systems. Automating entire departments.
Not typing better questions into ChatGPT.
The split is brutal:
> 90% of people = still using AI like a calculator
> 10% of people = treating it like infrastructure
In 5 years, the 10% will run everything with half the headcount.
The 90%? Replaceable.
Which group are you in?
Watch the full breakdown. This is the only skill gap that actually matters right now.
Bookmark this. You'll want to reference it.
In 1879, JP Morgan paid a man to invent the lie that is the foundation of modern economics.
A billionaire who helped start Amazon just exposed the whole thing on Diary of a CEO, and once you hear it you will never look at paychecks the same way again:
146 years ago, a guy named Henry George wrote a book called Progress and Poverty.
It was the first mainstream book about the rich systematically stealing from the poor, and It literally became the bestselling book in the history of the United States at the time.
The working class was reading it everywhere, and the people at the top of the economy completely lost their minds.
So JP Morgan personally brought a man named John Bates Clark to Columbia University, which was essentially the intellectual headquarters of Wall Street, and told him to fix the problem.
Clark wrote a book called The Distribution of Wealth. In it, he invented something called the "theory of marginal productivity," which claims that because markets are perfectly efficient, the amount of money you earn reflects EXACTLY the value you contribute to the economy.
If you make $15,000 a year, that's because you're providing $15,000 of value. If a hedge fund manager makes $500 million a year moving money around, that's an accurate reflection of the value he creates in the world.
And Clark literally said the quiet part out loud IN HIS OWN BOOK.
He wrote that they had to prove to working people that no matter how much they make, whether it's a little or a lot, it accurately reflects their value, because if workers ever concluded that their labor was worth more than they were being paid, they would revolt and destroy the entire system.
That was the whole point. The theory was built to prevent a revolution.
And it worked so well that it got absorbed into mainstream economics and is STILL taught as a foundational principle to this day.
Every time a CEO tells you "the market decides your salary," they're repeating a framework that was literally commissioned by JP Morgan in the 1800s to convince you not to ask for more.
Nick Hanauer, the billionaire who told this story, also shared the numbers that prove why it matters right now:
The median full-time worker in America earns about $60,000 a year. If that same worker had maintained the same share of GDP they held in 1975, they wouldn't be making $60,000. They'd be making $120,000. That gap goes all the way up to the 90th percentile. If you earn $180,000 today, you'd be earning $250,000 under the old distribution.
The ONLY people who benefited from 50 years of economic growth were the top 10%, and the vast majority of that went to the top 1%. That is trillions of dollars every single year that used to be wages for ordinary working people and now sits in the accounts of the wealthiest people on the planet.
This happened because of policy. Tax cuts for the rich, deregulation for the powerful, and wage suppression for everyone else, all justified by an economic theory that was invented specifically to make you believe you deserve exactly what you're getting.
And the craziest part is that GDP growth rates in America were 4 to 4.5% for decades when workers were included in prosperity. As soon as the neoliberals took over in the mid-1970s and implemented these policies, GDP growth fell to 3% and eventually to 2%.
Including people in the economy doesn't slow growth down. It's literally the thing that CREATES growth. And the theory that convinced the world otherwise was a hit job paid for by one of the richest men in history to keep workers quiet.
What do you think?
A freelance journalist who had never taken a statistics course wrote a 142-page book in 1954 that professional statisticians still hand to students before anything else, because nobody before him had bothered to explain the tricks in plain language.
His name was Darrell Huff. The book is called How to Lie with Statistics.
I read it in one sitting and spent the next three days noticing the tricks everywhere.
Over 1.5 million copies have sold in English alone. It became a standard college textbook in the 1960s and 70s. Seventy years later it is still in print, still assigned, still the first thing a working statistician reaches for when they want to teach someone to think clearly about numbers.
The man who wrote it was not a researcher. He was a freelancer who wrote how-to articles for magazines. He had no PhD, no academic post, no institutional affiliation. He just understood that numbers could lie without technically being wrong, and he thought someone should explain how.
His opening line sets the whole tone of the book.
"The crooks already know these tricks; honest men must learn them in self-defense."
That one sentence is the entire argument. The manipulation is not coming. It already happened. It happened this morning in the article you read and the chart someone showed you at work and the study your doctor quoted. The only question is whether you know what to look for.
Huff called the first trick the Well-Chosen Average.
When someone tells you the average salary at a company is $80,000, they have told you almost nothing. If the CEO earns $2 million and the 20 employees earn $30,000 each, the mean is $80,000. The median is $30,000. Both are technically correct. One is a lie. The person reporting the number chose which average to use, and they almost always chose the one that served their argument. Huff's rule: whenever you see an average with no description of which average it is, ask.
The second trick he named the Gee-Whiz Graph.
A line chart shows company profits rising. The line shoots nearly vertical, almost doubling in height across the chart. You feel impressed. Then you look at the y-axis and notice the chart does not start at zero. It starts at 94. The actual increase in profits was 3 percent. The dramatic visual was produced entirely by cropping the bottom of the chart. Nothing in the data changed. The picture changed everything.
Every news organization on earth still does this every day.
The third trick is the one that should change how you read every study you ever encounter. Huff called it Post Hoc Rides Again, which is short for the Latin phrase post hoc ergo propter hoc. After this, therefore because of this.
Cities with more churches have more violent crime. Therefore churches cause violence. The logic is airtight. The conclusion is absurd. Both church attendance and crime go up as population grows. The two numbers track each other because a third variable drives both. The correlation is real. The cause is invented.
Huff showed that this structure is not a rare mistake. It is the default pattern of almost every study reported in a newspaper, because causation is a boring word and because proves is a better headline than correlates with.
The fourth trick was the one that floored me. He called it the Semi-Attached Figure.
A headache pill company claims their product is twice as fast as the competition. The study behind the claim is real. The product was tested and the numbers are accurate. What the advertisement does not mention is that the study measured absorption rate into the bloodstream, not relief of headaches. The two things are related but not identical. The statistic is real. It is attached to the wrong conclusion.
Huff said this is the most dangerous trick of all because the number is never fabricated. You cannot fact-check a semi-attached figure by verifying the statistic. You have to ask whether the statistic actually measures what the claim requires it to measure.
Almost nobody asks.
There is one part of Huff's story that most people who recommend the book leave out.
Years after he wrote it, he was hired by the tobacco industry. He worked on a follow-up manuscript called How to Lie with Smoking Statistics, designed to cast doubt on the research connecting cigarettes to cancer. The book was never published. He testified before Congress in an attempt to undermine the statistical evidence against tobacco.
The man who wrote the clearest guide to spotting statistical deception spent the end of his career deploying those same tricks against evidence that was killing people.
That detail does not make the book wrong. The tricks he described are real and the defenses he taught are still the right ones. But it is a reminder that the tools in the book are neutral. Understanding how lies are built does not protect you from choosing to build one.
The crooks already know these tricks.
Some of them wrote the manual.
What is one statistic you have seen recently that you now think deserves a second look?
How to validate an idea using Reddit:
1. Search the Problem
Go to Reddit and search for the pain point (e.g., "hate managing invoices" or "can't find good freelance designers") your idea solves.
And set the filter to Top / All Time.
2. Read the Posts Like a Heatmap
If you see dozens of threads complaining about the same problem across multiple subreddits, that's a real problem.
Then, we all realize there is no solution or only a partial one.
3. Check the Upvotes / Comments
High upvotes + many comments = strong pain, active community, real demand.
Low engagement = niche problem or weak urgency, people don't care enough
4. "Someone Should Build This" Signal
Search phrases like "I wish there was an app", "why doesn't X exist", or "I'd pay for...", these are goldmines.
Users are literally handing you validated ideas.
5. Spot the Workarounds
If people are sharing DIY solutions (spreadsheets, manual processes, duct-tape tools), that's a strong signal.
This means the problem is real and no good solution exists yet.
6. Find Targeted Subreddit
Check which subreddit the complaints live in.
That community = your first customer base. You can post there, run surveys, or do direct outreach.
a stat that I cannot stop thinking about:
78% of people hire the first business that gets back to them.
not the best or the cheapest.
the FASTEST.
Here's an AI business idea that prints money off that one stat (start it this weekend with zero code):
Step 1: Pick the target.
Local service businesses with solid Google reviews (3.5 stars and up) and a website that looks like it was built in 2009. Or no website at all.
My pick: commercial cleaning companies. Recurring contracts, high lifetime value, and the space hasn't been bought up by private equity yet.
These businesses are bleeding demand.
Imagine how often someone Googles them at 9pm, lands on a dead website or a voicemail, then books with their competitor who had an auto-quote or answering service set up. The owner never even knows it happened.
Step 2: Build the machine. 0 code requires.
1. Rebuild their website for free. Modern, mobile, built for one job: turning a visitor into a booked appointment. You could host it on a GoHighLevel sub account so you control it.
2. Run Google ads to it. Their card, their ad account, so they watch every dollar.
3. Point an AI agent at their forms that answers, qualifies, and books appointments 24/7. The bar to beat is a competitor's voicemail. People would rather talk to an AI than nobody.
They touch none of it. That's the whole point.
Step 3: Make an offer they'd feel stupid saying no to.
You have zero case studies, so don't sell a retainer yet. Sell results.
They cover ad spend and hosting. They pay you only when a real appointment lands on their calendar. No retainer. No contract. Cancel anytime.
One rule: never say "leads." Every owner has been burned by Angi selling the same junk lead to five competitors. Say "exclusive booked appointments nobody else gets." Same thing. Completely different reaction.
Step 4: Once you have 2 or 3 wins, flip the model. Stop charging per appointment. Charge $5k/month flat plus ad spend, with a guarantee: X booked appointments a month or you don't pay the next one.
Now it's a managed service. They pay, they touch nothing, appointments show up, and you're on the hook if they don't. That's a client who never wants to leave.
How you land the first one (the part nobody shows you):
-Scrape Google Maps for cleaning companies in your city
-Outscraper does it for basically free
-Clean the list down to mobile numbers only
-Text them from your own phone.
-Don't pitch
-Open with a question that looks like a job: "are you guys still taking on new commercial accounts around [city]?"
-They reply yes. Then you pivot: "I actually reached out for a different reason. I don't need cleaning. I can send you clients. I'll build you a new site for free and only get paid when an appointment hits your calendar. Want me to mock yours up?"
-Build the free demo site that day
-Send them the live link
-Book the call
That's it. The website is the trojan horse.
The speed-to-lead AI is the actual product.
You need to move before every cleaning company in your city already has a guy.