Today we're launching Agent Analytics
Every team shipping an AI agent has the same blind spot. Offline evals pass, you ship, and then you have no idea what's happening in production. AI fails silently. Users ask a question and get different answers. They all look 'engaged' in a classic dashboard. You don't know who got a great response and who got a terrible one.
Agent Analytics solves it:
- Every session scored out of the box on task completion, response quality, friction, safety, and negative feedback
- Topic clustering across thousands of conversations, so you know if a failure hits 1 user or 10,000
- Eval agents that watch for regressions, and if you want will file a Linear ticket or the pull request themselves
- Agent quality sits next to product data, so 'payment scheduling fails 31%' becomes 'which renewals did that cost us?'
The Economist got their agent to a 96.9% task success rate and cut weekly failures 84%.
Included on every plan. Free tier included. https://t.co/K5vqrlMLvd
@Nirmal_Utwani Me reading this like why does this sound so familiar but also this story seems very specific / true to Nirmalโฆ scroll down and see Ted Lasso playing darts and the reference clicks ๐
We can ship features 10x faster now. We can also ship the wrong thing 10x faster.
Choosing what to build is the real leverage. Thatโs where @Amplitude_HQ's AI Analytics Platform comes in.
Pumped about the #AmplitudeAI launch, give it a try! https://t.co/sVm4tCyJts
Al has collapsed the time it takes to ship a feature. Now it's all about shipping the right thing. It's just as hard as it ever was.
As coding gets automated, agentic analytics becomes the bottleneck. Teams that win will build within context rich environments that capture user behavior and customer feedback, making that data accessible to both builders and agents.
Today @Amplitude_HQ is launching our Al Analytics Platform with Agents and MCP. Autonomous analytics that helps builders and coding agents ship products that their customers actually want.
Just as our partner @claudeai has reinvented software, Amplitude is reinventing analytics for the Al age.
We have achieved 76% accuracy for complex production grade queries and agentic usage has grown 10x in 3 months.
Don't just build fast, build what's right.
Try it for free, today.
The killing of Alex Pretti is a heartbreaking tragedy. It should also be a wake-up call to every American, regardless of party, that many of our core values as a nation are increasingly under assault.
@spenserskates So true. Our interns and junior engineers are amazing!
Spent an hour at lunch yesterday exchanging AI pro tips with a junior engineer, I learned so many interesting strategies to work with some new AI coding tools/agents
What Cursor did for eng, we just did for product analytics.
Went from typing-chaos Kermit to tea-time Kermit thanks to @Amplitude_HQโs Automated Insights.
Congrats @janakivivrekar & team! Check it out here: https://t.co/ZSyu9dmrVr
Introducing the holy grail of AI Analytics: Automated Insights
We've built an AI that works like an expert analyst. It proactively queries your data to figure out what's causing a change. See it in action:
We used Replit at Amplitude to help 8th graders at @MissionDAcademy learn to code, and it was such a hit, the kids loved it!
So cool to see the Replit team now tapping #AmplitudeAIFeedback to keep improving the product.
Replit serves 40M+ builders with just 2 PMs ๐คฏ
Head of Product Eng @amoljain_ shared how they pull it off:
1. Product-minded engineers
Engineers think like owners and deeply get the โwhy,โ so the team moves fast.
2. Tight feedback loops
Their global builder community drops feedback everywhere: app reviews, Zendesk, X, Reddit, and the team actually listens.
3. AI Feedback
Lets the team instantly parse all that noise and surface what truly matters.
Pumped to partner with the @Replit team as they help anyone go from idea โ app using natural language.
@amasad, @HayaOdeh, and team are building the future ๐ฅ
So easy to fall down the feedback rabbit holeโฆ ๐
#AmplitudeAIFeedback helps you climb back out... with insights that actually matter!
Check it out: https://t.co/vowwpNNCDK
Hey, SF! ๐ Free November 13?
Join us for our AI Wizard Bar! ๐ช You'll get hands-on demos, 1:1 office hours, and a sneak peek at how Amplitude MCP brings real product context into AI workflows. Save your spot: https://t.co/fo6QyXiiVA
Amplitude is now listed as an official MCP partner for Claude: https://t.co/uy9stoa02h
Try us out!
We're moving up in the AI world
Got that start of alphabet positioning too
The State of AI in SaaS:
SaaS is Dead, Long Live SaaS
This started as an analysis of Andrej Karpathy's excellent overview of AI's capabilities. Those who deeply understand this will make all the returns and everyone else will lose a lot of money.
"Ghosts" is a brilliant metaphor for what we've created. They're not animals, and they're definitely not human. They are imperfect replicas of us. Karpathy describes them as a "statistical distillation of humanity's documents".
AI channels that distribution more effectively than any human and can beat us at Go, schoolwork, analyzing medical images, and many well defined tasks. At the same time, it lacks the reward systems that humans use to improve including curiosity, empowerment, play, intrinsic motivation, and culture.
As a result, AI's capabilities are limited. If you watch how the best work gets done with AI, it happens in chunks where a human supervises the output and gives iterative feedback to the AI. Large scale autonomous agents are brittle and fail quickly. Watch anyone vibe code an application with any level of novel complexity.
AI also faces integration barriers into existing organizations. AI is not capable of pulling a lot of the levers you need to be effective like coordinating with multiple stakeholders, building trust, authenticity, and interacting with different modalities across time and space. You could argue that the average human doesn't either, but people know when they're interacting with an AI and don't allow it the same agency as they do to people. The most successful AI B2B companies actually need more humans to integrate what they've built (forward deployed engineers) than traditional B2B SaaS.
What's next?
Those who understand AI and its limitations will transform the industry.
Incumbents will be killed by those who know how to leverage AI. I've seen countless homepages talk about being the "AI platform for AI agents" but can't even string a demo together. Meanwhile they're trying to pitch a future where fully autonomous entities collaborate in parallel to write all the code and humans are useless. They will be the first to be replaced when they get surpassed by AI native companies.
The investors blindly throwing money into companies at 100x multiples are going to lose their money. I spoke with one of the most disciplined investors I know last week. They said they felt they had to play the game on the field even though they knew it didn't make sense. And this was from someone who is closer to the technology than 95% of investors.
On the other hand, companies who deeply understand AI will win everything. Cursor, Glean, Decagon, Sierra, Linear, Lovable, Replit, Bolt, and many AI natives are off to a great start.
While 90% of incumbents haven't adapted, it is possible. Figma, Notion, Vercel, Box, and Intercom have done a great job of tearing down what they have and rebuilding AI native products. They have teams who are close to the current capabilities of the models. They also understand their problem domain and as new capabilities come out know what capabilities will map well to what problems in what way. They are able to deliver on AI's promise to their customers. Whereas the majority of existing companies will die.
In analytics, the door is wide open. In spite of many of our competitors filling up their homepages with the text "AI", I haven't seen a single compelling demo. We're still working like it's 2015.
This will change. We have spent the last year at Amplitude rebuilding our team to be AI native. We've learned about what models are capable of, how to write prompts, and how to leverage evals for building great products. We've worked with our customers to see what gets used in practice and what doesn't. We are building a vision for the future of analytics. We are all in on AI at Amplitude.
Stay tuned for what's next. We're going to be fast and furious with AI products at Amplitude.
๐ฉ Red flags? Not with Amplitude.
Today, we're hard-launching โจ#AmplitudeforMarketers,โจ a new way to unify marketing and product data to drive conversions, lifetime value, and ROI. Learn more: https://t.co/SzzvZgsdqb