I think PMs and product leaders are under-investing in generative AI fluency & building the hard skills necessary to future proof their careers.
Sure folks can mumble about semantic search or "agents" or chat as an interface, but could they sit down and really spec a great AI app?
Product teams and leaders are responsible for understanding customer problems & goals, scoping out potential solutions, and providing enough detail that their partner design and dev teams can build a predictably high quality solution that can be tested in the market.
But I think 9/10 PMs would flounder when asked to write a decent PRD for an "AI-powered" product feature. It's simply not enough to say something something LLM chat agent beep boop. Because these systems can be highly non deterministic, there's a whole other set of requirements that need to be considered and outlined in order to go beyond demo apps to reliable production releases.
Often when people present gen AI ideas to me in various contexts, I ask: "How would you ensure this will be a high quality experience across the breadth of use cases you're describing" and often, the response is punted to engineers -- "oh, I'll set the goal and trust engineers can figure it out." Or worse--"oh, no one minds a few hallucinations."
But the reality is, just like with other products, a PM has to be able to articulate sufficient detail in goals, outcomes, and user experience to match the execution of the product with the problem they're solving. For gen AI applications, this is going to include:
- prompt strategy, writing, and testing
- scoping agent tasks
- retrieval strategies
- context management
- feedback strategies
- unit economics
- security & prompt injection hardening
- model selection
- identifying and structuring sources for fine tuning
- chains & agent orchestration
- analytics & quality management
None of these concepts are sitting out there in your PRD template, and while these seem like technical concepts -- they're not exclusively for SWEs to figure out (unless, of course--and this might be a good thing--more SWEs lead product management as well as technical execution.)
My 2c is that the only way to quickly ramp on hard skills is to get hands on: not just building things but *thinking about how you would get a team to build that thing.* What are the "requirements"? What would you measure? How do you share specs for a non-deterministic experience? How do you design for trust?
It's super early, and I believe that product leaders that invest now in what I'll call "hard AI product skills" are going to be way ahead of the game for when these capabilities inevitably become table stakes for software companies.
Don't miss out!
Here's a brief explainer on the difference between rectangles and pentagons.
This is especially important for my friends who think the AI apocalypse is coming soon — since this website and Wall Street got duped today by something much, much dumber.
Listen to this AI generated song featuring Drake & The Weeknd.
It goes so damn hard.
It's by "Ghostwriter977" on TikTok and it's blowing up on socials + streaming platforms.
UMG, which controls around 1/3 of the global music market, has already asked streaming platforms to ban AI.
A modern Napster moment.
Will be fascinating to watch this all unfold in real-time.
Do you ever feel that some cities are just more alive than others?
It's probably because they're "mixed-use".
This is a simple idea, but it changes everything...
@RGBParade@arunpattnaik@levelsio And, if it helps I’m on the new M2 MacBook Pro.
Digging through #Cython docs and webpages for the last hour didn’t lead me anywhere…