Product and tech leader, entrepreneur and advisor who loves building teams and products, mostly in agtech. Moonlights as (amateur) musician đžđčđ€ and runner đ
No-one has figured out how an eng team should work with agents yet. Be wary of anyone telling you they know how to do it. Keep exploring. https://t.co/QZ3RXEyIzZ
me and everyone around me prompt codex and chatgpt with voice. the more decisions you can spit out to codex the better your code will be, so you're only interface limited
the "everyone will use speech" guys were right, just 234 products and 18344 softwares too early
every new model generation you see the pinch of the bitter lesson.
harnesses, pipelines, rules which previously felt important now hold you back from innovating.
what took months of grind for you is now just a prompt away at œ the cost.
look for it and you will see. Both large and small companies re-evaluating. Company directions change before your eyes.
itâs a wild moment for our industry
In order to have a sense of AI adoption right now, you really need to have one foot at the bleeding edge with the kids Wispr-ing through a mic at Devin all day and one foot in the 99% enterprise world where theyâre still debating the merits of leaving âbundled-MSFT-ELA-Copilotâ
WTF are evals?
Evals are how you measure the quality and effectiveness of your AI system. They act like regression tests or benchmarks, clearly defining what âgoodâ actually looks like for your AI product beyond the kind of simple latency or pass/fail checks youâd usually use for software.
Evaluating AI systems is less like traditional software testing and more like giving someone a driving test:
1. Awareness: Can it correctly interpret signals and react appropriately to changing conditions?
2. Decision-making: Does it reliably make the correct choices, even in unpredictable situations?
3. Safety: Can it consistently follow directions and arrive safely at the intended destination, without going off the rails?
Evals are analogous to unit testing in some ways, with important differences.
Traditional software unit testing is like checking if a train stays on its tracks: straightforward, deterministic, clear pass/fail scenarios.
Evals for LLM-based systems, on the other hand, can feel more like driving a car through a busy city. The environment is variable, and the system is non-deterministic.
Unlike in traditional software testing, when you give the same prompt to an LLM multiple times, you might see slightly different responsesâjust like how drivers can behave differently in city traffic. With evals, youâre often dealing with more qualitative or open-ended metricsâlike the relevance or coherence of the outputâthat might not fit neatly into a strict pass/fail testing model.
Much more in this week's đ„ post by @amankhan
https://t.co/coexS7y2He
đ± Exciting news from Leaf! We've closed an $11.3M Series A round led by Spero Ventures. This funding will help us continue to help push food & agriculture forward by making it easy for companies to become compatible with data from any farm via a single integration.
A technical background is a superpower for PMs. You make better decisions, communicate with engineers with more confidence, and create more career opportunities for yourself.
In today's newsletter, Colin Matthews (instructor of the top-rated Maven course "Technical Foundations for Product Managers") breaks down in the simplest way possibleâand with hands-on practiceâAPIs, code deployments, system architectures, branches, databases, unit tests, and much more.
If youâve been embarrassed, or too busy, to ask your engineers how all this worksâthis post is for you.
https://t.co/B0En2YlLT9
Instead of âNow / Next / Laterâ
what about âNow / Probable / Possible.â
Youâre still communicating roughly the same thing, but with correct level of commitment.
#agtech software has struggled with adoption, will #AI help solve the problem?
@svnoles@jmatthewpryor of @tenaciousvc and I did a deep dive into trust, CX, and biz models for agtech software
Free report below
https://t.co/f7VrIR9LXs
Every time you ask the user click you lose half of them.
(AKA why tutorials, splash screens, and lengthy signup flows are a bad idea)
If youâve been building apps for a long time and have seen the results of a lot of A/B tests, you quickly realize that people are a flighty bunch. Ask them to download an app and 80% will bounce right on that page. Ask them to sign up and 90% will hit the back button to avoid putting in their email and password. Ask people whoâve arrived from Google to read an article, to subscribe and get more updates, and 99% will head back to find the next article.
In the early days of Uber the only way to sign up was to give your email address a bunch of other fields and also your credit card number. Some of the big early winds in acquiring customers was just to make it so that you could sign up with a phone number and a password, and put in your credit card lead in the flow. If memory serves me right, these were increases on the order of +50%.
You get the drift of what Iâm arguing.
So what happens when your designer has the fantastic idea of a stark and beautiful homepage for your new product that takes a few clicks to sign up, followed by a lengthy tutorial to explain all the features? Sometimes this becomes a life and death decision, because rather than signing up thousands of users into your private beta, which provides the traction to raise your next round of funding, instead only a few hundred make it through.
This is why, when I get feedback on a critical flow within a product, I always start by minimizing the number of clicks and steps. I asked whether each field in a sign-up form is really needed, or is optional. I ask the question of whether you need to user to do something now versus having them set it up in the future, when theyâre more bought into the product. I ask to remove all the glitzy, visual steps that explain things and just ask the user to hit next. I move the sign-up form to the first experience, whether thatâs on the homepage, or the opening screen of an app. If thereâs a call action, while the user is doing something else, like reading an article, my theory is that you should be very upfront with it and make it a blocking modal, or not do it at all. No half measures.
The point of all, this, of course, is to get people into the magic of your product. The magic is not in filling out forms or watching cute videos about your product, itâs about using your product as quickly as possible. As a result, the only acceptable forms of friction are ones that ultimately enhance the users ability to have a great experience. Thus product is much better experienced as an app, where you have a notifications channel and a richer experience, then, by all means, ask the user to download something. If a product is much better, when used with colleagues or friends, that it might make sense to take a lower conversion rate during the sign-up flow in exchange for some sharing or inviting functionality, that brings more people into the app. Ultimately, itâs all a trade-off, where every click drops off a huge number of users, so you need to spend that user intent very very well.
Ironically, it can also be an anti-pattern to not ask users to sign up or install or do anything at all, because once they bounce, which they will inevitably, do, you have no way to get them back. Thatâs why itâs all a trade-off, and one of the trickiest things about the user growth discipline is knowing when to add friction, and when to take it away.
Also, interestingly enough, as you make it easier and easier to sign up to reduce friction the quality and intent of the users also decreases. If you double the number of sign-up typically, you do not get twice the number of paying customers.
Nevertheless itâs an important thing to remember: Every time you ask the user click you lose half of them. Be careful.
mileiâs speech should be the media event of the year. every high school class in the US should break and watch it, then spend a week understanding the historical context behind his devastating truths. if you havenât watched, please do. and share widely.
The Obvious, the Easy, and the Possible
Much of the tension in product development and interface design comes from trying to balance the obvious, the easy, and the possible. Figuring out which things go in which bucket is critical to fully understanding how to make something useful.
Shouldnât everything be obvious? Unless youâre making a product that just does one thing â like a paperclip, for example â everything wonât be obvious. You have to make tough calls about what needs to be obvious, what should be easy, and what should be possible. The more things something (a product, a feature, a screen, etc) does, the more calls you have to make.
This isnât the same as prioritizing things. High, medium, low priority doesnât tell you enough about the problem. âWhat needs to be obvious?â is a better question to ask than âWhatâs high priority?â Further, priority doesnât tell you anything about cost. And the first thing to internalize is that everything has a cost.
Making something obvious has a cost. You canât make everything obvious because you have limited resources. Iâm not talking moneyâalthough that may be part of it too. Iâm primarily talking screen real estate, attention span, comprehension, etc.
Making something obvious is expensive because it often means you have to make a whole bunch of other things less obvious. Obvious dominates and only one thing can truly dominate at a time. It may be worth it to make that one thing completely obvious, but itâs still expensive.
Obvious is all about always. The thing(s) people do all the time, the always stuff, should be obvious. The core, the epicenter, the essence of the product should be obvious.
Beyond obvious, youâll find easy. The things that should be easy are the things that people do frequently, but not always. It all depends on your product, and your customer, but when you build a product you should know the difference between the things people do all the time and the things they do often. This can be hard, and will often lead to the most internal debates, but itâs important to think deeply about the difference between always and often so you get this right.
And finally are the things that are possible. These are things people do sometimes. Rarely, even. So they donât need to be front and center, but they need to be possible.
Possible is usually the trickiest category because the realistic list of things that should be possible will often be significantly longer than the list of things that should be obvious or easy. That means that some things on the possible list might be better off off the list completely. Instead of making them possible, maybe not making them at all is the right call.
Coming to know the difference between obvious, easy, and possible takes a lot of practice, deep thinking, critical analysis, and, often, debate. Itâs a constant learning process. It helps you figure out what really matters.
But once youâre able to see the buckets clearly, and you begin to think about things in terms of obvious, easy, and possible instead of high, medium, and low priority, youâre on your way to building better products.
If you're drowning in bureaucracy at work, your company probably isn't following Elon Musk's five-step process for improvement.
Here are the highlights:
"Everybody's been trained in high school and college that you have to answer the question. It's convergence logic. You can't tell a professor '"your question is dumb" or you'll get a bad grade. You have to answer the question. So without knowing it, people have mental straight jackets on. They will work on optimizing the thing that should simply not exist."
"It's particularly dangerous if the smart person gave you the requirements because you might not question them enough."
"If you're not occasionally adding things back in, you're not deleting enough process. The bias tends to be very strongly towards adding things."
(These quotes are paraphrased to make them easier to read).
Tl;dr
No amount of doc or figma review substitutes for using the product, hands on, like users do.
Spend time every week using your app, especially new stuff.
And ship in bits and speed up feedback cycles, even during the development phase.
Finally: details matter.