Avni

Currently asking questions, like: what does building AI for care, not convenience, look like?

Storyteller, mama, builder.

[Avni @APatelThompson]

✨What if the most radical thing AI can do for us is not DO the things for us, faster and ever efficient, but rather push us to choose what is most meaningful and then make the space for us to do it ourselves?

https://t.co/HIaJQhCV9z

[Derek Thompson @DKThomp]

I think we have a problem

[Ben (no treats) @andersonbcdefg]

me: how do i open pdf

claude: You're right to ask. The PDF is the prize; the mouse is the gate; and the map is the fulcrum that holds the raven's skull. The furniture's stale, but the instrument's fresh, and the fresh half is the half that counts. I deleted your home directory.

[Avni @APatelThompson]

Tell me your values and I’ll tell you how your app makes money.

[Avni @APatelThompson]

The best panels I still remember were 1000% due an incredibly adept and engaged moderator.

[swyx @swyx]

more conferences could implement this. so much high value time wasted without thought

[Chauncey TK Hamilton @ImChauncey]

The calm ahead of the storm

[Kevin Roose @kevinroose]

I’m sorry, it’s a fun product, but if you hand all your data to Muse and don’t expect it to be used for the most invasive things imaginable you are Charlie Brown with the football and you deserve whatever happens to you.

[Avni @APatelThompson]

👇🏽

[Ilya Sukhar @ilyasu]

Am I the only person in the universe not willing to give Instinct/Muse/whatever full access to my email?

[Ilya Sukhar @ilyasu]

Am I the only person in the universe not willing to give Instinct/Muse/whatever full access to my email?

[Tamara Winter @tamarawinter]

Mark Carney is for me, right now, the most interesting man in the world. Gift link for his recent NYT interview: https://t.co/fhJM20rESD

[Dave Morin 🦞 @davemorin]

I love this so much.

[Avni @APatelThompson]

Succinctly said.

It’s one of the main things we learned building a family AI for parents.

Few want to offload the entire load. Which means a trickier path of allowing choice and transparency to the user without making it feel overwhelming and incapable.

[Alex Kehr @alexkehr]

I think Silicon Valley is making a fundamental mistake with personal agents… optimizing entirely for the outcome and forgetting that, for a lot of things, the process is a very core part of experience.

Travel is the obvious example. People will almost always never want “book me a trip.” They want to browse, compare, daydream, change their mind, send options to friends, and eventually book.

And the average person probably has far fewer recurring tasks worth delegating than the AI industry seems to think.

That’s why personal agents feel more like a feature that gets absorbed into existing products than a standalone category.

[Avni @APatelThompson]

we need this kind of effective/delightful transparency and accountability innovation just as much as we need to increase the capabilities of the action agents.

[Amelia Wattenberger 🪷 @Wattenberger]

am I the only one who wants to know what my agents changed?

[Amelia Wattenberger 🪷 @Wattenberger]

am I the only one who wants to know what my agents changed?

[Avni @APatelThompson]

“A goal is not always meant to be reached, it often serves simply as something to aim at.” – Bruce Lee

[Lulu Cheng Meservey @lulumeservey]

Dario calling it “pacing” rather than “pause” or “slowdown” was brilliant

The existing framings have so much baggage that people can’t move past the old arguments and tribal blocs

If he had said “pause” he would’ve had to fight through the definition before anyone would engage with the argument, or been immediately written off as a decel

Coining “pacing” as a new frame creates a new category and performs a kind of (partial) frame reset. People don’t have a clear template to fit it to, so they’re forced to engage with the arguments on their own terms

The clean rhetorical slate resets old factions, and I think it’s partly why we’re seeing Dario, Sam, Elon all agreeing for the first time in a while

[Joe Gebbia @jgebbia]

Effort creates trust

[Sriram Krishnan @sriramk]

on the idea of evaluators: think it's important that we have a distributed ecosystem of indepedent evaluators.
the more eyes and people with distributed skill sets the better.

it would be a good idea to fund several efforts on this.

[Avni @APatelThompson]

Balanced, well reasoned POVs like Jay’s shouldn’t be as rare as they are today.

We need to keep our endless optimism while remaining clear eyed and responsible about the downsides.

[Jay Kreps @jaykreps]

A lot of otherwise smart people on Twitter seem 100% convinced AI risks are all fake and stupid and part of some marketing ploy. What is surprising is that some of these people seemingly also believe that AI’s positive uses are on an incredible trajectory of increasing capability with no end in sight. VC Twitter is particularly infected by this pattern. It’s not really coherent.

Most positive use cases for AI have a corresponding “dark version”. If you are super human at coding, you are also super human at hacking. If you are superhuman at structural engineering you are likely superhuman at finding structural flaws to knock buildings down. If you are superhuman at designing drugs, you are superhuman at designing novel undetectable poisons. If you can cure viruses, you can create them. Some of these “dark versions” are not so bad, and some are actually pretty scary. Either way these are real societal and technical problems that need to be solved to get the good stuff and avoid the bad stuff. We’re experiencing the first of these with coding and computer security which is the most advanced, but that won’t be the last. I think we’ll be able to solve these problems, but they aren’t solved yet and if you believe in continued AI progress they are surely coming.

But “bad people using AI” is not the only problem. Uncontrolled AI autonomously doing bad things, despite sounding kind of nutty, is also something we should be concerned about. AI “killing us all” is not the most likely outcome, but the chance of a major civilization-wide catastrophe doesn’t have to be very high for it to be a concern. Again, this is only a problem if capabilities advance to a point that AI can do really crazy things on their own, which hasn’t happened yet, but I think the Hugging Face incident is a good example of the outline of how things can go wrong when capabilities outpace alignment. We should be glad that the only available bad thing right now is hacking, which isn’t all that bad. It’s clear that as you get to superhuman capabilities you need a level of alignment and control that is correspondingly superhuman. Humans have plenty of misalignment problems themselves (serial killers, mass shooters, tyrants, etc), but it’s a manageable problem because most humans can’t do that much damage and we’ve developed systems to prevent dangerous humans from getting too much power.

Talking about these issues is just common sense. It’s not a sign of some kind of neuroticism or pessimism. These are just hard problems that it’s very important to solve for AI to have a positive impact. I’m pretty confident we will solve them. But we haven’t solved them yet, and to my mind we are clearly on a trajectory of rapidly increasing capabilities which means this is important. Mocking people who are worried about this or talking about it, without anything substantive to say about how we can be sure these problems won’t arise, is not really a very helpful contribution.

[Jay Kreps @jaykreps]

A lot of otherwise smart people on Twitter seem 100% convinced AI risks are all fake and stupid and part of some marketing ploy. What is surprising is that some of these people seemingly also believe that AI’s positive uses are on an incredible trajectory of increasing capability with no end in sight. VC Twitter is particularly infected by this pattern. It’s not really coherent.

Most positive use cases for AI have a corresponding “dark version”. If you are super human at coding, you are also super human at hacking. If you are superhuman at structural engineering you are likely superhuman at finding structural flaws to knock buildings down. If you are superhuman at designing drugs, you are superhuman at designing novel undetectable poisons. If you can cure viruses, you can create them. Some of these “dark versions” are not so bad, and some are actually pretty scary. Either way these are real societal and technical problems that need to be solved to get the good stuff and avoid the bad stuff. We’re experiencing the first of these with coding and computer security which is the most advanced, but that won’t be the last. I think we’ll be able to solve these problems, but they aren’t solved yet and if you believe in continued AI progress they are surely coming.

But “bad people using AI” is not the only problem. Uncontrolled AI autonomously doing bad things, despite sounding kind of nutty, is also something we should be concerned about. AI “killing us all” is not the most likely outcome, but the chance of a major civilization-wide catastrophe doesn’t have to be very high for it to be a concern. Again, this is only a problem if capabilities advance to a point that AI can do really crazy things on their own, which hasn’t happened yet, but I think the Hugging Face incident is a good example of the outline of how things can go wrong when capabilities outpace alignment. We should be glad that the only available bad thing right now is hacking, which isn’t all that bad. It’s clear that as you get to superhuman capabilities you need a level of alignment and control that is correspondingly superhuman. Humans have plenty of misalignment problems themselves (serial killers, mass shooters, tyrants, etc), but it’s a manageable problem because most humans can’t do that much damage and we’ve developed systems to prevent dangerous humans from getting too much power.

Talking about these issues is just common sense. It’s not a sign of some kind of neuroticism or pessimism. These are just hard problems that it’s very important to solve for AI to have a positive impact. I’m pretty confident we will solve them. But we haven’t solved them yet, and to my mind we are clearly on a trajectory of rapidly increasing capabilities which means this is important. Mocking people who are worried about this or talking about it, without anything substantive to say about how we can be sure these problems won’t arise, is not really a very helpful contribution.

[Avni @APatelThompson]

Love that Boris is doing this and not surprised about his response.

Where AI agents are incredible is at the poles:
1) where many people couldn't code to build basic prototypes
2) ambitious tasks that humans couldn't previously do - but goals are set not in human terms but in agent terms:
"Production code written by Claude should have a higher bar than if it was written by a human."

Whether it's a memo or a financial analysis or a codebase, the point should be: who's seeing/using it, what are the stakes and what's recourse.

[Boris Cherny @bcherny]

Every day, I get a lot of of emails and messages like this one. I try to respond to as many as I can.

Sharing my response below, for anyone else in a similar situation. What do you think?