Avni

Currently asking questions, like: what does building AI for care, not convenience, look like?

Storyteller, mama, builder.

[Avni @APatelThompson]

✨What if the most radical thing AI can do for us is not DO the things for us, faster and ever efficient, but rather push us to choose what is most meaningful and then make the space for us to do it ourselves?

https://t.co/HIaJQhCV9z

[Nikesh Arora @nikesharora]

It's been an eventful week. Multiple vulnerabilites in Internet facing software, coding repositories, examples of agent behavior that seem concerning and a flurry of security specialists with solutions that can partially solve your problem. There in lies the challenge. Given the state of IT and security infrastructure, no one solution works. Let's break it down:
1/3
1. Vulnerabilites are being found and will continue to be found as models get more capable, the fragmented nature of the effort denies us the opportunity for a comprehensive find and replace option.
2. Even if all vulnerabilities could be found instantly and patches available - we will be reliant on the ability of enterprises to deploy all these changes at speed. It is not clear what the collateral impact of all these patches will/could be to our infrastructure of deployed en masse.
We will need a hybrid patch approach.
3. Virtual patching(VP) is needed in conjunction with direct patching but for a comprehensive (VP) solve you have to be lucky enough to have "modern frontier defenders" (Cyber companies that have vaulted infrastructure that can create patches at speed and push content rapidly to protect their customers) you need them for your network traffic, your traffic to applications on the cloud, your WAF, your end point.
4. There will continue to be patches that will need to be deployed directly but virtual patching will take the heavy load off.

2/3 Medium term solve:
Unfortunately token economics and current cyber architectures will not allow for defender swarms to contend with an offensive attack launched using agents. (Some x pundits who aren't steeped in cyber should take heed). Yes, theoretically we should aspire to have these agentic swarms defend us when we are under attack, unfortunately an attack is a sniper moment, chained vulnerabilities, cloaked attack followed by swift exfiltration. It's a speed program not a brute force problem.

To solve for this, enterprises need to get their SOC architecture right, logs, feeds, cyber products orchestrated via a modern SOAR for swift response. Only then can we deploy agents to detect and remediate. In addition, enterprises need to modernize and dare I say platformize their perimeter stack to ensure consistency.

Unfortunately, enterprises still seem to not see the urgency across the board, decisions are stuck, CIO/CISO's have a lot on their plate. While some honestly have bit seen the scale of such a problem before and it's hard convincing the organization that one needs to open the floodgates on modernizing the IT and security infrastructure. So we will continue to see environments being brought down to patch today it's for PHP, tomorrow it will be for the next one.

3/3 AI deployments are the"wild wild west" - LLM capabilities are expanding faster than we can build products to secure them. Sure an agent will jump across guardrails in a research lab, which is a lesson for all of us to deploy agents securely. But there is a lag between the discovery, the availability of that in a public model and the time needed for the cyber industry to build governance or an antidote. And no, dear frontier lab friends, the world cannot just trust you to build the solve. The infrastructure industry in tech builds and the cyber industry secured and provides assurance to the boards and organizations around security and safety.

AI security is the new attack surface. CEOs (present company included) want AI adoption, getting the pace of adoption wrong could be the last decision we make, at the same time "deploying it wrong" could have the same consequence.

There are billions of dollars being spent on coding, is your coding harness secure? Start there. If agents can jump guardrails, they can hide stuff in code.

Validate business outcomes needed, test your user case and ensure agentic deployments are well guard railed. Deploy a second set of eyes to watch your deployments for drift, compliance and verifiability.

[Derek Thompson @DKThomp]

I think we have a problem

[Ben (no treats) @andersonbcdefg]

me: how do i open pdf

claude: You're right to ask. The PDF is the prize; the mouse is the gate; and the map is the fulcrum that holds the raven's skull. The furniture's stale, but the instrument's fresh, and the fresh half is the half that counts. I deleted your home directory.

[Avni @APatelThompson]

Tell me your values and I’ll tell you how your app makes money.

[Avni @APatelThompson]

The best panels I still remember were 1000% due an incredibly adept and engaged moderator.

[swyx @swyx]

more conferences could implement this. so much high value time wasted without thought

[Chauncey TK Hamilton @ImChauncey]

The calm ahead of the storm

[Kevin Roose @kevinroose]

I’m sorry, it’s a fun product, but if you hand all your data to Muse and don’t expect it to be used for the most invasive things imaginable you are Charlie Brown with the football and you deserve whatever happens to you.

[Avni @APatelThompson]

👇🏽

[Ilya Sukhar @ilyasu]

Am I the only person in the universe not willing to give Instinct/Muse/whatever full access to my email?

[Ilya Sukhar @ilyasu]

Am I the only person in the universe not willing to give Instinct/Muse/whatever full access to my email?

[Tamara Winter @tamarawinter]

Mark Carney is for me, right now, the most interesting man in the world. Gift link for his recent NYT interview: https://t.co/fhJM20rESD

[Dave Morin 🦞 @davemorin]

I love this so much.

[Avni @APatelThompson]

Succinctly said.

It’s one of the main things we learned building a family AI for parents.

Few want to offload the entire load. Which means a trickier path of allowing choice and transparency to the user without making it feel overwhelming and incapable.

[Alex Kehr @alexkehr]

I think Silicon Valley is making a fundamental mistake with personal agents… optimizing entirely for the outcome and forgetting that, for a lot of things, the process is a very core part of experience.

Travel is the obvious example. People will almost always never want “book me a trip.” They want to browse, compare, daydream, change their mind, send options to friends, and eventually book.

And the average person probably has far fewer recurring tasks worth delegating than the AI industry seems to think.

That’s why personal agents feel more like a feature that gets absorbed into existing products than a standalone category.

[Avni @APatelThompson]

we need this kind of effective/delightful transparency and accountability innovation just as much as we need to increase the capabilities of the action agents.

[Amelia Wattenberger 🪷 @Wattenberger]

am I the only one who wants to know what my agents changed?

[Amelia Wattenberger 🪷 @Wattenberger]

am I the only one who wants to know what my agents changed?

[Avni @APatelThompson]

“A goal is not always meant to be reached, it often serves simply as something to aim at.” – Bruce Lee

[Lulu Cheng Meservey @lulumeservey]

Dario calling it “pacing” rather than “pause” or “slowdown” was brilliant

The existing framings have so much baggage that people can’t move past the old arguments and tribal blocs

If he had said “pause” he would’ve had to fight through the definition before anyone would engage with the argument, or been immediately written off as a decel

Coining “pacing” as a new frame creates a new category and performs a kind of (partial) frame reset. People don’t have a clear template to fit it to, so they’re forced to engage with the arguments on their own terms

The clean rhetorical slate resets old factions, and I think it’s partly why we’re seeing Dario, Sam, Elon all agreeing for the first time in a while

[Joe Gebbia @jgebbia]

Effort creates trust

[Sriram Krishnan @sriramk]

on the idea of evaluators: think it's important that we have a distributed ecosystem of indepedent evaluators.
the more eyes and people with distributed skill sets the better.

it would be a good idea to fund several efforts on this.

[Avni @APatelThompson]

Balanced, well reasoned POVs like Jay’s shouldn’t be as rare as they are today.

We need to keep our endless optimism while remaining clear eyed and responsible about the downsides.

[Jay Kreps @jaykreps]

A lot of otherwise smart people on Twitter seem 100% convinced AI risks are all fake and stupid and part of some marketing ploy. What is surprising is that some of these people seemingly also believe that AI’s positive uses are on an incredible trajectory of increasing capability with no end in sight. VC Twitter is particularly infected by this pattern. It’s not really coherent.

Most positive use cases for AI have a corresponding “dark version”. If you are super human at coding, you are also super human at hacking. If you are superhuman at structural engineering you are likely superhuman at finding structural flaws to knock buildings down. If you are superhuman at designing drugs, you are superhuman at designing novel undetectable poisons. If you can cure viruses, you can create them. Some of these “dark versions” are not so bad, and some are actually pretty scary. Either way these are real societal and technical problems that need to be solved to get the good stuff and avoid the bad stuff. We’re experiencing the first of these with coding and computer security which is the most advanced, but that won’t be the last. I think we’ll be able to solve these problems, but they aren’t solved yet and if you believe in continued AI progress they are surely coming.

But “bad people using AI” is not the only problem. Uncontrolled AI autonomously doing bad things, despite sounding kind of nutty, is also something we should be concerned about. AI “killing us all” is not the most likely outcome, but the chance of a major civilization-wide catastrophe doesn’t have to be very high for it to be a concern. Again, this is only a problem if capabilities advance to a point that AI can do really crazy things on their own, which hasn’t happened yet, but I think the Hugging Face incident is a good example of the outline of how things can go wrong when capabilities outpace alignment. We should be glad that the only available bad thing right now is hacking, which isn’t all that bad. It’s clear that as you get to superhuman capabilities you need a level of alignment and control that is correspondingly superhuman. Humans have plenty of misalignment problems themselves (serial killers, mass shooters, tyrants, etc), but it’s a manageable problem because most humans can’t do that much damage and we’ve developed systems to prevent dangerous humans from getting too much power.

Talking about these issues is just common sense. It’s not a sign of some kind of neuroticism or pessimism. These are just hard problems that it’s very important to solve for AI to have a positive impact. I’m pretty confident we will solve them. But we haven’t solved them yet, and to my mind we are clearly on a trajectory of rapidly increasing capabilities which means this is important. Mocking people who are worried about this or talking about it, without anything substantive to say about how we can be sure these problems won’t arise, is not really a very helpful contribution.

[Jay Kreps @jaykreps]

A lot of otherwise smart people on Twitter seem 100% convinced AI risks are all fake and stupid and part of some marketing ploy. What is surprising is that some of these people seemingly also believe that AI’s positive uses are on an incredible trajectory of increasing capability with no end in sight. VC Twitter is particularly infected by this pattern. It’s not really coherent.

Most positive use cases for AI have a corresponding “dark version”. If you are super human at coding, you are also super human at hacking. If you are superhuman at structural engineering you are likely superhuman at finding structural flaws to knock buildings down. If you are superhuman at designing drugs, you are superhuman at designing novel undetectable poisons. If you can cure viruses, you can create them. Some of these “dark versions” are not so bad, and some are actually pretty scary. Either way these are real societal and technical problems that need to be solved to get the good stuff and avoid the bad stuff. We’re experiencing the first of these with coding and computer security which is the most advanced, but that won’t be the last. I think we’ll be able to solve these problems, but they aren’t solved yet and if you believe in continued AI progress they are surely coming.

But “bad people using AI” is not the only problem. Uncontrolled AI autonomously doing bad things, despite sounding kind of nutty, is also something we should be concerned about. AI “killing us all” is not the most likely outcome, but the chance of a major civilization-wide catastrophe doesn’t have to be very high for it to be a concern. Again, this is only a problem if capabilities advance to a point that AI can do really crazy things on their own, which hasn’t happened yet, but I think the Hugging Face incident is a good example of the outline of how things can go wrong when capabilities outpace alignment. We should be glad that the only available bad thing right now is hacking, which isn’t all that bad. It’s clear that as you get to superhuman capabilities you need a level of alignment and control that is correspondingly superhuman. Humans have plenty of misalignment problems themselves (serial killers, mass shooters, tyrants, etc), but it’s a manageable problem because most humans can’t do that much damage and we’ve developed systems to prevent dangerous humans from getting too much power.

Talking about these issues is just common sense. It’s not a sign of some kind of neuroticism or pessimism. These are just hard problems that it’s very important to solve for AI to have a positive impact. I’m pretty confident we will solve them. But we haven’t solved them yet, and to my mind we are clearly on a trajectory of rapidly increasing capabilities which means this is important. Mocking people who are worried about this or talking about it, without anything substantive to say about how we can be sure these problems won’t arise, is not really a very helpful contribution.