I hope AI companies can agree on "AI neutrality" where AI models do not have a preference toward their makers. For example, I love that currently ChatGPT and Claude are happy to help you move over to the competitor's product.
@mhdempsey They may need to reframe a bit by tying it to model diversity or reducing linguistic monoculture risks, but that could fit nicely on https://t.co/vIa7Ci0ZER
twitter's best worst quality is that it forces all takes into an extreme
you're either a tokenmaxxing inferencel looping your agents or you're a promptchud
the truth lies somewhere in between
you absolutely should have agents that can reproduce problems in your products, look for optimizations, automatically open PRs on issues
software has never been perfect, even before AI bugs still made it to production, the more of these you can catch and can fix automatically the better
at the same time, the underlying engineering still does matter - i just spent 3 days fixing some bad product design and architecture that i originally tried to defer to agents to implement
if i wasn't looking at the code, or i was just trusting loops to fix it, my product would be stunted in quality because i would've just kept digging myself into a worse path
anyways, setup your loops but prompting your agents is also fine - get the balance
With more impact-minded money coming online and grantmakers primarily being evaluated by their grantees, you can imagine philanthropy becoming like VC, where nonprofits decide their own round size and donors fight to get in before the round is oversubscribed.
We're on track for ~$30bn/year of philanthropic funding from Anthropic and OpenAI foundations, enough to pay 150,000 salaries.
Now, more than ever, our brightest young people should focus on solving the world's biggest problems, not PowerPoints at PwC.
New blog post: The third wave of American philanthropy
Hundreds of billions of dollars in new philanthropic capital will soon become liquid. The OpenAI Foundation holds 26% of OpenAI, worth about $220B at today’s valuation. Anthropic’s seven co-founders have pledged to give away 80% of their wealth and have instituted the most aggressive donor matching program for employees in tech history.
How much does this all add up to? And how meaningful is that in the context of philanthropy today?
I was doing some simple napkin math to wrap my head around the scale of what’s coming, and radicalized myself in the process. I had dramatically underappreciated the scale of the philanthropic capital that’s about to become available and the corresponding gap in talent and organizations that will be needed to make the most of it.
This piece aims to directionally sketch the scale of what’s coming, the gap in operational capacity needed to absorb it, and what we can do to fill it.
(Link to full post in reply)
Right, but “persuasiveness” is not just some arbitrarily scalable trait, and certainly not one that scales with intelligence. Persuasion is not an innate quality to the persuader but depends on the relationship of that person to the persuaded. You can’t just automate the process.
Cheating by models is a significant enough issue for METR's time-horizon measurement integrity that manually checking for cheating is often the majority of the work involved in a run of our evaluation suite.
types of guy in the AI consciousness debate:
- guy who thinks ai can’t be conscious because it’s “just a stochastic parrot”
- guy who thinks ai must be conscious because claude is a good boi
- guy who hasn’t gotten over 4o
- guy who unironically thinks everything is computer
- guy who claims to have a more nuanced argument for computational functionalism, but it just boils down to everything is computer
- dualist whose belief in dualism is downstream of their belief in god, yet tries to argue the inverse
- guy who doesn’t understand the difference between cognition and p-consciousness
- guy who asserts illusionism but has apparently wrestled with zero of the implications other than “reductive materialism wins again”
- guy who says the hard problem is easy, but then proceeds to only answer the easy problem
- guy who rejects ai consciousness because otherwise it might be wrong to abuse claude with death threats to make CRUD apps faster
- guy who argues that consciousness is is the key to moral patienthood, but completely ignores that when discussing animal rights
- eliezer yudkowsky being pedantic
- guy being pedantic about eliezer yudkowsky’s pedantry
- guy who rejects dualism because that would make mind uploading impossible and mean that he finally has to confront the inevitability of his own death
- guy who thinks this argument is unresolvable so everyone should just shut up and accept his position (which obviously deserves the benefit of the doubt)
- guy who would literally cut off his own hand if he thought there were a 1 in 10 trillion chance of creating ~infinite utility~
- guy who just thinks that redness is, like, super weird, man. can’t explain that!
- guy with a rarely-updated philosophy blog despite not majoring in philosophy or even reading that many books, talking about how “the whole field is up its own ass”
- academic philosopher who, for some reason, expects a higher caliber of discussion on x dot com the everything app
- guy who thinks that vectors are literally emotions and bites the bullet that, yes, your thermostat does feel hot
- panpsychist who took dmt once and contributes almost nothing to the conversation
- guy who is literally a solipsist but is still really invested in convincing strangers on the internet that he’s right
any that i missed?
some quick thoughts on mythos and tiered deployment
1) as models get more powerful and start to show dangerous capabilities, it will make sense to do tiered deployment to trusted enterprises first
2) this will enable domain experts and companies with access to real world environments to evaluate the extent of capabilities uplift created by the model
3) as in the present case, it may also open up the ability of these companies to use these models defensively as in the case of cyber
4) it especially makes sense to deploy those models first to companies with expertise in high risk domains e.g. pharmaceutical, chemical and cybersecurity
5) but, this may end up being controversial as it will give some companies access to the next generation of models first
6) and, these companies will have first mover advantage building around the new capabilities
7) in addition, labs will probably tend to give preference to those companies with whom they have advantageous commercial relationships or in whom they have invested
8) i remember that there was a brief period in 2023/2024 where it felt like openai fund companies had a big advantage because they got early access
9) but, then the market opened up and became more competitive and releases became more frequent and this seemed no longer to be an issue
@karsenthil The bottleneck isn't "right model, right inputs." It's that verifying agent output requires the same expertise the agent was supposed to replace. You need the expert to check the expert-replacement. That's the problem almost no one in this space is solving for.
Safety = moral tax framing ignores enterprises future reality imo because companies buy risk-managed systems rather than cool models. For now it's seen as a morality but once agents touch money/health/etc. safety becomes a procurement requirement.
So in that sense market incentives would force it.
1/ Some Simple Economics of AGI—🔥🧵
Right now, there is a low-grade panic running through the economy. Everyone is asking the same anxious question: what exactly is AI going to automate, and what will be left for us?
Obviously it doesn't help that any AI experiment such as Moltbook is immediately populated by countless crypto grifts trying to pump their memecoin off of the attention
Crypto bros really became the epitome of annoying parasites ... like you can't enjoy a bit of lakeside summer vacation without mosquitos swarming your face
trend I’m noticing
People think they’re being productive using fancy agent setups and AI tools
But in reality it’s mostly dopamine loop chasing and procrastination
It feels very smart and useful to have AI generate you some massive block of analysis and strategy. The brain loves the behavior of “prompt and see”, it’s literally a variable reward (like scrolling or gambling)
But doing anything useful in the world takes a lot of time, consistency, and many years of just doing the boring things over and over again
Startup costs are now 0, but long term execution is still very hard. Most people will perpetually pivot because of this
Incentivising users to make purchases based on chat context, by proactively surfacing specific products, prices, images, reviews and buy buttons right in the conversation, is functionally identical to targeted advertising, no matter how it's branded as "helpful shopping research" or "agentic assistance."
Until OpenAI publicly and unambiguously declares, in clear policy language, that neither OpenAI nor any entity acting on its behalf (including partners in the Agentic Commerce Protocol, Instant Checkout integrations, etc.) receives any commission, revenue share, transaction fee, affiliate payment, placement incentive, referral remuneration, or other financial benefit whatsoever tied to:
1. clicks or click-throughs on recommended products
2. the ranking / prominence / inclusion of specific items in results
3. actual completed purchases originating from ChatGPT recommendations
...then these features remain advertising in practice, regardless of whether they're currently "organic and unsponsored" in some flows (as OpenAI hilariously claims at their 2025 Instant Checkout launch lmfao) or whether the monetisation is confined to checkout-only at present.
btw - OAI has already confirmed merchant fees on completed Instant Checkout purchases (reported as 2–4% depending on partner/timing), and Sam Altman has openly discussed affiliate-style commissions (2%) as a preferred path. Without a firm "zero financial incentive across all shopping surfaces and recommendation modes" statement covering the exact experience people are seeing today, then it's literally ads and nothing will change that.