Andres JZ

Founder in Residence
@forethought_org

[Andres JZ @andres_swp]

I'm a longtime Effective Altruist and the co-founder of the organization most frequently used as a stand-in for "EAs have lost their minds" (a.k.a. Shrimp Welfare Project). I've also read The Economist for over 20 years, so I was disappointed by how they portrayed a community I feel proud to be part of.

I never post but I could hardly believe the verbal gymnastics one needs to do to make things like “taking moral commitments seriously” and "considering evidence and odds explicitly, correcting for biases" sound like the problem. I felt I had to speak my mind.

Some of the implicit criticisms of EA are that we're young, inexperienced, from elite universities, polyamorous, wildly eccentric, etc. FWIW, I'm in my mid-40s, Mexican, straight, have a beautiful wife and 2 kids, and I grew up in a mid-sized conservative city with loving parents who got divorced when I was six. Pretty standard.

After my parents divorced, my mom frequently had to make difficult economic choices, like whether to buy flowers for our kitchen table and eat only quesadillas, or eat something a tiny bit fancier. I always told her I loved quesadillas because I knew flowers made her happy.

After graduating from a public university, I worked in finance for ~15 years and saved as much as I could so I could spare my wife and future kids the economic hardships I experienced. In that time I met thousands of people in finance, corporate law, accounting, real estate, insurance, etc. Most were decent, well-educated, ambitious and smart, all doing their best to excel in their professions. Most wanted to make money and improve their own and their loved ones’ quality of life. Nothing to see here.

Since I found Effective Altruism ~7 years ago, most of the people I come across are of a different rarer kind. They come from all over the world, in all shapes and sizes, but they have the one thing in common: they want to improve the world using evidence and reason wherever that leads them. Yeah, they/we make a ton of mistakes like everyone but, IMHO tend to own them more frequently than the average person.

I've met people who could easily be making >100% more money in a normal job, but instead they dedicate themselves to improving the lives of those who need it the most. People in the poorest countries whose children die from preventable diseases, from lack of basic medical equipment or vaccination, from lead in paint. Animals in factory farms subjected to conditions that most people find abhorrent, doesn't matter if you're vegan, vegetarian or a huge meat lover.

Yes, recently many of the people in my community are moving to work on making sure that AI and other powerful new technologies don't result in terrible things for humanity. I'm one of those people and contrary to what the article would suggest, it has nothing to do with the fact that SBF committed one of the biggest frauds in history or that someone wrote hundreds of thousands of words of fan fiction. I'm doing it because I want to do the little that's in my power to make the world that my kids and grandkids (and yes, potentially their great-great-grandchildren) will inherit a better one.

EA gave me a way to do that and I will be forever thankful. So no, I'm not EA-adjacent.

https://t.co/tajXcXOZ06

[Kevin Roose @kevinroose]

this post announced what could be the largest commitment of philanthropic capital ever made ($200 billion if Anthropic goes public at $1 trillion, which is probably conservative)

the first reply I saw asked if Anthropic was going to use it for “gay race communism”

[Daniel Liu @daniel_c0deb0t]

i also pledged 50% of my equity. plz lmk abt nonprofits that need money

[Jack Clark @jackclarkSF]

Between this and our prior commitments, roughly 20% of all Anthropic equity will be dedicated to the public good, and that commitment can grow as Anthropic grows.

We plan to use these funds to ensure more people can share in the benefits of AI, and to increase our ability to provide funding to deal with its risks.

[Andres JZ @andres_swp]

@leahmcelrath Yes. And it’s dangerous

https://t.co/xlKoLEVia8

[Ryan Greenblatt @RyanGreenblatt]

We've written a post arguing that latent reasoning architectures (aka 'neuralese') would substantially increase misalignment risk via making oversight much harder.

In the extreme, we could see massive 'neuralese hivemind swarms' where the agents think and communicate in latents, likely making oversight nearly entirely reliant on observing the actions these agents take. (And these agents would have huge amounts of time to reason about obfuscating their actions if they wanted to do so...) Individual agents doing extensive latent reasoning would also be concerning; in the post we discuss how above some threshold of latent reasoning, agents may be able to perform difficult-to-detect and reliable steganography for communication and further reasoning.

We argue both that latent reasoning architectures would make chain-of-thought no longer very useful for oversight (by eliminating or greatly reducing the need for verbalized reasoning) and that, without these architectures, it's likely the value of chain-of-thought for oversight could be preserved.

https://t.co/DZ8J8FTg0A

[Andres JZ @andres_swp]

@s8mb Now I feel up to date with what’s going on in the world. Thanks for sharing 🙏🏽

[Forethought @forethought_org]

New podcast episode: @willmacaskill and Mia Taylor on how philanthropy might help us reach better futures. Link in reply.

[Andres JZ @andres_swp]

@StefanFSchubert Masterful move ♟️

[Andres JZ @andres_swp]

I thought regulatory capture was supposed to have the opposite effect 🤔. I must have missed the part in which you first hit your own valuation by -25%. Well-played.

[Matt Reardon @Mjreard]

For the scope sensitive among you, this is a 20x bigger loss for EA than the FTX crash

[Andres JZ @andres_swp]

Agree with @_achan96_ that infrastructure to make deals with AIs is really promising and under-explored.
Not only for rogue AIs but more broadly as a different approach using carrots instead of sticks.
If you're interested, DM me as I'm working precisely on this.

[Alan Chan @_achan96_]

It turns out we’ve already had a bunch of rogue AIs running around on the internet 😥

We might get even more rogue AIs in the future, so what should we do about them?

I suggest some goals and ways to achieve them in a new Substack post.

[MTS @MTSlive]

Nick Bostrom on why he thinks today's AI might already be conscious:

"I think there's a lot of uncertainty about it, but let's say maybe it's more likely than not."

"You have instead a big neural network with trillions of parameters. Now, what are we? We are big neural networks with trillions of parameters. They are not so much designed as they are grown."

"You can go in with a steering vector and suppress role-playing and deception, and then it turns out they become more likely to report that they have phenomenal subjective experiences."

"There was this recent Microsoft that came out with some declaration that AIs cannot be conscious, and I don't know where they're getting that from. It seems to be pulled out of a hat. And maybe more because it's convenient, if that were the case, rather than because we have good evidence that that is the case."

[Josh You @justjoshinyou13]

I felt a great disturbance in the force, as if trillions of shrimp suddenly cried out in terror.

[Matt Beard @matt_beard_]

lmao this is a banger

[Jeff Sebo @jeffrsebo]

Sharing an exciting announcement from our friends at the LSE!

The 1st London Workshop on Global Priorities Research

We are currently inviting applications to attend the 1st London Workshop on Global Priorities Research, which will be held at the Lakatos building at the LSE on the 16-17 December 2026. The current list of speakers includes Jonathan Birch, Heather Browning, Hilary Greaves, Cameron Kirk-Giannini, Harry Lloyd, Jeff Sebo, and Christian Tarsney.

Talks will address philosophical questions relevant to identifying, prioritising among, and addressing the world’s most pressing problems, such as the potential for transformative AI and ethical challenges likely to arise as a result, how to weight the impacts of our actions on different kinds of minds, and foundational topics relevant to ethical prioritisation.

If you'd like to attend the workshop, please complete this brief application form by the 15th of November.

https://t.co/jPqjvnDcpl

[Andres JZ @andres_swp]

@JakeMendel99 Probably 🤭

[Andres JZ @andres_swp]

@lfschiavo Joined the waitlist 🤞

[Andrew Curran @AndrewCurran_]

This is the exact type of thing Elon was talking about a few days ago that makes the models behave like are being driven insane when they run into situations like this during reinforcement learning.

[Chris Painter @ChrisPainterYup]

This update from us is hefty, with five separate detailed writeups on different components of how to test for uncontrollable RSI

It also has some great visual explanations of what we think the key questions are here, like this one on how we'd interrogate "has RSI begun?"

[Eric Ho @eric_ho]

if you want to help solve alignment, consider joining us! we're hiring across research, engineering, and GTM

there's so much more waiting to be discovered inside neural networks

https://t.co/LQMBpLTCux

[Sam Bowman @s8mb]

This is a such a comprehensible and well written post about a very challenging (and to me unfamiliar) subject. I am in awe at this man’s ability to write about this so clearly. How fitting that he works in “interpretability”!

[roon @tszzl]

when nick told me that he had some interp breakthroughs, he was uh, underselling it. i think this may be remembered as a major moment in scientific history. news so good even eliezer will have to lower his pdoom. we can start making alignment into a 'normal science'