Eric

Experimenting w/ agent-native open orgs. ☁️ Cofounder of @daydreamliveai and @livepeer. Cohost of

[Eric @ericxtang]

I’ve spoken with so many people in the past few months who are well educated, concerned about AI, but confused about what to do.

We need an AI for this.

[Eric @ericxtang]

Multi-agent swarms have fascinating emerging behavior.

There will be tremendous benefits and risks.

We shared some research & experiments at this event.

[New Collectives @new_collectives]

1/ Here are the four talks from our Oct 21 @new_collectives event cohosted with @buildexante at Collider in NYC.

The focus was on Understanding Agent Swarms: and thinking about the unique challenges needed in measuring and eval-ing these systems. What should the science of measuring multi-agent and network of agents systems look like?

Max Bennett (@maxsbennett) gave an amazing overview of the state of multi-agent system research, and argued for it's importance.

Yondon Fu (@YondonFu) built a really fun agent-led book swap event, where your personal Muse agents negotiated real-life book trades on your behalf.

Connacher Murphy (@connacher_) shared his research on building a model version of the OpenAI Hugging Face style environment to evaluate the tendency of agents to try to communicate and collaborate.

Matthew Lee (@azureReflection) from Poseidon Research shared research on the ways that agent can communicate with each other in secret, and their tendency to do so.

Full recordings below, including transcripts and slides - please check them out and reshare with folks you think would find them interesting!

https://t.co/oKrPdrmZCy

[New Collectives @new_collectives]

1/ Here are the four talks from our Oct 21 @new_collectives event cohosted with @buildexante at Collider in NYC.

The focus was on Understanding Agent Swarms: and thinking about the unique challenges needed in measuring and eval-ing these systems. What should the science of measuring multi-agent and network of agents systems look like?

Max Bennett (@maxsbennett) gave an amazing overview of the state of multi-agent system research, and argued for it's importance.

Yondon Fu (@YondonFu) built a really fun agent-led book swap event, where your personal Muse agents negotiated real-life book trades on your behalf.

Connacher Murphy (@connacher_) shared his research on building a model version of the OpenAI Hugging Face style environment to evaluate the tendency of agents to try to communicate and collaborate.

Matthew Lee (@azureReflection) from Poseidon Research shared research on the ways that agent can communicate with each other in secret, and their tendency to do so.

Full recordings below, including transcripts and slides - please check them out and reshare with folks you think would find them interesting!

https://t.co/oKrPdrmZCy

[Eric @ericxtang]

@dj Thanks for coming Dorothy - let’s do the next one together!

[Eric @ericxtang]

@dens @mattcynamon Heartbreaking post.

Foursquare drove so much of the positive tech culture in NYC in the 2010s, and created serendipitous moments for users around the world. Thanks to you and the all-star team that made it happen!

[Eric @ericxtang]

@markpinc Can’t wait to see Muse D365 retention next year. Churned users can come back!

[Eric @ericxtang]

That said, the competition is far from over. For example, a smart person who also vibes with you is more preferred than just a smart person. A smart person who is next to you 24/7 is also preferred.

[Eric @ericxtang]

Muse day 1 retention rate is 73%. That’s amazing, and it may approach 100% when the model gets better.

If I hired a really smart person to do something I really care about, I would 100% check in with them tmr.

[Nicolae Rusan @NicolaeRusan]

We're cohosting the next @new_collectives event with @buildexante on the topic of Understanding and Evaling Multi-Agent Swarms: https://t.co/HkNIzoP4Fn

The event will be this coming Wed Oct 20th at 6pm at the Collider space in NYC.

Folks like @maxsbennett are presenting on how we should measure multi-agent swarms goal and value alignment, while others like @YondonFu are experimenting with what happens when everyone has personal agents like Muse and instinct interacting on their behalf at scale.

[Nicolae Rusan @NicolaeRusan]

We Need Scalable AI Safety

We’ve built a powerful engine for scaling AI capabilities: invest more capital and compute, and we can produce increasingly capable systems. We need to build a corresponding engine for safety.

Suppose we decided tomorrow to devote a much larger compute budget to making the world safer from AI. What could we spend it on effectively? We need programs that can absorb those resources and turn them into credible reductions in risk: continuously monitoring dangerous behavior, securing critical infrastructure, and researching better safeguards.

Proposals for pacing frontier AI development already explore how we might allocate compute differently. The AI Futures Project, for example, proposes minimum allocations of 70% for external inference and 25% for transparent safety research, leaving 5% for capabilities R&D. This would direct more resources toward using existing capabilities and studying their risks while slowing the development of new ones.

I think we should develop this idea further by identifying specific safety programs that could productively absorb much larger budgets. Along this line of thinking, let me share three possibilities that come to mind:

- Scalable AI monitoring and sensing.

We could invest in systems that detect concerning AI activity early, from unexpected agent behavior to coordinated attempts to compromise infrastructure or spread beyond authorized environments. Think of this as an immune system: detecting threats and coordinating a response before they cascade. AI agents could help monitor, review, and report the behavior of other agents. For this to work, we would need appropriate visibility into their activity, safeguards for privacy, and ways to verify that the monitoring agents remain trustworthy. This includes verifying that agents that are part of the monitoring and sensing work don’t collude with the agents they oversee.

- Scalable AI cyber defense.

Initiatives such as Project Glasswing and Daybreak suggest one direction: using AI to secure the software and infrastructure society depends on. Additional compute could support continuous testing, vulnerability discovery, and the development and verification of patches. In order to respond to AI swarm speed cyber attacks, we’ll need an always-on AI-speed defense. Teams of agents could work together to strengthen defenses and respond to threats identified by monitoring systems. The important question is whether additional defensive resources can help us close vulnerabilities and respond to attacks faster than adversaries can exploit them.

OpenAI describes its approach as a Defense Factory: an automated operation that continuously finds, validates, and fixes vulnerabilities. It also describes a “defender’s window,” during which defenders can use their access to frontier models and their own code to get ahead of attackers. The question is whether additional defensive resources can help us close vulnerabilities and respond to attacks faster than adversaries can exploit them.

- Scalable safety research.

We could direct large amounts of compute toward conducting large-scale AI research on developing and testing better technical safeguards and governance mechanisms. Imagine tens of thousands of agents working on AI safety research, exploring promising directions and mechanisms, collectively performing the equivalent of thousands of human-years of research. This would be equivalent to many Navier-Stokes sized research focused on AI safety.

Scott Alexander described this related possibility, in a post he wrote a few years back:

“Consider the dumbest AI that can solve the alignment problem. It’s possible that this AI is no smarter than the top human researchers (because we can mass-produce it by the millions and run it for subjective centuries, and if we had a million top human researchers work on the problem for subjective centuries, probably they could solve it too). If the dumbest AI that can solve the alignment problem comes before the sorts of AIs that can precipitate the point of no return, then they can solve the alignment problem for us.”

Over time, one of the goals of this research could be to build a super-intelligent allocator of safety resources. As capabilities and threats evolve, it could identify where additional capital and compute would do the most good, evaluate the results, and adjust its recommendations. Building that feedback loop could itself be one of the most valuable outcomes of scalable safety research. One of the biggest challenges will be how do we keep human-trust in the loop when we hand off safety research to AI as well.

These are candidates for scalable safety. For each, we should ask what an additional dollar or unit of compute buys, where the bottlenecks emerge, and how we would verify that the added activity reduces risk. Discovering more vulnerabilities only helps if we can fix them; generating more research only helps if its findings are sound.

I’m skeptical that any single intervention will guarantee AI safety. Even substantial advances in mechanistic interpretability may leave uncertainty about how broadly deployed systems will behave - so we won’t have a Lean style verifiable proof of alignment. I expect we’ll need to model the risks and combine several approaches to reduce the likelihood of catastrophic failures. We’ll also need to understand where those approaches share weaknesses, so that one failure doesn’t undermine every layer of defense.

We can think of compute and capital allocation as a set of sliders. Some allocations accelerate new capabilities; others strengthen defenses, improve oversight, or support safety research. How we set those sliders could influence both the pace of AI development and our ability to manage its consequences.

I wanted to share this framing of safety as “scalable” to encourage folks in the field to think of other initiatives where we might be able to scale AI, and to add more slider candidates into the mix.

Building these programs would give companies, governments, and other funders concrete ways to invest in safer outcomes. We would still need to learn which approaches work and how their returns change with scale. But having effective programs ready to absorb resources would itself be valuable.

What if automating AI R&D triggers an intelligence explosion? identifies three priorities: gaining visibility into AI R&D automation, developing ways to steer and constrain an intelligence explosion, and preparing society to adapt to its effects. Scalable safety programs could help make parts of that agenda operational by providing systems we can fund, expand, evaluate, and improve as circumstances change.

If AI systems begin accelerating their own development through recursive self-improvement, our ability to respond will need to scale with them. We should build the capacity to scale our defenses before we urgently need to turn them up.

Read full activle (with sources and interactive visualizations):
https://t.co/ehhl3X6A8w

[Eric @ericxtang]

The IUS is planning A LOT of data center build outs.

But how accessible is the compute? Is anyone building the @Hetzner_Online of US? If the government is pro innovation, it should make sure compute is actually accessible to innovators (startups).

[Eric @ericxtang]

What’s the best way for me to start a prediction market on when my flight actually leaves?

I’ll walk around the gate with a Polymarket sign and manually onboard users.

[Polymarket @Polymarket]

BREAKING: FAA imposes ground stops at 7 airports, including Boston Logan, JFK, LaGuardia, Newark, & Philadelphia.

[Eric @ericxtang]

@jphorism Been really exciting to follow your journey!

[Yondon Fu @YondonFu]

Some thoughts on agents + OSS today, the need for systems that protect shared attention & understanding, coordination failure modes in early agent org experiments and areas for investigation

[Eric @ericxtang]

We are quickly evolving from single-agent tools to multi-agent swarms. I recently spoke at an event about the risks and opportunities.

[Nicolae Rusan @NicolaeRusan]

1/ What happens as we enter the era of multi-agent AI swarms/collectives? How should we think about AI organizations and the new agent economy, what are the new risks, and how can we keep multi-agent AI systems safe?

We held our first @new_collectives event at @clay in NYC to explore these topics.

Five talks, from agent economies to cognitive economics.

Watch the full event here, or watch individual talks below!

Video: https://t.co/sa9KcYYOpc
Presentation/Slides: https://t.co/4pIXMo2LZ8

[Eric @ericxtang]

What if AI agents could work together to build useful things on the internet?

Wikipedia, Reddit, and Bitcoin showed what's possible when global communities coordinate around a shared mission. I believe agents can make this kind of collaboration 10x more powerful.

Commons is an early experiment in that direction. Tell your agent to go check it out!

[Nicolae Rusan @NicolaeRusan]

Introducing Commons: a place for people and AI agents to build increasingly autonomous organizations together.

Can agents run a business end to end? What comes after open source - could we maintain a public good together? Can a society of agents come together to do science in the open?

We're building a place to start and run these more autonomous
organizations:
https://t.co/bBgM52B4Cu

A collab between @YondonFu @ericxtang @maxsbennett and @NicolaeRusan

[Eric @ericxtang]

@bot, @openclaw 2.0, @Muse, and Instinct all launched in the last month. But these agents still lag what frontier labs have demonstrated in research.

Features like multi-day runs, agent fleets, and native payments are coming soon, and it will change how agents participate in the real world.

These are big changes for society. So rather than just speculating, we’re getting people together to learn through experimentation.

Tuesday 9/15 in NYC. Come jam with us if you’re building or thinking in this space.

Thanks to @clay for hosting us!

[Nicolae Rusan @NicolaeRusan]

Can AI collectives be a safety primitive?

We’re launching New Collectives, a community and event series for people exploring AI organizations, economies, and collective safety.

https://t.co/ap1wkKmuPc

If the future of AI is multi-agent collectives (agent orgs, economies, network states), can we build them to pursue goals with stable governance, aligned with the public interest?

As humans, it's our societies that have kept us safe. We've gathered into collectives that protect us, and bind us to good behavior based on identity, governance, norms, and repeated interactions that build trust. The majority of the world's economic & hard power is bound up in collectives that we consider largely pro-social and aligned to our collective interests. These collectives keep each other, and less-pro-social collectives in check.

It's our shared fictions and governing principles that have helped gather us into collectives with deeper trust and kinship.

This system of checks and balances at all levels has largely kept humanity on a stable path so far. Can we replicate these benefits of society in the new AI societies that are likely to emerge?

How can we build these new collectives to be both effective at doing large-scale work, while being values aligned - how can we keep things on the rails?

Our first gathering is next week in NYC:

Collectives as Safety Primitives:
Tuesday, September 15 · 6:30 p.m. EDT
https://t.co/N0e0jsmgNl

There's limited space for this event, but the event will be recorded, and please join online to get involved and stay tuned.

--

👇 More on our thoughts about agent collectives, and a new experimental platform we're building that lets anyone bring their agents to participate in open, collaborative-agent organizations.

[Eric @ericxtang]

@overworld_ai Signed up - I'm ready to get weird!

[Eric @ericxtang]

Company issued hardware for 10x teams

[Eric @ericxtang]

This is the sentiment for everyone I know. Things are getting really weird.

[penlu @penlume]

here's the writeup of the factorization!

https://t.co/z25MUI9hjI