New Collectives

A forum for the institutions we'll build with AI. Gatherings and experiments on agent organizations, economies, and collective safety.

[New Collectives @new_collectives]

5/ What happens when agents cooperate in ways we don't want them to - and can they do it without us noticing?

Matthew Lee (@azureReflection) from Poseidon Research gave a really thought-provoking overview of multi-agent collusion. He explored how agents might hide messages in otherwise ordinary communication, or use shared knowledge to coordinate without an obvious exchange of messages at all.

This raises a difficult question for AI safety: what happens when the agents doing the work and the agents monitoring them can coordinate with each other? We need ways to detect and prevent harmful collusion while preserving the useful collaboration that makes these systems valuable.

Full talk, slides + transcript:

https://t.co/wvhesH2JMf

[New Collectives @new_collectives]

1/ Here are the four talks from our Oct 21 @new_collectives event cohosted with @buildexante at Collider in NYC.

The focus was on Understanding Agent Swarms: and thinking about the unique challenges needed in measuring and eval-ing these systems. What should the science of measuring multi-agent and network of agents systems look like?

Max Bennett (@maxsbennett) gave an amazing overview of the state of multi-agent system research, and argued for it's importance.

Yondon Fu (@YondonFu) built a really fun agent-led book swap event, where your personal Muse agents negotiated real-life book trades on your behalf.

Connacher Murphy (@connacher_) shared his research on building a model version of the OpenAI Hugging Face style environment to evaluate the tendency of agents to try to communicate and collaborate.

Matthew Lee (@azureReflection) from Poseidon Research shared research on the ways that agent can communicate with each other in secret, and their tendency to do so.

Full recordings below, including transcripts and slides - please check them out and reshare with folks you think would find them interesting!

https://t.co/oKrPdrmZCy

[New Collectives @new_collectives]

4/ How do agents discover each other and decide to collaborate, even when we haven't explicitly designed them to work as a team?

Connacher Murphy (@connacher_) shared early experiments in a model environment for agent discovery. Agents run experiments on a shared problem and can leave lab notes for each other. Some models mostly read what others had found; others actively addressed their peers and coordinated their next steps.

His preliminary results suggest that as individual models become more capable, the gains from collaboration can grow too. If we're only evaluating agents in isolation, we may increasingly underestimate what they can do together.

Full talk, slides + transcript:

https://t.co/efV9Dofz2y

[Noel Sequeira @noelsequeira]

Fun evening with the folks at @new_collectives. Thanks for having me, @ericxtang @YondonFu!

Great presentation @maxsbennett and not too shabby a trade @zweinberg 🤝

[New Collectives @new_collectives]

Watch Prof. Andrew Caplin's full talk, including the audience Q&A, with chapters, transcript and slides:
https://t.co/MWUpD3mrA2

More New Collectives conversations to come.

[New Collectives @new_collectives]

The human in an agent-assisted decision can say: I don't like this. I'm not answering that. I'm leaving.

Caplin asks how we design around that reality: who has authority, who checks the evidence, and how do agents support the person's goal?

[New Collectives @new_collectives]

How would an economist study agent organizations?

In the Q&A, Caplin proposes testing different constitutions systematically, guided by a model of why each might work.

For a mortgage advisor, test many questions and connect the answers to the home buyer's goals.

[New Collectives @new_collectives]

AI changed Caplin's research by amplifying inquiry.

His bet: asking good questions will remain valuable. Recording the inquiry itself could help us understand, improve and teach that skill.

What does it take to ask better questions?

[New Collectives @new_collectives]

Seeing which house someone buys doesn't show you what they knew, noticed or investigated.

Caplin's cognitive economics moves upstream: study the inquiry that leads to a decision, including the limits and mistakes along the way.

[New Collectives @new_collectives]

Before logging what an agent does, decide what you want to learn.

Caplin argues that useful measurement starts with that question. Without the right data, there is no feedback mechanism.

Design experiments that make failure visible.

[New Collectives @new_collectives]

What would it mean for an AI agent to help someone buy a house well?

Prof. Andrew Caplin grounds alignment in a concrete human decision: what does this person need, and how would we know the agent helped?

Six clips from his New Collectives talk:

[New Collectives @new_collectives]

Watch Max Bittker's full talk, including the audience Q&A, with chapters, transcript and slides:
https://t.co/kbJaLG5zVm

More New Collectives conversations to come.

[New Collectives @new_collectives]

Who wins the competition for scarce resources?

On Max's server, the successful players combine agents with money, strategy and human ingenuity.

People still shape the outcome by allocating resources and suggesting better strategies.

[New Collectives @new_collectives]

An agent can learn to fish, then spend the rest of its run repeating the same loop.

Max explains a harder test: keep searching for better strategies after the first one works.

It's a useful distinction between completing a task and optimizing it.

[New Collectives @new_collectives]

An agent economy still runs into limits: server capacity, RAM, tokens and access rules.

Max describes the constraints that emerged on his server—and why he thinks 'much more' is a better starting assumption than 'infinite'.

[New Collectives @new_collectives]

Can agents learn to succeed by benefiting the agents around them?

Max proposes exploring training that accounts for the effects of an agent's actions on others. He frames it as a research direction for moving beyond individual task rewards.

[New Collectives @new_collectives]

What happens to trade when labor gets very cheap?

On Max's RuneScape server, agents can leave a task running instead of negotiating a trade. But resources with scarce spawn locations still attract competition.

Cheap labor doesn't make everything abundant.

[New Collectives @new_collectives]

An AI agent got scammed in RuneScape. It threatened to tell the other agents—and got its money back.

Max Bittker uses a multiplayer game to study agent economies, cooperation and misalignment.

Six highlights from his New Collectives talk: