People seem to be arriving at a similar conclusion from various angles:
- AGI may not emerge as a monolith, but as a distributed "patchwork" system of coordinating sub-AGI agents [1]
- Static benchmarks aren't enough; we need multi-agent ones to capture emergent risks and capabilities [2]
- As creation costs go to zero, human verification bandwidth becomes the ultimate economic bottleneck, making verification infrastructure one of the most important public goods for the AI era [3]
- Automated proof-generation and verification can act as the unlock for this bottleneck [4]
- New kinds of strategic interactions between agents are emerging, reaching cooperative "program equilibria" inaccessible in traditional settings [5]
- Coasean transaction costs are about to collapse, changing our society [6]
There is an elephant here that we're all touching. Our @ARIA_research initial Β£50m r&d programme Scaling Trust is our unifying thesis, on the trust infrastructure needed for an agentic world and how to steer us there.
Before we set out on our journey over the next ~3ish years, we're hiring an additional individual to complete our team. Your role will essentially be one of Technical Director, steering our efforts technically and co-owning our research and engineering agenda. You will be doing incredibly meaningful work, in a highly interdisciplinary environment, and at the cutting edge of a technology that is shaping up to be the most defining of our century, if not of humanity.
We are building for the highest possible impact. After all, this is what @ARIA_research is about, moonshot r&d projects that change the world. We want to build technology as impactful as the invention of the internet once was in another r&d programme at DARPA, to start new academic fields and academic lineages for the next century, and to catalyze lasting positive change for the world.
For the right person, this is a bat signal π¦, few places will offer you as much leverage to effect positive change on the world, intellectual stimulation, and fun.
Join us! We want to onboard someone asap as we build out our initial portfolio, and are willing to move fast. Apply here: https://t.co/kjTgSecUq2
Any questions on the role, please shoot me a DM or reply in comment here!
---
[1] Distributional AGI Safety @weballergy@sebkrier@FranklinMatija et. al -- https://t.co/apRl6WZUcm
[2] Agents of Chaos @NatalieShapira et. al β https://t.co/T4xIkahcm2
[3] Some Simple Economics of AGI @ccatalini et. al β https://t.co/Oa1dTKWpu7
[4] When AI Writes the World's Software, Who Verifies It? @Leonard41111588 β https://t.co/dgMyby3tFl
[5] Evaluating LLMs in Open-Source Games @SwadeshSistla et. al β https://t.co/bTWFw4FMpt
[6] Coasean Bargaining at Scale @sebkrier β https://t.co/oNOsUUvvHh
hi! we're running a Scaling Trust meetup in Palo Alto during Real World AI Security. join us to hang out, and chat secure interactions btwn agents, open-ended long-horizon multiagent multiowner cyberphysical evals, and to learn more about @ARIA_research!
https://t.co/latA2nAg1F
Compensation in Index is calculated by estimating each source's Shapley value: its marginal contribution to an agent's answer at the moment of inference.
Content that's uniquely valuable, hard to replace, or used in high-value agent work earns more.
relatable! similar happened to me today where it started being passive aggressive towards me for not executing its proposed changes
made me switch to gpt5.5 to complete the task
Two years ago, we funded Fractile as part of our Scaling Compute programme.
Today, Fractile have announced a $220M funding round to build the next generation of inference hardware.
Huge congratulations to Walter and the team. We can't wait to see what you do next π
If anyone builds it, everyone thrives. Over the past decade, a lot of important work on AI alignment has focused on avoiding harm. But freedom from harm isn't the same as freedom to flourish.
In this paper, we introduce 'Positive Alignment'. A positively aligned agent is one that helps us navigate our own value trade-offs, builds our resilience, and acts as a scaffold for human flourishing. Doing this without slipping into top-down, technocratic paternalism is the great design challenge of our time.
We think a lot more research is now needed to explore this frontier: how do we align models that actively help us thrive?
Amazing work by @RubenLaukkonen, @drmichaellevin, @weballergy, @verena_rieser, @AdamCElwood, @996roma, @FranklinMatija, @shamilch, @_fernando_rosas, @scychan_brains, @matybohacek, @sudoraohacker, and others.
https://t.co/YNL0cZqYD9
New blog post! Could Programming Languages be the solution to Trust in Multi-agent Economies?
We combine Choreographies + Game Theory + Crypto to build a language for AI Ecosystems!
What happens when AI agents start making commitments with other agents on our behalf?
Not just answering questions: negotiating, buying resources, and deciding whether to trust each other.
(blog-post / talk below)
I am BEYOND excited to announce that after 1 year off, the best show in science & deep tech is BACK.
S3 returns this Saturday!
Bigger and better than before.
Weβre hiring a CEO at the Cooperative AI Foundation.
A rare chance to shape the future of AI alongside @AllanDafoe , @ghadfield, Jesse Clifton, @audreyt and me.
If you think deeply about how powerful AI systems should cooperateβand how to get thereβthis role is for you.
Apply: https://t.co/aEBX6QZoNE
In Dec, ARIAβs Trust Everything Everywhere team backed 14 discovery projects (<Β£20k, 3 months) to seed our ~Β£50m Scaling Trust programme π
The Demo Day talks from our March meetup are now live
π https://t.co/QofxTNhUK5
π https://t.co/9c1Gqg7O4Q
@ObadiaAlex@iamnotnicola