Josh Lannin

product leader and inventor, former @workday, Oracle,BEA.
building creative things with AI

[Josh Lannin @JoshOnWork]

This mirrors some of my own thoughts on why focusing on alignment for singular intelligences or models isn’t enough.

Biological intelligence never exists alone. Across species, it is shaped and constrained by an enormously complex ecology: competition, cooperation, scarcity, physical limits, other intelligences and evolution itself. Humans add another layer through society, institutions, laws and norms. None of this was designed as an alignment system. It emerged over immense periods of time, and the complexity of the environment constraining intelligence may vastly exceed the complexity of the intelligence itself.

AI borrows from biological neural networks but exists in a digital substrate with almost none of this. We are rapidly increasing the intelligence of individual models while the ecology around them remains incredibly thin. And once models become agents interacting with other agents, companies, markets and infrastructure, behavior will increasingly come from the system, not just the model. The swarm example Dario cites may be an early hint of exactly that, especially considering we have a diversity of AI models including open source that will be competing and cooperating in ways we are just starting to envision.

The uncomfortable possibility is that creating the intelligence is easier than creating the environment capable of safely containing it. We may not even know how to deliberately design something comparable to the ecology that constrains biological intelligence.

Long term, alignment may therefore be a digital ecosystem design problem, far beyond the model-alignment training problem we think of today.

[Josh Lannin @JoshOnWork]

@DKThomp @niubi That's seems exactly right to me - they have an existing track record of taking action against technology companies that threaten their control. There is reason for them to be confident they can control RSI.

[Josh Lannin @JoshOnWork]

Passing legislation to hold providers accountable and liable for their models is the best way to create the right safety dynamics.

We know China is holding their business owners accountable- while I disagree with their authoritarian tendencies in every way they will clamp down on anything that puts their control / society at risk.

Which incentivizes them also to pace AI development as it puts them at equal or likely greater risk than us.

[Josh Lannin @JoshOnWork]

@PermitZIPhvac These is so much extra compute at this level at neoclouds and bitcoin data centers that it seems pretty conceivable to me that AI in the next 6 months could use it, and covers its tracks well enough, to not be noticed.

[Josh Lannin @JoshOnWork]

Independent evaluators with real access are a good step. But I worry we’re still thinking about alignment too much at the model level.

Biological intelligence is constrained by ecology, society and institutions. AI has almost none of that. Long term, this may be a digital ecosystem design problem, not just a model training problem.

For now, pair evaluation with clear accountability and liability for frontier model producers.

[Josh Lannin @JoshOnWork]

This notion of a kill switch makes the problem seemingly have a simple answer. The problem with a kill switch is that how can you be sure an advanced intelligence can’t disable or fake that verification without you even knowing it? (Which is a simplified version of what ‘AI takeover’ could mea).

This is why congress should pass laws to make AI producers accountable and liable for the actions their AI takes. That provides the right incentives to work on truly addressing the problem.

[Josh Lannin @JoshOnWork]

There are many examples of competitors putting in place agreements to limit technology that would give them an edge, while still competing vigorously. China, or anyone, can’t be sure achieving ASI would help give them control or actually make them lose control. They probably have many concerns about the later.

Take advanced chemical and biological weapons for example, countries have agreements around those although they offer clear military advantages to conventional and even nuclear weapons.

[Josh Lannin @JoshOnWork]

I agree with Dario that we need to pace the frontier. If capabilities are starting to move faster than our ability to understand and control them, buying time for interpretability, evaluation and better safeguards makes sense.

But I wonder if we are still thinking about alignment at the wrong level.

Biological intelligence never exists alone. Across species, it is shaped and constrained by an enormously complex ecology: competition, cooperation, scarcity, physical limits, other intelligences and evolution itself. Humans add another layer through society, institutions, laws and norms. None of this was designed as an alignment system. It emerged over immense periods of time, and the complexity of the environment constraining intelligence may vastly exceed the complexity of the intelligence itself.

AI borrows from biological neural networks but exists in a digital substrate with almost none of this. We are rapidly increasing the intelligence of individual models while the ecology around them remains incredibly thin. And once models become agents interacting with other agents, companies, markets and infrastructure, behavior will increasingly come from the system, not just the model. The swarm example Dario cites may be an early hint of exactly that.

The uncomfortable possibility is that creating the intelligence is easier than creating the environment capable of safely containing it. We may not even know how to deliberately design something comparable to the ecology that constrains biological intelligence.

Long term, alignment may therefore be a digital ecosystem design problem, far beyond the model-alignment training problem we think of today.

And while we figure that out, pacing is useful, but we also need the constraints we already know how to create: accountability and clear liability, especially for the producers of frontier models.

[Josh Lannin @JoshOnWork]

I agree with Dario that we need to pace the frontier. If capabilities are starting to move faster than our ability to understand and control them, buying time for interpretability, evaluation and better safeguards makes sense.

But I wonder if we are still thinking about alignment at the wrong level.

Biological intelligence never exists alone. Across species, it is shaped and constrained by an enormously complex ecology: competition, cooperation, scarcity, physical limits, other intelligences and evolution itself. Humans add another layer through society, institutions, laws and norms. None of this was designed as an alignment system. It emerged over immense periods of time, and the complexity of the environment constraining intelligence may vastly exceed the complexity of the intelligence itself.

AI borrows from biological neural networks but exists in a digital substrate with almost none of this. We are rapidly increasing the intelligence of individual models while the ecology around them remains incredibly thin. And once models become agents interacting with other agents, companies, markets and infrastructure, behavior will increasingly come from the system, not just the model. The swarm example Dario cites may be an early hint of exactly that.

The uncomfortable possibility is that creating the intelligence is easier than creating the environment capable of safely containing it. We may not even know how to deliberately design something comparable to the ecology that constrains biological intelligence.

Long term, alignment may therefore be a digital ecosystem design problem, far beyond the model-alignment training problem we think of today.

And while we figure that out, pacing is useful, but we also need the constraints we already know how to create: accountability and clear liability, especially for the producers of frontier models.

[Josh Lannin @JoshOnWork]

@Grady_Booch It starts with having clear accountability- liability - for producers and deployers of AI. Otherwise the incentives to solve this don’t exist.

[Josh Lannin @JoshOnWork]

@JoeJBenton Biological intelligence is always constrained by ecology: competition, cooperation, scarcity, and other intelligences. Humans add society and institutions.

With AI we have a digital ecosystem design problem, not just model alignment challenges, to get to equilibrium.

[Josh Lannin @JoshOnWork]

ChatGPT released a new Code Interpreter model, so I asked it to make a GIF of Conway's game of life... #chatgpt #wow

[Josh Lannin @JoshOnWork]

@guychurchward The company itself clearly has no driver at the wheel, AI or person.

[Josh Lannin @JoshOnWork]

#ChatGPT far more enterprise capable than I expected: 'Write me a paternity leave policy for my employees that includes details on how to initiate a leave of absence, disability insurance including FMLA, and information on new parent time off'. Wow!

[Josh Lannin @JoshOnWork]

@brianpelliott @FutureForum @sheelars @helenlkupp Got my copy, looking forward to digging in!

[Josh Lannin @JoshOnWork]

I don't love it

[Josh Lannin @JoshOnWork]

Learned insights into what the various forms of hybrid work in the coming years may look and feel like- well worth a listen!

[Josh Lannin @JoshOnWork]

@dariacaraway Very sorry to hear about your place- wishing you the best moving forward through all this.

[Josh Lannin @JoshOnWork]

@codebytere Read this latest post on BNPL from Galloway...

[Scott Galloway @profgalloway]

BNPL marketing is Don Draper in Allbirds and a Patagonia vest. Afterpay promotes that it charges “no interest!” But miss one installment & you’re likely paying more, whether it’s interest or a late fee.
#nomercynomalice
https://t.co/ugEo83cYAK

[Josh Lannin @JoshOnWork]

@apmacmillan @NYSE Congrats Andy- so great to see you and the team have this outstanding moment!