Jack Orlik

Helping governments anticipate & shape change @OPSIgov | ex. @nesta_uk anthropology advocate tweeting about #innovation | Views my own & not those of @OECD

[Jack Orlik @jackorlik]

Not good news.

[Nature Human Behaviour @NatureHumBehav]

.@Moshe_Glickman and @affectivebrain reveal a human-AI feedback loop, where AI amplifies subtle human biases, which are then further internalized by humans. This cycle increases human bias over time across domains.
https://t.co/hyGntC4sNt

[Jack Orlik @jackorlik]

@GandonAmy Interesting points @GandonAmy - at @OPSIgov we've been exploring how strategic foresight/anticipatory governance can help address 1&3, learning from Finland and 14 other countries. Would be happy to chat!

[Séb Krier @sebkrier]

😈 A bit unsure about this but: one early rationale for dangerous capability evals was to demonstrate that models could in principle do risky things, at a time where the vibe du jour was 'models can't do X/Y/Z'. Gradually the fact that capabilities are continuously improving is becoming more accepted, and "I bet this stops here" is a bet fewer people make. So what does this mean for DC testing?

It doesn't go away: it's still a fundamentally necessary R&D function to better understand models and try to establish some initial idea of 'how badly could this be misused if one tried'. This is good and I'm glad this norm is being set. However I think there will be more work now to try and work out how exactly these capabilities might fit the 'supply chain' of harmful misuse/accident scenarios - i.e. a growing focus on the deployment side. For example if you're concerned about persuasion, you might then focus on "what if an AI therapist was system prompted to maintain engagement as long as possible?". This is because focusing on an in vitro capability eval result alone is unlikely in most cases to be persuasive enough warrant any action.

Not many people will like this, because most people are attracted to 'the model' like moths to fire. Governance folks in particular love finding general principles that apply across the board, 'regulating AI', and staying at this level of abstraction; few want to actually develop domain expertise in finance or healthcare or cybersecurity. Some people will tell you about the 2010 flash crash but can't tell you anything else about how AI is used in finance. This is problematic because you end up having many generalists theorise domain risks from first principles.

If the concern is alignment, then this is less clear - depending on how you understand the meaning of 'alignment' then you might focus on different parts of the value chain. If for a while you've thought 'models will be naturally power seeking' then you'll be inclined to focus on the model, which is where you'll want to find evidence of scheming. This seems good - different post-training approaches might cause unexpected model proclivities. If you think about alignment from a wider societal lens, you might focus more on the values used to align models, personalisation, and multi-agent dynamics.

Over time I would expect the former to look more and more like machine psychology (https://t.co/mrMvLRFb66) and the latter like multi-agent simulations (https://t.co/PUCEfeYog3). Still very early days of designing minds, and co-existing with them!

[Jack Orlik @jackorlik]

@mattsclancy Great idea to try this yourself. Did you collect any info on (self-professed) level of expertise? I'd love to see how literature grads fare!

[Jack Orlik @jackorlik]

Thought provoking piece from @stevenbjohnson on the potential of long-context AI models to improve institutional memory and collective intelligence. https://t.co/NDKAdZau3G

[Jack Orlik @jackorlik]

I've been getting Claude and ChatGPT to roast themselves to illustrate the features & limitations of LLMs. Just as I got to illustrating guardrails, I asked them to roast each other, and the results were *chef's kiss*.

[Frank Nagle @frank_nagle]

New paper alert: We (w/ @ichbinsnicht, @samjboysel, Sida Peng, Kevin Xu) just released the working paper version of our study “Generative AI and the Nature of Work” - https://t.co/wgFFbD8k1e. We seek to build upon research on #AI and productivity to better understand how #GenAI

[Paul Nightingale @Nightingale_P]

2. What is says is that experience leads people like me who do evaluations to expect that the net impact of interventions on what we hope to change is going to be zero.

It happens so often that we just assume we won't find an impact.

Its as brutal as that.

[Collective Intelligence @Collective_CI]

A UK parliamentary committee is interested in the best examples worldwide of parliaments using collective intelligence methods to crowdsource facts, ideas, evidence, experiences and questions. All examples welcome!

[Haydn Belfield @HaydnBelfield]

What if we took AI companies plans, investments & ambitions seriously? What should the UK Government do to prepare for "intelligence too cheap to meter"?

THREAD🧵

[Jack Orlik @jackorlik]

"To get organizational gains [from AI] you are largely going to have to do the R&D yourself" Excellent guidance from @emollick to discover and scale valuable use cases for GenerativeAI. https://t.co/XcqpB64n1G

[Jack Orlik @jackorlik]

5/5 This @EU_Commission funded project, implemented by @OPSIgov, has transformed how these governments anticipate change through pilots, workshops, training and peer learning from countries like Finland and Wales. Here's to more proactive governance across Europe & beyond!

[Jack Orlik @jackorlik]

1/5 Inspiring to hear outcomes from Project LIMinal presented by country partners at @OECD Gov Foresight Community! 🌟 Malta, Lithuania & Italy have made significant strides in strategic foresight and anticipatory governance. Let's dive into the concrete impacts:

[Jack Orlik @jackorlik]

4/5 🇮🇹 Italy:

- Developed new foresight training initiatives
- Identified support needs for Ministries and Public Administration
- Built network of stakeholders advancing strategic foresight

[Brian Merchant @bcmerchant]

Amid all the chaos at OpenAI I hope this paper in Nature gets some attention. It finds that larger language models become *less* reliable for a variety of reasons—for one, where earlier models avoided questions they couldn't answer, newer ones are more likely to make something up

[Jack Orlik @jackorlik]

Link: https://t.co/LRagW8RSt4

[Jack Orlik @jackorlik]

'What do you mean, "It's complex"?'

New diagram from UK MoD 'Global Strategic Trends: Out to 2055' shows interconnections of future trends. #futurespaghetti

[Ethan Mollick @emollick]

This paper offers insight into the implications of the “jagged frontier” of AI abilities that we found in our studies of AI use in companies

As models get better, they don’t improve on all skills equally, but bigger models are more opaque to users about what they do well & badly