Edward James Young

Co-founder and MTS at Geodesic Research

[Edward James Young @edwardjyoung]

We're excited about both empirical (e.g., model forensics, post-training diffing, interp) and theoretical (e.g., RL dynamics, behavioural selection theory) approaches to building understanding. If you're curious and want to develop a science of misalignment, please apply!

[Edward James Young @edwardjyoung]

@RadmardPuria and I are mentoring for the upcoming cohort at @pivotal_org!

We're building an understanding of how misalignment arises during reinforcement learning. In our stream, you'll build RL environments which induce pathological cognition.

Apply: https://t.co/RdNzw3Aa68

[Pivotal Research @pivotal_org]

Applications are open for Pivotal’s Q1 2027 Research Fellowship 💥

Work with mentors from @AnthropicAI , @GoogleDeepMind, @AISecurityInst, @ApolloResearch, and more.

📍 15 weeks in London (18th Jan - 30th Apr).
💸 £15-18k stipend, travel and housing support

Apply by Nov 1: https://t.co/uVlCCSXiS7 ↓

[Edward James Young @edwardjyoung]

You'll then set about understanding the dynamics of this cognition:
- How does it arise during RL?
- How is it selected for?
- What properties of the pre-RL checkpoint does it depend on?
- What aspects of the environment are responsible for its emergence?

[Xander Davies @alxndrdavies]

Our jailbreaking work is conducted by the Misuse Red Team (Eric Winsor, @giglema, @edwardjyoung), with contributions from @_robertkirk @AlexandraSouly @jaipatelAISI @abbydcruz__

Read the Frontier AI Trends Report! Lots of cool results now public. https://t.co/C0SgcYPayi

[Edward James Young @edwardjyoung]

@robertskmiles (He also calls them “cognitive misers”)

https://t.co/GtlTLTqUP9

[Edward James Young @edwardjyoung]

@robertskmiles Psychologist Keith Stanovich writes a lot about this! In his model, failures of rational thinking can either be “algorithmic mind failures”, which are intelligence based, or “reflective mind failures”, which are the lack of deployment of intelligence in the appropriate scenarios

[Edward James Young @edwardjyoung]

@AdriGarriga @BogdanIonutCir2 @xuanalogue I’m also skeptical that the Coconut exactly will work, but it seems likely that there’s *some* way to do RL effectively for latent space thoughts, and that once we figure out how it will be more compute efficient at inference time.

Are you bullish on neuralese translation?

[Edward James Young @edwardjyoung]

@AdriGarriga @BogdanIonutCir2 @xuanalogue Are you worried about coconut-style reasoning in latent space (https://t.co/bsOnV1SpKs)?

I expect these methods to be more compute efficient that reasoning in plain text, and for labs to migrate to them

[Emmett Shear @eshear]

Not being scared of AGI indicates either pessimism about rate of future progress synthesizing digital intelligence, or severe lack of imagination about the power of intelligence.

[Edward James Young @edwardjyoung]

Course schedule:
1. Introduction to RL
2. Dynamic Programming
3. Deep Q-Networks
4. Deep Deterministic Policy Gradients
5. Actor-Critic Methods
6. Proximal Policy Optimisation
7. Natural Policy Gradient Methods
8. Reward Hacking and Goal Misgeneralisation

[Edward James Young @edwardjyoung]

I've started a new deep reinforcement learning lecture series. New lecture every week. Here's the first one:
https://t.co/eSAia8pP1m