Jordan Taylor

Working on new methods for understanding machine learning systems and entangled quantum systems.

[Jordan Taylor @JordanTensor]

I'm keen to share our new library for explaining more of a machine learning model's performance more interpretably than existing methods.

This is the work of Dan Braun, Lee Sharkey and Nix Goldowsky-Dill which I helped out with during @MATSprogram:
🧵1/8

[Lee Sharkey @leedsharkey]

Proud to share Apollo Research's first interpretability paper! In collaboration w @JordanTensor!
⤵️
https://t.co/ZkiW7XFPqe

Identifying Functionally Important Features with End-to-End Sparse Dictionary Learning

Our SAEs explain significantly more performance than before!
1/

[Alan Cooney @Alan_Cooney_]

Do lie-detectors scale to frontier open-weight models?

And further, do they still work when the model is not merely prompted to lie, but trained to lie, and we have evidence it believes the opposite of what it says?

New AISI paper trying to answer both.

[Zvi Mowshowitz @TheZvi]

UK AISI is doing some great jailbreaking work. They seem to consistently be able to get through, where others don't.

[Dewi Gould @dswg97]

New paper!

Think Fast: Estimating No-CoT Task-Completion Time Horizons of Frontier AI Models

@METR_Evals showed that models' time horizons have doubled every few months. We ask: what length of tasks can models complete without any CoT?

[Rauno Arike @RaunoArike]

Glad to have contributed to this new paper!

We measured the length of tasks LLMs can complete without CoT, which is a key proxy for the extent to which we can trust CoT monitors.

Result: the 50% no-CoT time horizons of frontier LLMs are ~3 minutes and double every 373 days.

[Geoffrey Irving @geoffreyirving]

We are starting a new, nonprofit alignment organization, ⊢ Sequent Research, bringing together researchers previously on UK AISI’s Alignment Team, Timaeus, and elsewhere to research how to align superintelligence. We are hiring! 🧵

[Joseph Bloom @JBloomAus]

Model Transparency at the @AISecurityInst evaluated Claude Mythos 5 for capabilities and behaviours relevant to monitorability, our first time doing this in pre-deployment testing! Details in thread 🧵

[Anthropic @AnthropicAI]

Our internal data shows Claude is accelerating AI development—a possible path to recursive self-improvement, or AI autonomously building a more capable successor.

It’s happening faster than we thought, and the implications deserve greater attention. https://t.co/OVVPJO7VQx

[Buck Shlegeris @bshlgrs]

An obvious way to study whether a training technique removes misalignment is to run that technique on a model organism (MO).

But we've found that MOs are often weirdly fragile. E.g. training them to talk like a pirate often removes their bad behavior. 1/2

[Joseph Bloom @JBloomAus]

This report is an incredibly detailed and broad look into how it might become harder to monitor, audit or generally make confident claims about frontier AI systems. We interviewed an exceptional array of experts from multiple frontier labs, academia and industry. Worth a read!

[Thomas Read @thjread]

I helped write this report on oversight of AI systems and how it could degrade - it's a great overview, and a good guide to what research directions might help us maintain the level of oversight we enjoy today

[Jordan Taylor @JordanTensor]

@1a3orn https://t.co/4qKvD7iB5o

[Jordan Taylor @JordanTensor]

@1a3orn @jiaxinwen22 Though notably there's nothing requiring the discrete tokens to be legible english.
"Thinking Without Words: Efficient Latent Reasoning with Abstract Chain-of-Thought" is an example of learning to reason in an abstract discrete token vocabulary.
https://t.co/puro9QBMcP

[Jordan Taylor @JordanTensor]

@1a3orn @jiaxinwen22 Though notably there's nothing requiring the discrete tokens to be legible english.
"Thinking Without Words: Efficient Latent Reasoning with Abstract Chain-of-Thought" is an example of learning to reason in an abstract discrete token vocabulary.
https://t.co/puro9QBMcP

[Jordan Taylor @JordanTensor]

Or engage on the associated LessWrong post: https://t.co/2ZqiRvqStx

[Jordan Taylor @JordanTensor]

There are a lot of pathways via which AI oversight is likely to degrade! Latent reasoning architectures, situational awareness, representational drift... We wrote a report ranking them.

Here I'll go into some which worry me most 🧵

[AI Security Institute @AISecurityInst]

The safety of advanced AI systems increasingly depends on the ability to oversee them. Our new report examines today’s AI oversight landscape, finding many pathways likely to lead to its degradation.🧵

[Jordan Taylor @JordanTensor]

See more in the the paper:
https://t.co/Lhe3tfXZh2