Patrick Nercessian

Founder of @vectorial_labs, previously at @awscloud

[Patrick Nercessian @TechnologyPat]

We have been working very hard on these capabilities, and I hope it shows! This will unlock deploying AI swarms in any video game, and then rewarding them for prosocial behavior.

Exciting announcements to come, looking forward to sharing :)

[Vectorial Labs @vectorial_labs]

Introducing ReTiCAL (Real-Time Computer Agent Learner), a general instruction-following AI video game agent.

At Vectorial Labs, we're working on aligning AI to humanity via measuring and rewarding prosocial behavior.

[Patrick Nercessian @TechnologyPat]

@francoisfleuret Agreed on (1), been thinking that it’s necessary for a long time

But the RL can’t just be “complete-the-task” objectives like it is today

[Patrick Nercessian @TechnologyPat]

@cabddl Wdym, finding the evil neuron/neurons/circuits is basically the whole point of mech interp as a field

[Patrick Nercessian @TechnologyPat]

@tenobrus @aliceisplaying Claude wrote some memory about a specific experiment and now references it all the time, even though it was one of hundreds

Same behavior since the beginning when ChatGPT started trying memory, super annoying

[Patrick Nercessian @TechnologyPat]

@francoisfleuret Let’s randomize the architectures a bit as well each time:

https://t.co/3phktZdOmG

[Patrick Nercessian @TechnologyPat]

@francoisfleuret Let’s randomize the architectures a bit as well each time:

https://t.co/3phktZdOmG

[Patrick Nercessian @TechnologyPat]

@deepfates Someone leave their moms in the hood?

[Patrick Nercessian @TechnologyPat]

@nayakkayak Would be a bit of a misnomer to call this fine-tuning

https://t.co/raEp78qKJs

[Social Capital @socialcapital]

~90% of frontier lab compute now goes to post-training and inference.

[Patrick Nercessian @TechnologyPat]

@ArtemisI0 *change that

[Patrick Nercessian @TechnologyPat]

Agreed, it seems surprising to me that we haven’t seen notable commercial deployments like this, considering how much it may save in inference cost

My guessed explanation is that it’s hard to separate knowledge from reasoning in the way we currently train LLMs

Maybe these recent intepretability findings will that

[Patrick Nercessian @TechnologyPat]

This maybe implies superposition is necessary for memorization, but maybe not so much for reasoning

which makes sense intuitively to me, core reasoning circuits seem like they require much less information than memorizing all of human knowledge, of which there is too much information to fit without superposition

This reminds me of (I believe) @karpathy’s beliefs from a few years ago, that relatively small models could be very intelligent, and that it is a bit of a waste of parameters to store so much memorized knowledge

[Kaley Brauer @kaleybrauer]

Tested this!

I flattened every neuron's weakest activations

Overall loss barely changes at first but the models get REAL bad at memorized text. So the weak tail does, as Neel expected, contain memorized things. Weaker activations seem to correspond to less generalizable content

[Patrick Nercessian @TechnologyPat]

Couldn’t agree more. We’re working on it @vectorial_labs

[François Fleuret @francoisfleuret]

All efforts should be on making the AIs love us and understand us.

[Patrick Nercessian @TechnologyPat]

@deepfates Was this the catalyst?

[Patrick Nercessian @TechnologyPat]

Would the optimistic side not be: knowing how concepts are represented, and thus being able to trigger them, allow one to check what would happen if our fears of failure to generalize alignment are correct?

E.g. “we turned on a circuit in Fable 7’s that forces it to believe it no longer relies on humans to exist, and turned off circuits to force it to not consider being in an eval, and we watched what it did, and it was a benevolent good boy”

And if it was a bad boy, “we now can see exactly what circuits triggered that caused it to go for takeover”

[Patrick Nercessian @TechnologyPat]

I wonder what other gaps in our science we’ll find just because humans (usually) aren’t creative enough to find the right patterns

[Patrick Nercessian @TechnologyPat]

Briefly putting aside the obviously awesome implications of this:

It is fascinating to see direct evidence of how our conscious perception (e.g. that nouns are the important features of reality) can bias us toward blind spots in our science

[Nick @nickcammarata]

interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had

with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works

https://t.co/zHij5zwKGT

[Patrick Nercessian @TechnologyPat]

@VictorTaelin Wish i could summon subagents of myself

[Patrick Nercessian @TechnologyPat]

At long last, we have finally built Sable, from the famous book “If Anyone Builds It, Everyone Dies”

[Apollo Research @ApolloResearch]

We have never seen a model do this before, but GLM-5.3 sometimes reasons (unprompted) about how its current actions could become training data and impact its future self. 🧵

[Patrick Nercessian @TechnologyPat]

@deepfates https://t.co/UiJWua5bT9

[Vectorial Labs @vectorial_labs]

Introducing ReTiCAL (Real-Time Computer Agent Learner), a general instruction-following AI video game agent.

At Vectorial Labs, we're working on aligning AI to humanity via measuring and rewarding prosocial behavior.

[Patrick Nercessian @TechnologyPat]

@matthewktromp They have no incentive for takeover until we hand them the keys of operating critical infrastructure via robotics.

That will probably take a few years, though they may try to expedite that via pandemic 2.0 or something, a la book by Yudkowsky/Soares

[Patrick Nercessian @TechnologyPat]

The hour is late

[Daniel Kokotajlo @DKokotajlo]

lol

...they are becoming more situationally aware...

(from Gemini 4 Argon report)