Julia Rozanova

Working on Payments Foundation Models at @Visa in Cambridge, UK!
~ PhD in Natural Language Processing, Uni of Manchester ~
~ Catch me on the dance floor! ~

[Julia Rozanova @juliaroz]

This month I was formally awarded my PhD, and so I'm officially #PhDone!
Finishing off with an internship at the Amazon Alexa AI research group in Cambridge, and then -

I'm on the job market in the UK for 2024! :D
Shout if you can recommend any cool orgs hiring NLP people!

[Aryaman Arora @aryaman2020]

has anyone ever written a diss track of your paper

[Alejandro H. Artiles @alejadroHArt]

(Current) LLM-based ideation is biased toward what the field already finds easy to think.

We formalize this bias as cognitive availability and use it to identify coherent but overlooked research directions: The Alien Space of Science. 🧵

https://t.co/x23XNY19T3

[Hadas Orgad @ ICML @OrgadHadas]

Excited that our paper on Actionable Interpretability got accepted to ICML!
And just in time -- we also heard that our Actionable Interpretability workshop will be happening again, in COLM!
See you in Korea 🇰🇷 and SF🌉
[Arxiv paper link in the comment]

[Jack Morris @jxmnop]

if you write some really good code these days everyone thinks you did it with Claude Code. if you lose a lot of weight everyone thinks you did it with Ozempic. at least you can still be really good at dancing. they don't make ozempic or Claude Code for being good at dancing

[Julia Rozanova @juliaroz]

Ugh when did this whole "Advisory Capacity" Coauthor from a Famous™��� Institution thing start

[Adam Shai @adamimos]

Accepted to ICML. Excited to discuss with the community in Korea!

[elie @eliebakouch]

Qwen first release on interpretability (qwen scope) is very interesting

they use SAE features to identify what causes repetition in model outputs, then use steering to manufacture a "bad" rollout where the model repeats a lot. this gives RL a clear negative signal to learn from, since repetition barely shows up in normal rollouts so the model never gets punished for it

they also use SAE features as a fingerprint for benchmarks, you look at which features each benchmark activates and compare overlap. lets you find redundancy inside a benchmark and across benchmarks without running any model. for instance 63% of GSM8K features are in MATH but only 10% the other way

[wh @nrehiew_]

You should probably read this.

The best available overview of the systems required to scale large MoEs.

Notes below.

[Neil Renic @NC_Renic]

AI cheating has gotten so bad that I now feel genuine affection for horrifically bad essays clearly written by the student

[Ronak Malde @rronak_]

My takeaways from ICLR 2026

1. Recursive self improvement / continual learning is the next frontier of research. Several great papers in self distillation, auto agent harness optimization, learning from non verifiable reward, self-play are sarly signs of success

2. Multimodal models and world models are attaining emergent reasoning capabilities, opening up a near door to spatial understanding that was previously locked

3. Lots of concerns that the research community is currently too focused on benchmaxxing rather than improving the research process, and a call to action to address this, like Percy Liang’s fully open source training community.

4. Rio is possibly even better than San Diego 🇧🇷🏄

[Youssef El Manssouri @yoemsri]

@yoavgo The core lesson is that architecture sets capacity, but data sets behavior. If students understand that, they understand most of modern LLM progress.

[Aksel @akseljoonas]

For the last 72 hours since ml-intern launched we have had over 500+ autonomous AI research projects running on the Space at all times.

Some insane ones I saw:

1. A new AI paradigm from scratch — trying to replace transformers with a reasoning architecture based on energy minimization, binary sparse address tables and circular convolution binding. No GPU, no gradients, no training data — pure bitwise operations. Years of research done in 2 days. https://t.co/CE2j5HwybI

2. Someone took LoopLM (ByteDance's recurrent depth transformer with shared layers and infinite depth via looping) and crossed it with BitNet b1.58 (ternary 1.58-bit weights). The result: a model that's both infinitely deep AND uses almost no memory per parameter.

3. Designing a new attention mechanism modeled on the thalamo-cortical circuit in the human brain. Pulling from 2025/2026 research out of MIT, Harvard, and UF. The thalamus gates what information reaches the cortex. They're building a learnable gate that mimics this for transformer attention heads, combined with EEG datasets and a reinforcement learning loop. https://t.co/8QgXnQteVH

The use cases people bring are cooler and more impressive than anything we imagined when we built this.

[Julia Rozanova @juliaroz]

This is today at 3:15 - 5:45! Catch me in pavilion 3!

[Julia Rozanova @juliaroz]

I'm at ICLR with a poster on

*DMAP: A Distribution Map for Text*

led by the excellent Tom Kempton
(@UncleKempez), together with @Visa colleagues.

Pop by for a cool story on how our method detected a crucial data error in several major synthetic text detection papers!

[Julia Rozanova @juliaroz]

Made the social faux pas of bringing a vertical poster to a main track poster session. Humbled.

[Julia Rozanova @juliaroz]

@nrehiew_ Did they have a skill or did you stick in a screenshot? 😁

[𝙷𝚒𝚖𝚊 𝙻𝚊𝚔𝚔𝚊𝚛𝚊𝚓𝚞 @hima_lakkaraju]

📣 Excited to announce our oral presentation at #ICLR!

LLMs capture rich semantic structure, as evidenced by their strong performance across a wide range of language and reasoning tasks.

But Sparse Autoencoders (SAEs), a popular interpretability tool, mostly learn local, noisy, token-level features when applied to LLMs (e.g., hundreds of features for the word “the”).

So why aren’t SAEs finding that rich semantic structure?

👉 Because they ignore the sequential nature of language.

We introduce Temporal SAEs to bridge this gap.

https://t.co/HLvuAV7Qek

🧵 [1/N]

[GLADIA Research Lab @GladiaLab]

But what can we do with injectivity? Well, for one, we can invert language models!

We introduce SipIt, an algorithm that exactly reconstructs the input from hidden states in guaranteed linear time.

SipIt recovers inputs >100× faster than alternatives, while remaining exact.

(4/6)

[Julia Rozanova @juliaroz]

Paper:

https://t.co/IXiauLe6kb

[Julia Rozanova @juliaroz]

I'm at ICLR with a poster on

*DMAP: A Distribution Map for Text*

led by the excellent Tom Kempton
(@UncleKempez), together with @Visa colleagues.

Pop by for a cool story on how our method detected a crucial data error in several major synthetic text detection papers!

[Julia Rozanova @juliaroz]

@_joestacey_ @SheffieldNLP Woooh congrats on the Sheffield postdoc!

My PhD brother Marco Valentino is there too! 😁