Renato Lui Geh

PhD student at University of California, Los Angeles.

[Daniel Israel @danielmisrael]

What happens if you interpret attention scores as probabilities? We can then use sampling techniques in KV cache eviction. Check out our new paper!

[Daniel Israel @danielmisrael]

Congratulations @itisalex3 on acceptance to #ACL2026 🎉 If you are interested in KV cache compression, I highly recommend reading this paper.

[Daniel Israel @danielmisrael]

When you prompt an LLM for code, you get one deterministic program. However, the LLM actually defines a distribution over many programs, and existing methods discard it‼️ PPoT uses this distribution to extract free performance and efficiency gains. 🧵👇

[Renato Lui Geh @renatogeh]

Check out Alex's work on KV cache compression! Alex is an undergrad here at UCLA and is looking for PhD and industry positions! :)

[Alex Chen @itisalex3]

What happens when we compress the KV cache of prompts with multiple instructions? 🤔

Existing compression methods can lead to some instructions being ignored. 🙀

We propose simple changes to KV cache eviction that fix this problem alongside other pitfalls to be aware of. 💯

[Zilei Shao @zileishao]

What happens if we tokenize cat as [ca, t] rather than [cat]?

LLMs are trained on just one tokenization per word, but they still understand alternative tokenizations. We show that this can be exploited to bypass safety filters without changing the text itself.

#AI #LLMs #Token

[Renato Lui Geh @renatogeh]

Read the full paper for more details! https://t.co/bbUBlotNVj.

This work is a collaboration with @HonghuaZhang2, @KareemYousrii, @benjiewang_cs and @guyvdb.

5/5

[Renato Lui Geh @renatogeh]

We find that there exists significant signal in non-canonical tokenizations.

In fact, by computing a mixture of canonical and non-canonical tokenizations, we are able to achieve a consistent boost in accuracy for Q&A benchmarks!

4/5

[Renato Lui Geh @renatogeh]

Unfortunately, computing the marginal probability of text over all tokenizations is also computationally hard.

Instead, we approximate the marginal and by doing so are able to find evidence of signal in non-canonical tokenizations!

3/5

[Renato Lui Geh @renatogeh]

We show that, for autoregressive models, computing the most likely tokenization is computationally hard.

Despite this, we experimentally find that the canonical tokenization is overwhelmingly more likely compared to other tokenizations.

But this is not always true!

2/5

[Renato Lui Geh @renatogeh]

Where is the signal in LLM tokenization space?

Does it only come from the canonical (default) tokenization?

The answer is no! By looking at other ways to tokenize the same text, we get a consistent boost to LLM performance!

https://t.co/bbUBlotNVj

1/5