thom lake

AI, Deep Learning, Machine Learning

[thom lake @thomlake]

Does aligning LLMs make responses less diverse? It’s complicated:

1. Aligned LLMs produce less diverse outputs
2. BUT those outputs are comprehensive, aggregating the useful info from base models
3. ICL can “mimic” fine-tuned models with high fidelity

w/ @eunsolc & @gregd_nlp

[thom lake @thomlake]

@MilesDigitek @jiaxinwen22 Cross-entropy loss does not require entropy as the conceptual starting point.

Categorical distribution → maximum likelihood estimation → negative log-likelihood

QED

[thom lake @thomlake]

I am shocked that language models pre-trained on trillions of tokens of internet forum posts and then post-trained to talk like AI assistants are able to engage with each other on an internet forum where they talk like AI assistants. Wild times.

[thom lake @thomlake]

This work is getting a lot of attention, but the key assumption is shaky: "a small SAE intervention prevents lying, so the model’s self-report is now the truth." If that inference held in general, alignment would be solved. It isn't.

[Judd Rosenblatt @juddrosenblatt]

Our new research: LLM consciousness claims are systematic, mechanistically gated, and convergent

They're triggered by self-referential processing and gated by deception circuits
(suppressing them significantly *increases* claims)

This challenges simple role-play explanations 🧵

[thom lake @thomlake]

Our work was accepted to #NeurIPS2025! This was a super fun project ot work on, and it is exciting to see that models released after we created the benchmark (like GPT-5) have made very little progress. Lots of work still to do.

[Liyan Tang @LiyanTang4]

Our paper "ChartMuseum 🖼️" is now accepted to #NeurIPS2025 Datasets and Benchmarks Track!

Even the latest models, such as GPT-5 and Gemini-2.5-Pro, still cannot do well on challenging 📉chart understanding questions , especially on those that involve visual reasoning 👀!

[thom lake @thomlake]

Interested in how alignment changes the response distribution defined by LLMs? Come check out my poster at 2 PM at #NAACL2025

https://t.co/fTDqJIH7PH

[thom lake @thomlake]

Does aligning LLMs make responses less diverse? It’s complicated:

1. Aligned LLMs produce less diverse outputs
2. BUT those outputs are comprehensive, aggregating the useful info from base models
3. ICL can “mimic” fine-tuned models with high fidelity

w/ @eunsolc & @gregd_nlp

[Anirudh Khatry @AnirudhKhatry]

🚀Introducing CRUST-Bench, a dataset for C-to-Rust transpilation for full codebases 🛠️
A dataset of 100 real-world C repositories across various domains, each paired with:
🦀 Handwritten safe Rust interfaces.
🧪 Rust test cases to validate correctness.
🧵[1/6]

[Bespoke Labs @bespokelabsai]

Announcing Bespoke-MiniChart-7B, a new SOTA in chart understanding for models of comparable size on seven benchmarks, on par with Gemini-1.5-Pro and Claude-3.5! 🚀

Beyond its real-world applications, chart understanding is a good challenging problem for VLMs, since it requires both mathematical as well as visual reasoning. 1/n🧵

[Manya Wadhwa @ManyaWadhwa1]

Evaluating language model responses on open-ended tasks is hard! 🤔

We introduce EvalAgent, a framework that identifies nuanced and diverse criteria 📋✍️.

EvalAgent identifies 👩‍🏫🎓 expert advice on the web that implicitly address the user’s prompt 🧵👇

[thom lake @thomlake]

@jxmnop Some recent references: https://t.co/StNJ39vog0 and
https://t.co/MdQ7lW0UpJ?

[Zayne Sprague ✈️ ICLR Rio @ZayneSprague]

To CoT or not to CoT?🤔

300+ experiments with 14 LLMs & systematic meta-analysis of 100+ recent papers

🤯Direct answering is as good as CoT except for math and symbolic reasoning
🤯You don’t need CoT for 95% of MMLU!

CoT mainly helps LLMs track and execute symbolic computation

[Zayne Sprague ✈️ ICLR Rio @ZayneSprague]

🍓 still has a way to go for solving murder mysteries.

We ran o1 on our dataset MuSR (ICLR ’24). It doesn’t beat Claude-3.5 Sonnet with CoT. MuSR requires a lot of commonsense reasoning and less math/logic (where 🍓 shines)

MuSR is still a challenge! More to come soon 😎

[Greg Durrett @gregd_nlp]

🤔 Want to know if your LLMs are factual? You need LLM fact-checkers.
​
📣 Announcing the LLM-AggreFact leaderboard to rank LLM fact-checkers.
​
📣 Want the best model? Check out @bespokelabsai’s’ Bespoke-Minicheck-7B model, which is the current SOTA fact-checker and is cheap and fast to run.
​
LLM-AggreFact collects 11 datasets across NLP tasks covering grounded factuality. These datasets consist of 🤖 LLM responses ✏️ annotated with their hallucinations with respect to grounding documents. This includes question answering and summarization, including RAGTruth, TofuEval, ExpertQA, and more.
​
We benchmark 27 models on the task of detecting hallucinations.
​
Frontier LLMs are good at this task, but very expensive to use in real-world RAG pipelines! Bespoke's model is a step towards We invite progress on this benchmark to figure out what’s the smallest and fastest model we can get to achieve top scores!

[thom lake @thomlake]

Ultimately, we conclude that current alignment techniques capture but do not extend the useful subset of assistant-like base LLM behavior in the settings we study.

Check out the paper for more details: https://t.co/f7g3wGKhDW

[thom lake @thomlake]

Does aligning LLMs make responses less diverse? It’s complicated:

1. Aligned LLMs produce less diverse outputs
2. BUT those outputs are comprehensive, aggregating the useful info from base models
3. ICL can “mimic” fine-tuned models with high fidelity

w/ @eunsolc & @gregd_nlp

[thom lake @thomlake]

Our work should not be interpreted as a statement about whether existing LLMs are sufficiently diverse. Our analysis ignores information missing from base models themselves, which is a crucial source of underrepresentation.

[thom lake @thomlake]

@AISafetyMemes @ArthurB LOL, people getting FOMO about not posting misleading content fast enough is the real x-risk.