Suhong Moon

PhD student at Berkeley AI Research (@berkeley_ai)

[Suhong Moon @SuhongMoon]

Rich backstories enable deeper persona binding. LLMs conditioned with the backstories go beyond predicting human opinions to simulating how people perceive others and are perceived in return.

[Minwoo (Josh) Kang @joshminwookang]

🤔 Do LLMs exhibit in-group↔out-group perceptions like us?
❓ Can they serve as faithful virtual subjects of human political partisans?

Excited to share our paper on taking LLM virtual personas to the *next level* of depth!
🔗 https://t.co/LzeDAMtrEV 🧵

[Joseph Jeesung Suh @JosephJSSuh]

LLMs have dominated recent work on simulating human behaviors. But do you really need them?

In discrete‑choice settings, our answer is: not necessarily.
A lightweight graph neural network (GNN) can match or beat strong LLM-based methods.
Paper: https://t.co/WvMRy4DdjR 🧵👇

[Suhong Moon @SuhongMoon]

@redstone_hong Congrats!

[Kayo Yin @kayo_yin]

📅 The submission deadline has been extended to June 27th AoE! Looking forward to your ideas :)

[Suhong Moon @SuhongMoon]

@jmin__cho @unccs @JHUCompSci Congratulations!

[Nick Lee @nicholaszlee]

🚀 Excited to share that our paper on Plan-and-Act has been accepted to ICML 2025. Below is a TLDR:

🔎 Problem:
• LLM agents struggle on complex, multi-step web tasks (or API calls for that matter).
• Why not add planning for complex tasks and decouple planning and execution?
• Planning only helps if it’s accurate, and LLMs aren’t trained for that.
• Even small plan errors can drastically degrade performance.

💡 Thoughts:
• Separate PLANNER and EXECUTOR models: Web agents especially benefit, as acting on HTML needs different skills than step by step planning.
• Finetune each model with synthetic data to train the PLANNER and EXECUTOR models; no manual annotations or simulators needed.
• Plan-and-Act provides a scalable framework to create such synthetic data in a scalable manner for web tasks

📦 How we generate synthetic data:
• Use a Teacher LLM to generate new user queries from seed examples.
• A second Teacher tries to solve each query, generating action trajectories.
• We verify the success of each trajectory automatically.
• Another LLM reverse-engineers a plan from the trajectory.
• Finally, we expand this dataset further with more synthetic plans using LLMs.
• We then use this data to fine-tune the base models

⚡ Results:
🏆 New SOTA for text-only open-source models with up to 40% improvement with our synthetic finetuning approach:
• 57.58% on WebArena-Lite
• 81.36% on WebVoyager
• 48.15% on WebArena

Paper: https://t.co/fdpLV3fzns
Joint work w/ @eren_lutfi78249 @sehoonkim418 @SuhongMoon @frt03_ @GopalaSpeech @KurtKeutzer @amir__gholami

[Ritwik Gupta @Ritwik_G]

Do LLMs understand probability distributions? Can they serve as effective simulators of probability? No!
However, in our latest paper that via in-context learning, LLMs update their broken priors in a manner akin to Bayseian updating.

📝 https://t.co/Iab3kz2y7V

[Coleman Hooper @coleman_hooper1]

How can we efficiently scale up test-time compute with parallel tree search?

🚨 Introducing Efficient Tree Search (ETS): A new method for achieving efficient and accurate test-time search for LLM reasoning tasks!

- Test-time scaling has emerged as a new axis for improving model performance by leveraging additional computation at inference time in order to solve more challenging problems.
- One promising approach for scaling compute at test time is through search, where a model generates multiple potential candidates and we then filter these candidates down to a single final response. This type of search can be performed as a tree (where each level of the tree corresponds to taking an additional step towards solving the problem).
- Previous tree search methods have either yielded low accuracy (due to performing insufficient exploration) or else have substantial efficiency penalties due to the high cost of exploring diverse trajectories. These efficiency penalties are because of increased memory requirements due to reduced shared KV cache state during the search process.

Our method, ETS, addresses this challenge by encouraging KV cache sharing in the search process to reduce memory consumption, while maintaining the exploration of semantically diverse trajectories which is critical for attaining high accuracy.

Paper: https://t.co/Zzio6l53Id
Code: https://t.co/2LhXN5miql

Joint work with:  @sehoonkim418 @SuhongMoon Kerem Dilmen @sudomonish @nicholaszlee Michael Mahoney @Sophia_Shao_ @KurtKeutzer @amir__gholami

🧵 [1/7]

[Simon Guo @simonguozirui]

LLMs for GPU kernel🌽generation have been getting Pop🍿ular since our preview last Dec; excited to announce 📢 our full paper 📃 for KernelBench!

Turns out KernelBench is quite challenging 🧠 — frontier models outperform the PyTorch Eager baseline <20% of the time.

More 🧵👇

[Suhong Moon @SuhongMoon]

Surveys are key for public opinion research but are costly. Can AI help? We fine-tune LLMs on SubPOP, a newly curated large-scale dataset, reducing the human-LLM opinion gap by up to 46%. Please check our paper and code!

[Joseph Jeesung Suh @JosephJSSuh]

Can LLMs assist public opinion survey designs by predicting responses?

We fine-tune LLMs on our new large-scale survey response dataset, SubPOP, which reduces the distributional gap between human-LLM predictions by up to 46% 📊
A 🧵 on our findings: 👇

[Patrick Wu @tsunghan_wu]

🚨 Launching The Visual Haystacks (VHs) Benchmark: the first "visual-centric" Needle-In-A-Haystack (NIAH) benchmark to assess LMMs' capability in long-context visual retrieval and reasoning.

Check out the 🧵 to see our findings on models like #GPT4o, #Gemini, and more! (1/7)

[Alex Pan @aypan_17]

LLMs have behaviors, beliefs, and reasoning hidden in their activations. What if we could decode them into natural language?

We introduce LatentQA: a new way to interact with the inner workings of AI systems. 🧵

[Suhong Moon @SuhongMoon]

@woosuk_k I’m looking forward to seeing you!

[David Chan @_dmchan]

🚨 Call for Papers! 🚨

Join us at the Workshop on Human Alignment in AI Decision-Making Systems (IEEE CAI 2025) to explore challenges &amp; opportunities in aligning AI with human values &amp; societal norms📅 Papers Due 1/15/25

Details ➡️ https://t.co/WQpqDe4Oo0 #AIAlignment #AIethics

[Heesoo Jang @HeesooJang2]

South Korean President Yoon Suk Yeol just declared emergency martial law to "eradicate the shameless pro-North Korean anti-state forces that are plundering the freedom and happiness of our people."

This is a whole other level of authoritarian move we're seeing. #SouthKorea

[Suhong Moon @SuhongMoon]

@coleman_hooper1 This is so cool!

[Coleman Hooper @coleman_hooper1]

🚨 Introducing Squeezed Attention! A new fast attention method for long context inference providing up to 4x speed up 🚄

Many LLM applications require processing long input prompts for tasks such as document analysis and code generation; however, inference cost linearly increases with sequence length, making long context inference prohibitively expensive and slow.

One key trait for many LLM applications is that a large portion of the input prompt is fixed across successive user queries, for example, when a user asks multiple questions against a document, knowledge source, or codebase. This means that we have the opportunity to perform offline optimizations to be able to accelerate attention for user inputs when they are received. Our hierarchical method can reduce the complexity of attention from linear to logarithmic with respect to the fixed context length.

Squeezed Attention accelerates attention from the user input to the fixed context in the prompt by identifying which keys are important for a given query, and then only computing attention with these important keys. This identification is performed by comparing the query with clusters of keys, and then progressively refining this comparison using finer-grained clusters in order to identify the important keys for the query.

Paper: https://t.co/EjA0CRMKCb
Code: https://t.co/3yln96bkO6

Joint work with:  @sehoonkim418 @hiva_moh @sudomonish June Paik from @FuriosaAI  Michael Mahoney @KurtKeutzer @amir__gholami

🧵 [1/7]

[Suhong Moon @SuhongMoon]

Please check out our recent BAIR blog post (@berkeley_ai) for a deeper dive into Anthology and its potential to create realistic virtual personas with LLMs!

BAIR Blog: https://t.co/pyaHTJHsJq

[Suhong Moon @SuhongMoon]

(1/n) 🧵 Can Large Language Models simulate different individuals' beliefs and opinions?

Checkout our paper on conditioning LLMs to virtual personas for approximating individual human samples at #EMNLP2024!

Paper: https://t.co/xT7PD1dxmq
Code: https://t.co/i6WBoBO4IK

[Suhong Moon @SuhongMoon]

(12/n) This work would not have been possible without my amazing collaborators: @marwaabdulhai @joshminwookang @JeesungSuh20716, Widyadewi Soedarmadji, Eran Kohen Behar, @_dmchan, and John Canny