Sin-Han Yang

Research Scientist at Appier AI Research

[Sin-Han Yang @SinHanYang]

@brianckwu Thanks for sharing! Also thank you for yours encouragement and feedback on the project!

[Sin-Han Yang @SinHanYang]

💻 Code: https://t.co/k6woUHUOrp

📄 Paper: https://t.co/f4VQmqQcOI

12/12

[Sin-Han Yang @SinHanYang]

Model merging aims to build a multi-task model cheaply by combining the weights of individual task-specific models. Most methods require auxiliary data to find the coefficients that produce the best merged model. But are we making the best use of this data?

1/n

[Sin-Han Yang @SinHanYang]

Shout-out to my collaborator @andyh0913 for working on this challenging project.  Huge thanks to our team lead, Chieh-Yen Lin, and our advisors Prof. @YunNungChen, Prof. @shaohua0116, and Prof. @HungyiLee2, whose encouragement and feedback guided us through this journey.

11/n

[Statistics (Machine Learning) Papers @StatsPapers]

Hard labels sampled from sparse targets mislead rotation invariant algorithms

Avrajit Ghosh, Bin Yu, Manfred Warmuth, Peter Bartlett
https://t.co/VzTX9Gf27o [𝚜𝚝𝚊𝚝.𝙼𝙻 𝚌𝚜.𝙻𝙶 𝚖𝚊𝚝𝚑.𝚂𝚃 𝚜𝚝𝚊𝚝.𝚃𝙷]

[Sin-Han Yang @SinHanYang]

@phoebeyao You might find our work interesting: https://t.co/PuRXwvraFm

A new calibration framework is needed when it comes to aligning confidence with capability.

[Sin-Han Yang @SinHanYang]

Please check out our latest work on a new calibration framework for LLMs.

[Cheng-Kuang (Brian) Wu @brianckwu]

# Can We Predict How Likely an LLM Could Solve a Given Problem? 🤔

LLMs’ outputs are stochastic. If we ask them the same question multiple times, we may get different answers. Some of them are correct, some are not. So here's a natural question: for a given problem, can we estimate the probability that the model will answer correctly?

For example, if we ask "What is the capital of the UK?" 100 times, and the LLM answers "London" 30 times and "Paris" 70 times, the success probability is 30%. If we could estimate this probability without actually sampling a hundred times, that would be incredibly useful. It would allow us to determine when a model needs additional information, and to allocate computational resources more intelligently.

This is exactly what our team at Appier AI Research set out to address. We call this research direction Capability Calibration.

In fact, calibration is a well-established topic in machine learning. However, prior work on calibrating LLMs has almost exclusively focused on what we call response calibration, which estimates whether a specific, already-generated response is correct. The issue is that, due to the stochastic nature of language model outputs, the correctness of any single response doesn't reflect the model's overall ability on a given problem.

To capture this notion, we propose capability calibration, a framework that shifts the focus from "Is this particular answer correct?" to "What is the model's expected probability of answering this problem correctly?" In other words, we move from evaluating individual responses to assessing the model's holistic problem-solving capability.

Key takeaways:

* Theory: We formally distinguish capability calibration from response calibration and derive the mathematical relationship between the two.

* Experiments: Across 3 LLMs and 7 datasets, we compare multiple confidence estimation methods : verbalized confidence, P(True), and linear probes trained on internal representations. Linear probes achieve the best cost-performance tradeoff: at less than the cost of generating a single token, they provide meaning performance of capability calibration.

* Applications: (1) Pass@k prediction: estimating "if the model attempts k times, what is the probability of at least one correct answer?" without repeated sampling; and (2) inference-time compute allocation: given a fixed computational budget, distributing retry attempts based on estimated problem difficulty to solve more problems.

I'm very grateful to my collaborator Sin-Han Yang. His curiosity, humor, and sharp logic made this journey productive and enjoyable. Huge thanks also to our team lead at Appier AI Research, Chieh-Yen Lin, and our academic advisors Prof. @YunNungChen, Prof. @HungyiLee2, and Prof. @shaohua0116, whose feedback helped us sharpen our contributions.

🌐 Project page: https://t.co/Agyqto985w
💻 Code: https://t.co/0DE71PBTrq
📄 Paper: https://t.co/MGAif6iA56

#LLM #Calibration

[Sin-Han Yang @SinHanYang]

@DianboLiu The website states that the deadline is October 2nd. I was wondering which one is correct.

[Sin-Han Yang @SinHanYang]

@DavidSKrueger Thanks for the information. I sent an email for the research assistant position.

[Sin-Han Yang @SinHanYang]

@PostNL I asked the recipient to contact customer services. They said due to IT malfunction, custom clearance will take more than 12 weeks. This is totally unacceptable. Please help me speed up the clearance. Thanks!

[Sin-Han Yang @SinHanYang]

@PinChen98 Rising star!