Deema

This user runs on ambitions and matcha! @uiuc_nlp

[Deema @deema_cs]

🚨 New paper alert🧵
Can we detect and mitigate LLMs’ hallucinations before they happen? 🤔
🚀 Introducing FactCheckmate ♔— a lightweight framework for preemptive hallucination detection and mitigation in LMs using LMs' internal representations ✨
Paper: https://t.co/IFWiXiQs8j

[Deema @deema_cs]

@yacinelearning gotta admit the choice of the cover picture is brilliant

[Deema @deema_cs]

@yataobian Interesting read, it reminds me of this work https://t.co/kOxGLNsb7X

[Siddarth @siddarthv66]

> Be AI PhD student
> Submit paper to conference
> LLM slop reviews
> Rejected
> Concurrent paper with same method accepted
> Resubmit to next conference
> Reviewer points to concurrent paper which was accepted by last conference
> Lack of novelty
> Rejected

[Deema @deema_cs]

@iOmarCS يستهبلون احس

[ajay @ajayyy4y]

are you ready to date a model?

[Deema @deema_cs]

@FaisalDilaijan Rating feature would help alot 😂

[Rohan Paul @rohanpaul_ai]

Can we detect and mitigate hallucinations before they happen?

Internal neural patterns reveal hallucination risks before LLMs generate false outputs.

Basically Neural networks leak early warning signals when they're about to hallucinate

Original Problem 🔍:

LLMs frequently generate false or misleading information (hallucinations). Current solutions only detect hallucinations after they occur, adding significant overhead and missing opportunities to understand why they happen.

-----

Solution in this Paper ⚡:

• FactCheckMATE: A system that detects and prevents hallucinations before they occur

• Uses lightweight binary classifier analyzing model's hidden states from middle transformer layers

The system uses a lightweight binary classifier that takes the LM's hidden states as input and predicts hallucination probability. It averages the hidden states from middle transformer layers and passes them through a ReLU-MLP followed by a sigmoid function.

• When hallucination detected, adjusts hidden states using intervention model

• Works across multiple model families (Llama, Mistral, Gemma)

• Requires minimal computational overhead (3.16 seconds per inference)

-----

Key Insights from this Paper 💡:

• LLMs' internal representations contain predictive signals for hallucinations

• Middle layers of transformers show strongest hallucination detection capability

• Preemptive intervention is more efficient than post-hoc correction

• Hidden states can be steered to produce more factual outputs

-----

Results 📊:

• Over 70% preemptive detection accuracy across QA datasets

• 34.4% improvement in factual output generation after intervention

• Tested on multiple datasets: NQ-open (Wikipedia), MMLU (STEM), MedMCQA (medical)

• Consistent performance across different model sizes (7B to 13B parameters)

• Average inference overhead: 3.16 seconds

[Deema @deema_cs]

@Yazanmuteb ماشاء الله وين المكان👀

[Deema @deema_cs]

@gadahoth 🎤🫳

[Ahmad Beirami @abeirami]

The question that a reviewer should ask themselves is: Does this paper take a gradient step in a promising direction? Is the community better off with this paper published? If the answer is yes, then the recommendation should be to accept.

[Deema @deema_cs]

@XuanmingZhang07 Unfortunately, I am not. Have fun!

[Deema @deema_cs]

@Owainaa It was a cleaning day for me as well. Champaign misses you, tho 🐿️

[Deema @deema_cs]

@faisalbmahfoodh The test set is 1.8k datapoints in size, the repo will be released.

[Deema @deema_cs]

🚨 New paper alert🧵
Can we detect and mitigate LLMs’ hallucinations before they happen? 🤔
🚀 Introducing FactCheckmate ♔— a lightweight framework for preemptive hallucination detection and mitigation in LMs using LMs' internal representations ✨
Paper: https://t.co/IFWiXiQs8j

[Neeraja Kirtane @NeerajaKirtane]

Excited to share our new work, FactCheckmate ♔: a lightweight framework for preemptive hallucination detection and mitigation in LMs using their internal representations. Huge thanks to collaborators @deema_cs @MKhalifaaaa and advisor @haopeng_nlp
Paper: https://t.co/kLvlaW5xwf

[Muhammad Khalifa @MKhalifaaaa]

Inference-time techniques are just beginning to reveal their full potential. Great work by @deema_cs and @NeerajaKirtane!

[Deema @deema_cs]

Many thanks to the wonderful collaborators @neerajakirtane and @mkhalifaaaa, and advisor @haopeng_nlp
Paper Link: https://t.co/IFWiXiQs8j

[Deema @deema_cs]

With FactCheckmate ♔, we aim to open new avenues for understanding LMs by studying their internal workings and providing tools to create more reliable and truthful outputs.