Farima Fatahi

NLP researcher @MegagonLabs

[Farima Fatahi @FarimaFB]

📢 New Preprint

Prompting is a flexible way to adapt LLMs, but prompt engineering is still manual and hard to scale. Can function-calling LLMs act like prompt engineers: inspect failures, diagnose patterns, and revise prompts?

Meet Reflective Prompt Tuning (RPT): a diagnosis-driven framework for iterative prompt tuning with function-calling LLMs.

Paper: https://t.co/wNczUx32KM
@MegagonLabs
1/n

[Farima Fatahi @FarimaFB]

See our paper for more results and findings. Kudos to my co-authors @moinnas @PPezeshkpour @estevamhruschka
Paper: https://t.co/2rEJ9vkLIO
Code: https://t.co/fJEIfx7ae3

[Farima Fatahi @FarimaFB]

📢 New Preprint

Prompting is a flexible way to adapt LLMs, but prompt engineering is still manual and hard to scale. Can function-calling LLMs act like prompt engineers: inspect failures, diagnose patterns, and revise prompts?

Meet Reflective Prompt Tuning (RPT): a diagnosis-driven framework for iterative prompt tuning with function-calling LLMs.

Paper: https://t.co/wNczUx32KM
@MegagonLabs
1/n

[Farima Fatahi @FarimaFB]

Do prompt patches predict improvements?

For reasoning-heavy tasks, patches for step-by-step reasoning, output validation, and confidence calibration are associated with better performance and calibration.

For domain-specific numerical reasoning, gains are more mixed, suggesting some errors may require more than prompt edits alone. 7/n

[Andrew Ng @AndrewYNg]

The new White House policy requiring green card applicants to apply from outside the US is a capricious attack on legal immigration. It will hurt families, leave us with fewer doctors, teachers and scientists, and hurt American competitiveness in AI.

[Pouya Pezeshkpour @PPezeshkpour]

📢New Preprint📢

LLMs can solve many tasks. But who verifies an answer when the judge is also an LLM?

We introduce AutoPyVerifier: learning compact Python verifier sets from labeled LLM outputs, improving objective prediction by up to +55 F1.

Paper: https://t.co/VwJKCHR81u 1/n

[kooshiar @kooshiar]

Secretary @marcorubio

Iranian immigrants have long contributed to the U.S. economy, especially in STEM, medicine and AI. Most strongly oppose the Islamic regime and support the goal of a strong and secure USA under President Trump and your leadership.

Over the past few months, many have faced serious hardship due to the USCIS hold. They are here legally and often hold advanced degrees. Returning them to Iran would place them in danger and risk sending highly trained experts in strategic fields back to the Iranian regime.

We would greatly appreciate it if you could revisit this decision.

Thank you for your consideration.

[Pouya Pezeshkpour @PPezeshkpour]

📢New Preprint📢
Are LLM agents ready for real-world adoption? We evaluate LLM agents in a controllable interactive game with real-world stressors and find a consistent robustness gap: success drops, LLMs ranking flip, and agents show distinct strategies:
https://t.co/sDdrpgtjMY

[Farima Fatahi @FarimaFB]

Check out our paper for more results and findings!
Kudos to my co-authors @PPezeshkpour and @estevamhruschka.

[Farima Fatahi @FarimaFB]

🛠️ What is the effect of tool use on LLMs’ reasoning behavior?
We introduce 𝐓𝐨𝐨𝐥-𝐈𝐧𝐝𝐮𝐜𝐞𝐝 𝐌𝐲𝐨𝐩𝐢𝐚 — a new class of reasoning hallucination where LLMs swap coherent reasoning for tool outputs. https://t.co/oPxA6eDQ93
@MegagonLabs

[Farima Fatahi @FarimaFB]

💯Mitigating TIM Works
Our methods align TaLMs to use tools as reasoning aids, not shortcuts — improving both:
• 🎯 Final performance, and
• 🧠 Reasoning behavior