Shashank Sonkar

NLP+Education | @UCF @RiceECE @rice_dsp @OpenStax @IITKanpur

[SkyLight @skylight_org]

Sparse attention offers a viable path to solving the computational and memory issues of long-context LLM serving. However, we observe a surprising gap between the rapid pace of academic research and the reality of open-source adoption.

We identify the core barrier as fragmentation where new methods are published rapidly, but with missing comparative grounding.

Today we’re excited to share SkyLight, where we are building a unified framework designed to bridge this gap. SkyLight standardizes the implementation and evaluation of sparse attention to enable rigorous, controlled benchmarking.

Our first major effort is to understand the frontier of inference time sparse-decoding in LLMs.

1/N 🧵

[Shashank Sonkar @shashank_nlp]

@StevenJMoore Congratulations Steven 🎉

[Shashank Sonkar @shashank_nlp]

Excited to be part of this tutorial on Large Language Models for Education at #ACL2025! Join us tomorrow at 9 AM at the BEA Workshop (Rooms 1.85 & 1.86, Austria Center) to explore how NLP and AI are transforming education.

[Mrinmaya Sachan @mrinmayasachan]

If you're at ACL, join us for the tutorial "LLMs for Education: Understanding the Needs of Stakeholders, Current Capabilities and the Path Forward" at the BEA workshop (Room 1.85–86) 9:00-12:30am tomorrow (July 31st) @aclmeeting

[Shashank Sonkar @shashank_nlp]

@zouharvi Congrats Vilém!

[Jakub Macina @dmacjam]

Mistakes are key learning opportunities!🧑‍🎓 Can LLMs help students learn from them through dialog? 💬 While they often struggle to diagnose student errors when generating responses directly, adding a verification step ✅ could make a difference. #EMNLP2024

[Shehzaad Dhuliawala @shehzaadzd]

🚨"Towards Aligning Language Models with Textual Feedback" has been accepted at #EMNLP2024!
We explore if textual feedback can better align LLMs vs. numeric rewards. Our approach, ALT, adapts the Decision Transformer to condition responses on textual feedback. What we find 👇🧵

[Andreas Opedal @OpedalAndreas]

🚀 New paper on LLM reasoning 🚀 We present MathGAP, a framework for evaluating LLMs on math word problems with arbitrarily complex proof structures--resulting in problems that are challenging even for GPT-4o and OpenAI o1 💥 A thread 🧵
https://t.co/2k7CpIGtUv

[Shashank Sonkar @shashank_nlp]

Code and Models: https://t.co/AyN71ZP1jr

[Shashank Sonkar @shashank_nlp]

Excited that our paper "Pedagogical Alignment of LLMs" is accepted at EMNLP'24 findings 🎉

Thanks to all authors - Kangqi Ni, @Sapana_007, @rbaraniuk

Read here: https://t.co/ovMkO87imj

[Shashank Sonkar @shashank_nlp]

New contributions: We quantify using language modeling loss (perplexity) how likely LLMs are to give away direct answers vs hints (measure of LLM’s scaffolding behavior). And importantly how fine-tuning or policy optimization methods (DPO, IPO, KTO, etc) shift the scales.

[Shashank Sonkar @shashank_nlp]

@MFarajtabar Nice work --congrats ! Is the benchmark code available publicly?

[OpenStax @OpenStax]

🚀✨ OpenStax is proud to announce we have partnered with @GeminiApp to enable our library of resources to be discovered, searched, and available to users 18+ in the U.S.!

Read more here: https://t.co/L3wqGtcWRG

[Shashank Sonkar @shashank_nlp]

@singhanj13 @cab938 @xuwanghci @imjuhokim @rebquintana @ericson_barbara Woohoo 🎉 congratulations Anjali 👏🏼

[Shashank Sonkar @shashank_nlp]

@rasbt @kurtqian @natolambert @interconnectsai I see thank you :)

[Shashank Sonkar @shashank_nlp]

@rasbt @kurtqian @natolambert @interconnectsai Just curious is it possible to run 405B 8-bit model on A100s with litGPT? Seems like A100 does not support 8 bit (atleast with the huggingface library)!
https://t.co/3Fg7Hw8pYX

[Shashank Sonkar @shashank_nlp]

@conradborchers @JoonLin84 @zpardos @koedinger @cmuhcii @CAISnrw @EDMConf2024 Woohoo congratulations Conrad🎉👏🏼

[Google for Education @GoogleForEdu]

📚 We're also introducing new #Gemini features to help you learn more confidently. For example, Gemini will soon provide trustworthy responses based on textbooks from @OpenStax, a division of @RiceUniversity—including in-line citations and links to relevant peer-reviewed content.

[Shashank Sonkar @shashank_nlp]

@thedavenelson @thedavenelson thank you for sharing our paper! You accurately pointed out the challenges of long-form grading & the crucial role of human-AI collaboration. Well-designed rubrics are key for AI to effectively support graders & make their job manageable. Appreciate your insights!

[Shashank Sonkar @shashank_nlp]

You can also catch me to discuss how to build student simulation models using LLMs (where hallucinations are a desired feature and not a bug), and explore its connections to counterfactual reasoning!

[Shashank Sonkar @shashank_nlp]

🌟 Excited to announce we're presenting "Code Soliloquies for Accurate Calculations in LLMs" at #LAK24,#LearningAnalytics! Join us on Mar 21, 2024, Room K. Details 👉 [https://t.co/s9SWDPY6ps] This paper is part 2 in our series on developing pedagogically aligned LLM tutors.

[Shashank Sonkar @shashank_nlp]

3/ Paper 3: Discusses generating a reward dataset for reinforcement learning to ensure pedagogical alignment of LLMs. Find out more 👉 [https://t.co/ovMkO87QbR]