Rahul Krishna Thomas

EE PhD @ Columbia. Advised by @micahgoldblum and @adam_block66985

[Micah Goldblum @micahgoldblum]

LLM training runs can look healthy for billions of tokens or more then suddenly diverge. We find that quantization error in the LM head drives these delayed time-bomb loss spikes. A simple optimizer change can prevent them. 🧵 1/n

[Ishan Khare @ishanskhare]

Excited to announce Next-Concept Modeling (NCM): our new paradigm for fast and parallel, coarse-to-fine text generation! 🚀

Compared to strong diffusion language models (DLMs) at similar parameter sizes, NCM showcases:

⚡ 31× faster generation on downstream summarization and translation tasks.

🏎️ 10–21× faster unconditional generation across TinyStories, LM1B, and OpenWebText, with comparable or better generative perplexity using 3–5× fewer training tokens.

🧩 Better reasoning performance on tasks like Sudoku.

Check out the paper and codebase 👇

Arxiv: https://t.co/NUCOYycgcM
AlphaXiv: https://t.co/4brvP8GxDg
Code: https://t.co/CnkJHS4KQt

Enjoyed working with @KumbongHermann and advised by @simran_s_arora @Azaliamirh and @HazyResearch.

[Micah Goldblum @micahgoldblum]

🚨 LLMs are trained on past data and deployed in the future, but we evaluate them like time doesn't exist. By treating LLM evaluation as a forecasting problem, we can better predict how models will actually perform in the future. 📈 1/n

[Micah Goldblum @micahgoldblum]

Announcing Exploration-Distillation, a RL framework that unlocks aggressive exploration and large performance boosts over standard algorithms like DAPO. In principle, an LLM can discover new things during RL, but existing algorithms struggle. Here’s how we do it. 🧵1/n

[Rahul Krishna Thomas @rahulth03as]

@nikhil_suresh2 Excited for you! I know you're gonna do great things at Striker

[Mikaela Angelina Uy @mikacuy]

Happy to share our most recent work PL-NeRF to be presented @NeurIPSConf. We reformulate the volume rendering integral w/ piecewise linear opacity. #NeurIPS2023

Project: https://t.co/xlVOJYiAdq

Joint work w/@GeorgeNaka40190 @stevenygd @rahulth03as @GuibasLeonidas @KL_Div (1/3)