Paras Jain

AI at ION, prev student @acalonia_x ertswhile TokyoDS

[François Chollet @fchollet]

I would have assumed it was fairly obvious, but in case it's not: a million-line codebase (also known as a "harness"), running at inference time, orchestrating thousands of calls to a neural network for any given task, is the exact definition of a "neurosymbolic architecture"

[Horace He @cHHillee]

Presenting my grand unified theory of ML researcher impact: Your impact is directly proportional to how much pain you cause to infra.

Fundamentally, you can only inflict pain upon infra if your approach actually works. And the better your approach works the more pain infra is forced to endure.

So, to give some examples:
- MoE's add a ton of data-dependent computation => pain (shazeer++)
- GDN/KDA are the most complex architecture I've been forced to care about and a very annoying matrix inversion => pain (sonta++)
- Muon is much more annoying than Adam and causes annoying restrictions on parallelism => pain (keller/jeremy++)
- RL scaling forced many researchers to care about LLM inference and RL infra as a category => pain (tworek++)

Even papers like Attention Is All You Need have lead to significant pain! Before transformers were invented everyone was running small jobs and I never needed to think about kv-caches or 6D parallelism.

[Jacob Andreas @jacobandreas]

If you're excited about Anthropic's J-space work, definitely worth checking out the original paper on Jacobian lenses by @evanqed and @arnab_api! https://t.co/gttqzgB4VB

[Harshita Chopra @chopra_harshita]

Are you training or evaluating agents with LLM-based user simulators?

Most simulators inherit the behavior of their underlying models: cooperative, clear, and homogeneous - which is unrealistic! 🤖

Humans are messy: they falter, forget, push back, and behave in ways that are difficult to define. Manually writing personas becomes brittle and hard to scale. 😣

We introduce 𝗣𝗲𝗿𝘀𝗼𝗻𝗮 𝗣𝗼𝗹𝗶𝗰𝗶𝗲𝘀 (𝗣𝗣𝗼𝗹): an evolutionary framework that automatically discovers behaviors and instructions to generate diverse human-like user personas for any given task – ✨grounded in real dialogue traces✨

[Paras Jain @prsjna]

@bernhardsson did not know I'd find such a fine match of a role I had been looking for all this while, that too @modal -- Stockholm, let's go!

[Jo Kristian Bergum @jobergum]

You know me as the BM25 guy, but embeddings are cool too.

New post from the @HornetDev team just dropped. ANN tuning at 100M scale, covering embedding bias, graph connectivity, and quantization ceiling

https://t.co/aPWYLXiGtK

[Paras Jain @prsjna]

@EmtiyazKhan Congratulations Emti!
Ah always the masters requirement for PhD, would have loved to apply and work with you

[Paras Jain @prsjna]

@alfcnz the modern day dl-book-authoring gstack

[Sayak Paul @RisingSayak]

Editing images is a series of state transitions between the source image and the edited image that we want. Yet, the existing paradigm doesn't explicitly include any transitioning priors in the editing process.

This becomes particularly prevalent for edits, involving causal dynamics (e.g., refraction, deformation).

To model this kind of physics-informed information, we leverage the rich priors present in videos and introduce PhysicEdit 🔥

TL;DR: We fine-tune QwenImage Edit on a curated dataset of videos with reasoning traces and fixed-length transition queries to do solid physics-aware image editing!

In the process, we introduce a cool dataset "PhysicTran38K", consisting of 38K transition trajectories across five physical domains and devise a method to provide supervision from it QwenImage Edit.

Hop in to learn more ⬇️

[Angjoo Kanazawa @akanazawa]

If you let VLMs experiment on their own, they can do surprising things!

From an image, we let a VLM code a 3D scene from scratch in Blender, and then render to verify/refine in a loop. Even when each step is imperfect, it gets results like this, with zero training.

[Paras Jain @prsjna]

@sarahookr @adaptionlabs see you there!

[Nirosha J. Murugan @niroshajmurugan]

This @Nature study shows why energy and information must be understood together.

Biology does this too: mitochondria manage energetic flux, while biochemical and electrical signals encode information about stress, state, and repair.

Health depends on keeping these two modes aligned: 1) efficient energy distribution and 2) accurate information about perturbations.

https://t.co/0fKZ5s5igb

[Bidipta Sarkar @bidiptas13]

Introducing 🥚EGGROLL 🥚(Evolution Guided General Optimization via Low-rank Learning)! 🚀 Scaling backprop-free Evolution Strategies (ES) for billion-parameter models at large population sizes

⚡100x Training Throughput
🎯Fast Convergence
🔢Pure Int8 Pretraining of RNN LLMs

[Paras Jain @prsjna]

@jxmnop good luck!

[Percy Liang @percyliang]

You spend $1B training a model A.
Someone on your team leaves and launches their own model API B.
You're suspicious. Was B was derived (e.g., fine-tuned) from A?
But you only have blackbox access to B...
With our paper, you can still tell with strong statistical guarantees (p-values < 1e-8).
Idea: test for independence of A's training data order with likelihoods under B.
There are crazy amounts of metadata about training process baked into the model that can't be washed out, like a palimpsest...

[Paras Jain @prsjna]

. @myogiadityanath @myogioffice @CMOfficeUP

[Dr. Atul Kumar Jain @DrAtulKJain]

माननीय महोदय,
निवेदन है कि जैन धर्म स्थानक (मंदिर) के. बी.46 कवि नगर को बचाने/संरक्षित /सुरक्षित रखने के संबंध में प्रकरण आपके संज्ञानार्थ प्रस्तुत कर रहा हूँ।

आपसे विनम्र अनुरोध है कि कृपया विषय पर संज्ञान लेकर आवश्यक एवं उचित कार्रवाई करने की कृपा करें।

आपकी अति कृपा होगी।

[Dimitris Papailiopoulos @DimitrisPapail]

We don’t even understand a two layer perceptron

[Shashwat Goel @ShashwatGoel7]

There's been a hole at the heart of #LLM evals, and we can now fix it.

📜New paper: Answer Matching Outperforms Multiple Choice for Language Model Evaluations.

❗️We found MCQs can be solved without even knowing the question. Looking at just the choices helps guess the answer and get high accuracies. This affects popular benchmarks like MMLU-Pro, SuperGPQA etc. and even "multimodal" benchmarks like MMMU-Pro, which can be solved without even looking at the image ⁉️.

Such choice-only shortcuts are hard to fix. We find prior attempts at fixing them-- GoldenSwag (for HellaSwag) and TruthfulQA v2 ended up worsening the problem. MCQs are inherently a discriminative task, only requiring picking the correct choice among a few given options. Instead we should evaluate language models for the generative capabilities they are used for. We show discrimination is easier than even verification, let alone generation.

🤔 But how do we grade generative responses outside "verifiable domains" like code and math? So many paraphrases are valid answers... We show a scalable alternative--Answer Matching--works surprisingly well. Its simple--get generative responses to existing benchmark questions that are specific enough to have a semantically unique answer without showing choices. Then, use an LM to match the response against the ground-truth answer.

👨‍🔬We conduct a meta-evaluation by comparing to ground-truth verification on MATH, and human grading on MMLU-Pro and GPQA-Diamond questions. Answer Matching outcomes give near-perfect alignment, with even small (recent) models like Qwen3-4B. In contrast, LLM-as-a-judge, even with frontier reasoning models like o4-mini, fares much worse. This is because without the reference-answer, the model is tasked with verification, which is harder than what answer matching requires--paraphrase detection--a skill modern language models have aced💡

Lets shift the benchmarking ecosystem from MCQs to Answer Matching. Impacts:
Leaderboards: We show model rankings can change and accuracies go down making benchmarks seem less saturated.
Benchmark Creation: Instead of creating harder MCQs, we should focus our efforts on creating questions with for answer matching, much like SimpleQA, GAIA etc.
🤑 Cost: Finally, to our great surprise, answer matching evals are cheaper to run than MCQs!

See our paper for more, its packed with insights. 🧵 has paper and more result figures.

[Essential AI @essential_ai]

[1/5]

🚀 Meet Essential-Web v1.0, a 24-trillion-token pre-training dataset with rich metadata built to effortlessly curate high-performing datasets across domains and use cases!