Charvi Jain

PhD student, AI for Automated Scientific Discovery & Synthesis
@tudresden_de

[Charvi Jain @charvijain16_]

@anjusingh2512m 💯

[Charvi Jain @charvijain16_]

To all the matchapaglus, it is mehndi ka paani (henna water) 🤢

Hats off to whoever has done the marketing for matcha. 10 euros for a 30 g pack, WOW.

[Charvi Jain @charvijain16_]

@justinskycak 💯 as much as I love AI for writing all the boiler plate code, I hate it when it starts to go out of my control

[Charvi Jain @charvijain16_]

@PAHoyeck I am at 9 and haven’t even done 1 yet.

[Charvi Jain @charvijain16_]

Doomscrolling wasn’t the point. This is.

[Jash Dholani @oldbooksguy]

Mozart on when his ideas flowed best

[Charvi Jain @charvijain16_]

@alec_helbling hmm, explains the reason why it often coughs errors

[Charvi Jain @charvijain16_]

@tomjiralerspong Can't relate to it any stronger. At the same time, I am also in awe of how teams like Mirendil are able to pretrain models at scale without a pretraining team, and here I am just figuring out how not to fall into the psychosis pit.

[Charvi Jain @charvijain16_]

@gabriel1 Every time I open X, I get a minor concussion from this unfathomable depth of wisdom, stopp..

[Cameron R. Wolfe, Ph.D. @cwolferesearch]

Tomorrow morning, I’ll release a blog that outlines a detailed roadmap for setting up evals on subjective / open-ended tasks. The blog also covers a variety of stats / ML tricks for reducing noise and increasing confidence in your evals.

Here is a list of blogs, papers, and other resources (written both by me and by others) I often return to when thinking about this topic:

- High-quality human data: https://t.co/5o8flZLzzX
- Human evaluation guidelines: https://t.co/BwgTwv26e3
- LLM-as-a-Judge core concepts: https://t.co/ZVVIqezskV
- LLM-as-a-Judge practical guide: https://t.co/Qsv78WlBfB
- LLM-as-a-Judge best practices: https://t.co/S76u1d4NS1
- Rubric-based judges (RL focus): https://t.co/rNOAiEL59D
- Rubric-based judges (eval focus): https://t.co/jUrQIgQrxz
- Stats for LLM evals: https://t.co/Qsu4vBZ1DH
- Adding error bars to evals: https://t.co/hYMrnaiGxb
- Anatomy of a benchmark: https://t.co/7nFvsMCyR0
- Agent evals: https://t.co/LkdsoLXYGu
- Full ecosystem for agent evals: https://t.co/jkA8DT8t8W

Would highly recommend reading all of these if you're interested in building better evals. Each of these blogs also link to further papers / resources on each topic that are incredibly useful.

[Bassel Al Omari @BasselAlOmari_]

AI research agents can generate hundreds of ideas in seconds, but evaluating each can take days of GPU time. When compute is limited, which idea deserves execution?

We introduce AI Research Preference Models (RPMs) to assess ideas & focus compute on the most promising paths 🧵👇

[Charvi Jain @charvijain16_]

@RespectfulMemes I still get them all the time 😭

[Jarred Sumner @jarredsumner]

I’m very much not a mathematician. I dropped out of high school at 16. I took about half of sophomore year geometry before dropping out. I didn’t contribute any math to the paper. I mostly just told Claude variations of “keep going” and “believe in yourself”

[Charvi Jain @charvijain16_]

Impressive 🙌🏻

[Jeff Dean @JeffDean]

@chhaviyadav_ Soccer (2 games/wk in my 25-and-over league), gym 2x/wk, yoga 2x/wk, run to work 3-4 directions/wk (8k each way), bike to work 3-4 directions/wk (8k each way), longer run or bike on weekend.

[​𝐥𝐲𝐫𝐚 @sunnkssdseraph]

You'll regret it if you get married, you'll regret it if you don't get married. You'll regret it if you have kids and you'll regret it if you don't. Kierkegaard said this 200 years ago as follows - whatever you choose, you'll regret it, because the problem isn't in your choices, it's in romanticizing your life. A person always finds an untravelled path alluring and mysterious. That's why the issue isn't making the right choice. It's choosing and deciding which regret you'll live with. In life, we must choose our regret.

[shego 🐲 @hydrang9a]

My refusal to allow any kind of fun in my life if i am working on something

[Nithya Shri @Nithya_Shrii]

Some of my best decisions started with “actually, I don’t want to do this anymore.”

[Jakob Foerster @j_foerst]

A few years ago I worked on a conceptually interesting problem: how do we prevent agents from developing their own cryptic communication protocols?
I didn't think this would become relevant so quickly.
Here's the paper https://t.co/Y5w4qqe3KE

[Le Cong@Stanford, AI+Bio+Gene-Editing @lecong]

Thrilled to release two new preprints on intelligent labs for driving science and innovation. This is in close coordination with Aviv Regev, Jian Ma (@jmuiuc), Michelle Lee (@michellearning), and the teams at @Genentech, @SCSatCMU, and @Princeton University.

In our perspective, we argue that the next generation of labs should be human-in-the-lead and AI-empowered, integrating human intent, machine reasoning, and physical experimentation through scientific world models and an agentic harnessing layer.

Done responsibly, these systems have the potential to make scientific discovery more programmable, reproducible, adaptive, and scalable while enabling scientists to focus on higher-level scientific reasoning and discovery.

It's been a privilege to pursue this Perspective with @MengdiWang10 and an outstanding group of scientists and innovators advancing the intersection of computation, AI, science, and medicine. We're excited to continue exploring where this vision leads.

Our preprint: https://t.co/aKEkUIaNwb
Preprint led by Jian and Aviv team: https://t.co/yOzQ6cI9XD

This also kicks off a new Gladstone-Stanford AI Hub efforts, led by Katie Pollard at @GladstoneInst and myself, with an amazing team of scientists including Emma Lundberg (@Prof_Lundberg), Brian L Trippe (@brianltrippe ), Anshul Kundaje (@anshulkundaje ), Barbara Engelhardt (@BeEngelhardt), Christina Theodoris (@TheodorisLab), Catherine Tcheandjieu (@ines_catherine), Bruce Conklin, Alexander Marson, Stacie Dodgson (@StacieDodgson), Seth Shipman (@seth_shipman), Vijay Ramani, Danielle Swaney (@dlswaney), and Nevan Krogan. Excited to be building together across two great institutions!