kuanchen

Working on foundation model applications in biomedical image @modella_ai

[Surya Ganguli @SuryaGanguli]

Teaching a new course @Stanford this quarter on explainable AI, motivated by neuroscience. I have curated a paper list 4 pages long (link in comment). What are your favorite papers on explainable AI/mechanistic interpretability that I am missing? Please comment or DM. thanks!

[Gokul Swamy @g_k_swamy]

It was a dream come true to teach the course I wish existed at the start of my PhD. We built up the algorithmic foundations of modern-day RL, imitation learning, and RLHF, going deeper than the usual "grab bag of tricks". All 25 lectures + 150 pages of notes are now public! 🧵

[Sara Hooker @sarahookr]

It is critical for scientific integrity that we trust our measure of progress.

The @lmarena_ai has become the go-to evaluation for AI progress.

Our release today demonstrates the difficulty in maintaining fair evaluations on @lmarena_ai, despite best intentions.

[Xavier Bresson @xbresson]

My course lecture on Diffusion Models from statistical first principles:
https://t.co/vqP5xJkb8r

The PyTorch notebooks that implement Diffusion Models from scratch w/ Transformer and UNet:
https://t.co/TfD8CZXYUz

[Arash Vahdat @ArashVahdat]

My GTC talk highlighting some of the Gen AI for science projects from my team at NVIDIA and the lessons we've learned along the way is now publicly available. Bonus point, I used some new pictures of Peanut in the presentation.
https://t.co/dc5NIU5QRk

[Jonathan Frankle @jefrankle]

The hardest part about finetuning LLMs is that people generally don't have high-quality labeled data. Today, @databricks introduced TAO, a new finetuning method that only needs inputs, no labels necessary. Best of all, it actually beats supervised finetuning on labeled data.

[Anthropic @AnthropicAI]

Introducing Claude 3.7 Sonnet: our most intelligent model to date. It's a hybrid reasoning model, producing near-instant responses or extended, step-by-step thinking.

One model, two ways to think.

We’re also releasing an agentic coding tool: Claude Code.

[ModellaAI @modella_ai]

🎉✨We are excited to report that PathChat™ DX, our clinical-grade, generative AI co-pilot for pathology, has officially received Breakthrough Device Designation from the FDA! This marks a pivotal step forward in our quest to transform biomedicine with generative and agentic AI.🌟🚀

📖 Read our press release: https://t.co/pdBkkHBtGm
🎥 See our latest demo for PathChat™ 2a below 👇
📄 Read the PathChat™ article in Nature: https://t.co/Xm19ewviWC

We’re excited to continue pushing the boundaries of innovation in healthcare! #DigitalPathology #ComputationalPathology #AI4Pathology #pathology #ai

[clem 🤗 @ClementDelangue]

Our science team has started working on fully reproducing and open-sourcing R1 including training data, training scripts,...

Full power of open source AI so that everyone all over the world can take advantage of AI progress! Will help debunk some myths I’m sure too.

Thanks @deepseek_ai!

[Sander Dieleman @sedielem]

📢PSA: #NeurIPS2024 recordings are now publicly available!

The workshops always have tons of interesting things on at once, so the FOMO is real😵‍💫 Luckily it's all recorded, so I've been catching up on what I missed.

Thread below with some personal highlights🧵

[Francis Bach @BachFrancis]

My book is (at last) out, just in time for Christmas!
A blog post to celebrate and present it: https://t.co/ttRo04efDF

[Sander Dieleman @sedielem]

This blog post provides a very accessible overview of flow matching / rectified flow and reflow, based on intuitions from physics, rather than starting from probability distributions. The visualisations and animations are excellent, and the whole thing is also a colab!

[Ekdeep Singh Lubana @EkdeepL]

Paper alert—accepted as a NeurIPS *Spotlight*!🧵👇

We build on our past work relating emergence to task compositionality and analyze the *learning dynamics* of such tasks: we find there exist latent interventions that can elicit them much before input prompting works! 🤯

[Epoch AI @EpochAIResearch]

1/10 Today we're launching FrontierMath, a benchmark for evaluating advanced mathematical reasoning in AI. We collaborated with 60+ leading mathematicians to create hundreds of original, exceptionally challenging math problems, of which current AI systems solve less than 2%.

[Dawn Song @dawnsongtweets]

Large Language Model Agents is the next frontier. Really excited to announce our Berkeley course on LLM Agents, also available for anyone to join as a MOOC, starting Sep 9 (Mon) 3pm PT! 📢
Sign up & join us: https://t.co/lOHc6HLtmG

[Manolis Kellis @manoliskellis]

The field of #ComputationalBiology has changed dramatically in the last few years with #GenerativeAI.
I'm excited to be recording my lectures again this year, and to be teaching with the awesome @EjAlm.
You can find the first lecture here: https://t.co/dBuv9M2aOX
and more will be coming in the playlist below.
Fall 2024: https://t.co/o5RBYluZXI
Spring 2021: https://t.co/1rHgfil6W2
Fall 2020: https://t.co/ekCtSr6qMn
#CompBio #DeepLearning #MachineLearning #Genomics #Therapeutics #RepresenationLearning #Chemistry #ProteinFolding #ClinicalRecords #SingleCell

[Timothy Gowers @wtgowers @wtgowers]

Google DeepMind have produced a program that in a certain sense has achieved a silver-medal peformance at this year's International Mathematical Olympiad. 🧵

https://t.co/DIcsYXUv97

[Andrej Karpathy @karpathy]

⚡️ Excited to share that I am starting an AI+Education company called Eureka Labs.
The announcement:

---
We are Eureka Labs and we are building a new kind of school that is AI native.

How can we approach an ideal experience for learning something new? For example, in the case of physics one could imagine working through very high quality course materials together with Feynman, who is there to guide you every step of the way. Unfortunately, subject matter experts who are deeply passionate, great at teaching, infinitely patient and fluent in all of the world's languages are also very scarce and cannot personally tutor all 8 billion of us on demand.

However, with recent progress in generative AI, this learning experience feels tractable. The teacher still designs the course materials, but they are supported, leveraged and scaled with an AI Teaching Assistant who is optimized to help guide the students through them. This Teacher + AI symbiosis could run an entire curriculum of courses on a common platform. If we are successful, it will be easy for anyone to learn anything, expanding education in both reach (a large number of people learning something) and extent (any one person learning a large amount of subjects, beyond what may be possible today unassisted).

Our first product will be the world's obviously best AI course, LLM101n. This is an undergraduate-level class that guides the student through training their own AI, very similar to a smaller version of the AI Teaching Assistant itself. The course materials will be available online, but we also plan to run both digital and physical cohorts of people going through it together.

Today, we are heads down building LLM101n, but we look forward to a future where AI is a key technology for increasing human potential. What would you like to learn?
---

@EurekaLabsAI is the culmination of my passion in both AI and education over ~2 decades. My interest in education took me from YouTube tutorials on Rubik's cubes to starting CS231n at Stanford, to my more recent Zero-to-Hero AI series. While my work in AI took me from academic research at Stanford to real-world products at Tesla and AGI research at OpenAI. All of my work combining the two so far has only been part-time, as side quests to my "real job", so I am quite excited to dive in and build something great, professionally and full-time.

It's still early days but I wanted to announce the company so that I can build publicly instead of keeping a secret that isn't. Outbound links with a bit more info in the reply!

[Thomas Wolf @Thom_Wolf]

There was a super impressive AI competition that happened last week that many people missed in the noise of AI world. I happen to know several participants so let me tell you a bit of this story as a Sunday morning coffee time.

You probably know the Millennium Prize Problems where the Clay Institute pledged a US$1 million prize for the first correct solution to each of 7 deep math problems. To this date only one of these, the Poincaré conjecture, has been solved by Grigori Perelman who famously declined the award (go check Grigori out if you haven't the guy has a totally based life).

So this new competition, the Artificial Intelligence Math Olympiad (AIMO) also came with a US$1M prize but was only open to AI model (so the human get the price for the work of the AI...). It tackle also very challenging but still simpler problems, namely problems at the International Math Olympiad gold level. Not yet the frontier of math knowledge but definitely above what most people, me included, can solve today.

The organizing committee of the AIMO is kind-of-a who-is-who of highly respected mathematicians in the world, for instance Terence Tao widely famous math prodigy widely regarded as one of the greatest living mathematicians.

Enter our team, Jia Li, Yann Fleuret, and Hélène Evain. After a successful exit in a previous startup (that I happen to have know well when I was an IP lawyer in a previous life but that's for another story) they decided to co-found Numina as a non-profit to do open AI4Math.

Numina wanted to act as a counterpoint to AI math efforts like DeepMind's but in a much more open way with the goal to advance the use of AI in mathematics and make progress on hard, open problems. Along the way, they managed to recruit the help of some very impressive names in the AI+math world like Guillaume Lample, co-founder of Mistral or Stanislas Polu, formerly pushing math models at OpenAI.

As Jia was participating in the code-model BigCode collaboration with some Hugging Face folks, came the idea to collaborate and explore how well code models could be used for formal mathematics.

For context, olympiad math problems are extremely hard and the core of the issue is in the battle plan you draft to tackle each problem. A first focus of Numina was thus on creating high quality instruction Chain-of-Thought (CoT) data for competition-level mathematics. This CoT data has already been used to train models like DeepSeek Math, but is very rarely released so this dataset became an unvaluated ressource to tackle the challenges.

BigCode's lead Leandro put Jia in touch with the team that trained the Zephyr models at Hugging Face, namely, Lewis, Ed, Costa and Kashif with additional help from Roman and Ben and the goal became to have a go at training some strong models on the math and code data to tackle the first progress prize of AIMO.

And the trainings started:

Jia being an olympiad coach, was intimately familiar with the difficulty level of these competitions and able to curate an very strong internal validation set to enable model selection (Kaggle submissions are blind). While iterating on dataset construction, Lewis and Ed from Hugging Face focused on training the models and building the inference pipeline for the Kaggle submissions.

As often in competition it was an intense journey with Eureka and Aha moments pushing everyone further.
Lewis told me about a couple of them which totally blow my mind. A tech report is coming so this is just some "along the way" nuggets that will be soon gathered in a much more comprehensive recipe and report.

Learning to code: The submission of the team relied on self-consistency decoding (aka majority voting) to generate N candidates per problem and pick the most common solution. But initial models trained on the Numina data only scored around 13/50... they needed a better approach. They then saw the MuMath-Code paper (https://t.co/9KGmjGJvT7) which showed you can combine CoT data with code data to get strong models. Jia was able to generate great code execution data from GPT-4 to enable the training of the initial models and get to impressive boost in performance.

Taming the variance: Another Ahah moment came at some point when a Kaggle member shared a notebook showing how DeepSeek models worked super well with code execution (the model breaks down the problem into steps and each step is run in Python to reason about the next one).

However, when the team tried this notebook they found this method had huge variance (the scores on Kaggle varied from 16/50 to 23/50).

When meeting in Paris for a hackathon to improve this issue (like the HF team often does) Ed had the idea to frame the majority voting as a "tree of thoughts" where you'd progressively grow and prune a tree of candidate solutions (https://t.co/dkKtBMrIPm).

This had an impressive impact on the variance and enabled them to be much more confident in their submissions (which showed in how the model ended up performing extremely well on the test set versus the validation set)

Overcoming compute constraints: the Kaggle submissions had to run on 2xT4s in under 9h which is really hard because FA2 doesn't work and you can't use bfloat16 either. The team explored quantization methods like AWQ and GPTQ, finding that 8-bit quantization of a 7B model with GPTQ was best

Looking at the data: a large part of the focus was also on checking the GPT-4 datasets for quality (and fixing them) as they quickly discovered that GPT-4 was prone to hallucinations and failing to correctly interpret the code output. Fixing data issues in the final week led to a significant boost in performance.

Final push: The result were really amazing and the model climbed to the 1 place. And even more, while tying up for first place on the public, validation leaderboard (28 solved challenges versus 27 for the second place), it really shined when tested on the private, test leaderboard where it took a wide margin solving 29 challenges versus 22 for the second team.

As Terence Tao himself set it up, this is "higher than expected"

Maybe what's even more impressive about this competition, beside the level of math these models are already capable of is how ressource contraint the participants were actually, having to run inference in a short amont of time on T4 which only let us imagine how powerful these models will become in the coming months.

Time seem to be ripe for GenAI to have some impact in science and it's probably one of the most exciting thing AI will bring us in the coming 1-2 year. Accelerating human development and tackling all the real world problems science is able to tackle.

[Faisal Mahmood @AI4Pathology]

What would you do with 1000+ spatial transcriptomics samples with corresponding H&E-stained whole-slide images? Meet HEST-1k, a collection of 1,108 ST samples assembled from 131 public and internal cohorts encompassing 25 organs, 2 species. HEST-1k includes over 1.5 million expression–morphology pairs.

🔍Explore 3 use-cases for HEST-1k:
- HEST-Benchmark: Evaluate gene expression prediction from histology across 10 organs and 9 cancer types, testing multiple foundation models for pathology including UNI, and GigaPath.
- HEST for discovery: Explore our proof-of-concept for multimodal biomarker characterization using Xenium breast cancer samples.
- HEST for fine-tuning pathology foundation models: See how HEST-1k can enhance foundation models for histology with expression-guided fine-tuning.

📄Preprint: https://t.co/X43hYUKQMR
👩‍💻 Code and Data access: https://t.co/Sy5j8LVFNk

Congratulations to @guillaumejaume, @pauldoucet, and everyone else who contributed to this work. Huge thanks to everyone who helped curate the dataset. #SpatialTranscriptomics #ComputationalPathology #CancerResearch #Bioinformatics