The era of throwing millions of random samples into the GPU woodchipper is over. We’re drowning in mediocre bits while starving the critical "tails" of our data distributions.
My take: More data isn't better data. Relying on sterile simulation mirages or unchecked "AI agents" to clean up our datasets is just a pedagogical hack that risks model collapse. Responsible AI requires active data stewardship, human-in-the-loop verification, and strict dataset provenance.
Read the full essay on why it's time to move past the Great Data Delusion: https://t.co/RRi7gjoJCV
Life Update: After 14 years living abroad in the USA and Switzerland, I am looking forward to returning to India 🇮🇳 and restarting my academic career in Computer Science at the University of York, Mumbai https://t.co/VkVloYLoyI Excited for the adventure!
I packaged up the "autoresearch" project into a new self-contained minimal repo if people would like to play over the weekend. It's basically nanochat LLM training core stripped down to a single-GPU, one file version of ~630 lines of code, then:
- the human iterates on the prompt (.md)
- the AI agent iterates on the training code (.py)
The goal is to engineer your agents to make the fastest research progress indefinitely and without any of your own involvement. In the image, every dot is a complete LLM training run that lasts exactly 5 minutes. The agent works in an autonomous loop on a git feature branch and accumulates git commits to the training script as it finds better settings (of lower validation loss by the end) of the neural network architecture, the optimizer, all the hyperparameters, etc. You can imagine comparing the research progress of different prompts, different agents, etc.
https://t.co/YCvOwwjOzF
Part code, part sci-fi, and a pinch of psychosis :)
This repository gives you virtually infinite FREE Computer Science Education:
"Computer Science Courses With Video Lectures"
Enjoy!
https://t.co/0LGgKPMa4S
Today, Michael Levin and I are proud to announce the unveiling of
The Institute for Computationally Designed Organisms,
a joint venture between The University of Vermont and Tufts University.
https://t.co/nt0mA2r1PN
A *free* online (duh) summer school in Algorithms, Combinatorics, and Complexity. Featuring three awesome lecturers, Maria Chudnovsky, Fedor Fomin, and Madhu Sudan. From May 24 to May 28, register here https://t.co/14BN26hvlg
Two weeks into the semester, there's already an incredible energy in everyone. To inspire our efforts in 2021, we look back today on some 2020 highlights of our department.
"Have a great Winter and Spring, learn a lot, and let's keep our dialogue going." -@ChristianSkalka
Are you a student studying open source software and you need to cover software costs or registration fees?
Do you have a great idea for an experiment or in need of computing time?
An association promoting open practices in science and tech?
All of the above? We want to help!