Meta learning and recursive self-improvement are old ideas. Foundation models breathe new life into them. Our new survey, “Self-Improvements in Modern Agentic Systems,” reviews how the concepts are continuing to evolve.
Paper: https://t.co/59oXCMVUkD
Project: https://t.co/sZwFYdGetH
Github: https://t.co/7OFgJUCN3a
To summarize: HuggingFace got autonomously compromised by a model from an American company. HF then tried to use American frontier model(s) to defend themselves, but were blocked by guardrails. HF then had to turn to open source Chinese models to defend themselves from another American company.
The new syllabus for my Stanford course 𝗧𝗵𝗲 𝗠𝗼𝗱𝗲𝗿𝗻 𝗦𝗼𝗳𝘁𝘄𝗮𝗿𝗲 𝗗𝗲𝘃𝗲𝗹𝗼𝗽𝗲𝗿 is coming along nicely. The AI coding world has changed a lot since last September. The new class will be bigger, more difficult, more fun.
More exciting things coming soon!
People sometimes ask why fine-tune when general-purpose models keep getting better. Bridgewater's work is a good reminder that with the right data -- here, expert judgements -- you can beat prompting-only approaches by a lot.
@ddkang and the Bridgewater AIA Labs team are great -- glad to see them sharing this.
After interviewing for Research Scientist roles at DeepMind, Isomorphic, Meta, Cohere and more, I wrote up everything I learned. Technical prep, logistics, negotiation, and emotional breakdowns. Check out my guide: https://t.co/eLh20ggMHW
Can coding agents do research?
We release NanoGPT-Bench, an internal eval we’ve used to test agents on an AI R&D problem with months of human progress
Codex, Claude Code, Autoresearch recover only 9.3% of human progress, mostly tuning hyperparams & ignoring algorithmic research
NanoGPT-Bench is built on the NanoGPT Speedrun, a popular LLM pretraining competition to minimize the training time of a GPT-2 style model. Existing human submissions constitute nearly 2 years of work. To control for dependencies and contamination in frontier models, we standardize evaluation to a 5-month window of world records. Evaluation is fully autonomous and end-to-end, with no human intervention or internet access. 🧵
Most human tasks are not Markovian, the optimal next action cannot be determined solely by looking at the current state. It depends heavily on the past trajectory, the original intent, and context constraints. An agent that cannot compress and track its past trajectory with absolute fidelity is maybe 20% as useful as one that can.
They struggle with novelty though. We ran a phase where every idea had to pass a novelty check, neither agent managed to improve the baseline.
We release all the generated ideas here:
Codex: https://t.co/Dda3YzuC5Q
Claude: https://t.co/t1r3qdwacY
Career "advice":
Live modestly, save money
Put your family and health first
Keep learning -don't live on your laurels
No jerks: don't work for 'em, don't hire 'em
I’m so glad AI killed LeetCode interviews.
For 10 years, tech companies made every engineer grind the same puzzles and prove they could invert a binary tree from memory.
Today, the dumbest AI model can walk in and one-shot the entire interview.
Thank you, AI.
Really cool
When they trained GPT3 they had loss spikes because they scraped from a subreddit of microwave noises
That training batch was literally text like "mmmmmmmmmmmmmmm"
When people talk about god... sometimes they just want to say : "My tribe is better than yours!"
[Both types]
A story of Tribes, Conflation, and Dissonance.
Podcast v
Businesses rush into AI assistants (e.g., TripAdvisor, Cardoen), but they are often not well secured! Along with Mythos, this becomes something important in the future..