ARC-AGI-4 will be a benchmark for autonomous open-ended innovation. It will continue our commitment to open-source, giving the research community a shared target for progress that benefits all of humanity.
Despite rapid model progress, humans still significantly outperform AI at open-ended invention. This is the meta-skill that unlocks progress across every field of technology.
Advanced AI capable of scientific innovation will lead to tremendous new technology, knowledge, and understanding. This is a positive-sum future. We are deeply committed to advancing it.
Open source is the foundation for that progress.
The knowledge behind frontier AI, not just the technology itself, should be broadly distributed among researchers, academics, and organizations. Any coordinated effort by the AI industry to reduce openness or concentrate access to frontier AI would undermine that positive-sum future. We are committed to advancing a future where everyone can contribute to and benefit from AI progress.
The ultimate goal is getting paid to be yourself. To solve problems you can't help but solve. To study what you can't pull yourself away from. To eliminate any work that does not challenge you creatively.
Airbnb just published how they run evals internally, and it reads like a job description for an AI engineer
Three layers. Programmatic checks first, an LLM judge second, humans last and only to calibrate the judge.
The numbers they work to: golden sets of 50 to 100 examples that have to include failures, judges calibrated to high 80s or 90s agreement with a human, measured with Cohen's kappa. 5% of live traffic sampled every day.
The admission that makes it real: roughly three quarters of their LLM-generated reference answers came out different on every labeling run. Their eval was measuring its own noise.
They got a full cycle from weeks down to a day, mostly by caching identical outputs and training tiny LoRA adapters.
Nobody in a job ad calls this eval engineering, but this is the work. Full breakdown in the article below.
Bookmark this
Robot learning is one of those fields where you can actually catch up from scratch.
The cheat code? ETH Zurich’s Robot Learning course. It basically gives you the roadmap: imitation learning, RL, visual control, manipulation, and the papers that shaped the field.
Then just nerd-maxx the papers and build the experiments.
Open-source AI models do not threaten frontier lab profitability.
The most valuable sectors of the economy - investing, product development, war, cybersecurity, even scientific discovery - are adversarial and competitive in nature.
You pay to win, or someone else will.
WAIT A MINUE, so Claude is not only the most expensive frontier model, but Claude Code is also the most token-hungry harness?
the Claude Code enterprise bills are starting to make sense.
We can finally talk about it:
We found a way to extract hidden reasoning of frontier models using a vulnerability in the APIs of every frontier AI company.
We verified that our reasoning token count matches billed API thinking tokens 1:1 for most of the prompts we queried.
Agency > Intelligence
I had this intuitively wrong for decades, I think due to a pervasive cultural veneration of intelligence, various entertainment/media, obsession with IQ etc. Agency is significantly more powerful and significantly more scarce. Are you hiring for agency? Are we educating for agency? Are you acting as if you had 10X agency?
Grok explanation is ~close:
“Agency, as a personality trait, refers to an individual's capacity to take initiative, make decisions, and exert control over their actions and environment. It’s about being proactive rather than reactive—someone with high agency doesn’t just let life happen to them; they shape it. Think of it as a blend of self-efficacy, determination, and a sense of ownership over one’s path.
People with strong agency tend to set goals and pursue them with confidence, even in the face of obstacles. They’re the type to say, “I’ll figure it out,” and then actually do it. On the flip side, someone low in agency might feel more like a passenger in their own life, waiting for external forces—like luck, other people, or circumstances—to dictate what happens next.
It’s not quite the same as assertiveness or ambition, though it can overlap. Agency is quieter, more internal—it’s the belief that you *can* act, paired with the will to follow through. Psychologists often tie it to concepts like locus of control: high-agency folks lean toward an internal locus, feeling they steer their fate, while low-agency folks might lean external, seeing life as something that happens *to* them.”
A SpaceX Falcon 9 rocket was seen streaking across the skies of southern California Monday, after it launched from Vandenberg Space Force Base on a mission to launch Starlink satellites. #spacexlaunch#SpaceX
100% power Aeon R full engine test, right out of the gate on our Block 2 engine iteration 😎 this one is 268,000 pound thrust vs 258,000 before, +many design upgrades. Block 3 is being built in the factory now. Extreme iteration speed for our reusable Terran R rocket 🦾🇺🇸 @relativityspace