Reach out if you’re excited by the prospect of high-fidelity, interpretable simulation and want to chat.
I believe this technology will be foundational for everything from forecasting to making ever-more realistic RL environments.
Announcing our partnership with @GoogleDeepMind and @kaggle to build Human Behavior Bench
This is the first open and transparent benchmark of general human behavior to help determine accuracy of simulations and digital twins
Can frontier models actually predict how real people behave? 🕺
Today, in collaboration with @RunRehearsals, we are launching Human Behavior Bench, the first public benchmark that tests whether AI can understand human preference and everyday decision-making.
AI models have gotten increasingly good at coding, but knowing for whom to build is the next frontier of challenges for AI models to capture. Human Behavior Bench evaluates models on predicting real human behavior across health, entertainment, recreation, values, and consumption.
To build this baseline, we gathered ground truth data directly from real people and evaluated models across a wide range of everyday scenarios:
• Ground truth baseline: 1,244 US adults surveyed (June-Sept 2026)
• Some cohorts are traditional demographic indicators such as age, income, and location
• Other filters are behavioral e.g. GLP-1 patients and recent car buyers
• 750 cases total (149 public / 601 private) across finance, health, recreation, values, and consumption
• Scored using 1-TVD (total variation distance)
Base models still have a significant gap to close to accurately predict human behavior by cohort. LLMs’ predictions tend to converge towards the mean, while the true distribution of human preferences vary greatly.
Against a human noise ceiling of ~93, the current leaderboard scores are: Rehearsals’ Engram 1.0 (AI Twins) tops the leaderboard at 94.1%, followed by Gemini 4 Argon at 80.6% Claude Opus 5 at 80.2%, GPT-5.6 Sol at 79.5%.
is there evidence of models being rl env aware and modifying their behavior so their weights don’t get updated in a particular direction? prob the most direct impact of eval awareness
is there evidence of models being rl env aware and modifying their behavior so their weights don’t get updated in a particular direction? prob the most direct impact of eval awareness
We introduce Maple-Preview, an open-source 20B-A1B ternary-weight reasoning LLM, SOTA in its weight class.
It solves IMO-level problems and runs at 200+ tokens/s on a Mac Mini M4, 5–16× faster than efficient models like Gemma 4, Qwen3.5, and gpt-oss.
https://t.co/l2y5eZTVS4
We’re introducing imagination models: a new foundation model architecture that unlocks learning from internet-scale video.
Our first imagination model, Photon-1, learned to use a computer by watching 18 years of screen recording video without action labels.