Harnesses often get dismissed as just scaffolding, just prompt engineering, and not real research. But that couldn't be farther from the truth. The same model weights that score 30% on ARC-AGI score 95% with a better harness.
So we gathered a group of researchers and founders working at the frontier to do a deep dive into the state of harnesses.
We cover how we got to this point, the case for making your harness as expressive as possible, and what YC learned building an agent for every employee in the company.
00:00 - @FrancoisChauba1: Why harnesses matter
04:27 - Building an auto-researcher by accident
07:13 - A five minute history of harnesses
13:56 - Self-improving harnesses
18:35 - @sethkarten: Prime Agent, a self-improving RLM harness
21:50 - Context as an L1, L2, L3 cache
24:51 - From Turing machine to von Neumann computer
28:33 - Messaging between agents
30:04 - ARC-AGI results
33:09 - Emulator Bench and GPU kernels
37:30 - @JonSaadFalcon: OpenJarvis, personal AI on personal devices
38:26 - How far behind are local models
39:21 - The five primitives of a personal AI stack
42:47 - Letting cloud models optimize your local stack
43:53 - 800x cheaper than the cloud
45:58 - @josh__france and @jbellregan: QM, YC's agent harness for work
47:29 - A history of YC's internal agents
49:24 - OpenClaw and a fleet of 50 agents
51:04 - Pulling the brain out of the sandbox
54:43 - Letting the agent choose its own sandbox and model
57:16 - The grind tool: budgets on goals
58:50 - Agents don't understand social context
This perspective from Bruce Wittmann on computational protein engineering is phenomenal. Ignoring library construction and costs is a big gap between researchers only training and benchmarking models and those applying them.
https://t.co/3fftIyzsGX
This is arguably one of the best technical books you can read on how LLMs are trained at scale:
> GPU memory and profiling
> tiling, kernel fusion, FlashAttention
> Data / Tensor / Pipeline / Context Parallelism
I’ve already read the online free version, but still had to get the physical copy for the library lol.
You can also read it here on @huggingface: https://t.co/zkFhABMOTw
Today we're releasing data on models accelerating research at OpenAI.
Recursive self-improvement could be the most important contributor to AI capabilities over the next few years, but by default it will only be seen inside a few frontier AI labs. Being transparent is more urgent than ever, so we can inform the public discussion on whether and how to pace model development. I ask other AI companies to do the same.
https://t.co/iLKbrLcBAI
Try this today
> go to https://t.co/Mraotu5K3s
> grab Qwen3.5-0.8B (base, not instruct)
> grab Jiayi Pan's Countdown-Tasks-3to4 dataset
> skip sft entirely, go straight to RL
> write a reward fn: did it hit the target number run?
> run grpo for a few hundred steps watch it start reasoning on its own
boom, there's your RL project to showcase
Learning resources play a key role in shaping where talent goes. Today I’m launching my side project, BioTorch, in public beta to give researchers a clearer path into biological AI.
LLMs have a wealth of great tutorials, courses, and minimal implementations from great teachers. They make the path from curiosity to contribution easy to see and offer researchers from fields like math or physics a legible roadmap to transition into AI research.
In bio AI, that path is much harder to piece together. That friction can mean fewer researchers entering the field, fewer startups getting built, and less investment allocated to promising ideas. This ultimately leads to fewer life-saving cures being discovered than would otherwise be possible.
BioTorch is my attempt to help change that.
It starts with 116 PyTorch exercises and 17 model guides. You implement the building blocks, test your code, and see how the pieces fit into seminal models like AlphaFold2, ESM2, and RFdiffusion.
The goal is to make influential papers in bio AI something you can understand and build on, bringing researchers up to the frontier as rapidly as possible.
I want more people working on biology’s hardest problems. My mission is to help researchers find their way into the field, and BioTorch is the result of that ambition.
BioTorch is completely free while in beta.
Try your first problem now!
https://t.co/BHmN59S3bI
China published the most uncomfortable paper on vibe coding.
ETH Zurich tested 100 developers in a controlled, commercial-grade vibe coding environment to see who actually succeeds.
The findings are brutal.
The researchers tracked computer science achievement, written communication skills, and general cognitive reasoning.
They wanted to see what actually predicts vibe coding proficiency when you never touch a line of source code yourself.
Two major predictors emerged.
Written communication proficiency mattered. The ability to structure thoughts and articulate intent unambiguously in text directly impacts what the AI builds.
But that wasn't even the main takeaway.
Computer science achievement was a massive, dominant predictor of success.
Even when researchers controlled for general intelligence and reasoning skills, CS background still heavily dictated who built working software and who completely crashed.
In fact, CS knowledge contributed roughly twice the unique predictive variance of writing skills alone.
Why? Because vibe coding isn't about writing code. It’s about debugging logic.
When an AI agent builds a complex application and quietly breaks an edge case under the hood, a non-technical user looks at the glowing UI and assumes it works.
They don't know what questions to ask. They don't know what logic to challenge. They lack the mental models to recognize architectural catastrophe.
You can prompt your way past syntax.
You cannot prompt your way past a fundamental lack of engineering intuition.
The hype told us that learning to code is dead because language is all you need.
The data just proved the opposite.
To truly master the vibe, you still need to understand how the machine thinks.
we published a blog on hugging face at possibly the worst time yesterday lol
congrats to the HF team on the big news! 💚
here's a fine-tuning tutorial showing how to
• fine-tune a tiny LFM2.5-350M model
• in 100 GRPO steps using TRL
• for better structured outputs
blog: https://t.co/0ijRPNQdzH
colab: https://t.co/QAAPY5a6Vn
The most important skills for using AI coding agents effectively. Presenting the AI Engineering Skills Map for using coding agents. https://t.co/GrEw7wG5Wz
This Microsoft research paper is f*cking mind-blowing
Microsoft researchers just proved that a tiny 7B open-source model can beat OpenAI o1 on Olympiad mathematics without copying a single token from larger teacher models
Their breakthrough paper, rStar-Math, solves the holy grail of test-time compute: combining Monte Carlo Tree Search, step-level code verification, and a 4-round self-evolution loop
How Microsoft turned a 7B model into an Olympiad-level reasoning engine:
→ The Harness Layer (Code-Augmented MCTS): executes Python verifiers at every reasoning step to filter out invalid intermediate calculations before expanding the tree
→ The Graph Layer (Process Preference Model): uses Q-value branch comparisons to score individual reasoning steps rather than relying on noisy final-answer rewards
→ The Loop Layer (4-Round Self-Evolution): bootstraps the policy model and verifier in iterative cycles over 747k problems, unlocking autonomous self-improvement from scratch
The Benchmark Numbers:
• MATH Benchmark → Qwen2.5-Math-7B surged from 58.8% to 90.0% (beating OpenAI o1-preview by +4.5%)
• Phi3-Mini (3.8B) → exploded from 41.4% to 86.4% (+45.0% accuracy leap)
• AIME Olympiad 2024 → solved 53.3% (8/15) of competition problems, outperforming top human math Olympians
Hypothesis Verification & Practical Validation:
Microsoft hypothesized that combining step-by-step code-verified tree search (Graph) with self-evolving process preference models (Loop) allows small models to outperform frontier reasoning models without distillation. Practical validation confirmed the findings: deploying Microsoft's rStar-Math tree search and harness verifier across our internal reasoning pipeline drove small model task accuracy from 52.4% to 88.6% (+36.2% gain) while slashing intermediate token waste by 47% via step-level Q-value pruning
The future of AI reasoning is not training trillion-parameter black boxes. The true leverage comes from test-time compute, sandboxed code verifiers, and iterative search graphs
Read the complete breakdown in the article below ↓
Banger paper from the Qwen team.
If you evaluate agents on anything longer than a single session, this one is worth your time.
(bookmark it)
E-Commerce Bench runs an agent through a simulated 365-day year operating several online stores at once.
18 frontier models are scored across seven dimensions and no single model dominates.
GPT-5.6 Sol earns the most, growing a 100,000 opening stake into 1,431,425, then ranks 16th of 18 on fraud avoidance and trails Fable 5 on operational efficiency.
Among open-weight models, Qwen3.8-Max-Preview leads at 416,252, 38% above GLM 5.2 (high), and shows the strongest learning over the horizon by progressively bargaining suppliers down across repeated orders.
Paper: https://t.co/pX3u6GABA6
Chat with Paper: https://t.co/SXhS6zAxBF
Training tiny models for special purpose use cases works so incredibly well if you have a great self improving recursive flywheel. Shopify ML team is on fire.
finetuned 0.8b model beats GPT 5.6-sol xhigh in this very specialized task.
Today we're releasing TimesFM 3.0 on Hugging Face
- Open foundation model for time series forecasting
- Complex multivariate scenarios with zero-shot
- 330m parameters
Excited about the research and new use cases this will unlock!
Introducing TimesFM-3, a state-of-the-art time series foundation model that enables accurate multivariate time series forecasting in a single forward pass, significantly outperforming other forecasting models across major benchmarks.
More on the blog →https://t.co/uSlnIdUJ4Q
Interesting technical work from Microsoft.
Provides a better understanding on SFT and how to leverage it better for RL.
Microsoft researchers asked whether a standard SFT pipeline actually produces the model you want to run RL on.
Their answer is no.
Standard SFT keeps spending gradient on sequences the model has already fit, which narrows the distribution RL later needs to explore.
TailSFT filters those sequences out during training and concentrates learning on the under-modeled tail of the data.
That is the only modification they implement.
Results:
On OLMo-3 7B, pass@16 improves by up to 16.8 points absolute on coding and 3.1 on math. Those higher-coverage checkpoints then lift final pass@1 after GRPO by up to 3.9 points, and in some settings early reward climbs 2.5x faster than the matched standard SFT run.
Paper: https://t.co/QocPRtNhjH
Chat with Paper: https://t.co/wfTytUm5jp