Learn to build evals, folks.
Highly recommended reading on using decision models for evals.
From our internal evals, System One models like Jev are a great fit for LLM Judges.
If you want to go beyond Hamel's great write-up, I've also put together a few resources on applying decision models to LLM judges.
(bookmark it)
1) Jev-as-a-Judge for Agent Evaluations - https://t.co/SkXGd4uA2J
2) Jev-as-a-Judge: Evaluating Agents with a Decision Model - https://t.co/SqGSzlMFut
Bonus paper I would recommend, with a few other ideas that combine System One and System Two models: Jev-as-a-Judge paper - https://t.co/8Oi6oBEt2A
This semester I am teaching a new course on Agentic Systems, which explores the full stack of AI systems: from building and optimizing agent harnesses to serving and optimizing LLM serving infrastructure.
Students learn by building, measuring, and optimizing real agents and systems, balancing capability, latency, and cost.
Our goal is not just to teach today's AI technologies, but to prepare students to build tomorrow's.
All lectures, readings, and assignments are publicly available at https://t.co/z69r5AwsKX
reading feynman lecturess as feynman intended: with accompanying ai-made demo sections for each chapter
rapid acquisition of knowledge is possible thanks to superintelligence
a full humanoid robot used to cost more than a car. berkeley just made one for under $5k.
berkeley humanoid lite changes everything:
> modular, 3d-printed gearboxes
> off-the-shelf hobby parts
> cad, firmware, and the full rl training stack included
a five-figure parts order becomes a weekend of printing.
that’s the difference between reading about humanoid robots and having one on your bench.
100% open-source.
Stanford dropped CS329Z today. Engineering AI Agents. compound systems, build RAG/tool use from scratch, then frameworks, then how you actually evaluate this stuff.
gonna follow along online. looks pretty neat.
https://t.co/7To6HNMqxu
We're flying students to the biggest blockchain student conference on Earth.
We partner with @UBC_Conference to fly LATAM students to the conference.
1,000+ students from 100+ universities will be there.
In two years, UBC has sent 80+ students into frontier tech, with teams joining Y Combinator, a16z Speedrun, and Colosseum.
This opportunity could change your life, apply now.
https://t.co/LWHiueNGVN
My Stanford course 𝗧𝗵𝗲 𝗠𝗼𝗱𝗲𝗿𝗻 𝗦𝗼𝗳𝘁𝘄𝗮𝗿𝗲 𝗗𝗲𝘃𝗲𝗹𝗼𝗽𝗲𝗿 starts next Tuesday.
This Github repository will hold all the assignments (and has the ones from last year).
Bookmark it. See you next week.
https://t.co/b6o254kwon
The robotics company’s prisoner’s dilemma: don’t use GPT Astra and your competitors will show better demos and capabilities. Use it, and your data & moat will be absorbed by the next version of the LLM.
We are hiring at Supabase!
There are 60 open positions across engineering, sales, design, and many more, so there should be something that would fit you!
We are a remote company, so where you live doesn't matter. If you are awesome at what you do, we'd love to work with you!