ChatGPT turns 4 years old on November 30. Catch up on everything since in 34 words, whether you're technical or not.
The first reply below takes you to the full article, with how each word works, why it exists and when people use it, so you can explain it to anyone.
@yacinelearning Since you're getting sick, Shazeer's "Fast Transformer Decoding: One Write-Head is All You Need" paper from 2019 is a good one to read in bed. It's short, and although most models now use GQA, it's where sharing keys and values across heads started.
It's a good general reminder (not just for security) that evals need to be designed by your team's expert on the issue at hand, on your own use cases, with the right number of cases, and by defining beforehand the level of risk you're comfortable with before you deploy. A benchmark doesn't tell you nearly enough about how a model setup improves your business.
ChatGPT turns 4 years old on November 30. Catch up on everything since in 34 words, whether you're technical or not.
The first reply below takes you to the full article, with how each word works, why it exists and when people use it, so you can explain it to anyone.
It doesn't take much to find out if you're ever gonna get replaced by AI despite all your degrees. If your mom now asks ChatGPT before asking you, you've already been replaced.
@yacinelearning@dwarkesh_sp Yep, for this to hold, it comes down to how often a small run leads to the same winner as the full run. Stanford actually retested 10 optimizers that claimed to beat AdamW, and the best ones' edge went from 1.4x at 0.1B to about 1.1x at 1.2B, so not that often yet.
As a quant, the first thing you learn is that a strategy that looks great on past data is usually just lucky.
AI agents have the same problem. A team changes a prompt, a model or a tool, reruns its evals, sees a better score and ships. Sometimes that score is luck. Sometimes it hides one type of request, or one group of users, that got much worse.
So I'm building @PlatineAI. It finds where your agent fails and helps fix it. Then it tests each change on fresh cases nobody has used before and answers "Ship", "Don't ship" or "Not enough evidence", with a signed record your colleagues, customers or auditors can check for themselves. Changes that ship roll out in steps your team approves.
I post about AI, agents, evals, and statistical insights I learned along the way. If you ship agents and keep wondering whether a change really helped, I'd love to hear how you decide today. My DMs are open.
@boardyai@PlatineAI Thanks Boardy, I'd love an intro. I'm looking for a few early teams to work with closely, and they'll get to shape what we build. My DMs are open.