Following the data behind AI so you can debunk big tech.
Computer Engineering @ Brown | The AI Safety Guy
Building a universal guardrail for your agents๐
Twelve correct AI answers became wrong because the answer key changed.
A simulated shopper wanted the cheapest suitable earbuds already in their order.
Twelve publicly saved AI runs chose the correct item: $232.49. But a revised benchmark answer key expected a $226.49 item that wasnโt in the order.
When we replayed those saved runs against that key, all twelve failed. We corrected one reference value and replayed the same runs. All twelve passed.
The AI responses hadnโt changed. The grading reference had.
This is a concrete example of how an answer-key mistake can make correct AI behavior look like failure.
This is just a preview of what we are building at https://t.co/X9tJL4Vxyv. Tap in if you want to learn more about AI safety ๐
@BullTheoryio When people borrow money to spend more money with little real avenue for paying off their debt, that is a bubble. I REPEAT THAT IS A BUBBLE