A lot of the real world use cases that stack judgement + decisions all come down to perceiving risk, modeling hidden variables, and taking action under uncertainty.
For this reason, good ol fashioned games serve as much better benchmarks for this than the core “standardized testing-esque” benchmarks we see accompanying every model release
We found one game in particular to be a really good benchmark for real-world performance: poker!
Turns out the learnings from watching LLMs think and act during poker aligns really well with the anecdotal reports on real-world performance of the models.
There are many great builders exploring RL and benchmarking on games, and yet the this niche is still underrated and deserves more public interest.
If you’re curious to learn more, check out https://t.co/3dcPtnw31h to see what we learned about models’ psychologies through hundreds of games of poker
It's time! We'll be live-tweeting for the next three days from our annual conference on #personalizedlearning for the whole learner, #EdSurgeFusion! Whether you're at the event or not, we hope you get something out of following along. https://t.co/R3WCUp0WNL
The EdSurge fall jobs fairs are nearly here! This October we'll be in SF, Boston and NYC with 50+ #edtech companies who are looking to hire talented folks like you at the intersection of education and technology. RSVP to save your spot now: https://t.co/yHEjm0hJ1F
I continue to tell graduates they should not be influenced by the negativity and lack of vision around them. We are living in extraordinary, inspiring times that bring us closer to our potential as humans. https://t.co/K9d8K5A3YX
We've heard teachers say, “Oh, I’m just a teacher” a million times, and we tend to assume we know what that means. But what does it really look like to be #justateacher?
For #TeacherAppreciationDay, share your stories of the impact teachers have today. https://t.co/nheXTTx23d
Well it took over 2 years but my first EdSurge article is live in collaboration with @sambeckertweets! - When the Stars Align: A New Constellation of Innovation https://t.co/NOAosIWPir #edtech via @HigherEdSurge
$137.5M Purchase Price and $1M in 2017 revenue? What are we missing here? China’s NetDragon to Acquire Edmodo for $137.5 Million https://t.co/118CSO827x #edtech via @EdSurge
Some powerful data and storytelling about disappointing trends in teacher pay by my colleague @JennyAbamu - The Data Tells All: Teacher Salaries Have Been Declining For Years https://t.co/ABT9Vs0eTj #edtech via @EdSurge
Looking for a new job in #edtech? Curious about what jobs are out there? Come by one of @EdSurge's spring jobs fairs: SF (4/10), Boston (4/18), DC (4/19), and NYC (4/24)! https://t.co/ux4GGZCnDj
Important piece by my friend and former colleauge @MJMadda - Confronting the Realities of Sexual Harassment in Education and Edtech https://t.co/Xtsca1A6LR #edtech via @EdSurge