"One of the very confusing things about the models right now: how to reconcile the fact that they are doing so well on evals.
And you look at the evals and you go, 'Those are pretty hard evals.'
But the economic impact seems to be dramatically behind.
There is [a possible] explanation. Back when people were doing pre-training, the question of what data to train on was answered, because that answer was everything. So you don't have to think if it's going to be this data or that data.
When people do RL training, they say, 'Okay, we want to have this kind of RL training for this thing and that kind of RL training for that thing.'
You say, 'Hey, I would love our model to do really well when we release it. I want the evals to look great. What would be RL training that could help on this task?'
If you combine this with generalization of the models actually being inadequate, that has the potential to explain a lot of what we are seeing, this disconnect between eval performance and actual real-world performance"
@mikeeisenberg@Apple@eden I remember when @eden showed me his new android device back in 2009 when we were both teaching at IDC. I am still bullish on Apple after all these years. Patience and wisdom. I hope they won’t disappoint.
Like many nonprofits, some of the major human rights organizations (Amnesty, Human Rights Watch) have been taken over by left-wing activists and lost sight of their missions | How Doctors Without Borders Became a Political Actor in Gaza https://t.co/jotRpzoCAV
Taiwan should be independent
Somaliland should be independent.
South Yemen should be independent.
Northern Cyprus is occupied.
Tibet is occupied.
There’s a genocide in Xinjiang.
If any of this angers you, and you also agitate for a Palestinian state, you are transparent.
@chelseakomlo@benediktbuenz Chelsea congrats! This is a game changer; reducing bandwidth and computation overhead while keeping it verifiable is impressive.