recently, i’ve been thinking about ways to design ai systems to be more compatible with slow thinking 🐌.
you can check out the full blogpost here 🤗:
https://t.co/3hdYCIpuoN
What? Pre-training? No, no, no, no. No pre-training. Why would you do pre-training?!
If you do pre-training, people will ask "HOW MANY TOKENS?" And it will never be enough. The model that was the frontier breakthrough becomes "just distillation from chatgpt"
But if you just do SFT, some RL on benchmarks? You're efficient. You're doing *reasoning*.
It's not about capabilities, it's about the benchmark score. And who tops the leaderboard? Labs doing RLFT on a Chinese frontier base.
The United Nations in #Myanmar is calling for an immediate de-escalation of the situation in Sanchaung, Yangon, and to allow protestors prevented from returning home by security forces to so do safely.
Some reflections on #myanmar op-eds by Myanmar outsiders these days. A thread:
I understand why Myanmar folks, some of whom are in exile, might write op-eds that sound resigned in regard to the viability of the #myanmar protest movement. They have their reasons.
What if I flip a coin 1e6 times, throw out half the heads, train an unbiased estimator on the remaining data, and then use it to decide who has to stay in jail pretrial?
Every day some new xxx-BERT gets released, I have nearly lost track of their names. If u told me u r reading DOSA-BERT or IDLY-BERT, there is a 99.9% chance that I would believe you. #NLProc#BERT#deeplearning
Update: A near-total internet shutdown is now in effect in #Myanmar.
Network data show a collapse of connectivity to 16% of ordinary levels from ~2 pm local time 📉
The information blackout is likely to severely limit coverage of anti-coup protests 📵
📉https://t.co/Jgc20OBk27
Myanmar leaders including Aung San Suu Kyi and the president have been detained in an early-morning raid, according to a report, amid fears of a coup https://t.co/Z1myZJi5aK
Breaking News: Myanmar’s leader, Daw Aung San Suu Kyi, was detained in an early-morning raid, the government said, fueling concern of a possible coup. https://t.co/vs2CW9p40Y