In 2021, a landmark study found that the Amazon rainforest is now a net emitter of greenhouse gasses. We have passed a tipping point.
#SocialismorExtinction
I'm calling the Myth of Context Length:
Don't get too excited by claims of 1M or even 1B context tokens. You know what, LSTMs already achieve infinite context length 25 yrs ago!
What truly matters is how well the model actually uses the context. It's easy to make seemingly wild claims, but much harder to solve real problems better.
I highly recommend "Lost in the Middle: How Language Models Use Long Contexts" from Stanford. It's jam-packed with rigorous experiments that put popular long-context models to test. Key findings:
▸ Figure 1 (left): models are good at using information located at the very beginning or end of its context, but significantly worse at the middle. This isn't unique to GPT architecture - encoder-decoder like Flan-T5 also suffers in the middle
▸ Figure 2 (right): models that are natively longer context do NOT actually use the context better. You can see that the curves of GPT-3.5 (4k vs 16k) almost completely overlap.
▸ Model performance substantially decreases as input contexts grow longer, regardless of their native length.
We don't need more tokens. We need models that actually pay attention to them (pun intended).
Is there anything followers would like to see us cover around the environmental crisis? The script for For Land is pretty set in stone now, but we will be continuing to produce shorts around the subject for a while yet, and we can be quite flexible on the content.
Drinking water from nearly half of U.S. faucets likely contains “forever chemicals” that may cause cancer and other health problems, a new government study says.
https://t.co/C5GTOsaBiO
My immunocompromised patients have to get into a packed hospital elevator to get to the clinic 13 floors up. Half the people in there are unmasked these days, and most of them are staff/employees btw, not patients. But tell me again how IPAC is protecting high risk patients.
Adding to this, there are a couple of reasons we're seeing temps spike higher outside of just carbon emissions:
- Potential El Nino weather pattern return: https://t.co/zTfCxZLWlz
- Termination shock from sulphur emission reductions: https://t.co/t4TmBA07UU
This June was the hottest June on record. The 3rd and 4th of July this year were the two hottest days on the planet since records began.
We are already in a new climate.
#SocialismorExtinction
Good thing the US military rescinded the COVID vaccination requirement but kept the requirements for polio, chickenpox and flu vaccinations
https://t.co/t9dP3KpR7v
"'... they may have had half a dozen prior infections that they did a nasal swab at home and were never reported,'"
A half-dozen prior infections. That's the quiet part out loud
https://t.co/7D2XPTC8nF
In other words, I am planning on having a virtually "normal" trip by identifying where the risk is, and taking very simple mitigating steps where needed.
That said, I will mask in grocery stores, drug stores, and anywhere else that vulnerable people need to visit too.
6/7
Detalles: El Senado convoca sesión extraordinaria mañana, viernes, para atender, entre otros, el nombramiento de Ángel Toledo como secretario de Educación. https://t.co/CmeYRW4RH4
⚡️Ukrainian Ministry of Health has published instructions on how to act in the event of a “terrorist attack at the ZNPP” for those who will be in the zone of a radiation accident, if the settlement is not subject to evacuation.
Ah, look, it's Twitter Community Notes saying USA military bases in Puerto Rico "do not count" because it's a "territory" (= colony) and the USA has "total sovereign control over the island" (and asserts the privilege to do whatever they want to us whether we like it or not).