I've made that point before:
- LLM: 1E13 tokens x 0.75 word/token x 2 bytes/token = 1E13 bytes.
- 4 year old child: 16k wake hours x 3600 s/hour x 1E6 optical nerve fibers x 2 eyes x 10 bytes/s = 1E15 bytes.
In 4 years, a child has seen 50 times more data than the biggest LLMs.
1E13 tokens is pretty much all the quality text publicly available on the Internet. It would take 170k years for a human to read (8 h/day, 250 word/minute).
Text is simply too low bandwidth and too scarce a modality to learn how the world works.
Video is more redundant, but redundancy is precisely what you need for Self-Supervised Learning to work well.
Incidentally, 16k hours of video is about 30 minutes of YouTube uploads.
@Vertex and @CRISPRTX Therapeutics Announce US FDA Approval of CASGEVY™ (exagamglogene autotemcel) for the Treatment of Sickle Cell Disease
– First-ever approval of a #CRISPR-based gene-editing therapy in the U.S., 16,000 with severe #SickleCellDisease eligible #futureofhealth
#Blogalert: #Parkinson's disease can be predicted by eye scans - Dr Antonio Giordano
It is possible to identify invisible irregularities appearing up to 7 years before the clinical manifestation of Parkinson's disease by scanning the retina
Read more: https://t.co/YsZ7CHwqIE
biotech company @BlueRockTx Therapeutics says implants of lab-made neurons introduced into the brains of 12 people with #parkinson safe and may have reduced symptoms #future
We’re excited to launch 𝗦𝘆𝗻𝘁𝗵𝗜𝗗 today with @GoogleCloud: a digital tool to watermark and identify AI-generated images. 🖼️
It will be available on Imagen, one of @Google’s latest text-to-image models.
Here’s how it works. 🧵 https://t.co/p8s5XK5Ekr #GoogleCloudNext
The University of Toronto made a video for a non technical audience in which I explain how deep learning works, the enormous promise of this technology and some of the potential risks. https://t.co/d0Izl3RYbS
I'm waking up to the prospect that in my prime years, I'll see both mainstream superconducting and AGI. The former will propel the latter, and the latter will propel every scientific breakthrough.
These should've stayed in sci-fi for another 20 yrs. But somehow, they are eerily within reach.
And we as a species will soon take these for granted and move on to the next thing that should've stayed in sci-fi.
do not microwave in #plastic containers - 75 percent of cultured #kidneys cells died after being exposed for 48 hours to the particulates from the microwaved baby food containers; a new study out of the University of Nebraska, Kazi Albab Hussain https://t.co/VTmJK71UI8
July 20, 1969 is a date that represents a pinnacle of human achievement - the #Apollo11 Moon landing. And while it was certainly special, it was only part of a long and challenging effort that involved thousands of people working hard and doing their part, resulting in the subsequent Apollo missions taking Americans to the Moon and returning them safely to Earth. It truly was America at its best.