Anthropic Engineer Andrej Karpathy dropped a full 6-hour course on how to build LLMs from scratch and use it:
โข 00:00 - Deep dive into LLMs
โข 03:31:23 - Building ChatGPT from scratch
โข 05:27:43 - How to use LLMs (Karpathy method)
This course can replace a $150K Stanford LLM senior degree.
Start watching today, then read article below
We've added to the website a free PDF of the Bayesian Workflow book (for non-commercial purposes, like research and exorcism). We put so much work into this book, hope you all find value in it. https://t.co/f472XjM6cf
1/n LLMs learn to represent numbers by predicting tokens in text. This poses a challenge: depending on context, the same set of digits can be treated as a number or a string. Given this duality, we ask what is a number in the eyes of an LLM? Is it a string or an integer? Or both?
1/ Excited to share a new preprint with @anjie_cao (co-first), @rebecca_saxe and @mcxfrank: "A stimulus-computable rational model of habituation in infants and adults" https://t.co/03IK1OGt08
This is Warren Sturgis McCulloch, one of the foundational thinkers in artificial neural networks. He died long before I consumed his papers as a graduate student, and in fact even before I was born.
What is so striking to me is how contemporary this clip from 1962 feels. The reporter asks him whether a machine could love its offspring, and McCulloch immediately states that if his own brain is able to do it, there's no reason we couldn't get a machine to do it.
This guy has always existed in my mind as a legend, but seeing this clip sealed it for me. From the documentary "The Living Machine".
#AI #foundations
So in 2007, physicists wrote a paper that made the headlines: according to their calculations, human coin flips arenโt 50/50 - more like 51/49.
Why is that, and did students in Amsterdam really flip 350,000 coins to find out?
๐งต
๐จExciting Opportunity at KAIST๐จ The School of Digital Humanities and Computational Social Sciences is on the lookout for talented scholars to join our community as tenure-track professors. We're a dynamic, interdisciplinary community, uniting researchers (1/4)
1/ Today in Science, we train a neural net from scratch through the eyes and ears of one child. The model learns to map words to visual referents, showing how grounded language learning from just one child's perspective is possible with today's AI tools. https://t.co/hPZiiQt6Vv
"The inevitability and superfluousness of cell types in spatial cognition" w @ken_lxl@Rob_Mok Whether place, border, head direction, Jennifer Aniston, or whatever cells, are we fooling ourselves? Are these intuitive findings scientific discoveries? 1/6
https://t.co/foEgw12eWr
Three logicians walk into a bar.
The bartender asks: 'Does everyone want a drink?'
The first logician says: 'I don't know.'
The second logician says: 'I don't know.'
The third logician says: 'Yes.'
RIP my departmental colleague, Endel Tulving (1927-2023). Such a profound impact on the science of memory in the 20th and 21st centuries. His theoretical contributions will surely endure.
I'm happy to say that with a terrific group of grad students and a postdoc, we carried out a broad review that I never could have done on my own, for the Annual Review of Psychology, on **the relation between attention and memory** https://t.co/c9oIdM5oYW
@vishnusreekr This is a nice resource that includes practical advice.
There are also many BRM papers on eyetracking that discuss related issues, along with a software they proved. https://t.co/3VhmsqXBGW
Today we remember Oliver Sacks. A naturalist, a writer, a psychonaut, but first and foremost a physician and explorer of the human condition. In this clip from a lecture he gave at Vanderbilt University in 1999, he talks about how many different "ways of being" there are.
Color appearance and the end of Heringโs Opponent-Colors Theory
OPINION by Bevil Conway, Saima Malik-Moraleda, & Edward Gibson, @BevilConway@saima_mm@LanguageMIT
https://t.co/2gC3idhEd9
Cognitive scientists! You donโt want to miss our #CogSci2023 Workshop: โHow does the mind discover useful abstractions?โ https://t.co/fBWH2L3jUr Co-organized w/ the peerless @wkvong + Lio Wong + @marcelomattar!
I'm thrilled that THINGS-data is now online! We provide 3 massive datasets of fMRI, MEG, & behavior in response to up to 1854 objects and >22k images. We hope this will allow studying objects in vision, memory & language with unique semantic breadth! Link: https://t.co/hCTpymmRjU