Science Fiction: This last week, I think humanity has reached what I dreamed about when I watched IBM's Deep Blue beat Garry Kasparov in 1997.
The more I learn about recent results in mathematics and theoretical computer science, the more I realize we're in superhuman territory.
Two years ago, trekking to Everest Base Camp reminded me of the Romanian countryside of my childhood. 40–80 years ago, my parents and grandparents walked to neighboring villages to buy something, close a deal, or deliver a message. Many households had no car or telephone. Some had horses. Getting anywhere on foot requires enormous effort. In many ways, that's where we've been intellectually.
Hans Moravec observed that we evolved to survive, not to do mathematics. Our working memory holds only a handful of things. Learning enough to contribute at the frontier takes years, and making progress requires enormous intellect, effort, and specialization. We've been scratching at open problems with primitive tools and goldfish-sized working memories.
Now we have machines designed to reason about mathematics, physics, and code. Systems that can run many reasoning agents in parallel and draw on the whole of human knowledge. Scientific progress has been constrained by the limits of human thought.
We're going to discover what's possible when intelligence is no longer human limited. This is the science fiction I dreamed about as a middle school student in 1997.
I'm mostly writing this as a note to myself in the future, to remember how I felt this last week.
@kellerjordan0 Can insiders really know that "for a fact"? Pre-training is automated pipelines; no one sees every data point, at best the distribution. And if user prompts are in the mix, users attempting novel problems with LLMs may already have put the "untrained" task in the data.
@karthikponna19 Gpt-oss 20gb hosted on 16gb laptop in lmstudio. I use it on non WiFi flights and places and always surprised by the quality of responses.
@shreyas The correct answer is clarifying the main goal.
If we understand the main goal- It keeps you honest and gives you less opportunities to distract the mind with irrelevant details/processes. This is the hallmark for building great taste.
@soiled_plants2 Water and fasting. Typically, after 12 hrs of only-water-fasting keto cycle kicks in where you get lot of mental energy and focus. Also saves time in the logistics of eating
@rasbt@karpathy Not sure - the final llm in production is also trained via RLHF that is likely not arxiv. The best format as you said would be - how it was trained on. So I guess different for different llm. Only the llm labs know this answer!
@karpathy@karpathy "Move 37" is arrived when "relatively" different minded super-users (artists, scientists, philosophers, politicians) can appreciate responses given by the LLM.
It is hard for one person or a set of people to judge if the LLM is executing move 37.
@vhmth If you already don’t know what to do, then I will suggest why not mentor the child in you. Instead of you founding something why not join the board of existing startups and help the next Vinay. Join an accelerator program. Mentor a college startup. Build with the next generation.
@Uber_Support : Driver with license no. KA03AB3048 , Name: Yogananda
has stolen my phone. Please help. When I connect with the driver he is not picking up my phone. Please help- the phone has last images of my dad.
@makemytripcare I have also spoken to the airlines(cathay pacific). They also insist, make my trip is responsible for this, given the attached information presented to me.
@makemytripcare : I have booked a ticket (NN2302817813895) on your website. While booking the ticket, it showed date changes are allowed. Now, I want to change the dates of the ticket to 11th Feb 2018 from 29th Jan 2018. Your cc says now,date changes are not possible.