Dickson Xavier

[Jaynit @jaynitx]

Elon Musk reveals why, measured against the sun's energy, human civilization barely registers as existing at all:

"Right now we're very low on the Kardashev one scale. If you say what proportion of our planet's power are we harnessing, it's a very, very tiny number. And we're harnessing almost nothing of our star's power. The sun is truly an immense thing. It's difficult with words to characterize just how immense the sun is"

"The sun is about 99.86% of all mass in the solar system. Most of the remaining 0.14% is Jupiter. The entire mass of Earth is in the tiny miscellaneous category. Earth is a tiny dust mote compared to the sun"

"We currently use much less than a trillionth of the power output of the sun, and a trillion is a million times a million. So basically we're practically nowhere on the Kardashev two scale. On the Kardashev scale, we're still non-existent. We're not registering"

[Jikku Jose @jikkujose]

We have genuinely crossed character & temporal consistency challenges!

It’s only skill issue that we don’t have a proper movie built using AI.

[Dickson Xavier @Dxnxj]

Inference closure: deployed. Representation closure: still research. Is the blocker continual learning without forgetting, or the world model?

[Andrej Karpathy @karpathy]

I'm noticing that due to (I think?) a lot of benchmarkmaxxing on long horizon tasks, LLMs are becoming a little too agentic by default, a little beyond my average use case.

For example in coding, the models now tend to reason for a fairly long time, they have an inclination to start listing and grepping files all across the entire repo, they do repeated web searchers, they over-analyze and over-think little rare edge cases even in code that is knowingly incomplete and under active development, and often come back ~minutes later even for simple queries.

This might make sense for long-running tasks but it's less of a good fit for more "in the loop" iterated development that I still do a lot of, or if I'm just looking for a quick spot check before running a script, just in case I got some indexing wrong or made some dumb error. So I find myself quite often stopping the LLMs with variations of "Stop, you're way overthinking this. Look at only this single file. Do not use any tools. Do not over-engineer", etc.

Basically as the default starts to slowly creep into the "ultrathink" super agentic mode, I feel a need for the reverse, and more generally good ways to indicate or communicate intent / stakes, from "just have a quick look" all the way to "go off for 30 minutes, come back when absolutely certain".

[Dickson Xavier @Dxnxj]

@karpathy May be llm should have a provision to generate tools or skill of its own and make it at its disposal. Evaluate their skills and improve themselves by rewriting the skill codes... May be we should look on llm writing a narrative or itself and embedded beliefs through rewards.

[Andrej Karpathy @karpathy]

We're missing (at least one) major paradigm for LLM learning. Not sure what to call it, possibly it has a name - system prompt learning?

Pretraining is for knowledge.
Finetuning (SL/RL) is for habitual behavior.

Both of these involve a change in parameters but a lot of human learning feels more like a change in system prompt. You encounter a problem, figure something out, then "remember" something in fairly explicit terms for the next time. E.g. "It seems when I encounter this and that kind of a problem, I should try this and that kind of an approach/solution". It feels more like taking notes for yourself, i.e. something like the "Memory" feature but not to store per-user random facts, but general/global problem solving knowledge and strategies. LLMs are quite literally like the guy in Memento, except we haven't given them their scratchpad yet. Note that this paradigm is also significantly more powerful and data efficient because a knowledge-guided "review" stage is a significantly higher dimensional feedback channel than a reward scaler.

I was prompted to jot down this shower of thoughts after reading through Claude's system prompt, which currently seems to be around 17,000 words, specifying not just basic behavior style/preferences (e.g. refuse various requests related to song lyrics) but also a large amount of general problem solving strategies, e.g.:

"If Claude is asked to count words, letters, and characters, it thinks step by step before answering the person. It explicitly counts the words, letters, or characters by assigning a number to each. It only answers the person once it has performed this explicit counting step."

This is to help Claude solve 'r' in strawberry etc. Imo this is not the kind of problem solving knowledge that should be baked into weights via Reinforcement Learning, or least not immediately/exclusively. And it certainly shouldn't come from human engineers writing system prompts by hand. It should come from System Prompt learning, which resembles RL in the setup, with the exception of the learning algorithm (edits vs gradient descent). A large section of the LLM system prompt could be written via system prompt learning, it would look a bit like the LLM writing a book for itself on how to solve problems. If this works it would be a new/powerful learning paradigm. With a lot of details left to figure out (how do the edits work? can/should you learn the edit system? how do you gradually move knowledge from the explicit system text to habitual weights, as humans seem to do? etc.).

[Dickson Xavier @Dxnxj]

Make chatGPT troll itself.
'Troll yourself creating Ghibli images for everyone'

[Dickson Xavier @Dxnxj]

@AnthropicAI Like a connectome project for llm

[Anthropic @AnthropicAI]

New Anthropic research: Tracing the thoughts of a large language model.

We built a "microscope" to inspect what happens inside AI models and use it to understand Claude’s (often complex and surprising) internal mechanisms.

[Anthropic @AnthropicAI]

We’re launching a new blog: Engineering at Anthropic.

A hub where developers can find practical advice and our latest discoveries on how to get the most from Claude.

[Andrej Karpathy @karpathy]

"Move 37" is the word-of-day - it's when an AI, trained via the trial-and-error process of reinforcement learning, discovers actions that are new, surprising, and secretly brilliant even to expert humans. It is a magical, just slightly unnerving, emergent phenomenon only achievable by large-scale reinforcement learning. You can't get there by expert imitation. It's when AlphaGo played move 37 in Game 2 against Lee Sedol, a weird move that was estimated to only have 1 in 10,000 chance to be played by a human, but one that was creative and brilliant in retrospect, leading to a win in that game.

We've seen Move 37 in a closed, game-like environment like Go, but with the latest crop of "thinking" LLM models (e.g. OpenAI-o1, DeepSeek-R1, Gemini 2.0 Flash Thinking), we are seeing the first very early glimmers of things like it in open world domains. The models discover, in the process of trying to solve many diverse math/code/etc. problems, strategies that resemble the internal monologue of humans, which are very hard (/impossible) to directly program into the models. I call these "cognitive strategies" - things like approaching a problem from different angles, trying out different ideas, finding analogies, backtracking, re-examining, etc. Weird as it sounds, it's plausible that LLMs can discover better ways of thinking, of solving problems, of connecting ideas across disciplines, and do so in a way we will find surprising, puzzling, but creative and brilliant in retrospect. It could get plenty weirder too - it's plausible (even likely, if it's done well) that the optimization invents its own language that is inscrutable to us, but that is more efficient or effective at problem solving. The weirdness of reinforcement learning is in principle unbounded.

I don't think we've seen equivalents of Move 37 yet. I don't know what it will look like. I think we're still quite early and that there is a lot of work ahead, both engineering and research. But the technology feels on track to find them.

https://t.co/JCxTdKpuzv

[A Man Of Memes @RickyDoggin]

Kid Rock

[clem 🤗 @ClementDelangue]

Our science team has started working on fully reproducing and open-sourcing R1 including training data, training scripts,...

Full power of open source AI so that everyone all over the world can take advantage of AI progress! Will help debunk some myths I’m sure too.

Thanks @deepseek_ai!

[OpenAI @OpenAI]

Today we’re rolling out a beta version of tasks—a new way to ask ChatGPT to do things for you at a future time.

Whether it's one-time reminders or recurring actions, tell ChatGPT what you need and when, and it will automatically take care of it.

[Andrej Karpathy @karpathy]

I still do this most days and I think it works great. My morning brain (right after 1hr exercise and 1 coffee) is quite eager to work and I go directly to the one top priority item. The energy decreases over time and with every distracting item loaded into the context window.

[Andrej Karpathy @karpathy]

Of ~200 books I've read, the few that stayed with me over time and I find myself often thinking back to or referring to, in ~random order:

All short stories by Ted Chiang, especially Exhalation, Division By Zero, Understand, The Story of Your Life, Liking What You See, The Lifecycle of Software Objects, What's Expected of us, just excellent themes ideas and reading all around.

The Selfish Gene (nonfiction) - a classic for understanding evolution and natural selection, especially the realization that the gene is closer to the real unit of selection more than an individual, explaining altruism and colonies and a lot more.

The Lord of the Rings (fantasy) - I return to LoTR all the time for comfort. I don't think anyone else has created a high fantasy Universe this complex, with so much mythology, symbolism, new languages, mysterious system of magic, ancient and powerful beings and artifacts, beautiful writing and dialog, themes of courage, friendship and heroism, the list goes on and on... You're thrown into a world with characters and references to so many things that are part of this ancient world and never really introduced. There's always more to find on each reading.

The Martian (~scifi) - top tier science porn, competence porn, fast paced and fun.

The Vital Question (nonfiction) - First time I intuitively grokked the bridge from geology to biology, the origin of life, and likelihood of life in the Universe at large at various stages of complexity and development. Also all other Nick Lane books.

How To Live by Derek Sivers (nonfiction) - 27 conflicting answers to how to live life. Emphasizing the diversity of consistent and possible answers to the meaning and goals of life.

1984 (nonfiction) - Classic. Newspeak, Ministry of Truth, Doublethink, Thoughtcrime, Facecrime, Unperson, the list just keeps on going. Chilling world-building and the realization that weaker equivalents of everything exist.

In Defense of Food by Pollan (nonfiction/food) - Eat food. Not too much. Mostly plants. The book that first taught me to avoid the entire center of every grocery store and only shop on the outer ring. The realization that the food industry is out of control and the things they do with your food, what they put into it, what they are allowed to do, and how they are allowed to market it to you is quite a lot worse than I thought.

The Accidental Superpower by Zeihan (nonfiction/geopolitcs) - I've found Zeihan to be a bit of a mixed bag over time but I still remember his books (esp this one) to be elucidating on geopolitics.

Countdown to Zero Day (nonfiction/cyberwarfare) - Goes into detail on Stuxnet, imo very important and highly elucidating reading on cybersecurity, the future of warfare, and AGI.

A Fire Upon the Deep (scifi) - Chapter one only, incredible portrayal of what superintelligence will be like that has stayed with me since.

Guns Germs and Steel (nonfiction/history) - I'd probably recommend a summary of this book more than the book itself. I remember it being very dry, but it was very interesting because it is a comprehensive analysis of the resources grid (food, animals, freshwater, climate, ...) in our real-world game of Civilization, and the implications there of.

Flowers of Algernon (scifi) - Just a totally crushing masterpiece on intelligence.

Atlas Shrugged (scifi) - No one finishes this I think but the first few chapters and its worldbuilding are enough and, once seen in an exaggerated form in fiction, elements of it cannot be fully unseen in reality.

An Immense World (nonfiction/bio, by Yong, among others of his) - Nice book on so many different sensors used by various animals, you repeatedly realize human senses are super inadequate and that we only measure such a tiny sliver of reality.

The Master Switch (nonfiction/tech history, by Wu) - history of information technologies telegraph, telephony, radio, television, film, cable television, internet and the pattern of "The Cycle", where each medium starts decentralized, open and idealistic and then progresses towards centralization, control and oligopoly, for the very similar reasons, by very similar means, and usually at the expense of diversity, innovation and technological progress. Quite a few connections to draw on for LLMs, which are after all an information technology too.

(I take recommendations for more that are likely to make this list!)

[Carlos E. Perez @IntuitMachine]

Other Nobel prize awards for AI researchers as predicted by GPT-4o, o1 and Claude 3.5-sonnet.

[Dickson Xavier @Dxnxj]

@karpathy @theworldlabs Thank you

[Dickson Xavier @Dxnxj]

@imPenny2x @lexfridman @karpathy That is going to be luciferian.. may be fascism. collective good and evil may not be in alignment with each person's amygdala response and existential struggles . Is that going to open another Pandora box? Creating more conflicts.

[Phil Scarborough @phsjr1]

He is the one.