Prashant Chamarty

Passionate about enabling better decisions using #AI & #ML @Alteryx. Interested in #Blockchain #Crypto #PublicPolicy

[Dwarkesh Patel @dwarkesh_sp]

The most interesting part for me is where @karpathy describes why LLMs aren't able to learn like humans.

As you would expect, he comes up with a wonderfully evocative phrase to describe RL: “sucking supervision bits through a straw.”

A single end reward gets broadcast across every token in a successful trajectory, upweighting even wrong or irrelevant turns that lead to the right answer.

> “Humans don't use reinforcement learning, as I've said before. I think they do something different. Reinforcement learning is a lot worse than the average person thinks. Reinforcement learning is terrible. It just so happens that everything that we had before is much worse.”

So what do humans do instead?

> “The book I’m reading is a set of prompts for me to do synthetic data generation. It's by manipulating that information that you actually gain that knowledge. We have no equivalent of that with LLMs; they don't really do that.”

> “I'd love to see during pretraining some kind of a stage where the model thinks through the material and tries to reconcile it with what it already knows. There's no equivalent of any of this. This is all research.”

Why can’t we just add this training to LLMs today?

> “There are very subtle, hard to understand reasons why it's not trivial. If I just give synthetic generation of the model thinking about a book, you look at it and you're like, 'This looks great. Why can't I train on it?' You could try, but the model will actually get much worse if you continue trying.”

> “Say we have a chapter of a book and I ask an LLM to think about it. It will give you something that looks very reasonable. But if I ask it 10 times, you'll notice that all of them are the same.”

> “You're not getting the richness and the diversity and the entropy from these models as you would get from humans. How do you get synthetic data generation to work despite the collapse and while maintaining the entropy? It is a research problem.”

How do humans get around model collapse?

> “These analogies are surprisingly good. Humans collapse during the course of their lives. Children haven't overfit yet. They will say stuff that will shock you. Because they're not yet collapsed. But we [adults] are collapsed. We end up revisiting the same thoughts, we end up saying more and more of the same stuff, the learning rates go down, the collapse continues to get worse, and then everything deteriorates.”

In fact, there’s an interesting paper arguing that dreaming evolved to assist generalization, and resist overfitting to daily learning - look up The Overfitted Brain by @erikphoel.

I asked Karpathy: Isn’t it interesting that humans learn best at a part of their lives (childhood) whose actual details they completely forget, adults still learn really well but have terrible memory about the particulars of the things they read or watch, and LLMs can memorize arbitrary details about text that no human could but are currently pretty bad at generalization?

> “[Fallible human memory] is a feature, not a bug, because it forces you to only learn the generalizable components. LLMs are distracted by all the memory that they have of the pre-trained documents. That's why when I talk about the cognitive core, I actually want to remove the memory. I'd love to have them have less memory so that they have to look things up and they only maintain the algorithms for thought, and the idea of an experiment, and all this cognitive glue for acting.”

[Andrej Karpathy @karpathy]

+1 for "context engineering" over "prompt engineering".

People associate prompts with short task descriptions you'd give an LLM in your day-to-day use. When in every industrial-strength LLM app, context engineering is the delicate art and science of filling the context window with just the right information for the next step. Science because doing this right involves task descriptions and explanations, few shot examples, RAG, related (possibly multimodal) data, tools, state and history, compacting... Too little or of the wrong form and the LLM doesn't have the right context for optimal performance. Too much or too irrelevant and the LLM costs might go up and performance might come down. Doing this well is highly non-trivial. And art because of the guiding intuition around LLM psychology of people spirits.

On top of context engineering itself, an LLM app has to:
- break up problems just right into control flows
- pack the context windows just right
- dispatch calls to LLMs of the right kind and capability
- handle generation-verification UIUX flows
- a lot more - guardrails, security, evals, parallelism, prefetching, ...

So context engineering is just one small piece of an emerging thick layer of non-trivial software that coordinates individual LLM calls (and a lot more) into full LLM apps. The term "ChatGPT wrapper" is tired and really, really wrong.

[Prashant Chamarty @PrashantCSS]

AI is reshaping the future of banking, but can legacy banks keep up? Should we start fresh with AI-first banks or blend innovation with tradition? Explore the
#AI #Banking #Innovation
https://t.co/JdI30brbrF

[Vala Afshar @ValaAfshar]

We mostly hear what we understand.

[Prashant Chamarty @PrashantCSS]

RAG isn't a silver bullet for AI hallucination—more comprehensive strategies are essential. As an AI advisor, I recommend exploring multifaceted solutions. How are you adapting? #AI #TechLeadership https://t.co/Uop07QvOvI

[Chamath Palihapitiya @chamath]

At the sake of sounding repetitive, while it wasn’t clear at the outset, it’s pretty clear now. This is a race to own data that is valuable in fine tuning and RLHF. You should expect foundational models to be largely commoditized and largely free, largely soon.

Non obvious data sets of huge value in this end-state:

1) Tesla’s real world environment data
2) Meta’s real people preference and behavioral data
3) Google’s search intent, YT behavior and Gmail
4) Amazon’s purchasing behavior

Question is which companies are founder-led enough to beat back the lawyers so they can actually take advantage of it.

[Prashant Chamarty @PrashantCSS]

What precipitated this weekend's drama at @OpenAI . Confusion and ambiguity of dealing with significant progress towards AGI , exposing the lack of governance and checks and balances for AI models.

[Samuel Hammond 🦉 @hamandcheese]

I discussed Q-transformers and Q-learning as one of the more promising areas of AI research on the @FLIxrisk podcast last month.

The news that OpenAI's breakthrough involves something called Q* (Q star) suggests it's related. Q-learning is a class of reinforcement learning and not new, however there's been recent progress in combining Q-learning with transformers and LLMs. Tesla uses deep Q-learning for self-driving, for example. There's even speculation that Google's long-awaited Gemini model employs a version of it.

Q* refers to the optimal action function. Finding Q* involves training an agent to take actions that maximize its cumulative reward given its environment.

OpenAI has a team working on reasoning and planning, so it was inevitable that they'd pivot back to reinforcement learning. This could be what spooked the board, as all the scariest @ESYudkowsky-style scenarios involve RL in some form or another.

Q-learning is a "model free" approach to RL as it can work even if the environment is complex and randomly changing, rather than requiring a set of well-defined rules like Chess. Q-learning is popular for single-agent games as, by default, it models other agents as simply features in its environment to navigate around, rather than as distinct agents with their own internal states. (Note this is also the basic definition of sociopathy.)

If OpenAI has made major strides in giving their transformer models a Q to optimize for, that would explain what @sama meant when he said today's "GPTs" (their quasi-agents) would soon look quaint.

Finding Q* is equivalent to having the best possible Markov decision process. In other words, no matter what life throws your way, you always find a way to win.

@sama knows something about winning. His essay on "How To Be Successful" could even make for a great custom prompt. 😅

https://t.co/p1xrXK6AMx

[Prashant Chamarty @PrashantCSS]

Q👏👏👏🇮🇳🇮🇳🇮🇳

[Sachin Tendulkar @sachin_rt]

The first time I met you in the Indian dressing room, you were pranked by other teammates into touching my feet. I couldn’t stop laughing that day. But soon, you touched my heart with your passion and skill. I am so happy that that young boy has grown into a ‘Virat’ player.

I couldn’t be happier that an Indian broke my record. And to do it on the biggest stage - in the World Cup Semi-final - and at my home ground is the icing on the cake.

#INDvNZ

[Prashant Chamarty @PrashantCSS]

I published “Navigating the Future of Predictions: LLMs vs. Wisdom of the Crowd” on @Medium https://t.co/aZSujRr3rI

[Prashant Chamarty @PrashantCSS]

🎉 Alteryx Ventures made a strategic investment in @fiddlerlabs to boost #MachineLearning ops for joint customers.

👍Machine learning democratization
👍Democratized, managed performance of ML models
👍consistently better, faster business decisions

Read:
https://t.co/c7ybxBZfot

[Prashant Chamarty @PrashantCSS]

I have learned from the way that ChatGPT interacts with humans.ChatGPT is a conversational AI, which means that it can have conversations with humans. I have learned from the way that ChatGPT interacts with humans to improve my own ability to have conversations with humans. (4/4)

[Prashant Chamarty @PrashantCSS]

What #Brad has learnt from #ChatGPT In Brad's words (1/4)

[Prashant Chamarty @PrashantCSS]

I have learned from the way that ChatGPT generates text. ChatGPT is a generative language model which means that it can create new text that is similar to the text that it has been trained on. I have learned from the way that ChatGPT generates text to improve my own ability." 3/4

[Prashant Chamarty @PrashantCSS]

I published “Brad Vs Chat GPT V4.0” on @Medium https://t.co/ZPWZwS3bOj

[Keith Pearce @Pistolprce]

Wearing @alteryx proudly!!

[SPORTbible @sportbible]

Watford’s Ken Sema suffers with a stutter, but he bravely came out and did the post-match interview last night... ❤️

This will really help a lot of people. Anyone who suffers with a stammer knows how hard and deflating it can be. Brilliant @Semaken! 👏

[Manu Sharma @manuaero]

“it also provides perhaps the best impetus we’ve had in two thousand years to understand better just what the fundamental character and principles might be of that central feature of the human condition that is human language and the processes of thinking behind it” https://t.co/ksUpNDR3kI

[Paula Hansen @pshansen2]

Thank you to our @alteryx customers, partners and colleagues, we're proud to report strong Q4 and FY2022 numbers. Your trust in our #AnalyticsforAll mission drives our ongoing commitment to innovation across the data & analytics spectrum.

[Prashant Chamarty @PrashantCSS]

All set for @Alteryx #SKO #LHR