If you are someone who has not closely followed the Deep Learning and NLP literature but is curious about what innovations led to ChatGPT, this is my attempt at making that material accessible. Hope you enjoy reading this as much as I enjoyed writing it. https://t.co/tcXTix2VBS
A few thoughts,
1. We have learned to lose fidelity on memory of the distant past in a way that we can reconstruct most of it. Compaction needs to do something similar. Not summarize, but optimize for reconstruction fidelity, may be an auto-encoder that optimizes for, forgetting what LLM already knows, and keeping new info in a compact representation.
2. Retrieval should be coupled with inference. IT could be that we emit a set of retrieval instructions that is more complex than vector search. This is an RL problem.
3. Context needs to be a stack, not a compressed linear thing. We know when to discard near-term memory when we are done with something specific.
A mentor once told me: "We don't make right choices. We make our choices right." I don't have all the answers. This is a hypothesis. But I wrote a longer version with frameworks for healthcare, finance, law, engineering.
How about a metric based on Conditional Kolmogorov Complexity? Define the "Slop Index" as the ratio of the length of the text to the length of the shortest prompt required to reproduce it (or a close semantic match).
High Slop = A short, low-effort prompt generating massive, smooth, generic text. As you noted, easy to intuit, but computationally intractable to solve for the optimal prompt.
@krudin founded the growth function at both Meta (when it was Facebook) and Google. He led the transformation of ThoughtSpot from Sales lead GTM to Hybrid, PLG, and SLG motion. Now he is helping startups use these ideas. There is no one better than Ken to learn about product-led growth.
I sat down with Ken for the latest Effortless Podcast episode to dig into what actually works in growth. We covered everything from early-stage tactics to scaling challenges https://t.co/uY5bACdugG
@krudin founded the growth function at both Meta (when it was Facebook) and Google. He led the transformation of ThoughtSpot from Sales lead GTM to Hybrid, PLG, and SLG motion. Now he is helping startups use these ideas. There is no one better than Ken to learn about product-led growth.
I sat down with Ken for the latest Effortless Podcast episode to dig into what actually works in growth. We covered everything from early-stage tactics to scaling challenges.
Listen here: https://t.co/MdtidWczq5
Introducing AmpUp!
Having spent my time at Google building AI agents, I’ve seen firsthand the immense potential of LLM agents, but also the deep technical challenges that remain. Many of the critical components, like robust planning, tool use, and reliable evaluation are still open research questions.
Generic, off-the-shelf agents often fail because they lack the specific context of a business. They don’t understand how a specific team actually works and wins.
That’s the challenge we’re tackling with AmpUp. We’ve built a platform that allows sales teams to create their own custom, self-learning AI agents. These agents are built on the knowledge, strategies, and insights from your own top performers, creating a system that truly understands your business. We are making it possible for great performance to be repeatable.
Our goal is to move beyond generic AI and empower teams with tools built from their own expertise.
Early results have been promising.
Sales teams are seeing their execution variance drop significantly.
The future of sales isn't hoping for heroes; it's building a system of excellence.
AmpUp AI learns from what your best reps actually do—not generic best practices.
Connect it to your existing tools, and it starts identifying the patterns that separate top performers from everyone else.