Ground Zero

Everything AI and Culture • Deep Tech Podcasts • partner: [email protected]

[Ground Zero @groundzero_twt]

Update.

We just published all Ground Zero Episodes on Spotify and Apple Podcast for better listening experience.

On Spotify: https://t.co/D8JzBxS3Zr

On Apple Podcasts: https://t.co/aNOImcfQTA

[himanshu @himanshustwts]

i think @groundzero_twt convos lately are truly living up on the frontier..with the kind of guests bringing alphas from their niche.

next, i am looking to bring more eps around mech interp, token efficient archs, multimodality, robotics and more state-of-the-art. if you are working around these, please hmu!

[himanshu @himanshustwts]

does AI make being self-taught easier, or does it trap you in infinite explanations without ever doing the reps?

it definitely makes it easier, and it definitely traps people. a lot of the self-learning you can do with LLMs sometimes is borderline on the kind of thing that you do when you watch 3b1b. he is wonderful creator, but in 25 minutes you don't learn anything. there is nothing for free. there must be some pain and some work.

what you get through 3b1b is - if you've done some work before then maybe it helps you visualize. it helps you rethink something completely, puts it into words. but from scratch, i don't know anybody who's been learning programming completely from scratch at the moment. i am really curious what it looks like.

does it matter, by the way? does it matter that you're learning if you can do the thing? if the agents do it for you then what does it matter? a lot of the things you don't need to learn, you don't need to understand at all. you need to know how to use them or get them to the people that need them that are gonna pay you for it.

the latest models are so good, even at pretty advanced mathematics. but that's the difference between being able to visualize something and explain it at a high level and having the formal capacity to use the tools. LLMs can help generating exercise sheets or whatever but basically it's up to your personality to keep you in check and tell you - if i don't just grind exercises, then maybe I know this, but i know this in a very superficial way.

cc @groundzero_twt / pod with Ludwig

[_doola @doolasux]

good/refreshing stuff, full of honesty and valuable insight from @ludwigABAP. rare that i see a podcast and immediately give it a listen, because they are almost always largely performative or void of substance

well led @himanshustwts, has that genuine feel i miss so v much 🤝❤️

[v. @prnshtwt]

absolute gold of a podcast, thank you .@himanshustwts .@ludwigABAP

[presilence @presilences]

Just listened to the whole thing, good podcast! The host had some really nice questions especially bouncing back on some of what Lud said, I enjoyed my time

[Ground Zero @groundzero_twt]

Watch out conversation with Ludwig!

[himanshu @himanshustwts]

A Quest to Formalize Intelligence ft. @ludwigABAP

0:00:00 - INTRO
0:01:06 - What is he most excited about these days
0:03:31 - All arcs are isomorphic, Dropping out in 7th grade
0:09:31 - Does skipping formal education make you naturally broader?
0:10:11 - Growing up on 4chan, Mathematics is humbling
0:14:52 - Mentors, First programming job, Internet microcultures
0:21:47 - Culture in SF, Why he won't start a company
0:25:21 - Does AI make self-teaching easier or trap you in infinite explanations?
0:29:49 - The 20% doing 80% inside a company
0:33:24 - Company culture and Building subcultures
0:35:56 - Define understanding, Compression equals Prediction
0:45:44 - Understanding vs knowing, Michael Levin, Cognitive light cones
0:54:39 - Starting a Research lab, Sheaf theory and Grothendieck
0:58:43 - Mapping Levin's biology onto proof space with MCTS and Lean4
1:02:49 - Active inference, Distributed systems and Morphogenesis
1:12:28 - Signal vs noise on X, The unit distance proof, Flanderization
1:17:39 - Community Questions
1:21:39 - Shape rotator vs wordcel, Tenstorrent, How to hire great engineers
1:28:01 - Building mental space to do things that will be left to do
1:33:42 - Advice to a 20-year-old. Be a capable thinker, you have 20k days left

[Ground Zero @groundzero_twt]

@himanshustwts @ludwigABAP .@haizelabs has been sponsor for this episode. They are building expert-level agents for mission-critical work, powered by proprietary Reliability Harness. Also, checkout articles by @leonardtang_

To sponsor future episodes, visit https://t.co/bJvpiY3C2H

[himanshu @himanshustwts]

A Quest to Formalize Intelligence ft. @ludwigABAP

0:00:00 - INTRO
0:01:06 - What is he most excited about these days
0:03:31 - All arcs are isomorphic, Dropping out in 7th grade
0:09:31 - Does skipping formal education make you naturally broader?
0:10:11 - Growing up on 4chan, Mathematics is humbling
0:14:52 - Mentors, First programming job, Internet microcultures
0:21:47 - Culture in SF, Why he won't start a company
0:25:21 - Does AI make self-teaching easier or trap you in infinite explanations?
0:29:49 - The 20% doing 80% inside a company
0:33:24 - Company culture and Building subcultures
0:35:56 - Define understanding, Compression equals Prediction
0:45:44 - Understanding vs knowing, Michael Levin, Cognitive light cones
0:54:39 - Starting a Research lab, Sheaf theory and Grothendieck
0:58:43 - Mapping Levin's biology onto proof space with MCTS and Lean4
1:02:49 - Active inference, Distributed systems and Morphogenesis
1:12:28 - Signal vs noise on X, The unit distance proof, Flanderization
1:17:39 - Community Questions
1:21:39 - Shape rotator vs wordcel, Tenstorrent, How to hire great engineers
1:28:01 - Building mental space to do things that will be left to do
1:33:42 - Advice to a 20-year-old. Be a capable thinker, you have 20k days left

[Ground Zero @groundzero_twt]

@joel_the_ai @himanshustwts @ludwigABAP yes

[himanshu @himanshustwts]

this was a crazy conversation and i can say that @ludwigABAP is an absolute mogger ahaha

scoop: he has codex /goal running for 23 hrs on continuous automata.

didn't realize how two hours passed ngl and i can say there is lot to learn from him. publishing soon.

[himanshu @himanshustwts]

in another news, ludwig is joining us on ground zero podcast soon.

the internet has created a new kind of engineer - self taught, taste driven and adversarial to credentials. i believe ludwig is a case study of that archetype.

drop your questions for him around the concept of intelligence, morphogenesis, understanding systems or ai in general.

[himanshu @himanshustwts]

i think @groundzero_twt convos lately are truly living up on the frontier..with the kind of guests bringing alphas from their niche.

next, i am looking to bring more eps around mech interp, token efficient archs, multimodality, robotics and more state-of-the-art. if you are working around these, please hmu!

[himanshu @himanshustwts]

https://t.co/3vgtNkh8v0

[himanshu @himanshustwts]

new kind of data ahead might not be a new category of labels or trajectories but the data that closes the loop making AI good enough to teach humans back.

there is AI (artificial intelligence) where humans teach AI and make it smarter. And then there is IA (intelligence augmentation) where AI teaches humans and makes them smarter. as a human, IA is better -- you don't want to spend whole life draining your brain to make AI better but rather have AI make you better.

but then there is a loop. AI won't be good enough to teach humans and tutor them and coach them unless the AI is really good first. and the only way to make the AI good is for humans to provide a lot of data and information and reasoning, and for researchers to work on algorithms, and for hardware folks to make things scale. everyone is doing their part across those four quadrants but IA is the promise.

cc @groundzero_twt / pod with Curtis from Handshake AI.

[himanshu @himanshustwts]

How does GPT-5 become GPT-6?

Everything on the internet was already trained on a year, year and a half ago. New data is being added but it's marginal and a lot of it is AI slop. There is interesting private data (like JP Morgan's data) but nobody is opening that up.

You don't go from GPT-5 to GPT-6 in isolation because that's not how learning works. Every learning that has ever occurred comes from having information to learn. You can't learn no information.

cc @groundzero_twt / pod with Curtis from Handshake AI.

[himanshu @himanshustwts]

reflecting on this again.

"mid-training, pre-training, post-training -- these are all made-up words. there were perfectly fine words before them like "supervised learning" instead of "SFT." when a technology has a profound impact on society and people start asking questions that don't have well-defined answers, we make up names.

there are definitions for these things, and people who can articulate them precisely are valued. but there's also value in zooming out and recognizing there's a lot of ways to skin a cat."

cc @groundzero_twt / pod with Curtis from Handshake AI.

[himanshu @himanshustwts]

There is a lot of noise about what is the diff between "good" and "bad" quality of data to train models.

Curtis (Director of AI Research, Handshake) sent Slack message to about 50 AI researchers telling them they need to stop just saying "bad quality" or "good data" because those phrases are meaningless without precision.

He broke it down into specific things people actually mean when they say "bad quality":

1. The prompts are ambiguous. There can be more than one correct response from the model. This is one specific type of bad quality.

2. The grader is unreliable. Let's say there is an AI grader checking if responses are right or wrong, and then humans checking if the grader is right or wrong, and the inter-rater reliability of those humans is low. So three human reviewers can't agree on the same answer. This is a completely different problem from ambiguous prompts.

One should define the metric first, stand by it and work on it.

[Ground Zero @groundzero_twt]

Must watch episode if you are a bit interested in Human Data, RL Envs, Benchmarks and Evals.

[himanshu @himanshustwts]

The Delta of Intelligence is Human Data!

@cgnorthcutt is Director of AI Research at Handshake AI. In this episode, Curtis shares his unique perspectives on human data, RL envs, four quadrants of the AI market, journey of Handshake and much more.

0:00:00 - INTRO
0:02:08 - The Snowman Effect: From Rural Kentucky to 7 Years at MIT
0:09:50 - Confident Learning, Two Systems of Intelligence, Classical ML Has to Prevail
0:20:37 - Origin of CleanLab and Handshake acquisition 0:28:00 - How does GPT-5 become GPT-6?
0:31:20 - The Fastest growing Data Lab: What makes Handshake different?
0:35:32 - Difference between Good Data and Bad Data 0:38:17 - New kind of Data, IA and AI
0:42:04 - Scaling Coding Benchmarks and Efficiency in Long Horizon Tasks
0:49:12 - On Pre-Training, Misconception around Human Data
0:57:13 - Open Question: Taste and Personality, Quick Fire
01:05:15 - Advice to 20yo : Skill and Obsession

[himanshu @himanshustwts]

This episode is quite a learning experience for human data, RL Envs and market around it.

> Everything on the internet has already been trained on so new data being added is marginal and a lot of it is AI slop. Private data like enterprise ones would be valuable but no one is opening that up.

> So when you ask what actually takes a model from GPT-5 to GPT-6, the answer is human data. This is the delta of intelligence. (Essentially not a new architecture and not more GPUs.) New information from human minds which is formatted into something learnable.

> The AI market is really just four quadrants - GPUs, researchers, consultants, and human data. Only one of those four (human data) - actually provides the new information that makes models smarter. The other three enable it but don’t create it.

> Handshake’s revenue has roughly 10x’d within two months of the Cleanlab acquisition. This was news to me.

> The “expert data is fake” narrative is wrong but for the wrong reasons. The issue isn’t that PhDs aren’t doing expert work but the long tail of human capability is so massive that even non-expert tasks still haven’t been fully captured by AI. Yes, long way to go.

There are so many alphas in this one. Def worth to watch!

[himanshu @himanshustwts]

The Delta of Intelligence is Human Data!

@cgnorthcutt is Director of AI Research at Handshake AI. In this episode, Curtis shares his unique perspectives on human data, RL envs, four quadrants of the AI market, journey of Handshake and much more.

0:00:00 - INTRO
0:02:08 - The Snowman Effect: From Rural Kentucky to 7 Years at MIT
0:09:50 - Confident Learning, Two Systems of Intelligence, Classical ML Has to Prevail
0:20:37 - Origin of CleanLab and Handshake acquisition 0:28:00 - How does GPT-5 become GPT-6?
0:31:20 - The Fastest growing Data Lab: What makes Handshake different?
0:35:32 - Difference between Good Data and Bad Data 0:38:17 - New kind of Data, IA and AI
0:42:04 - Scaling Coding Benchmarks and Efficiency in Long Horizon Tasks
0:49:12 - On Pre-Training, Misconception around Human Data
0:57:13 - Open Question: Taste and Personality, Quick Fire
01:05:15 - Advice to 20yo : Skill and Obsession