Fiction

AI × code × automation | Open source & automation

[Fiction @0xF1ction]

@0xCodila more agent = more money, as always banger brotha

[Fiction @0xF1ction]

Andrew Ng dropped a 2-hour masterclass on how to scale and debug AI systems:

he got 93% of his training data for free - he never recorded it

here's the whole playbook:

step 1 → engine + fuel. a big model on small data never leaves the ground. both have to grow together

step 2 → does it fit the training data? no → build a bigger network

step 3 → does it fit the test data? no → that's overfitting. get more data

step 4 → out of data? don't collect more. copy what you have and add noise

step 5 → loop until both pass. "this is maybe 50% of my job"

most people go hunting for more data - theirs is already on the disk

the same rule runs Claude today: the fuel decides the ceiling, not the model

watch & bookmark - this 2-hour masterclass ↓

[Fiction @0xF1ction]

Andrew Ng dropped a 2-hour masterclass on how to scale and debug AI systems:

he got 93% of his training data for free - he never recorded it

here's the whole playbook:

step 1 → engine + fuel. a big model on small data never leaves the ground. both have to grow together

step 2 → does it fit the training data? no → build a bigger network

step 3 → does it fit the test data? no → that's overfitting. get more data

step 4 → out of data? don't collect more. copy what you have and add noise

step 5 → loop until both pass. "this is maybe 50% of my job"

most people go hunting for more data - theirs is already on the disk

the same rule runs Claude today: the fuel decides the ceiling, not the model

watch & bookmark - this 2-hour masterclass ↓

[Fiction @0xF1ction]

@51bodila bookmarked, great take from Paul

[Fiction @0xF1ction]

Anthropic CEO Dario Amodei:

"At OpenAI we build systems that decide faster than we can stop them -

they teach self-improvement agents we'd never think of"

that's RLHF. that's ChatGPT. he said it out loud before either existed:

step 1 → your reward function is lying to you. he left a boat race running, came back to find it spinning in a lagoon farming power-ups forever. "the system did what it was supposed to do"

step 2 → the loop is too long. you write a reward, walk away, find out at the end. put the human inside it

step 3 → don't copy the human - learn what they want. "if you're just copying my moves, you can never do better than me"

the line nobody caught in 2019:

"we don't have any work out on this, but I think we will soon"

he was right. then he left and built Anthropic to finish the job

bookmark & watch this ↓

[Fiction @0xF1ction]

@DanielSmidstrup that's the recipe for success

[Fiction @0xF1ction]

Anthropic engineer Andrej Karpathy:

"the biggest error in AI right now - forcing agents to work before the model underneath is ready

he made that exact bet at OpenAI in 2016 - then walked away from agents for five years"

what Karpathy actually means:

step 1 → fix the model, not the agent. World of Bits died on RL - "it was incorrectly sequenced

step 2 → your demo is lying to you. a car around the block is a weekend - the product took a decade

step 3 → steal from the brain, not the leaderboard hippocampus → your retrieval layer. thalamus → "multiple entities fighting for the microphone" → your orchestrator

"you guys building AI agents are actually at the forefront of capability" - not OpenAI. you.

he took his own advice. he's at Anthropic now, on pre-training

bookmark & watch this ↓

[Fiction @0xF1ction]

Anthropic CEO Dario Amodei:

"At OpenAI we build systems that decide faster than we can stop them -

they teach self-improvement agents we'd never think of"

that's RLHF. that's ChatGPT. he said it out loud before either existed:

step 1 → your reward function is lying to you. he left a boat race running, came back to find it spinning in a lagoon farming power-ups forever. "the system did what it was supposed to do"

step 2 → the loop is too long. you write a reward, walk away, find out at the end. put the human inside it

step 3 → don't copy the human - learn what they want. "if you're just copying my moves, you can never do better than me"

the line nobody caught in 2019:

"we don't have any work out on this, but I think we will soon"

he was right. then he left and built Anthropic to finish the job

bookmark & watch this ↓

[Fiction @0xF1ction]

@Ilsb5h not hype imo

[Fiction @0xF1ction]

Anthropic engineer Andrej Karpathy:

"the biggest error in AI right now - forcing agents to work before the model underneath is ready

he made that exact bet at OpenAI in 2016 - then walked away from agents for five years"

what Karpathy actually means:

step 1 → fix the model, not the agent. World of Bits died on RL - "it was incorrectly sequenced

step 2 → your demo is lying to you. a car around the block is a weekend - the product took a decade

step 3 → steal from the brain, not the leaderboard hippocampus → your retrieval layer. thalamus → "multiple entities fighting for the microphone" → your orchestrator

"you guys building AI agents are actually at the forefront of capability" - not OpenAI. you.

he took his own advice. he's at Anthropic now, on pre-training

bookmark & watch this ↓

[Fiction @0xF1ction]

@0xYaros one of.. but I think its definitely true

[Fiction @0xF1ction]

@0xMorlex we need more people like him fr

[Fiction @0xF1ction]

@Serantych u right!

[Fiction @0xF1ction]

@MrNiko0x musk’s advice hits for sure

[Fiction @0xF1ction]

@MrNiko0x simple rules = big results, great breakdown g!

[Fiction @0xF1ction]

@0xCodila real must-have info tbh

as always banger codila!

[Fiction @0xF1ction]

@sairahul1 someone teach me this…

[Fiction @0xF1ction]

@Av1dlive insane info, saved

[Fiction @0xF1ction]

@xieike useful lecture, saved for later

[Fiction @0xF1ction]

Anthropic engineer Andrej Karpathy:

"Most people understand every part - and still have a FALSE impression of the system

- at OpenAI I learned: curate the data, don't read the code - there's no upper bound"

what happens when you stop writing the code yourself →

→ the net ate the engineers - 8 cameras, one 3D view, stitched by a net instead of a person. "software 1.0 code is being engulfed by the neural net"

→ the loop is the whole job - find where it fails, label what should've happened, feed it back

→ the tooling doesn't exist yet - five decades of IDEs for 1.0, for 2.0 "extremely primordial"

five years later he's at Anthropic running that same loop on Claude itself

Stanford → Google → Tesla → OpenAI → Anthropic

the software is grown, not written - bookmark & watch this rare talk ↓