alex paden

building

[Andrej Karpathy @karpathy]

Nice new read on tokenization!
You've heard about the SolidGoldMagikarp token, which breaks GPT-2 because it was present in the training set of the Tokenizer, but not the LLM later.

This paper digs in in a lot more depth and detail, on a lot more models, discovering a less extreme version of the above - partially-trained tokens in both open/closed models. You have to be careful with a lot of small details and implications - weight sharing, constants in residual streams, weight-decays, regex splitting patterns, BPE, UTF-8, etc.

TLDR Tokenization remains a major pain and a large LLM attack surface. Including these partially-trained tokens in your prompts drifts the model out of distribution into undefined regions of the dynamics, areas that the model is not used to. They confuse the LLM. The paper's focus is discovery and not engineering, but it seems likely one can find "token attacks" that reliably induce target weirdness: pop-off safety, alter personality or behaviors (?), any other kind of ... otherwise undefined behavior, whatever that may look like.

Now go ask GPT-4 about _ForCanBeConverted, $PostalCodesNL, useRalative, and _typingsJapgolly :)
(or see Figure 4 of the paper at the very end for simple examples)

[alex paden @shoni_eth]

@slashreboot Better if you give ratio scale response than brag about being past fable

[alex paden @shoni_eth]

@naygozalova im not sure ai is complicated enough for that yet

[alex paden @shoni_eth]

I’m gonna vote for people enabling ai

[alex paden @shoni_eth]

@naygozalova oh okay. bump

[sarah guo @saranormous]

Really fun to hang again with my friend 🃏 @polynoamial (OpenAI research scientist, our first guest ever on @NoPriorsPod in early 2023) to talk about the implications of large test-time compute, and what happens when models are given $10M budgets to spend on a single task. Topics:

01:23 – Why Benchmarks Are Broken
04:19 – Compute Budgets and Projections
06:48 – How Long Should Models Think?
08:01 – Benchmarkmaxxing
09:48 – Noam's Evals
12:40 – Safety (When Model Capability Scales With Spend)
16:09 – Implications For the Model Release Cycle
18:34 – Latent Model Capability
22:27 – Limits on Recursive Self-Improvement
28:38 – Large-Scale Multi-Agent Coordination
30:39 – Competition at the Frontier
33:19 – Breaking the Benchmark Grid Equilibrium
34:57 – Why Benchmarks Should be Scaled by Cost

[alex paden @shoni_eth]

@om was awesome and kind

[alex paden @shoni_eth]

@karpathy one of the ideas i really like the concept of is the org ide and files/data oriented on the people and team and operations more broadly

im excited to try this new implementation as well!

[bodila @51bodila]

Andrej Karpathy told Elon Musk that Tesla will grow from $45B to $1.5T using AI - while competitors spent billions doing it the old way

after this presentation Tesla's stock dropped 3.9%

Karpathy stood up and explained why every competitor will lose

"lidar is a crutch - it sidesteps the fundamental problem "

Musk: "every mile driven is training the network - whether autopilot is on or off "

simply because every single driver was training Tesla's model for free

that $45B company is now worth $1.5T

Waymo spent $27 billion. Tesla spent $0 on data collection

then Andrej left Tesla and joined Anthropic - to build their competitor

bookmark and watch today ↓

[Siew Win @ 🇲🇾 Network School @siewwwin]

🥇Won at @arkivnetwork builder challenge! @ns 😭

Cortex is what I wish every agent had:
memories that behave like human memory, forget like humans do.

use it in a decision → it stays.
ignore it → it decays.

Owned by your wallet. @claudeai MCP + Openclaw plug-in available.

[alex paden @shoni_eth]

@Rainmaker1973 Is it a wheel

[Daily Dose of Data Science @DailyDoseOfDS_]

Claude Code fully dissected!

Researchers from UCL reverse-engineered the leaked Claude source. What they found changes how you should think about agent design.

Only 1.6% of the codebase is AI decision logic.

The other 98.4% is operational infrastructure. Permission gates, tool routing, context compaction, recovery logic, session persistence. The model reasons. The harness does everything else.

This is the opposite of what most agent frameworks do today.

LangGraph routes model outputs through explicit state machines. Devin bolts heavy planners onto operational scaffolding. Claude Code gives the model maximum decision latitude inside a rich deterministic harness, and invests all its engineering effort in that harness.

The core loop is a simple while-true. Call model, run tools, repeat.

But the systems around that loop are where the real design lives:

A permission system with 7 modes and an ML classifier. Users approve 93% of prompts anyway, so the architecture compensates with automated layers instead of adding more warnings.

A 5-layer context compaction pipeline. Each layer runs only when cheaper ones fail. Budget reduction, snip, microcompact, context collapse, auto-compact.

Four extension mechanisms ordered by context cost. Hooks (zero), skills (low), plugins (medium), MCP (high). Each answers a different integration problem.

Subagents return only summary text to the parent. Their full transcripts live in sidechain files. Agent teams still cost roughly 7x the tokens of a standard session.

Resume does not restore session-scoped permissions. Trust is re-established every session. That friction is the point.

The bet behind all of this is simple. As frontier models converge on raw coding ability, the quality of the harness becomes the differentiator, not the model.

Paper: Dive into Claude Code (arXiv:2604.14228)

We've shared an article on Agent Harness and what every big company is building.

Read it below.

[alex paden @shoni_eth]

?

[How To Prompt @HowToPrompt__]

A toothpaste company has quietly killed the entire market research industry and nobody is talking about it.

Colgate published a paper showing you can predict real purchase intent at 90% accuracy by simply asking LLMs to roleplay customers.

And this is beyond insane.

If you ask an AI, "Rate this product from 1 to 5," it gives safe, middle-of-the-road garbage.

So researchers invented a method called Semantic Similarity Rating (SSR).

Instead of asking the AI for a number, they asked it to roleplay.

They gave the LLM a demographic profile. They showed it a product concept. And they asked it to write down its raw, unfiltered thoughts.

Then, they used a semantic model to translate those written thoughts into a numerical score.

The results are staggering.

Tested against 57 real corporate surveys and 9,300 actual human responses, the synthetic AI consumers matched real human buying behavior with 90% reliability.

They perfectly mirrored how different age brackets and income levels react to price changes.

And they provided detailed, qualitative feedback that was deeper and more critical than what actual humans wrote.

This destroys the economics of traditional market research.

You don't need to wait a month to see if a product will sell.

You can simulate 1,000 hyper-targeted customer interviews overnight.

You can A/B test pricing across every demographic instantly.

[GRITCULT @GRITCULT]

dont be confident, be over confident. be delusionally confident. be so confident that doors open for you and reality bends for you. be sure that GOD walks beside you. The meek inherit nothing if they never believed they were worthy of more. be too much. be impossible to ignore.

[a16z @a16z]

.@pmarca: "The person who writes down the thing has tremendous power."

@david_perell

[alex paden @shoni_eth]

@claudeai i'm unable to use fable because i have native memories in cybersecurity (the same prompt works in incognito)

[Rahul @sairahul1]

This is the best site on the internet to learn harness engineering.

Free. Completely.

Most AI engineers have never heard the term.

https://t.co/bwDbTTYsjM

Bookmark this site.

Then read this setup ↓

[Miles Cranmer @MilesCranmer]

This is an insane paper and I love it

https://t.co/DP8OR5NJf2