Ariel N. Lee

Senior Research Engineer @ Code Metal | Data Provenance Initiative | Open-Platypus | prev: Founding team @ Sciforium, Raive | @UCLA @BU_ece

[will brown @willccbb]

we raised $130M @ $1B for our series A

>$100M run rate

we’re just getting started

[codila @0xCodila]

Anthropic Engineer Andrej Karpathy:

"The biggest mistake in AI right now - people are forcing agents to work instead of mastering the model first

We made that mistake in 2016 at OpenAI - It cost us 5 years "

what Karpathy actually means:

step 1 → stop forcing your agent to do everything, understand the model underneath first

step 2 → demos are easy - products take a decade. self-driving proved it - if you skip the foundation, everything breaks

step 3 → the agent is not the product. the foundation is. build that - and agents emerge on their own

"you building agents right now - you're at the forefront. not OpenAI. not DeepMind. you "

watch - bookmark, then read article below ↓

[NVIDIA AI @NVIDIAAI]

We took a 30B model and split it in two to write tokens in parallel instead of one at a time.

Introducing Nemotron-Labs-TwoTower: a diffusion language model from NVIDIA Research adapted from Nemotron-3-Nano-30B-A3B. Here’s how it works: one half holds the context, the other writes the tokens, with both reusing the pretrained model instead of training a new one from scratch.

We found it kept 98.7% of the original model’s quality at 2.42× faster generation.

[John Schulman @johnschulman2]

People sometimes ask why fine-tune when general-purpose models keep getting better. Bridgewater's work is a good reminder that with the right data -- here, expert judgements -- you can beat prompting-only approaches by a lot.
@ddkang and the Bridgewater AIA Labs team are great -- glad to see them sharing this.

[Anthropic @AnthropicAI]

Claude Fable 5 will be available again globally tomorrow.

After a series of productive conversations with the US government, we're redeploying the model with a new set of classifiers to target and block more cybersecurity tasks. In the near term, some routine tasks like coding and debugging will fall back to Opus 4.8. We’ll continue to refine these classifiers over the coming weeks to reduce false positives and better distinguish genuine misuse from legitimate requests.

We’ve also begun drafting a consensus framework—with Amazon, Microsoft, Google, and other Glasswing partners—for assessing the severity of AI jailbreaks and how AI developers should respond to them. We invite other industry partners and model providers to join us in this effort.

Finally, we’re scaling up our collaboration with the US government on model testing and safeguards. This will include pre-release access to models and safeguards for evaluation, information sharing on jailbreaks and misuse, and dedicated resources for joint research.

Thank you to our users for your patience, and to our partners across the government, industry, and the research community who worked alongside us to make Fable 5 available again.

Read our full blog: https://t.co/VHyum831ri

[Nathan Lambert @natolambert]

Not enough people studying SFT methods. It’s a foundation of post training with limited literature that seems very serious in an empirical sense.

[Alex Volkov @ AI Engineer @altryne]

MidJourney just announced... a full body ultrasound! Yup... read on because it's as crazy as it sounds.

"As powerful as MRI and as casual as a trip to the spa"

They are calling it "the @midjourney scanner"

Insane details:

- First, the scale. The device uses 8,960 individual transducers arranged in a ring around your body

- The precision is the most jaw-dropping part: it resolves motion at the picometer range. It can image internal tissues finer than the width of an atom. We are talking sub-atomic level diagnostic capability

- The compute requirement is massive. The system processes 17 gigabytes of data per second.

It takes 40GB of raw data to reconstruct just one cross-sectional slice. And they are planning to scan 100 slices?

- Midjourney claims that fewer than 12 of these machines could perform more full-body scans than every MRI machine on Earth combined.

Welcome to the future of healthcare!

Not only these scanners are announced, they will exist in a "Midjourney SPA" - with hot tubs, saunas, cold plunges, and 9-10 whole body scanners.

[Susan Zhang @suchenzang]

current LLMs fundamentally consist of four main components:

- input layer: where input "words" (prompt) get mapped to "latents" aka some-model-representation-you-don't-understand-unless-you-start-reading-tea-leaves-of-spurious-correlations (some quite compelling à la word2vec style; latents is also unnecessary lingo so i will refer to these as "inputs" with quotes from now on)

- mixing layers: where you jumble all your "inputs" together to see if any correlations between "inputs" can become useful (commonly used to compress or expand dims; predicting a single classification target == compress to a single dim, etc)

- attention layers: where you learn how "inputs" relate to each other (aka discern what's important to remember vs fluff)

- residuals: where you short-circuit a mixing/attention layer because it's probably adding too much confusion (aka avoid overthinking for simple things)

-----

a "big" LLM simply scales two things:

- width == how many dimensions you give to your "inputs" (the more dims, in theory the more unique/discerning/precise/complex your knowledge can become)

- depth == how many mixing/attention/residual layers you can stack/loop between (aka "reason" over, where more of these ~= more "reasoning" abilities)

"capabilities" that seem impressive to humans usually arise from taking advantage of both depth & width: where a model seemingly makes connections between disparate ideas, beyond what an average human can hold in working memory.

this requires models to "completely light up" when responding to a "hard prompt", where effectively no param/layer goes unused.

-----

the anatomy of a "model capability" is precisely the same mechanism that can be co-opted for a jailbreaking exploit:

your goal is simply to "light up" as much of the model as possible, dodging any shallow input-classifiers at the beginning by triggering as many disparate "input ideologies" as possible, and subsequently have these "inputs" relate to each other in seemingly unrelated-yet-related ways that ideally have similar "complexity" as your jailbreak goal (to make it past enough layers of the model).

think of the attack-vector as bundling your goal in a series of schizo-nerd-snipes:

a sufficiently capable model will try to reason through everything all at once, eliminate the dead-ends, and successfully deliver the one jailbreak use-case you bubble-wrapped for.

of course, there's an art to the above, and some are already extraordinarily proficient at the trojan-horse-packaging, but at some point there's no difference between "a capability" and "a jailbreak", though i'll be happy to be proven otherwise.

-----

tl;dr ant flew too close to the sun, better kiss the ring or get buried.

[Google AI Developers @googleaidevs]

DiffusionGemma, our experimental open model released under an Apache 2.0 license, explores text diffusion, an exceptionally fast approach to text generation.

Here’s how DiffusionGemma accelerates development:

+ Faster token output: By shifting the bottleneck from memory bandwidth to raw compute, the model generates up to 4x faster token output on dedicated GPUs
+ Accessible hardware footprint: Activates just 3.8B parameters during inference, fitting comfortably within 24GB-VRAM high-end consumer GPUs when quantized
+ Novel workflows: Parallel token generation enables self-correction, making it ideal for code infilling, in-line editing, and non-linear structures

DiffusionGemma prioritizes speed over raw quality and accelerates best on compute-bound hardware (like @NVIDIAAI GPUs). Standard @GoogleGemma 4 remains recommended for production quality and memory-bound devices.

[Thomas Jiang @thomasjiangcy]

today is my last day at anthropic.

leaving to build the next generation of devtools.

for-loop as a service; waitlist coming soon

[Nous Research @NousResearch]

We have been working closely with @nvidia to ensure Hermes Agent works smoothly on their new @NVIDIARTXSpark superchip and integrates with the new OpenShell runtime, which connects Hermes to @Microsoft's security primitives.

Watch our feature in the big announcement at Computex:

[Ryan Aytay @ryanaytay]

AI is rapidly creating a trust gap between what models can generate and what can be safely deploy into prod.

Closing the gap requires a new approach.

@Code_Metal_AI focuses on safely delivering the last mile for mission-critical systems and self improves over time.

[Muratcan Koylan @muratcan]

Gradient descent for SKILL.md files sounds interesting, maybe a bit complex but it's becoming a real part of agent harness.

SkillOpt is one of the first papers to treat markdown skill files as trainable parameters and provides a proper optimization framework for them.

A few things I learned that you should consider too.

1. The validation gate is the only thing that matters in a self-editing loop.

Held-out set, strict improvement, ties rejected. End-to-end, their best skills land with 1 to 4 accepted edits total. If your "self-improving agent" is accepting most of what it proposes, you're shipping slop.

2. Bounded edits are better than full rewrites. 4 to 8 edits per step is the sweet spot.

Remove the budget and performance collapses. This is the textual analog of learning rate, and it transfers to any LLM-as-author loop. If you're using an agent to refactor your docs, your prompts, or your skills, cap the diff size.

3. Compactness wins. Median final skill: ~920 tokens.

Skills do not need to be long. They need to be high-signal. Most skill files I see are bloated because length feels like effort. It isn't.

4. The harness is becoming less important; the skill is becoming more important.

A Codex-trained skill ported into Claude Code hit +59.7 points on SpreadsheetBench. Procedural knowledge is more general than the runtime that
produced it.

5. Frozen model + trained context is the practical adaptation.

GPT-5.4-nano with a SkillOpt'd skill ≈ frontier behavior on procedural benchmarks. Cheaper, portable, inspectable, zero inference-time cost. This is
the answer to "how do we adapt a frontier model for our domain" for almost everyone who isn't training their own models.

6. Verification is the bottleneck.

Every gate in this paper depends on an auto-grader. That works for benchmarks. It fails for writing, design, and strategy, exactly the open-ended work we want to automate. Whoever builds the verifier for open-ended tasks owns the next stage.

There are also two leassons I learned while shipping v2.3.0 of my Context Engineering Agent Skills repo, measured across composer-2, claude-opus-4-7,
gpt-5.5, and gemini-3.1-pro via the @cursor_ai SDK:
- Description and body are two different surfaces. The router only sees the description. The agent sees the body once activated. They can quietly disagree, and only end-to-end task tests catch it.
- Aggregate accuracy is the wrong unit. When I rewrote three descriptions, the corpus average moved ~1pp. Individual skills moved 23–25pp. Per-skill effect size is where the action is.

Also, in Feb 2026 I shared a piece called Personal Brain OS arguing that the markdown file is a first-class substrate for agent state. SkillOpt is the optimizer-shaped version of that same argument: not "store memory in files" but "treat files as trainable parameters with proper optimization machinery around them." That's the move from static to measured.

The fast/slow split they describe already lives implicitly in the digital-brain-skill repo:
- voice-guide and tone-of-voice.md are slow-state (rarely touched)
- posts.jsonl and bookmarks.jsonl are fast-state

What SkillOpt adds that I didn't have is a protected section invariant, a structural guarantee that fast edits cannot overwrite slow lessons. Removing that mechanism cost them 22 points on SpreadsheetBench. Worth borrowing.

If you're building agents, SkillOpt: Executive Strategy for Self-Evolving Agent Skills is a good paper to read: https://t.co/ZS9SZXQ6Mv

[Rohan Paul @rohanpaul_ai]

WOW, 🤯 A leaked audio from Meta’s April 30 all-hands.

Meta is reportedly using its own engineers’ work traces to train coding AI while cutting thousands of jobs.

Here Zuckerberg arguing that models learn better when they watch “really smart people” perform tasks, meaning Meta’s internal code, tool use, clicks, and problem-solving can become higher-grade training data than contractor-written examples.

The idea is behavior cloning: instead of only feeding an AI finished code, Meta can feed it the step-by-step path a strong engineer takes, including edits, tests, mistakes, fixes, and tool choices.

That can teach a model not just what correct code looks like, but how a skilled developer moves from a vague task to a working solution.

Meta is reportedly cutting about 8,000 jobs, roughly 10% of its workforce, and additionaly moving about 7,000 employees toward AI-focused work, so the hard realy is that human expertise is being turned into training data before some of those humans leave.

The story is not fully independently verified, but the shift is happening for sure: tech companies no longer see AI as a tool sitting beside workers, but as a system that can absorb worker patterns and then compress them into software.

[Nous Research @NousResearch]

Today we release Lighthouse Attention, a selection-based hierarchical attention for long-context pre-training that delivers a 1.4-1.7× wall-clock speedup at 98K context.

It runs the same forward+backward pass ~17× faster than standard attention at 512K context on a single B200, without a custom sparse attention kernel, a straight-through estimator, or an auxiliary loss.

During training, queries, keys, and values are pooled symmetrically into a multi-resolution pyramid. We then score every pyramid heads, and a top-k cascade selects a small hierarchical dense sub-sequence, and after a sorting pass that enforces causality, we use standard attention for token mixing. A brief full attention resume at the end converts the checkpoint back into a competent dense-attention model.

Validated this using 530M parameter Llama-3 models across 50B tokens, with up to 1M-token benchmarks across 32 B200s under context parallelism.

The work on Lighthouse Attention was led by @bloc97_, @SubhoGhosh02, and @theemozilla.

[Nous Research @NousResearch]

Today we release Token Superposition Training (TST), a modification to the standard LLM pretraining loop that produces a 2-3× wall-clock speedup at matched FLOPs without changing the model architecture, optimizer, tokenizer, or training data.

During the first third of training, the model reads and predicts contiguous bags of tokens, averaging their embeddings on the input side and predicting the next bag with a modified cross-entropy on the output side. For the remainder of the run, it trains normally on next-token prediction. The inference-time model is identical to one produced by conventional pretraining.

Validated at 270M, 600M, and 3B dense scales, and at 10B-A1B MoE.

The work on TST was led by @bloc97_, @gigant_theo, and @theemozilla.

[Rohan Paul @rohanpaul_ai]

WSJ: Anthropic is wrapping up a deal to set up a joint venture with Blackstone, Goldman Sachs, and other Wall Street firms, with the goal of selling AI tools to private-equity backed companies, according to people familiar with the matter.

Anthropic, Blackstone, and Hellman & Friedman are expected to put in about $ 300M each, while Goldman Sachs is expected to invest about $ 150M.

The new company would work like an AI deployment arm, meaning it would not just sell Claude access, but help companies rebuild workflows around LLMs, from customer support and finance to coding, legal review, and internal research.

Private-equity firms are the target because they own many companies, measure every cost tightly, and can force software changes faster than slow public companies.

Anthropic gets distribution, Wall Street gets a stake in the AI services layer, and portfolio companies become a large testing ground for enterprise AI.

The deeper move is that AI labs are no longer only competing on model quality, but on who controls the path from model to business process.

---

wsj .com/business/deals/anthropic-nears-1-5-billion-joint-venture-with-wall-street-firms-8f5448ee

[Aakash Gupta @aakashgupta]

Karpathy told Dwarkesh that a 1 billion parameter model, trained on clean data, could hit the intelligence of today's 1.8 trillion parameter frontier.

That is a 1,800x compression claim. The math behind it is more defensible than it sounds.

When researchers at frontier labs look at random samples from their training corpus, they see stock ticker symbols, broken HTML, forum spam, autogenerated gibberish. Not Wikipedia. Not the Wall Street Journal. The actual pretraining dataset is mostly noise, and the model is burning parameters to vaguely remember all of it.

One estimate pegs Llama 3's information compression at 0.07 bits per token. Well-structured English carries around 1.5 bits per token of real information. The trillion-parameter model is holding a roughly 5% resolution image of the internet it trained on.

So when a lab ships a 1.8 trillion parameter model, the overwhelming majority of those weights are handling rough memorization. They are compression overhead for a noisy training set, taking up capacity that could be doing reasoning instead.

Karpathy's proposal is to separate the two. Build a cognitive core: a small model that contains only the algorithms for reasoning and problem-solving, stripped of encyclopedic memorization. Pair it with external memory the model queries when it needs a fact. A 1 billion parameter reasoner plus retrieval beats a 1.8 trillion parameter model trying to do both.

The data already supports this direction. GPT-4o runs at roughly 200 billion parameters and outperforms the original 1.8 trillion GPT-4. Inference costs for GPT-3.5 level performance fell 280x between 2022 and 2024, driven almost entirely by smaller, cleaner, better-architected models. The trend line is pointing where Karpathy says it should.

The real implication for anyone tracking the AI trade: data quality is the actual constraint. The companies winning the next phase will be the ones who figured out what to train on, and what to throw away.

[clem 🤗 @ClementDelangue]

Introducing Kernels on the Hugging Face Hub ✨

What if shipping a GPU kernel was as easy as pushing a model?

- Pre-compiled for your exact GPU, PyTorch & OS
- Multiple kernel versions coexist in one process
- torch.compile compatible
- 1.7x–2.5x speedups over PyTorch baselines

[James Campbell @jam3scampbell]

anthropic roommate came back sloppy drunk at 3am last night and had a full scale crash out through tears and slurred words about how the world will never be the same

glad to hear the mythos release was received well internally