Clive Chan

chips @openai / formerly a car @tesla dojo

[Clive Chan @itsclivetime]

@elonmusk lfg looking forward to the race

[Clive Chan @itsclivetime]

honestly pretty solid idea

[Kevin Frazier @KevinTFrazier]

New @nytimes op-ed by @BernieSanders calls for sovereign wealth fund tied to the stock of frontier labs.

Whether or not you back this idea, his closing is a reminder that people support what they help build. Absent more meaningful mechanisms for people to share their views on AI and shape its development, the backlash will grow and “missed uses” will become the default (ie we will fail to realize the most beneficial uses of AI).

“It must be decided by workers, parents, teachers, artists, scientists, communities and the American people. It’s our future. We must decide it.”

[Clive Chan @itsclivetime]

most confidential filing in the history of filings, maybe ever

[Anthropic @AnthropicAI]

Anthropic has confidentially submitted a draft S-1 registration statement to the Securities and Exchange Commission.

Pending completion of SEC review, this gives us the option to pursue an initial public offering.

Read more: https://t.co/onGZAhRLvD

[Clive Chan @itsclivetime]

@tchirnhaus20039 a single call into the driver queues hundreds of kernels in a cuda graph, instead of just one in a normal kernel launch

[Clive Chan @itsclivetime]

Improving CPU speed by 10x should not affect training speed essentially at all.

The CPU's main job is to kick off the real work on the GPU. If your kernels are sane (fused etc), the time to launch a kernel on the CPU is <<1% of the kernel runtime, even in Python.

[Elon Musk @elonmusk]

SpaceX has almost finished writing V1.0 of an in-house AI training stack in C that exact-maps to 220k GB300s with 800G NICs, making heavy use of pipeline parallelism and getting as close to bare metal as possible.

The potential speed improvement vs JAX for large training runs is over an order of magnitude.

[Clive Chan @itsclivetime]

@FelixCLC_ @jonmasters @lauriewired @FritzchensFritz interesting!

[Clive Chan @itsclivetime]

@jonmasters @lauriewired @FritzchensFritz what i've always wondered is... does scheduling stuff actually reveal materially interesting ip? why would intel ever care what amd's latencies are, they have their own pipelines to work on

feels like old company habits dying hard

[Clive Chan @itsclivetime]

@xiaosun86 @uncledoomer underrated comment

[Clive Chan @itsclivetime]

@tenderizzation 10 wide ooo cpus are literal witchcraft and i refuse to believe otherwise

[Hieu Pham @hyhieu226]

https://t.co/y0tc2tyjgE 😂

[Clive Chan @itsclivetime]

@hyhieu226 LMAO

[Clive Chan @itsclivetime]

@bubbleboi yes, 100%

elon's post is just about replacing JAX entirely though. which is not the relevant part

[Clive Chan @itsclivetime]

@bubbleboi in the right places certainly yes! replacing JAX is not the right place, replacing NCCL is. this won't get you 10x

&gt; how do you coordinate and schedule compute between nodes efficiently

this is handled by the user (writing code in JAX) for the most part. not something C related

[Clive Chan @itsclivetime]

@jedpolglase i mean these are all pretty straightforward to implement in python no? not sure why C comes into the picture

[Clive Chan @itsclivetime]

@sakurayukiai yep, definitely makes sense for low batch inference. but surely jax already can do cudagraphs, it's been around for many years

[Clive Chan @itsclivetime]

&gt;hundreds of parallel subagents

gonna hit your quota in like 3 seconds

[Claude @claudeai]

Also new in Claude Code: dynamic workflows (research preview).

For the hardest tasks, Claude makes a plan, runs hundreds of parallel subagents, and verifies its work before reporting back. Think a migration touching hundreds of files.

Read more: https://t.co/7gt06kGkDN

[Clive Chan @itsclivetime]

@LiuYunlong63318 yea pipelining is a really good usecase for a compiler. writing the code by hand, especially when supporting multiple pipeline schedules, is nightmare material

[Clive Chan @itsclivetime]

@tenderizzation tpu folks flexing on us cuda plebs

[Clive Chan @itsclivetime]

@William_Blake is 96.5% of that trillion due to anthropic

[Clive Chan @itsclivetime]

@timzaman 10x of the overall training wouldn't happen from that, but yea maybe could 10x the individual comms kernels for some collectives

[Clive Chan @itsclivetime]

@botirkhaltaevv hard to write CUDA C that beats nvidia's own custom assembly kernels in cudnn/etc.