Anikate Kaw

silicon engineer

[Anikate Kaw @anikatekaw]

For everyone interested in finding out where AI silicon is headed it’s really important you understand this.

In inference and many other compute workloads, the challenging part is always moving data around. Doing the actual compute is often trivial. Be it a GPU or AI ASIC, a lot of time is spent in fetching the data, worrying about cache coherency and waiting for the data to be sent out.

This is why it’s important to have memory close to silicon.

There are many techniques used for this. The latest is stacking memory on top of logic. This is not unique to fractile. Many companies are doing this and many companies will continue to do this. D matrix, cerebras and now fracture have all talked about it. I am sure Nvidia will do this too.

So don’t think this is something unique and can’t be reproduced. Most of the heavy lifting here is done by packaging companies and tsmc. Not the actual chip makers.

[Fractile @fractile_ai]

We’re delighted to announce that Fractile has raised $720M in a Series C led by @thrivecapital @foundersfund @lightspeedvp and @redpoint, with further participation by brilliant new and existing investors.

[Anikate Kaw @anikatekaw]

@saranormous @goodwin_ml @fractile_ai It’s already a sea of dram stacking bets - cerebras, d matrix, fracture, tenspresso.

This is the new default

[Anikate Kaw @anikatekaw]

@fractile_ai @ThriveCapital @foundersfund @lightspeedvp @Redpoint Stacking memory over logic is the next big thing!

Everyone’s on it

[Anikate Kaw @anikatekaw]

I think one part of chip design cycle that is never talked about in discussions on speeding up chip design is how compute bound it is.
Spec to gds involves a lot of simulations in verification, synthesis and physical design. Even though the engineering cost will drop
because of AI the compute cost won’t. Solving this is key to a true Cambrian explosion in silicon

[Anikate Kaw @anikatekaw]

@AryaTschand Going through the slides is giving me FOMO! Wish I could be there

[Anikate Kaw @anikatekaw]

For anyone who thinks hardware engineering is software engineering in disguise please see this

[Anikate Kaw @anikatekaw]

@cerebras @toprak_dikici 👏👏👏

[Anikate Kaw @anikatekaw]

For anyone who thinks hardware engineering is software engineering in disguise please see this

[Cerebras @cerebras]

Fearless Engineering: The CS-4 is our first engineering documentary.

Through lab footage and conversations with the engineers behind each breakthrough, the film follows the creation of the CS-4, our latest wafer-scale AI system.

0:00 introducing the engineering team
1:09 Wafer-scale history
2:28 Powering the wafer
4:00 The cooling problem
4:18 Designing the cold plate
5:11 Holes in the wafer
6:33 the alignment problem
7:47 The CS-4 PRD
8:35 Designing for deployability and serviceability
11:01 Removing the motherboard
11:25 Removing internal cooling loops
11:41 inventing the power rack
12:35 A one-person lab deployment
14:03 Backpack: 50% fewer components
14:55 Connecting the compute backpack to cooling manifold
17:58 shifting Cerebras’s focus from training to inference
19:09 Building the Cerebras Cloud
20:01 CS-3 vs. CS-4 performance
20:49 3D Stacked DRAM

[Anikate Kaw @anikatekaw]

@andrewashur https://t.co/Vt8pJ5d820

[Anikate Kaw @anikatekaw]

ways in which hardware engineering is not like software engineering
1. You cannot git revert anything physical
2. Bugs are wayyyy more expensive
3. Verification is more important because you cannot release a patch to change circuits or physical objects
4. You have to deal with physical constraints like area
5. The environment affects your product. Temperature, humidity and cosmic radiation doesn’t cause software issues.
6. You have to deal with a supply chain

This is just off the top of my head there are many more. With vibecoding a lot of people in software land think that hardware is solved too.

This is because a hardware development involves simulations which makes people feel like its software. This is obviously a super surface level take. People who have actually built hardware know it’s not true

[Anikate Kaw @anikatekaw]

@damnang2 Like what? Be specific

[Anikate Kaw @anikatekaw]

@irinatoxi https://t.co/tOghDGEjYp

[Anikate Kaw @anikatekaw]

Chip Design Verification is slowly becoming a purely computer bound problem because of AI. Humans are just overseers.

This will mean that your spec to gdsii timeline will go down exponentially now. Not enough people understand this and are building chatgpt wrappers for chip design.

[Vincent Zhong @vincentzed_cuda]

A chip that’s 3× faster but idle half the day isn’t 3× cheaper:

A prefill or decode device only pays off when your traffic matches the ratio you bought, and that ratio keeps changing. Your GPUs can switch roles when that happens, but specialized silicon just sits idle while you keep paying for it. Software keeps moving the bar too, since megakernels alone have pushed low-concurrency decode on ordinary GPUs up by 40–75% this year, and every one of those gains is another knock on specialized chips.

Consider: “how fast?” but “how fast, times how often it’s actually busy, after moving the KV cache over the network?” If your GPUs run at 70% and their chip averages 40% across a real year of traffic, it needs to be ~1.75× better just to tie. For a single model with very stable traffic (like a premium fast tier), that can work out, but in many case, general-purpose computing can be more useful.

[Anikate Kaw @anikatekaw]

@vincentzed_cuda Is “how often it’s actually busy” same as “throughput”? Or is it some other metric?

Still trying to fully grasp dosage

[Anikate Kaw @anikatekaw]

@baseduart No one is talking about lighting an led on a breadboard

[Anikate Kaw @anikatekaw]

ways in which hardware engineering is not like software engineering
1. You cannot git revert anything physical
2. Bugs are wayyyy more expensive
3. Verification is more important because you cannot release a patch to change circuits or physical objects
4. You have to deal with physical constraints like area
5. The environment affects your product. Temperature, humidity and cosmic radiation doesn’t cause software issues.
6. You have to deal with a supply chain

This is just off the top of my head there are many more. With vibecoding a lot of people in software land think that hardware is solved too.

This is because a hardware development involves simulations which makes people feel like its software. This is obviously a super surface level take. People who have actually built hardware know it’s not true

[Paul Graham @paulg]

"Most hardware engineering is actually software engineering in disguise."

[Anikate Kaw @anikatekaw]

@EERandomness Will - maybe
does it rn - no

[Anikate Kaw @anikatekaw]

@satnam6502 Im tuned! Its super cool to see how much silicon design and verification is changing after spending almost 10 years in this industry

[Anikate Kaw @anikatekaw]

@satnam6502 By data movement I mean when the computed result of a gpu, mmu etc. needs to be transported using a dma which has arbitration rules or industry protocols where it’s not just math but sequential logic

[Anikate Kaw @anikatekaw]

@fcvprzhfgsybj I think because both involve logic and math people think they are similar/same.

Once you look into what it takes to have production hardware vs production software the difference becomes obvious

[Anikate Kaw @anikatekaw]

@fcvprzhfgsybj @zettascale One step from spec to gdsii which can’t be done in Claude code/codex?