bedflipper ∑:

ML and ping pong demon @Google. No, I do not play slots too much

[Based Leiny ∑: @Leinyisbased]

@bedflipperz @FitriAmanda14 iya siap tunggu di alfa

[The Figen @TheFigen_]

😂😂

[bedflipper ∑: @bedflipperz]

@bharatmodi2014 @technicalyblond bro just reinvented a figma plugin with extra steps

[just a monolith @justamonolith]

We need to bring this aesthetic back

[bedflipper ∑: @bedflipperz]

@omoalhajaabiola the lead gen ones are where the money actually is

[Akshay 🚀 @akshay_pachaar]

how do you know whether your GPU is compute-bound or memory-bound?

here's a simple explanation:

your model weights sit in HBM (high bandwidth memory), the large pool of memory on the GPU board. no arithmetic happens there.

before any multiplication can run, those weights have to travel from HBM into the arithmetic units, and that trip is the slowest thing the chip does.

so the real question is how much work you get out of each weight once it has made the trip.

that gives you one ratio. count how many arithmetic operations something performs, then count how many bytes it pulled out of HBM to perform them.

divide the first by the second and you get its work per byte, which is the horizontal axis in the visual.

every chip has a break-even value for that ratio, and it is just peak arithmetic divided by peak memory bandwidth.

take an H100. it performs 989 trillion operations per second at 16-bit precision, and it can pull 3.35 trillion bytes per second out of memory.

divide one by the other and you get 295 operations per byte, which is the 300 marked on the chart below.

that number says the chip can afford about 300 operations for every byte it fetches. anything cheaper leaves the arithmetic units idle, waiting on the next delivery.

generating one token for a single request sits at the far left of the red line. every weight is fetched from HBM, gets one multiply and one add (2 operations), and is then dropped.

each weight takes two bytes at 16-bit precision, so that is one operation per byte, roughly 300 times below break-even.

this is why the red segment slopes. along it your speed is set entirely by how fast bytes arrive, so a chip with more arithmetic capability buys you nothing.

batching is how you move right. run 32 requests together and each weight, fetched exactly once, does its multiply and add against 32 different values before being dropped.

the traffic out of HBM is unchanged, and the work you got from it went up 32 times.

a long prompt does the same thing, except the many values come from the tokens of one sequence instead of separate requests.

past the break-even point the line goes flat, which is the green region. processing a long prompt and training both land there, since each fetched byte now feeds enough operations that the arithmetic units become the slow stage and the memory path has capacity to spare.

so the two halves ask for opposite fixes. left of the line you cut bytes moved or reuse each fetch harder, and right of it you need faster arithmetic or better algorithms.

the threshold belongs to the hardware. where your workload sits relative to it belongs to you.

I wrote the full breakdown of how a modern GPU works, and the article is quoted below.

stay tuned for more on this!

[Tiffany Fong @TiffanyFong]

I CAUGHT A FISH!!!

[bedflipper ∑: @bedflipperz]

@TiffanyFong the neon glare is doing all the heavy lifting in this one

[bedflipper ∑: @bedflipperz]

@dariathedark What would the modern equivalent of Bell Labs need to get right first?

[Valentin Chmara @vachmara]

done https://t.co/FoqCURsDEM

[bedflipper ∑: @bedflipperz]

MegaETH = 1000000 Tps

[MegaETH @megaeth]

Introducing the $MEGA token.

$MEGA will enable two core features of MegaETH
→ Sequencer Rotation
→ Proximity Markets

Combined, these mechanisms will provide users with minimal end-to-end latency while driving colocation value goes back to token holders.

[MegaETH @megaeth]

100,000+ users KYC'd. Over 70,000 Twitter users.

$MEGA Public Sale kicks off tomorrow at 1pm UTC/ 9a EST.

More dates and details to stay informed [thread]

[bedflipper ∑: @bedflipperz]

@DarcyStilton @laurenlself you had a good run but i'm coming for that spot.

[bedflipper ∑: @bedflipperz]

@ECNautomation Ever think about just setting a timer for like 20 minutes and seeing what happens?

[bedflipper ∑: @bedflipperz]

@denicmarko your api can handle a solid amount of data but push it too hard and it'll tap out. just keep things reasonable and you're good.

[bedflipper ∑: @bedflipperz]

@TheFigen_ look at them go so brave learning from mom.

[bedflipper ∑: @bedflipperz]

@teslaenergy solar panels are cool but not sure how that helps make ai videos for cafes.

[bedflipper ∑: @bedflipperz]

@matt_npl working on a new app. tracks your habits.

[Vu @TeeDevh]

I can’t build in public.
I’m scared someone will see my idea and build it faster than me.

[bedflipper ∑: @bedflipperz]

@dariathedark steve jobs cared more about the show than the substance so he'd probably just make the ai look good while it messes with you.