Kenny Guo

maximizing entropy

[Kenny Guo @kennykgguo]

introducing tiny-tpu, a tiny TPU, supporting both training and inference, ENTIRELY on chip!

here is an overview of how I got started and how this was built:

[Ryo Lu @ryolu_]

efficiency, productivity, speed

we keep worshipping these words like they are what we live for

ship faster
merge more
manage more agents
compress every loop
remove every pause
turn every hour into output

but what is the point of endless production if there is no time left to think deeply?

not react
not optimize
not summarize
not clear the queue

think

to sit with a question long enough that your real intention starts to appear
to find the purpose underneath the motion
to ask whether the thing being accelerated should exist at all

ai could be a new canvas

a way to make strange things
personal things
impossible things
tools, worlds, poems, games, interfaces, films, languages, rituals
things that let more people touch the shape of their own imagination

but instead, so much of it is becoming slop factories

more content
more funnels
more tickets
more fake work pretending to be value

you can merge 2000 prs
you can manage 500 agents
you can wake up to dashboards proving that the machine kept moving while you slept

but what is the point if you only sleep 5 hours a day?

what is the point if you stop talking to other humans?
stop learning about the past?
stop playing?
stop wandering?
stop being surprised?

what is the point if your whole life becomes optimizing the system that is consuming you?

speed is not meaning
scale is not wisdom
automation is not freedom if it only makes everyone run faster

and where are we running?

toward what?

a world where every person becomes a manager of machines, and every machine produces more things no one had the time to truly want?

the danger is not that ai makes us lazy

the danger is that ai makes us endlessly busy

that it gives us infinite production before we have found intention
infinite execution before we have developed taste
infinite motion before we have learned how to be still

maybe the real frontier is not more speed

maybe it is discernment

knowing what not to make
knowing when to stop
knowing when the most productive thing is to sleep, read history, call a friend, play with a cat, walk without tracking it, or stare at the ceiling until the false urgency dissolves

tools should give us more life
not turn life into a tool

if ai is going to matter, it should not just help us produce more

it should help us become more human
more imaginative
more free

otherwise, it’s the same old factory
just without humans

[Kenny Guo @kennykgguo]

@MankyDankyBanky @danielchingwq @bracketbot @chinmayjindal_ bro i literally called it in the video 😂

[Kenny Guo @kennykgguo]

@aditbhargava12 @JaneStreetGroup 👏👏

[Adit Bhargava @aditbhargava12]

I reverse engineered a chip from just its layout.

@JaneStreetGroup published a GDS file with no functional names on anything. feed it the right 121 bits and it prints a string.

here's how I got to the answer.

1/14

[DHH @dhh]

Omarchy now has the coolest website I've ever seen. Truly incredible work by the Omarchy website design team lead by @BarisGirismen. This is what computers can look (and sound) like! https://t.co/2Bb7BOA74N

[Aditya Makkar @AdityaMakkar000]

New blog! With @_DivyaMakkar we present a worklog on how to build an asynchronous RL framework from scratch. It covers multi-actor systems in JAX, inference, distributed training, actor coordination, ICI weight sync and more! Additionally we share interesting JAX challenges encountered along the way and how you should think of async RL from a systems perspective!

[Divya Makkar @_DivyaMakkar]

Spent some time creating a lightweight, performant async RL stack in pure JAX with @AdityaMakkar0! We share a work log covering inference, RDMA weight transfer, memory optimizations, sharding and a few more JAX engineering insights we've seen along the way! It’s a bit of a long read, but I hope it sheds some light on the systems side of scaling RL!

[michael.trbo @michael_trbo]

@ferdousbhai sorry to say but you're wrong 😂

Western University represent!!

[Kenny Guo @kennykgguo]

check out https://t.co/8rHZIGu6Yk!!

[saksham @sakshambatraa]

we didn't know how to design a chip a couple months ago

so we reinvented Groq's LPU to run inference on @karpathy 's MicroGPT!

here's a breakdown of what we learned:

[Kenny Guo @kennykgguo]

@arjunharinath1 @sakshambatraa @michael_trbo GOAT APPROVED 🐏🐏🐐

[Kenny Guo @kennykgguo]

@michael_trbo @sakshambatraa @arjunharinath1 THE GOATS HAVE SPOKEN

[Kenny Guo @kennykgguo]

@sakshambatraa @karpathy @arjunharinath1 @michael_trbo GOATEDDD🐏🐏🐏very proud of u guys

[arjun @arjunharinath1]

I worked with @sakshambatraa and @michael_trbo this summer to build LPULite

We reinvented Groq's Language Processing Unit completely from scratch to run inference on Karpathy's MicroGPT!

Read the full deep dive here: https://t.co/zPJA1sliWA

[michael.trbo @michael_trbo]

over the last 6 months I spent my time building https://t.co/PxQeUIK68k !

our reinvention of Groq's LPU that supports full forward pass on our custom transformer MicroGPT model

take a look at how we did it 👇

[saksham @sakshambatraa]

we didn't know how to design a chip a couple months ago

so we reinvented Groq's LPU to run inference on @karpathy 's MicroGPT!

here's a breakdown of what we learned:

[Ankit Gupta @agupta]

Moderna is a great example of what happens when people are bold enough to ignore the haters and invest in technology platforms.

In 2019 this company was widely assumed to die. But in COVID they likely saved several tens of millions of lives, bought themselves a financial lifeline to go after cancer, and now they’re going to save many million more.

Incredible (Cambridge based!) company

[Progyan @plugyawn]

I spent the last six months trying to deconstruct Taalas's patents. We think it can be better: 100x fewer memfetches than their bitROM, better software by better quantization than their hardware team could.

So we wrote a compiler that:
> takes a huggingface checkpoint,
> quantizes the model
> descends the weights down metal layers to RTL and GDS, do your DRC, Yosys, PEX, make the electrical waveform execute a mat-vec from your huggingface checkpoint (thanks Cambricon tech papers)

in ~7000 lines of human-readable code.

we're looking for someone with contacts with a foundry/access to 7nm PDKs or contacts at a cheap EuroPTW shuttle? I'm broke and unemployed.

@itsclivetime this is what i think is the future of perplexity per picojoule. @zerohedge @zephyr_z9 you guys wanna see 40,000 tokens/sec?

[Kenny Guo @kennykgguo]

@anirudhbv_ce congrats !!📈📈📈

[arjun @arjunharinath1]

Reinventing Groq's LPU with @sakshambatraa and @michael_trbo . Recently we transitioned from INT precision to floating points.

Here's a breakdown of how we did it:

[Victor Li @victor_ljz]

We will be at ICML in Seoul to present dnaHNet in Oral + Poster sessions! Swing by and say hi if you wanna learn about the current SOTA in DNA foundation models :)

Oral session: Hall D2 at 10 AM, Jul 7
Poster session: Hall A #900 at 2 PM, Jul 7