Last Year We Built Megakernels. This Year We Deleted Them.
One year ago, we helped introduce some of the first AI megakernels to the world. Since then, we’ve enjoyed seeing their wide adoption from friends and researchers. However, while whole model megakernels are a helpful abstraction, we’ve found that they overcomplicate and are not always an ideal optimization target.
Below, we discuss our journey to understand and push past these limitations. We start with a deep dive into two contrasting paradigms of ML optimization – Torch Compile and Megakernels
Plaid is about to become a lot more valuable once the market realizes it is quietly becoming the context layer for all useful agentic finance applications
we will need to rebuild and rewrite much of software to adapt to a new, faster pace.
code used to arrive one drop at a time. soon, if not already, code will be written as if it were a continuous stream, like a violent river twisting and turning as it pleases.
this is not a new thing: big tech cos have had to deal with this with the enormous scale of their engineering teams. but now ai has brought an incredibly elastic and capable workforce wielded by every engineer. this is your problem now, and the only way to adapt is to evolve your systems so agents - even and especially the ones with little context about the right way of doing things - can thrive. make the easy thing the right thing.
your codebase is becoming organic architecture. agent contributions grow around the constraints and guardrails you’ve designed. good software engineering matters even more now! and no, the continuous stream of code is not something you have to watch helplessly. be more ambitious and design systems, frameworks, and languages to control the chaos.
the time for mourning coding is long over
I almost didn't get hired at Dropbox. I did well in some parts of the interview, not so well in others, and a couple of people had to go to bat for me. Then I had to do a take home. And then, I think I just wore them down and they hired me.
That job ended up changing my career.
That's why I've been a little skeptical of how much you can realistically learn about someone in a couple of hours of interviews.
I've become more willing to take chances on people. If someone seems smart, curious, hungry and has a track record of building things, I want to give them a shot and see how they do.
https://t.co/JMMhHNC4QS
Hi all, I have a new result I'm excited to share. It turns out that *all existing implementations* of softmax and cross-entropy (that's JAX, PyTorch, textbooks, etc) have a subtle bug --- two bugs, actually. A simple rewrite fixes both, and eliminates "slingshots" when grokking:
arXiv paper: https://t.co/bwZo3IZr65
HTML version of the arxiv paper: https://t.co/AykiAWT5C7
The interactive widget for the video in the first post: https://t.co/B57vLPBERp
Jean-Denis Greze, founder of @TownAI, on the question every AI app founder gets asked: what stops the big labs from crushing you?
"It is risky always to be competing with your suppliers, right? We're competing against our suppliers. So you gotta ask yourself, is that naturally the right state of the world?"
"Someone at OpenAI, at Anthropic, at Google could wake up and decide to do that. So the better answer actually is to think about our customers and what they want."
"It is our job to determine from all the AI capabilities, frontier model, open weight, what is the right thing for you as a business. And I think you can save money there, but also you can de-risk ever having made the wrong partner."
"There are companies that picked OpenAI like two years ago, and then Anthropic was better at coding for a 12-month period. And those companies had to use both because they realized to be competitive, they needed Claude."
"If you're a smart business, you really think about the neutrality of who's providing intelligence for you and who's routing."
@jgreze@arampell