Kalpak

Distributed Systems Engineer at Amazon Web Services.

[Kalpak @KalpakSeal]

@AnjneyMidha @kukreja_abhinav If and when

[Kalpak @KalpakSeal]

@kunchenguid IPO baby 🍼🍼

[Kalpak @KalpakSeal]

@_mohansolo I really didn't like this messing around with my workflow and all the shenanigans.

Please think through the experience for the next releases

[Kalpak @KalpakSeal]

@KevinNaughtonJr Prozac

[Kalpak @KalpakSeal]

@championswimmer It's all in the momentum

[Kalpak @KalpakSeal]

@AnjneyMidha @Goltra Shall i post this then??

[Kalpak @KalpakSeal]

@BenjDicken Lets do something around learning Golang please 🥺

[Charles 🎉 Frye @charles_irl]

👀

https://t.co/SfNUbRSXT0

[Kalpak @KalpakSeal]

@arpit_bhayani Somewhere, some how, AGI vanished like potholes before election.

[Soumith Chintala @soumithchintala]

The Jensen + @dwarkesh_sp podcast was fantastic.
Jensen is someone who understood how ecosystems work and someone who understands real-world trade, policy and controls work. And in some deeper sense how AI will actually diffuse into the world.
In this podcast, Dwarkesh came off as someone who picked up talking points from an AGI party in the SF Mission District.
And the contrast was so evident.
As someone who understood ecosystems relatively deepy, maybe I understood Jensen's take more than others did (idk).

Mythos, that Dwarkesh kept bringing up, is not a single absolute turning point in the AI development landscape. Take a state-of-the-art Chinese open-source model, and give it three orders of magnitude more test-time compute + post-training algorithmic advances that haven't been published yet. That's the baseline. It was evident that in whatever bubble Dwarkesh is in, that is seen as a naive or illogical baseline.

When AI has such a complex development cycle, it's evident that America needs many levers of policy intervention across multiple layers in a dominant ecosystem that ideally the Western world controls.

The entire premise that a particular model with AI development will have a critical phase change is neither correct nor does evidence point to it. OpenAI made this point with GPT-4, Anthropic made this point with Mythos, but neither stood / will stand the test of time.

I think Jensen's repeated emphasis within the podcast to try to make this point mostly didn't get Dwarkesh's attention. And Dwarkesh (in this podcast) represents an entire cult of AI researchers and decision-makers that are going to influence policy.

The thing with policy interventions is that if you do too much too early, you shoot yourself in the foot. There's a good reason American foreign policy and general sanctions of all kinds are measured and continuous.

Despite Jensen's attempt at educating the "Anthro" audience how ecosystems work, I'm also not super hopeful a lot of people who've taken the extreme position will change their thought after listening to this podcast. I do think there's a certain religiousness that has permeated some of that community that would make it hard to understand ecosystems at a deeper level.

[Kalpak @KalpakSeal]

@joecarlsonshow Zuck over Satya ?

[Kalpak @KalpakSeal]

@sriramk @dwarkesh_sp The podcast, nonetheless, is so well done. It would stand the test of time, imo.

Podcasts should aim for such exchange of thoughts.

[Abhinav Upadhyay @abhi9u]

Found a neat use of the min-cut algorithm inside torch.compile.

During training, the compiler captures both forward and backward graphs.

The backward pass needs certain activations from the forward pass to compute gradients. The obvious option is to simply save all such tensors for the backward pass. But for large models this can be memory-intensive.

So the compiler decides which activations to save and which to recompute during backward.

PyTorch uses a min-cut based partitioning on the joint forward+backward graph to decide the minimum set of activations to save, recomputing the rest during the backward pass.

[Kalpak @KalpakSeal]

@Yuchenj_UW @jenzhuscott Intuit-ive 😅

[Kalpak @KalpakSeal]

@aiexplorations @AnjneyMidha Beyond that, I mostly find it to be folks with capital taking strategic bets on people based on network effects .

[Kalpak @KalpakSeal]

@aiexplorations @AnjneyMidha Expertise in the AI space is often a blue / red pill situation.

Expertise is required & found at the model's training & inference layer and in the stack below it (chip design, infrastructure , power management , etc. ).

People with credentials for building them over year.

[Kalpak @KalpakSeal]

@kirillk_web3 Thanks for sharing.

[Kalpak @KalpakSeal]

@AnjneyMidha @aiexplorations Isn't executions just capital allocation ( bets and hedges) + accountability (proof of work ).

Leadership subsequently becomes a ceremonial robe.

[krish @IamIronLAN]

Super excited to watch this!! @michael_nielsen is one of my favorite people on the internet.

He’s a physicist who has been my go to to read about deep learning, blockchain, spaces recognition, how to read papers/math, what deep creative work looks like, what science and open science look like and ofc the very little I know about quantum computing.

He’s collaborated and introduced me to so many thoughtful people I also love including:
- Chris Olah (@ch402)
- Andy Matushak (@andy_matuschak )
- Kanjun (@kanjun)
- Devon (@devonzuegel)
- P Collison (@patrickc)

He’s been my def for what an independent researcher looks like.

[Kalpak @KalpakSeal]

@arpit_bhayani He's Arpit. Period. Full stop .