Desert Keyboard

Geeking out on tech,AI hardware, investing, & travel just to try great food. Not financial advice. Likes are just likes!

[Desert Keyboard @DesertKeyboard]

@dylan522p Nice setup Dylan and STEEL team! Great to have a new big lab coming online

[Desert Keyboard @DesertKeyboard]

@0xSero @exolabs Is that 4 x 5090’s running in the background?!

[Desert Keyboard @DesertKeyboard]

@semivision_tw @BenBajarin @bubbleboi has $INTC sent you these socks yet?

[Desert Keyboard @DesertKeyboard]

Is this considered AI TokenMaxing or Chipotle BurritoMaxing?

[dax @thdxr]

someone made a fork of opencode that routes through the unsecured ai endpoints from chipotle

[Desert Keyboard @DesertKeyboard]

Damnang posts great deep dives and this post on the optical interconnect ecosystem is a good one to read given AI systems are going CPO/NPO

[Damnang2 @damnang2]

https://t.co/6hYI1f7pTD

[Desert Keyboard @DesertKeyboard]

@insane_analyst I’m hopeful, but doesn’t the added latency still cause idle XPUs?

[Desert Keyboard @DesertKeyboard]

@chamath When CEOs hire a PwC type consulting firm vs. an AI lab, isn’t CYA aka “the consultant confirmed this was a great idea” a big reason the CEO still would want to hire a consulting firm?

[Desert Keyboard @DesertKeyboard]

Chip Packaging examples from 1950’s to 2000’s to now…connections get smaller and then chips go burrrrr faster

[Pushkar Ranade @magicsilicon]

7 decades of semiconductor packaging!
From 2D to 3D.
From wire bonding to direct Copper-Copper bonding.
From millimeters to nanometers.

[Dwarkesh Patel @dwarkesh_sp]

Did a very different format with @reinerpope – a blackboard lecture where he walks through how frontier LLMs are trained and served.

It's shocking how much you can deduce about what the labs are doing from a handful of equations, public API prices, and some chalk.

It’s a bit technical, but I encourage you to hang in there - it’s really worth it.

There are less than a handful of people who understand the full stack of AI, from chip design to model architecture, as well as Reiner. It was a real delight to learn from him.

Recommend watching this one on YouTube so you can see the chalkboard.

0:00:00 – How batch size affects token cost and speed
0:31:59 – How MoE models are laid out across GPU racks
0:47:02 – How pipeline parallelism spreads model layers across racks
1:03:27 – Why Ilya said, “As we now know, pipelining is not wise.”
1:18:49 – Because of RL, models may be 100x over-trained beyond Chinchilla-optimal
1:32:52 – Deducing long context memory costs from API pricing
2:03:52 – Convergent evolution between neural nets and cryptography

[Desert Keyboard @DesertKeyboard]

@0xSero What’s your hardware/workstation setup for this as an individual lab? Threadripper for 6x PCIE 5 and multi-PSU?

[Desert Keyboard @DesertKeyboard]

@0xSero @huggingface How about best model at the 32Gb /Nvidia 5090 or at the 48GB unified memory Mac M4 Pro level?

[Desert Keyboard @DesertKeyboard]

@Alisvolatprop12 @grok can you translate this to English?

[Brian Armstrong @brian_armstrong]

How long until agentic payments overtake human payments?

[Ben Bajarin @BenBajarin]

The new system of compute from NVIDIA.

Extreme co-design = 35X more throughput per MW.

[Desert Keyboard @DesertKeyboard]

@claudeai @threadreaderapp unroll

[Desert Keyboard @DesertKeyboard]

Great comment from BG on where AI software vs. hardware opportunity may land in the long run…and how new open source voice models are changing business lines

[Bill Gurley @bgurley]

People really don’t understand what a competitive strategic weapon open source has become and how it works. Alfred Marshall would be proud. Always good to reread the cathedral and bazaar. https://t.co/Il8BsDuDeI

[Desert Keyboard @DesertKeyboard]

@Austen Yep those API calls will get expensive/likited. Tried running a local model yet? Maybe time for a Mac Studio 128GB+ of RAM?

[Desert Keyboard @DesertKeyboard]

Great take on new tool and tech productivity boosts from @karpathy (Ex-OpenAI and Tesla)

[Andrej Karpathy @karpathy]

I've never felt this much behind as a programmer. The profession is being dramatically refactored as the bits contributed by the programmer are increasingly sparse and between. I have a sense that I could be 10X more powerful if I just properly string together what has become available over the last ~year and a failure to claim the boost feels decidedly like skill issue. There's a new programmable layer of abstraction to master (in addition to the usual layers below) involving agents, subagents, their prompts, contexts, memory, modes, permissions, tools, plugins, skills, hooks, MCP, LSP, slash commands, workflows, IDE integrations, and a need to build an all-encompassing mental model for strengths and pitfalls of fundamentally stochastic, fallible, unintelligible and changing entities suddenly intermingled with what used to be good old fashioned engineering. Clearly some powerful alien tool was handed around except it comes with no manual and everyone has to figure out how to hold it and operate it, while the resulting magnitude 9 earthquake is rocking the profession. Roll up your sleeves to not fall behind.

[Desert Keyboard @DesertKeyboard]

@BenBajarin @BenBajarin is it odd if the goal was an acqui-hire type deal? Been seeing more of these in the software space in SV, maybe time it comes to chips

[Desert Keyboard @DesertKeyboard]

AI inference inference deals seems to ramping up….End of year licensing for cool tech stack from Groq

[Jonathan Ross @JonathanRoss321]

Today Groq entered into a non-exclusive licensing agreement with Nvidia for Groq’s inference technology. Along with other members of the Groq team, I’ll be joining Nvidia to help integrate the licensed technology. GroqCloud will continue to operate without interruption.

Learn more here: https://t.co/1Lgv1EKZNH