𝚓𝚎𝚜𝚜𝚎𝚕

devAI @ meta

[𝚓𝚎𝚜𝚜𝚎𝚕 @0xjessel]

☺️

[Austen Allred @Austen]

Yes I lost this one fair and square.

Send me your Venmo or bank info.

[𝚓𝚎𝚜𝚜𝚎𝚕 @0xjessel]

@ricefield aka english lit 🥀

[David Hoang @davidhoang]

Millennials were built for Agent Orchestration.

[Hasnain Lakhani @mhlakhani]

Today I learned Meta has an LLM-agent bot (called Devmate) that looks at test failures, diagnoses them, and puts up diffs to fix them.

People have been talking about AI doing these types of workflows and whether it's practical or not for quite some time...

1/N

[Haider. @haider1]

🚨 Sam Altman predicts,

software engineering by the end of 2025 looks very different than software engineering at the beginning of 2025

[Yishan @yishan]

I think the Deepseek moment is not really the Sputnik moment, but more like the Google moment.

If anyone was around in ~2004, you'll know what I mean, but more on that later.

I think everyone is over-rotated on this because Deepseek came out of China. Let me try to un-rotate you.

Deepseek could have come out of some lab in the US Midwest. Like say some CS lab couldn't afford the latest nVidia chips and had to use older hardware, but they had a great algo and systems department, and they found a bunch of optimizations and trained a model for a few million dollars and lo, the model is roughly on par with o1. Look everyone, we found a new training method and we optimized a bunch of algorithms!

Everyone is like OH WOW and starts trying the same thing. Great week for AI advancement! No need for US markets to lose a trillion in market cap.

The tech world (and apparently Wall Street) is massively over-rotated on this because it came out of CHINA.

I get it. After everyone has been sensitized over the H1BLM uproar, we are conditioned to think of OMG Immigrants China as some kind of Alien Other. As though the Alien-Other Chinese Researchers are doing something special that's out of reach and now China The Empire is somehow uniquely in possession of Super Efficient AI Power and the US companies can't compete. The subtext of "A New Fearsome Power Now Under The Command of the CCP" is what's driving the current sentiment, and it's not really valid.

Like, no. These are guys basically working on the same problems we are in the US, and not only that, they wrote a paper about it and open-sourced their model! It is not actually some sort of tectonic geopolitical shift, it is just Some Nerds Over There saying "Hey we figured out some cool shit, here's how we did it, maybe you would like to check it out?"

Sputnik showed that the Soviets could do something the US couldn't ("a new fearsome power"). They didn't subsequently publish all the technical details and half the blueprints. They only showed that it could be done.

With Deepseek, if I recall correctly, a lab in Berkeley read their paper and duplicated the claimed results on a small scale within a day.

That's why I say it's like the Google moment in 2004. Google filed its S-1 in 2004, and revealed to the world that they had built the largest supercomputer cluster by using distributed algorithms to network together commodity computers at the best performance-per-dollar point on the cost curve.

This was in contrast to every other tech company, who at that time just bought what were essentially larger and larger mainframes, always at the most expensive leading edge of the cost curve. (To the young people reading this, this will sound incredible to you)

I worked at PayPal at the time, and in order to keep pace with the rising transaction volume, the company was forced to buy bigger and bigger database servers from Oracle. We were totally Oracle's bitch. At one point when we ran into scalability issues, the Oracle reps told us we were their biggest installation so they had no other reference point on how to help us overcome our scalability issues. We literally resorted to flipping random config switches and rebooting it.

(This heavily influenced me when I was a young manager later at Facebook. I deliberately torpedoed an Oracle salesman's pitch to try and get us to switch from open source MySQL databases to an Oracle contract: of course we had scalability problems, but at least when we had them, we could open up the hood and figure out how to fix it ... assuming we had good enough engineers, and we did. When it's closed-source infra, you're at the mercy of the vendor's support engineers)

Back to Google - in their S-1, they described how they were able to leapfrog the scalability limits of mainframes and had been (for years!) running a far more massive networked supercomputer comprised of thousands of commodity machines at the optimal performance-per-dollar price point - i.e. not the more expensive leading edge - all knit together by fault-tolerant distributed algorithms written in-house.

Some time later, Google published their MapReduce and BigTable papers, describing the algorithms they'd used to manage and control this massively more cost-effective and powerful supercomputer.

Deepseek is MUCH more like the Google moment, because Google essentially described what it did and told everyone else how they could do it too. In Google's case, a fair bit of time elapsed between when they revealed to the world what they were doing and when they published a papers showing everyone how to do it. Deepseek, in contrast, published their paper alongside the model release.

Now, I've also written about how I think this is also a demonstration of Deepseek's trajectory, but that's also no different from Google in ~2004 revealing what it was capable of. Competitors will still need to gear up and DO the thing, but they've moved the field forward. But it's not like Sputnik where the Soviets have developed technology unreachable to the US, it's more like Google saying, "Hey, we did this cool thing, here's how we did it."

There is no reason to think nVidia and OAI and Meta and Microsoft and Google et al are dead. Sure, Deepseek is a new and formidable upstart, but doesn't that happen every week in the world of AI? I am sure that Sam and Zuck, backed by the power of Satya, can figure something out. Everyone is going to duplicate this feat in a few months and everything just got cheaper. The only real consequence is that AI utopia/doom is now closer than ever.

====

Bonus: This is also a little similar the Ethereum PoS moment, when AI finally has a counterpoint to the environmentalists who say AI uses so much electricity. We just brought down the cost of inference by 97%!

[Andrej Karpathy @karpathy]

We have to take the LLMs to school.

When you open any textbook, you'll see three major types of information:

1. Background information / exposition. The meat of the textbook that explains concepts. As you attend over it, your brain is training on that data. This is equivalent to pretraining, where the model is reading the internet and accumulating background knowledge.

2. Worked problems with solutions. These are concrete examples of how an expert solves problems. They are demonstrations to be imitated. This is equivalent to supervised finetuning, where the model is finetuning on "ideal responses" for an Assistant, written by humans.

3. Practice problems. These are prompts to the student, usually without the solution, but always with the final answer. There are usually many, many of these at the end of each chapter. They are prompting the student to learn by trial & error - they have to try a bunch of stuff to get to the right answer. This is equivalent to reinforcement learning.

We've subjected LLMs to a ton of 1 and 2, but 3 is a nascent, emerging frontier. When we're creating datasets for LLMs, it's no different from writing textbooks for them, with these 3 types of data. They have to read, and they have to practice.

[low yield lucy @picotop]

you are not allowed to wear Carhartt if you know what KPI means.

[Kingsley Uyi Idehen @kidehen]

This time around, its a reply from my #Mastodon account to @0xjessel's #Threads account. It just works!

The #Fediverse has taken a while to put together, but it will ultimately end the myopic reign of frustration unleashed by old platforms like @X.

#OpenWeb #SocialMedia

[Denny Klisch @KiwiDenny]

Best MVP description I have ever seen.

[𝚓𝚎𝚜𝚜𝚎𝚕 @0xjessel]

chat are we cooked

[Ricky @rickyrobinett]

What can an 8-year-old build in 45 minutes with the assistance of AI?

My daughter has been learning to code with @cursor_ai and it's mind-blowing🤯

Here are highlights from her second coding session. In 45 minutes she built a chatbot powered by @CloudflareDev Workers AI 👀

[𝚓𝚎𝚜𝚜𝚎𝚕 @0xjessel]

@bioreconstruct @wheelsee @instagram @bioreconstruct are you not able to sign in on instagram or just threads?

[𝚓𝚎𝚜𝚜𝚎𝚕 @0xjessel]

Do you remember when you joined X? I do! #MyXAnniversary

[vjeux ✪ @Vjeux]

It's been super inspiring to see Threads being developed so quickly internally. This is a sneak peak into what was required to do it :)
https://t.co/dNep7y2Waf

[Engineering at Meta @Meta_Engineers]

With today’s roll out of keyword search on Threads to countries where most people post in English or Spanish, Jesse Chen, engineering manager at Meta, shares the journey of Threads thus far, and what lies ahead for Meta’s newest app.
https://t.co/sfueWaEuuF

[Gergely Orosz @GergelyOrosz]

Learned that the Threads app was built in 5 months (!!), project started with maybe a dozen (senior) engineers. On launch week, it hit 100M downloads / new users. This speed is... very fast.

More details from the Threads engineering team coming in today's @Pragmatic_Eng issue.

[vitalik.eth @VitalikButerin]

What do I think about Community Notes?

https://t.co/9A0PnanndM

[𝚓𝚎𝚜𝚜𝚎𝚕 @0xjessel]

💯

[Peter Yang @petergyang]

All else being equal, the team that moves faster will win.

So how fast are you shipping and learning?

From the book Working Backwards:

[𝚓𝚎𝚜𝚜𝚎𝚕 @0xjessel]

@nikitabier what app is #1?

[𝚓𝚎𝚜𝚜𝚎𝚕 @0xjessel]

@alex193a 😂😅