erbear

Heretic

[Prakash @8teAPi]

Zuck on Dwarkesh

TLDR: AI winter is here. Zuck is a realist, and believes progress will be incremental from here on. No AGI for you in 2025.

1) Zuck is essentially an real world growth pessimist. He thinks the bottlenecks start appearing soon for energy and they will be take decades to resolve. AI growth will thus be gated on real world constraints.
> "I actually think before we hit that, you're going to run into energy constraints. I don't think anyone's built a gigawatt single training cluster yet. You run into these things that just end up being slower in the world."
> "I just think that there's all these physical constraints that make that unlikely to happen. I just don't really see that playing out. I think we'll have time to acclimate a bit."

2) Zuck would stop open sourcing if the model is the product
> "Maybe the model ends up being more of the product itself. I think it's a trickier economic calculation then, whether you open source that."

3) Believes they will be able to move from Nvidia GPUs to custom silicon soon.
> "When we were able to move that to our own silicon, we're now able to use the more expensive NVIDIA GPUs only for training. At some point we will hopefully have silicon ourselves that we can be using for at first training some of the simpler things, then eventually training these really large models."

Opinion
Overall, I was surprised by how negative the interview was.
A) Energy - Zuck is pessimistic about the real world growth necessary to support the increase in compute. Meanwhile the raw compute per unit energy has doubled every 2 years for the last decade. Jensen also is aware of this, and it beggars belief that he does not think of paths forward where he has to continue this ramp.
Over at https://t.co/mzPa6JREGw, the launch announcement read "At xAI, we have made maximizing useful compute per watt the key focus of our efforts."
So energy efficiency, algorithmic and otherwise are obvious areas where firms will be focused on. Zuck meanwhile is planning to move of Nvidia chips soon, and basically believing that the value in AI available to having a share of the most advanced compute clusters will declline.

B) AGI Negative
Zuck fundamentally
> does not believe the model, the AI itself, will be the product.
> It is the context, the network graph of friendships per user, the moderation, the memory, the infrastructure that is the product.
> Allows him to freely release open source models, because he has all of the rest of the pieces of user facing scaffolding already done.

An actual AGI
> where the a small model learns and accompanies the user for long periods
> while maintaining its own state
> with a constitution of what it can or cannot do
> rather than frequent updates from a central server
> would be detrimental to Meta’s business,
> would cause a re-evaluation of what they are doing

C) Summary
Zuck
> essentially settled into the trap of believing in incrementalism
> Advised by the smartest people in the world.
> Technically competent
> But he does not believe in states of the world where a 100x improvement from GPT-4 are possible, or that AGI is possible within a short timeframe.
> But he's also… he’s not raising capital.

The three people who are raising capital: Sam Altman, Elon Musk and Dario Amodei, are all on record expecting dramatic increases in capability. They could be hyping because they need higher valuations. I don’t know.

And so again we wait for GPT-5. If it is 10-100x as good as GPT-4, the current benchmarks won’t even work (how does one measure 100x as good on an MMLU scale of 1-100 ?).

If the models deliver value, and deliver large amounts of value exceeding manifold the capital deployed to develop them, then progress will continue. If not, not.

For me, the most exciting part of all of this is.. 🍿🍿🍿 drama. You get to see who is bluffing, with billions of dollars on the line, on a fairly short timeframe. And I, for the most part, am in it for sheer entertainment value in spectating potentially the greatest game mankind has ever played.

Happy Friday 🥂

I’d like to congratulate Dwarkesh for an excellent interview… and for not great ad reads as well 😜

The above was excerpted from my daily AI newsletter, full edition goes out Monday.

Sign up below!

[Michal Oblak @michaloblak]

#polskiprzemysl bez przestojów. 💪 Frekwencja pracowników produkcyjnych na poziomie 63% według #SmartLunchIndex ➡ https://t.co/tchkAuZbvC

#koronawiruspolska #CoronaVirusUpdate #koronawirus @DziennikPL @gazeta_wyborcza @Money_pl @PracodawcyRP @PolskiPrzemysl

[erbear @erbbear]

Grow the Puzzle Around You https://t.co/KZeAqmSBw0

[erbear @erbbear]

Amazon Strategy Teardown: Amazon’s Barreling Into Physical Retail, Financial Services, Healthcare, And AI-Led Computing https://t.co/v1IgnwGu8O

[Patrick Collison @patrickc]

I'd pay a *lot* more for books if I could see the highlights, annotations, and marginalia of friends or people I follow.

If books really do matter in the world, feels like we'd benefit from a lot more reading technology. Getting books onto the screen was a good *first* step.

[Michal Oblak @michaloblak]

QUESTION: How many transactional emails is your SaaS sending per month, and how many users do you have?
It feels like Intercom/Drip pricing per user and not per sent email is kind of broken.

[erbear @erbbear]

@michaloblak Enjoy the nature not a screen! Have fun!

[Michal Oblak @michaloblak]

Don't predict your future revenue, until you've discovered a *really* repeatable sales process. https://t.co/gGuG7gr0l1

[Neil Cremins @ncremins]

Current state of my inbox:

We’ve updated our privacy policy.
We’ve updated our privacy policy.
We’ve updated our privacy policy.
We’ve updated our privacy policy.
We’ve updated our privacy policy.
We’ve updated our privacy policy.
We’ve updated our privacy policy.
…

[erbear @erbbear]

One good example how leadership can be blind in the organization. Sadly it's in the @magicleap 😔 https://t.co/ew7zZ5RuYo

[erbear @erbbear]

@michaloblak See it for free here: https://t.co/K7CWC4am2a

[@levelsio @levelsio]

Adding ⭐️ ratings (from @goodreads) to a BUY button just doubled daily sales.

🙂 Now I understand why Amazon has been doing it since the start

[Esther Schindler @estherschindler]

In 1999, I asked David Gerrold to write a "future of computing" prediction for the magazine where I was Technology Editor. Here's what he wrote.

[Michal Oblak @michaloblak]

This 🐟 is 🐠 awesome! 🐡 https://t.co/TX0pi2fhPn

[erbear @erbbear]

How to blast through silo mentality to create a culture of experimentation https://t.co/5fOhIyMMxN

[erbear @erbbear]

Feedback Driven Growth: A Process For Sustainable Growth https://t.co/PdH8bYuicv

[erbear @erbbear]

Tipping Point https://t.co/BhdUEEFzeV

[erbear @erbbear]

How to Fall Asleep in 2 Minutes or Less https://t.co/MGhuVcV8SR