stats

Statistics, but* naked.

[stats @NakedStatistics]

@FuzzyPandaShort @grok analyse the critically analyse this thread

[stats @NakedStatistics]

@VitalikButerin Side question: AI models are finding crack in softwares, have they found issues with the blockchain? And how does one mitigate for that if frontier models are getting released so often?

[Andrej Karpathy @karpathy]

Finally had a chance to listen through this pod with Sutton, which was interesting and amusing.

As background, Sutton's "The Bitter Lesson" has become a bit of biblical text in frontier LLM circles. Researchers routinely talk about and ask whether this or that approach or idea is sufficiently "bitter lesson pilled" (meaning arranged so that it benefits from added computation for free) as a proxy for whether it's going to work or worth even pursuing. The underlying assumption being that LLMs are of course highly "bitter lesson pilled" indeed, just look at LLM scaling laws where if you put compute on the x-axis, number go up and to the right. So it's amusing to see that Sutton, the author of the post, is not so sure that LLMs are "bitter lesson pilled" at all. They are trained on giant datasets of fundamentally human data, which is both 1) human generated and 2) finite. What do you do when you run out? How do you prevent a human bias? So there you have it, bitter lesson pilled LLM researchers taken down by the author of the bitter lesson - rough!

In some sense, Dwarkesh (who represents the LLM researchers viewpoint in the pod) and Sutton are slightly speaking past each other because Sutton has a very different architecture in mind and LLMs break a lot of its principles. He calls himself a "classicist" and evokes the original concept of Alan Turing of building a "child machine" - a system capable of learning through experience by dynamically interacting with the world. There's no giant pretraining stage of imitating internet webpages. There's also no supervised finetuning, which he points out is absent in the animal kingdom (it's a subtle point but Sutton is right in the strong sense: animals may of course observe demonstrations, but their actions are not directly forced/"teleoperated" by other animals). Another important note he makes is that even if you just treat pretraining as an initialization of a prior before you finetune with reinforcement learning, Sutton sees the approach as tainted with human bias and fundamentally off course, a bit like when AlphaZero (which has never seen human games of Go) beats AlphaGo (which initializes from them). In Sutton's world view, all there is is an interaction with a world via reinforcement learning, where the reward functions are partially environment specific, but also intrinsically motivated, e.g. "fun", "curiosity", and related to the quality of the prediction in your world model. And the agent is always learning at test time by default, it's not trained once and then deployed thereafter. Overall, Sutton is a lot more interested in what we have common with the animal kingdom instead of what differentiates us. "If we understood a squirrel, we'd be almost done".

As for my take...

First, I should say that I think Sutton was a great guest for the pod and I like that the AI field maintains entropy of thought and that not everyone is exploiting the next local iteration LLMs. AI has gone through too many discrete transitions of the dominant approach to lose that. And I also think that his criticism of LLMs as not bitter lesson pilled is not inadequate. Frontier LLMs are now highly complex artifacts with a lot of humanness involved at all the stages - the foundation (the pretraining data) is all human text, the finetuning data is human and curated, the reinforcement learning environment mixture is tuned by human engineers. We do not in fact have an actual, single, clean, actually bitter lesson pilled, "turn the crank" algorithm that you could unleash upon the world and see it learn automatically from experience alone.

Does such an algorithm even exist? Finding it would of course be a huge AI breakthrough. Two "example proofs" are commonly offered to argue that such a thing is possible. The first example is the success of AlphaZero learning to play Go completely from scratch with no human supervision whatsoever. But the game of Go is clearly such a simple, closed, environment that it's difficult to see the analogous formulation in the messiness of reality. I love Go, but algorithmically and categorically, it is essentially a harder version of tic tac toe. The second example is that of animals, like squirrels. And here, personally, I am also quite hesitant whether it's appropriate because animals arise by a very different computational process and via different constraints than what we have practically available to us in the industry. Animal brains are nowhere near the blank slate they appear to be at birth. First, a lot of what is commonly attributed to "learning" is imo a lot more "maturation". And second, even that which clearly is "learning" and not maturation is a lot more "finetuning" on top of something clearly powerful and preexisting. Example. A baby zebra is born and within a few dozen minutes it can run around the savannah and follow its mother. This is a highly complex sensory-motor task and there is no way in my mind that this is achieved from scratch, tabula rasa. The brains of animals and the billions of parameters within have a powerful initialization encoded in the ATCGs of their DNA, trained via the "outer loop" optimization in the course of evolution. If the baby zebra spasmed its muscles around at random as a reinforcement learning policy would have you do at initialization, it wouldn't get very far at all. Similarly, our AIs now also have neural networks with billions of parameters. These parameters need their own rich, high information density supervision signal. We are not going to re-run evolution. But we do have mountains of internet documents. Yes it is basically supervised learning that is ~absent in the animal kingdom. But it is a way to practically gather enough soft constraints over billions of parameters, to try to get to a point where you're not starting from scratch. TLDR: Pretraining is our crappy evolution. It is one candidate solution to the cold start problem, to be followed later by finetuning on tasks that look more correct, e.g. within the reinforcement learning framework, as state of the art frontier LLM labs now do pervasively.

I still think it is worth to be inspired by animals. I think there are multiple powerful ideas that LLM agents are algorithmically missing that can still be adapted from animal intelligence. And I still think the bitter lesson is correct, but I see it more as something platonic to pursue, not necessarily to reach, in our real world and practically speaking. And I say both of these with double digit percent uncertainty and cheer the work of those who disagree, especially those a lot more ambitious bitter lesson wise.

So that brings us to where we are. Stated plainly, today's frontier LLM research is not about building animals. It is about summoning ghosts. You can think of ghosts as a fundamentally different kind of point in the space of possible intelligences. They are muddled by humanity. Thoroughly engineered by it. They are these imperfect replicas, a kind of statistical distillation of humanity's documents with some sprinkle on top. They are not platonically bitter lesson pilled, but they are perhaps "practically" bitter lesson pilled, at least compared to a lot of what came before. It seems possibly to me that over time, we can further finetune our ghosts more and more in the direction of animals; That it's not so much a fundamental incompatibility but a matter of initialization in the intelligence space. But it's also quite possible that they diverge even further and end up permanently different, un-animal-like, but still incredibly helpful and properly world-altering. It's possible that ghosts:animals :: planes:birds.

Anyway, in summary, overall and actionably, I think this pod is solid "real talk" from Sutton to the frontier LLM researchers, who might be gear shifted a little too much in the exploit mode. Probably we are still not sufficiently bitter lesson pilled and there is a very good chance of more powerful ideas and paradigms, other than exhaustive benchbuilding and benchmaxxing. And animals might be a good source of inspiration. Intrinsic motivation, fun, curiosity, empowerment, multi-agent self-play, culture. Use your imagination.

[stats @NakedStatistics]

@CroConMedia @TripleNetInvest Mom’s spaghetti, the bag and these weak arms are heavy.

[Albert Alan, MD @AlbertAlan]

Dear $OPEN Investors,

Something Doesn’t Add Up…

You know what I find interesting?

There’s a gentleman named Eric Jackson, founder of EMJ Capital, who’s been pretty vocal about his belief that OpenDoor could reach $82 per share.

But when we look at the data straight from SEC 13F filings and our own proprietary tools his trading history tells a different story.

Let’s break it down:

September 30, 2021:
-EMJ Capital bought 62,000 shares of OpenDoor at an average price of $16.73.

December 31, 2021 (just 3 months later):
-They sold all 62,000 shares at around $14.61, locking in a 12.67% loss.

Then it gets even more interesting…

March 31, 2022:
-EMJ bought 135,000 shares at an average of $9.69.
-June 30, 2022 (again, just 3 months later):
-They sold everything when the stock was around $5.66, taking a 41.58% loss.

And it doesn’t stop there…

September 30, 2022:
-They came back in, buying 70,000 shares at $4.74.

December 31, 2022:
-Again, sold all shares at just $1.16, down 75.53% from the buy-in.

So here’s the big question…

If someone truly believed OpenDoor was an asymmetric opportunity a stock that could moon to $82 why sell at such massive losses, repeatedly?

We’re not here to throw shade. We respect Eric Jackson and anyone who puts their ideas out there. But as an educational platform focused on financial literacy through first principles thinking, we encourage everyone to be data-driven.

Let’s not speculate we follow the filings.

Personally, I’m curious to see EMJ’s next 13F. Will there be another OpenDoor move? We’ll see. Until then, the numbers tell a story that doesn’t quite match the headline.

[Triple Net Investor @TripleNetInvest]

Warren Buffett thoughtfully explains why investing in stocks/equities is better than real estate, during Berkshire's latest annual meeting:

"In respect to real estate, it's so much harder than stocks in terms of negotiation of deals, time spent, the involvement of multiple parties in the ownership.

Usually when real estate gets in trouble, you find out you're dealing with more than equity holder. But there have been times when large amounts of real estate...

I've changed hands at bargain prices, but usually stocks were cheaper, but there were a lot easier to do.

Charlie did more real estate. Charlie enjoyed real estate transactions, and he actually did a fair number of them in the last five years of his life.

But he was playing a game that was an interesting game to him. But I think if you'd asked him to make a choice when he was 21, he'd either be in stocks exclusively the rest of his life or real estate the rest of his life. He would have chosen stocks in a second.

There's just so much more opportunity, at least in the United States. There's so much more opportunity that presents itself in the security market than it does in real estate and in real estate.

You're usually dealing with a single owner or a family that owns maybe a large property they've had a long time. Maybe they've borrowed too much money against them. Maybe the population trends are against them. But to them, it's an enormous...

When you walk down to the New York Stock Exchange, you can do billions of dollars worth of business totally anonymous, and you can do it in five minutes. And the trades are complete when they're complete.

In real estate, when you make a deal, a big deal with a distressed lender, when you sign the deal, then you go into another phase. Then people start negotiating more things and more things. It's a whole different game. And a different type of person, to some extent, enjoys the game.

We did a few real estate deals that came our way in 2008 and 2009, but the amount of time that they would take us compared to doing something intelligent and probably better in securities, there was just no comparison.

I mean, in a real estate deal, every sentence is important. In stocks, if somebody needs to sell 20,000 shares of Berkshire or something and they call us and the price is right, it's done in five seconds. And it closes all the time."

[stats @NakedStatistics]

@doctoroak Reddit seem to have forgotten Reddit has a developer majority

[dscvr.one @DSCVR1]

Reddit’s API changes are unacceptable. Greed permeates every corner of each Big Tech social platform, but that’s why we’re here.

-Transparency
-Open API
-Listening to mods & users
-Delivering on desired features
-No politics
-Actually fun

#RIPReddit #RedditBlackout

[Angry Torontonian @Angry6ian]

Wow, when did @Reddit becoming another @digg? Please listen to your users or else they leave.
https://t.co/mcmcec9EdM

[no force majeure @no_forcemajeure]

This is insane
#RedditBlackout

[Aakash Gupta @aakashgupta]

Twitter revealed its algorithm to the world.

But what does it mean for you?

I spent the evening analyzing it.

Here’s what you need to know:

[stats @NakedStatistics]

@Tesla @elonmusk Parked Tesla cars becoming an owner scheduled power bank to help stabilise grid, has that been considered?

[Alex @alex_avoigt]

Elon reportedly send this mail to Tesla employees:

1) Avoid large meetings
Large meetings waste valuable time & energy
- They discourage debate
- People are more guarded than open
- There’s not enough time for everyone to contribute
Don’t schedule large meetings unless ...

[Sawyer Merritt @SawyerMerritt]

https://t.co/ZG0no4q5Zp

[Marius Hauken @mhauken]

If you're not using AI, you're losing out.

8 mindblowing AI tools (you can use today) that will save you hours:

[Robin Wheeler  @robinw]

https://t.co/05QY6u4FSD

[stats @NakedStatistics]

LUSD looks great! https://t.co/zclQs8KpbS

[stats @NakedStatistics]

One of the better takes I’ve heard ++@MBRichardson87 @Here2DeFi

[Johnny_TVL @john_tv_locke]

The @Bancor Network paused depositor principal protection 50 days ago.

The team and community are grinding to find a way forward. Can they save the protocol and survive? I propose a solution.

Power through your Sunday hangover for 8 minutes and learn something today.
1/n

[Scott Bryan @scottygb]

INCREDIBLE SCENES

“THE BENNY HILL THEME TUNE” IS BEING BLASTED OUT ON SKY NEWS.

[stats @NakedStatistics]

@davidasinclair @mdlaplante It’d be great to entice another billion people into ageing researching