Francis

"Blow a kiss, fire a gun"

[Jakeup @myhandle]

writing fables with Fable, call me Aeslop

[Francis @dabblingfrancis]

@prajdabre Task is too difficult, too much zero reward

[Rory Sutherland @rorysutherland]

I would also add that good air conditioning units also operate as heat pumps in cold weather. Almost all modern hotel rooms are both warmed and cooled this way. Heat pumps are vital in getting people to heat homes electrically rather than using gas.

Second point. Never mind polar bears. Electrification is valuable simply because electrical devices are omnivorous and can use energy generated from any source. Gas boilers are like pandas: they can only run on gas/bamboo. This makes them vulnerable to bottlenecks, supply shocks, etc.

[Francis @dabblingfrancis]

@digvijoy_c He didn't play against France

[Francis @dabblingfrancis]

@JustinBleuel @ChatGPTapp Displaying math equations Latex can be so slow sometimes (in Android)

[Francis @dabblingfrancis]

@SOURADIPCHAKR18 @lateinteraction @amritsinghbedi3 @NoahZiems Hello, thank you for this great blog post. This is a very refreshing take on improving RL sample efficiency.

It seems that for a given value of \beta, the teacher model can compensate for spikes by adding many predictable filler tokens. Have you noticed longer trajectories?

[Francis @dabblingfrancis]

@dosco "A harness for every task" - Anthropic

[David Bessis @davidbessis]

To further explain why I think your position is untenable:
AI can easily produce new fugues in the style of Bach and it's already hard (or will soon be) to justify that they could be "weaker" in objective dimensions. Yet I don't think you could become seriously interested in those fugues.

The reasons seems backstory-related:

1/ the backstory of authenticity, connecting the work to a specific person, from a specific century, at a certain moment in the history of music (this is the same reason that prevents neoclassical copycat for being seriously considered).

2/ the backstory of how you were connected to the fugues, at a certain moment of your cognitive journey. Your adult brain might be less willing to engage with new fugues with the same intensity...

3/ but it's likely to be different if you're fed a specific story that you buy into (eg, that the fugue is a lost work by Bach that was miraculously recovered.)

I even think this is testable, with a group of Bach lovers being presented the fake AI fugue as AI generated, and another group being told the "recovered lost work" story. I'm pretty certain the two groups would come up with very distinct esthetic judgments.

[Caleb Gross @noperator]

https://t.co/gOSvgSg8kv

[Souradip Chakraborty @SOURADIPCHAKR18]

🚨Typical RL algorithms and on-policy distillation methods are blind samplers: they use privileged info to score rollouts, but not to *find* them.

We ask: can we use privileged info to *actively sample* the rollouts RL wishes it can stumble upon with compute?

⤵️ Pedagogical RL

[Francis @dabblingfrancis]

@sean_a_mcclure On- vs off-policy

[Sky @SkyTheViking]

I CAN'T BREATHE 🤣🤣🤣

[Francis @dabblingfrancis]

@Porkchop_EXP QTing yourself with another visual banger. Extraordinarily powerful. Thank you for this masterclass.

[Francis @dabblingfrancis]

@hjluks Surely there are better strength indicators than a handgrip test?

[François Chollet @fchollet]

I keep reading this take (below) every few months, presented as if extremely profound, and it is just offensively dumb. It confuses data and information, it ignores the fact that not all information is equally valuable, and it ignores the importance of retention rate.

As a thought experiment: if this were true, if your retina cell count were 10x greater, you'd be "trained on 10x more tokens" and therefore you'd be way smarter. Same if their firing frequency were 10x greater. With 10x more retina cells firing 10x faster you'd be "trained on 100x more tokens"!

Obviously this makes no sense -- the signal coming from these cells is extremely correlated over space and time, so their raw information content (what remains post-compression) is extremely low compared to the "raw bit" encoding. The human visual system actually processes 40 to 50 bits per second after spatial compression. Much, much less if you add temporal compression over a long time horizon.

Latest LLMs get access to approximately 3 to 4 orders of magnitude of information more than a human by age 20 (post compression in both cases). About O(10T) bits vs O(10-100B) bits.

And that's just *raw information* but of course not all information is equal, otherwise we wouldn't be spending tens of billions of dollars on training data annotation and generation. Plus, that's only *information intake* but of course humans have far lower retention than LLMs (by 3-4 OOM). You could write a short essay about how incredibly off the mark this take is.

[Francis @dabblingfrancis]

@BakerBarbell @sam_gzstrength People who train, when thinking of "new lifters", tend to think of themselves at the start of their lifting career at 14yo.

[Francis @dabblingfrancis]

@nntaleb This is the IQ story all over again. And to think I Almost got tricked into doing a VO2 max test.

[Francis @dabblingfrancis]

@sheggle_ @DmitryRybin1 @stalkermustang I think they were planning on making the rest of the games easier due to the models' bad performance during the preview phase

[Francis @dabblingfrancis]

@MaximeRivest Great point about abstraction. People are not thinking about the right level of abstraction.