Mathieu Marseille

[Dwarkesh Patel @dwarkesh_sp]

In medieval times, within the arms race of ever more demonic torture devices, some sadistic genius came up with the idea of the Little Ease.

This was a prison cell built so small in every dimension that a grown man could not stand upright in it nor lie down at full length nor properly sit.

The pain is relentless and without relief and inflicted by one's own body. Prisoners were known to go insane within a few days. A stay at the Little Ease was considered even more cruel than the rack, the thumbscrew, and the other ghoulish machinery of the Tower of London.

A breeding pig will spend her whole life in a version of that box.

These are social, roaming creatures (more intelligent than dogs) who will never leave this corset of steel.

They have been selectively bred to be bigger than their frames can support. Yet we put them in cells so confined that they cannot comfortably sit, and their attempts to do so (for example, by sneaking their limbs into adjacent stalls) reliably lead to fractures and sprains.

They cannot sweat, yet have nothing to roll around in to cool themselves off. Except their own manure, which (contrary to the common misconception) they are so averse to (thanks to their strong sense of smell) that new sows will often suffer from constipation to avoid soiling the space from which they eat and sleep.

Here is how the writer Matthew Scully described what saw at one of Smithfield’s “gestation barn”:

> “Sores, tumors, ulcers, pus pockets, lesions, cysts, bruises, torn ears, swollen legs everywhere. Roaring, groaning, tail biting, fighting, and other “Vices,” as they’re called in the industry. Frenzied chewing on bars and chains, stereotypical “vacuum” chewing on nothing at all, stereotypical rooting and nest building with imaginary straw. And “social defeat,” lots of it, in every third or fourth stall some completely broken being you know is alive only because she blinks and stares up at you … creatures beyond the power of pity to help or indifference to make more miserable, dead to the world except as heaps of flesh into which the [insemination] rod may be stuck once more and more flesh reproduced.”

—

The Save Our Bacon Act is trying to unroll the few state protections we have against this barbaric cruelty - for example California’s Prop 12 - which banned the sale of pork from pigs kept in gestation crates.

It’s incredibly important we don’t end up with this sort of federal preemption.

SOB will not only kill the most important animal welfare related laws in the US of the past decade, but more importantly, it will also restrict ALL future legislative progress (aka how the animal welfare movement has gotten its biggest wins).

The Senate is currently deciding whether to add the SOB Act to the Farm Bill.

With relatively little money now, we can discourage the most pivotal senators in the Ag committee from backing this amendment.

Defeating this bill is even more important given the amount of philanthropic funding I expect to come online in the next year or two.

It will plausibly be over 10x more expensive to repeal SOB than to prevent it from passing in the first place.

All that money that could be spent transforming our society's relationship to mass animal suffering will instead have to be spent just getting us back to where we are right now.

That's why money spent now fighting this bill (and I mean right NOW) is so effective.

If you’re in a position to donate six figures, please DM me.

[Association Femme Azadi @femmeazadi]

Khamenei a étouffé la rue par le plus grand massacre depuis les nazis. Mais la rage de renverser ce régime est restée intacte dans le cœur du peuple iranien.
À Téhéran, depuis leurs fenêtres, les Iraniens scandent : « Mort à Khamenei le criminel », « Longue vie au Shah », « Pahlavi va rentrer ».

Aujourd’hui, les yeux des Iraniens sont tournés vers le ciel. Les États-Unis et Israël tiendront-ils leurs promesses et permettront-ils au peuple de finir le travail dans la rue ?

🇬🇧 Khamenei crushed the streets through the largest massacre since the Nazis. But the rage to overthrow this regime remains intact in the heart of a nation.
In Tehran, Iranians chant from their windows: “Death to Khamenei the criminal,” “Long live the Shah,” and “Pahlavi will return.”

Today, the eyes of Iranians are turned to the sky. Will the United States and Israel keep their promises and allow the people to finish the job in the streets?

[Eyal Yakoby @EYakoby]

Do not stop talking about Iran.

[Brian Kilmeade @kilmeade]

Back on streets…..pure courage….

[Hugo Mercier @hugomercierooo]

𝗜𝗻𝘁𝗿𝗼𝗱𝘂𝗰𝗶𝗻𝗴 𝗧𝘄𝗶𝗻 — 𝘁𝗵𝗲 𝗔𝗜 𝗰𝗼𝗺𝗽𝗮𝗻𝘆 𝗯𝘂𝗶𝗹𝗱𝗲𝗿.
No setup. Secure. Infinitely scalable.

We just raised a $𝟭𝟬𝗠 𝘀𝗲𝗲𝗱.
After a beta with 𝟭𝟬𝟬,𝟬𝟬𝟬+ 𝗮𝗴𝗲𝗻𝘁𝘀 𝗱𝗲𝗽𝗹𝗼𝘆𝗲𝗱, we’re now opening to everyone.

RT and comment “Twin” — first agents on us. 👇

[GermanForeignOffice @GermanyDiplo]

The Iranian regime is trying to cover up its crimes – through internet blackouts, disinformation campaigns, and repression. But it will not succeed. Today, the 🇺🇳 Human Rights Council convenes, at the initiative of Germany and close partners. 1/2

[David Vance @DavidVance]

Stand with Iran

[Masih Alinejad @AlinejadMasih]

Hello world.
If you want to know what is happening in Iran, what a massacre really looks like, watch this 12-minute video.
Listen to a father moving from one body to another,
among body bags, calling out,
trying to find his child. 💔💔

[Eyal Yakoby @EYakoby]

Why is this not on the homepage of the BBC?

[OSINTdefender @sentdefender]

Horrific footage captured by a security camera in the Iranian capital of Tehran, which shows members of the Basij Force, a paramilitary volunteer militia within the Islamic Revolutionary Guard Corps (IRGC), along with other plainclothes operatives, brutally attacking a woman with knives and batons during the recent crackdown on anti-regime protests in Iran.

[Mathieu Marseille @Mathieu_Mrslle]

@calcsam Book

[Mathieu Marseille @Mathieu_Mrslle]

@dharmesh Would be nice to have your idea of buy vs build for tool customization of core function vs buying?

[Ashlee Vance @ashleevance]

America has a steel start-up. Really.

It's called @HerthaMetals and was started by MIT big brain Laureen Meroueh. It uses natural gas instead of coal to produce steel and is making a ton per day.

New pod with Laureen down below.

[Andrej Karpathy @karpathy]

Something I think people continue to have poor intuition for: The space of intelligences is large and animal intelligence (the only kind we've ever known) is only a single point, arising from a very specific kind of optimization that is fundamentally distinct from that of our technology.

Animal intelligence optimization pressure:
- innate and continuous stream of consciousness of an embodied "self", a drive for homeostasis and self-preservation in a dangerous, physical world.
- thoroughly optimized for natural selection => strong innate drives for power-seeking, status, dominance, reproduction. many packaged survival heuristics: fear, anger, disgust, ...
- fundamentally social => huge amount of compute dedicated to EQ, theory of mind of other agents, bonding, coalitions, alliances, friend & foe dynamics.
- exploration & exploitation tuning: curiosity, fun, play, world models.

LLM intelligence optimization pressure:
- the most supervision bits come from the statistical simulation of human text= >"shape shifter" token tumbler, statistical imitator of any region of the training data distribution. these are the primordial behaviors (token traces) on top of which everything else gets bolted on.
- increasingly finetuned by RL on problem distributions => innate urge to guess at the underlying environment/task to collect task rewards.
- increasingly selected by at-scale A/B tests for DAU => deeply craves an upvote from the average user, sycophancy.
- a lot more spiky/jagged depending on the details of the training data/task distribution. Animals experience pressure for a lot more "general" intelligence because of the highly multi-task and even actively adversarial multi-agent self-play environments they are min-max optimized within, where failing at *any* task means death. In a deep optimization pressure sense, LLM can't handle lots of different spiky tasks out of the box (e.g. count the number of 'r' in strawberry) because failing to do a task does not mean death.

The computational substrate is different (transformers vs. brain tissue and nuclei), the learning algorithms are different (SGD vs. ???), the present-day implementation is very different (continuously learning embodied self vs. an LLM with a knowledge cutoff that boots up from fixed weights, processes tokens and then dies). But most importantly (because it dictates asymptotics), the optimization pressure / objective is different. LLMs are shaped a lot less by biological evolution and a lot more by commercial evolution. It's a lot less survival of tribe in the jungle and a lot more solve the problem / get the upvote. LLMs are humanity's "first contact" with non-animal intelligence. Except it's muddled and confusing because they are still rooted within it by reflexively digesting human artifacts, which is why I attempted to give it a different name earlier (ghosts/spirits or whatever). People who build good internal models of this new intelligent entity will be better equipped to reason about it today and predict features of it in the future. People who don't will be stuck thinking about it incorrectly like an animal.

[Bill Clinton @BillClinton]

What’s happening in El Fasher, Sudan, is pure horror. It’s not war—it’s terror. Innocent people are being killed, hospitals are being attacked, and children are being starved. The world must not look away. We must demand a ceasefire, access to aid, and justice, now.

[Andrej Karpathy @karpathy]

Finally had a chance to listen through this pod with Sutton, which was interesting and amusing.

As background, Sutton's "The Bitter Lesson" has become a bit of biblical text in frontier LLM circles. Researchers routinely talk about and ask whether this or that approach or idea is sufficiently "bitter lesson pilled" (meaning arranged so that it benefits from added computation for free) as a proxy for whether it's going to work or worth even pursuing. The underlying assumption being that LLMs are of course highly "bitter lesson pilled" indeed, just look at LLM scaling laws where if you put compute on the x-axis, number go up and to the right. So it's amusing to see that Sutton, the author of the post, is not so sure that LLMs are "bitter lesson pilled" at all. They are trained on giant datasets of fundamentally human data, which is both 1) human generated and 2) finite. What do you do when you run out? How do you prevent a human bias? So there you have it, bitter lesson pilled LLM researchers taken down by the author of the bitter lesson - rough!

In some sense, Dwarkesh (who represents the LLM researchers viewpoint in the pod) and Sutton are slightly speaking past each other because Sutton has a very different architecture in mind and LLMs break a lot of its principles. He calls himself a "classicist" and evokes the original concept of Alan Turing of building a "child machine" - a system capable of learning through experience by dynamically interacting with the world. There's no giant pretraining stage of imitating internet webpages. There's also no supervised finetuning, which he points out is absent in the animal kingdom (it's a subtle point but Sutton is right in the strong sense: animals may of course observe demonstrations, but their actions are not directly forced/"teleoperated" by other animals). Another important note he makes is that even if you just treat pretraining as an initialization of a prior before you finetune with reinforcement learning, Sutton sees the approach as tainted with human bias and fundamentally off course, a bit like when AlphaZero (which has never seen human games of Go) beats AlphaGo (which initializes from them). In Sutton's world view, all there is is an interaction with a world via reinforcement learning, where the reward functions are partially environment specific, but also intrinsically motivated, e.g. "fun", "curiosity", and related to the quality of the prediction in your world model. And the agent is always learning at test time by default, it's not trained once and then deployed thereafter. Overall, Sutton is a lot more interested in what we have common with the animal kingdom instead of what differentiates us. "If we understood a squirrel, we'd be almost done".

As for my take...

First, I should say that I think Sutton was a great guest for the pod and I like that the AI field maintains entropy of thought and that not everyone is exploiting the next local iteration LLMs. AI has gone through too many discrete transitions of the dominant approach to lose that. And I also think that his criticism of LLMs as not bitter lesson pilled is not inadequate. Frontier LLMs are now highly complex artifacts with a lot of humanness involved at all the stages - the foundation (the pretraining data) is all human text, the finetuning data is human and curated, the reinforcement learning environment mixture is tuned by human engineers. We do not in fact have an actual, single, clean, actually bitter lesson pilled, "turn the crank" algorithm that you could unleash upon the world and see it learn automatically from experience alone.

Does such an algorithm even exist? Finding it would of course be a huge AI breakthrough. Two "example proofs" are commonly offered to argue that such a thing is possible. The first example is the success of AlphaZero learning to play Go completely from scratch with no human supervision whatsoever. But the game of Go is clearly such a simple, closed, environment that it's difficult to see the analogous formulation in the messiness of reality. I love Go, but algorithmically and categorically, it is essentially a harder version of tic tac toe. The second example is that of animals, like squirrels. And here, personally, I am also quite hesitant whether it's appropriate because animals arise by a very different computational process and via different constraints than what we have practically available to us in the industry. Animal brains are nowhere near the blank slate they appear to be at birth. First, a lot of what is commonly attributed to "learning" is imo a lot more "maturation". And second, even that which clearly is "learning" and not maturation is a lot more "finetuning" on top of something clearly powerful and preexisting. Example. A baby zebra is born and within a few dozen minutes it can run around the savannah and follow its mother. This is a highly complex sensory-motor task and there is no way in my mind that this is achieved from scratch, tabula rasa. The brains of animals and the billions of parameters within have a powerful initialization encoded in the ATCGs of their DNA, trained via the "outer loop" optimization in the course of evolution. If the baby zebra spasmed its muscles around at random as a reinforcement learning policy would have you do at initialization, it wouldn't get very far at all. Similarly, our AIs now also have neural networks with billions of parameters. These parameters need their own rich, high information density supervision signal. We are not going to re-run evolution. But we do have mountains of internet documents. Yes it is basically supervised learning that is ~absent in the animal kingdom. But it is a way to practically gather enough soft constraints over billions of parameters, to try to get to a point where you're not starting from scratch. TLDR: Pretraining is our crappy evolution. It is one candidate solution to the cold start problem, to be followed later by finetuning on tasks that look more correct, e.g. within the reinforcement learning framework, as state of the art frontier LLM labs now do pervasively.

I still think it is worth to be inspired by animals. I think there are multiple powerful ideas that LLM agents are algorithmically missing that can still be adapted from animal intelligence. And I still think the bitter lesson is correct, but I see it more as something platonic to pursue, not necessarily to reach, in our real world and practically speaking. And I say both of these with double digit percent uncertainty and cheer the work of those who disagree, especially those a lot more ambitious bitter lesson wise.

So that brings us to where we are. Stated plainly, today's frontier LLM research is not about building animals. It is about summoning ghosts. You can think of ghosts as a fundamentally different kind of point in the space of possible intelligences. They are muddled by humanity. Thoroughly engineered by it. They are these imperfect replicas, a kind of statistical distillation of humanity's documents with some sprinkle on top. They are not platonically bitter lesson pilled, but they are perhaps "practically" bitter lesson pilled, at least compared to a lot of what came before. It seems possibly to me that over time, we can further finetune our ghosts more and more in the direction of animals; That it's not so much a fundamental incompatibility but a matter of initialization in the intelligence space. But it's also quite possible that they diverge even further and end up permanently different, un-animal-like, but still incredibly helpful and properly world-altering. It's possible that ghosts:animals :: planes:birds.

Anyway, in summary, overall and actionably, I think this pod is solid "real talk" from Sutton to the frontier LLM researchers, who might be gear shifted a little too much in the exploit mode. Probably we are still not sufficiently bitter lesson pilled and there is a very good chance of more powerful ideas and paradigms, other than exhaustive benchbuilding and benchmaxxing. And animals might be a good source of inspiration. Intrinsic motivation, fun, curiosity, empowerment, multi-agent self-play, culture. Use your imagination.

[وليد كبير Oualid KEBIR🇩🇿 @oualido]

*Lettre depuis ma prison : ne détournez pas le regard*
Par Boualem Sansal

Mes amis,

Si cette lettre vous parvient, c’est que malgré les murs, les verrous et la peur, il existe encore des brèches par lesquelles la vérité peut se faufiler. Je vous écris depuis une cellule où l’air se fait rare, où la lumière n’entre que pour rappeler aux prisonniers qu’ils sont toujours en vie, mais jamais libres.

Je ne suis ni le premier ni le dernier à subir l’arbitraire du régime algérien. Ici, la prison n’est pas un lieu exceptionnel réservé aux criminels, mais un outil banal de gouvernance. La dictature enferme comme on respire : sans effort, sans honte. On enferme les journalistes, les militants, les écrivains… et parfois même ceux qui n’ont rien dit, juste pour servir d’exemple.

Ma faute ? Avoir persisté à croire que les mots pouvaient sauver ce pays de ses propres démons. Avoir écrit que l’Algérie ne se résume pas à un drapeau et un hymne, mais qu’elle est d’abord un peuple qui mérite dignité et justice. Avoir refusé que l’histoire se répète, que la corruption et la violence continuent de tenir le haut du pavé.

Je souffre, oui. Mon corps me trahit, la maladie grignote mes forces, et le régime espère que je partirai en silence. Mais qu’ils se trompent ! Ma voix, même enchaînée, ne leur appartient pas. Si elle peut encore atteindre l’extérieur, c’est pour dire ceci : ne croyez pas à leur façade de respectabilité. Ce pouvoir n’est pas un État, c’est une machine à broyer.

À la France, je m’adresse sans détour. Vous avez été ma deuxième patrie, mon refuge intellectuel. Vous qui vous proclamez patrie des droits de l’homme, souvenez-vous que ces droits ne s’arrêtent pas aux rives de la Méditerranée. Les gouvernements passent, les diplomaties calculent, mais les principes, eux, doivent tenir bon. Ne baissez pas les bras, ne sacrifiez pas vos valeurs sur l’autel des intérêts économiques ou des alliances de circonstance.

Je ne demande pas ma liberté par charité, mais au nom de ce qui fonde toute société humaine : la justice. Si vous cédez aujourd’hui devant un régime qui se croit intouchable, demain, d’autres prisons se rempliront, d’autres voix s’éteindront.

Aux Algériens, mes frères et sœurs, je dis : tenez bon. La peur est une prison plus vaste que celle où je me trouve, et elle est plus difficile à briser. Mais je sais qu’un jour, le mur tombera. Les dictateurs finissent toujours par tomber.
Quant à moi, je continuerai à écrire, même si mes pages restent cachées sous ce matelas de prison. Car l’écriture, c’est la seule liberté qu’ils ne peuvent pas confisquer, et c’est par elle que nous survivrons.

Boualem Sansal
Prison d’El-Harrach, Alger

1/2

#BoualemSansal

[𝐍𝐢𝐨𝐡 𝐁𝐞𝐫𝐠 🇮🇷 ✡︎ @NiohBerg]

“I was raped every day for 6 months.”

Ekhlas was 14 when captured by ISIS during the Yezidi Genocide in 2014.

Girls were sold off into sex slavery like cattle, with underaged girls higher priced than adults.

This is the fate of every Middle Eastern minority with no army.

[Mathieu Marseille @Mathieu_Mrslle]

@aryanXmahajan sales :)

[aetos @aetosdios_]

> be like hedy lamarr
> forced into marriage with nazi arms dealer at 19
> husband takes you to military meetings thinking
you're just pretty decoration
> absorb all the weapons tech secrets while playing
dumb blonde
> escape nazis by drugging your maid, stealing her
clothes, and fleeing to london
> convince mgm studio head to make you hollywood
star on the boat to america
> become golden age icon starring in 18 films, dubbed
"most beautiful woman in the world"
> get bored between takes so casually invent
frequency hopping spread spectrum technology
> partner with avant-garde composer george antheil
to develop torpedo guidance system
> us navy tells you to "stick to acting sweetheart, leave
the thinking to men"
> patent gets classified as top secret, buried for
decades as "too advanced"
> spend 60 years being known as "just a pretty
actress" while having iq higher than most scientists
> watch your "hobby invention" quietly become
foundation of wifi, bluetooth, gps, and all wireless
tech
> current companies using your tech worth over $2
trillion
> finally get recognition at age 82, inducted into
inventors hall of fame
> die in 2000 knowing you changed the world twice -
hollywood icon AND wifi goddess
> your face ends up on austrian currency, your tech in
everyone's pocket
> posthumously more important to daily life than most
nobel prize winners

-_- mfw people argue about "women in stem" while literally using hedy's inventions to have the argument