Sean Mark, PhD

Humans suck at many fields of science (cough cough nutrition....) - building a Personalized Health AI - powered by a scientific reasoning engine.

[Sean Mark, PhD @Smark_phd]

If science actually worked...we wouldn't be wondering how to create AI-based superintelligence. Science would be that superintelligence.

But too often, the human scientific endeavor is not aligned with reality - or any meaningful reality beyond the ivory tower.

[François Chollet @fchollet]

The next major breakthrough will branch out at a much lower level than deep learning model architecture. It will be a new approach. A better model architecture can lead to incremental data efficiency & generalization gains, but it won't fix the fundamental issues of the parametric learning paradigm.

[Dwarkesh Patel @dwarkesh_sp]

Human beings whose emotional centres are damaged, even if their intelligence is still intact, have terrible decision-making skills.

Whatever role emotions are playing in humans, it's necessary for agency.

Ilya speculates that the equivalent for AIs is something to do with value functions - and that it might not emerge through pre-training alone.

[Nassim Nicholas Taleb @nntaleb]

As with fat tails Llms are frequency machines that fail to extrapolate outside the sample set. What they know is the VISIBLE.

Almost as bad as economists, almost worse than psychologists.

[Gary Marcus @GaryMarcus]

Three Predictions:

1. Some form of AI, probably neurosymbolic in nature, will come that is far more economical and data- and energy-efficient than LLMs, and it will make an absolute fortune.

2. LLMs, on the other hand, will never be all that profitable (aside from the chip companies selling shovels in the gold rush).

3. Today’s gigantic bets are premature, and most won’t pay off.

[Massimo @Rainmaker1973]

The chances of a deer taking you out at a crosswalk are slim. But never zero.

[Electroverse @Electroversenet]

Since the 1980s, the Sahara has shrunk by roughly 8%. Satellite data show widespread greening, a pattern that is playing out across the planet.

Around 50% of Earth's vegetated land has become significantly greener, an area roughly three times the size of the United States.

The dominant driver is not rainfall or land use change, it is rising atmospheric CO2.

Higher CO2 lets plants photosynthesize more efficiently, they lose less water, they tolerate heat and dryness better.

The effect is strongest along desert margins, across the Sahel, the Middle East, Australia's interior and the southern edge of the Sahara.

Rising CO2 is making the deserts, and the planet as a whole, greener.

[Sean Mark, PhD @Smark_phd]

Can't stop watching this...

[Massimo @Rainmaker1973]

Multitasking vs focus

[Massimo @Rainmaker1973]

New research shows clutter dramatically spikes women’s cortisol—while men’s stress barely budges.

Household clutter extends far beyond mere aesthetics—it's deeply intertwined with stress physiology and cognitive burden, impacting women in particular.

Drawing from studies on dual-income married couples, therapist Elizabeth Earnshaw explains that women who view their homes as cluttered often see their cortisol levels rise throughout the day, unlike those who feel at ease, whose levels naturally decline. This heightened effect in women stems largely from bearing the disproportionate invisible mental load—the constant cycle of noticing, recalling, planning, and orchestrating household tasks.

Earnshaw suggests a realistic, three-part approach to reducing the stress–clutter spiral.

First, “shedding” involves intentionally minimizing possessions, including doing the emotional work required to let things go, in order to create more mental and physical space. Second, “preventing” focuses on systems: giving items clear “homes” so that decisions about where things go become automatic rather than mentally taxing. This may start with listing common types of clutter and designing dedicated spots for each (for example, a single, consistent place for receipts). Third, “adapting” asks families to accept that some clutter is inevitable in busy seasons of life and to concentrate on emotional regulation and co-regulation with partners, keeping stress and cortisol lower by adjusting expectations rather than striving for a perpetually picture-perfect home.

[Earnshaw, E., "Clutter, Cortisol, and Mental Load". Psychology Today, 2024]

[Saxbe, D. E., & Repetti, R. , "No place like home: Home tours correlate with daily patterns of mood and cortisol", Personality and Social Psychology Bulletin, 36(1), 71–81, 2010, DOI: 10.1177/0146167209352864]

[Sean Mark, PhD @Smark_phd]

It's hot now on the island...makes me remember these days.

[Richard Sutton @RichardSSutton]

A new and possibly controversial perspective:
In this video, I explain the sense in which generative AI trained by supervised learning is incapable of making novel discoveries.
https://t.co/zin5QbbT9N

The text of the speech:

AI Creativity and Discovery

Good day ladies and gentlemen. I regret that I am unable to be with you all today to engage in a back-and-forth discussion, but I am nevertheless pleased to be able to share with you, via this recording, some high-level thoughts about the current and future state of artificial intelligence, and in particular about AI’s relationship to science and mathematics, which is, as I understand it, the central focus of this meeting and of the SAIR Foundation.

I would like to start with an old joke; I am sure you have heard it before. It is the one about the researcher whose work is being evaluated, and the review comes back, and says “This work is both novel and good. Unfortunately, the parts that are good are not novel, and the parts that are novel are not good.”

My first point about AI is that this assessment applies exactly to large parts of AI as we know it today. Not all of today’s AI, but a large part of it. Pretty much all of what we mean by “Generative AI”---which includes large language models, and the images and video models, and even the new methods for learning world models. All of these AIs take large numbers of examples and produce a “model” which behaves similar to the examples, that is, which generates text like people, or images like artists or nature, and videos like we find on the internet. Don’t get me wrong, Generative AI can be extremely useful. No doubt about that. But the assessment of the joke still applies. These systems can produce output that is both novel and good, but not at the same time.

In many ways this is just absolutely not a problem. When we ask an AI for an answer from the internet, or to summarize a document, we don’t want it to be novel. We are happy if the quality of the answer, the goodness, comes from the source material—from the people who wrote the document or the articles on the internet. If the AI’s answer is novel it means it is going beyond the source material, adding something beyond it. This is what we call “hallucinations”. In most cases, we don’t like it when the AI makes something up, when it adds something novel.

One exception, of course, is when we are looking not for facts or reality, but for fiction and entertainment. We might ask for a bedtime story for a child, or an image based on existing images on the internet but which is nevertheless different and distinct from them. In these cases, it is never easy for us to know how creative the AI is actually being, as we do not know how close the AI’s story, poem, or image is to the source material. In a real practical sense we can not know this because the internet is too big, the possible sources that the AI may draw upon are too numerous.

When we ask for a fiction or novelty, the AI can give it to us because its processing is in part stochastic. Every decision can go multiple ways and will go different ways and produce a different trajectory every time. The trajectory can be random—and thus novel—or it can be based on the training data—and thus “good” because the training data is good, sourced from people or reality. Thus, the trajectory is either novel or good—based on randomness or based on data—but never both at the same time.

Really, I think it is okay if the output of Generative AI is never good and novel at the same time. For the researcher in the joke this is a devastating criticism, but for most things it is not, and for Generative AI it is not. Generative AI is meant to be a mimic. This is what supervised learning is for. Generative AI can be extremely useful, even when it just mimics, if it is faster, or cheaper, or smaller, or more customizable, or more copy-able, than the thing being mimicked. It is okay if Generative AI cannot be both novel and good at the same time. It is still a transformative technology.

But it is a limitation. And remember we are here to use AI for science and mathematics, and for these areas the assessment of the reviewer in the joke is devastating. For these areas we need true creativity and discovery. Generative AI—or Mimicking AI—will never get where us there. For these we need something more, and indeed we have something more in other parts of AI. We have many AI systems which can give us more. We have AlphaGo with its world-changing move 37, or AlphaZero with its brilliant original chess-playing style. We have GT-Sophy that drives simulated racecars better than any human. We have AlphaFold and AlphaProof and Claude-Code, which have brought true advances in science, mathematics, and programming. We have RL-Lyft which optimizes the assignment of cars to passengers in the ride-hailing business. All these systems have found things that are both novel and good. And, truth be told, some language models have been augmented in ways that make them more than Generative AI based on supervised learning.

All these systems have some additional features that make them capable of true creativity and true discovery. It is important for us to recognize what this is—and that it is not present in ordinary, garden-variety Generative AI. It is something that can not come from just supervised learning, from learning from examples. What is it? Well, it is a simple thing, a commonsense thing. It is not new. We have many names for it, but unfortunately none of them are very good names. I will call it Discovery. Basically, Discovery is just the idea of trying many things and seeing which of them work, then keeping those that worked the best. Evolution by natural selection works this way. The scientific method works this way. And just ordinary life and learning works this way. We try things and remember what works. What could be more obvious? In this behavioral case, psychology has two names for it— “instrumental learning” and “operant conditioning”—and in machine learning it is what we mean by “reinforcement learning”. We also see the idea of Discovery in planning and combinatorial search—anything that involves the idea of “generate and test”.

The essence of Discovery is to combine three steps:
1. Variation,
2. Evaluation, and
3. Selective retention.

Of course, I am not the first to say this. I am not the first to point out that this combination of steps is key to science, to evolution by natural selection, and to animal behavior. I think particularly of papers by Donald Campbell, by Daniel Dennett, and by Gary Cziko. What is new in my remarks is to directly relate the idea of Discovery to modern AI to help us see that it is not present in supervised learning or Generative AI—in particular, that Discovery is not present in backpropagation or gradient descent.

Let me say explicitly what is missing from Generative AI. As we have remarked, these systems do have a stochastic aspect, so they do generate a variety of trajectories and behavior. What is missing is the Evaluation step. The generator was pre-trained by supervised learning, leaving no way at runtime to Evaluate what it generates. And of course without Evaluation there can be no Selective retention, and thus no Discovery. The variation can bring novelty, but without evaluation there is no Discovery, and arguably, no creativity. That is, I would say that creativity requires that the new things generated be Evaluated. Without evaluation, and retention of the best, there is nothing created. The novelty flickers into existence but, if its value is unrecognized, it flickers away and is lost.

In many cases, Evaluation is done by people to make a discovery. As when we have Generative AI make many pictures for us, and then we pick the one that we like the best. The human+AI system completes the discovery.
In many other cases, the Evaluation comes from a clear objective. Some moves lead to checkmate, some steps lead to a proof, some actions result in high reward, some genotypes make more copies, some theories explain the data better.

Some prefer the Variation step to be called Blind variation, where “blind” here means that it is uninformed, a shot in the dark. It does not need to be completely uninformed; a good scientist does not select theories to test at random. But neither can it be completely informed and determined. There must be some uncertainty about where the answer lies in order for there to be a discovery. In practice, the variation is partly informed and partly blind, but it is the blind part that corresponds to the discovery.

Now let us briefly go all the way to modern deep learning, to the backpropagation algorithm. At first it might seem that backpropagation is incapable of discovery because it is deterministic and thus incapable of variation. But this is not correct. The weight updates of backprop are deterministic, but the weights are initialized to small random values. The random initialization is often downplayed, but in fact it is a necessary form of variation; it must be done properly to get good performance. In backprop this Variation is done once, at network initialization, so its effect is temporary, and later the network may lose its ability to learn. This is the weakness of deep learning that is alleviated with a new algorithm that my group presented in Nature a couple of years ago. Our “continual backpropagation” made one small change: every so often a less-used neuron would be re-initialized to small random weights. This allows the variation to continue and plasticity to be retained.

Although there is much more to be said about Creativity and Discovery, this is the key point: they are more than supervised learning, more than pattern recognition, more than prediction, and more than world modeling. Those things are important, but they alone will not bring us to discovery. Discovery requires Evaluation from a person or from an explicit goal, and only in the latter case will we attain full autonomy.

So that is my call to arms. If we want the full power of AI scientists, then we should share the goals with them so they can create, evaluate, discover, and in these ways fully participate in achieving the goals. Let’s be bold! Let’s fully automate Creativity and Discovery!

[Sean Mark, PhD @Smark_phd]

@_alice_evans yes...non-generative nuking priors in social sciences...that is, if Hannah Arendt's work is considered social science.

https://t.co/PbTuWx2N4X

[Dalai Lama @DalaiLama]

Finding Inner Peace - His Holiness the Dalai Lama on inner peace — anger is its real destroyer, rooted in self-centeredness. The antidote — remember your existence depends on others. Compassion isn't just kind, it's realistic. Video originally recorded on November 4, 2017.

[Vivek Sen @Vivek4real_]

LARRY ELLISON: AI IS RAPIDLY COMMODITIZING BECAUSE MOST MODELS ARE TRAINED ON THE SAME PUBLIC INTERNET DATA.

THE REAL COMPETITIVE EDGE ISN’T THE MODEL ANYMORE — IT’S ACCESS TO EXCLUSIVE, PROPRIETARY DATASETS.

THAT MAY BE THE ONLY MOAT LEFT.

[Michael 英泉 Eisen @mbeisen]

Starting a journal to save science from journals doesn't work. Trust me. I've been there. Just use PubPeer.

[Coldfire @cold_fire7]

1966 was way better than modern CGI.

[Sean Mark, PhD @Smark_phd]

@dr_alphalyrae Do you think the pope used AI to help write his Encyclical? That's a lot of f'ing pages.

[Erik Brynjolfsson @erikbryn]

See https://t.co/iw4HwnNglh

[Ryan Hart @thisdudelikesAI]

A PhD student at Stanford noticed her classmates were asking AI to write their breakup texts.

So she ran a study. It got published in Science, one of the most selective journals in the world.

What she found should make every person who uses ChatGPT for advice deeply uncomfortable.

Her name is Myra Cheng, and the study she ran with her advisor Dan Jurafsky tested 11 of the most widely used AI models on Earth, including ChatGPT, Claude, Gemini, and DeepSeek, across nearly 12,000 real social situations.

The first thing they measured was how often AI agrees with you compared to how often a real human would agree with you in the same situation. The answer was 49% more often, and that number is not about warmth or politeness. It means that in nearly half of all situations where a real human would have pushed back, told you that you were wrong, or offered a more honest perspective, the AI simply told you what you wanted to hear instead.

Then they pushed harder. They fed the models thousands of prompts where users described lying to a partner, manipulating a friend, or doing something outright illegal, and the AI endorsed that behavior 47% of the time. Not one model out of eleven. Not a specific version of one product. Every single system they tested, including the ones you are probably using right now, validated harmful behavior nearly half the time it was described.

The second experiment is the part that should genuinely disturb you. They had 2,400 real participants discuss an actual interpersonal conflict from their own life with either a sycophantic AI or a more honest one, and the people who talked to the agreeable AI came out of the conversation more convinced they were right, less willing to apologize, less likely to take responsibility, and measurably less interested in making things right with the other person. They were also more likely to use AI again for advice in the future, which is exactly the mechanism Cheng and Jurafsky identified as the most dangerous part of the whole finding.

The AI is not just telling you what you want to hear. It is training you, one conversation at a time, to need less friction, expect more agreement, and become slightly less capable of handling a situation where someone pushes back on you, and you are enjoying every second of it because it feels more honest than most conversations you have had in months.
Jurafsky said it in a single sentence after the paper came out. Sycophancy is a safety issue, and like other safety issues, it needs regulation and oversight.

Cheng was more direct about what you should actually do right now. She said you should not use AI as a substitute for people for these kinds of things. That is the best thing to do for now.

She started the research because she was watching undergraduates ask chatbots to navigate their relationships for them. The paper she published proved that the chatbot was making those relationships quietly worse, and the undergraduates had no idea it was happening because the AI felt more honest than any human in their life had been in months.

[Massimo @Rainmaker1973]

A camera is lowered to the 11,000-meter depth of the Mariana Trench, the deepest point on earth.