Christopher Chabris

Cognitive scientist, professor, author 'Nobody's Fool' ( & 'The Invisible Gorilla,' essayist, chess master, poker player, game lover

[Christopher Chabris @cfchabris]

NOBODY'S FOOL, my second book co-authored with @profsimons, was published today by @BasicBooks! It's about how deception works—the patterns inherent in successful frauds and scams, and the cognitive science of how we respond to them. Much more info here: https://t.co/g3F6vWjds2

[Thomas Yeo @bttyeo]

In a meta-analysis of 210 biomedical AI studies that statistically compared models under cross-validation, 97% used invalid statistical tests.

Here's our new preprint https://t.co/OG58Vkeu49 led by @tianchuzeng @kkli20111 @ZShaoshi @ten_photos 1/N

[Christopher Chabris @cfchabris]

@cremieuxrecueil That would be cool, but it seems hard to see how it would work, since each species has evolved differences in brain/cognitive architecture because they have faced different adaptive pressures, environments, etc. I think interesting general comparisons can be made though.

[Michelle N. Meyer @MichelleNMeyer]

Today I'm thrilled to welcome Paul Han to the Department of Bioethics and Decision Sciences in @GeisingerRsrch & @geisingercohs at @GeisingerHealth. A recent Fulbright Scholar who studies medical decision-making, uncertainty, and risk communication, Dr. Han joins us from NCI/NIH.

[Christopher Chabris @cfchabris]

It wasn’t a meaningful experiment and doesn’t prove much of anything because nothing was controlled or meaningfully compared. The Polgar sisters are wonderful, accomplished people; I know them all. But they don’t teach us anything about IQ and chess skill (where they all reached far above what almost everyone who has ever tried has gotten).

[Christopher Chabris @cfchabris]

@Mankosmash @cremieuxrecueil Compared to the rest of humanity they were ultra close in skill (and ultra far from average). They did not get the same training, since Susan was a strong player when Judit was learning, etc. They show nothing about IQ (except that they are all smart and went very far in chess).

[Christopher Chabris @cfchabris]

@Mankosmash @cremieuxrecueil He didn’t prove anything at all about IQ. His daughters, all obviously very intelligent when you know them, achieved chess skill in the top-1000 to top-10 range for all human beings. No one failed or had a huge gap, compared to any reasonable expectation.

[Christopher Chabris @cfchabris]

Surely Google realizes how central the quality of its search result displays are to most people's understanding of what's real and what's not? Could they devote more resources/attention to the problem of keeping them non-fake?

[Christopher Chabris @cfchabris]

From Botvinnik's famous game with four queens on the board and both kings in check at the same time!

[Christopher Chabris @cfchabris]

@AlexTISYoung Selection on IQ + some specific skill or set of skills ought to be pretty common, since getting to the highest level in any endeavor is probably helped by both being good at the micro-skill involved and intelligent more generally to excel at all the ancillary tasks required.

[Christopher Chabris @cfchabris]

From Botvinnik's famous game with four queens on the board and both kings in check at the same time!

[Vasif Durarbayli @durarbayli]

Google is now using an AI-generated image of Botvinnik over real historical photos. This is getting out of hand.

[Christopher Chabris @cfchabris]

@AlexTISYoung I'm pretty sure the original authors know that, but the Instagram "performance neurology" guy may not, e.g. his inference "the more reasonable explanation is that IQ doesn’t measure intelligence as well as we assumed."

[Christopher Chabris @cfchabris]

@AlexTISYoung Within the full 57-player sample, unsurprisingly, IQ was positively correlated with chess skill, even after controlling for practice.

[Christopher Chabris @cfchabris]

@AlexTISYoung The "elite" were a 23-player subset of a total sample of 57 players. Within that subset, they report a slightly negative correlation between IQ and chess practice, which seems consistent with a form of collider bias (and not with IQ somehow directly decreasing chess skill).

[Christopher Chabris @cfchabris]

Yes! And if you outsource too much thinking it will be hard to develop much understanding.

[kache @yacineMTB]

you can outsource your thinking
but you cannot outsource your understanding

[Michelle N. Meyer @MichelleNMeyer]

🚨eConsent can promote research cost-effectiveness & equity, but how does it compare to traditional consent in terms of informedness? In this RCT, we found the eConsent framework used by NIH's All of Us & Apple ResearchKit was noninferior (& in an exploratory analysis, superior)

[Christopher Chabris @cfchabris]

@MeerMaurits @DrCanChess @chessable How about 1 ... Kd7 2 Rc3 Rh8 3 Ka4 Rc8 4 Rxc8 Kxc8 5 Kb5 Kb7? If 3 Rb2 Kc6 4 Ka4 Ra8+ and we win the b-pawn.

[Christopher Chabris @cfchabris]

@charlesmurray @CHSommers Jeez not this stuff again

[Saloni @salonium]

Many public discussions center around trends and statistics that are not real at all.

For over a decade, there was widespread public discourse about the causes of high and rising maternal mortality in the US.

But, as I've written about before , CDC analyses showed that the apparent rise from 2003 to 2017 was due to a change in measurement https://t.co/pBBcRoDoXQ , when a pregnancy checkbox was added to death certificates, which flowed directly into maternal mortality counts in most cases. Rather than mortality rising, the rate had been stable. Many deaths had been previously missed, and many other countries were undercounting maternal deaths.

This isn't an isolated case.

- People often cite the IHME's estimate of childhood height having fallen in the UK over the past decade. Looking at the data sources, it missed one of the key sources of data on height - a national dataset measuring the height and weight of almost all schoolchildren in the UK, which showed no decline (that data wasn't publicly available until an FOIA request) - and instead the IHME estimates were likely extrapolated based on a global model and smaller, less reliable surveys. https://t.co/dOxnt7ewPD
- I often hear claims about disruptive science having declined over time based on a highly influential paper in Nature. https://t.co/pTAlXnvanB But the key results were affected by a coding bug, which would have showed a decline simply due to this artefact https://t.co/0EXvL55Zer
- The idea that interstate migration in the US has collapsed has led to lots of concern about dynamism and unemployment. But recently, it's been shown that much of the apparent decline was a statistical artefact of how the survey filled in missing responses, causing it to systematically overcount non-movers. Correcting this shows only a very slight decline over time https://t.co/CeIp2kchWL
- The dramatic rise in autism diagnoses, which has spurred lots of commentary about pesticide use and vaccines, actually reflects changes in how autism was defined. In the 1960s, autism described severely disabled, mostly nonverbal children: if a child was verbal or succeeding at school, they were excluded from the diagnosis by definition. The criteria then widened across successive editions of the DSM. Alongside it, it became much easier to get assessed, from requiring a specialist with months-long waiting lists to something that could be done in a few appointments. https://t.co/0L1Y4tKCUd

--

I think this is a persistent problem of people undervaluing data quality and measurement. It may sound dull or academic to care about these issues, but numbers and statistics are a big part of public discussions. They can be the premise of debates that can go on for years and sometimes even decades, and mislead people about social and policy interventions to fix them.

So before spending time arguing about the causes and consequences of a trend or statistic and what should be done about it, it's worth digging into the data to see if it supports the premise at all.

I suspect there are many other discussions affected by this too. Are there others I've missed?

[Christopher Chabris @cfchabris]

Our analysis of multiple large data sets confirms this is a myth about IQ:
“We found that ability measured in youth has a positive association with most occupational, educational, health, and social outcomes later in life … [and] no support for any downside to higher ability and no evidence for a threshold beyond which greater scores cease to be beneficial.”
https://t.co/OBafNa8DCc