Rickard Carlsson

Open Science. Stats. (Board) games. Cargo bikes. Editor at Meta-Psychology.

[Rickard Carlsson @RickCarlsson]

Join our conference on Open Science and Reproducibility. Keynote by @lakens and @annemscheel, with @nulliusverbapod live on stage at the record shop / “folkölscafé” (bar) Mono. Does the fake code make you cringe? Then you will love the replication games! URL below.

[Spencer Greenberg 🔍 @SpencrGreenberg]

The Dunning-Kruger Effect is one of the most famous psychology findings. It's the idea that people of low competence/ability overestimate their ability. But is it REAL? At Clearer Thinking, we investigated (with surprising twists and turns): 🧵

[Matthew B. Jané @MatthewBJane]

Error Report #2: https://t.co/9xyqIVUele

The meta-analysis by Stanyer et al. (2022) contains substantial data extraction errors (i.e., effect size calculation errors). Due to the severe data extraction errors and the meta-analytic results for memory retention scores are incorrect and hyper-inflated. The reported effects from the word-pair associate outcomes are 5x larger than the corrected effects (meta-analytic mean of corrected effects: d=0.19, mean of Stanyer et al. reported effects: d=0.98).

[Ruben C. Arslan @rubenarslan]

New comment in NHB by Farid Anvari, me and friends arguing that psychology is fragmented into the study of too many constructs and measures with too few links.

[Daniël Lakens @lakens]

Very good start of the new Metaror journal as the public peer reviews are as critical of @jamesheathers recent claim that 1 in 7 papers are fraud as I want my peer metascientists to be of such badly supported bold claims. Read the reviews here: https://t.co/VInnAk37HI

[Carlisle Rainey @carlislerainey]

My commentary on Isager, van ‘t Veer, and Lakens’ “Replication value as a function of citation impact and sample size” is now accepted at Meta-Psychology.

CC: @lakens @peder_isager

Preprint: https://t.co/HdfGxqhOWn

[Rickard Carlsson @RickCarlsson]

Hattie’s Visible Learning is a good start!

[Alec Stapp @AlecStapp]

Fun question from @Sam__Enright:

What’s the “anti-reading list” in your field of expertise?

[Rickard Carlsson @RickCarlsson]

@squig @lakens @mVranka Ok that is fair!

[Rickard Carlsson @RickCarlsson]

@squig @lakens @mVranka What IRB is ok with the deliberate act of collecting human (however trivial) data and not publishing the results in any form? As someone who teaches research ethics, I am deeply intrigued!

[Rickard Carlsson @RickCarlsson]

@squig @lakens @mVranka There are, of course, situations where the benefits of research can outweigh some rights of participants, but a cavalier attitude towards it is quite schocking to me. Where I come from (Sweden) this type of deliberate behavior is probably even a punishable offense.

[Rickard Carlsson @RickCarlsson]

@lakens @Kim_De_Roover It is a real problem though. Many researchers shares the view that the journals are crappy and don’t review for it. Everyone loses on the prestige game.

[Rickard Carlsson @RickCarlsson]

@lakens @squig @mVranka I would prefer A. You are contributing to a problematic culture in academia by keeping your information secret. You then talk about all these replications you have done but people can’t verify it. It adds confusion and the net problems for everyone else is worse than your gain.

[Rickard Carlsson @RickCarlsson]

@lakens @squig @mVranka I think that participation in research is a cost by itself regardless of monetary compensation. Participants have the right to learn about the results of the study they took part in

[Rickard Carlsson @RickCarlsson]

@squig @lakens @mVranka I can in principle agree that data can be used without being shared in such a way that it’s balanced with costs. Rare but possible. Also if data is eg simulations it’s different.

[Rickard Carlsson @RickCarlsson]

@squig @lakens @mVranka The idea is that involving non-scientists in research incurs an ethical cost benefit analysis and that involves actually using the data. How unethical depends on type of data. It’s always problematic to collect and throw away but level varies wildly.

[Rickard Carlsson @RickCarlsson]

@lakens @squig @mVranka @Meta_Psy I think that pre-registrations should have a preprepared reporting function and that everyone always upload data meta-data and main results. It can be pretty much automated. Embargo shouldn’t be on lifting the registration but on sharing the results!

[Rickard Carlsson @RickCarlsson]

@lakens @squig @mVranka I would be very happy working on how we can create a good format for this. We tried with file drawer reports at @Meta_Psy but not with great success. Hard to review, reproduce and little interest in submitting. Clearly a lot more work is needed on that format to work.