Stephanie Tully

Marketing professor @USCMarshall, and visiting scholar @FederalReserve. Researching consumer finances and experiences.

[Stephanie Tully @StephMTully]

Tenured! I am so grateful for all of the people who led me to today. Too many to name you all, but a special thanks to my amazing advisor Tom Meyvis who forever changed my life (I miss you), my awesome colleague & tenure chair @real_k_diehl, & my great coauthor @EeshaSharma.

[Stephanie Tully @StephMTully]

@andre_quentin @alexpmil just had a paper published based on a dataset of A/B tests. He may know.

[Stephanie Tully @StephMTully]

Definitely concerning. The silver lining is that this may make researchers think harder. It puts more emphasis on the need for stronger theory (given less data availability). Tenure committees will need to consider productivity expectations as a result of this though.

[Ethan Mollick @emollick]

Online research surveys may be a dying methodology as techniques for verifying respondents as humans are failing: “Automatic AI detection systems are currently completely unusable… Individual attention checks will no longer be a sufficient tool to ensure good data quality.”

[Stephanie Tully @StephMTully]

Interesting new research on the prevalence of p-hacking in A/B testing in e-commerce by my awesome colleague @alexpmil . Check it out!

[Alex P. Miller @alexpmil]

💡I am happy to announce new research with @KHosanagar. Available below and forthcoming in 𝘐𝘯𝘧𝘰𝘳𝘮𝘢𝘵𝘪𝘰𝘯 𝘚𝘺𝘴𝘵𝘦𝘮𝘴 𝘙𝘦𝘴𝘦𝘢𝘳𝘤𝘩.💡

Link (PDF): https://t.co/HObSypD81s

We study p-hacking, A/B testing, data-driven decision making, and more.

Short thread below👇

[Stephanie Tully @StephMTully]

Why great PhD programs should think like farmers:

[Eric Partaker @Eric_Partaker]

Why great leaders think like a farmer:

[Stephanie Tully @StephMTully]

How do AI disclosures on social media affect our engagement with content? Check out this thread to learn about our new research. Led by two great USC PhD students - Steve Carney & @ignacioriverosg

[Stephanie Tully @StephMTully]

Amazingly accurate!

[Neil Renic @NC_Renic]

Academia is held together by caffeine, free labour, and the two people who nod encouragingly during presentations

[Stephanie Tully @StephMTully]

This is a huge win for the folks at @DataColada and an even bigger win for science!

[Joe Simmons @jpsimmon]

Gino's case against us has been dismissed.

Scientists cannot effectively sue other scientists for exposing fraud/errors in their work.

Those who work to correct the scientific record can sleep better tonight. Those who don’t want it corrected, well, I don’t care how they sleep.

[Stephanie Tully @StephMTully]

Job market candidates - reframe the purpose of your job talk accordingly! I used to view my talks as people evaluating me and my competency. Once I reframed them as an opportunity to tell people what I've learned through research, my talks improved so much.

[Stephanie Tully @StephMTully]

Agreed! Moreover, it's not just that new settings may show different results, but different executions aimed at addressing these behavioral science principles within the same setting can show different results.

[Oleg Urminsky @OlegUrminsky]

"Read about 4 #BehavioralScience principles that #FinTech designers should consider testing in their own context," because the generalizability and application of even highly robust behavioral science principles to a specific context is not trivial.

[Stephanie Tully @StephMTully]

@PeterMcGraw “The Good Place” illustrates this so well!

[Stephanie Tully @StephMTully]

@JZBerman While I think weird wouldn’t upset lots of people (many like to be weird), I think the idea is that republicans in particular like to fit in and being weird is more problematic to them. So it’s more to annoy them than to persuade others.

[Stephanie Tully @StephMTully]

@lakens @andre_quentin @RJ_Youngling Again, I think highly of you and am not trying to pick a fight. At the end of the day, I think we both want high quality research published.

[Stephanie Tully @StephMTully]

I'm disappointed to hear this take. We need more people to pre-reg, not less. The sad truth is that there are always ways a researcher can get around a pre-reg, even with more specificity. (1/4)

[Daniël Lakens @lakens]

This paper will be the starting point for a discussion about whether we should from now on treat an AsPredicted preregistration as No Preregistration.

The template has so little detail you can still publish a set of 5 studies, none of which replicate.

That is problematic.

[Stephanie Tully @StephMTully]

@lakens @andre_quentin @RJ_Youngling As you'll see in my original thread, I do not think any pre-reg is a silver bullet & all research should be critically evaluated w/ and w/o pre-reg. I 100% agree with you that we don't want non-replicable research published & offered another approach in my OT.

[Stephanie Tully @StephMTully]

@lakens @andre_quentin @RJ_Youngling S1 pre-reg had poor wording choice “at least 330”, but the supp materials indicate they had 337 responses which seems consistent w/ posting for 330 on an online platform.

[Stephanie Tully @StephMTully]

@RJ_Youngling Also you posting that I shouldn’t be allowed to express my disagreement with the original thread is quite a hypocritical stance.

[Stephanie Tully @StephMTully]

@RJ_Youngling The original thread says that the failed rep should be a call to question all AP pre-reg. But the OP assumes this w/o reading. A read of the paper, the supp materials, & the pre-regs shows that QRPs are unlikely the issue & other pre-reg wouldn’t fix it.

[Stephanie Tully @StephMTully]

The best way to improve science is to encourage integrity in research practices. Rather than criticizing pre-reg methods, we should be talking about changing the systems and incentives that encourage people to publish non-replicable results. (4/4)

[Stephanie Tully @StephMTully]

Reviewers & readers need to individually evaluate the value of any pre-reg in conjuction with the data. But telling people an imperfect pre-reg is worthless isn't going to bring more people on board, & more specificity in pre-reg isn't going to erase failures to replicate. (3/4)