PS...I usually include an f-bomb or two in my blogs. I saved them for the postscript this time.

FFFFFFFFFFUUUUUUUUUUUUUCCCCCCCCCKKKKKKKKKKKK!!!!

And a second one: FFFFUUUUUCCCCCKKKKK YOOOOOOOOOUUUUUUU FORMER WILL!!!

There, that feels better. I wanted to use this postscript for some context setting, and to muse about this whole experience.

Context

By my timeline, the initial study was conducted sometime 2009-2010ish. Science ended up being the third journal we tried (I know, right?). I think it was under review there for about a year? Could be wrong.

For the replication, Bob Calin-Jageman and his team first got in touch with me (according to my email) in May 2014. So 2.5 years from first contact to publication. Anyone who claims that replicators are in it for the short easy process or the fame are crazy. All credit to Bob and his team: they kept me in the loop throughout. I did my best to help them recreate the study I ran (even though I wasn't super duper optimistic). They were beyond cordial and professional throughout. On my end, and I hope on theirs, everything was positive.

I did get to review the manuscript, after it had already passed an initial round of review at PLoS. I recommenced acceptance, and I think my feedback was pretty minimal (word choice here, different stats presentation there). Typical trivial Reviewer 3 BS.

Musing

Facts aside: Is it fun to find out that a study you published in a high profile outlet back in the day does not hold up well to more rigorous scrutiny? Oh hell no. I highly recommend you avoid the experience.

How do you avoid the experience? Make sure you're more rigorous up front. More power. Open science, etc. Follow Mickey's advice and CHECK YOURSELF BEFORE YOU WRECK YOURSELF while being open to REVISING YOUR BELIEFS. Embrace methodological reform. It isn't going away. We can look at old papers and say "sure, but we didn't know better then." Even the famous False Positive Psychology paper is already dated (25 [oops: 20] per cell, ha!). Maybe we will be able to pick a year and say "LOOK everyone, we get it, before YEAR XXXX we didn't always know better. But after YEAR XXXX nobody has an excuse for not changing yet. Nobody can claim ignorance on these issues after YEAR XXXX." I have no idea which year counts as that YEAR XXXX. But I am quite confident that it is already in the past rather than still in the future.

I think my Methodological Awakening (tm) started in ~2012 and continues to this day. I'm optimistic about the direction our field is headed (though occasionally dismayed to see studies at least as weak as mine published in 2016 in places like Psychological Science, JPSP, PNAS, and everywhere. Progress ain't linear). And I'm super proud of the steps I've taken to both 1) educate grad students about methods (my favorite class to teach), and 2) upgrade my own science. My best 2-3 papers HANDS DOWN are currently under review. I hope I can say the same thing again next year (assuming, of course, that I'm not talking about the same papers still being under review).

Science is dead. Long live science.