I discovered something interesting at SPUDM24 last week. (That is the European Judgement & Decision Making Conference which was held in Barcelona this year.) I was speaking about things that are happening in psychology having to do with replication and publication when I mentioned that there was something that each person in the (surprisingly large) audience could do on his or her own to help force new norms on the journals:
“When a journal asks you to review an empirical manuscript, write back to the editor and say you will do it if only if you can get the data.”
Wow. I hadn’t realized the power of that idea until I said it aloud and saw/heard the reaction. We are authors AND we are reviewers. So, if we start asking for the data, we have to be willing to be asked for the data. And we have to recognize that asking (or asking for all of it) is not appropriate in all cases. But when you do so appropriately, what is an action editor / journal then to do?
I don’t know whether or not I love this idea. But I think it’s worth more thought.