Frequently Asked Questions (FAQ)
Q: Won't people just lie when providing additional information for the public spreadsheet?
A: We think they won't. We are confident that in spite of recent high-profile cases, actual data and reporting fraud is very infrequent in psychology. Much more common -- and therefore worrisome -- is the extent to which post-hoc reporting decisions based on significance have become acceptable. We think almost all researchers want to play by the rules of the game; we just propose realigning the rules so that they conform to statistical reality and common-sense ideas of honesty.
Q: I'm concerned that if I report the practices we actually used, people will question my research.
A: Your article represents a great achievement. It has addressed an interesting question and passed through a tough process of peer review. We don't think supplying additional information will take that achievement away from you. However, we hope you will agree that as scientists, our primary commitment is to the truth -- not to any given idea, no matter how personally invested we are in it. Ultimately, truth does not reside in any one study, any one paper, or any one lab, but in the overall body of evidence. More accurate reporting in any one article will allow better assessment of this body of evidence overall.
Q: What exactly is sound reporting practice?
A: We have chosen to focus this project on the goal of increasing the amount of information in a research report rather than defining what is good and bad research. However, our basic assumption is that sound practice is principled. That is, it accurately reports any ideas and procedures determined a priori; if these were modified in the course of the research, it reports this fact, along with the reason for doing so. We are not seeking to impose standards that further restrict how people can run studies or analyze their data. We are encouraging standards under which people can feel free to report honestly what they did and why.
Q: If I don't respond, will my research be judged as dishonest?
A: Because only 50% of the authors in your journal and issue have been randomly chosen to take part, there will be no way of identifying you if you choose not to participate. Information regarding who was and was not contacted will be kept strictly confidential as research data. Using this selection approach, we are emphasizing the positive benefits of adding information to your article, rather than the negative judgment of not doing so.
Q: Which journals are being targeted and why?
A: We are focusing on articles published in all 2012 issues (and onward) of Journal of Personality and Social Psychology, Psychological Science, Journal of Experimental Psychology: Learning, Memory, and Cognition, and Journal of Experimental Psychology: General because they represent prominent journals in psychology that are widely read. However, our effort may eventually expand to other psychology journals and other publication years.