Frequently Asked Questions (FAQ)

Q: Won't people just lie when providing additional information for the public spreadsheet?
A: We think they won't. We are confident that in spite of recent high-profile cases, actual data and reporting fraud is very infrequent in psychology. Much more common -- and therefore worrisome -- is the extent to which post-hoc reporting decisions based on significance have become acceptable. We think almost all researchers want to play by the rules of the game; we just propose realigning the rules so that they conform to statistical reality and common-sense ideas of honesty.

Q: I'm concerned that if I report the practices we actually used, people will question my research.
A: Your article represents a great achievement. It has addressed an interesting question and passed through a tough process of peer review. We don't think supplying additional information will take that achievement away from you. However, we hope you will agree that as scientists, our primary commitment is to the truth -- not to any given idea, no matter how personally invested we are in it. Ultimately, truth does not reside in any one study, any one paper, or any one lab, but in the overall body of evidence. More accurate reporting in any one article will allow better assessment of this body of evidence overall.

Q: What exactly is sound reporting practice?
A: We have chosen to focus this project on the goal of increasing the amount of information in a research report rather than defining what is good and bad research. However, our basic assumption is that sound practice is principled. That is, it accurately reports any ideas and procedures determined a priori; if these were modified in the course of the research, it reports this fact, along with the reason for doing so. We are not seeking to impose standards that further restrict how people can run studies or analyze their data. We are encouraging standards under which people can feel free to report honestly what they did and why.

Q: If I don't respond, will my research be judged as dishonest?
A: Because only 50% of the authors in your journal and issue have been randomly chosen to take part, there will be no way of identifying you if you choose not to participate. Information regarding who was and was not contacted will be kept strictly confidential as research data. Using this selection approach, we are emphasizing the positive benefits of adding information to your article, rather than the negative judgment of not doing so.

Q: Which journals are being targeted and why?
A: We are focusing on articles published in all 2012 issues (and onward) of Journal of Personality and Social Psychology, Psychological Science, Journal of Experimental Psychology: Learning, Memory, and Cognition, and Journal of Experimental Psychology: General because they represent prominent journals in psychology that are widely read. However, our effort may eventually expand to other psychology journals and other publication years.


Q: Can I submit disclosure information for articles not covered by your initiative?
A: Yes! Please see Contact Us page for details on how to e-mail us your information.


Q: Won't journals or editors be upset because some of the disclosures will involve information on how the reviewing process itself sometimes leads to selective reporting?
A: Yes it is possible that disclosures revealing questionable editorial practices (QEPs) might irk some journals given that these practices have now been identified as clearly inadmissible or indefensible. However, we view this possibility as a small cost compared to the much larger benefit that disclosing this information will have for improving research practices in our field


Q: But why is an independent group of researchers requesting this information? Shouldn't journals be asking for this information?
A: We can't agree more that it is indeed the journals that **SHOULD** be asking for this information rather than an independent group of researchers!!! (Trust me, I'd definitely rather be doing science than spending hundreds of hours e-mailing hundreds of corresponding authors!!!). The fact of that matter is, however, that journals are **NOT** asking for this information at this time. As a (bottom-up) strategy to move us closer to this reality, our initiative aims to raise awareness regarding our journals' completely ineffective reporting standards with the hope that our website inspires journal editors to consider changing editorial policies whereby the 4 categories of methodological details disclosed on this website become a required component of submitted manuscripts. Indeed, there is evidence that our website is already having such an effect.


Q: I've been randomly selected to disclose the methodological information and I support your cause, but isn't unfair because opponents in our intense theoretical debate can now use this information about minor procedural imperfections to advance their cause?
A: We sympathize with this concern, however, we believe that it is completely scientifically justified to then turn around and email your opponents requesting such methodological details (not required to be disclosed) for papers containing findings implicated in the debate! This is not only fair, but completely scientifically reasonable given that it is crucial to know these methodological details to accurately interpret the reported published findings.