[Click here for article (@ Perspectives on Psychological Science) reporting on our initiative.]
We're glad to report that the success of PsychDisclosure.org has influenced the journal Psychological Science to raise their reporting standards! Since January 1, 2014, all authors who submit a manuscript to Psychological Science must excplicitly confirm that they have disclosed all information covered by the 4 categories (see here and here for more details on Psychological Science's new submission guidelines). Also, we're excited to announce that disclosure information available at PsychDisclosure.org will soon be integrated at CurateScience.org!
Recently, several common research practices in psychology have been highlighted as impeding cumulative knowledge development and hurting the reputation of our field. For instance, it has become acceptable -- and action editors often have required authors -- to selectively exclude and report measures, manipulations, samples, and analyses on the basis of whether these practices yield statistically significant results or tell more compelling stories rather than for principled reasons (John, Loewenstein, & Prelec, 2012; Stapel final report, 2012). (Though of course many methodological details are also often not reported for reasons which have nothing to do with increasing the statistical significance or compellingness of the story.)
Regardless of the source of these suboptimal research practices, it is our belief that many of us would appreciate the opportunity to provide more details about the methods actually used to obtain findings reported in published articles. Our initiative provides this opportunity. Our effort builds upon a Simmons, Nelson, and Simonsohn's (2012) recently proposed initiative wherein authors submitting manuscripts for publication voluntarily include a 21-word disclosure statement regarding crucial methodological details that are not required to be disclosed under currently accepted reporting standards.
The primary benefits of our initiative include the following:
We aim to achieve this by inviting (via email) a subset of corresponding authors of recently published articles (2012 and onward) in prominent psychology journals to publically disclose important methodological details, derived from the four methodological categories from Simmons et al.'s 21-word disclosure statement initiative. (We are now accepting disclosure information for any article in any journal; please see Contact Us page for more details.)
For all studies in your recently published article titled [publication title], please endorse the following statements: (please type an X to indicate your answer)
We reported the total number of observations which were excluded (if
any) and the criterion for doing so. (If no observations excluded, please
Yes: ___ No: ___
If no, please report this information here (e.g., data from 3 participants in Study 2 excluded due to computer malfunction; 4 participants in Study 1 excluded for not following instructions):
We reported all tested experimental conditions, including failed manipulations.
Yes: ___ No: ___
If no, please provide brief explanation for not reporting this information (e.g., critical software implementation error; editorial request):
We reported all administered measures/items. Yes: ___ No: ___
If no, please provide brief explanation for not reporting this information (e.g., measures not related to research question; scores from unreported measure insufficiently reliable):
We reported (a) how we determined our sample size and (b) our data collection stopping rule. Yes: ___ No: ___
If no, please describe (a) the basis for the sample sizes used and (b) how you decided to stop collecting data (e.g., decided ahead of time to collect data until minimum sample size achieved and this was followed; sample size determined by power analysis but did not achieve it by the end of term):
Our initiative has received appropriate ethics clearance in accordance with APA guidelines. To protect the anonymity of non-respondents, only a randomly determined subset (i.e., half) of the corresponding authors in your journal and issue will be contacted.
We emphasize that the additional information requested is not intended to question or stigmatize published research, but to give a more accurate picture of the actual methods used to obtain the findings, correcting for artificially rigid standards of evidence in publication. The project is committed to transparency and open science practices (see all project information and materials here).