| In press | LeBel & Campbell | Heightened sensitivity to temperature cues in highly anxiously attached individuals: Real or elusive phenomenon? |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: Full Disclosure (details provided in the preregistered study protocols, accessible via links in the article)
|
| Mar 2013 | Ackerman, Kashy, Donnellan et al. | The Interpersonal Legacy of a Positive Family Climate in Adolescence |
| | - Exclusions: We excluded some people from the sample because they did not have the observational data in adolescence or data about marital romantic partnerships in adulthood. We did not exclude anyone who otherwise met the selection criteria for our study.
- Conditions: Full Disclosure (We had no manipulations. In our analyses we did examine a couple of parenting variables but then we learned that another team working with the same data set was using those outcomes so we dropped them.)
- Measures: No. There are many more measures of different constructs in the dataset. We selected those most relevant to our investigation.
- Sample Size: The data were already collected so we simply analyzed what was available to us from this existing project.
|
| | Goldfarb & Treisman | Counting Multidimensional Objects: Implications for the Neural-Synchrony Theory |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: We decided ahead of time to collect data until minimum sample size was achieved and this was followed.
|
| | Hehman, Leitner, Deegan et al. | Facial Structure Is Indicative of Explicit Support for Prejudicial Beliefs |
| | - Exclusions: Full Disclosure
- Conditions: In Study 3, there was no difference between conditions on any of our dependent measures, nor interactions. Therefore, there was no evidence that this manipulation was effective or that it even functionally "existed."
- Measures: Full Disclosure
- Sample Size: In Study 1, we collected as many as possible during a semester. In Study 2, we aimed for 100 participants as we were unsure what effect size to expect, and stopped when we reached that goal. In Study 3, we based our sample goal on the size of the effect demonstrated in Study 2.
|
| | Schneider, Eerland, van Harreveld et al. | One Way and the Other: The Bidirectional Relationship Between Ambivalence and Body Movement |
| | - Exclusions: Full Disclosure
- Conditions: Ambivalence is hard to manipulate experimentally among Dutch students, we pretested more self-written articles but used only the strongest manipulation. Because there were no differences in ambivalence, we did not analyze these data further, and as such, this information was not interesting.
- Measures: Some were significant, some were not, but the most important reason was doubts regarding validity. However, we mention the additional measures in the paper and interested researchers may contact us about these measures.
- Sample Size: We decided ahead of time to collect data until minimum sample size achieved, or data collection period ended, and this was followed.
|
| | Sutin, Terracciano, Milaneschi et al. | The Effect of Birth Cohort on Well-Being: The Legacy of Economic Hard Times |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: The Baltimore Longitudinal Study of Aging (BLSA) is an on-going epidemiological study of normal aging. BLSA participants undergo extensive testing during each visit that lasts for 2-3 days. This testing includes numerous measures of physical, cognitive, and emotional health. The National Health and Nutrition Examination Survey (NHANES I) was likewise a large study that included numerous measures of health and nutrition. From both studies, we selected the measure that was relevant to our research question.
- Sample Size: We selected every participant who had completed the CES-D from the time it was introduced into the BLSA (1979) to the time of the initial data analysis (2010). BLSA participants continue to fill out the CES-D at every visit. From NHANES I, we selected adult participants who completed the CES-D.
|
| | Van der Burg, Awh, & Olivers | The Capacity of Audiovisual Integration Is Limited to One Item |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: We included a predetermined number of subjects in each experiment. This was based on experience.
|
| Feb 2013 | Caparos, Linnell, Bremner et al. | Do Local and Global Perceptual Biases Tell Us Anything About Local and Global Selective Attention? |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Participants performed two blocks of trials. Only the data obtained in the first block are reported in the paper. The effects (reported in the paper) were also present in the second block, however, there were also carry-over effects that were not sufficiently reliable to be reported. The paper format (brief report) was not adequate for us to discuss these effects.
- Sample Size: We aimed to test at least 50 participants in each group (a group of British participants and a group of traditional African participants). In Africa, two weeks of testing were dedicated to data collection for this experiment. We tested as many participants as we could during these two weeks (reaching a sample size of 58 in the African group). We then tested an equivalent number of British participants.
|
| | Jamieson, Koslov, Nock et al. | Experiencing Discrimination Increases Risk Taking |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: We collected pre-experiment questionnaires on-line that all participants completed and were beyond the scope of the current article. Measures included items such as intergroup contact, personality measures, and other individual differences
- Sample Size: As in all the studies in my lab we decide on the targeted N based on previous studies and power analysis. We then run 10% over the targeted amount due to typical loss in physiolgocial measures and biological samples (due to electrical interference, loss of signal, contaminated saliva samples, etc). We *never* analyze our data until the study is complete primarily because we send out biological samples in batch so that they are assayed at one time. I didn't respond (yes or no) above because this stopping rule is not stated explictly, but we do cite standard articles and chapters that outline this protocol and space restraints prevent this type of extra information.
|
| | Laran & Salerno | Life-History Strategy, Food Choice, and Caloric Consumption |
| | - Exclusions: Full Disclosure
- Conditions: An entire study, from the first submission, did not make the final version of the paper as per editorial request.
- Measures: In study 2, we included a few other filler questions unrelated to our research questions that were included to support our cover story. These measures did not vary as a function of our experimental conditions.
- Sample Size: Study 1: We aimed to collect at least 25 participants per cell. We obtained our final sample by asking our undergraduate research assistants to recruit as many participants as they could over a two day period of a few hours each day and ended up with more participants than the 25 per cell initially expected (n = 121). Studies 2 and 3: We aimed to collect 40 subjects per cell given that the dependent variable was binary. Our total n was lower than expected in Study 2 (n = 238) and Study 3 (n = 144) based on fluctuation in attendance rates for the sessions held at our lab.
|
| | Marinovic, Pearce, & Arnold | Attentional-Tracking Acuity Is Modulated by Illusory Changes in Perceived Speed |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: Based on past experience of motion adaptation phenomena, we determined on a sample size of 8, as this should be more than ample to detect a low-level visual aftereffect. We stopped testing once we had tested all the participants.
|
| | Simonsohn & Gino | Daily Horizons: Evidence of Narrow Bracketing in Judgment From 10 Years of M.B.A. Admissions Interviews |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: Full Disclosure
|
| Jan 2013 | Briñol, Gascó, Petty et al. | Treating Thoughts as Material Objects Can Increase or Decrease Their Impact on Evaluation |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: We excluded any items that were unrelated to the research questions or that were included for exploratory purposes. Furthermore, we focused on the items that were included in all the studies within the paper in order to maintain convergence across experiments.
- Sample Size: The number selected was based on our prior experience with this research topic and the number of participants that could be successfully recruited within an academic term (without crossing terms). Also, given that not all participants who signed up for the experiments in advance showed up to participate, the total number of subjects per cell was not identical in all cases. Also, we did not conduct any statistical tests until we were done collecting data.
|
| | Cook, Johnston, & Heyes | Facial Self-Imitation: Objective Measurement Reveals No Improvement Without Visual Feedback |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: Decided ahead of time to collect data until minimum sample size achieved and this was followed
|
| | Kille, Forest & Wood | Tall, Dark, and Stable: Embodiment Motivates Mate Selection Preferences |
| | - Exclusions: Data from 2 participants were excluded: 1 was unable to sit in either of our chairs—which constituted our manipulation of physical stability—due to due his/her weight, and 1 did not comply with the researcher's assignment to condition. When these participants are included in our analyses (the participant who was unable to use our chair was assigned to a separate "stable" chair), results remained significant.
- Conditions: Full Disclosure
- Measures: We also gathered measures to address a separate research question regarding participants’ perceptions of the stability of their own singlehood status that we did not report. As we predicted, we found that participants in the physically unstable (vs. stable) condition felt that their singlehood was less likely to last. After participants completed all of the measures reported in the paper, they went on to complete measures assessing their preferences for products (e.g., Aerobics step bench) unrelated to relationships.
- Sample Size: We recruited participants in a campus student center, which requires reserving time slots in advance. We reserved a number of timeslots that we felt would give us adequate access to participants to obtain at least 20 participants per cell in our design and collected the data until our slots were completed.
|
| | Lerner, Yi, & Weber | The Financial Costs of Sadness |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: We aimed for 30 subjects per cell, based on past experience with these kinds of studies. We did not conduct a power analysis. Once we reached at least 30 per cell, we continued running until all previously scheduled subjects had been run
|
| | Spunt & Lieberman | The Busy Social Brain: Evidence for Automaticity and Control in the Neural Systems Supporting Social Cognition and Action Understanding |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Participants in the study completed several validated personality questionnaires following their MRI session. To be perfectly honest, their inclusion was primarily motivated by convenience: given that MRI data is expensive to collect, we often include additional measurements that are secondary to the main purpose of the study but which will permit theoretically-related follow-up analyses (for instance, examining the moderating influence of a personality variable on the strength of an observed group effect). For the published study in question, I have not had the time to even begin to look at this individual difference data.
- Sample Size: We determined sample size in heuristic-fashion based on our previously published studies using this paradigm (Spunt, Satpute, & Lieberman, 2012, Journal of Cognitive Neuroscience; Spunt, Falk, & Lieberman, 2010, Psychological Science). We collected a few more subjects than in those previous studies given that this study was examining the moderating effect of an additional manipulation (i.e., memory load). I completely acknowledge that this is a highly informal procedure; at the time it was unclear how best to formally determine sample size. While there are still many ambiguities in how to best determine sample size for fMRI studies, recent publications and software-releases (e.g., http://fmripower.org/) are beginning to clarify things.
|
| Dec 2012 | Berntsen, Johannessen, Thomsen et al. | Peace and War: Trajectories of Posttraumatic Stress Disorder Symptoms Before, During, and After Military Deployment in Afghanistan |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure (As stated in the supplementary material as well as in the paper: The reported study was part of a large survey conducted through the military. It included many questionnaires and we had to focus on the ones of key relevance. This is often the case with this size of data bases (unlike the typical lab experiment).)
- Sample Size: Full Disclosure
|
| | Gaissmaier & Gigerenzer | 9/11, Act II A Fine-Grained Analysis of Regional Variations in Traffic Fatalities in the Aftermath of the Terrorist Attacks |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure (Note: We analyzed publicly available observational data from the 50 US states (+DC) only and thus did not have any experimental conditions)
- Measures: In response to the editor’s and reviewers’ comments, we conducted and/or discussed some additional analyses, which were only presented to the editors and reviewers, but not included in the paper – either because they yielded redundant results (e.g., statistics per inhabitant with driver’s licence rather than per each inhabitant) or because the number of observations was too small to yield reliable results (e.g., number of drunk driving citations on a state-by-state level)
- Sample Size: Full Disclosure (Note: The sample size was simply determined by the number of states)
|
| | Korjoukov, Jeurissen, Kloosterman et al. | The Time Course of Perceptual Grouping in Natural Scenes |
| | - Exclusions: In the “Size” experiment, described in the Appendix, we excluded data from 15 participants due to a technical error. Another data set, collected over 12 participants, was excluded due to a difference in procedure (difference in the overall number of trials and session duration).
- Conditions: We did not report the differences between three d-conditions in the experiments because they are irrelevant for main focus of the study.
- Measures: Full Disclosure
- Sample Size: We found that the results were highly significant after we tested a predetermined number of participants.
|
| | Rodeheffer, Hill & Lord | Does This Recession Make Me Look Black? The Effect of Resource Scarcity on the Categorization of Biracial Faces |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: For both studies, we decided ahead of time to aim for 30 participants per cell and stopped data collection once we reached that target. In Study 1 we went slightly over (N = 35) and in Study 2 we were slightly under (N = 27). Discrepancies were due to fluctuations in participant attendance rates. We did not look at my data before data collection ended, nor did we run more participants once we had decided when the last experiment session would be.
|
| Nov 2012 | Matthews | How Much Do Incidental Values Affect the Judgment of Time |
| | - Exclusions: When participants were excluded on the basis of their responses to questions asked during the task (e.g., extreme values), I explained how many participants were excluded and the basis for exclusion in the Supplementary Materials for the paper (which describe the methods in detail). In addition, several of my studies were run on-line. For these studies, I applied eligibility criteria to determine whether the participant was eligible to be included in the sample. These included age (participants had to be at least 16), answering all questions (i.e., not choosing to withdraw from the task), and not having an ip address that appeared earlier in the study or in one of the earlier studies in the series. These eligibility criteria are fully documented in the Supplementary Materials. Ineligible responses were never analysed and there is no way of knowing, for example, how many actual participants they represent (e.g., duplicate ip addresses may be one person or several), so I did not report the precise numbers of responses that were screened on these grounds (although I am happy to provide that information).
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: Sample sizes were based on power analysis and sampling continued until a minimum sample size was achieved. It was not possible to specify in advance precisely what sample size would be tested because I could not control exactly how many eligible people would sign up. The policy was to recruit more participants than were needed to give high power to detect the effect of interest (see Table 1 of the paper). That is, I aimed to “over-shoot” slightly so as to have high power after removing ineligible respondents. Samples were intentionally larger for on-line studies because (a) participants were easier to recruit, and (b) the more heterogeneous sample and testing environment might reduce effect size. There was no optional stopping.
|
| | Stallen, De Dreu, Shalvi et al. | The Herding Hormone: Oxytocin Stimulates In-Group Conformity |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: The number of participants was determined before starting the study, and in line with typical sample sizes used in studies in this field. Data collection was terminated upon reaching the predefined sample size
|
| | Weems, Scott, Banks & Graham | Is TV Traumatic for All Youths? The Role of Preexisting Posttraumatic-Stress Symptoms in the Link Between Disaster Coverage and Stress |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure (non-experimental research)
- Measures: Data was from a larger longitudinal study and we do reference this fact and the other work (previously published studies) in the paper. We also tested a number of alternative explanations with additional measures and this we report in our supplemental data available online.
- Sample Size: Full Disclosure
|
| Oct 2012 | Berman & Small | Self-interest without selfishness: The hedonic benefit of imposed self-interest |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: We took additional measures that were not reported in the final manuscript. These measures were not included because: a) they were irrelevant to the main hypothesis and were not analyzed; b) they were removed during the review process; or c) reporting the results did not fit within the manuscript word count.
- Sample Size: All of our sample sizes were determined in advance of collecting data and data collection stopped when the target sample sizes were reached. For study 2, we purchased a set of gift cards ahead of time in bulk, and stopped when we ran out of gift cards.
|
| | Brascamp & Blake | Inattention abolishes binocular rivalry: Perceptual evidence |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: We empirically determined the amount of data needed to get a clear and interpretable data pattern in our two reference conditions (called 'Attended' and 'Absent'). For our condition of interest ('Unattended') we then collected the same amount of data.
|
| | Fairbanks, Way, Breidenthal et al. | Maternal and offspring dopamine D4 receptor genotypes interact to influence juvenile impulsivity in vervet monkeys |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: The sample size was not determined a priori. We tested all of the juvenile monkeys available in our colony during the 7 year time period of the research.
|
| | Grossman, Karasawa, Izumi et al. | Aging and wisdom: Culture matters |
| | - Exclusions: Data from 22 American participants excluded for being outside the matching age range of the corresponding Japanese sample (25-75). The survey company in charge of subject recruitment in Japan did not recruit Japanese over 76 yrs. To match the age range of its Japanese equivalent, we reduced the American sample. Results remain virtually identical when examining all American adults. The full American sample was reported in the initial paper from this project (collected before the Japanese counterpart; Grossmann et al., 2010). Results are very similar across both types of samples.
- Conditions: Full Disclosure
- Measures: The study was part of a large-scale project examining cultural differences between Americans and Japanese in cognition and emotion. Thus, in other sessions participants were tested for a variety of instruments dealing with cultural constructs of independence vs. interdependence; holistic attention; positivity bias in memory, etc. Reporting these measures was outside the scope of the paper both thematically and in terms of page length. Further, many of these tasks were not yet entirely coded and analyzed at the time this paper was in press.
- Sample Size: We used an age-stratified random sample with oversampling. The latter was done to ensure that we have a comparable number of individuals of both genders, different levels of education (junior High vs. college), and in each of the three age groups (25-40; 41-55; 60-75). Our goal was to have at least 25 people in each cell. In the U.S., we stopped collecting data when we achieved this quota for the cells we have to oversample; in Japan a survey company made a corresponding decision.
|
| | Hu, Rosenfeld, & Bodenhausen | Combating automatic autobiographical associations: The effect of instruction and training in strategically concealing information in the autobiographical Implicit Association Test |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: Sample sizes are determined based on previous similar experiments conducted by the first author. We decided to stop data collection after we reached the predetermined number, which is N=16 in each condition.
|
| | Shalvi, Eldar, & Bereby-Meyer | Honesty requires time (and lack of justifications) |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: We instructed the RA to target at 30-35 participants per cell, based on our experience with the strength of studied effects and a general convention of good practice in the field. Data collection was stopped once this target was met. We ended up with a slightly higher n-per-cell due to good show up rates in some experimental sessions.
|
| | Ybarra, Lee, & Gonzalez | Supportive social relationships attenuate the appeal of choice |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Study 1 contained measures not related to research question.
- Sample Size: No (Disclosure statement coming soon.)
|
| Sep 2012 | Cain, Vul, Clark et al. | A Bayesian optimal foraging model of human visual search. |
| | - Exclusions: Full Disclosure
- Conditions: All between-subjects conditions and manipulations were reported. We attempted a within-subjects version but it was too difficult for participants and was canceled.
- Measures: We also collected additional demographic information (e.g. video game playing behavior) for future recruitment purposes. This were not analyzed in relation to the dependent measures of this study.
- Sample Size: We collected 10 participants per group (30 total) and examined the results. The results were unclear so we decided to collect an additional 5 participants per group. At that point a clearer picture had emerged and we stopped data collection. These values were informed by previous studies from our lab using related paradigms that tested 12 participants per group.
|
| | Emery, Finkel, & Pedersen | Pulmonary function as a cause of cognitive aging |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure (Not applicable.)
- Measures: Other measures assessed but not reported because data came from longitudinal, population based study of multiple outcomes
- Sample Size: Coming soon.
|
| | Grant & Dutton | Beneficiary or benefactor: Are people more prosocial when they reflect on receiving or giving? |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: We collected additional questionnaire measures unrelated to the research question.
- Sample Size: For Study 1 the sample was the population of employees at the call center. For Study 2 we set our data collection termination rule in advance based on power calculations from Cohen (1992 PB) and sample size availability in the behavioral lab. We did not modify the rule in the course of the research.
|
| | O’Hara, Gibbons, Gerrard et al. | Greater exposure to sexual content in popular movies predicts earlier sexual debut and increased sexual risk taking |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure (non-experimental)
- Measures: These data came from an extensive longitudinal study of media and health and many measures were not related to this research question.
- Sample Size: A power analysis was used to determine that the analyses from the original grant proposal required successful follow-up with 2200 never-smokers at baseline resulting in an original sample of 6522 participants at Time 1.
|
| | Raby, Cicchetti, Carlson et al. | Genetic and caregiving-based contributions to infant attachment: Unique associations with distress reactivity and attachment security |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Our study was part of' a longitudinal project that collected several other measures not relevant to our hypotheses. However exploratory analyses (involving measures of infant temperament) were completed at a reviewer's request but the results were not reported because the measures were not sufficiently reliable.
- Sample Size: Full Disclosure
|
| Aug 2012 | Monti, Parsons, & Osherson | Thought beyond language: Neural dissociation of algebra and natural language |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: Sample size (20 subs + 1 pilot) was decided at the time we put in our request MRI scanning slots for the study and set in line with typical sample sizes in the field. Collection was terminated upon reaching the target N.
|
| | Pleskac | Comparability effects in probability judgments. |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: I sought to obtain 30 participants. The design was completely within subjects and so this sample size provides sufficient power (greater than 80%) at the aggregate level. Note also I collected enough observations per subject 450 so I can actually treat each participants as his or her own experiment.
|
| Jul 2012 | Bélanger, Slattery, Mayberry et al. | Skilled deaf readers have an enhanced perceptual span in reading. |
| | - Exclusions: One participant was excluded from the experiment based on he/she not meeting our inclusion criterion on non-verbal IQ. This was not reported because of the limited space available to report more relevant results.
- Conditions: Full Disclosure
- Measures: We had a background test to assess ASL skills and the test was not well suited to our adult population. Participants's scores reached ceiling or near ceiling score so it could not be used as a covariate as originally planned.
- Sample Size: We ran the maximum number of people that we could find in our special population that also met our inclusion criteria (those were included in the paper).
|
| | Fuller-Rowell, Evans, & Ong | Poverty and health: The mediating role of perceived discrimination |
| | - Exclusions: Full Disclosure (We used FIML estimation in order to be able to include all individuals who participated in W3 of the study in the models.)
- Conditions: Full Disclosure (Not applicable. Our study was not experimental.)
- Measures: Study included a large number of measures. We only discussed the measures relevant to the analyses presented in our paper.
- Sample Size: Coming soon.
|
| | Leander, Chartrand, & Bargh | You give me the chills: Embodied reactions to inappropriate amounts of behavioral mimicry |
| | - Exclusions: Full Disclosure
- Conditions: In the third study we ran (now Study 1), I attempted to add a second, male experimenter, but his data were uninterpretable and only seemed to add error variance. That is why all studies specifically report using only a female experimenter.
- Measures: Scales/questionnaires unrelated to the research question were not reported. The DVs were the first things we assessed after the manipulations and we included additional questionnaires afterwards so as to make full use of the participants' time while we had them in the lab. It seems superfluous and distracting to report such information if it is independent of the study procedure and would not be meaningful for the purpose of someone trying to replicate the findings (which, in my mind, is the essence of how to write a research report).
- Sample Size: An a priori decision was made to stop data collection at the end of the given block/semester.
|
| | Longo, Long, & Haggard | Mapping the invisible hand: A body model of a phantom limb |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: This was a single case-study of an individual with congenital limb absence so the issue of determining sample size is not applicable.
|
| | Mazerolle, Régner, Morisset et al. | Stereotype threat strengthens automatic recall and undermines controlled processes in older adults |
| | - Exclusions: We removed 4 participants (two young and two old participants) for being outliers (on Cook's D and SSD, following Judd & McClelland, 1989; McClelland, 2000). Because of PS word count for short reports, we didn'tmentioned these informations.
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: A french version of the process dissociation procedure (PDP) was constructed in French, including four sets of words (similar in letters and syllables numbers, and frequency, based on Jacoby's recommendations, 1998) corresponding to four instructions conditions (inclusion, inclusion filler, exclusion, exclusion filler). To account for possible differences in words sets, we counterbalanced each set, creating 4 PDP versions. Then, we counterbalanced each PDP version with threat conditions and age groups. We decided that 56 participants for each PDP version was sufficient to account for potential differences, resulting in 56 participants X 4 PDP versions = 224 participants. Analysis didn't shown any difference between the 4 PDP versions. Sample size was decided ahead and was followed. Again, because of PS word count for short reports, we didn't mentioned these informations.
|
| | Parise & Csibra | Electrophysiological evidence for the understanding of maternal speech by 9-month-old infants |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: 14 infants / group were targeted but the sample overshoot because less infants were excluded due to bad data than expected.
|
| | Wolfe | Saved by a log: How do humans perform hybrid visual and memory search? |
| | - Exclusions: We excluded outlier trials with RTs > 7000 msec. That seems to have eliminated 8 of 12000+ trials. Usually we report that exclusion. I think that must have fallen victim to the word count restriction in Psych Sci.
- Conditions: Frankly this is a bit of a silly question. We would typically run various pilot versions of experiments to make sure that the code works that we know how long the task takes etc. Once we are sure we are not wasting our time collecting garbage we would run a decently powered experiment. What you (I assume) really want to know is whether we ran more or less the same experiment 20 times and are only reporting the one time that p scraped over 0.05. That we did not do.
- Measures: Oh come now we record all sorts of things out of a sense of completeness. For example in this experiment I know the location on the screen of every target item. There are undoubtedly effects of this variable on reaction time. Those effects might be interesting. We do not happen to have analyzed that variable. Is it unrelated to the research question? I dont know. Might be a good exploration for a rainy day or for someone who asks for our data.
- Sample Size: Many years of experience with experiments of this sort suggest that if we collect on the order of 50 data points in each cell of the experiment (in this case 50 trials target present and absent for each combination of visual and memory set size) and if we run 10-12 observers that our results will have sufficient power to see differences between conditions when such differences exist.
|
| Jun 2012 | Donkin & Nosofsky | A power-law model of psychological memory strength in short- and long-term recognition |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: We decided to collect four participants (each of whom completed 10 sessions) because our goal was to fit a quantitative model to individual subject response time distributions. This number is standard for this type of analysis. The experiment was a replication of a previous study and our own pilot studies revealed that the effect was remarkably robust in individuals (even when they completed the task for just one hour) which told us that we did not need to collect more participants than is standard.
|
| | Hirsh, Kang, & Bodenhausen | Personalized persuasion: Tailoring persuasive appeals to recipients' personality traits |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: Sample size was determined a priori using power analysis software based on effect size estimates from previous research. Data collection was stopped once we achieved the target sample size.
|
| | Hofmann, Vohs, & Baumeister | What people desire, feel conflicted about, and try to resist in everyday life |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Other measures were assessed but not reported because they were unrelated to the research question (Large experience sampling project addressing multiple research questions not all of which were addressed in the above publication as explicitly stated.)
- Sample Size: The goal was to collect as many participants as possible with the available project funds.
|
| | Hulme, Bowyer-Crane, Carroll et al. | The causal role of phoneme awareness and letter-sound knowledge in learning to read: Combining |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Article reports analyses of a subset of measures from an earlier study. Article only reports data relevant to assessing a specific hypothesis that the effects of the reading and phonology training in the previous study were mediated by changes in phoneme awareness and letter-sound knowledge.
- Sample Size: The data come from a previously conducted randomized controlled trial wherein sample size was based on a power calculation and we recruited samples that were as large as possible within the time and resources available.
|
| | Keysar, Hayakawa, & An | The foreign-language effect: Thinking in a foreign tongue reduces decision biases |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: For experiments conducted on campus: We targeted around 30 subjects per condition in advance. For experiments abroad and out of state, we instructed RAs to recruit as many subjects as they could within their limited time-frame.
|
| May 2012 | Bernard, Gervais, Allen et al. | Integrating sexual objectification with object versus person recognition: The sexualized-body-inversion hypothesis. |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: We also measured potential moderators: ambivalent sexism (ASI) internalization of beauty standards (SATAQ) and self-objectification. We did not find any significant correlations and we decided to report all tested experimental conditions (i.e. recognition of inverted males upright males inverted females and upright females) without mentioning these additional measures (i.e. moderators). Participants were also asked to complete two other tasks unrelated to the body inversion paradigm we used.
- Sample Size: Before data collection we decided to test approximately 80 participants, based on past studies we have done using this task. We tested during several testing sessions and we stopped data collection after the last testing session (when we had > 80 participants)
|
| | John, Loewenstein, & Prelec | Measuring the prevalence of questionable research practices with incentives for truth telling |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: Full Disclosure (As stated in the supplement, “We stopped collecting data approximately ten days after the final follow-up email was sent. By this point, the rate of incoming responses had dropped off substantially (from over 100 per day in the days immediately following the first solicitation email, to on average fewer than one respondent per day).” That is, the decision to stop was independent from results of data analysis.
|
| | Sweeny & Vohs | On near misses and completed tasks: The nature of relief. |
| | - Exclusions: We excluded one participant in Study 2 because the RA failed to record the experimental condition for that session.
- Conditions: Full Disclosure
- Measures: Other measures were assessed but were not reported because they were not related to the research question.
- Sample Size: We aimed for approximately 100 participants for each study and this was followed. In Study 1 we went slightly over (n = 114) before noting the sample size and cutting off recruitment and in Study 2 we didn't quite reach 100 (n = 79) by the end of the data collection period at the end of term.
|
| | Terburg, Aarts, & van Honk | Testosterone affects gaze aversion from angry faces outside of conscious awareness. |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Measure of digit ratio was included in the research design as a possible mediator in the effects of testosterone. This was however not the case which was not reported for reasons of space/word-limits. These additional data are however reported and discussed in: Terburg D. & van Honk (in press). Approach-avoidance versus dominance-submissiveness: A multilevel framework on how testosterone promotes social status Emotion Review
- Sample Size: Sample size was predetermined based on earlier testosterone administration studies; We collected data until the sample-size as written in the protocol (N=20) was reached.
|
| | Vess | Warm thoughts: Attachment anxiety and sensitivity to temperature cues |
| | - Exclusions: Full Disclosure (Criterion used for 2 participants excluded in Study 2 was studentized-deleted residual values greater than |3.0|. This criterion was provided in the original submission, but was excluded in the final version due to strict word limits)
- Conditions: Full Disclosure
- Measures: I included 2 items regarding participants experience with the sentence unscrambling task. These items assessed task difficulty and task enjoyment. Description of these items were included in a supplement to the original submission but were excluded in the final version due to space restrictions and their null impact on the primary results.
- Sample Size: A minimum sample size was targeted and each study was opened on-line for a set amount of time. Because the minimum sample size was met in both studies after this set time, each study was stopped at that point.
|
| Apr 2012 | Chandler & Pronin | Fast thought speed induces risk taking |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: In Study 2 we asked participants items from the CARE (a measure of risk taking) that pertained to sex substance abuse and disorderly conduct. However due to researcher (my) error participants also completed a single item from the sports subscale of the CARE (the extent to which participants enjoy skiing) at the end of the questionnaire. We did not report this item because (aside from the fact that we did not intend to include it) it seemed inappropriate to make claims about behaviors measured by this subscale when it was represented by only a single item. All descriptive results reported in the paper are virtually identical and all statistical tests of significance are unchanged if this item is included.
- Sample Size: We had a sense that the effect would likely be large based on earlier research using similar manipulations so we were not too concerned about obtaining a large sample. This was really one of those situations where sample size was determined by resource limitations rather than a solid methodological rationale - I had to be present in the eating hall with the RA while data were collected and so we could only collect at times both of us were free. The rule was collect until the end of the semester and see how things looked then. In both cases we collected a single semester's worth of data checked and terminated.
|
| | O'Brien & Ellsworth | More than skin deep: Visceral states are not projected onto dissimilar others |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: We included a question about perspective taking (To what extent did you step inside Jim's shoes while reading the story? from 0-9); there were no differences on this measure & it didn't affect the results, and a number of people left it blank or put question marks next to it. We also included 5 yes/no questions taken from Van Boven & Loewenstein 2003: Have you ever been lost in the woods? Engaged in backpacking? Engaged in mountaineering? Engaged in hiking? Engaged in wilderness activities? Almost everyone circled No, and many people left them blank. They were printed on the back page of the last sheet of the study packet, so I think some people forgot to flip it over. Hence, due to methodological difficulties and to fit word limits, we dropped these measures.
- Sample Size: The general rule of thumb is that we always try to get at least 20-30 people per cell before looking at any of the data, and if more data are needed to run additional blocks of 20-30 before looking. There's also some practical constraints. For example, I think we stopped data collection in Study 2 because we ran out of subject pool hours (the 20-30 rule had also been met).
|
| Mar 2012 | Forest & Wood | When social networking is not working: Individuals with low self-esteem recognize but do not reap the benefits of self-disclosure on Facebook |
| | - Exclusions: Full Disclosure (Criteria for exclusion were reported in the paper. In general, data from participants who complete a survey multiple times or double-submit pages of the survey are also discarded, or the first set of responses is retained but subsequent responses are discarded). Without going back to the raw user-input data, I cannot be certain which of these strategies was employed, or whether there were any such participants who completed the survey multiple times. However, discarding data from participants who submit multiple times is always done before any analyses are conducted.)
- Conditions: Full Disclosure
- Measures: Other measures included for example, measures of the Big 5 Personality traits and narcissism, questions about participants' Facebook settings--that were collected for purposes unrelated to the main research questions addressed in the paper and were therefore not reported in the paper.
- Sample Size: These data were collected several years ago and I do not recall the specific reasons for sample size decisions in these particular studies. In general, we terminated data collection at the end of an academic term or when a given study had reached its maximum credit allocation from the research participant pool, unless a sample size we deemed sufficient was reached prior to these cutoffs.
|
| | Gupta, Jang, Mednick et al. | The road not taken: Creative solutions require avoidance of high-frequency responses |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: We collected the data during one term and then spent the next year applying mathematical models to these data. There was no modification of sample size.
|
| | Jackson, Thoemmes, Jonkmann et al. | Military training and personality trait development: Does the military make the man, or does the man make the military |
| | - Exclusions: Full Disclosure
- Conditions: N/A (non-experimental study)
- Measures: There were scores of unreported measures and items given the study was part of a large, multi-wave longitudinal study.
- Sample Size: Sample size determined by power analysis for initial grant that took into account the number of schools that we would need to sample (assuming a particular response rate).
|
| | McCaffrey | Innovation relies on the obscure: A key to overcoming the classic problem of functional fixedness |
| | - Exclusions: Full Disclosure
- Conditions: Another condition was testing a secondary hypothesis, which did not reach significance. We reported on the primary hypothesis but not the secondary hypothesis.
- Measures: Full Disclosure
- Sample Size: Full Disclosure
|
| Feb 2012 | Frankenstein, Mohler, Bülthoff et al. | Is the map in our head oriented north |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: We decided ahead of time to collect a sample size of 30 participants (15 male 15 female). Due to time constrains of the project and the lab space used by a lot of research groups (i.e. we had only a limited amount of time to collect data in that lab facility) we were not able to run 30 participants and had a few less. All data collection was finished before starting any analyses no additional participants were run after analyses started.
|
| | Hodson & Busseri | Bright minds and dark attitudes: Lower cognitive ability predicts greater prejudice through right-wing ideology and low intergroup contact. |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure (We conducted secondary analyses of large-scale datasets with our analyses focusing on the key variables reported by the original authors.)
- Sample Size: Full Disclosure (We used the participant samples as used by the original authors in our secondary analyses.)
|
| | Howell & Shepperd | Reducing information avoidance through affirmation |
| | - Exclusions: Full Disclosure
- Conditions: When we conducted Study 1 we included a manipulation intended to have the opposite effect of affirmation. However manipulation check measures suggested that our manipulation failed to produce the intended psychological effect. Thus we dropped it from the remaining studies and do not report it in the paper.
- Measures: We included some measures that were not related to the research question. We also included a variety of process variables which either were not reliable or did not predict any variance in our outcomes. We chose to stick to our primary effects for publication because of space constraints and to streamline our story.
- Sample Size: We determined that we were going to collect 20-25 participants per condition in advance of the study based on standard power recommendations for the analyses we intended. We ran our research in our university's human-subjects participant pool and uploaded blocks of participation slots each week. For the first study we stopped when the semester ended. Our second two studies included more than 25 participants per cell because of an unanticipated influx of signups at the end of the studies.
|
| | O'Brien & Ellsworth | Saving the last for best: A positivity bias for end experiences |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: We also measured current mood, current hunger level, and general enjoyment of chocolate (on 0-10 scales); there were no differences between groups, and none of these variables influenced the results; I think I dropped them to fit word limits.
- Sample Size: We stopped data collection simply because I had to travel to Poland for a summer research exchange program, and I wanted to finish data collection before I left. I collected data up until the last possible day before the trip.
|
| | Wang, Li, Fang et al. | Individual differences in holistic processing predict face recognition ability |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: At the first step we planned to collect 500 subjects. (For genetic studies usually you need three independent samples and the first sample is for exploratory investigation and the second and the third samples for validation. That is the sample size is determined by the genetic study). However we were only able to collect data from 337 subjects in this step. We happened to learn that our data were also capable of addressing the relation between face recognition and holistic face processing so we used this set of data. We used the data designated for another study (i.e. the genetic basis for face recognition). We used the full data set collected.
|
| Jan 2012 | Duguid & Goncalo | Living large: The powerful overestimate their own height |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: Full Disclosure
|
| | Hehman, Gaertner, Dovidio et al. | Group status drives majority and minority integration preferences. |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Article was based on data from a larger multi-year project hence multiple measures were collected many unrelated to the research question addressed in the manuscript and so were not included. Additionally since we created the scale used for this research several items originally intended to measure our construct of interest were cut from analysis based on a confirmatory factor analysis (though the use of a CFA is reported in the manuscript).
- Sample Size: Our original goal was for 150 participants of each type of student (Black or White) at two universities or 600 participants. We quickly realized collecting data from 150 White participants at one university (a historically Black college) was unrealistic. For these groups we collected as many as possible and stopped collecting at the end of the semester. For the groups for which we were able to meet our target goal we stopped when hitting that goal (~150 participants).
|
| | Sandman, Davis, & Glynn | Prescient human fetuses thrive |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure (We didn't have experimental conditions but naturalistic observations)
- Measures: We also assessed hundreds of other measures in this longitudinal project--everything from fetal growth (using ultrasound) to childhood MRI; We did not report these other measures because it was not be feasible.
- Sample Size: Longitudinal Study; Sample included all subjects for whom complete data were available for the variables of interest. For the NIH grants that supported the studies detailed descriptions of power were included.
|
| | Sternberg & McClelland | Two mechanisms of human contingency learning |
| | - Exclusions: Full Disclosure
- Conditions: For comparison with the causal framing instructions, we required a comparison framing condition that led to comparable learning of the direct contingencies in the training phase of the experiment across the causal framing and comparison framing condition. Two conditions that did not meet this requirement were tried before the object framing condition. Details are reported in the first author's dissertation (Sternberg, 2012).
- Measures: Participants were also asked to give subjective ratings about the probabilities of the outcome for each item at the very end of the experiment, immediately before debriefing. These are not reported in the paper, as we found early on that while they demonstrated participants had declarative knowledge of the direct contingencies, they were in general not reliably sensitive to the observed indirect effects in either task.
- Sample Size: As cue competition/indirect effects in fast-paced response time tasks have not to our knowledge been previously observed in the contingency learning literature, we could not perform a direct power analysis based on previous findings. However, we decided on specific target sample sizes (48 per condition in the RT task and 24 per condition, in the prediction task, respectively) from the outset, and stuck with them.
|
| | Szpunar, Addis, & Schacter | Memory for emotional simulations: Remembering a rosy future. |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: Sample size was determined on basis of results of previous studies run in the lab. We aspired to reach 24 participants in each delay condition half using the memory sampling technique and half using the list sampling technique. This goal was not modified in the course of the experiment.
|
| | Waytz & Young | The Group-Member Mind Trade-Off: Attributing Mind to Groups Versus Group Members |
| | - Exclusions: Full Disclosure
- Conditions: Full Disclosure
- Measures: Full Disclosure
- Sample Size: Targeted sample sizes were based on previous similar studies, taking into account the particular design of the study at hand, and the total number of cells. We stopped data collection when we reached our pre-determined targets. The first two studies relied on item-wise analyses, so we aimed for approximately 20 subjects accounting for data loss/gain that results from typical discrepancies between MTurk's reported number of hits accepted and the actual numbers of participants that we identified as completing the study upon data inspection. Studies 3 and 4 relied on subject-wise analyses and either a mixed design (Study 3) or a within-subjects design (Study 4). The main analysis of Study 3 involved a 3x2 ANOVA (6 cells) as well as a between-subjects factor, included in an initial analysis, reported in the paper. The primary analysis of Study 4 was a 2x2 ANOVA (4 cells). We aimed for approximately 60 subjects and 30 subjects for Studies 3 and 4 respectively, again accounting for data loss/gain.
|