A5070157740- Mobile Crowdsensing and Crowdsourcing
- Mental Health Research Topics
- Social and Intergroup Psychology
- Psychology of Moral and Emotional Judgment
- Decision-Making and Behavioral Economics
- Behavioral Health and Interventions
- Meta-analysis and systematic reviews
- Digital Marketing and Social Media
- scientometrics and bibliometrics research
- Human Mobility and Location-Based Analysis
- Open Source Software Innovations
- Digital Mental Health Interventions
- Innovative Human-Technology Interaction
- Social Media and Politics
- Digital Games and Media
- Data-Driven Disease Surveillance
- Media Influence and Health
- Action Observation and Synchronization
- Advanced Causal Inference Techniques
- Advanced Text Analysis Techniques
- Social Media in Health Education
- Biomedical and Engineering Education
- Language, Metaphor, and Cognition
- Cultural Differences and Values
- Psychology of Social Influence
Mathematica Policy Research
2015-2023
Princeton University
2010-2022
University of Michigan
2010-2022
Ann Arbor Center for Independent Living
2015-2020
John Wiley & Sons (United States)
2020
Hudson Institute
2020
Erasmus University Rotterdam
2018
Queen's University
2018
Stockholm School of Economics
2016
Michigan United
2013-2016
Reproducibility is a defining feature of science, but the extent to which it characterizes current research unknown. We conducted replications 100 experimental and correlational studies published in three psychology journals using high-powered designs original materials when available. Replication effects were half magnitude effects, representing substantial decline. Ninety-seven percent had statistically significant results. Thirty-six results; 47% effect sizes 95% confidence interval...
Abstract Although Mechanical Turk has recently become popular among social scientists as a source of experimental data, doubts may linger about the quality data provided by subjects recruited from online labor markets. We address these potential concerns presenting new demographic subject population, reviewing strengths relative to other and offline methods recruiting subjects, comparing magnitude effects obtained using traditional pools. further discuss some additional benefits such...
Mechanical Turk (MTurk), an online labor market created by Amazon, has recently become popular among social scientists as a source of survey and experimental data. The workers who populate this have been assessed on dimensions that are universally relevant to understanding whether, why, when they should be recruited research participants. We discuss the characteristics MTurk participant pool for psychology other sciences, highlighting traits samples, why people participants, how data quality...
Although replication is a central tenet of science, direct replications are rare in psychology. This research tested variation the replicability 13 classic and contemporary effects across 36 independent samples totaling 6,344 participants. In aggregate, 10 replicated consistently. One effect – imagined contact reducing prejudice showed weak support for replicability. And two flag priming influencing conservatism currency system justification did not replicate. We compared whether conditions...
Although participants with psychiatric symptoms, specific risk factors, or rare demographic characteristics can be difficult to identify and recruit for participation in research, these are crucial research the social, behavioral, clinical sciences. Online general crowdsourcing software particular may offer a solution. However, no date has examined utility of conducting on psychopathology. In current study, we prevalence several disorders related problems, as well reliability validity...
We conducted preregistered replications of 28 classic and contemporary published findings, with protocols that were peer reviewed in advance, to examine variation effect magnitudes across samples settings. Each protocol was administered approximately half 125 comprised 15,305 participants from 36 countries territories. Using the conventional criterion statistical significance ( p < .05), we found 15 (54%) provided evidence a statistically significant same direction as original finding....
Amazon Mechanical Turk (MTurk) is widely used by behavioral scientists to recruit research participants. MTurk offers advantages over traditional student subject pools, but it also has important limitations. In particular, the population small and potentially overused, some groups of interest are underrepresented difficult recruit. Here we examined whether online panels can avoid these Specifically, compared sample composition, data quality (measured effect sizes, internal reliability,...
The Internet has enabled recruitment of large samples with specific characteristics. However, when researchers rely on participant self-report to determine eligibility, data quality depends honesty. Across four studies Amazon Mechanical Turk, we show that a substantial number participants misrepresent theoretically relevant characteristics (e.g., demographics, product ownership) meet eligibility criteria explicit in the studies, inferred by previous exclusion from study or experiences...
Abstract Using capture-recapture analysis we estimate the effective size of active Amazon Mechanical Turk (MTurk) population that a typical laboratory can access to be about 7,300 workers. We also time taken for half workers leave MTurk pool and replaced is 7 months. Each has its own which overlaps, often extensively, with hundreds other laboratories using MTurk. Our based on sample 114,460 completed sessions from 33,408 unique participants 689 across seven in US, Europe, Australia January...
Although replication is a central tenet of science, direct replications are rare in psychology. This research tested variation the replicability thirteen classic and contemporary effects across 36 independent samples totaling 6,344 participants. In aggregate, ten replicated consistently. One effect – imagined contact reducing prejudice showed weak support for replicability. And two flag priming influencing conservatism currency system justification did not replicate. We compared whether...
The online labor market Amazon Mechanical Turk (MTurk) is an increasingly popular source of respondents for social science research. A growing body research has examined the demographic composition MTurk workers as compared with that other populations. While these comparisons have revealed ways in which are and not representative general population, variations among samples drawn from received less attention. This article focuses on whether sample varies a function time. Specifically, we...
This crowdsourced project introduces a collaborative approach to improving the reproducibility of scientific research, in which findings are replicated qualified independent laboratories before (rather than after) they published. Our goal is establish non-adversarial replication process with highly informative final results. To illustrate Pre-Publication Independent Replication (PPIR) approach, 25 research groups conducted replications all ten moral judgment effects last author and his...
In this chapter, we outline the common concerns with MTurk as a participant pool, review evidence for those concerns, and discuss solutions. We close Table of considerations that researchers should make when fielding study on
Clinical psychological research studies often require individuals with specific characteristics. The Internet can be used to recruit broadly, enabling the recruitment of rare groups such as people disorders. However, Internet-based relies on participant self-report determine eligibility, and thus, data quality depends honesty. For those groups, even low levels dishonesty lead a substantial proportion fraudulent survey responses, all will include careless respondents who do not pay attention...