Rater bias in psychological research: When is it a problem and what can we do about it?

• DOI: 10.1037/1082-989x.5.1.64
• Publication Date: 2005-09-23T15:46:35Z
• Authors: William T. Hoyt
• Keywords: Observer Variation; Models, Statistical; Bias; Psychometrics; 05 social sciences; Humans; 0501 psychology and cognitive sciences

Rater bias is a substantial source of error in psychological research. Bias distorts observed effect sizes beyond the expected level attenuation due to intrarater error, and impact not accurately estimated using conventional methods correction for attenuation. Using model based on multivariate generalizability theory, this article illustrates how affects research results. The identifies 4 types that may affect findings observer ratings, including biases traditionally termed leniency halo errors. depends which classes rating design used, formulas are derived correcting (due variance) inflation covariance) each these classes. rater suggests procedures researchers seeking minimize adverse study findings.