A Meta‐Analysis of Reliability Coefficients in Second Language Research

• DOI: 10.1111/modl.12335
• Publication Date: 2016-04-30T09:21:03Z
• Authors: LUKE PLONSKY; DEIRDRE J. DERRICK
• Keywords: Inter-Rater Reliability; Intra-rater reliability; Sample (material)

Ensuring internal validity in quantitative research requires, among other conditions, reliable instrumentation. Unfortunately, however, second language (L2) researchers often fail to report and even more interpret reliability estimates beyond generic benchmarks for acceptability. As a means guide interpretations of such estimates, this article meta‐analyzes coefficients (internal consistency, interrater, intrarater) as reported published L2 research. We recorded 2,244 537 individual articles along with study (e.g., sample size) instrument features item formats) proposed influence reliability. also coded the indices employed alpha, KR20). The were then aggregated (i.e., meta‐analyzed). three types varied, consistency lowest: median = .82. Interrater intrarater substantially higher (.92 .95, respectively). Overall found vary according proficiency (low .79, intermediate .84, advanced .89) target skill writing .88 vs. listening .77). use our results inform encourage relative larger field well substantive methodological particular studies subdomains.