Comparing 2 Approaches for the File Review of Residency Applications

• DOI: 10.4300/jgme-d-20-00619.1
• Publication Date: 2021-02-17T21:03:03Z
• Authors: Nada Gawad; Julia Younan; Chelsea Towaij; Isabelle Raiche
• Keywords: Inter-Rater Reliability; Trait

The residency selection process relies on subjective information in applications, as well assessment of applications by reviewers. This inherent subjectivity makes prone to poor reliability between those reviewing files.We compared the interrater 2 tools during file review: one rating applicant traits (ie, leadership, communication) and other using a global application elements curriculum vitae, reference letters).Ten reviewers were randomized into groups, each scored 7 general surgery from 2019-2020 cycle. first group used an element-based (EB) scoring tool, while second trait-based (TB) tool. Feedback was collected, discrimination capacities measured variation scores, (IRR) calculated intraclass correlation (ICC) 2-way random effects model.Both identified same top-ranked bottom-ranked applicants; however, discrepancies noted for middle-ranked applicants. score range 5 applicants greater with TB tool (6.43 vs 3.80), which also demonstrated fewer tie scores. IRR superior EB (ICC [2, 5] = 0.82 0.55). required only raters achieve ICC ≥ 0.70.Using review strategy can facilitate improved EB, spread candidate potentially offers programs feasible way optimize reflect their institution's core values process.