Borderline grades in high stakes clinical examinations: resolving examiner uncertainty

• DOI: 10.1186/s12909-018-1382-0
• Publication Date: 2018-11-20T08:29:39Z
• Authors: Boaz Shulruf; Barbara-Ann Adels...; Arvin Damodaran; Peter Harris; Sean Kennedy; Anthony O’Sullivan; Silas Taylor
• Keywords: Confusion; Raw score

Objective Structured Clinical Exams are used to increase reliability and validity, yet they only achieve a modest level of reliability. This low is due in part examiner variance which greater than the students. often represents indecisiveness at cut score with apparent confusion over terms such as "borderline pass". It amplified by well reported failure fail. A borderline grade (meaning performance neither clear pass nor fail) was introduced high stakes undergraduate medical clinical skills exam replace (which historically resolved 50%) 4 point scale (distinction, pass, borderline, fail). Each Borderline then into Pass or Fail formula referencing difficulty station same domain student other stations. Raw fail grades were unaltered. Mean scores 95%CI calculated per for unmodified modified scores/grades (results presented on error bars). To estimate defensibility these modifications, similar analysis took place P F resulted from modification B grades. Of 14,634 observations 4.69% Borderline. Application did not impact mean each but rate increased 0.7 4.1%. Examiners students expressed satisfaction grade, resolution outcomes. (by stations domains respectively) whose significantly higher their counterparts F. study provides feasible defensible situations where examinee's fail, demonstrating application exam. does create new standard utilises real data make judgements about small number candidates. perceived fair approach Pass/Fail decisions.