Using artificial intelligence to assess personal qualities in college admissions

• DOI: 10.1126/sciadv.adg9405
• Publication Date: 2023-10-12T17:57:22Z
• Authors: Benjamin Lira; Margo Gardner; Abigail Quirk; Cathlyn Stone; Arjun Rao; Lyle Ungar; Stephen Hutt; Louis Hickman; Sidney K. D’Mello; Angela L. Duckworth
• Keywords: Graduation (instrument); Disadvantaged; Sample (material); Prosocial Behavior; Personal life; Personal computer

Personal qualities like prosocial purpose and leadership predict important life outcomes, including college success. Unfortunately, the holistic assessment of personal in admissions is opaque resource intensive. Can artificial intelligence (AI) advance goals admissions? While cost-effective, AI has been criticized as a "black box" that may inadvertently penalize already disadvantaged subgroups when used high-stakes settings. Here, we consider an approach to assessing aims overcome these limitations. Research assistants officers first identified presence/absence seven n = 3131 applicant essays describing extracurricular work experiences. Next, fine-tuned pretrained language models with ratings, which successfully reproduced human codes across demographic subgroups. Last, national sample (N 309,594), computer-generated scores collectively demonstrated incremental validity for predicting 6-year graduation. We discuss challenges opportunities qualities.