Ex Machina: Personal Attacks Seen at Scale

• DOI: 10.48550/arxiv.1610.08914
• Publication Date: 2016-01-01
• Authors: Ellery Wulczyn; Nithum Thain; Lucas Dixon
• Keywords: Crowdsourcing

The damage personal attacks cause to online discourse motivates many platforms try curb the phenomenon. However, understanding prevalence and impact of in at scale remains surprisingly difficult. contribution this paper is develop illustrate a method that combines crowdsourcing machine learning analyze scale. We show an evaluation for classifier terms aggregated number crowd-workers it can approximate. apply our methodology English Wikipedia, generating corpus over 100k high quality human-labeled comments 63M machine-labeled ones from as good aggregate 3 crowd-workers, measured by area under ROC curve Spearman correlation. Using scores, allows us explore some open questions about nature attacks. This reveals majority on Wikipedia are not result few malicious users, nor primarily consequence allowing anonymous contributions unregistered users.