Feature-based detection of automated language models: tackling GPT-2, GPT-3 and Grover

• DOI: 10.7717/peerj-cs.443
• Publication Date: 2021-04-06T07:54:11Z
• Authors: Leon Fröhling; Arkaitz Zubiaga
• Keywords: Feature (linguistics)

The recent improvements of language models have drawn much attention to potential cases use and abuse automatically generated text. Great effort is put into the development methods detect machine generations among human-written text in order avoid scenarios which large-scale generation with minimal cost undermines trust human interaction factual information online. While most current approaches rely on availability expensive models, we propose a simple feature-based classifier for detection problem, using carefully crafted features that attempt model intrinsic differences between Our research contributes field producing method achieves performance competitive far more methods, offering an accessible “first line-of-defense” against models. Furthermore, our experiments show different sampling lead types flaws