Fighting Fire with Fire: Adversarial Prompting to Generate a Misinformation Detection Dataset

• DOI: 10.48550/arxiv.2401.04481
• Publication Date: 2024-01-01
• Authors: Shrey Satapara; Parth Mehta; Debasis Ganguly; Sandip Modha
• Keywords: Misinformation; Hatred

The recent success in language generation capabilities of large models (LLMs), such as GPT, Bard, Llama etc., can potentially lead to concerns about their possible misuse inducing mass agitation and communal hatred via generating fake news spreading misinformation. Traditional means developing a misinformation ground-truth dataset does not scale well because the extensive manual effort required annotate data. In this paper, we propose an LLM-based approach creating silver-standard datasets for identifying Specifically speaking, given trusted article, our proposed involves prompting LLMs automatically generate summarised version original article. prompts act controlling mechanism specific types factual incorrectness generated summaries, e.g., incorrect quantities, false attributions etc. To investigate usefulness dataset, conduct set experiments where train range supervised task detection.