Hypothesis Only Baselines in Natural Language Inference

• DOI: 10.18653/v1/s18-2023
• Publication Date: 2018-05-30T04:40:05Z
• Authors: Adam Poliak; Jason Naradowsky; Aparajita Haldar; Rachel Rudinger; Benjamin Van Durme
• Keywords: FOS: Computer and information sciences; Computer Science - Computation and Language; 0202 electrical engineering, electronic engineering, information engineering; 02 engineering and technology; Computation and Language (cs.CL)

We propose a hypothesis only baseline for diagnosing Natural Language Inference (NLI). Especially when an NLI dataset assumes inference is occurring based purely on the relationship between a context and a hypothesis, it follows that assessing entailment relations while ignoring the provided context is a degenerate solution. Yet, through experiments on ten distinct NLI datasets, we find that this approach, which we refer to as a hypothesis-only model, is able to significantly outperform a majority class baseline across a number of NLI datasets. Our analysis suggests that statistical irregularities may allow a model to perform NLI in some datasets beyond what should be achievable without access to the context.<br/>Accepted at *SEM 2018 as long paper. 12 pages<br/>