ClarET: Pre-training a Correlation-Aware Context-To-Event Transformer for Event-Centric Generation and Classification

• DOI: 10.18653/v1/2022.acl-long.183
• Publication Date: 2022-06-03T01:34:53Z
• Authors: Yucheng Zhou; Tao Shen; Xiubo Geng; Guodong Long; Daxin Jiang
• Keywords: Complex Event Processing

Generating new events given context with correlated ones plays a crucial role in many event-centric reasoning tasks. Existing works either limit their scope to specific scenarios or overlook event-level correlations. In this paper, we propose pre-train general Correlation-aware context-to-Event Transformer (ClarET) for reasoning. To achieve this, three novel objectives, i.e., whole event recovering, contrastive event-correlation encoding and prompt-based locating, which highlight correlations effective training. The proposed ClarET is applicable wide range of scenarios, considering its versatility (i) types (e.g., causal, temporal, contrast), (ii) application formulations (i.e., generation classification), (iii) abductive, counterfactual ending reasoning). Empirical fine-tuning results, as well zero- few-shot learning, on 9 benchmarks (5 4 classification tasks covering diverse correlations), verify effectiveness generalization ability.