SemEval-2015 Task 4: TimeLine: Cross-Document Event Ordering

• DOI: 10.18653/v1/s15-2132
• Publication Date: 2015-12-15T14:22:40Z
• Authors: Anne-Lyse Minard; Manuela Speranza; Eneko Agirre; Itziar Aldabe; Marieke van Erp; Bernardo Magnini; German Rigau; Ruben Urizar
• Keywords: 01 natural sciences; Digital Humanities and Cultural Heritage; 0105 earth and related environmental sciences

This paper describes the outcomes of the TimeLine task (Cross-Document Event Ordering), that was organised within the Time and Space track of SemEval-2015. Given a set of documents and a set of target entities, the task consisted of building a timeline for each entity, by detecting, anchoring in time and ordering the events involving that entity. The TimeLine task goes a step further than previous evaluation challenges by requiring participant systems to perform both event coreference and temporal relation extraction across documents. Four teams submitted the output of their systems to the four proposed subtracks for a total of 13 runs, the best of which obtained an F1-score of 7.85 in the main track (timeline creation from raw text).