A5032668442- Natural Language Processing Techniques
- Topic Modeling
- Software Engineering Research
- Speech and dialogue systems
- Economic and Technological Systems Analysis
- Digital Transformation in Industry
- Advanced Text Analysis Techniques
- Software-Defined Networks and 5G
- Adversarial Robustness in Machine Learning
- Explainable Artificial Intelligence (XAI)
- Semantic Web and Ontologies
- Agricultural economics and policies
- Agriculture, Plant Science, Crop Management
- Impact of AI and Big Data on Business and Society
- Advanced Malware Detection Techniques
- Network Security and Intrusion Detection
- Data Mining Algorithms and Applications
- Video Analysis and Summarization
- Multi-Agent Systems and Negotiation
- Smart Cities and Technologies
- Multimodal Machine Learning Applications
University of Aberdeen
2018-2024
Dublin City University
2024
Edinburgh Napier University
2021-2024
Most Natural Language Generation systems need to produce accurate texts. We propose a methodology for high-quality human evaluation of the accuracy generated texts, which is intended serve as gold-standard evaluations data-to-text systems. use our evaluate computer basketball summaries. then show how gold standard can be used validate automated metrics.
Emiel van Miltenburg, Miruna Clinciu, Ondřej Dušek, Dimitra Gkatzia, Stephanie Inglis, Leo Leppänen, Saad Mahamood, Emma Manning, Schoch, Craig Thomson, Luou Wen. Proceedings of the 14th International Conference on Natural Language Generation. 2021.
Abstract While conducting a coordinated set of repeat runs human evaluation experiments in NLP, we discovered flaws every single experiment selected for inclusion via systematic process. In this squib, describe the types discovered, which include coding errors (e.g., loading wrong system outputs to evaluate), failure follow standard scientific practice ad hoc exclusion participants and responses), mistakes reported numerical results numbers not matching experimental data). If these problems...
Digital Twins (DT) have become crucial to achieve sustainable and effective smart urban solutions. However, current DT modelling techniques cannot support the dynamicity of these city environments. This is caused by lack right-time data capturing in traditional approaches, resulting inaccurate high resource energy consumption challenges. To fill this gap, we explore spatiotemporal graphs propose Reinforcement Learning-based Adaptive Twining (RL-AT) mechanism with Deep Q Networks (DQN). By...
Human evaluation is widely regarded as the litmus test of quality in NLP. A basic requirementof all evaluations, but particular where they are used for meta-evaluation, that should support same conclusions if repeated. However, reproducibility human evaluations virtually never queried, let alone formally tested, NLP which means their repeatability and results currently an open question. This focused contribution reports our review experiments reported papers over past five years we assessed...
The Shared Task on Evaluating Accuracy focused techniques (both manual and automatic) for evaluating the factual accuracy of texts produced by neural NLG systems, in a sports-reporting domain. Four teams submitted evaluation this task, using very different approaches techniques. best-performing submissions did encouragingly well at difficult task. However, all automatic struggled to detect errors which are semantically or pragmatically complex (for example, based incorrect computation inference).
We propose a shared task on methodologies and algorithms for evaluating the accuracy of generated texts, specifically summaries basketball games produced from box score other game data. welcome submissions based protocols human evaluation, automatic metrics, as well combinations evaluations metrics.
It is unfair to expect neural data-to-text produce high quality output when there are gaps between system input data and information contained in the training text. Thomson et al. (2020) identify narrow Rotowire, a popular dataset. In this paper, we describe study which finds that state-of-the-art produces higher output, according extraction (IE) based metrics, additional carefully selected from newly available source. remains be shown, however, whether IE metrics used correlate well with...
This paper proposes an approach to NLG system design which focuses on generating output text can be more easily processed by the reader. Ways in cognitive theory might combined with existing techniques are discussed and two simple experiments content ordering presented.
Neural data-to-text systems lack the control and factual accuracy required to generate useful insightful summaries of multidimensional data. We propose a solution in form data views, where each view describes an entity its attributes along specific dimensions. A sequence views can then be used as high-level schema for document planning, with neural model handling complexities micro-planning surface realization. show that our view-based system retains while offering output tailored based on...
In today's networked world, Digital Twin Networks (DTNs) are revolutionizing how we understand and optimize physical networks. These networks, also known as 'Digital (DTNs)' or 'Networks Twins (NDTs),' encompass many from cellular wireless to optical satellite. They leverage computational power AI capabilities provide virtual representations, leading highly refined recommendations for real-world network challenges. Within DTNs, tasks include performance enhancement, latency optimization,...
Intrusion detection systems are integral to the security of networked for detecting malicious or anomalous network traffic. As traditional approaches becoming less effective, machine learning and deep learning-based intrusion vital research areas improved systems. Past into computer vision using revealed that classifiers themselves vulnerable adversarial attacks, these attacks have been investigated extensively. However, restricted not only domain image recognition. indicated by previous...
Most Natural Language Generation systems need to produce accurate texts. We propose a methodology for high-quality human evaluation of the accuracy generated texts, which is intended serve as gold-standard evaluations data-to-text systems. use our evaluate computer basketball summaries. then show how gold standard can be used validate automated metrics
We propose a shared task on methodologies and algorithms for evaluating the accuracy of generated texts. Participants will measure basketball game summaries produced by NLG systems from box score data.
The Shared Task on Evaluating Accuracy focused techniques (both manual and automatic) for evaluating the factual accuracy of texts produced by neural NLG systems, in a sports-reporting domain. Four teams submitted evaluation this task, using very different approaches techniques. best-performing submissions did encouragingly well at difficult task. However, all automatic struggled to detect errors which are semantically or pragmatically complex (for example, based incorrect computation inference).