A5004412943- Topic Modeling
- Natural Language Processing Techniques
- Multimodal Machine Learning Applications
- Speech and dialogue systems
- Speech Recognition and Synthesis
- Text Readability and Simplification
- Multi-Agent Systems and Negotiation
- Modular Robots and Swarm Intelligence
- Distributed Control Multi-Agent Systems
- Semantic Web and Ontologies
- Domain Adaptation and Few-Shot Learning
- Robotic Path Planning Algorithms
- AI-based Problem Solving and Planning
- Robotics and Sensor-Based Localization
- Artificial Intelligence in Games
- Robotics and Automated Systems
- Reinforcement Learning in Robotics
- Advanced Text Analysis Techniques
- Robot Manipulation and Learning
- Evacuation and Crowd Dynamics
- Human Pose and Action Recognition
- Machine Learning and Algorithms
- Educational Assessment and Pedagogy
- Biomedical Text Mining and Ontologies
- Artificial Intelligence in Law
IT University of Copenhagen
2023
Tokyo Institute of Technology
2023
Administration for Community Living
2023
American Jewish Committee
2023
Meta (Israel)
2017-2022
University of Southern California
2022
University of Washington
2015-2022
University of California, Irvine
2022
Allen Institute for Artificial Intelligence
2022
Southern California University for Professional Studies
2022
Language model pretraining has led to significant performance gains but careful comparison between different approaches is challenging. Training computationally expensive, often done on private datasets of sizes, and, as we will show, hyperparameter choices have impact the final results. We present a replication study BERT (Devlin et al., 2019) that carefully measures many key hyperparameters and training data size. find was significantly undertrained, can match or exceed every published...
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, Luke Zettlemoyer. Proceedings of the 58th Annual Meeting Association for Computational Linguistics. 2020.
Large pre-trained language models have been shown to store factual knowledge in their parameters, and achieve state-of-the-art results when fine-tuned on downstream NLP tasks. However, ability access precisely manipulate is still limited, hence knowledge-intensive tasks, performance lags behind task-specific architectures. Additionally, providing provenance for decisions updating world remain open research problems. Pre-trained with a differentiable mechanism explicit non-parametric memory...
We explore story generation: creative systems that can build coherent and fluent passages of text about a topic. collect large dataset 300K human-written stories paired with writing prompts from an online forum. Our enables hierarchical generation, where the model first generates premise, then transforms it into passage text. gain further improvements novel form fusion improves relevance to prompt, adding new gated multi-scale self-attention mechanism long-range context. Experiments show...
We introduce the first end-to-end coreference resolution model and show that it significantly outperforms all previous work without using a syntactic parser or hand-engineered mention detector. The key idea is to directly consider spans in document as potential mentions learn distributions over possible antecedents for each. computes span embeddings combine context-dependent boundary representations with head-finding attention mechanism. It trained maximize marginal likelihood of gold...
This paper demonstrates that multilingual denoising pre-training produces significant performance gains across a wide variety of machine translation (MT) tasks. We present mBART—a sequence-to-sequence auto-encoder pre-trained on large-scale monolingual corpora in many languages using the BART objective (Lewis et al., 2019 ). mBART is first method for complete model by full texts multiple languages, whereas previous approaches have focused only encoder, decoder, or reconstructing parts text....
This paper demonstrates that multilingual denoising pre-training produces significant performance gains across a wide variety of machine translation (MT) tasks. We present mBART -- sequence-to-sequence auto-encoder pre-trained on large-scale monolingual corpora in many languages using the BART objective. is one first methods for complete model by full texts multiple languages, while previous approaches have focused only encoder, decoder, or reconstructing parts text. Pre-training allows it...
We introduce a new deep learning model for semantic role labeling (SRL) that significantly improves the state of art, along with detailed analyses to reveal its strengths and limitations. use highway BiLSTM architecture constrained decoding, while observing number recent best practices initialization regularization. Our 8-layer ensemble achieves 83.2 F1 on theCoNLL 2005 test set 83.4 CoNLL 2012, roughly 10% relative error reduction over previous art. Extensive empirical analysis these gains...
This paper presents USARSim, an open source high fidelity robot simulator that can be used both for research and education. USARSim offers many characteristics differentiate it from most existing simulators. Most notably, constitutes the simulation engine to run virtual robots competition within Robocup initiative. We describe its general architecture, examples of utilization, provide a comprehensive overview those interested in simulations education, competitions.
Much of human dialogue occurs in semi-cooperative settings, where agents with different goals attempt to agree on common decisions. Negotiations require complex communication and reasoning skills, but success is easy measure, making this an interesting task for AI. We gather a large dataset human-human negotiations multi-issue bargaining task, who cannot observe each other’s reward functions must reach agreement (or deal) via natural language dialogue. For the first time, we show it possible...
Practical applications of abstractive summarization models are limited by frequent factual inconsistencies with respect to their input. Existing automatic evaluation metrics for largely insensitive such errors. We propose QAGS (pronounced “kags”), an protocol that is designed identify in a generated summary. based on the intuition if we ask questions about summary and its source, will receive similar answers factually consistent source. To evaluate QAGS, collect human judgments consistency...
Sebastian Schuster, Sonal Gupta, Rushin Shah, Mike Lewis. Proceedings of the 2019 Conference North American Chapter Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019.
This paper introduces the task of questionanswer driven semantic role labeling (QA-SRL), where question-answer pairs are used to represent predicate-argument structure.For example, verb "introduce" in previous sentence would be labeled with questions "What is introduced?", and something?", each paired phrase from that gives correct answer.Posing problem this way allows themselves define set possible roles, without need for predefined frame or thematic ontologies.It also scalable data...
We present a new visual reasoning language dataset, containing 92,244 pairs of examples natural statements grounded in synthetic images with 3,962 unique sentences. describe method crowdsourcing linguistically-diverse data, and an analysis our data. The data demonstrates broad set linguistic phenomena, requiring set-theoretic reasoning. experiment various models, show the presents strong challenge for future research.
Writers often rely on plans or sketches to write long stories, but most current language models generate word by from left right. We explore coarse-to-fine for creating narrative texts of several hundred words, and introduce new which decompose stories abstracting over actions entities. The model first generates the predicate-argument structure text, where different mentions same entity are marked with placeholder tokens. It then a surface realization structure, finally replaces placeholders...
Task oriented dialog systems typically first parse user utterances to semantic frames comprised of intents and slots. Previous work on task intent slot-filling has been restricted one per query slot label token, thus cannot model complex compositional requests. Alternative parsing have represented queries as logical forms, but these are challenging annotate parse. We propose a hierarchical annotation scheme for that allows the representation queries, can be efficiently accurately parsed by...
Despite much progress in training artificial intelligence (AI) systems to imitate human language, building agents that use language communicate intentionally with humans interactive environments remains a major challenge. We introduce Cicero, the first AI agent achieve human-level performance Diplomacy, strategy game involving both cooperation and competition emphasizes natural negotiation tactical coordination between seven players. Cicero integrates model planning reinforcement learning...
Sewon Min, Mike Lewis, Luke Zettlemoyer, Hannaneh Hajishirzi. Proceedings of the 2022 Conference North American Chapter Association for Computational Linguistics: Human Language Technologies. 2022.
Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, Hannaneh Hajishirzi. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.
We introduce a new approach to semantics which combines the benefits of distributional and formal logical semantics. Distributional models have been successful in modelling meanings content words, but is necessary adequately represent many function words. follow mapping language representations, differ that relational constants used are induced by offline clustering at level predicate-argument structure. Our algorithm highly scalable, allowing us run on corpora size Gigaword. Different...
We explore story generation: creative systems that can build coherent and fluent passages of text about a topic. collect large dataset 300K human-written stories paired with writing prompts from an online forum. Our enables hierarchical generation, where the model first generates premise, then transforms it into passage text. gain further improvements novel form fusion improves relevance to prompt, adding new gated multi-scale self-attention mechanism long-range context. Experiments show...
We introduce a new CCG parsing model which is factored on lexical category assignments.Parsing then simply deterministic search for the most probable sequence that supports derivation.The parser extremely simple, with tiny feature set, no POS tagger, and statistical of derivation or dependencies.Formulating in this way allows highly effective heuristic A * parsing, makes fast.Compared to standard C&C parser, our more accurate out-of-domain, four times faster, has higher coverage, greatly...