A5051652099- Natural Language Processing Techniques
- Topic Modeling
- Speech Recognition and Synthesis
- Text Readability and Simplification
- Speech and dialogue systems
- Text and Document Classification Technologies
- Neural Networks and Applications
- Handwritten Text Recognition Techniques
- Phonetics and Phonology Research
- Model Reduction and Neural Networks
- Linguistic Variation and Morphology
- Linguistics and Discourse Analysis
- Speech and Audio Processing
- Music and Audio Processing
- Multimodal Machine Learning Applications
- Data Mining Algorithms and Applications
- Anomaly Detection Techniques and Applications
- French Language Learning Methods
- Semantic Web and Ontologies
- Biomedical Text Mining and Ontologies
- Algorithms and Data Compression
- Adversarial Robustness in Machine Learning
- Meteorological Phenomena and Simulations
- Data Quality and Management
- Translation Studies and Practices
Lamsade
2021-2025
ESPCI Paris
2020-2025
Université Paris Dauphine-PSL
2020-2024
Université Paris-Saclay
2017-2023
Laboratoire d'Informatique pour la Mécanique et les Sciences de l'Ingénieur
2011-2023
Centre National de la Recherche Scientifique
2011-2023
Université Paris Sciences et Lettres
2021-2022
Laboratoire Interdisciplinaire des Sciences du Numérique
2021
Institut national de recherche en informatique et en automatique
2021
Laboratoire d'Informatique de Grenoble
2020
This paper introduces a new neural network language model (NNLM) based on word clustering to structure the output vocabulary: Structured Output Layer NNLM. is able handle vocabularies of arbitrary size, hence dispensing with design short-lists that are commonly used in NNLMs. Several softmax layers replace standard layer this model. The depends which uses continuous representation induced by GALE Mandarin data was carry out speech-to-text experiments and evaluate On well tuned baseline...
Language models have become a key step to achieve state-of-the art results in many different Natural Processing (NLP) tasks. Leveraging the huge amount of unlabeled texts nowadays available, they provide an efficient way pre-train continuous word representations that can be fine-tuned for downstream task, along with their contextualization at sentence level. This has been widely demonstrated English using contextualized (Dai and Le, 2015; Peters et al., 2018; Howard Ruder, Radford Devlin...
Self-Supervised Learning (SSL) using huge unlabeled data has been successfully explored for image and natural language processing. Recent works also investigated SSL from speech. They were notably successful to improve performance on downstream tasks such as automatic speech recognition (ASR). While these suggest it is possible reduce dependence labeled building efficient systems, their evaluation was mostly made ASR multiple heterogeneous experimental settings (most of them English). This...
This paper extends a novel neural network language model (NNLM) which relies on word clustering to structure the output vocabulary: Structured OUtput Layer (SOUL) NNLM. is able handle arbitrarily-sized vocabularies, hence dispensing with need for shortlists that are commonly used in NNLMs. Several softmax layers replace standard layer this model. The depends based continuous representation determined by Mandarin and Arabic data evaluate SOUL NNLM accuracy via speech-to-text experiments. Well...
In this paper we explore a POS tagging application of neural architectures that can infer word representations from the raw character stream.It relies on two modelling stages are jointly learnt: convolutional network infers representation directly stream, followed by prediction stage.Models evaluated and morphological task for German.Experimental results show meaningful representations, while stage, well designed structured strategy allows model to outperform stateof-the-art results, without...
Deep Reinforcement Learning (DRL) is applied to control a nonlinear, chaotic system governed by the one-dimensional Kuramoto-Sivashinsky (KS) equation. DRL uses reinforcement learning principles for determination of optimal solutions and deep Neural Networks approximating value function policy. Recent applications have shown that may achieve superhuman performance in complex cognitive tasks. In this work, we show using restricted, localized actuations, partial knowledge state based on...
Despite being the cornerstone of deep learning, backpropagation is criticized for its inherent sequentiality, which can limit scalability very models. Such models faced convergence issues due to vanishing gradient, later resolved using residual connections. Variants these are now widely used in modern architecture. However, computational cost remains a major burden, accounting most training time. Taking advantage residual-like architectural designs, we introduce Highway backpropagation,...
Conditional expectation \mathbb{E}(Y \mid X) often fails to capture the complexity of multimodal conditional distributions \mathcal{L}(Y X). To address this, we propose using n-point quantizations--functional mappings X that are learnable via gradient descent--to approximate This approach adapts Competitive Learning Vector Quantization (CLVQ), tailored for distributions. It goes beyond single-valued predictions by providing multiple representative points better reflect structures. enables...
Large Language Models (LLMs), when used for conditional text generation, often produce hallucinations, i.e., information that is unfaithful or not grounded in the input context. This issue arises typical generation tasks, such as summarization and data-to-text where goal to fluent based on contextual input. When fine-tuned specific domains, LLMs struggle provide faithful answers a given context, adding generating errors. One underlying cause of this rely statistical patterns learned from...
Given the high flexional properties of French language, transcribing broadcast news (BN) is more challenging than English BN. This in part due to largenumber homophones inflected forms. paper describes advances automatic processing speech based on recent improvements LIMSI system. The main differences between and BN systems are: a 200k vocabulary overcome lower lexical coverage (including contextual pronunciations model liaisons), case sensitive language model, use POS impact homophonic...
This paper presents continuation of research on Structured OUtput Layer Neural Network language models (SOUL NNLM) for automatic speech recognition. As SOUL NNLMs allow estimating probabilities all in-vocabulary words and not only those pertaining to a limited shortlist, we investigate its performance large-vocabulary task. Significant improvements both in perplexity word error rate over conventional shortlist-based are shown challenging Arabic GALE task characterized by recognition...
The paper reports on an investigation of open vocabulary recognizer that allows new words to be introduced in the recognition vocabulary, without need retrain or adapt language model. This method uses special word classes, whose n-gram probabilities are estimated during training process by discounting a mass probability from out words. A part-of-speech tagger is used determine classes model and for adaptation. Metadata information provided French audiovisual archive institute identify...
This paper describes our statistical machine translation systems based on the Moses toolkit for WMT08 shared task. We address Europarl and News conditions following language pairs: English with French, German Spanish. For Europarl, n-best rescoring is performed using an enhanced n-gram or a neuronal model; condition, models incorporate extra training data. also report unconvincing results of experiments factored models.