Ordering-Sensitive and Semantic-Aware Topic Modeling

• DOI: 10.1609/aaai.v29i1.9501
• Publication Date: 2022-06-23T19:04:01Z
• Authors: Min Yang; Tianyi Cui; Wenting Tu
• Keywords: Perplexity; Word embedding; Latent semantic analysis

Topic modeling of textual corpora is an important and challenging problem. In most previous work, the “bag-of-words” assumption usually made which ignores ordering words. This simplifies computation, but it unrealistically loses information semantic words in context. this paper, we present a Gaussian Mixture Neural Model (GMNTM) incorporates both meaning sentences into topic modeling. Specifically, represent each as cluster multi-dimensional vectors embed corpus collection generated by mixture model. Each word affected not only its topic, also embedding vector surrounding The components documents, can be learnt jointly. Extensive experiments show that our model learn better topics more accurate distributions for topic. Quantitatively, comparing to state-of-the-art approaches, GMNTM obtains significantly performance terms perplexity, retrieval accuracy classification accuracy.