Mental health predictive systems typically model language as if from a single context (e.g. Twitter posts, status updates, or forum posts) and often limited to level of analysis either the message-level user-level). Here, we bring these pieces together explore use open-vocabulary (BERT embeddings, topics) theoretical features (emotional expression lexica, personality) for task suicide risk assessment on support forums (the CLPsych-2019 Shared Task). We used dual based approaches (modeling...

In human-level NLP tasks, such as predicting mental health, personality, or demographics, the number of observations is often smaller than standard 768+ hidden state sizes each layer within modern transformer-based language models, limiting ability to effectively leverage transformers. Here, we provide a systematic study on role dimension reduction methods (principal components analysis, factorization techniques, multi-layer auto-encoders) well dimensionality embedding vectors and sample...

Targeting of location-specific aid for the U.S. opioid epidemic is difficult due to our inability accurately predict changes in mortality across heterogeneous communities. AI-based language analyses, having recently shown promise cross-sectional (between-community) well-being assessments, may offer a way more longitudinally community-level overdose mortality. Here, we develop and evaluate, TROP (Transformer Opiod Prediction), model community-specific trend projection that uses social media...

Much of natural language processing is focused on leveraging large capacity models, typically trained over single messages with a task predicting one or more tokens. However, modeling human at higher-levels context (i.e., sequences messages) under-explored. In stance detection and other social media tasks where the goal to predict an attribute message, we have contextual data that loosely semantically connected by authorship. Here, introduce Message-Level Transformer (MeLT) – hierarchical...

Assessing risk for excessive alcohol use is important applications ranging from recruitment into research studies to targeted public health messaging. Social media language provides an ecologically embedded source of information assessing individuals who may be at harmful drinking.Using data collected on 3664 respondents the general population, we examine how accurately used social classifies as at-risk problems based Alcohol Use Disorder Identification Test-Consumption score benchmarks.We...

Natural language is generated by people, yet traditional modeling views words or documents as if independently. Here, we propose human (HuLM), a hierarchical extension to the problem where human- level exists connect sequences of (e.g. social media messages) and capture notion that moderated changing states. We introduce, HaRT, large-scale transformer model for solving HuLM, pre-trained on approximately 100,000 users, demonstrate it’s effectiveness in terms both (perplexity) fine-tuning 4...

Human natural language is mentioned at a specific point in time while human emotions change over time. While much work has established strong link between use and emotional states, few have attempted to model Here, we introduce the task of

Swanie Juhng, Matthew Matero, Vasudha Varadarajan, Johannes Eichstaedt, Adithya V Ganesan, H. Andrew Schwartz. Proceedings of the 61st Annual Meeting Association for Computational Linguistics (Volume 2: Short Papers). 2023.

Social science NLP tasks, such as emotion or humor detection, are required to capture the semantics along with implicit pragmatics from text, often limited amounts of training data. Instruction tuning has been shown improve many capabilities large language models (LLMs) commonsense reasoning, reading comprehension, and computer programming. However, little is known about effectiveness instruction on social domain where pragmatic cues needed be captured. We explore use for tasks introduce...

Many recent works in natural language processing have demonstrated ability to assess aspects of mental health from personal discourse. At the same time, pre-trained contextual word embedding models grown dominate much NLP but little is known empirically on how best apply them for assessment. Using degree depression as a case study, we do an empirical analysis which off-the-shelf model, individual layers, and combinations layers seem most promising when applied human-level tasks. Notably,...

Recent works have demonstrated ability to assess aspects of mental health from personal discourse. At the same time, pre-trained contextual word embedding models grown dominate much NLP but little is known empirically on how best apply them for assessment. Using degree depression as a case study, we do an empirical analysis which off-the-shelf language model, individual layers, and combinations layers seem most promising when applied human-level tasks. Notably, find RoBERTa effective and,...

Natural language is generated by people, yet traditional modeling views words or documents as if independently. Here, we propose human (HuLM), a hierarchical extension to the problem whereby human-level exists connect sequences of (e.g. social media messages) and capture notion that moderated changing states. We introduce, HaRT, large-scale transformer model for HuLM task, pre-trained on approximately 100,000 users, demonstrate its effectiveness in terms both (perplexity) fine-tuning 4...

Adithya V Ganesan, Vasudha Varadarajan, Juhi Mittal, Shashanka Subrahmanya, Matthew Matero, Nikita Soni, Sharath Chandra Guntuku, Johannes Eichstaedt, H. Andrew Schwartz. Proceedings of the Eighth Workshop on Computational Linguistics and Clinical Psychology. 2022.

Much of natural language processing is focused on leveraging large capacity models, typically trained over single messages with a task predicting one or more tokens. However, modeling human at higher-levels context (i.e., sequences messages) under-explored. In stance detection and other social media tasks where the goal to predict an attribute message, we have contextual data that loosely semantically connected by authorship. Here, introduce Message-Level Transformer (MeLT) -- hierarchical...

In human-level NLP tasks, such as predicting mental health, personality, or demographics, the number of observations is often smaller than standard 768+ hidden state sizes each layer within modern transformer-based language models, limiting ability to effectively leverage transformers. Here, we provide a systematic study on role dimension reduction methods (principal components analysis, factorization techniques, multi-layer auto-encoders) well dimensionality embedding vectors and sample...