A5017109645- Emotion and Mood Recognition
- Deception detection and forensic psychology
- Social Robot Interaction and HRI
- Anomaly Detection Techniques and Applications
- Sentiment Analysis and Opinion Mining
- Domain Adaptation and Few-Shot Learning
- Online Learning and Analytics
- Humor Studies and Applications
- Topic Modeling
- Autism Spectrum Disorder Research
- Emotions and Moral Behavior
- Natural Language Processing Techniques
- Multimodal Machine Learning Applications
- Assistive Technology in Communication and Mobility
- Educational Games and Gamification
- Speech Recognition and Synthesis
- Psychopathy, Forensic Psychiatry, Sexual Offending
- Family and Disability Support Research
- Empathy and Medical Education
- BIM and Construction Integration
- Multi-Agent Systems and Negotiation
- Teaching and Learning Programming
- Color perception and design
- Music and Audio Processing
- Innovative Teaching Methods
University of Southern California
2018-2023
Southern California University for Professional Studies
2021-2023
Carnegie Mellon University
2023
Viterbo University
2021
Automated systems that detect deception in high-stakes situations can enhance societal well-being across medical, social work, and legal domains. Existing models for detecting videos have been supervised, but labeled datasets to train rarely be collected most real-world applications. To address this problem, we propose the first multimodal unsupervised transfer learning approach detects real-world, with-out using labels. Our subspace-alignment (SA) adapts audio-visual representations of...
Virtual and robotic agents capable of perceiving human empathy have the potential to participate in engaging meaningful human-machine interactions that support well-being. Prior research computational has focused on designing empathic use verbal nonverbal behaviors simulate attempt elicit responses from humans. The challenge developing with ability automatically perceive elicited humans remains largely unexplored. Our paper presents first approach modeling user during a agent. We collected...
Background and Context We replicated expanded on previous work about how well students learn dynamic programming, a difficult topic for in algorithms class. Their study interviewed number of at one university single term. recruited larger sample size students, over several terms, both large public private universities as liberal arts colleges.Objective Our aim was to investigate whether the results generalized other also groups students.Method who completed relevant portions their class,...
Automated systems that detect the social behavior of deception can enhance human well-being across medical, work, and legal domains. Labeled datasets to train supervised detection models rarely be collected for real-world, high -stakes contexts. To address this challenge, we propose first unsupervised approach detecting realworld, high-stakes in videos without requiring labels. This paper presents our novel affect-aware Deep Belief Networks (DBN) learn discriminative representations...
The self-supervised objective of masked prediction has led to promising performance gains on a variety downstream tasks. However, while most approaches randomly mask tokens, there is strong intuition that deciding what can substantially improve learning outcomes. We investigate this in continued pretraining setting which pretrained models continue pretrain domain-specific data before performing some task. introduce Difference-Masking, masking strategy automatically chooses during by...
Automated deception detection systems can enhance health, justice, and security in society by helping humans detect deceivers high-stakes situations across medical legal domains, among others. This paper presents a novel analysis of the discriminative power dimensional representations facial affect for automated detection, along with interpretable features from visual, vocal, verbal modalities. We used video dataset people communicating truthfully or deceptively real-world, courtroom...
Humans are social beings; we pursue goals in our daily interactions, which is a crucial aspect of intelligence. Yet, AI systems' abilities this realm remain elusive. We present SOTOPIA, an open-ended environment to simulate complex interactions between artificial agents and evaluate their In environment, role-play interact under wide variety scenarios; they coordinate, collaborate, exchange, compete with each other achieve goals. the interaction LLM-based humans within task space performance...
Multimodal foundation models that can holistically process text alongside images, video, audio, and other sensory modalities are increasingly used in a variety of real-world applications. However, it is challenging to characterize study progress multimodal models, given the range possible modeling decisions, tasks, domains. In this paper, we introduce Holistic Evaluation Models (HEMM) systematically evaluate capabilities across set 3 dimensions: basic skills, information flow, use cases....
In recent decades, the field of affective computing has made substantial progress in advancing ability AI systems to recognize and express phenomena, such as affect emotions, during human-human human-machine interactions. This paper describes our examination research at intersection multimodal interaction computing, with objective observing trends identifying understudied areas. We examined over 16,000 papers from selected conferences interaction, natural language processing: ACM...
Building socially-intelligent AI agents (Social-AI) is a multidisciplinary, multimodal research goal that involves creating can sense, perceive, reason about, learn from, and respond to affect, behavior, cognition of other (human or artificial). Progress towards Social-AI has accelerated in the past decade across several computing communities, including natural language processing, machine learning, robotics, human-machine interaction, computer vision, speech. Natural particular, been...
Automated deception detection systems can enhance societal well-being by helping humans detect deceivers and support people in high-stakes situations across health, social work, legal domains. Existing computational approaches for detecting have not leveraged dimensional representations of affect, specifically valence arousal, expressed during communication. My research presents a novel analysis the potential including affect machine learning models deception. work informs motivates...
In our multicultural world, affect-aware AI systems that support humans need the ability to perceive affect across variations in emotion expression patterns cultures. These must perform well cultural contexts without annotated datasets available for training models. A standard assumption affective computing is recognition models trained and used within same culture (intracultural) will better than on one different cultures (intercultural). We test this present first systematic study of...
In recent decades, the field of affective computing has made substantial progress in advancing ability AI systems to recognize and express phenomena, such as affect emotions, during human-human human-machine interactions. This paper describes our examination research at intersection multimodal interaction computing, with objective observing trends identifying understudied areas. We examined over 16,000 papers from selected conferences interaction, natural language processing: ACM...
The self-supervised objective of masking-and-predicting has led to promising performance gains on a variety downstream tasks. However, while most approaches randomly mask tokens, there is strong intuition that deciding what can substantially improve learning outcomes. We investigate this in continued pretraining setting which pretrained models continue pretrain domain-specific data before performing some task. introduce Difference-Masking, masking strategy automatically chooses during by...
In our multicultural world, affect-aware AI systems that support humans need the ability to perceive affect across variations in emotion expression patterns cultures. These must perform well cultural contexts without annotated datasets available for training models. A standard assumption affective computing is recognition models trained and used within same culture (intracultural) will better than on one different cultures (intercultural). We test this present first systematic study of...
A vast majority of the world's 7,000 spoken languages are predicted to become extinct within this century, including endangered language Ladin from Italian Alps. Linguists who work preserve a language's phonetic and phonological structure can spend hours transcribing each minute speech native speakers. To address problem in context Ladin, our paper presents first analysis representations machine learning models for classifying 32 phonemes Ladin. We experimented with novel dataset Fascian...
Automated systems that detect the social behavior of deception can enhance human well-being across medical, work, and legal domains. Labeled datasets to train supervised detection models rarely be collected for real-world, high-stakes contexts. To address this challenge, we propose first unsupervised approach detecting in videos without requiring labels. This paper presents our novel affect-aware Deep Belief Networks (DBN) learn discriminative representations deceptive truthful behavior....
Virtual and robotic agents capable of perceiving human empathy have the potential to participate in engaging meaningful human-machine interactions that support well-being. Prior research computational has focused on designing empathic use verbal nonverbal behaviors simulate attempt elicit responses from humans. The challenge developing with ability automatically perceive elicited humans remains largely unexplored. Our paper presents first approach modeling user during a agent. We collected...