Predicting animal behavior from neuronal miniscope data: a deep learning approach

• DOI: 10.1117/12.3006089
• Publication Date: 2024-04-02T17:29:08Z
• Authors: Subhrajit Das; Janaka Senarathna; Emma Ouyang; Amit Banerjee; Arvind P. Pathak
• Keywords: Robustness; Modalities

The field of systems neuroscience strives to comprehend the intricate links between brain activity and behavior, with goal understanding neurological disorders targeted interventions, notably in feedback-based therapies like Deep Brain Stimulation (DBS). However, challenges persist correlating neuroimaging data especially freely behaving animals across different imaging modalities. In last decade, deep learning has emerged as a crucial tool for analyzing image enabling correlation or prediction from This study investigates application sophisticated architecture, specifically pre-trained ResNet50-BiLSTM model, predict behaviors data. acquired multicontrast (fluorescence FL, intrinsic optical signal IOS, laser speckle contrast LSC) synchronized behavioral five awake, healthy mice. Behavioral "syllables" (i.e. running, nest building/eating, minimally mobile) classes were annotated using Behavior Cloud software. Neural FL channel) sequences associated syllables, consisting 12 frames spanning one minute. A architecture was used sequence classification. We added fully connected layer after BiLSTM fine-tuned it compute class scores. Since we had only mice, evaluated model's performance generalizability 5-fold cross-testing strategy. Test accuracy utilized assess reliability robustness predictions, evaluation. Generalizability results showcase promising model predicting behavior neural images test reaching 93.75%. intend conduct additional research on this pipeline's by confusion matrices, ROC curves, AUCs, within One-vs-All (OvA) framework. initial lays groundwork exploration multi-modal approaches (e.g. vision transformers autoencoders) that encompass modalities accurate prediction.