Automating sedation state assessments using natural language processing

• DOI: 10.1111/jnu.12968
• Publication Date: 2024-03-27T06:44:46Z
• Authors: Aaron Conway; Jack Li; Mohammad Goudarzi...; Blair Warren; Sebastian Mafeld; Babak Taati
• Keywords: 03 medical and health sciences; machine learning; 0302 clinical medicine; nursing; 610; clinical documentation; Original Article; natural language processing; artificial intelligence; procedural sedation; 620

Abstract Introduction Common goals for procedural sedation are to control pain and ensure the patient is not moving an extent that impeding safe progress or completion of procedure. Clinicians perform regular assessments adequacy in accordance with these inform their decision‐making around titration also documentation care provided. Natural language processing could be applied real‐time transcriptions audio recordings made during procedures order classify states involve movement pain, which then integrated into clinical systems. The aim this study was determine whether natural algorithms will work sufficient accuracy detect sedation. Design A prospective observational conducted. Methods Audio from consenting participants undergoing elective performed interventional radiology suite at a large academic hospital were transcribed using automated speech recognition model. Sentences text used train evaluate several different NLP pipelines classification task. we evaluated included simple Bag‐of‐Words (BOW) model, ensemble architecture combining linear BOW model “token‐to‐vector” (Tok2Vec) component, transformer‐based RoBERTa pre‐trained Results total 15,936 sentences 82 analysis. achieved highest performance among three models area under ROC curve (AUC‐ROC) 0.97, F1 score 0.87, precision 0.86, recall 0.89. Ensemble showed similarly high AUC‐ROC 0.96, but lower 0.79, 0.83, 0.77. approach 0.97 0.7, 0.83 0.66. Conclusion best performance. Further research required confirm pipeline can accurately classifications data allow state assessments. Clinical Relevance Automating would more timely received by sedated patients, and, same time, decrease burden clinicians. Downstream applications generated classifications, including example visualizations state, may facilitate improved communication between clinicians, who performing supervision remotely. Also, accumulation multiple reveal insights efficacy particular sedative medications identify where current analgesia optimal (i.e. significant amount time spent “pain” “movement” states).