The Utility of General Domain Transfer Learning for Medical Language Tasks

• DOI: 10.48550/arxiv.2002.06670
• Publication Date: 2020-01-01
• Authors: Daniel Ranti; Katie Hanss; Shan Zhao; Varun Arvind; J. Titano; Anthony Costa; Eric K. Oermann
• Keywords: Transfer of learning

The purpose of this study is to analyze the efficacy transfer learning techniques and transformer-based models as applied medical natural language processing (NLP) tasks, specifically radiological text classification. We used 1,977 labeled head CT reports, from a corpus 96,303 total evaluate pretraining using general domain corpora combined with bidirectional representations transformers (BERT) model for Model performance was benchmarked logistic regression bag-of-words vectorization long short-term memory (LSTM) multi-label multi-class classification model, compared published literature in BERT either set pretrained checkpoints outperformed achieving sample-weighted average F1-scores 0.87 biomedical-domain model. General may be viable technique generate state-of-the-art results within NLP tasks on corpora, outperforming other deep such LSTMs. could serve facilitate creation groundbreaking uniquely challenging data environment text.