Multimodal skin lesion classification using deep learning

• DOI: 10.1111/exd.13777
• Publication Date: 2018-09-06T01:58:46Z
• Authors: Jordan Yap; William Yolland; Philipp Tschandl
• Keywords: Binary classification; Skin lesion; Contextual image classification; Multiclass classification

Abstract While convolutional neural networks (CNNs) have successfully been applied for skin lesion classification, previous studies generally considered only a single clinical/macroscopic image and output binary decision. In this work, we presented method which combines multiple imaging modalities together with patient metadata to improve the performance of automated diagnosis. We evaluated our on classification task comparison as well five class representative real‐world clinical scenario. showed that multimodal classifier outperforms baseline uses macroscopic in both melanoma detection ( AUC 0.866 vs 0.784) multiclass mAP 0.729 0.598). addition, quantitatively diagnosis lesions using dermatoscopic images obtains higher when compared images. performed experiments new data set 2917 cases where each case contains image, metadata.