Advancing skin cancer detection through deep learning and fusion of patient metadata and skin lesion images

Comprehensive Summary

This study explores whether combining patient metadata with skin lesion images can improve artificial intelligence-based (AI) detection of suspicious skin lesions during teledermatology triage. The researchers analyzed 79,246 dermoscopic and DSLR images from 39,623 skin lesions collected with 22 meta-features from 19,295 patients across UK diagnostic clinics. Multiple EfficientNet-B2 AI models were developed: one was trained only using metadata, one was trained only using images, and the last was trained using both the metadata and images. Lastly, the one training had decisions of the AI models fused through a majority voting technique. Metadata-only models showed the weakest performance, and the image-only models had high sensitivity, but limited specificity. Fusing images with metadata improved both metrics, and the ensemble decision-making yielded the highest specificity. These findings highlight the value of using patient metadata as an addition to image-based AI.

Outcomes and Implications

Rising skin cancer incidences in the last three decades and the shortage of dermatology specialists have increased delays in diagnosis, making the need for efficient triage tools increasingly necessary. Improving specificity can reduce unnecessary clinic visits. This multi-modal AI model could be implemented as a supplementary decision making tool in teledermatology workplaces, assisting clinicians in more effective decisions.