How Working with AI May Make Us Better Dermatologists

Dr. Veronica Rotemberg presented information about the current state of artificial intelligence (AI) tools in dermatology. There are few high-quality studies of AI for dermatologic purposes.

First, Rotemberg discussed the challenges dermatologists face when working with AI models. AI models that are trained using photographs may become less accurate over time and may incorporate details that dermatologists would otherwise ignore (e.g., pen markings). AI models depend on the statistical distribution of their training data and the pretest probability of individual clinicians. Even experts can be influenced by incorrect answers from AI models.

Second, Rotemberg described her research on AI for melanoma detection. The goal of melanoma detection is to identify as many melanomas as possible and biopsy as few benign lesions as possible. Prospective studies of AI models in their intended use setting are critical.

The PROspective Validation of AI MOdels for Dermoscopy study evaluated the diagnostic accuracy of the open-source AI algorithm “All Data Are Ext” for detecting melanoma and its utility for dermatologists. The study included 603 lesions selected for biopsy by a dermatologist at Memorial Sloan Kettering Cancer Center. The researchers established a predetermined 95% sensitivity threshold. The model demonstrated 96.8% sensitivity and 37.4% specificity. There was no significant difference in sensitivity for invasive versus in situ melanoma. Model specificity was worse in older patients, head and neck lesions, large lesions, and lesions with photodamage. The model was most successful at identifying nevi and least successful for identifying keratinocyte carcinoma and other nonmelanotic lesions (e.g., actinic keratosis, lentigo).

Overall, the AI model was more accurate than the dermatologists at a prespecified sensitivity level. After considering results from the AI model, dermatologists avoided skin biopsies for 173 lesions. Rotemberg discussed the strengths and limitations of this study and plans for future research.

Finally, Rotemberg shared information about additional opportunities for dermatologists to collaborate with AI including apps, AI scribes, and large language models (LLMs). The FDA has not commented on any dermatology apps; little information is available about their training data. A study by Rotemberg evaluating publicly available dermatology apps against a benchmark dataset found they were less than 20% sensitive for melanoma. Although Rotemberg does not recommend these apps, dermatologists should be familiar with them because patients may ask about them. Data on AI scribes specific to dermatology are limited. Results from a preliminary study showed they may reduce documentation time but increase note length. To date, no prospective studies of LLMs have demonstrated benefits; benchmarking experiments showed hallucination rates of 10-20%. Rotemberg encouraged dermatologists to experiment with these tools but emphasized that high levels of human supervision are needed.

Join us for 2026 DF Clinical Symposium, January 28–31. Register today!