Use of Smartphone Apps to Diagnose Skin Cancer: Beware

These days, it seems that there is a smartphone app for just about anything. The medical field is no stranger to apps, including ones that check for medication interactions, those that help people learn anatomy, and those that check for the possible reasons for various signs and symptoms. There are several apps that exist for patients that allow them to take a picture of a mole and have it analyzed to determine if it is likely benign (non-cancerous) or likely to be cancer.

Cancerous skin lesions are referred to as melanomas. Some moles are describes as dysplatic nevi, meaning that they are usually benign but may resemble a melanoma.

It is essential to detect dysplatic nevi and melanomas early so they can be removed before skin cancer spreads throughout the body. This requires going to a dermatologist for a full body skin check, typically about once a year. However, many people do not do this because of time pressure, financial stress, lack of health insurance, difficulties finding a dermatologist (skin doctor) in their area, and embarrassment about needing to remove their clothing during the physical exam. For such people, free or low cost medical apps can be an appealing alternative.

The problem, however, is that such apps are not subject to any type of regulation or oversight. The app makers protect themselves legally by requiring the user to accept a disclaimer saying they are only meant for educational purposes, but many patients will still likely rely on the results as medical advice. If the results of such apps are typically wrong, however, this can lead to a very dangerous situation in which patients believe that a melanoma is actually non-cancerous and do not have it evaluated by a dermatologist until it has worsened and spread throughout the body. In some cases, this can potentially lead to death.

In an upcoming issue of JAMA Dermatology, researchers from the University of Pittsburgh Medical Center tested four smartphone apps designed for melanoma detection with existing photographs. The pictures submitted were those that had undergone tissue analysis and were reviewed by a board certified dermatologist. In total, 188 images were used, 60 of which were melanomas. The others were benign.

The first three apps used algortithms to classify the images. Application 1 determined if the image was “problematic” or “okay.” Application 2 determined if the image was a “melanoma or if it “looks good.” Application 3 determined in the image was “high risk,” “medium risk,” or “low risk.” The fourth app did not use an algorithm but sent the image to a board-certified dermatologist for evaluation and provided results within a day. This app classified images as “atypical” or “typical.” Each app also classified images as unevaluable in some way, if necessary.

The results of the study were very concerning because there were too many false negatives (classifying abnormal results as normal) for the three automated programs. Even the best of these classified 30% of melanomas as benign. One of these programs only had a sensitivity of 7%, meaning that it only correctly identified 7% of melanomas as abnormal. This program was the most specific (94%) meaning that it correctly identified benign lesions as normal, but this is hardly reassuring since the program classified almost all images (including melanomas) as normal. The three other programs had specificities ranging from 30 to 39%.

The most sensitive program of the four was the only one that used a physician to review the images. This program only falsely classified 2% of melanomas as benign. The cost of this increased sensitivity was increased price of the product, as it was the most expensive of the four, costing five dollars per image analysis and a 24-hour turn-around time for results. The other programs were free or $4.99 for unlimited use and provided immediate results. As the saying goes “You get what you pay for.”

The critical statistics for classification accuracy are the positive predictive values (PPV) and negative predictive values (NPV). PPV indicates the proportion of positive test results that are true diagnoses. All four apps performed poorly in this regard, with PPVs ranging from 33 to 42%. For the app using the physician, this is because it classified many normal images as abnormal. The NPV for this app though, was 97%, meaning that 97% of negative test results were correct. The other programs had NPVs of 65 to 73%.

Suggested reading: Beating Melanoma: A Five-Step Survival Guide
 
Reference: Wolf JA, Moreau J, Akilov O, Patton T, English JC, Ho J, Ferris LK. (2013). Diagnostic Inaccuracy of Smartphone Applications for Melanoma Detection. JAMA Dermatol. 16:1-4.