A decade-old CNN beats frontier LLMs at diagnosing skin cancer

Magnus Lynch, a consultant dermatologist and Mohs surgeon at Guy's Hospital and King's College London, set up a simple experiment. He took the two most capable language models of October 2026 — GPT-6 Astra and Claude Fable 5.1 — and compared them with Inception v3, a convolutional network first published by Google in 2015. He fine-tuned the old network in roughly twenty minutes on a single H100 GPU, using 68,011 clinical and dermoscopic images from the public ISIC archive.

The test

Lynch drew 200 skin lesions at random from MILK10k, a public dataset released in 2025. Each lesion is photographed twice: an ordinary clinical close-up and a shot through a dermoscope. All 200 diagnoses were confirmed by histopathology, including the benign ones — which makes the sample deliberately hard: a mole someone chose to excise is one a clinician found worrying.

Half were lesions that must not go untreated: 30 melanomas, 25 basal cell carcinomas, 25 squamous cell carcinomas and 20 actinic keratoses. The other half were 50 harmless naevi and 50 seborrhoeic keratoses. All three were asked to pick one of six diagnoses, give a confidence and a second choice, and decide whether the patient could safely be discharged.

62% against 42%

The convolutional network named the right diagnosis for 125 of 200 lesions (62%). Both language models managed only 84 and 85 (42%). Allowing a second choice, the network reached 173 against 134 and 138. It called melanoma first choice 13 times out of 30, the models 8 and 11.

The widest gap was on the harmless lesions. The models also failed in telling, predictable ways: GPT-6 Astra labelled 75 lesions basal cell carcinoma when only 25 were, while Claude Fable 5.1 overused "actinic keratosis".

One figure from the article says it all: an invasive melanoma on the ear was called a harmless seborrhoeic keratosis by both LLMs, which advised discharge. The network recognised the melanoma.

Caveats

Lynch is candid about the limits. The sample is small, and 30 melanomas leave wide uncertainty. Each LLM case was run once, and answers vary. MILK10k has been public since 2025, so the models may have seen it in training — which would only have helped them. Finally, Inception v3 is a deliberately outdated baseline; the regulated systems in clinical use perform considerably better.

His conclusion is plain: a small, old, narrowly specialised network on an obsolete architecture recognises skin lesions far better than two frontier language models.

Source: Hacker News