Science and medicine are supposedly based on rigor—a rigor where theories are only correct if you can replicate results. It turns out, though, that the software used to analyse medical images of your brain gives wildly different answers if it’s run on Mac or PC.
The finding is completely crazy, and casts doubt over the way many scans—such as CT and MRI—are analysed in routine practice.
A team of German researchers took data from 30 brain scans and analysed them using a package called FreeSurfer—one of the major medical image analysis programs, which can be used to measure the size of different parts of the brain.
They ran the software on Windows, and also on Macs running different versions of Mac OS, each time using the software to measure the size and thickness of various structures of the brain. They found that not only did Windows and Mac OS installs of the software throw up different results—but the version of Mac OS also had an impact.
Across most sections there was at least a 2-5 percent variation in the answers. But in the parahippocampal and entorhinal cortex, the answers diverged by as much as 15 per cent. A 15 per cent variation just because of a Mac OS update. The results appear in PLoS One.
Sadly, it’s not clear why the problem arises. Indeed, that’s something that medical professionals and software developers need to work on—and quick. Because it’s a terrifying idea that a single piece of software that can be used to measure parts of the brain—including tumors—can throw up different results after even a simple operating system update.
That means that not only might different hospitals choose to treat patients differently, but the same hospital could in theory change its diagnosis as a result of an IT upgrade.
As NeuroSkeptic points out, it would be wrong to lay the blame solely at the feet of the software developers behind FreeSurfer, because this is probably a wide-spread problem across other imaging platforms, too. But regardless of whose fault it is, something needs to happen to address the problem, and the sooner, the better. [PloS One via NeuroSkeptic]