This post is based on a submission by reader Professor Daniel Bitran.

Definitions

fMRI is a way of measuring blood-flow in the brain or spinal cord and, by extension, neural activity in those areas. A false positive is the sort of mistake your smoke detector makes when it goes off, but there's no smoke.

Summary

Commentary

The image above is striking because the false positives seem to show neural activity in a dead salmon's brain. According to researchers at UCLA Santa Barbra, these errors are due to a problem of multiple comparisons.

Imagine we're playing One of These Things is not Like the Others with several tin cans of Atlantic salmons. At first, it's hard to tell which one is not like the others—there's a bunch of canned salmon. They have similar color, weight, shape, etc. But as we add different ways of comparing the cans (or more cans to compare), we increase the probability that there will be some way in which one of them differs from the rest—particularly because of small differences, say manufacturing defects.

[Note: Corrections appreciated.] Now imagine we're collecting data for an fMRI. Each each point (called a voxel) is measured several times with certain extreme values discarded. Now we want to figure out which of the voxels is not like the others—that's because that's where we expect to see differences in blood flow. However, by comparing voxels we're actually comparing multiple measurements of each voxel to multiple measurements of other voxels. This is like adding more ways of comparing the cans. Moreover, because there is a small bit of noise, the measurements for each voxel can be slightly different each time. This is why we occasionally find some difference between neighboring voxels that isn't really there—it's a false positive. Luckily there are ways of correcting for this sort of error, but unfortunately, it is not applied as frequently as it should.

Meta

What are other examples of widespread errors of multiple comparisons or false positives?

Acknowledgements

Thanks to Craig Bennett of Prefrontal.org for providing a high resolution version of the Atlantic salmon fMRI.

See Also

• Human Brain Mapping 2009 – Presentations at Prefrontal.org for the conference posters & slides.

• Multiple Comparisons with Repeated Measures by David Howell for a lengthy discussion on the subject.

• The Importance of Numeracy