With the start of the summer Olympics in Beijing just days away, the attention of many throughout the world is turning towards athletic competition and the national pride it can bring. However, even with all the celebration and pageantry, the specter of doping weighs over the games more heavily than ever before.
In order to ensure a level playing field for all athletes, the World Anti-Doping Agency, International Olympic Committee, and individual national delegations have worked together to establish testing protocols designed to catch athletes who, despite the growing sophistication of testing methods, choose to pharmaceutically enhance their chances. In this week's Nature, Dr. Donald A. Berry, chair of the Department of Biostatistics at University of Texas, writes that the methods used to catch doping athletes have substantial statistical flaws.
Dr. Berry's analysis pertains mostly to testing for the metabolites of banned substances. Many synthetic forms of testosterone (far and away the most common form of doping among athletes) are metabolized and passed from the body through urine as a variety of derivatives that occur naturally. The metabolite concentrations in urine can vary based on diet and natural variations in hormone levels. The World Anti-Doping Agency determines whether an athlete has doped by testing for unusual ratios among these various metabolites that are indicative of synthetic hormone use.
This type of testing assumes that an unusual result is proof of guilt, a flawed form of reasoning known as the prosecutor's fallacy. As an example (lifted from Dr. Berry's commentary), consider a criminal suspect who is accused based solely on blood testing that matches only 1 out of every 1000 people. The prosecutor of the case may try to convince a jury that the odds of this person being guilty are 999:1, but in a city of 1,000,000 people, if there is no other evidence, the true odds of guilt are actually 1001:1 against. This, in essence, is the same reasoning applied to many forms of drug testing approved by the World Doping Agency; an athlete is assumed to be guilty simply by having a relatively rare test result.
Dr. Berry states that this does not mean we cannot determine cheaters by these testing methods—he argues that we simply need to apply the appropriate statistical methods to the tests. In order to determine whether a test result is simply unlucky or the result of doping, we need to know two statistical properties of the test: the odds that a true doper tests positive ('sensitivity'), and the chances that an honest athlete tests positive (one minus 'specificity').
Herein lies the problem – the Wold Doping Agency has neither conducted nor published the necessary studies to establish sensitivity and specificity in these metabolite ratio tests. An analysis of the case of Floyd Landis from the Tour de France, assuming reasonable values of sensitvity and specificity, indicates that there was between an 8 and 34 percent chance of registering a false positive. Given that he had eight different opportunities to test positive (far more than the average Tour rider because Mr. Landis was a front-runner throughout the race), the case against him suddenly looks substantially weaker.
It is important to note that the current statistics are equally as poor at proving innocence as they are at proving guilt. Considering the extreme ramifications of testing positive, however, the World Anti-Doping Agency owes it to the athletes to establish more statistical rigor in testing methods, and should be far more transparent when it comes to its methods. It is quite clear that, with current methods, some cheating athletes are going undetected while a finite number of innocent athletes are being ruined by false-positive tests.
Nature, 2008. DOI not yet available.