Colleague Commentary - Clinical Trials: What's the Harm?

DR. LAZAR J. GREENFIELD

Surgeons have always been concerned about their outcomes, tending to publish their results as observational studies that include complications and deaths. Other "cognitive" physicians have sometimes criticized these reports, placing their faith in randomized clinical trials (RCTs) that compare treatment with control groups. So it must have been a shock for them to learn that some RCTs have been parading on feet of clay.

The problem is that RCT reporting emphasizes efficacy of the drug or treatment, but frequently obscures the adverse events or "harms" that occur. For example, if you have an effective arthritis drug like rofecoxib (Vioxx), which seems to have minimal side effects and a lower incidence of ulcers than NSAIDs do, it will be a blockbuster on the market. The initial favorable report was based on 6 months' worth of data, not the full year's worth of available information (JAMA 2000;284:1247-55). Guess which was more favorable to publish? Not only was there a comparable incidence of GI side effects at 1 year, there was also evidence of increased cardiovascular events (JAMA 2001;286:954-9).

The same adverse effect was noted when rofecoxib was compared with naproxen. But in that report, the higher incidence of myocardial infarcts was relegated to a short paragraph in the discussion section of the published paper. The authors claimed that the appearance of more cardiovascular events in the Vioxx group was really due to the protective effect provided to naproxen users. So the evidence of potential harm in 2001 was ignored for more than 3 years (JAMA 2004;292:2647-50).

Then, another trial to examine the efficacy of Vioxx in preventing recurrent colon polyps showed a dramatic increase in myocardial infarcts, and the company withdrew the drug. And where was the FDA during all this? Despite an authoritative report on the drug's cardiovascular toxicity (JAMA 2001;286:954-9), the FDA never insisted on a trial to define the problem, and in fact administratively suppressed the concerns and findings of its own investigators.

For this reason, many medical journals and editorial groups have endorsed the revised Consolidated Standards of Reporting Trials (CONSORT) statement, which provides a checklist aimed at improving the quality of reporting of harms-related data (Ann. Intern. Med. 2004;141:781-8). But why use the term "harms" instead of "adverse events"? Adverse events are limited to harmful side effects that occur during the trial and that can be linked to the intervention statistically, as well as those that may be contributed by the underlying disease process.

Once the trial is over, subsequent events may be reported by study participants spontaneously or by active surveillance. These harms are the totality of adverse consequences of the intervention and are the direct opposite of benefits, against which they must be compared. So the current recommendation is to report harms and avoid the reassuring term "safety," which can obscure results if there is no obvious evidence of harm, particularly when reported as a percentage. A treatment that is 95% safe means that 1 in 20 had a problem, which may be unacceptable in an otherwise low-risk population.

When an epidemiologist looked at the protocols of 122 trials registered with Danish ethics committees, he found that more than half of the outcomes measured were missing from the published papers. When asked about this, the authors usually denied that the data were recorded, despite evidence to the contrary. Of course, ambiguous or negative results are not likely to ever be published in the first place. A French team showed that only 40% of trials registered with its ethics committees in 1994 had been published by 2002 even though more than twice as many had been completed (BMJ 2005;331:19).

The potent combination of personal ambition, financial gain, and industry pressure makes it essential that journal editors adopt CONSORT and other measures to clean up clinical trial reporting. An important step in this direction is mandatory registering of all trials. In 2004, the International Committee of Medical Journal Editors declared that its members would not publish the results of trials that had not been placed in a public registry. To date, there are several registries in existence, including one run by the U.S. government. The World Health Organization is working on an online portal that would bind these databases into a single source. (For information on trial registration, go to http://register.clinicaltrials.gov.)

Drug companies are reluctant to support public disclosure of a trial's primary objective, likening it to a description of an invention before it is patented. And there might well be problems from patients demanding access to investigational drugs. But the results of all trials should be made public, not just the ones likely to be profitable.

So the next time you hear about an exciting new drug or treatment, remember "what's the harm?"