Outcome switching

Outcome switching is the practice of changing the primary or secondary outcomes of a clinical trial after its initiation. An outcome is the goal of the clinical trial, such as survival after five years for cancer treatment. Outcome switching can lead to bias and undermine the reliability of the trial, for instance when outcomes are switched after researchers already have access to trial data. That way, researchers can cherry pick an outcome which is statistically significant.

Problem

Outcome switching can undermine the reliability of the trial, for instance when outcomes are switched when researchers already have access to the trial data. It can lead to bias in terms of benefits and harms. For example, when the findings using the original protocol were statistically insignificant, a study may cherry pick a new outcome measure that is statistically significant. If there are a large number of outcomes to choose from, it is likely at least one will show significant findings, assuming no correction is made for multiple testing. Incomplete or misleading reporting of outcomes is an example of unethical research practice.

Possible solutions

Primary outcomes for clinical trials need to be defined upfront to prevent a biased selection of outcomes. Some medical journals require adherence to the CONSORT standards, which require authors to document any deviations from the initial study protocol and explain why changes were made. For instance, JAMA and the British Medical Journal require inclusion of the CONSORT checklist. When pre-specified outcomes are defined but unclear, researchers still have wriggle room to choose the most favourable option. A clear pre-defined outcome is therefore important to avoid outcome switching.
Study authors may engage in outcome switching due to high pressures to publish. It is more difficult to publish negative results, especially in high-impact journals. Registered reports—a type of journal article where editors pledge to publish clinical trials irrespective of their results—may help combat this pressure to produce positive results. Initial peer review takes place based on the methodology and the reasoning behind the study.

Prevalence

Despite problems with outcome switching, the practice is common. Changes in primary outcome metrics are present in nearly one in three studies. Outcome switching also occurs frequently in follow-up studies. In an analysis of oncology trials, outcome switching was more common in studies with a male first author, and in studies funded by non-profits. One study analysed outcome switching in five top medical journals, writing letters for each misreported trial outcome. Journal editors and clinical trial authors typically responded dismissively when concerns were raised, misrepresenting ethical standards and including ad hominem attacks.

Examples

A drug against major depressive disorder, paroxetine, was marketed for children and teenagers for years, based on a flawed trial that involved outcome switching. The trial's protocol had described two primary and six secondary outcomes by which it would measure efficacy. The data showed that, according to those eight outcomes, paroxetine was no more effective than placebo. According to Melanie Newman, writing for the BMJ, "he drug only produced a positive result when four new secondary outcome measures, which were introduced following the initial data analysis, were used instead. Fifteen other new secondary outcome measures failed to throw up positive results."