The company had invested hundreds of millions of dollars investigating a new potential drug to treat a mental ailment, and they had zeroed in on a compound that showed promise but the compound was not passing the statistical tests required by the FDA. Elder Research was hired to examine the data and determine the drug’s viability.
The Challenge
To be approved, the compound had to be significantly more effective than a placebo — where patients react physically to a treatment with no compound. (In fact, the “placebo effect” is so powerful that some of the scientists believed it accounted for 70% of the efficacy of their existing drugs.) Crushingly, after months and $10 million invested in a double-blind trial with 1,000 patients, the compound was not passing the statistical tests the FDA would use. To go forward with the multi-year FDA application process would require another billion dollars of investment — an amount so large, they’d have to team with a larger rival company to proceed.
The Solution
Three competing tests were being used to evaluate the treatment’s effect — all accepted in the academic literature. Yet all three were failing. To examine these results, Elder Research took three steps.
- We determined there was information in each test; they were correlated but sufficiently different to provide more information together than alone.
- The time-critical business need was to make a rational investment decision on whether or not to go forward, not to pass the specific FDA tests. So, we focused on visualizing the effects of treatment, rather than on statistical significances.
- Previous analyses had measured patient outcome on an absolute scale. However, distributions seemed to vary significantly by several factors so we focused instead on relative improvement (or worsening) after treatment.
In Figure 1, the 500 patients who received the placebo are summarized in a three-dimensional density plot we custom-designed for this project. The 500 points are summarized by density manifolds; that is, by “shrink wrapped” shapes that depict the 3D data quartiles. The outer red shape is the smallest shape that contains three-quarters of the data. Inside it, the green shape is the smallest shape that contains half the data. Lastly, the two blue shapes contain the densest quarter of the data. All the points started at the origin in the center of the cube. Those that moved toward the upper-right corner got better, and those that moved toward the lower-left got worse. Examining the density, a small group of patients moved up very noticeably, but an apparently larger group got somewhat worse.
By way of contrast for the drug, as shown in Figure 2, the movement toward getting better (upper-right) is very strong and not countered in the data depicted. Clearly, the compound is having a very positive effect.
Results
When shown the plots and the research behind them, The client moved forward confidently and partnered with Pfizer to make the $1 billion investment. Years of further work proved the drug’s efficacy to the regulatory agency; it was approved, adopted, and a blockbuster success. Such an outcome is so rare that it was one of only three drugs the firm created in that decade. The data science triumphs were discovering a real effect where experts in the domain had been stymied, and communicating decisive results to decision makers with a novel visualization. Ultimately, the client experienced a huge business success and millions of their patients benefited from improved treatment.
