The client’s risk model had been developed and deployed by dozens of expert statisticians over many years. The client was interested in whether any new insights would be produced by using modern machine learning techniques. To be adopted, the new model needed to have significantly better predictive performance and be suitable to run in the client’s production scoring environment.
The Challenge
The Solution
Elder Research applied a robust testing framework to evaluate alternative machine learning approaches— many not common in the industry — to find more effective techniques for predicting the probability that credit card accounts would default within three months. We built and tested numerous advanced algorithms to determine which technique was most effective. We also tested and ranked the predictions from many combinations of the best algorithms as shown in the figure below and found an ensemble model (MPN) that demonstrated record-breaking performance.
Results
The new model ensemble reduced the number of credit card accounts that defaulted on the client’s evaluation data set by more than 10% when compared to their production model.