In this paper, John Elder describes the power of Target Shuffling to evaluate the validity of your discovery. It’s a method that is particularly useful
for identifying false positives, or when two events or variables occurring together are perceived to have a cause-and-effect relationship, as opposed to a coincidental one. In data mining the more variables you have in your predictive model, the easier it becomes to ‘oversearch’ and identify false patterns among them—what Dr. Elder calls the ‘vast search effect.’