MDR Analysis in Imbalanced Datasets

Our paper on "A balanced accuracy function for epistasis modeling in imbalanced datasets using multifactor dimensionality reduction" has been accepted for publication in Genetic Epidemiology and will appear later this year. This paper describes a balanced accuracy function for estimating accuracy when the number of cases and controls is not equal. This new function has been implemented in the latest MDR software package.

Velez, D.R., White, B.C., Motsinger, A.A., Bush, W.S., Ritchie, M.D., Williams, S.M., Moore, J.H. A balanced accuracy function for epistasis modeling in imbalanced datasets using multifactor dimensionality reduction. Genetic Epidemiology, in press (2007).

Abstract:

Multifactor dimensionality reduction (MDR) was developed as a method for detecting statistical patterns of epistasis. The overall goal of MDR is to change the representation space of the data to make interactions easier to detect. It is well-known that machine learning methods may not provide robust models when the class variable (e.g. case-control status) is imbalanced and accuracy is used as the fitness measure. This is because most methods learn patterns that are relevant for the larger of the two classes. The goal of this study was to evaluate three different strategies for improving the power of MDR to detect epistasis in imbalanced datasets. The methods evaluated were: 1) over-sampling that resamples with replacement the smaller class until the data are balanced; 2) under-sampling that randomly removes subjects from the larger class until the data are balanced; 3) balanced accuracy [(sensitivity+specificity)/2] as the fitness function with and without an adjusted threshold. These three methods were compared using simulated data with two-locus epistatic interactions of varying heritability (0.01, 0.025, 0.05, 0.1, 0.2, 0.3, 0.4) and minor allele frequency (0.2, 0.4) that were embedded in 100 replicate datasets of varying sample sizes (400, 800, 1600). Each dataset was generated with different ratios of cases to controls (1:1, 1:2, 1:4). We found that the balanced accuracy function with an adjusted threshold significantly outperformed both over-sampling and under-sampling and fully recovered the power. These results suggest that balanced accuracy should be used instead of accuracy for the MDR analysis of epistasis in imbalanced datasets.

This work was supported by National Institutes of Health grants AI59694, HD047447, LM009012, RR018787, GM62758 and AG20135.