Comparative study of gene set enrichment methods
Open Access
- 2 September 2009
- journal article
- research article
- Published by Springer Science and Business Media LLC in BMC Bioinformatics
- Vol. 10 (1), 275
- https://doi.org/10.1186/1471-2105-10-275
Abstract
Background The analysis of high-throughput gene expression data with respect to sets of genes rather than individual genes has many advantages. A variety of methods have been developed for assessing the enrichment of sets of genes with respect to differential expression. In this paper we provide a comparative study of four of these methods: Fisher's exact test, Gene Set Enrichment Analysis (GSEA), Random-Sets (RS), and Gene List Analysis with Prediction Accuracy (GLAPA). The first three methods use associative statistics, while the fourth uses predictive statistics. We first compare all four methods on simulated data sets to verify that Fisher's exact test is markedly worse than the other three approaches. We then validate the other three methods on seven real data sets with known genetic perturbations and then compare the methods on two cancer data sets where our a priori knowledge is limited. Results The simulation study highlights that none of the three method outperforms all others consistently. GSEA and RS are able to detect weak signals of deregulation and they perform differently when genes in a gene set are both differentially up and down regulated. GLAPA is more conservative and large differences between the two phenotypes are required to allow the method to detect differential deregulation in gene sets. This is due to the fact that the enrichment statistic in GLAPA is prediction error which is a stronger criteria than classical two sample statistic as used in RS and GSEA. This was reflected in the analysis on real data sets as GSEA and RS were seen to be significant for particular gene sets while GLAPA was not, suggesting a small effect size. We find that the rank of gene set enrichment induced by GLAPA is more similar to RS than GSEA. More importantly, the rankings of the three methods share significant overlap. Conclusion The three methods considered in our study recover relevant gene sets known to be deregulated in the experimental conditions and pathologies analyzed. There are differences between the three methods and GSEA seems to be more consistent in finding enriched gene sets, although no method uniformly dominates over all data sets. Our analysis highlights the deep difference existing between associative and predictive methods for detecting enrichment and the use of both to better interpret results of pathway analysis. We close with suggestions for users of gene set methods.Keywords
This publication has 27 references indexed in Scilit:
- Modeling Cancer Progression via Pathway DependenciesPLoS Computational Biology, 2008
- Module-Based Outcome Prediction Using Breast Cancer CompendiaPLOS ONE, 2007
- Random-set methods identify distinct aspects of the enrichment signal in gene-set analysisThe Annals of Applied Statistics, 2007
- Statistical assessment of functional categories of genes deregulated in pathological conditions by using microarray dataBioinformatics, 2007
- Oncogenic pathway signatures in human cancers as a guide to targeted therapiesNature, 2005
- Gene set enrichment analysis: A knowledge-based approach for interpreting genome-wide expression profilesProceedings of the National Academy of Sciences of the United States of America, 2005
- Global detection of molecular changes reveals concurrent alteration of several biological pathways in nonsmall cell lung cancer cellsMolecular Genetics and Genomics, 2005
- Ontological analysis of gene expression data: current tools, limitations, and open problemsBioinformatics, 2005
- Cancer genes and the pathways they controlNature Medicine, 2004
- The Hallmarks of CancerCell, 2000