A quasi‐exact test for comparing two binomial proportions

Abstract

The use of the Fisher exact test for comparing two independent binomial proportions has spawned an extensive controversy in the statistical literature. Many critics have faulted this test for being highly conservative. Partly in response to such criticism, some statisticians have suggested the use of a modified, non‐randomized version of this test, namely the mid‐P‐value test. This paper examines the actual type I error rates of this test. For both one‐sided and two‐sided tests, and for a wide range of sample sizes, we show that the actual levels of significance of the mid‐P‐test tend to be closer to the nominal level as compared with various classical tests. The computational effort required for the mid‐P‐test is no more than that needed for the Fisher exact test. Further, the basis for its modification is a natural adjustment for discreteness; thus the test easily generalizes to r × c contingency tables and other discrete data problems.