The Cost of Fraud Prediction Errors

Abstract

The paper provides a cost-based explanation for decision makers’ reluctance to use fraud prediction models, particularly as these models have nearly doubled their success at identifying fraud (true positive rates) when compared to the initial models in Beneish (1997, 1999). We estimate the costs of fraud prediction errors from the perspective of auditors, investors, and regulators, and find that the costs of errors differ both within and across fraud/non-fraud groups. Because metrics commonly used to compare models assume costs equality within or across classes, we propose a cost-based measure for model comparison that nets the costs avoided by correctly anticipating instances of fraud (true positives), against the costs borne by incorrectly flagging non-fraud firms (false positives). We find that the higher true positive rates in recent models come at the cost of higher false positive rates, and that even the better models trade false to true positives at rates exceeding 100:1. Indeed, the high number of false positives makes all seven models we consider too costly for auditors or regulators to implement. For investors, M-Score and, in some cases, the F-Score are the only models providing a net benefit. This is because the main component of investors’ false positive costs is the profit foregone (or the loss avoided) by not investing in a falsely flagged firm, and these models are based on fundamental signals that have been shown to predict future earnings and returns. In sum, our evidence shows that as the number of false positives increases, the use of fraud prediction models becomes a value-destroying proposition. Hence, it suggests that researchers focus on lowering the false positive rates of their models rather than pursuing higher true positive rates.