The Expected Performance Curve
Samy Bengio, Mikaela Keller and Johnny Mariéthoz
IDIAP RR, Martigny, Switzerland.
ROC and DET curves are often used in the field of person authentication to assess the quality of a model or even to compare several models. We argue in this paper that this measure can be misleading as it compares performance measures that cannot be reached simultaneously by all systems. We propose instead new curves, called Expected Performance Curves (EPC). These curves enable the comparison between
several systems according to a criterion, decided by the application, which is used to set thresholds according to a separate validation set. A free sofware is available to
compute these curves. A real case study is used throughout
the paper to illustrate it. Finally, note that while this study was done on an authentication problem, it also applies to most 2-class classification tasks.