Effect of reject option on classifier performance

S. Dreiseitl, M. Osl

Research output: Chapter in Book/Report/Conference proceedingsConference contributionpeer-review

Abstract

Binary classifier systems that provide class membership probabilities as outputs may be augmented by a reject option to refuse classification for cases that either appear to be outliers, or for which the output probability is around 0.5. We investigated the effect of these two reject options (called "distance reject" and "ambiguity reject", respectively) on the calibration and discriminatory power of logistic regression models. Outliers were found using one-class support vector machines. Discriminatory power was measured by the area under the ROC curve, and calibration by the Hosmer-Lemeshow goodness-of-fit test. Using an artificial data set and a real-world data set for diagnosing myocardial infarction, we found that ambiguity reject increased discriminatory power, while distance reject decreased it. We did not observe any influence of either reject option on the calibration of the logistic regression models.

Original languageEnglish
Title of host publication23rd European Modeling and Simulation Symposium, EMSS 2011
Pages176-180
Number of pages5
Publication statusPublished - 2011
Event23rd European Modeling and Simulation Symposium, EMSS 2011 - Rome, Italy
Duration: 12 Sep 201114 Sep 2011

Publication series

Name23rd European Modeling and Simulation Symposium, EMSS 2011

Conference

Conference23rd European Modeling and Simulation Symposium, EMSS 2011
CountryItaly
CityRome
Period12.09.201114.09.2011

Keywords

  • Classifier systems
  • Performance evaluation
  • Reject option

Fingerprint Dive into the research topics of 'Effect of reject option on classifier performance'. Together they form a unique fingerprint.

Cite this