Cost-Sensitive Classifier Evaluation Using Cost Curves

The evaluation of classifier performance in a cost-sensitive setting is straightforward if the operating conditions (misclassification costs and class distributions) are fixed and known. When this is not the case, evaluation requires a method of visualizing classifier performance across the full range of possible operating conditions. This talk outlines the most important requirements for cost-sensitive classifier evaluation, and introduces a technique for classifier performance visualization – the cost curve – that meets all these requirements. This talk should be of interest to anyone who works in areas that use classifiers, for example, machine learning, pattern recognition, biometrics, and diagnosis.

Presenter(s)

Professor Robert Holte of the Computing Science Department at the University of Alberta is a former editor-in-chief of the journal “Machine Learning” and co-founder and former director of the Alberta Innovates Center for Machine Learning (AICML, now known as Amii). His current research is on single-agent heuristic search, with seminal contributions on bidirectional search, methods for predicting the run-time of a search algorithm, and the use of machine learning to create heuristics. Professor Holte was elected a Fellow of the AAAI in 2011.