Title: Auditing: Active Learning with Outcome-Dependent Query Costs

Abstract: We propose a learning setting in which unlabeled data is free, and the cost of a label depends on its value, which is not known in advance. Specifically, we study binary classification in an extreme case, where the algorithm only pays for negative labels. Our motivation is applications such as fraud detection, in which investigating an honest transaction should be avoided if possible. We term the setting "auditing", and consider the "auditing complexity" of an algorithm. We design auditing algorithms for simple hypothesis classes,
and show that with these algorithms, the auditing complexity can be significantly lower than the active label complexity. We also consider a general competitive approach for auditing,

and demonstrate its potential for linear classification.

Joint work with Anand Sarwate and Nati Srebro from TTI-Chicago