Between Yes and No: looking beyond binary

Abstract: Often we attempt to answer a question with a “yes” or a “no” by developing predictive models (“Will the small remaining population of axolotls survive outside of their native wetlands?”) or by implementing binary classifiers (“Is this a cat or a dog?”). However, the answers that are provided by our models are often given in terms of probabilities. Even more confusing, different models - equally good according to accuracy metrics - can produce conflicting answers. In this talk I will explore these issues and discuss their implications. How do we interpret an answer that is neither “yes” nor “no”? For example, a PCR test for COVID yields a probability. How does the choice of threshold affect the individual? How does it affect policy decisions, or the course of the disease? How can we disentangle the predictions given by competing models , i.e. how can we deal with predictive multiplicity? For example, if two models disagree on whether or not someone is a loan risk, which one should be trusted? Which groups are most affected? What new metrics can be used to compare models? This talk is a survey intended for a general math audience.

Back to Search Research Symposium Abstracts