NTR Webinar: Theoretical Analysis of Performance Measures for Classification and Clustering

NTR organizes and hosts scientific webinars on neural networks and invites speakers from all over the world to present their recent work. 

On December 14 Liudmila Prokhorenkova, Yandex Research, Moscow, Russia, presented a technical Zoom webinar on Theoretical Analysis of Performance Measures for Classification and Clustering. 

To compare different algorithms for a given task, several performance measures can be used. For instance, for classification, one can use accuracy, F-measure, or other options. 

Thus, there is a question of how to choose a proper measure. In other words, how can we validate validation measures? 

To answer this question, we conducted a systematic analysis of classification and clustering performance measures: we formally define a list of desirable properties and theoretically analyze which measures satisfy which properties. 

We also proved an impossibility theorem: some desirable properties cannot be simultaneously satisfied. 

Then we proposed a new family of measures satisfying all desirable properties except one. 

The talk is based on the papers “Systematic analysis of cluster similarity indices: How to validate validation measures” (ICML 2021) and “Good Classification Measures and How to Find Them” (NeurIPS 2021).

Slide deck link.

Moderator and contact: NTR CEO Nick Mikhailovsky: nickm@ntrlab.com.

Leave a Reply

Your email address will not be published. Required fields are marked *