Guest speaker series on Graphical Models and Neural Networks

Gaussian Graphical Models on top of Softmax Neural Classifiers: Out-of-distribution Detection and Noisy Labels

Given or to build a soft-max neural classifier, we propose new methods for detecting out-of-distribution samples (which are drawn far from the training distribution) and handling noisy labels during training. In both methods, our common idea is inducing the generative classifier (e.g., Gaussian graphical models) on top of fixed features from the pre-trained model. Such “deep generative classifiers” have been largely dismissed for fully-supervised classification settings as they are often substantially outperformed by discriminative deep classifiers (e.g., softmax classifiers). In contrast to this common belief, we show that it is possible to formulate a simple generative classifier that is significantly more robust to out-of-distribution test samples as well as labeling noise during training without much sacrifice of the original discriminative performance. This lecture is based on two papers: arxiv.org/abs/1807.03888 and arxiv.org/abs/1901.11300.