Bayesian Additive Regression Networks: A Machine Learning Approach and Software Library
When
Where
Speaker: Danielle Van Boxel, Program in Applied Mathematics, University of Arizona
Title: Bayesian Additive Regression Networks: A Machine Learning Approach and Software Library
Abstract: We adapt principles for Bayesian sampling of decision trees to ensembles of neural networks for regression and classification tasks. Using Markov Chain Monte Carlo, we sample from the posterior distribution of single hidden layer neural networks. On regression test data benchmarks, BARN averaged between 5% and up to 20% lower root mean square error. For classification, we introduce an additional Probit-like latent target in order to use the same regression approach. Here, we find that BARN is relatively best in specific scenarios, and even then by only a few percentage points as measured by test accuracy. BARN's performance was robust even without cross-validation, making it time-competitive with other methods. This robustness and accuracy makes BARN an attractive algorithm for regression and classification modeling.
Additionally, we make BARN available as a Python package, `barmpy`, with documentation at https://dvbuntu.github.io/barmpy/ for general machine learning practitioners. Our object-oriented design is compatible with SciKit-Learn, allowing usage of their tools like cross-validation. To ease learning to use barmpy, we produce a companion tutorial that expands on reference information in the documentation. Any interested user can `pip install barmpy` from the official PyPi repository.