Machine Learning for Science via Fast Bayesian Optimisation and Quadrature

Masaki Adachi -
Data Science

Date: -
Location: Eurecom

Abstract: Bayesian inference plays a critical role in enhancing the reliability of machine learning by transparently revealing predictive uncertainty, a crucial aspect for science and engineering. Nonetheless, the high computational cost of Bayesian inference has limited its applicability. To address this issue, Bayesian quadrature was developed, which can efficiently solve the combined problems of function approximation and numerical integration through model-based active learning. Despite its numerous successes, the computational overhead of Bayesian quadrature poses a challenge for handling large datasets, expensive simulators, and more general problems such as combinations, permutations, graphs, and molecules. To tackle this issue, we have developed two algorithms: BASQ and SOBER. BASQ is a model-agnostic and parallelisable solution for Bayesian quadrature that offers significant speed gains with negligible overhead. On the other hand, SOBER offers a data-type-agnostic quadrature approach and offers global optimisation at the same time. Our methods have been successfully applied to real-world science and engineering problems such as drug discovery, battery control model selection, predictive maintenance for solar off-grid systems, and the detection of new molecules through spectra from extraterrestrial stars. Brief Bio: Masaki is a 2nd-year DPhil student in Engineering Science (Information Engineering) in the Bayesian Exploration Lab | Machine Learning Reading Group at the University of Oxford, co-supervised by Prof Michael A. Osborne and Prof David Howey from Battery Intelligence Lab. Masaki received a master of engineering from the University of Tokyo with President's award in 2014, then has been working for Toyota Motor Corporation as Data Scientist for electric vehicles, and now is an assistant manager at the headquarter as well as a Ph.D. student. References: Adachi, M., Hayakawa, S., Jørgensen, M., Oberhauser, H., & Osborne, M. A. (2022). Fast Bayesian inference with batch Bayesian quadrature via kernel recombination. NeurIPS 2022, arXiv:2206.04734. https://arxiv.org/abs/2206.04734 Adachi, M., Hayakawa, S., Hamid, S., Jørgensen, M., Oberhauser, H., & Osborne, M. A. (2023). SOBER: Scalable Batch Bayesian Optimization and Quadrature using Recombination Constraints. arXiv preprint arXiv:2301.11832. https://arxiv.org/abs/2301.11832 Adachi, M., Kuhn, Y., Horstmann, B., Osborne, M. A., & Howey, D. A. (2022). Bayesian Model Selection of Lithium-Ion Battery Models via Bayesian Quadrature. arXiv preprint arXiv:2210.17299. https://arxiv.org/abs/2210.17299