Computational Math and Statistics Seminar by Aleksei Sorokin: Fast Gaussian Process Regression with Derivative Information

Speaker: Aleksei Sorokin, Ph.D. candidate, Illinois Institute of Technology

Title: Fast Gaussian Process Regression with Derivative Information

Abstract:

Gaussian process regression, or kriging, is a method for high dimensional interpolation which has gained popularity thanks to the ability to encode assumptions about the underlying simulation into the covariance kernel and the ability to quantify prediction uncertainty. A significant drawback is that fitting a kriging model to n data points typically costs O(n^3) as it is required to solve a system involving the n x n Gram matrix of pairwise covariance kernel evaluations. Hickernell and Jagadeeswaran show in [1,2] that when one has control over the design of numerical experiments then specially chosen sampling locations and matching covariance kernels yield structured Gram matrices for which we can complete all necessary computations in O(n \log n). Specifically, lattice and digital sequences from Quasi-Monte Carlo paired with matching kernels yield circulant and block-Toeplitz Gram matrices respectively.

We have extended the cubature rules implemented in QMCPy to surrogate modeling with support for noisy observations and derivative information. We present methods and Julia software for fitting these fast kriging models in O(n \log n), including software to generate Quasi-Monte Carlo point sets. 

[1] Jagadeeswaran, R., & Hickernell, F. J. (2022). Fast Automatic Bayesian Cubature Using Sobol’ Sampling. In Advances in Modeling and Simulation: Festschrift for Pierre L'Ecuyer (pp. 301-318). Cham: Springer International Publishing.

[2] Jagadeeswaran, R., & Hickernell, F. J. (2019). Fast automatic Bayesian cubature using lattice sampling. Statistics and Computing, 29(6), 1215-1229.

 

Computational Mathematics and Statistics Seminar