Marrying Stochastic Control and Machine Learning: From Bermudan Options to Natural Gas Storage and Microgrids

Simulation-based strategies bring the machine learning toolbox to numerical resolution of stochastic control models. I will begin by reviewing the history of this idea, starting with the seminal work by Longstaff-Schwartz and through the popularized Regression Monte Carlo framework. I will then describe the Dynamic Emulation Algorithm (DEA) that we developed, which unifies the different existing approaches in a single modular template and emphasizes the two central aspects of regression architecture and experimental design. Among novel DEA implementations, I will discuss Gaussian process regression, as well as alternative simulation designs (space-filling, sequential, adaptive, batched). The DEA template is illustrated with multiple examples drawing from Bermudan option pricing, natural gas storage valuation, and optimal control of back-up generator in a power microgrid. This is partly joint work with Aditya Maheshwari (UCSB).