Most of supervised machine learning problems, including deep learning, are routinely solved via optimization recently. We will discuss several widely used optimization algorithms and mention how the current theory is (not) reflected in the practice. Lastly, we will talk about several challenges the field is currently facing.
Filip Hanzely is now PhD Student (Optimization) on KAUST. He received his Bc degree (Economics and Financial mathematics) at the Comenius University, Bratislava, Slovak Republic in 2016, and MSc degree (Mathematics and Statistics) at the University of Edinburgh in 2017. He had interned in Amazon, Berlin and in Google Research, New York, in summer 2018 and summer 2019, respectively.