ORF 544 – Stochastic Optimization and Learning
Professor Warren Powell
Department of Operations Research and Financial Engineering
Last taught Spring, 2019
Not offered 2019-2020 (I am on sabbatical). But I am willing to give a tutorial lecture (~2 hours) if there is interest. Go to http://tinyurl.com/sequentialdecisionstutorial to sign up (so I can see how many are interested).
Course syllabus (updated January 28)
Making decisions under uncertainty is a universal human activity, something we have all done our entire lives, and generally something we do every day. The study of this rich problem area can be organized 
under a broad umbrella called “stochastic optimization.” Normally taught as a mathematically deep topic, ORF 544 will be taught with a primary emphasis on proper modeling, and the design and analysis of practical algorithms.
The course will present a unified framework for stochastic optimization that cuts across the many communities that contribute to the general problem of the design and control of systems under uncertainty (I call this the “jungle of stochastic optimization”). This modeling framework has been tested in problem domains spanning transportation and logistics, energy, health, finance, internet search, and even the laboratory sciences.
Audience: The course is appropriate for students in operations research, economics, computer science, applied math, and any field of engineering (e.g. for students interested in engineering controls). It is open to undergraduates with a strong interest in models and algorithms. The course requires a basic background in probability and statistics at the undergraduate level (e.g. ORF 245 – if you have ORF 309, even better). There is a small amount of material where a background in linear programming is useful, but this will not be required on problem sets or exams. I will occasionally bridge to more advanced probabilistic concepts, but this will be aimed at students without this background and is not required.
Readings: The course will be taught from a new book, Stochastic Optimization and Learning: A Unified Framework, that is being written. To download the book:
Click here to download book (updated March 21 2019)
Please enter any comments/suggestions at
http://tinyurl.com/OUUcomments
The book will introduce students to different notational systems, and will cover problems, modeling frameworks and algorithmic strategies from all of the books shown above.
Lecture slides for spring, 2019.
Week 1 – Introduction and overview
Week 3 – Derivative-based stochastic optimization
Week 4 – Derivative-free stochastic optimization Part I: PFAs and CFAs
Week 5 – Derivative-free stochastic optimization Part II: VFAs and DLAs
Week 6 – Notation and the unified modeling framework (pdf) (powerpoint)
Week 7 – Uncertainty and designing policies (pdf) (powerpoint)
Week 8 – Policy function approximations (PFAs) and policy search (pdf) (powerpoint)
Week 9 – Cost function approximations (CFAs) and introduction to Markov decision processes (pdf) (powerpoint)
Week 10 – Approximate dynamic programming and Q-learning (VFAs) (pdf) (powerpoint)
Week 11 – Approximate dynamic programming – Monotonicity and convexity (pdf) (powerpoint)
Week 12 – Lookahead policies (pdf) (powerpoint)