The courses below all focus on the theme of making decisions over time under uncertainty. These are topics that fall under the broad heading of “stochastic optimization” (or many other names – see jungle.princeton.edu), which is a topic traditionally taught at a very mathematical level. Notably, two are taught at the undergraduate level. The graduate course is designed for a broad audience, and typically draws students from a wide range of departments (and is open to undergraduates).
In order of increasing complexity and sophistication, the themes of the courses are:
- ORF 418 – Optimal learning
- This course focuses on pure learning problems, where decisions are to acquire information. Students learn about proper modeling of sequential decision problems, and are introduced to the four classes of policies, but in the context of relatively simple problems.
- ORF 411 – Sequential Decision Analytics and Modeling
- This course introduces students to the proper modeling of sequential decision problems, illustrated on a broad set of applications, spanning various resource allocation problems (asset selling, energy storage, and blood management), pure learning problems, and problems that combine physical resources and learning. The course uses a “teach by example” style, where the problems are chosen to bring out all four classes of policies, along with the challenges of learning and modeling uncertainty. Lectures are organized around an on-line book written specifically for this course, with a supporting set of python modules to illustrate the algorithms.
- ORF 544 – Stochastic Optimization and Learning
- This course is more of a classic methods-based introduction to stochastic optimization and learning, spanning both pure learning problems (derivative-based and derivative-free), as well as the entire spectrum of state-dependent optimization problems. The course uses a new book (in progress) that is being written entirely around a unified framework that optimizes over policies, and is organized around the four major classes of policies. The course emphasizes modeling and algorithms, and is aimed at a broad audience (primary prerequisite is a basic course in probability and statistics).
- ORF 411 – Sequential Decision Analytics and Modeling
- Newly re-designed, ORF 411 tackles the universal problem of making decisions over time, under different forms of uncertainty. The course introduces a universal framework that brings together all the different tools in the “jungle of stochastic optimization.”
- ORF 418 – Optimal learning
- This course focuses on the challenges of making good decisions that balance “learning while earning” – we want to do the best, but we have to learn while we are doing. Applications range from the internet (choosing the best ad to maximize ad-clicks) to a variety of problems in engineering, science, and everyday life where you have to use intelligent trial-and-error.
- ORF 544 – Stochastic optimization and learning
- ORF 544 is a course that teaches a unified framework for stochastic optimization and learning, spanning classical fields such as stochastic search, dynamic programming, stochastic programming, and optimal (stochastic) control, and related fields including reinforcement learning/approximate dynamic programming, model predictive control, online computation, and decision trees. We consider both offline formulations (learning in the lab) and online (learning in the field), in a simple, integrated framework. Students will see how these diverse fields can be viewed from this common framework, opening up both new problem classes, as well as new solution approaches. The course will emphasize proper modeling, and the study of general computational frameworks rather than the highly specialized problems that yield analytical solutions. General modeling formulations will be illustrated with a wide range of applications drawn from different fields (depending on the makeup of the class), spanning business, economics, engineering, and the sciences. The course will be guided by extensive experience with computational work on a range of applications.
These are not taught 2019-2020 (I am on sabbatical, working to finish the ORF 544 text).
ORF 411 – Sequential Decision Analytics and Modeling
Newly re-designed, ORF 411 tackles the universal problem of making decisions over time, under different forms of uncertainty. The course introduces a universal framework that brings together all the different tools in the “jungle of stochastic optimization.”
The course focuses on modeling, using an application-based teaching approach that uses a variety of problems to teach different modeling devices. The goal is to then design effective policies for making decisions under uncertainty. Students learn the four classes of policies; one of these four classes (or a hybrid) will provide an effective policy (sometimes optimal) for any problem.
ORF 411 is open to undergraduates (juniors and seniors) and graduates from any background, with just a course in probability and statistics (some python coding is required). Students from engineering, social and physical sciences, computer science and economics are welcome.
This course focuses on the challenges of making good decisions that balance “learning while earning” – we want to do the best, but we have to learn while we are doing. Applications range from the internet (choosing the best ad to maximize ad-clicks) to a variety of problems in engineering, science, and everyday life where you have to use intelligent trial-and-error.
ORF 544 – Stochastic optimization and learning
ORF 544 is a course that teaches a unified framework for stochastic optimization and learning, spanning classical fields such as stochastic search, dynamic programming, stochastic programming, and optimal (stochastic) control, and related fields including reinforcement learning/approximate dynamic programming, model predictive control, online computation, and decision trees. We consider both offline formulations (learning in the lab) and online (learning in the field), in a simple, integrated framework. Students will see how these diverse fields can be viewed from this common framework, opening up both new problem classes, as well as new solution approaches.
The course will emphasize proper modeling, and the study of general computational frameworks rather than the highly specialized problems that yield analytical solutions. General modeling formulations will be illustrated with a wide range of applications drawn from different fields (depending on the makeup of the class), spanning business, economics, engineering, and the sciences. The course will be guided by extensive experience with computational work on a range of applications.