Sequential Decision Analytics and Modeling

Warren B. Powell
Professor Emeritus, Princeton University
Chief Innovation Officer, Optimal Dynamics

Sequential decision problems arise in virtually every form of human processes: transportation and logistics, supply chain management, energy, health (from public health to medical decision making), finance, e-commerce, laboratory sciences, … 

Sequential Decision Analytics and Modeling uses a teach-by-example style to illustrate a universal framework for modeling sequential decision problems.  The universal framework applies to any sequential decision problem, from active learning problems up through complex resource allocation problems.  Chapters are accompanied by python modules that have implemented the models, but the book should be of value even to people not interested in writing code.

There are two ways to get the book:



I recently designed a new way of teaching introduction to optimization for undergraduates and masters students.  This strategy is summarized in a monograph which provides a course outline in the form of 11 topics, many of which reference Sequential Decision Analytics and Modeling. The new approach is given at:



Chapter 1 illustrates the core modeling framework using two inventory examples, and introduces the four classes of policies that encompass any method used for making decisions.

Chapters 2-6 use a series of examples designed to illustrate each of the four classes of policies, along with different styles for modeling uncertainty.  We pause in chapter 7 to revisit the universal modeling framework, drawing on the applications from chapters 2-6 to illustrate state variables (including belief states), scalar and vector-valued decisions, uncertainty modeling, different types of objective functions, and the four classes of policies.

The book is designed to be used as a standalone introduction to the vast field of sequential decision analytics.  For more information, see the sequential decision analytics website. A youtube video introduction to sequential decision analytics is available here. A webpage of links to books, videos, discussions, and suggested courses is available here.

For advanced readers interested in developing models and algorithms, I recommend my new book Reinforcement Learning and Stochastic Optimization: A unified framework for sequential decisions


While the entire book is focused on sequential decisions under uncertainty, the book assumes no background in any form of stochastic optimization or dynamic programming. I taught out of an earlier version of this book in an undergraduate course at Princeton. The complete lecture notes are available at  (scroll down to the “Undergraduate course”).  I also suggest an introductory weekly seminar on the same webpage (near the top).

Supplementary material

There is additional material, such as spreadsheets and datasets, at