Supplementary material for Reinforcement Learning and Stochastic Optimization

This page contains different forms of supplementary material for Reinforcement Learning and Stochastic Optimization: A unified framework for sequential decisions (referred to below as RLSO).The page is a work in progress, although I anticipate steady additions (even after the book is published).

Companion document

RLSO is a graduate level book focusing on modeling and algorithms, with supporting applications.  A helpful companion document is

Sequential Decision Analytics and Modeling

Companion python modules for most chapters available here.

This is a teach-by-example book that I first wrote for an undergraduate course, but it is helpful for anyone learning this material for the first time.  Most of the chapters (other than chapters 1, 7 and 8) are focused on a specific application, chosen to bring out specific modeling issues.  For example, chapters 2-6 walk the reader through applications that illustrate all four classes of policies.  Each application chapter follows the same outline (as you can see looking at the table of contents).  

The book will undergo a major revision fall, 2021, but will always be available for free download.

Readers may contribute their own chapters at 

Sequential Decision Analytics and Modeling: The public version

This book can be edited by the public.  I am looking for models of your own problems.  My hope is that these are not too complicated, and can be written in no more than 5-6 pages.


Link to Github library for python modules for Sequential Decision Analytics and Modeling:

Github library

Python module for dynamic shortest path problem


I will be making datasets needed for various exercises here.

Chapter 10: Uncertainty modeling

Spreadsheet of electricity price data