Large-scale applications
Our work in approximate dynamic programming evolved entirely from industrial applications. Below is a list of more recent projects (when our notation matured) of projects we have taken on for large-scale applicactions.
Locomotive planning at Norfolk Southern Railroad - ADP was used to produce a model used for strategic and tactical planning of locomotives at NS. The model optimizes over long horizons (we use a month for strategic planning, and a week for tactical planning). ADP allows us to model each locomotive individually, capturing operations at a very high level of detail. Click here for a complete summary and downloadable papers.
Fleet management at Schneider National: ADP was used to develop an optimization model for strategic planning to model Schneider's 5,000 long-haul drivers at a high level of detail. The model involves 50,000 variables (dimensions) per time period over 60 time periods, and a state variable with 10^{20} dimensions.
Simao, H. P. A. George, Warren B. Powell, T. Gifford, J. Nienow, J. Day, "Approximate Dynamic Programming Captures Fleet Operations for Schneider National," Interfaces, Vol. 40, No. 5, pp. 342-352, 2010. (c) Informs - This is a light version of the paper with no equations.
Simao, H. P., J. Day, A. George, T. Gifford, J. Nienow, W. B. Powell, “An Approximate Dynamic Programming Algorithm for Large-Scale Fleet Management: A Case Application,” Transportation Science, Vol. 43, No. 2, pp. 178-197 (2009). (c) Informs. This version has all the equations.
Managing high value spare parts at Embraer - The problem involves planning up to 1000 different types of high value spare parts. We used ADP to derive policies that achieve aggregate targets on service and costs (which prevented us from solving each problem individually). Demand rates varied from several per week, to less than one per year. In some cases, we would have to allocate a half dozen spare engines spread among 20 different locations. Lead times might be as long as six months, and we also had to determine whether to stock a part at a central depot (with longer, more expensive response times) or in the field (which limited the geographical region where a part could be used). Click here for a short paper summarizing the project.
Car distribution at Norfolk Southern - This was our first industrial application of ADP using piecewise linear, separable value functions. The system was used as a tactical planning system, planning movements for two weeks into the future. Click here for a book chapter that summarizes the model.
SMART: A stochastic, multiscale model for energy policy. This problem models hourly wind variations in an investment planning problem that spans 20 years (175,000 time periods). This is a policy model developed for Lawrence Livermore National Laboratory.