Research activities at PENSA

Our research focuses on the design and control of energy systems under uncertainty. Much of the research at PENSA has a significant dimension of mathematical modeling and algorithms. We also feature selected senior theses which tend to focus on applications in energy systems.

Research papers and projects

Undergraduate senior theses and thesis topics

The undergraduates contribute in a significant way to our understanding of energy systems through a diverse range of projects. Below are some selected theses from recent years.

Undergraduate senior theses from 2014-2015

Undergraduate senior theses from 2013-2014

Undergraduate senior theses from 2012-2013

Undergraduate senior theses from 2011-2012

Undergraduate senior theses from 2010-2011

Undergraduate senior theses from 2009-2010

Research papers and projects

M. Coulon, J. Khazaei, W.B. Powell, "SMART-SREC: A Stochastic Model of the New Jersey Solar Renewable Energy Credit Certificate Market,"

Markets for solar renewable energy certificates (SRECs) are gaining in prominence in many states, stimulating growth of the U.S. solar industry. However, SREC market prices have been extremely volatile, causing high risk to participants and potentially less investment in solar power generation. Such concerns necessitate the development of realistic, flexible and tractable models of SREC prices that capture the behavior of participants given the rules that govern the market. We propose an original stochastic model called SMART-SREC to fill this role, drawing on established ideas from the carbon pricing literature,
and including a feedback mechanism for generation response to prices. We calibrate the model to the New Jersey market, analyze parameter sensitivity, and demonstrate its ability to reproduce historical dynamics, while also inferring current expectations of future solar growth. Finally, we investigate the role and impact of regulatory parameters, thus providing insight into the crucial role played by market design.

W. R. Scott and W. B. Powell, “Portfolio Selection and Covariance Matrix Estimation using an Errors-in-Variables Factor Model with an Application to the PJM Electricity Market,"

This paper considers the problem of purchasing portfolios of virtual bids (or financial trading rights) over the PJM grid. The idea is to use classical Markowitz theory to purchase robust portfolios, but it is well known that this does not work if standard methods are used to estimate the covariance matrix. This paper describes a new method based on insrumental variables (see the next paper) that makes it possible to estimate covariance matrix from a factor model on just a few days of data. We found that this makes it possible for the model to capture short periods of volatility and therefore respond in the makeup of the portfolio by using a more responsive covariance matrix.

 

Warren Scott, W. B. Powell,  “Approximate Dynamic Programming for Energy Storage with New Results on Instrumental Variables and Projected Bellman Errors,”

This paper considers a fundamental configuration of energy from wind (stochastic supply), the grid (stochastic price), stochastic, time varying loads, and a storage device. The problem is modeled in 15 minute increments in steady state, which requires a discount factor of .999 to produce realistic results. While this appears to be a small, stylized problem, it is not possible to solve it optimally (even with reasonable levels of discretization). We create a series of benchmark problems on simplified versions of this problem. We then test two classes of approximate dynamic programming algorithms: one based on Bellman-error minimization, and one based on direct policy search. Several variations of Bellman-error minimization are considered, including least squares approximate policy iteration (LSAPI), and two variants: one which uses instrumental variables, and one which uses projected Bellman error minimization. We show that these two variants are actually equivalent, and produce results that are much better than vanilla LSAPI, with results that are 70-80 percent of optimal. However, direct policy search was found to produce results that are about 95 percent of optimality.

Donghun Lee, W. B. Powell, "An Intelligent Battery Controller Using Bias-Corrected Q-learning," AAAI 2012 (to appear)

The transition to renewables requires storage to help smooth short-term variations in energy from wind and
solar sources, as well as to respond to spikes in electricity spot prices, which can easily exceed 20 times their
average. Efficient operation of an energy storage device is a fundamental problem, yet classical algorithms such
as Q-learning can diverge for millions of iterations, limiting practical applications. We have traced this behavior
to the max-operator bias, which is exacerbated by high volatility in the reward function, and high discount
factors due to the small time steps. We propose an elegant bias correction procedure and demonstrate its effectiveness.

Jae Ho Kim, W. B. Powell, “Optimal Energy Commitments with Storage and Intermittent Supply,” Operations Research, Vol. 59, No. 6, pp. 1347-1360 (2011).

Energy storage has long been viewed as a critical dimension in the use of energy from intermittent sources. One dimension of our work has focused on designing simple analytical policies that guide the process of making commitments for energy from wind in the presence of finite capacity solar devices. This work has produced a simple formula for the value of storage, making it possible for managers and analysts to perform quick, simple analyses of the value of energy storage while assuming a practical, implementable policy for making wind commitments. This work is summarized in

Kim, Jae Ho, Powell, Warren B., An hour-ahead prediction model for heavy-tailed spot prices, Energy Economics (2011), doi:10.1016/j.eneco.2011.06.007

A second line of research addressed the problem of energy arbitrage - storing energy when prices are low and releasing it when prices are higher. We designed a simple control limit policy that is easy to implement and avoids assumptions that implicitly require knowing the future. In the process, we discovered that electricity prices are highly volatile, and do not fit standard jump diffusion models that are popular in the academic literature. In fact, we found strong evidence to support the hypothesis that electricity spot prices exhibit infinite variance. We derived a new pricing model based on the concept of median-reverting price processes. The model uses a Cauchy distribution, and produces a much more accurate fit to prices for both Ercot and PJM West (on the left) than we were able to achieve using a jump diffusion model (on the right).

Powell, W. B., George, A., A. Lamont and J. Stewart, “SMART: A Stochastic Multiscale Model for the Analysis of Energy Resources, Technology and Policy,”

SMART is a stochastic, multiscale energy resource planning model that can capture hourly variations in wind, solar and demand over a multidecade horizon (over 175,000 time periods), capturing uncertainty in wind, prices and demand as well as climate technology and policy. The model makes decisions that range from hourly dispatch and storage decisions to yearly investment decisions in conversion, transmission and storage technologies. The technology for making these decisions is based on the modeling and algorithmic framework of approximate dynamic programming.

The model is currently being tested on a spatially aggregate dataset, which is allowing us to benchmark the ADP results against optimal decisions produced by a linear programming model, but this only applies to deterministic settings. This research introduces new theoretical and algorithmic challenges, and is leading us to new fundamental research in machine learning and approximate dynamic programming.

A companion paper provides a theoretical proof of convergence of the algorithm in SMART, but limited to the energy storage problem within a year:

J. Nascimento, W. B. Powell, “An Optimal Approximate Dynamic Programming Algorithm for the Energy Dispatch Problem with Grid- Level Storage,”

L. Hannah, W. B. Powell, and J. Stewart, “One-Stage R&D Portfolio Optimization with an Application to Solid Oxide Fuel Cells"

We have a line of research focusing on the general problem of R&D portfolio optimization. One of the challenges in energy involves deciding what project to work on. We have considered the problem of choosing a subset of research projects to achieve the best improvement in performance in a complex device such as a solid-oxide hydrogen fuel cell. This research is summarized in:

Optimal design of wind farm portfolios

When a grid operator uses inputs from multiple wind farms, the combined effect of all the wind farms will generally produce lower volatility than a single wind farm. We are developing algorithms for finding the best combination of wind farm locations to determine the lowest possible volatility (see graph to the right). This may have an impact on the ability of the grid to use wind energy in the most effective way.

J. Enders, W. B. Powell, D. Egan, “A Dynamic Model for the Failure Replacement of Aging High-Voltage Transformers,” Energy Systems Journal, Vol. 1, No. 1, pp. 31-59 (2010) (c) Springer

This paper introduces a model that supports these efforts by optimizing the acquisition and the deployment of high-voltage transformers dynamically over time. We formulate the problem as a Markov Decision Process which cannot be solved for realistic problem instances. Instead we solve the problem using approximate dynamic programming using three different value function approximations, which are compared against an optimal solution for a simplified version of the problem. The methods include a separable, piecewise linear value function, a piecewise linear, two-dimensional approximation, and a piecewise linear function based on an aggregated inventory that is shown to produce solutions within a few percent with very fast convergence. The application of the best performing algorithm to a realistic problem instance gives insights into transformer management issues of practical interest.

 

Undergraduate senior theses from 2014-2015:

Saumya Singh (2015) - Princetonian Electricity: Managing an Isolated Microgrid - This is a stochastic model of Princeton university as a microgrid, performed using SMART-ISO. The thesis includes a formal statement of the stochastic optimization model, and a complete set of simulations evaluating the cost of different configurations of generation and storage. Available for download are:

• A short executive summary
• The full senior thesis
• Powerpoint presentation of the research

Undergraduate senior theses from 2013-2014:

We have begun a robust grid project with SAP and PSE&G to minimizes outages from storms. Theses contributing to this project include:

Kevin Cen (2014), Entropy Minimization and Locating Faults Across the Electrical Network using Customer No Light Calls- PSE&G depends on customer-reported outages to indicate when storm damage has caused an outage. This thesis uses these phone calls to estimate the likelihood of an outage at each point in a circuit given the breaker configuration.

Mark Holekamp (2014), Keeping the Lights On: An Analysis of the Dynamic Allocation Problem of Assigning Utility Repair Trucks to Outages - Using the (imperfect) state of the grid, the next step is to actually dispatch utility trucks. This research uses a storm simulator to test the effectiveness of different policies.

Daniel P. Chen (2014), Analyzing Transformer Replacement Policies: A Simulation Approach to Reducing Failure Risk
- Utilities have to deal with the challenge of replacing/repairing transformers that are close to the end of their lifetime. Such transformers are more vulnerable to failures from storm-related events which create current shortages. The problem is that we generally do not know the state of the transformer. This thesis tests policies for minimizing the risk of excessive replacements despite having an imperfect knowledge of the actual state of each transformer.

Other energy related research includes:

Henry Chai (2014), A Statistical Model for Simulating Solar Intensity in New Jersey
- We are beginning research to study the impact of dramatic increases in solar energy in New Jersey. This research builds a statistical model of solar energy using a very detailed dataset of solar output from PSE&G.

Luke L. Cheng (2014), Solar, Wind, and Storage: Optimizing for Least Cost Configurations of Renewable Energy Generation in the PJM Grid
- Sometimes simple models provide valuable insights. This thesis revisits a recent paper that argues that 99 percent of our electricity can come from wind and solar. In our effort, using wind and solar data for the PJM region, we allow ourselves to purchase energy from the grid at a potentially very high price. Even at high prices, we show that the optimal configuration never approaches 99 percent.

Kevin Lin (2014), Approximate Dynamic Programming Applied to Biofuel Markets in the Presence of Renewable Fuel Standards
- One way to reduce emissions from transportation fuels is to subsidize credits for the use of biomass. This thesis develops a stochastic optimization model to analyze the dynamics of such a market.

Oladoyin F. Phillips (2014), Policies for Investing in Nigeria’s Power Delivery Capabilities - Nigeria has a relatively simple energy system which is unable to meet the needs of the country. This thesis identifies three types of flows: energy (natural gas), electricity, and money. They are tightly linked, and all three networks struggle with outages and "leaks". This thesis develops a stochastic model that links all three flows to develop an understanding of how an investment in one component interacts with the others.

Undergraduate senior theses from 2012-2013:

Haotian (Cosmo) Zhong (2013), Replicating Electricity Spot Prices Through Inverse Optimization of Supply Shifts

Shreyashi Ghosh (2013), The Future of Solar:  An Analysis of New Jersey's Market for  Solar Renewable Energy Credits (SRECs)

Taman Narayan (2013), Modeling Government Contracting: A Principal-Agent Approach with Imperfect Monitoring and Constrained Rewards, Certificate Program in Applied and Computational Mathematics, Economics Department.

Alexander Ogier (2013), Optimizing Princeton’s Energy Use, Department of Computer Science

Tarun Sinha (2013), Resource Optimization in the Princeton University Energy System, Department of Mechanical and Aeronautical Engineering

Tarun Sinha (2013), A Stochastic Gradient Method to Match Actual Resource Demand in Energy Management Systems, Certificate Program in Applied and Computational Mathematics, Department of Mechanical and Aeronautical Engineering

Timothy Wenzlau (2013), Nested Newsvendor Optimal Commitment Policies in Day-Ahead and Hour-Ahead Electric Capacity Forward Markets

Undergraduate senior theses from 2011-2012:

A Stochastic Unit Commitment Model in the Presence of Offshore Wind Energy

Kevin Kim

This thesis documents the work that Kevin has contributed to the SMART-ISO model on the day-ahead problem. He solves the day-ahead unit commitment problem, which is a large integer programming model adjusted to handle significant levels of uncertainty from wind.

 

 

Natural Gas Power Generation in the Presence of Wind: A Mixed Integer Linear Programming Approach to the Hour-Ahead Unit Commitment Problem

Steven Chen

This thesis documents the work that Steve has contributed to the SMART-ISO model on the hour-ahead problem, which is an integer programming model for planning natural gas generation. This model plans 1-2 hours into the future in five-minute increments. Using the SMART-ISO model, with Kevin Kim's day-ahead model, Steve ran an extensive series of simulations testing the effect of different levels of wind. The goal is to understand how short-term variations from the forecast can be compensated using natural gas combustine turbine.

 

 

Dynamic Pricing of Electric Vehicle Charging Locations: An Application of Optimal Learning

Yu-Sung Huang

Yu-Sung addressed the problem of quickly learning optimal prices for recharging electric vehicles at parking garages. He compared a number of learning algorithms, and found that the knowledge gradient using a parametric belief model quickly found the best price.

 

Undergraduate senior theses from 2010-2011:

Cell Charging Challenges:An Optimal Pricing Strategy fo ra Solar Mobile Charging System in Africa

Megan Wong

Millions of Africans use cell phones, with limited access to electricity. This introduces the challenge of how to charge cell phone batteries. Because cell phones have their own batteries, this makes them a natural candidate or energy from wind and solar. But African commnities do not have the economics and infrastructure to build and support large wind and solar farms. This thesis is looking into entrepreneurial models to develop and spread the use of energy from small solar panels and micro wind turbines.

Exploiting the Inverse Capacity-Rate Relationship in a Stochastic Setting:
Control Algorithm Development for Hybrid Energy Storage
in Renewable Energy Applications

Sami Yabroudi (Electrical engineering)

Energy storage devices come in a variety of styles in terms of cost, storage capacity, and the rate at which energy can be stored or withdrawn. Wind is a source of energy that arrives at different speeds. When the wind is strong, we need storage devices that can store energy at high speed with minimum loss, but these devices tend to be expensive and therefore with limited capacity. Other technologies have a lower throughput, but are cheaper and offer more storage. This thesis explores both the engineering of these devices, and offers a control policy based on approximate dynamic programming for deciding when to store energy in each device, as a function of the wind speed.

 

Estimating the Number of Plug-in Hybrid Electric Vehicles Necessary to Offset Volatility
from 20% Wind Generation

Ben Sheng

Plug-in electric vehicles offer a significant source of stored energy that utilities are interested in using to offset the volatility of energy from wind turbines. This thesis formulates the problem of when to charge and discharge batteries for electric vehicles in a region, modeled as a single energy aggregator. The model is solved using dynamic programming to find an optimal policy for charging and discharging, and the model is then used to develop an understanding of the number of cars needed to balance a set of wind turbines.

Controlling the Elements: Regulating Wind with Hydro in China

Hui (CinCin) Fang

This thesis explores the idea of using the Three Gorges Dam to perform limited regulation of variable generation of electricity using wind. The idea is to exploit the huge storage potential of the Three Gorges Dam, while recognizing that because of downstream uses of water, we are limited in the degree to which the dam can be regulated. However, hydroelectric power offers the significant advantage of being dispatchable, which means that it can be adjustesd on very short notice. This thesis sets up the regulation problem as a Markov decision process and quantifies the degree to which hydroelectric power might be used as a form of regulation.

 

 

Wind on Water: Powering China with an Integrated Wind-Hydro System

Rui Zhang

One way to reduce the aggregate variability of energy generated from wind is to create a portfolio over space that strikes a balance between the amount of energy generated from wind and the covariances from different locations. Rui Zhang uses historical data from different locations around China to estimate a covariance matrix, and then uses Markowitz portfolio theory to find optimal locations which balance average wind energy and the variance of the entire portfolio.

Undergraduate senior theses from 2009-2010:

20% Wind Generation and the Energy Markets

Jessica Zhou

Supervised by: Warren Powell

Increasing the supply of electrical energy from wind to 20 percent will dramatically increase the overall volatility of the supply of energy, with implications for the overall energy infrastructure. To try to quantify these impacts, Jessica develops a detailed unit commitment model to optimize 50 energy generation sources on the PJM network, including coal, nuclear and natural gas along with wind, scaled to provide 20% of total electricity. The unit commitment model, which is a large integer program solved with Cplex, is imbedded in a larger simulator that solves the unit commitment problem once each day, and then steps forward hour by hour, simulating uncertainty in demand, prices and supply from wind. Tunable policies are used to make adjustments between the advance commitments and actuals. She shows, for example, that increasing wind to 20% of total, at zero cost, reduces total costs by about 12 percent, much less than expected. Additional studies are performed to understand the how this number changes with changes in the accuracy of wind forecasts and the amount of storage.

 

 

Wind and Pumped Hydro-Power Storage: Determining Optimal Commitment Policies with Knowledge Gradient Nonparametric Regression

Christine Schoppe

Supervised by: Warren Powell

Pumped hydro is the most popular form of energy storage used in the United States, and the type of storage which can store the most energy for the longest periods. This thesis presents models of rainfall, wind and electricity prices to develop an accurate model of energy generation from wind and water. She develops a mathematical model and proposes a policy for controlling the storage of excess wind energy in storage, which she runs using five years of data from PJM. The storage policy is tuned using the knowledge gradient algorithm, and the results are calibrated against those of a real application.

 

 

The Valuation of Natural Gas Storage: A Knowledge Gradient Approach with Nonparametric Estimation

Jennifer Schoppe

Supervised by: Warren Powell

This thesis begins with a thorough introduction of the natural gas market, including extraction, production, transport, storage, and distribution. She also provides a good description of electricity pricing models. She describes storage and valuation strategies based on the spread option approach and approximate dynamic programming. She then describes a policy for optimizing the use of pumped hydro storage in the presence of stochastic processes for wind, rain and prices. She tunes the parameters of the pricing model using the knowledge gradient algorithm, and then uses the resulting policy to value the natural gas storage. Her valuations compare quite well against industrial estimates for a storage facility.