Fast Transactive Control for Frequency Regulation in Smart Grids with Demand Response and Energy Storage

Abstract

This paper proposes a framework for controlling grid frequency by engaging the generation-side and demand-side resources simultaneously, via a fast transactive control approach. First, we use a proportional frequency-price relation to build and analyze a transactive frequency droop controller for a single-area power grid. Then, we develop a transactive demand response system by incorporating a large population of thermostatically controlled air conditioning loads. A proportional-integral controller is used to adjust the setpoint temperature of the air conditioners based on price variations. A battery storage system is then developed and augmented to the system to capture the energy arbitrage effects. A nonlinear price-responsive battery management system is developed to enable effective charging and discharging operations within the battery’s state-of-charge and power constraints. Simulation results indicate that the proposed transactive control system improves the steady-state and transient response of the grid to sudden perturbations in the supply and demand equilibrium. To decouple frequency from price during daily operation and maintain frequency near the nominal value, we propose adding a feedforward price broadcast signal to the control loop based on the net demand measurement. Through various simulations, we conclude that a combination of feedback transactive controller with feedforward price broadcast scheme provides an effective solution for the simultaneous generation-side and demand-side energy management and frequency control in smart power grids.

1. Introduction

This paper investigates an alternative solution to the load frequency control (LFC) problem in smart grids through a fast transactive approach. The main benefit of the transactive control is enabling a direct and transparent medium for compensating the services provided by the demand-side and energy storage resources for frequency regulation in power grids. This paper investigates scenarios in which thermostatically controlled loads (TCLs) in buildings can be used alongside battery storage and generation-side resources to balance power supply and demand in the grid.

Energy conservation in buildings has become one of the top priorities of electric utilities in the past few decades [1,2]. New technologies have been developed to propel rapid information and statistics for the energy conservation efforts [3,4], and the expanding usage of distributed renewable energy [5,6]. However, due to the high intermittency of renewable sources, the power supply and demand balancing paradigms have to shift from traditional LFC and automatic generation control (AGC) [7,8] to a mix of general-side and demand-side control strategies [9,10].

Studies on demand-side energy management expand in many different fields of literature. Economic incentives such as rebate programs and time-of-use electricity pricing have been the primary tools to achieve effective demand response [11,12,13]. The authors in Reference [11] investigate the consumer behavior in response to peak-time rebate programs using a multi-stage stochastic optimization process and conclude that a rational consumer changes his/her consumption pattern in order to increase his/her welfare. The paper cautions that depending on whether the economic incentive is lower or higher than the retail price of electricity, this behavior may result in potentially larger overall energy consumption despite reduction during peak hours. Reference [12] studies the optimal response of household appliances with thermostatic controllers to time-varying electricity price signals. The appliances reduce their demand during peak hours and coordinate with one another to maximize the utilization of the on-site energy production such as solar power. The authors in Reference [13] propose a two-level day-ahead energy planning process that uses a dynamic benefit function to maximize consumer benefit while reducing the financial risks imposed on the energy providers. It is shown that, through his process, the grid load curve flattens, and consumers see about 20% cut in their energy bills.

In several studies, energy demands for residential units have been modeled by major thermostatically controlled loads (TCLs) such as refrigerators, air conditioners (ACs), and water heaters [14,15,16,17,18,19]. These loads are deemed as flexible because their demands can be shifted significantly through small offsets in their setpoint temperatures or on-off schedules. References [14] and [15] develop diffusion and transport-based partial differential equation models for the aggregate air conditioning loads, as well as setpoint offset algorithms to balance power supply and demand in the presence of renewable power. In Reference [16], a quadratic programming-based hybrid controller is proposed to achieve the optimal TCL response to a time-varying electricity price signal, using an experimentally validated air conditioner model. Reference [17] applies dynamic programming to solve the optimal TCL control problem and discovers that precooling under the real-time electricity market can result in more economic benefits than that under the day-ahead market. Packetized energy management (PEM) is another recently developed framework for controlling the aggregate TCL load [18,19]. In this framework, which is inspired by digital communications, the end-users request for certain amount of energy packets over fixed periods of time. The PEM algorithm then decides whether to accept or deny the packets based on the energy availability, thereby balancing the power supply and demand.

Several studies on transactive grid control have analyzed how a price moderator sets up a clearance price that factors in generation costs along with energy demands [20,21,22,23]. Reference [20] models a transactive control scheme for a single commercial building showing that the price market can passively influence the heating, ventilation, and air conditioning (HVAC), lighting, and electric vehicle (EV) loads in real-time. The simulation results show improvements in energy savings and efficiency. Reference [21] proposes a study for transactive control and the charging of 200 EVs with a day-ahead forecast and a real-time EV charging management. The authors use a model predictive control method to allow the scheduler/aggregator to handle the uncertainty of various users’ charging demands while making profit to regulate the grid. Reference [22] proposes an energy management system based on transactive control to compare its efficacy against a more centralized method. The authors find the advantages of transactive control for multiple energy systems are scalability for having multiple energy hubs, flexibility for the aggregators, and privacy-preserving for consumers. Studies of transactive control also show the ability to handle the intermittencies of renewable energy. Reference [23] implements transactive control on a conventional model of an energy grid with cloud computing to improve robustness and flexibility in the exchange of information so that intermittencies with renewable energy sources like wind energy can be better transmitted in frequency regulation of the grid. Results show improvements versus conventional hardware. The authors warn that cyber security and robustness challenges will need to be addressed.

One of the main challenges in implementing some of these solutions is the delay in the process due to the addition of a third party, and time needed to converge to a clearance price. In practice, to have many devices communicate and measure data in the smart grid environment involves the concept of a middleware structure [24]. The middleware architecture features end user domains, aggregators, and power generator domains that are essential for incorporating transactive control policies.

Linking electricity price to grid frequency has been the subject of a few studies. One of the early efforts used a proportional price-frequency relation to achieve frequency droop control in a power network with price responsive generators [10,25]. In these studies, the electricity price is proposed to be set as a linear function of frequency within a certain range around the nominal frequency. In Reference [10], the price signal is used to control the generators, whereas loads such as TCLs are controlled directly using the frequency signal. Therefore, the supply and demand-side resources are not controlled via a common variable. Other studies investigate combining the day-ahead electricity market prices with the real-time frequency-based price adjustments [26,27] to achieve optimal transactive frequency control. These studies use different optimization schemes such as linear programming [26] and model-predictive control [27] to optimize the dispatch schedules and reference setpoints for an optimal grid performance. Therefore, an optimization process is placed within the feedback loop, which can significantly increase the computational requirement compared to the conventional AGC-based algorithms.

To develop a fully automatic feedback control scheme operated based on a single market-wide control variable, i.e., electricity price, this paper develops a transactive feedback controller that performs similarly to a conventional frequency droop controller without requiring any real-time optimizations. To maintain the frequency deviations near zero, the controller is equipped with a feedforward component based on the real-time demand measurements. We develop an integrated network of generation-side and demand-side resources such as steam generators, renewables, TCLs, and batteries to demonstrate the effectiveness of the proposed control strategy. The individual TCL and battery controllers are developed to synergistically respond to the proposed transactive frequency controller. We investigate the response of the system under various step changes in the demand, and use the California ISO power demand and renewable energy data to simulate the system for a 24-h power cycle. Various simulations are presented to show the improvements and drawbacks of the proposed transactive and the subsystem-level controllers.

Remainder of the paper is organized as follows: Section 2 presents the generation-side transactive frequency control. Section 3 develops the TCL demand response model. Section 4 adds the battery storage system. Section 5 presents the 24-h simulations based on California ISO data. Section 6 formulates and integrates the feedforward controller, and Section 7 summarizes the paper’s key conclusions.

2. Generation-Side Transactive Frequency Control

In the traditional automatic generation control, the grid frequency is mainly regulated via a control scheme called frequency droop control [7]. In this scheme, as shown in Figure 1, the frequency is measured and scaled by a negative feedback gain called the droop constant. The scaled signal is then fed into a low-pass filter to mitigate the feedback noise. The filtered signal is then used as the reference position for the generator’s input valve to control the power output.

Steam generators and grid frequency dynamics have been mainly modeled by simplified linear dynamics in the automatic generation control literature [7,8]. Under the frequency droop control scheme, multiple generators can be connected to the grid simultaneously in what is called a lumped area dynamic model. In a large grid, a selected number of generators use an additional integral controller to eliminate the error residuals.

2.1. Transactive Frequency Control

Frequency deviations in a power grid result from supply and demand imbalances. When the demand is larger than the supply, the generators’ rotational speed slows down, resulting in a grid-wide frequency drop from the nominal value. Contrarily, when the supply is larger, the generators speed up and the grid frequency increases.

Generally, in a large power grid, the cost of electricity generation increases with the increase in demand. This is mainly because more expensive generators have to be brought on to meet the added demand. In addition, the locational marginal prices of electricity could grow significantly larger in the event of transmission line congestion during peak hours. As the demand peaks, the generation cost and the market price of electricity peak as well. In decentralized power networks, the wholesale transactions are usually conducted in the day-ahead and the real-time markets [28]. In the day-ahead market, the participants commit to sell or buy wholesale electricity a day before the operation based on an approximate forecast of the demand. Additional adjustments are made in the real-time market based on an improved forecast. The time scale of the real-time market can range from a few minutes to up to an hour before the production. None of these markets address the frequency regulation, which requires a much faster balancing action, i.e., every 2–4 s. The frequency control loop is carried out based on the frequency feedback only, and the participants receive compensation based on a pre-set frequency regulation market contract.

The goal of transactive frequency control in this paper is to demonstrate the possibility of using fast economic transactions to regulate grid frequency. This requires a universal price signal to be generated and broadcasted to all participants every few seconds. When the price is high, the generators will ramp up their production to gain as much economic benefit as possible, and the demand-side resources would cut their consumption. This would result in a downward frequency trajectory. When the price is low, the price-responsive loads and energy storage systems would take the advantage and increase the demand while the generators will reduce their production, leading to an upward frequency drift. Therefore, the price can be tied to frequency through a continuous antisymmetric function as follows:

$$\begin{gathered} P{r}_{p.u.}\left(t\right)=f\left(Fre{q}_{p.u.}\left(t\right)\right) \\ f\left(0\right)=0 and f\left(x\right)\left|f\left(x\right)\right|<0 \forall x\ne 0 \end{gathered}$$

(1)

where $P{r}_{p.u.}$ and $Fre{q}_{p.u.}$ represent the normalized deviations of electricity price and grid frequency with respect to their nominal values in per unit (p.u.). For example, $Fre{q}_{p.u.}=\frac{\left(Freq-Fre{q}_{nom}\right)}{Fre{q}_{nom}}$.

There are many possibilities for function $f$, such as a line, a third-order polynomial, a tangent function (to explode the price), or even an arctangent or sigmoid function (to saturate the price). These functions may create different effects on the price/frequency dynamics, but they behave similarly near the nominal frequency. In this paper, we use a simple linear relation between the price and frequency, similar to Reference [10], as follows:

$$P{r}_{p.u.}\left(t\right) =-k_{pr} Fre{q}_{p.u.}\left(t\right)$$

(2)

where $k_{pr}$ is a constant positive gain to be tuned by trial and error or based on the conventional frequency droop controller.

2.2. Power Generation Response to Price Variation

As discussed earlier, the cost and price of electricity is an increasing function of power demand in large power grids with many generators connected. In small microgrids with one or more generators, we can assume or enforce a similar trend to achieve a stable system behavior. In this paper, we adopt a second-order polynomial for the power generation vs. price function:

$$P{g}_{p.u.}=aP{r}_{p.u.}^2+bP{r}_{p.u.}$$

(3)

where $P{g}_{p.u.}$ represents the generation output in p.u., and $a$ and $b$ are constants to be determined based on the generator’s operating costs. In practice, the price and the generation output will be subject to lower and upper bounds.

Figure 2 depicts the second-order polynomial used in this study to simulate generation behavior vs. the electricity price. This polynomial has been tuned in such a way that for 50% drop in price with respect to nominal, the power generation also drops by 50%, and for 100% increase in the price, the power generation increases by 50%. It is important to note that any lower or higher order polynomial can be used for this function as long as the closed-loop system maintains sufficient stability margins.

2.3. Closed-Loop System Analysis

To demonstrate the transactive frequency control scheme, we use the same steam plant and grid model in Figure 3. The frequency is converted to price using Equation (2) and fed into the generation control unit. The polynomial in Equation (3) converts the price to a reference valve position, which is then filtered and passed to the generator plant. The generator output will converge to the pre-filtered signal at steady-state since both transfer functions on its path have a DC gain of 1.

To analyze the stability of the closed-loop system with the nonlinear generation vs. price function, we can linearize it around a given frequency or price point (e.g., $P{r}_0$), and use the slope of the line as the effective gain of the nonlinear block. The intercept of the linearized function does not affect the closed-loop stability around the linearized point since it can be treated as an exogenous input. Figure 4 depicts the equivalent linearized system diagram.

The loop gain transfer function of the linearized system is given by:

$$L\left(s\right)=\frac{k_{pr}\left(2aP{r}_0+b\right)\left(F_{HP}{T}_{RH}s+1\right)}{\left(T_{G}s+1\right)\left(T_{CH}{T}_{RH}{s}^2+\left(T_{CH}+T_{RH}\right)s+1\right)\left(Ms+D\right)}$$

(4)

The characteristics equation of the closed-loop system (i.e., 1 + L(s) = 0) can then be derived as:

$$\begin{gathered} c_4{s}^4+c_3{s}^3+c_2{s}^2+c_{1}s+c_0=0 \\ \begin{array}{l}{c}_4= T_{CH} T_{RH}M T_G\\ c_3=T_{CH} T_{RH}\left(D T_G+M\right)+\left(T_{CH}+T_{RH}\right)M T_G\\ c_2=M{T}_G+T_{CH} T_{RH}D+\left(T_{CH}+T_{RH}\right)\left(M+D T_G\right)\\ c_1=M+\left(T_{CH}+T_{RH}+T_G\right)D+k_{pr}{F}_{HP}{T}_{RH}\left(2aP{r}_0+b\right)\\ c_0=D+k_{pr}\left(2aP{r}_0+b\right) \end{array} \end{gathered}$$

(5)

For a given set of parameter values, the roots of the characteristics equation can be evaluated numerically or using the Routh-Hurwitz criterion to check the stability of the system. Alternatively, one can investigate the stability margins of the closed-loop system using the Bode or Nyquist analysis of $L\left(s\right)$.

To demonstrate the stability of the system, we adopt the parameter values for the steam generator and the grid frequency dynamics from Reference [7]. To assign a reasonable value for the frequency-price gain, $k_{pr}$, we match the transactive and the traditional droop controllers at the nominal price of $P{r}_0$ = 0 p.u. This would result in the equivalent value of $k_{pr}=1/\left(bR\right)$.

Table 1. System parameters mainly adopted from Reference [7].

Parameter	Value	Parameter	Value
M	10.0 s	D	1.0
TRH	7.0 s	R	0.05
TCH	0.3 s	kpr	24
TG	0.2 s	a	−0.33
FHP	0.3	b	0.83

summarizes the parameter values adopted from Reference [7], as well as those obtained from the generation-price function, and the resulting equivalent $k_{pr}$ value.

To evaluate the stability of the transactive controller, Figure 5 demonstrates the Bode diagram of the linearized loop gain transfer function for the transactive controller in the price range of $P{r}_0$ @ [−1.0 1.0] p.u. As can be seen, in the worst-case scenario, when $P{r}_0$ = −1.0 p.u., the gain margin of the closed-loop system is 16.7 dB, which is far away from instability. The phase margin of the transactive controller is not affected by the price variation since there are no dynamic terms included in the frequency-price relation or the generation-price function.

2.4. Transactive Frequency Control Simulations

To provide a comparison between the conventional and the transactive frequency droop controllers, we apply a ±10% step change in the demand. Figure 6 presents the resulting responses. As can be seen, both controllers perform similarly in preventing the frequency to drift away from the nominal value. Compared to the no control scenario, where the frequency deviates by about 0.1 p.u., both controllers are able to maintain the frequency at around 0.005 p.u. at steady state, thanks to the droop constant value of 0.05.

The conventional controller remains symmetric with respect to the direction of the demand offset due to linearity of the system. However, the transactive controller exhibits a small amount of asymmetricity in the frequency and power output responses due to the adopted nonlinear generation-price function. The resulted electricity price signals from the transactive controller are shown in Figure 6c. Despite the observed differences, the transactive controller provides a fairly comparable response to the changes in the power demand. In addition, the transactive controller provides a useful signal, i.e., the electricity price that can be used to control demand-side resources such as energy storage devices and systems directly and autonomously. In the next section, we introduce the TCLs as a well-known resource for price-based demand-side energy management for frequency regulation.

3. Demand Response of Thermostatically Controlled Loads

One of the key advantages of the proposed transactive frequency control scheme is the ability to directly incentivize price-responsive loads. TCLs are great candidates to be integrated with a grid-wide transactive frequency control policy, thanks to their setpoint flexibility that provides a control over their thermal storage capacity. In this paper, we adopt a widely used TCL model for air conditioners (ACs) to demonstrate the synergistic integration of TCLs with the transactive frequency control scheme. We assume that the AC thermostats have access to the real-time electricity price, either through a communication network like internet or via local frequency measurement and conversion to price in accordance with a prior grid-wide agreement. We also assume that thermostats are allowed to move the setpoint temperature smoothly and continuously within a limited range specified by the user.

3.1. Aggregate AC Load Model

In this study, we use a first-order differential equation model to simulate the indoor temperature dynamics of TCLs as follows [14,28]:

$${\dot{T}}_i=\frac{1}{C_i{R}_i}\left(T_{\infty }-T_i-s_i\left(t\right)R_i{Q}_i\right), i=1,2, \dots , N_{AC}$$

(6)

where i is the load index, T is the indoor temperature, $T_{\infty }$ is the ambient temperature, C represents the thermal capacity and R is the thermal resistance of the building, Q is the energy transfer rate, and s represents the switching control logic imposed by the thermostat:

$$s_i\left(t\right)=\{\begin{array}{c}0, if s_i\left(t-\epsilon \right)=1 and T_i\left(t\right)\le T_{min,i} \\ 1, if s_i\left(t-\epsilon \right)=0 and T_i\left(t\right)\ge T_{max,i}\\ s_i\left(t-\epsilon \right), otherwise\end{array}$$

(7)

where ε is an infinitesimal time step, and T_min and T_max represent the thermostat’s lower and upper dead-band limits, respectively. These variables are related to the setpoint temperature (T_sp) and the thermostat’s dead-band range, δ_db.

$$T_{min,i}=T_{sp,i}-\frac{{\delta }_{db}}{2}, T_{max,i}=T_{sp,i}+\frac{{\delta }_{db}}{2}$$

(8)

The aggregate AC power can be calculated by adding the individual AC power demands adjusted by their respective coefficient of performance, ${\eta }_c$:

$$P_{Agg}\left(t\right)={\displaystyle \sum }_{i=1}^{N_{AC}}{s}_i\frac{Q_i}{{\eta }_{ci}}$$

(9)

In this study, we assume that each thermostat is able to offset the user-specified setpoint temperature up or down, continuously, within a limited range:

$$T_{sp,i}\left(t\right)=T_{sp0,i}+∆T_{sp,i}\left(t\right), \left|∆T_{sp,i}\left(t\right)\right| \le ∆T_{max,i}$$

(10)

where T_sp0 is the user-specified setpoint temperature, ΔT_sp is the temperature offset, and ΔT_max is the maximum allowed setpoint temperature deviation.

In this study, we used Simulink to simulate the aggregate AC system model, as shown in Figure 7. All the parameters and signals have been vectorized to facilitate the model development and simulation. The AC temperatures are initialized inside the integrator, and a custom relay function is developed based on Equation (7) to allow arbitrary initial values for the switching state, s. The parameters of the system are adopted from Reference [14] and given in

Table 2. System parameters adopted from Reference [14].

Parameter	Mean Value	Unit	Rel. Stand. Deviation
R, Thermal Resistance	2	°C/kW	0.1
C, Thermal Capacitance	10	kWh/°C	0.1
${\eta }_c$, Coefficient of Performance	$2.5$	-	0.0
δdb, Thermostat Deadband	0.5	°C	0.0
Q, Energy Transfer Rate	14	kW	0.1

. The table also contains the relative standard deviation of the parameters used to create a realistic population from normal (Gaussian) distributions.

To simulate the aggregate system response, a 24-h ambient temperature profile is created based on temperature data for the city of San Jose, California, in a warm summer day. The number of loads (N_AC) is set to be 5000 to create a smooth aggregate response. The setpoint temperatures are distributed normally with 20.0 °C mean value and 10% relative standard deviation. The system is simulated first with temperatures initially set to be the same as the initial ambient temperature. The simulation is then repeated by setting the initial temperature values same as the final temperature values from the first run. This would represent a more realistic simulation, as the initial temperatures are dependent on the historical ambient temperature trajectory and the user setpoints.

Figure 8 shows the simulation result, with the ambient temperature, aggregate AC power, and 5 sample indoor temperature trajectories with different setpoint values being plotted. As can be seen, the average AC power follows the ambient temperature with a small time delay. During peak time, the average AC power reaches to around 45% of the maximum potential power the ACs can demand (2.5 kW vs. 5.6 kW max power per AC unit). This is because of the fact that the loads turn on and off at different times due to different initial conditions and thermal properties, as seen from the sample indoor trajectories. From the individual temperature trajectories, we can see that the lower the setpoint temperature, the higher the AC activity.

3.2. AC Price Response Controller

The aggregate power demand response of TCLs to setpoint input is similar to that of a linear underdamped second-order system [29]. Both linear and nonlinear controllers have been used to achieve power tracking and frequency regulation [29,30,31,32,33]. Most of these controllers have a common element of using an integral controller. For frequency control, adding a proportional term may be necessary to deal with the added grid frequency dynamics. Therefore, we propose using a simple proportional-integral (PI) controller to adjust the AC setpoints using real-time price signal:

$$∆T_{sp}\left(t\right)=k_{p}P{r}_{p.u.}\left(t\right)+k_i{{\displaystyle \int }}_0^t{\mathrm{Proj}}_{∆T_{sp}}\left[P{r}_{p.u.}\left(\tau \right)\right]d\tau$$

(11)

where $k_p$ and $k_i$ are the proportional and integral gains, and ${\mathrm{Proj}}_{∆T_{sp}}$ is an integrator anti-windup operator defined as:

$${\mathrm{Proj}}_{∆T_{sp}}\left[P{r}_{p.u.}\right]=\{\begin{array}{l}0 if ∆T_{sp}\ge ∆T_{max} and P{r}_{p.u.}>0\\ 0 if ∆T_{sp}\le -∆T_{max} and P{r}_{p.u.}<0\\ P{r}_{p.u.} otherwise\end{array}$$

(12)

Based on the proposed PI controller, when the price (or frequency) rises above the nominal value, the setpoint temperature would increase, thereby decreasing the power demand in the cooling mode. During the cold season, when the ACs are used for heating, we can use the same PI controller with a flipped price signal sign to achieve the same balancing effect. Each AC may have different response sensitivity and setpoint flexibility range (i.e., different $k_p$, $k_i$, and $∆T_{max}$ values) set by the user. However, for an effective overall control performance, these parameters must be kept within a certain range. For the simulations of this study, we use the same PI controller and temperature limit for all the TCLs.

The block diagram of the integrated demand-side and generation-side transactive frequency control system is shown in Figure 9. The PI controller receives the price signal and determines the change in the setpoint temperature of the TCLs according to Equation (11). In practice, the PI controller computations can be done within each thermostat through sensing the frequency, conversion to price, and then adjustment of the setpoint temperature.

To evaluate the proposed transactive TCL control scheme, we first disconnect the steam plant from the simulation model and run the TCL-based control only. Then, we evaluate the integrated system. The ambient temperature is kept at the constant value of 30 °C during this simulation, and the controller’s proportional and integral gains are obtained by a trial and error process and set to 0.2 and 0.0002, respectively. Larger control gains may improve the system’s response time, but they could also trigger the internal dynamics of the aggregated system. To convert the aggregate AC power to the p.u. system, we divide it by the total number of ACs multiplied by the maximum power of each AC, i.e., 5000$\times$5.6 W = 28,000 W.

Figure 10 summarizes the TCL simulation for controlling the frequency when there is a ±10% step change in the load. We compare 4 different cases: No control; proportional (P) control only; integral (I) control only; and PI control. As can be seen, the 10% step load results in around 10% offset in the frequency in the absence of any control action. The integral controller is able to revive the frequency to a great extent, but very slowly and with a lot of steady-state fluctuations. These fluctuations mainly result from the randomness infused in the TCL population. The proportional controller acts quickly to stabilize the frequency at the beginning but drifts away over time. The PI controller provides the most stable performance by acting quickly through its proportional channel and maintaining the frequency at a steady level through its integrator.

From the resulting setpoint offset trajectories in Figure 10, we can see that there is a steady temperature drift over time after the initial transient period in all cases with active control. This drift would continue until reaching the limit set by the user unless the supply and demand imbalance is mitigated by other resources. This implies that the frequency control by TCLs during power imbalance periods is subject to capacity limits, as with any energy storage system. Similar to the previous simulations, there are asymmetricities in the frequency and temperature responses between the step-up/down scenarios because of the nonlinearity of the price-generation function.

To evaluate the combined effects of demand-side and generation-side frequency control, we compare three cases: TCL control only; generation-side control only; and the combined control. Figure 11 summarizes this comparison, indicating that the generation-side controller provides a much faster and a more steady response compared to the TCL demand response alone. The integrated system, however, provides an even faster response with a smaller frequency deviation, as expected. Besides, the TCL setpoint temperature drift is slowed down because of burden sharing between the two resources.

In the next section, we further extend the transactive frequency control study by adding a battery storage component to the system.

4. Energy Storage

Transactive frequency control provides a unique opportunity to engage batteries for active participation in frequency regulation. In this section, an equivalent-circuit battery cell model is used and scaled up to capture the main energy storage and transient dynamics of electrochemical batteries connected to the grid. A multi-level control algorithm is then developed to operate the battery within its safe constraints in response to the real-time electricity price signal.

4.1. Battery System Model

The battery system model adopted in this paper is based on a widely used equivalent circuit model shown in Figure 12 [34]. The system’s state equations are given by:

$$\{\begin{array}{c}\dot{SOC}\left(t\right)={\eta }_e\frac{1}{Q}I\left(t\right) \\ {\dot{V}}_d\left(t\right)=-\frac{1}{R_d{C}_d}{V}_d\left(t\right)+\frac{1}{C_d}I\left(t\right)\end{array}$$

(13)

where SOC stands for the battery state-of-charge, ${\eta }_e$ represents coulombic efficiency, $Q$ is the charge capacity, $I$ is the charging current, $V_d$ represents the diffusion voltage, and $R_d$ and $C_d$ are the diffusion resistance and capacitance, respectively. The battery terminal voltage is given by:

$$V\left(t\right)=V_{oc}\left(SOC\left(t\right)\right)+V_d\left(t\right)+R_{0}I\left(t\right)$$

(14)

where $V_{oc}$ is the SOC-dependent open-circuit voltage, and $R_0$ is the internal resistance of the battery.

The block diagram of the system is shown in Figure 13. A function block is used to create the nonlinear SOC-dependent open-circuit voltage relation. The battery power is calculated by multiplying the input current by the terminal voltage as shown in the diagram.

To scale the battery cell’s power to the grid level represented in p.u., we can divide the cell power by the maximum allowed cell power and multiply by the maximum grid-level power in p.u., as follows:

$$P_{b,p.u.}\left(t\right)= P_{cell}\left(t\right)\frac{P_{bmax,p.u.}}{P_{cell, max}}$$

(15)

where $P_{bmax,p.u.}$ represents the maximum battery power seen by the grid (e.g., 0.1 p.u.) and $P_{cell,max}$ stands for the maximum allowed cell power (e.g., 7.2 w).

It is important to note that there is round-trip efficiency loss in the battery due to electrical energy losses in resistors $R_0$ and $R_d$. Additional losses due to the power electronic circuits have been ignored in this study, but can be accounted for by increasing $R_0$ accordingly.

4.2. Transactive Battery Management System

The transactive battery power control is done in two steps: First, we obtain the desired amount of battery power based on the real-time electricity price signal. Then, we command it to the battery controller which delivers the desired power, as long as the battery power and SOC remain within specified limits.

A quadratic relation is proposed here to relate the desired battery power to the electricity price function as follows:

$$P_{bdes,p.u.}\left(t\right)=\{\begin{array}{c}-min\left\{\eta P{r}_{p.u.}^2\left(t\right),P_{bmax,p.u.}\right\}, P{r}_{p.u.}\left(t\right)\ge 0\\ min\left\{\eta P{r}_{p.u.}^2\left(t\right),P_{bmax,p.u.}\right\}, P{r}_{p.u.}\left(t\right)<0\end{array}$$

(16)

where $\eta$ is a constant positive parameter for tuning the battery system’s sensitivity to the price signal. Figure 14 demonstrates this function for the different values of $\eta$ in the range of 0.5 to 10. Based on this relation, the battery discharges to the grid when the price rises above the nominal value (i.e., 0 p.u.) and charges from the grid when the price drops below the nominal value. The battery charging/discharging power increases quadratically as the price deviates from zero until it hits the maximum limit, beyond which the desired power remains constant. There is a nearly flat region around the nominal price, where the battery’s activity is marginalized. The larger $\eta$, the narrower the flat region and the higher the sensitivity of the desired power to the price signal.

It is important to note that the price function can be replaced with other similar functions such as a linear relation subject to a dead-zone region or a third-order polynomial. It can be shown that the proposed function would not impact the stability of the grid’s frequency dynamics as long as it remains continuous and non-increasing with respect to price. It is also important to note that the nominal price is subject to changes due to significant energy price fluctuations over time. A slowly changing moving average of the electricity price may be a good representative of the nominal price.

Once the desired battery power is determined in p.u., it can be converted to cell-level power using the inverse of the conversion factor used in Equation (15):

$$P_{cell,des}\left(t\right)=P_{bdes,p.u.}\left(t\right) \frac{P_{cell, max}}{P_{bmax,p.u.}}$$

(17)

We can further impose the SOC constraints by limiting the charging and discharging power through a pair of scaling factors as follows:

$$P_{cell,cmd}\left(t\right)=\{\begin{array}{l}min\left\{P_{cell,des}\left(t\right), {\alpha }_c{P}_{cell, max} \right\} P_{cell,des}\left(t\right)\ge 0\\ max\left\{P_{cell,des}\left(t\right),-{\alpha }_d{P}_{cell, max} \right\} P_{cell,des}\left(t\right)<0\end{array}$$

(18)

where $P_{cell,cmd}$ is the final commanded cell power applied to the battery cell, and ${\alpha }_c$ and ${\alpha }_d$ are the SOC-dependent power constraint factors during charging and discharging, defined as:

$${\alpha }_c\left(SOC\right)=\{\begin{array}{l}0 SOC\ge SO{C}_{max} \\ \left(SO{C}_{max}-SOC\right)/\delta SO{C}_{max}-\delta <SOC<SO{C}_{max}\\ 1 SOC\le SO{C}_{max}-\delta \end{array}$$

(19)

$${\alpha }_d\left(SOC\right)=\{\begin{array}{l}0 SOC\ge SO{C}_{min} \\ \left(SOC-SO{C}_{min}\right)/\delta SO{C}_{min}<SOC<SO{C}_{min}+\delta \\ 1 SOC\ge SO{C}_{min}+\delta \end{array}$$

(20)

where $SO{C}_{min}$ and $SO{C}_{max}$ are the lower and upper SOC limits, respectively, and $\delta$ determines the SOC range within which the battery power slows down near the lower or upper limit. Figure 15 shows the shape of ${\alpha }_c$ and ${\alpha }_d$ for the admissible SOC range of 0.1–0.9 and $\delta$ value of 0.1.

Once the commanded cell power is calculated via Equations (18)–(20), the input current can be determined by dividing the power by the measured cell voltage.

The proposed logic ensures a smooth operation of the battery within desirable SOC and power limits. Additional safety measures can be taken in a similar fashion to protect the battery based on voltage and temperature limits if necessary.

4.3. Transactive Battery-Grid System Simulation

Figure 16 shows the block diagram of the transactive frequency control system with AGC, AC, and battery storage. The battery management block contains the algorithms and procedures discussed in the previous section. It receives the electricity price, battery SOC, and battery voltage as inputs, and outputs the commanded cell current. To avoid algebraic loop in the simulation, a one-time-step delay is applied in the voltage feedback path. For the simulations of this study, the parameters and the open-circuit voltage curve of the battery model are adopted from Reference [34]. The maximum battery power is set to be 0.1 p.u., and the maximum cell current is set to ±1C, corresponding to full charging and full discharging within 1 h. This would create the storage capacity that can address nearly 8% of the grid’s demand for about 1 h if the batteries are fully charged. Note that the battery system will always have an unused 20% capacity (approximately, 2% of the grid load) due to the imposed SOC constraints.

The first simulation of this section looks into the grid’s frequency when only the battery is responsible for the grid’s frequency regulation. The generator is disconnected from the grid, and the AC system’s demand response controller is turned off, making it a passive load on the grid. The battery’s initial SOC for this simulation is set to 0.5. A 10% step load is applied after 5 minutes. The simulation results are shown in Figure 17 for the different values of parameter $\eta$. As can be seen, the battery is able to maintain the grid frequency near the nominal value as long as the SOC is above 0.2. Once the SOC drops in the δ range, i.e., below 0.2, after about 20 minutes, the battery slows down discharging, and the frequency is dropped to near the uncontrolled level of -0.1 p.u. when SOC reaches the lower limit of 0.1. The larger the value of $\eta$, the more aggressive the battery discharging, and the higher the initial frequency control performance. However, the battery provides a shorter response due to capacity limitation. In practice, this gain must be tuned to provide the maximum economic return from the daily price fluctuations on the grid.

The second simulation of this section explores the integrated system when all the components discussed previously are actively responding to the electricity price signal. Figure 18 compares the three cases of AGC only, AGC plus AC demand response, and all the three systems together, for the case of 10% step increase in the load. As can be seen, the addition of battery provides a significant improvement to the grid’s transient response. This is mainly due to the battery’s instantaneous response to price variation, which prevents the frequency from dipping significantly. There is a small improvement in the steady-state response of the system as well. However, the battery’s share of steady-state power contribution is only about 15%, which is the smallest among the three systems for this particular simulation.

To further evaluate the overall system’s performance under real loading conditions, we study a set of 24-h period simulations with real power demand and renewable energy data.

5. Daily Grid Simulations

In this section, we use the California Independent System Operator (CAISO) data to obtain sample grid demand and renewable power profiles for a full daily cycle simulation. The data for the same day as that of the ambient temperature shown in Figure 8 is used for consistency. To separate the aggregate AC power from the total grid demand, we first divide the grid demand by its maximum value to convert it in the p.u. scale, and then subtract the open-loop aggregate AC power to get the base demand trajectory. Figure 19 shows the different power trajectories including the renewable power for a 30-hour time period.

To start the simulation from an equilibrium condition and near the nominal price, we set the simulation start time to 2 a.m., where the net demand (grid demand minus renewable power) is near its mid-point. The battery control parameter $\eta$ is tuned in such a way that the battery’s main charging and discharging events occur during price valley and peak periods. To achieve this goal, we use two different values for $\eta$, one for charging (${\eta }_{charge}$ = 0.85) and another for discharging (${\eta }_{discharge}$ = 0.45), obtained based on trial and error for this particular simulation. The AC demand response controller’s parameters are kept the same as before ($k_p$ = 0.2 and $k_i$ = 0.0002) since they result in a reasonable setpoint temperature variation, i.e., around 1 °C during the simulation period.

We perform the simulation for three scenarios: Controlling the grid frequency with transactive controller using (i) AGC only, (ii) AGC with AC demand response, and (iii) AGC with AC and battery demand response. Figure 20 provides a comparison among the three scenarios. We observe that adding AC demand response significantly reduces the frequency deviation, peak demand, and peak price. The AC demand is shifted to the left to avoid the peak price period. This is an effect known as pre-cooling, which is produced by the proposed transactive control framework, automatically. Adding the battery further reduces the peak demand. However, its contribution is smaller than AC demand response due to the small storage capacity considered in the simulation. The battery also fills the demand valley, making the overall load profile more uniform. The net change in frequency is 0.0207 p.u. for the case of AGC only, 0.0148 p.u. for the case of AGC plus AC, and 0.0134 when all three systems are active. Therefore, the total frequency deviation improves by 29% through the AC demand response, and by another 7% when the battery is included. Similar improvement percentages are obtained for the price and net demand trajectories.

To further demonstrate the AC system’s demand response, Figure 21 shows the setpoint offset for the 3 different scenarios. As can be seen, the net setpoint temperature change for the duration of the simulation is around 1 °C, which indicates a small but manageable compromise in the building occupants’ comfort as a result of the demand response operation. In return, the average AC energy cost is reduced by 13% as also shown in Figure 21. The addition of the battery further reduces the energy cost by 0.5% for the ACs.

The battery power, SOC, and economic profit are shown in Figure 22. The peak charging power coincides with the net demand valley between 8–9 a.m., and the peak discharging power coincides with the net demand peak as expected. The maximum charging power is around 0.02 p.u., which corresponds to 0.2 C, and the discharging power is around 0.15 C. The SOC trajectory remains within the 10–90% range as a result of active constraint control scheme developed in the previous section. The net 24-h profit is around 2 cents per kWh of energy storage capacity. This amount can be larger or smaller depending on the daily price variation. In practice, the price variations could be significantly larger than that in this simulation during certain high-demand periods. As discussed in the previous section, battery can also mitigate transient frequency fluctuations during contingencies.

The simulations of this section indicate the effectiveness of the proposed transactive frequency control and demand response algorithms. One of the shortfalls of this method is the considerable frequency deviation from the nominal value during the day, which is the main driving factor for creating the price differential required for the transactive controller and demand response operations to work. In the next section, we propose a modified control strategy that can resolve the frequency deviation problem while maintaining the necessary price differential factor.

6. Feedforward Price Broadcast for Frequency Error Suppression

To attenuate frequency deviation despite the variation of electricity price, the grid operator can broadcast a feedforward price signal based on the measured instantaneous demand, and the electricity cost function parameters provided by the generator. The feedforward price signal is a function of the grid’s net demand and is calculated as:

$$P{r}_{ff,p.u.}=\frac{-b+\sqrt{b^2+4a{D}_{net,p.u.}}}{2a}$$

(21)

where $a$ and $b$ are the price function coefficients from Equation (3), and $D_{net,p.u.}$ is the net demand measured by the grid. The feedforward price can be fed into the control loop as follows:

$$P{r}_{p.u.}\left(t\right) =P{r}_{ff,p.u.}\left(t\right)-k_{pr} Fre{q}_{p.u.}\left(t\right)$$

(22)

where $k_{pr}$ is the same feedback gain used previously in Equation (2). This modification will significantly decouple frequency from the price, thereby allowing the price to move up and down freely while the frequency remains near its nominal point.

The impacts of the feedforward price broadcast addition are shown in Figure 23 for the case of having all three systems (i.e., AGC, AC, and battery) actively responding to the price signal. As can be seen, the frequency variation drops to near 0, while all the other trajectories remain nearly the same as before. This simulation proves the importance of the feedforward price broadcast if the frequency deviation has to be maintained within a narrow range.

Since the feedforward price is dependent on the net grid demand, there may be a measurement and communication time delay. Besides, the price signal may be broadcast at much slower intervals than the frequency control loop. Therefore, it is reasonable to investigate the impacts of such time delays on the grid frequency. Figure 24 compares the three levels of broadcast delays on the frequency: No time delay; 1-min demand averaging and broadcast intervals; and 5-min intervals. The time delay involved in averaging and broadcast is clearly creating chatter in the frequency signal. The longer the delay, the larger is the chatter. Nonetheless, the maximum frequency deviation remains within a small range, i.e., around 0.05% and 0.2% with 1 min and 5 min broadcast intervals, respectively. Without any feedforward price information, the maximum frequency deviation is around 0.7% as depicted in Figure 23.

7. Conclusions

In this paper, a transactive controller was proposed to simultaneously manage grid frequency and demand response resources via a universal electricity price signal. The price can be related to grid frequency and measured locally by the generators and loads, or broadcast via a communication infrastructure to all entities. A frequency droop controller was adopted from the conventional automatic generation control framework to build a simulation model. To incorporate a demand response capacity, a population of thermostatically controlled air conditioning (AC) loads was added to the grid. The loads were assumed to have flexible setpoint temperatures that can be adjusted continuously based on the price signal. A PI controller was used to control the AC system’s response. To capture energy arbitrage dynamics on the grid, an equivalent-circuit battery model was adopted. A nonlinear control scheme was developed for the battery to provide appropriate charging and discharging cycles in accordance with the price signal.

Simulation results indicate that the transactive frequency controller can lead to similar results to the conventional frequency droop controller. Besides, it can directly invite the AC loads and energy storage resources to actively participate in the power balancing effort. The AC demand response reduces the steady-state frequency deviation, and the battery system improves the transient response in the event of contingencies. During the daily operation, the ACs reduce the peak demand and the battery fills the demand valley, thereby creating a more uniform load profile. To minimize frequency deviation, a feedforward price broadcast strategy was implemented. The feedforward price can significantly reduce the frequency deviations, particularly when it is computed and broadcast promptly, leading to a transactive control system suitable for modern power grids.