The Impact of Battery Storage on Power Flow and Economy in an Automated Transactive Energy Market

Abstract

This article explores the use of battery energy storage in a transactive energy approach for a heavily solar-penetrated community. We hypothesize that the efficient market interactions between independently acting, fully automated agents (some equipped with battery energy storage) can result in both bill savings and improvements in power flow, without explicitly optimizing for power flow ahead of time. A test setting of nine typical residential prosumers and one heavily loaded prosumer is used. The heavily loaded prosumer initially experiences multiple undervoltage violations, and a 13.5 kWh battery is installed to alleviate the problem. Two profile-shaping strategies are compared. The first scenario uses greedy control that maximizes self-sufficiency, while the second scenario uses a local market to enable energy trading between participants and a rule-based trading and management agent for control. The results show that the first scenario made minimal improvements to power flow, but the second scenario eliminated all occurrences of under-voltage violations. Furthermore, the total amount of energy from the grid is reduced by 24.3%, and the amount of energy injected into the grid is reduced by 39.2%. This results in lower bills for every participant and a bill reduction of 16.7% for the community as a whole.

1. Introduction

Technological breakthroughs in renewable power generation, battery storage, electric mobility, and advanced data logistics are changing the way the electric grid is designed and operated. A large contributor to this change will be the large amount of distributed energy resources (DERs) expected to be installed in the near future. These DERs, which include rooftop solar, battery storage, and electric vehicles, are more intermittent and less predictable in nature. This makes it difficult for electric grid utility companies to balance generation with loads at the right time. If left unchecked, this could lead to a lower quality of service. Phenomena, such as brownouts and blackouts, voltage fluctuations, reverse power flow, and degraded power quality [1,2,3], may increase in frequency.

Overcoming these challenges requires finding more sophisticated device and energy management strategies. Transactive energy (TE) is one popular framework [4,5] that promises to achieve flexible, robust, and adaptive energy management systems geared towards integrating DERs into the existing grid. The GridWise Architecture Council (GWAC) defines TE as “a set of economic and control mechanisms that allows the dynamic balance of supply and demand across the entire electrical infrastructure using value as a key operational parameter” [6]. This high-level definition provides some clues as to how TE should work but leaves the specific implementation details to system designers.

In our previous work [7], we examined aspects of an economy-based TE system designed to use a group of independent reinforcement learning agents to manage DERs in a community. DERs within the community are coordinated through trading in a local energy market based on double auctions (see Appendix A for details). We concluded that, given the right market design, the proposed TE system can produce responsive and relevant prices. This would, in turn, translate into bill savings for some participants. However, participants did not have profile shaping capabilities; therefore, additional benefits were left untapped. To the best of our knowledge, there has been no previous contribution that studies the relationship between efficient market interactions by independently acting, fully automated agents equipped with battery storage, and the resulting effects on power flow. This article fills this research gap and explores the effects of battery-storage-based profile shaping in a similar setting as our previous work. We hypothesize that, in addition to bill savings, improvements in power flow can be observed as a result of efficient market interactions between participants.

This article is organized in five sections. Section 2 describes the necessary background information and provides a review of related work, focusing on topics related to transactive energy and automated demand response. Section 3 describes the experiments carried out to study the effects of battery energy storage in a local energy market. Specifically, it compares the resulting power flow and end-customer bills of two battery-based profile-shaping strategies. The main conclusions are summarized in Section 5, along with possible directions for future work.

2. Background and Related Work

The addition of DERs, especially at the edge of the grid, can lead to decreased flexibility, resiliency, and quality of service of the electric grid. To combat these problems, demand response techniques have been used to shape loads at end-customer sites that are more closely aligned with the generation profiles. Demand response (DR) techniques can be divided into two categories: direct and indirect. Direct DR refers to methods in which the electric utility company is given direct control over heavy-loading appliances, such as HVACs, to increase or decrease the load on the grid as needed. Direct DR is limited in scope due to the small variety of controllable appliances available and the large infrastructure overhead [4,5,8]. It is also considered more invasive and tends to create more friction between the utility company and the customers. Indirect DR, on the other hand, uses incentive signals to influence customer behavior in the hope that enough customers will respond to make a difference. Popular methods include time-of-use (TOU) pricing, critical peak pricing, and dynamic pricing (DP) [8]. A survey by Chen et al. [4] concludes that customers generally prioritize comfort over responding to price signals, making indirect DR largely ineffective.

Given these challenges and limitations, transactive energy has been gaining interest as a modern DR framework that flexibly offers a mixture of direct and indirect DR. There is a large body of research on TE in the form of transactive control (TC), which uses the TE framework to generate a combination of direct and indirect control signals to achieve optimal power flow (OPF). Typically, transactive control schedules are calculated via a utility function in conjunction with forecasts and expert-designed behavioral models [4,5,8]. Some notable TC approaches include the following.

• Works by Hu et al. [9], who proposed an aggregator-based optimization approach that generates charging/discharging schedules for electric vehicles (EVs). A simulation was performed on a Danish distribution network to show the decrease in the frequency of line congestion and voltage violation.
• Nazir et al. [10] used an aggregator-based model incorporated into a model predictive control (MPC) framework to calculate optimal price signals and to control thermostatically controlled loads (TCLs) and storage devices to decrease power oscillations at substation feeders.
• Soarez et al. [11] introduced a comparable aggregator-based approach using a dual decomposition algorithm. To validate the efficacy of the algorithm, they performed a field test involving six houses. Unfortunately, the results were inconclusive due to the lack of flexibility at the test site.

A key takeaway of the research performed by Soarez et al. is that TC may not be flexible enough to dynamically adapt to changes in real time. Fundamentally, OPF-based TC is not much different from contemporary DR techniques. Therefore, it still suffers from the same issues [4,8]. In an effort to solve these issues, research on TE approaches that do not use OPF as a primary objective has been ongoing. A promising approach is economy-based TE. Unlike TC, economy-based TE does not aim to create schedules that must be strictly adhered to in order to achieve OPF. However, improvements in power flow can be observed as a result of efficient market interactions between participants. Some notable research in this area includes the following.

• Zhou et al. [12] used a system that combines fuzzy rules and Q-learning to train agents exchanging energy in a peer-to-peer energy market that directly ties prices to the supply/demand ratio. They investigated several community configurations with different battery energy storage and renewable generation assets and evaluated their performance. This system uses an aggregator-based method to predetermine local energy prices in a centralized energy pool, and agents are trained to only control battery energy storage in reaction to these price signals. While lower bills are shown against TOU and net-billing, the authors do not show the effects of battery control on power flow. In contrast, our work presented in this article will provide a more holistic view with perspectives on both economic benefits and effects on power flow.
• Zhang et al. [13] trained a reinforcement learning agent to manage a community-shared battery. The battery is allowed to trade on a peer-to-peer TE market, with the goal of maximizing overall economic performance. The authors show that positive economic benefits can be achieved, even when considering the running costs of the battery. This system designates the battery as both the market and the only peer that participants can trade with. In contrast, our work is more representative of a real-world economy by allowing all participants to trade with each other. Energy storage is an optional add-on to any participant for energy profile shaping.
• In our previous work [7], experiments were performed to test the effects of different properties of the double auction market on the development of reinforcement learning policies. This study was performed using Automated Local Energy Exchange (ALEX, see Appendix B for an overview). The results showed that given rational agents, the market must be truthful and budget balancing must be weakened to develop policies that reflect supply and demand. We also showed that using the market as a DER coordination mechanism can lower bills compared to TOU and net billing (Appendix C).

The content of this article uses the information presented in our previous work [7], and we recommend reading it to gain the relevant context.

3. The Effects of Battery Storage in a Local Energy Market

Although the relevant research presented in Section 2 shows promising economic gains achievable through economy-based TE, there is a lack of understanding of its effects on power flow. The experiments described in this contribution are designed to evaluate the effects of including battery energy storage in a local energy market on both power flow and bills.

3.1. Experimental Design

This experiment requires power flow simulations to demonstrate the possible effects of profile shaping. Since our application of profile shaping is focused on the grid edge, the CIGRE North America low voltage residential benchmark circuit [14,15] (shown in Figure 1) is a suitable circuit to perform this study on. Due to its previous inavailability, we created a circuit model of this circuit for OpenDSS and made it available on GitHub [16]. Simulations were performed and evaluated against the results published in the CIGRE report [14] for validation, which produced node voltages within 2% of the reported values. As node voltages are the primary metric used for evaluation, this model is considered sufficiently accurate for the purposes of this study.

3.2. Use Case Scenarios

The energy profiles used are selected from the Smart* SunDance Dataset [17,18]. To somewhat ensure location relevance, the selection is limited to the Denver time zone. Monte Carlo search is used to create approximately 500 combinations of energy profile-to-node mappings, each followed by a 24-h time series power flow simulation to produce the same number of time series node voltage plots. The node voltage plots are then inspected both programmatically and visually to establish the baseline configuration (B1). The ideal outcome should have a single participant who causes consistent voltage violations, and its surrounding neighbors should have excess generation to spare. The set of profile IDs for the corresponding node locations that meet this criterion is shown in

Table 1. Profile IDs and corresponding node locations.

Node	Profile ID
R5	1001230
R6	1001974
R7	100500
R8	10068
R9	10011
R10	10063
R11	100124
R12	100502
R13	1003842
R14	1002705

. The energy profiles and the resulting node voltage plots are shown in Figure 2.

In this case study, R10 has been assigned the energy profile with the largest load (peaking at over 15 kW). As expected, this causes multiple occurrences of under-voltage violations throughout. Consequently, it is the location where the BESS should be deployed. The BESS used are modeled after a typical Tesla Powerwall 2, with a usable capacity of 13.5 kWh and a continuous charge/discharge rating of 5 kW. Profile-shaping strategies can then be deployed and examined. In this study, two strategies are implemented to examine the effects of enabling DER coordination through a local energy market. The strategies are described as follows:

B2:

Attach a BESS to R10. Use greedy control with no local market access.

B3:

Attach a BESS to R10. Use a rule-based controller with access to the local market.

B2 is designed to emulate contemporary control strategies that focus on self-sufficiency. Generally, this involves charging the battery (if there is residual generation) and discharging (if there is residual load). Charge/discharge times may be modified by a schedule if time-of-use or dynamic pricing is in play. In general, the success of this strategy relies on the availability of an oversized solar generation system and is often infeasible due to a variety of constraints, such as cost, regulatory, roof area, etc.

The purpose of B3 is to create the simplest scenario that can demonstrate the effects of maximizing the utilization of local DERs. The use of a single profile-shaping capable participant with a rule-based agent makes the outcomes for this participant deterministic. The rules are described by the pseudocode shown in Algorithm 1. The Python implementation used is available in the project repository [19] as basic_trader.py.

Algorithm 1 Energy Trading and Management Algorithm for B3

for each market round do     for last settled round do         adjust BESS schedule based on successful settlements     end for     if residual load > 0 then         if BESS is available then            if next_settle is between 8AM to 4PM (inclusive) then                Battery is allowed to charge                Submit bid where,                quantity = residual load + max charge capacity                price = $0.14/kWh            else                Battery is allowed to discharge                Submit ask where,                quantity = max(0, max discharge capacity − residual load)                price = $0.07/kWh            end if         else            Submit bid where,            quantity = residual load            price = $0.14/kWh         end if     else         if residual generation > 0 then            if BESS is available then                if next_settle is between 7AM to 3PM (inclusive) then                    Battery is allowed to charge                    Submit bid where,                    quantity = max(0, max charge capacity − residual generation)                    price = $0.14/kWh                else                    Battery is allowed to discharge                    Submit ask where,                    quantity = max discharge capacity + residual generation                    price = $0.07/kWh                end if            else                Submit ask where,                quantity = residual generation                price = $0.07/kWh            end if         end if     end if end for

3.3. Evaluation of Economic Performance

This algorithm is designed to maximize market interactions using extreme bid and ask prices. Therefore, it does not reflect the equilibrium prices. However, Equation (1) from our previous work [7] can be applied after simulation to approximate equilibrium prices at each time step based on total supply and demand, which can then be used to calculate and compare the economic performance between B2 and B3. This equation is reproduced below for convenience.

$$P(s,d)=\left\{\begin{array}{c}{P}_{\mathrm{NB},\mathrm{load}}^{\mathrm{grid}}\mathrm{if}P(s,d)\ge P_{\mathrm{NB},\mathrm{buy}}^{\mathrm{grid}}\hfill \\ P_{\mathrm{NB},\mathrm{gen}}^{\mathrm{grid}}\mathrm{if}P(s,d)\le P_{\mathrm{NB}}^{\mathrm{grid}},\mathrm{sell}\hfill \\ -0.0254\frac{s}{d}+0.1426C_{H,M,L}^{\mathrm{TOU}}\mathrm{if}\mathrm{buying}\hfill \\ -0.0280\frac{s}{d}+0.1299C_{H,M,L}^{\mathrm{TOU}}\mathrm{if}\mathrm{selling}\hfill \end{array}\right.$$

(1)

where s is energy supply, d is energy demand, $P_{\mathrm{NB},\mathrm{load}}^{\mathrm{grid}}$ is the price of electricity when buying from the grid with net billing, $P_{\mathrm{NB},\mathrm{gen}}^{\mathrm{grid}}$ is the price of electricity when selling to the grid with net billing, and $C_{H,M,L}^{\mathrm{TOU}}$ are the adjustment factors when TOU is used.

4. Results and Discussion

Figure 2 shows the energy profiles and node voltages for the baseline. The experiment takes place over three summer days (1–3 June 2015) to show the effects of leftover energy from the previous day(s). The top plot shows the hourly net loads of all participants, where positive means net consumption, and negative means net generation. The bottom plot shows the node voltages, with the major y-axes highlighting the ± 3% and ± 5% thresholds. With the exception of R10, the other profiles show behaviors that are representative of residential prosumers. Consequently, R10 experiences four under-voltage violations throughout the experiment (hours 18, 41, 42, and 66), which can be corrected using a battery storage system and profile shaping strategies.

Figure 3 shows the energy profiles and node voltages for applying strategy B2 on participant R10. Upon first inspection, the difference compared to the baseline is minuscule. This should be expected, as R10 only has short windows where net generation is available for storage (hours 12, 60 to 63, peaking at approximately 2.5 kW), which is not enough to make this strategy viable. As a result, the loads at hours 13 and 64 decrease, but not enough to eliminate voltage violations. Although it may be possible to optimize this strategy by predicting load spikes, predicting these spikes with a high degree of accuracy is difficult in the real world and is one of the contributors to the general failure of schedule-based transactive control [11].

Figure 4 shows the energy profiles and node voltages for applying strategy B3 to participant R10, where local energy trading is enabled. The local energy market allows R10 to buy and store excess generation from its neighbors and bypass some of the aforementioned constraints regarding the purely self-sufficient strategy. This results in clear and drastic improvements compared to B1 and B2, the most obvious being the elimination of all voltage violations.

Comparing the state of charge (SoC) of the battery between B2 and B3 (Figure 5) further shows the improved utilization available through local energy trading. B3 manages to almost always fully charge the battery each day, whereas B2 is unable to utilize the full capacity due to R10’s lack of residual generation.

As mentioned above, the rule-based energy trading algorithm is designed to maximize market interactions, and therefore prices do not accurately reflect the observed supply and demand as a learning agent would. At the time of writing, learning agents are still being investigated and developed by colleagues. In lieu of learning agents, Equation (1) can be applied post-simulation to approximate equilibrium prices at each time step based on total supply and demand. The resulting price curve over the simulation is shown in Figure 6. The economic performance of all participants under B3 is recalculated by applying this price curve for local energy transactions, then compared to B2. The results of this comparison are summarized in

Table 2. Comparisons of key metrics between B2 and B3.

	Net Profit ($)			Energy (kWh) Bought from Grid			Energy (kWh) Bought Locally		Energy (kWh) Sold to Grid			Energy (kWh) Sold Locally
Participant	B2	B3	%	B2	B3	%	B2	B3	B2	B3	%	B2	B3
R5	4.68	6.17	24.15	25.65	25.40	−0.99	0	0.25	121.77	70.81	−41.85	0	50.96
R6	−19.48	−18.10	7.62	134.51	91.88	−31.70	0	42.63	0.21	0.12	−45.12	0	0.10
R7	4.98	6.15	19.02	11.14	11.04	−0.95	0	0.11	95.63	58.15	−39.19	0	37.48
R8	0.45	1.08	58.33	12.96	10.47	−19.19	0	2.49	33.80	18.44	−45.44	0	15.36
R9	0.99	1.28	22.66	9.60	9.45	−1.59	0	0.15	34.57	24.78	−28.31	0	9.79
R10	−51.04	−47.44	7.59	352.23	245.77	−30.23	0	106.47	0.00	0.00	-	0	0.00
R11	−2.62	−2.48	5.65	19.31	14.04	−27.31	0	5.27	2.60	2.26	−13.12	0	0.34
R12	−0.46	0.32	243.75	30.59	28.90	−5.52	0	1.69	57.61	29.21	−49.30	0	28.40
R13	−10.05	−8.84	13.69	106.91	94.01	−12.07	0	12.90	78.82	40.75	−48.30	0	38.07
R14	−4.68	−4.29	9.09	52.89	41.07	−22.35	0	11.82	43.22	39.93	−7.63	0	3.30
Total	−77.21	−66.14	16.74	755.81	572.02	−24.32	0	183.78	468.23	284.45	−39.25	0	183.78

.

The results in Table 2 show that the combination of BESS and the local energy market provides an advantage to the community by socializing the benefits of DERs. When local energy trading is enabled, the amount of energy from the grid is reduced by $24.3\%$, and the amount of energy injected into the grid is reduced by $39.2\%$. As a result of using more locally available energy, the overall cost of the bill decreases by $16.7\%$. In addition to end-customers, the electric utility company may also benefit from this setup. Correcting the under-voltage violation can involve the addition or upgrade of equipment. A battery installed at the affected site may be a less expensive and more effective option under the right conditions.

5. Conclusions and Future Work

The transition to renewable energy leads to the deployment of more distributed energy resources. As the energy mix becomes more complex, automated management and coordination is required to maintain the quality of service expected of the electrical grid. This article continues to explore the economy-based transactive energy approach and focuses on the effects of battery energy storage on both power flow and end-customer bills. A heavily solar-penetrated community, consisting of nine typical residential prosumers (with peak net loads around 1 kW and 5 kW) and one heavily loaded prosumer (R10, with peak loads around 15 kW), who experiences multiple under-voltage violations, is used as the test setting. A 13.5 kWh battery is installed in R10 to test and compare profile-shaping strategies. The first case (B2) uses a common greedy control strategy that maximizes self-sufficiency. The second strategy (B3) uses a local market that allows energy trading between participants and uses a rule-based energy-trading and management agent for control.

In scenario B2, because R10 does not have enough excess generation to be stored, the battery was unable to be fully utilized. The impact on power flow is minimal, and voltage violations still exist. However, because scenario B3 uses the local market as a DER coordination mechanism, R10 is able to access the energy generated by the surrounding participants. The full capacity of the battery can be used, and peak loads are reduced by up to the maximum allowable discharge rate (5kW). As a result, all under-voltage violations are eliminated. Improvements in power flow are observed as a result of efficient market interactions between participants.

The combination of battery storage and the local energy market is also shown to socialize the benefits of DERs. Compared to B2, B3 reduces both the total amount of energy from the grid by $24.3\%$ and the amount of energy injected into the grid by $39.2\%$. From this, every participant receives lower bills. The community as a whole achieves a reduction in bills by $16.7\%$. In addition to end-customers, the electric utility company may also see benefits. Correcting the under-voltage violation may involve expensive equipment additions or upgrades. A battery installed at the affected site may be a less expensive and more effective option.

The findings presented in this article lay the crucial foundation for future work. We plan to extend this study by investigating a larger variety of participant compositions, as well as using more sophisticated reinforcement learning agents that can learn equilibrium prices autonomously. The addition of these learning agents makes the overall system stochastic and will require an extensive amount of case studies to be performed. Statistical analysis of the stochastic behaviors of both individual agents and the overall system will be conducted.