Special Issue “Innovation, Policy, and Regulation in Electricity Markets”

The rise of intermittent renewable energy generation, the coming mass penetration of electric vehicles and moves to decarbonise the gas grid are leading to widespread innovation experiments within electricity systems and their associated markets. These innovative experiments give rise to policy and regulatory questions, which must be addressed if innovations are to become business as usual within the lower voltage electricity distribution grid.

We invited colleagues to submit papers to this Special Issue on the lessons from electricity innovation experiments across the world and what implications they have for policy and regulation. We have eleven contributions referring to the latest developments in Australia, Europe, Ghana, Japan, and the US.

Many of our papers focus on the key issue of how to manage the electricity distribution grid more flexibly in the light of the move away from fossil fuel based electricity production to intermittent renewable energy from wind and solar. We briefly review each of the papers in the Special Issue in this editorial.

We begin with a couple of linked papers looking at electricity innovation experiments and how they are regulated.

Anaya and Pollitt [1] present a comparison of 13 experiments in the use of smart platforms to trade flexibility services within seven jurisdictions. The platforms are operated by distribution system operators (DSOs) or by independent parties. The seven jurisdictions are Australia, France, Germany, Great Britain, Japan, The Netherlands, and Norway. All have been implemented recently (from 2017 onwards). In each case, the architecture of the smart solution is discussed with respect to market design, price formation, and procurement methods. The used cases illustrate the importance of the user interface, clear specification of the service being acquired, the need for proper valuation of flexibility, the need for more experimentation around competitive auction design, and the key role of regulation in overseeing that the final customers are benefiting from competitive procurement.

Anaya and Pollitt [2] discuss the theme of regulation of flexibility service procurement in their second paper. Looking at the seven jurisdictions in their first paper they survey distribution utilities, energy regulators, energy marketplaces, energy associations, and relevant experts to identify what the top regulatory issues that need to be addressed are in these countries. The main conclusion—across the range of stakeholders—is that better regulation can promote more use of flexibility solutions by distribution system operators to solve constraints in their grids. However, how widespread the benefits of competitive procurement are remains unproven, given the small scale of such markets. Peer to peer (P2P) trading of electricity, with a view to reducing grid constraints, is not considered a near term possibility. While dynamic distribution tariffs have been considered, DSOs and their regulators remain reluctant to implement them. Coordination between transmission and distribution system operators (TSOs and DSOs) is considered important, but can also be a source of tension between grid operators (e.g., in Australia). The paper observes that a common cost benefit methodology for evaluating the benefits of flexible solutions is required in many of the jurisdictions.

Next, we turn to a paper that puts the current wave of electricity innovation in its historical context in Europe.

Schittekatte et al. [3] identifies four waves of new entrants that pushed innovation forward in electricity markets. The authors also discuss why and how market rules in Europe have evolved with each wave. The first wave they identify is the entry of national utilities in each other’s markets, the second is the entry of utility-scale renewable project developers in electricity markets, the third is the entry of asset-light software companies that often became independent aggregators, and the fourth is the entry of communities as a business model and communities as social movements and citizen cooperatives. The authors conclude that new players tend to improve the sustainability of the electricity sector in environmental, social, or economic terms but might also present new risks that require intervention by regulators. Regulators in Europe are aware of this as dynamic regulation is high on their agenda.

We move on to the United States with our fourth paper. Here, the focus is on an experiment with a truly distributed market-based solution to the issue of flexibility.

Arlt et al. [4] discusses an important field experiment around transactive energy. Transactive energy is where individual devices bid to consume more or less according to the market price, aka: ‘pricing to devices’. This has an important application when it comes to flexible loads such as air conditioning, heating, and EVs. These specific loads can be set up to reveal their bid schedule to market and hence be turned up or down according to the condition of the grid. This source of deeper flexibility could be very valuable in an electricity system characterised by intermittent renewable sources of generation. The paper describes a trial in progress in a co-operative utility in Colorado using the Transactive Energy Service System (TESS) software where this sort of load management was undertaken. The authors show that while transactive energy has potential in this context there are significant barriers to be overcome, most notably the need for transactive energy prices to sit alongside existing tariffs—which can be highly distortionary—and the fact that the value of offering response to the participants is difficult to predict in advance.

A distinguishing feature of electric systems with distributed energy resources and digitization is that they are more decentralized than previous grid architectures, both requiring and making possible a different approach to coordination of demand and supply. This is the topic of our fifth paper, which features a German case study.

Zeiselmayr and Koppl [5] model their proposed algorithm for using markets to increase flexibility as a constrained optimization matching problem. They include a variety of resources and stakeholders as available options in their optimization, which minimizes the combined costs of positive and negative flexibility subject to a set of operational constraints (e.g., limits on length of call and number of calls per day on a resource) and to the grid operator’s demand for flexibility. This analysis yields a market clearing algorithm, which they tested in a field experiment in the Altdorfer Flexmarkt.

Our sixth paper, focuses on whether the regulatory environment can facilitate the new business models which increased flexibility in the electricity system requires.

Schneiders and Shipworth [6] evaluate the existing legal environment in which new business models and forms of energy trading are emerging at the local level. Examples of “regulatory sandbox” projects in the UK suggest that blockchain has potential as a useful technology for matching supply and demand at local levels by enabling prosumers and consumers to trade. Candidate organizational forms for creating legal clarity in such local markets are Co-operative Societies and Limited Liability Partnerships, both of which have member benefits as an objective. Their analysis suggests that these existing legal structures would still face data-related challenges, specifically uncertainty associated with data privacy, and as local energy communities evolve, they may create new organizational and regulatory institutions that are better suited to the context of P2P energy trading.

Our seventh and eighth papers, draw on Italian case studies to examine the role of different actors within a more flexible electricity system.

Medinicino et al. [7] discuss the system architecture for the exploitation of flexibility from nanogrids. A nanogrid is a small DC system with its own power generation and load, which is connected to the AC distribution grid. It could be a single house or an energy community as defined by the European Union (EU). Mendincino et al. [7] suggest that internal and external optimisation can occur, such that the nanogrid matches its own supply and demand in such a way as to minimise its grid related costs, but also that it can supply services to the distribution grid. The discussion is framed in the context of the Italian power market and potential DSO-TSO co-ordination on ancillary services procurement via the GLASM market platform. The paper discusses how nanogrids might be contracted to provide congestion management and voltage support services to the DSO.

Schwidtal et al. [8] discuss a pilot project to open the Italian balancing market for participation by distributed flexibility. It was a large-scale pilot with 1 GW of distributed flexibility that participated over a period of 18 months from January 2019 to June 2020. The pilot was designed by the regulator to make sure that Italy can comply with the EU Clean Energy Package. The Italian balancing market used to follow a central dispatching approach with a strongly proactive TSO relying on large power plants. The pilot was executed by the TSO with participation by aggregators of distributed flexibility. The authors studied the outcome of the tenders and the bidding behavior. They conclude that there is an impressive potential for new flexibility resources in Italy if the smaller units would be allowed to participate in balancing markets. They propose to separate the submission of bids for upwards and downwards services, and to review the incentives that were ap-plied in the auction. The authors suspect that some players only wanted to cash in on the availability payments from the auction by making themselves available at the high price cap to avoid activation.

Next we move to Africa and an innovation project which focuses on a different issue: the problem of electricity theft and its reduction by the use of smart metering.

Otchere-Appiah et al. [9] examine the problem of electricity theft, meter tampering, and other non-technical energy losses at the distribution level in Ghana. They analyze data from a digital prepaid meter pilot project, in which tamper-proof prepaid meters replaced older, tamper-prone meters for 46% of 1666 residential customers participating in the project. This project structure allowed them to perform a causal inference analysis of the impact of digital prepaid meters on electricity theft, measured electricity consumption, and utility revenues. They found that electricity consumption increased by 13.2% in the group with prepaid meters, and that measured consumption increased in all income quintiles except for the lowest quintile. Despite data limitations, their results suggest that digital electric technologies can improve the efficient operation of the distribution grid.

We conclude our Special Issue with two surveys of recent papers, which focus on the promotion of flexibility service procurement in Europe.

Valarezo et al. [10] selected thirteen flexibility market pilots and four aggregator platforms from Europe. The authors compare the characteristics of the platforms (e.g., their pricing method, market frequency, bidding period, settlement…), and the buyers (e.g., Balancing Responsible Parties, Distribution System Operators, Transmission System Operators) and the sellers (e.g., Residential or industrial Aggregators, Home systems, Microgrids, Virtual Power Plants) that are active on these platforms. They conclude with trends and insights. They observe that some of these platforms are operated by experienced market operators. They recommend linking flexibility mechanisms with existing markets. They also highlight that there are many different use cases that are being tested for the use of flexibility from balancing to congestion management, voltage control, and controlled islanding of distribution networks.

Forouli et al. [11] assess to what extent electricity markets are open for participation by demand side flexibility in the United Kingdom, Belgium, Italy, and Greece. They conclude that the first two countries are more advanced than the latter. For each country, they discuss the enablers and the obstacles, and provide policy recommendations. They for instance recommend allowing aggregators to contract directly with prosumers without interacting with suppliers and Distribution System Operators. They also propose to re-duce the bidding volume and bid duration requirements in electricity markets. They remind us of how important it is to roll out smart metering systems and introduce cost reflective tariffs. They also underline the important role for DSOs in this transition, and the need for changes in the regulatory framework for DSOs.

The eleven published papers confirm that a lot is happening in terms of innovation at the level of the distribution system operator (DSO) within the electricity system. We are very grateful to all of our fellow authors and commend their careful analysis of real world electricity innovation experiments to our readers.