Evolving Bidding Formats and Pricing Schemes in USA and Europe Day-Ahead Electricity Markets ^†

Abstract

This paper compares the evolution of USA and European power markets and evaluates the suitability and future challenges of their designs in the context of the transition to a low-carbon power system. The analysis focuses on bidding formats (the way in which organized electricity markets allow participants to express their operational constraints) and pricing schemes (how agents recover their short-term costs from market prices). The radical evolution of the power mixes worldwide already experienced in the last decade and the larger one to come, with even greater shares of renewable energy and a more prominent role for storage resources, exposes limitations in current market designs. We develop an in-depth and comprehensive review of best practices from both sides of the Atlantic, and learning from them, we draw recommendations to evolve these market design elements.

1. Introduction

In the context of liberalized electric power systems, organized short-term electricity markets (as, for instance, the ones run by power exchanges in the European context (See www.europex.org)) not only help participants manage their risks, but mainly serve as a tool to facilitate an efficient matching of supply and demand, ideally contributing to the goal of maximizing market welfare. While electricity is often defined as a commodity (in the sense that one MWh of electricity is indistinguishable from another), experience has shown that for electricity markets to perform these tasks—aligning the economic utility functions of market agents and the physical constraints conditioning supply, it is more than suitable to allow for some complexity to the bidding and clearing procedures.

In markets for most commodities, only the willingness to buy/sell is relevant, but in the case of electricity, a proper consideration of the physical and economical constraints of agents is instrumental to achieve efficient clearing results. Bidding formats allow agents to express in a complex format their willingness to buy or sell electricity, reflecting how their particular constraints lead to the need to respect quantities and intertemporal links. Two very different approaches have been followed in the USA and Europe as regards to how to design these bidding formats, and in both cases, these formats are experiencing limitations to deal with the new paradigm.

Pricing electricity poses several challenges, mostly derived from the presence of non-convexities (such as non-convex costs). As is well known, in a non-convex context, there may be no linear prices (constant prices that remunerates all quantities) that are able to support a competitive market equilibrium. To deal with this issue, again, USA and Europe have opted for different approaches, and in both cases, these different schemes are being challenged by the penetration of new resources.

Bidding formats and pricing rules are key market design elements to allow for an efficient and massive integration of new resources such as renewables and battery storage systems. The objective of this paper is to develop an in-depth analysis on these elements, including the latest and most up-to-date discussions and challenges in USA ISOs (Independent System Operators) and EU Power Exchanges at the time of this writing. The paper is structured as follows:

In Section 2 we describe how USA and European markets followed very different approaches to the design of bidding formats and pricing rules.

Section 3 describes the context that motivates the evolution of power market design. Power system operation is becoming increasingly complex by the introduction of renewable energy resources, and new market players with new requirements are gradually entering into play, such as storage resources and aggregators. Other studies have discussed the impact of these changes in power markets performance, e.g., Hu et al. [1] and IRENA [2] develop comprehensive reviews on overall system needs, and Anuta et al. [3] focuses on the specific case of storage, but this paper focuses on bidding formats in greater detail and encompasses power system resources with more generality.

Section 4 analyzes performance implications of alternative designs and explores potential improvements for current bidding formats, especially in the European context where more limitations have been identified. We also assess other key elements linked to bidding formats, such as the design of clearing and pricing rules. Section 5 provides final conclusions and recommendations.

2. Bidding Formats and Pricing Rules in USA and the EU: Two Different Approaches

USA- and EU-organized power markets, from their initial implementation, opted for significantly different approaches to design their bidding formats and market clearing rules. The reasons for these differences were diverse, but maybe the most relevant one was the fact that from the very beginning, USA markets run by Independent System Operators (ISOs) were based on a pre-existent integrated structure (the Regional Transmission Operators) who had the responsibility to determine the economic dispatch in a centralized way. Meanwhile, in Europe, the market implementation was focused on prioritizing the economic dispatch (previously, run independently by different utilities) in a single market supported by Power Exchanges (initially, mostly national in scope and mostly non-compulsory). Green [4] and Conejo and Sioshansi [5] develop good descriptions of the fundamental differences of both approaches.

2.1. Markets in the United States: Resource-Specific Bidding Formats

USA markets use multi-part offers, which explicitly reflect generating units’ operational and opportunity costs (such as start-up costs), and their technical constraints (e.g., ramp rates). Multi-part offers are clearly motivated by the market clearing approach adopted by ISOs, which is nothing other than the straightforward application of the Security Constrained Unit Commitment and Economic Dispatch optimization models used before the liberalization of the power industry.

Table 1. Typical multi-part offer structure in ISO (Independent System Operator) markets.

Operating Costs		Technical Constraints
Energy offer curve	MWh, $/MWh	Economic min	MWh
Piecewise linear or stepwise linear function with multiple MW/Price pairs		Economic max	MWh
		Ramp rate	MWh/hour
No-load offer	$/hour	Min/Max run time	hours, min
		Min down time	hours, min
Start-up offer	$	Notification time	hours, min
Available for different types of start-ups (hot/intermediate/cold)		Cooling time	hours, min
		Start-up time	hours, min

highlights the typical offer parameters that ISOs make available to thermal units (see, for example, exhibit 4–6 at MISO (Midcontinent Independent System Operator) [6]).

In some cases, additional parameters allow multi-stage resources to represent their different operating regimes, and transition costs and constraints between modes (i.e., combined cycle gas turbines, which allow multiple configurations of their gas and steam turbines, and therefore have multiple commitment decisions to take). The bidding parameters highlighted here focus on the energy market, but USA markets also optimize operating reserve provision, and other bidding parameters are provided to this end. So-called flexibility products (which are close in nature to reserve products) are also out of the scope of this paper. Jacob [7] presents a good review of current discussions around flexibility products.

The archetypical multi-part offer is the thermal unit bidding format (predominant type in USA systems), but other bidding formats have also been implemented for different types of resources. For instance, in Section 3.2, we describe recent developments to improve bidding formats available to pumped-storage hydro and other storage units. Not all market agents require complex multi-part offers, and it is possible to submit only price-quantity bids, which could be the case for renewable generators and load serving entities.

In summary, ISOs attempt to represent the power system with the highest detail possible in their clearing algorithm, including the technical characteristics of each generator individually, apart from all the constraints required to ensure reliability. This complex model allows ISOs to optimally schedule resources, while enabling competition in the provision of energy and electricity services.

Pricing Approach

These multi-part bidding formats make the clearing problem non-convex, causing well-known challenges in the computation of marginal prices [8,9,10,11,12,13,14,15]. The basic matter of this non-convexity is that the marginal cost of the system may be lower than the average cost of some units. For example, a power plant may be block-loaded, meaning its minimum output constraint is equal to its maximum capacity. These units are usually fast-start gas turbines that only operate economically at full load. A block-loaded unit may be committed by the clearing algorithm but, because it cannot supply the next marginal increment of load, it cannot set marginal prices. In this case, a lower-cost unit could set the marginal cost of the system, making the market price lower than the average cost of the block-loaded unit. For this reason, marginal prices are complemented with uplift payments, which are separate payments that compensate generators for the costs incurred above the revenue earned through market prices. Uplift payments are also referred to as side-payments or make-whole payments.

Uplifts are unavoidable in an optimal dispatch-based market, required to support the welfare, maximizing commitment and dispatch (i.e., to provide a remuneration aligned with dispatch orders). The underlying problem with uplift payments is that they create a discriminatory pricing regime, where not all agents benefit from the same prices, potentially creating misaligned incentives. This means price signals do not fully reflect operational costs, which can also have an effect in long-term investment decisions [16].

Pricing in USA markets, especially in recent years, has deviated from pure marginal costs in an attempt to reduce the weight of uplift, and to internalize, as much as possible, all operational costs into market prices. A notable example is the “hybrid pricing” approach first implemented in 2001 by NYISO (New York Independent System Operator) [17], and that is continuously updated and under review [18,19,20]. The NYISO pricing approach allows block-loaded units (as the one in the previous example) to artificially become marginal in an ex-post run of the dispatch problem, where the inflexible bid is treated as flexible (as if it could be dispatched at any level between zero and its maximum power output). This way, block-loaded units can set prices, although NYISO only applies this method for a subset of fast-start units.

The more general term used for this practice is Integer Relaxation (IR), which involves relaxing binary constraints in an ex-post pricing run of the commitment and dispatch problem (see [12] or [21] for more detailed discussions). However, the exact method is more nuanced and varies from one ISO to another. Indeed, most ISOs apply some type of IR, but they differ in which units can set prices, and whether they consider start-up and no-load cost in the pricing problem. In some cases (for instance, in the original NYISO hybrid pricing), only the minimum output constraint is relaxed in the pricing run, so only variable costs can impact prices; this practice is frequently called “EcoMin relaxation”.

In addition, “fast-start pricing” is also a common term in practice because the relaxation often involves only fast-start units. Allowing fast-start units to set marginal prices can have positive effects, such as sending efficient signals to price-responsive load [22], or incentives to fast-start units to improve their performance or bid their true cost [23]. Fast-start pricing has been a hot topic in USA markets in the last years, with some relevant ISOs not fully allowing prices to reflect the short-term true costs. This led the Federal Energy Regulatory Commission (FERC) on 2019 to, for example, require PJM Interconnection and New York Independent System Operator (NYISO) to “implement tariff changes to ensure their pricing practices for fast-start resources were just and reasonable” [20]. The measures included, among others, using the same time granularity in the ex-post pricing run of the model and in the previous commitment and dispatch problem, or further relaxing the capacity of the fast-start units that are flexible in the price computation.

Fast-start block-loaded resources are certainly a very relevant part of the uplift problem, but this is not the only non-convexity causing price distortions. Start-up and no-load costs of thermal units can potentially cause uplift, both in the real-time and day-ahead markets. A more inclusive approach is applied in MISO (based on a simplified version of Convex-Hull pricing, see [24]); called approximated ELMP (extended locational marginal pricing). This approach is essentially an IR, but it is more comprehensive than NYISO’s hybrid pricing. MISO includes start-up and no-load costs in pricing and applies ELMP to all fast-start resources (not only to block-loaded ones). Indeed, MISO broadened the definition of fast-start resources to allow more peaking units to set prices [25]. MISO continues to search for improvements to its pricing approach; during 2019, MISO studied the practical application of new formulations of the convex envelope for ELMP [26], with the objective of changing the methodology when the cost-benefit analysis was clear.

2.2. Markets in Europe: Abstract Bidding Formats

European power exchanges follow a completely different approach; their main goal is to provide a platform for market agents to trade electricity, simplifying as much as possible the consideration of physical constraints, under the presumed objective to facilitate trading and maximize market clearing replicability and transparency. System operation is decoupled from the market, and left to transmission system operators, which eventually enforce reliability constraints. This vision shifts part of the responsibility in optimizing the operation of generation resources to market agents and expects them to express their willingness to buy or sell power in a simpler way. For instance, most European power exchanges allow portfolio bidding, i.e., generation companies that own several generation units in the same pricing area can submit combined offers, and then internally decide the operation of each unit to reach the required production.

The basic bidding format in Europe is the price-quantity pair; however, a set of more complex bidding formats (or order types or market products, in the European terminology) is also available, as shown in

Table 2. Bidding formats in EUPHEMIA (acronym of Pan-European Hybrid Electricity Market Integration Algorithm).

Bid Format	Description
Simple orders:	-
Hourly step orders	Buy or sell orders for a given volume and a limit price. It can be partially accepted if the market clearing price is equal to the bid price.
Hourly linear piecewise order	Buy or sell order for a given volume and a pair of prices: An initial price at which the orders begin to be accepted and a final price at which the order is totally accepted.
Block orders:	-
Regular block order	Buy or sell order for a single price and volume and a period of consecutive hours that can only be totally accepted.
Profile block order	Regular block order that can be partially accepted, it includes a minimum acceptance ratio condition.
Exclusive block orders	Set of block orders in which the sum of accepted ratios cannot exceed one.
Linked block orders	Set of block orders where the acceptance of some blocks (children) is conditioned to the acceptance of others (parents).
Flexible block order	Price and volume combination that can be accepted in several consecutive periods within a defined delivery range.
Complex conditions:	-
Minimum Income	Condition to reject all hourly orders of a resource if its daily remuneration does not reach the minimum income amount, defined by a fixed and a variable component.
Load gradient	Limit to the variation between the accepted volume at a period and the accepted volume at the adjacent periods

.

Hourly step and linear piecewise orders resemble the variable cost component in USA multi-part offers, but in this case, all operational costs must be internalized in the offered price (no additional components such as start-up cost are explicitly considered).

Complex conditions can be added to hourly orders to reflect more sophisticated constraints [27]. The minimum income condition available in the Iberian market can constrain the hourly orders of a unit, so they are only accepted if the income of the resource throughout the day reaches a fixed amount (representing, for example, the start-up cost) plus a variable cost component. The minimum income condition, combined with the load gradient condition, represents some, but not all, of the features of multi-part offers. However, the fixed and variable cost components are not considered for the maximization of market welfare, only to reject some hourly orders when the minimum income condition is not met.

Alternatively, block orders are bids that apply to multiple consecutive periods simultaneously, instead of a single hourly period, and are accepted or rejected based on the average price for those periods [28]. Resorting back to the example of the thermal unit, this could allow offering to start a power plant for a given set of hours, internalizing the start-up cost in the average price. Block orders can be combined using exclusive or linking conditions to represent more complex possibilities.

All order types in Table 2 are implemented in the single clearing algorithm EUPHEMIA (acronym of Pan-European Hybrid Electricity Market Integration Algorithm) [28]. However, the orders available in the power exchange designated in each country (the Nominated Electricity Market Operator, NEMO) differ. The integration of power exchanges through the Price Coupling of Regions (PCR) initiative has achieved some standardization of market products, but for the moment, NEMOs have not fundamentally modified the orders available in their territory. For instance, complex conditions were, and still are, only available in the OMIE exchange (for Spain and Portugal), while Nord Pool (Nordic-Baltic region) and EPEX SPOT (central Europe) allow the use of block orders.

Bidding formats are now regularly reviewed; the first proposal being submitted jointly by all NEMOs dates back from November 2017 [29]. This proposal did not include significant changes besides updating some definitions. For instance, hourly orders were defined as Market Time Unit (MTU) orders, and any references to hourly periods were modified accordingly; this was to allow changes in the definition of MTU in the future (the goal is to move from hourly to 15-min periods). In addition, it generalized some definitions to allow the use of all orders as both supply and demand. For example, the “maximum payment condition” was introduced as the demand-side version of the minimum income condition. It is worth noting that, although the 15-min change has not been implemented at the time of this writing (it is expected for 2021), it currently focuses most of the efforts in EUPHEMIA developments [30].

The day-ahead products were approved by all Regulatory Authorities and agreed to at the Energy Regulators’ Forum on 23 January 2018. Every two years, NEMOs shall consult the products that should be included in the day-ahead market. The last consultation proposal in April 2020 [31] includes a number of amendments to the current list of day-ahead market products, with the most relevant being the inclusion of a new complex condition: the Scalable Complex Orders (SCO). Unlike the classical MIC order that imposes a minimum income condition (MIC) expressed using a fixed cost and a variable cost, the Scalable Complex Order does not use the former variable cost, and instead uses the prices of the hourly suborders as variable cost on top of a fixed cost. As pointed out by EIRGRID et al. [32], the theoretical merit of SCO over MIC is to improve EUPHEMIA performance, but this merit can only materialize if the SCO eventually replaces (not complements) the classical complex orders.

2.2.1. Pricing Approach

European bidding formats, although seemingly simpler than their USA counterpart, also create non-convexities in the clearing problem with similar implications in the determination of market prices. However, the market clearing approach is not a pure welfare-maximization; the essence of European markets is to determine the highest welfare solution possible that also meets these two constrains:

• The market must be cleared with uniform prices; this entails market prices (without uplifts) must suffice to compensate all accepted bids.
• Simple bids (hourly orders) must be fully accepted if in the money, i.e., if the market price is above a generation offer, this offer must be fully accepted.

The uniform-pricing principle is often rephrased as a restriction that does not allow the existence of paradoxically accepted bids (PAB). PABs would be offers accepted in the market which are unprofitable at market prices, so it can be likened to units that require uplift payments in the USA context. Since PABs are not allowed in European markets, uplifts are consequently not needed either. The second principle refers, in the European terminology, to paradoxically rejected bids (PRB), which are those rejected bids that would apparently be profitable at given market prices [28]. As stated previously, simple bids cannot be paradoxically rejected, but PRBs are allowed in European clearing rules for complex conditions and block orders.

These two conditions applied simultaneously constrain the welfare-maximization problem, leading (by definition) to a generally sub-optimal market welfare. This is a matter of trade-offs; in the European context, uniform-pricing is, as an objective worth, the potential loss in short-term efficiency. Among the advantages of uniform-pricing is that demand and generation interact in the market in equal terms, and it is not necessary to define rules to allocate uplift that would inevitably send inefficient signals.

3. The Increasing Need for More Complex Bidding Formats

Both USA and European markets feature different kinds of complex bidding formats, revealing the higher complexity of electricity markets compared to other commodities. Both USA and EU day-ahead markets were originally designed for a predominantly thermal generation mix (with some notable exceptions), and most complex bidding formats were justified by the operational characteristics of thermal generation resources. While this is quite clear in USA multi-part bids (see Table 1), complex European bidding formats tackle the same problem from a different angle. Section 3.1 explains how the needs of thermal resources are addressed (with some limitations) by European bidding formats, and how the penetration of renewable energy sources makes these complex bidding formats even more necessary.

The transition to a low-carbon power system will most likely necessitate from new energy resources (such as batteries), which will bring their own operational constraints, requiring, as well, new bidding formats. This further justifies the need for complex bidding formats, as discussed in Section 3.1.3.

3.1. Operation of Thermal Resources

The following example describes the bidding requirements of thermal resources, building from the simplest bidding format possible, to progressively introducing more complexity as the limitations of the simpler formats arise.

This section focuses especially on the challenges derived from the variability and uncertainty of renewable energy sources. Each step of this sequence faces a trade-off between the operational efficiency lost in day-ahead dispatch decisions from using too simple bidding formats, and the additional computational burden required to introduce complex bids.

3.1.1. Initial Setting

As a starting point, the simplest design possible is a single-period simple auction, where market agents submit price-quantity pairs to express their available production or desired consumption, and their production cost or demand utility. This design takes the assumption that producers’ cost structure consists fundamentally only of variable costs, and/or producers are able to properly predict how their plants will be committed (so they can efficiently internalize their non-convex costs in their bids). This could be a reasonable proxy in power systems dominated by thermal power plants and characterized by a rather flat net thermal demand (i.e., hydrothermal systems) or at least characterized by a highly predictable one. This is the foundation of European power exchanges, and, for instance, the Italian day-ahead market which still uses only simple orders (still, the original designs were quickly complemented with intraday markets to allow market agents to rectify dispatch decisions, see, for example, [31]) [33]. If they ever did, these assumptions do no longer hold in practice in the vast majority of European markets, so there is a severe risk that this approach does not provide the most efficient, or even a feasible dispatch.

Traditionally, another way of facilitating dispatch decisions, despite the simplicity of the power exchange, was to allow portfolio bidding. The lack of complexity can be compensated by managing a large portfolio instead of an individual power plant. Large generation companies with sufficiently diverse portfolios mitigate the impact of an infeasible outcome of the market clearing algorithm, since a large portfolio allows “absorbing” potential inefficiencies. However, in this context, this approach is nothing but a potential market barrier for potential new entrants, and eventually an alternative to exercise some degree of market power. By using simpler bids, generation companies benefit from disclosing the minimum amount of information about their operating cost structures, as limiting the amount of information contained in bids complicates monitoring of market power, since it is very difficult to link bid parameters to actual operating costs [27].

3.1.2. Variability

One of the reasons why this simple model can lead to inefficiencies is that, in the real multi-period problem, it cannot reflect constraints coupling different periods. For example, thermal power plants have ramping constraints that make the production available in one period dependent on the production in the preceding and following periods. One of the effects of the introduction of renewable energy sources is an increase of the cycling regime of thermal units [34]; in summary, ramping constraints are expected to be binding more frequently, and not incorporating this constraint in the day-ahead schedule can significantly degrade the efficiency of the dispatch.

Dispatch efficiency could be improved by incorporating ramping constraints in the optimization model (as in USA markets, or using the load gradient condition), at the expense of some market transparency, but this is not the only way to face this problem. For instance, block orders allow bypassing this problem, if used to offer a predefined production profile (the so-called profiled block orders), as shown in Figure 1.

3.1.3. Uncertainty

Using a block order requires the producer to take, prior to the market clearing, a decision about what would be the best production profile to offer into the market. The underlying assumption has been that producers can easily forecast market conditions (not only market prices, but also the resulting unit commitment), and offer the most efficient production profiles. In reality, the market outcome is uncertain, and generators need to account for the uncertainty of demand forecasts, competitors’ bids, and renewable production schedules [35].

Linked and exclusive block orders can mitigate some of this uncertainty, allowing producers to express a wider range of potential operating profiles for the clearing algorithm to choose. For example, Figure 2 shows how two additional block orders (orange and green) can be linked to the previous order to potentially extend the range of hours where the unit is operating. Linked orders can only be accepted if the previous (parent) order is accepted.

The uncertainty associated to renewable production has greatly increased the need to model the complexities of power system operation. In USA markets, day-ahead bidding formats already represent operational constraints with detail, and renewable energy sources (RES) does not involve either a change in the way agents bid in the market nor an increase in the number of bids. However, in European markets, the use of block orders has increased significantly in recent years. Vázquez et al. [36] analyzed this effect for the Spanish case.

Figure 3 shows the average and maximum number of block orders used in the PCR region (data from European Stakeholder Committee of the Price Coupling of Regions [37,38]. Only annually aggregated data was available for the period 2011–2013, and monthly for 2014–2017). Not only has the total number of block orders almost tripled from 2011 to 2017, the use of the most complex block types has had even greater growth. As discussed in Section 4.1.2, the use of block orders and complex conditions is expected to keep on rising in the following years. This represents a major challenge from the computational point of view.

The use of exclusive orders remarks the fact that, in the uncertain context resulting from renewable production, producers cannot easily plan the operation of their units. Exclusive orders allow expressing multiple possible production profiles of which only one can be accepted by the market, therefore, it makes it easier for market agents to make offers for different scenarios. For example, the orders shown in Figure 2 express three different production profiles, which could also be represented by three exclusive orders. Exclusive orders can sometimes express a wider range of possibilities than linked orders, since exclusive orders do not need a common parent block. In the example in Figure 4, a unit does not know what the best time to sell its production is, so it offers three different blocks and the clearing algorithm will select the best one (maximizing market welfare) only.

3.2. New Bidding Formats for New Resources

The development of bidding formats has been clearly influenced by the needs of thermal power plants, not only in the USA where multi-part bids are used, as previously described complex European bidding formats have also been tailored to the needs of thermal units.

The transition to low-carbon power systems will likely reduce reliance on thermal resources as other flexible energy technologies enter into play to compensate for the variability and uncertainty of renewables. It is difficult to predict what resources will play that role in the future, but whether it is in the form of batteries, hydrogen or other storage resources, all are clear candidates.

Pumped-hydro storage has been present in power systems for many decades, and for the same reasons that complicate the operations of thermal power plants, the participation of resources with (limited) storage capabilities now requires more complex bidding formats.

The key challenge for storage arises when bidding formats force these resources to decide, in advance, when (in which time intervals) to bid as a generator and when to bid as demand. The volume of generation and consumption possible from a storage resource are interdependent. Bidding is especially difficult for new electro-chemical storage resources (such as Lithium-Ion batteries) because of their limited storage capacity. Grid-scale batteries, due to their high cost, are usually sized to store energy only for a few hours at nominal capacity, while pumped-hydro resources can have up to weekly or monthly planning cycles. Although both resources have limitations to participate in power markets with current bidding formats, small storage resources have clearly more limitations.

3.2.1. Developments in USA Markets

Resource-specific bidding formats in USA markets clearly provide almost perfectly-adapted bidding parameters for a selection of resources, but on the downside, discriminate potential new resources which cannot enter the market with ease until specific bidding formats are designed for them. For instance, pumped-hydro resources have participated in ISO markets for many years, but smaller storage resources (e.g., batteries, flywheels) have different constraints that cannot be represented with existing bidding formats.

The abovementioned created concerns that unnecessary barriers to storage resources were limiting competition, and the FERC initiated a consultation in November 2015, which culminated in Order 841 in February 2018, entitled “Electric Storage Participation in Markets Operated by Regional Transmission Organizations and Independent System Operators” [39]. The requirements most relevant to the topics discussed in this paper are the following [40]:

• ISOs must include a participation model for electric storage resources (ESRs) that allows them to participate in energy markets (also in ancillary service and capacity markets) when technically capable of doing so. This participation model has to prevent conflicting dispatch signals in the same market interval (charging and discharging at the same time).
• ISOs must allow self-management of state of charge (SOC).
• ESRs must be able to set the wholesale price both as seller or as a buyer, when it turns out to be the marginal resource.
• ISOs must account for physical parameters of ESRs through bidding or otherwise.

As it is usually the case, the FERC order allows a high degree of freedom for ISOs to customize rules to their specific context. Therefore, these requirements will not be homogeneous across ISOs, but the rule provides an interesting judgement on what are the most relevant constraints of batteries.

Table 3. Multi-part bid for storage.

Charging	Discharging	Unit
Max charge limit	Max discharge limit	MWh
Min charge limit	Min discharge limit	MWh
Charge ramp rate	Discharge ramp rate	MWh/min
Max charge time	Max run time	hours, min
Min charge time	Min run time	hours, min
Energy bid curve	Energy offer curve	MWh, $/MWh
State of charge parameters
Initial state of charge	-	p.u.
Max state of charge	-	p.u.
Min state of charge	-	p.u.

summarizes the potential bidding format for storage following ISOs’ resource-specific approach.

The first characteristic that makes these bidding parameters different from traditional multi-part bids is that it allows for both charging (consuming) and discharging (generating) regimes, in a single bid. Before, storage resources needed to present separate offers as generators and consumers.

Although many of the bidding parameters are equivalent to the ones used in multi-part bids— maximum/minimum operating limits, ramp rates and maximum/minimum run times— a new participation model is necessary because the constraints are applied simultaneously for charging and discharging. Furthermore, new constraints are necessary to represent the limited energy storage. The state of charge represents how much energy is stored in a battery with respect to its maximum capacity. The definition of models to manage the state of charge has been one of the most open design elements, and has led to different approaches [41]. All ISOs have implemented, as an option, the so-called self-schedule model, where storage operators are responsible for dispatching the output (independently from the market clearing algorithm).

On the other extreme, storage operators may prefer to entirely leave to the ISO the responsibility of achieving a feasible dispatch. In this model (known as ISO-state of charge-management), storage operators would not submit hourly price-quantity offers, but rather, the technical parameters allowed in the market. This model has been implemented in CAISO (California ISO), NYISO, and PJM (but only for pumped-hydro storage).

Other models are possible between these two approaches. For instance, the so-called “self-state of charge management”, where the operator of the storage facility is still responsible to achieve a feasible dispatch, but may present simple bids in the market to optimize its schedule. This model has been implemented in CASIO, NYISO, MISO, SPP (Southwest Power Pool) and PJM. This model is fundamentally aimed at the real-time market, where the operator can monitor the state of charge and update market bids.

The deadline for ISOs to implement Order 841 was December 2019 Most ISO/RTOs have achieved compliance with the order, but two are still on track to meet the requirements. The New York ISO (NYISO) requested an extension to 2020 (accepted), and Midcontinent ISO (MISO) to 2022 (also accepted).

3.2.2. Developments in European Markets

Arguably, the approach implemented in EU power exchanges provides a general set of “abstract” bidding formats that can be used by any type of resource. However, the current design was not conceived for storage resources.

As described for the USA context, the main bidding requirement of storage resources is to represent the physical link between supply offers and demand bids. In this regard, linked block orders provide a limited way to represent this constraint. For example, as shown in Figure 5, a storage resource could submit a purchase block order and a linked sell block order. This way, the sell order will not be accepted if the purchase order has not been accepted as well. In other words, the battery will only be discharged if it has been charged before.

This use of linked orders has two main limitations for storage resources. First, market agents must decide, in advance, two potential periods to buy and sell power, so the use of storage is not fully optimized. Potentially, this limitation could be addressed by submitting multiple pairs of linked buy-sell orders, including an exclusive constraint so only one of the pairs is accepted. However, linked and exclusive orders cannot be combined.

Second, the linked order guarantees a feasible schedule (since the battery will not be discharged if it has not been charged before), but because the link can only go in one direction, the parent purchase order could be accepted without the sell order. This would produce a feasible but clearly suboptimal schedule, leaving the battery charged without a commitment to sell its energy. This creates a risk to incur losses if using this bidding format. To address the latter limitation, EPEX Spot introduced a new type of bidding format called “loop order” [42]. Loop orders allow submitting two (and only two) blocks which will be executed or rejected together by the clearing algorithm. The introduction of this new order type highlights that bidding formats need to be continuously updated as the needs of market agents evolve; and shows a shift from the “abstract” bidding format approach to much more resource-specific products.

It is worth noting the Nordic power exchange features flexible orders, which is an extremely useful bidding format for limited-energy resources, although it is aimed mainly at storage resources with a longer than one-day planning cycles (such as hydro storage). Flexible orders allow participants to express the maximum volume of energy they are willing to sell, and the clearing algorithm will select its optimal allocation. However, flexible orders share some of the abovementioned limitations for small storage resources. The problem of coordinating the sale and purchase of power within a single day remains because it is not possible to combine flexible and linked or exclusive orders.

The current discussion on the products’ definition (the aforementioned consultation process launched in April 2020) includes a number of minor amendments to the products, none of them affecting the capability to properly bid storage resources in the day-ahead market.

4. Challenges Ahead

As described in the previous section, the operational needs of thermal units in a context of increased renewable penetration, and the potential needs of new energy resources, call for rethinking bidding formats. However, reforming such a fundamental market design element opens some additional debates. Section 4.1 discusses some of the computational challenges that could arise from the introduction of complex bidding formats, and Section 4.2 elaborates on the implications in Europe of introducing complex bidding formats similar to those in the USA.

4.1. Computational Tractability

In both USA and European markets, the computational complexity of the clearing problem is an instrumental factor that conditions what bidding formats can be implemented in practice. USA markets use market welfare-maximizing optimization models to clear markets—which despite their large size are reasonably tractable problems—followed by separate pricing models. European markets, despite including less detail in modeling the physical constraints of the system, combine clearing and pricing in a single more computationally complex model.

4.1.1. Challenges in USA Markets

As previously discussed, USA markets use detailed multi-part bids, which capture most of the complexity of thermal generating units. This model is well prepared to face the introduction of larger shares of renewable production. ISOs have progressively increased the modeling detail in their markets [43], as made possible by optimization software improvements and developments in computing technology. This does not mean the USA model is not constrained by its computational tractability, but for the moment, computational improvements have continued to allow for incremental modeling enhancements. For instance, some ISOs have already implemented new bidding formats for storage, similar to the one previously described, see e.g., [44,45]. However, computational problems could arise, not because of the complexity of these bidding formats, but due to the larger number of participating resources. New storage resources could be 1 MW or less in size, which is orders of magnitude smaller than traditional resources, meaning the number of market participants could be hundreds of times the current amount. The size of the resulting commitment and dispatch problem could become intractable, and indeed, FERC Order 841 [39] included provisions to allow increasing minimum bid size requirements:

We are also not concerned about the potential availability of software solutions as multiple RTOs/ISOs already provide a minimum size requirement of 100 kW for all resources and have not expressed similar concerns regarding the minimum size requirement. While establishing a minimum size requirement of 100 kW for the participation model for electric storage resources will result in some smaller resources entering the markets in the near term, we do not expect an immediate influx of these smaller resources or any resulting inability to model and dispatch them. However, we recognize this finding is based on the fact that there are currently fewer 100 kW resources than there may be in the future. Therefore, in the future, we will consider requests to increase the minimum size requirement to the extent an RTO/ISO can show that it is experiencing difficulty calculating efficient market results and there is not a viable software solution for improving such calculations.

This computational problem in ISO markets results from the combination of two factors: complex bidding formats and the number of resources. Therefore, the scalability issue can be confronted from both sides. Increasing the minimum size requirement is a way to reduce the number of resources, but this also limits competition, so it is only acceptable as a short-term solution. This measure should be accompanied by the development of rules for the participation of aggregations of resources, which opens all sorts of new questions. For instance, defining bidding parameters for aggregators cannot follow the usual resource-specific approach in ISO markets, since this participation model calls instead for general bidding parameters.

An alternative approach would be to simplify existing bidding formats. Just as creating participation models for aggregations rather than individual resources, this approach would reduce the ability of ISO markets to model the physical system accurately. Taking any of these solutions would significantly change the current modus operandi in ISO markets, but there are several reasons why ISO markets will not need to simplify its approach all the way to European-like bidding formats. As already discussed, the welfare-maximizing clearing approach allows for much more complex bidding formats than the uniform-pricing clearing approach, now and in the future. Furthermore, ISOs have not shown interest in facing one of the greatest challenges of European markets (see next section), which is to integrate several states/countries in a continent-scale market. For now, each ISO market has a well-defined footprint, and although some markets are expanding their geographical scope (for example, California ISO’s real-time market has been opened to neighboring balancing authority areas through the Energy Imbalance Market (See www.westerneim.com)), no plausible plans exist to further integrate all North American ISOs. Such an objective in the future, however, would most likely require taking modeling simplifications.

4.1.2. Challenges in European Markets

As discussed in Section 2.2.1, the uniform-pricing rule conditions the clearing problem in European markets. This approach combines clearing and pricing in a single, more computationally complex problem. Computational complexity becomes especially relevant when considering the ultimate goal of European markets is to integrate all European member states in a single clearing algorithm. Computational problems have already surfaced during the first years of operation of the PCR, and Eastern European markets are to join the common platform in the upcoming years. Probably, the most relevant concern nowadays in European markets is the existence of PRBs. As previously described, PRBs are unavoidable under uniform price-based clearing; however, in certain cases, bids may be incorrectly rejected due to the computational complexity of the algorithm. As pointed out by the Market Parties Platform (MPP) in the European Stakeholder Committee of the Price Coupling of Regions [46]:

There may exist false PRBs: rejected in-the-money blocks that could have been accepted and result in a better (higher welfare) solution. MPP asks for more transparency on optimality, to prove the absence of false PRBs.

The reason behind this matter is that, mathematically, the clearing problem is a non-linear and non-convex problem, for which it is difficult to prove the optimality of a solution, or to take a quantitative measure of the quality of a solution. This may hinder the confidence of market participants, together with a lack of clarity in the public documentation of the clearing algorithm [29]. The joint response of ACER (Agency for the Cooperation of Energy Regulators) and CEER (Council of European Energy Regulators) [47] to the European Commission’s Consultation on a new Energy Market Design claims that: “We would particularly like to see clearer rules and greater transparency around the market coupling algorithm (EUPHEMIA)”.

Computational complexity may limit the scalability of European markets, not only to integrate more Member States in the PCR, but also to cope with the increasing trend in the number of block bids submitted to the market.

Performance and scalability are two pillars that need to be reinforced in the algorithm, and as a consequence, ACER Decision 08/2018 on the Algorithm Methodology on 26th July 2018 established that NEMOs have to report regularly on the following aspects regarding day-ahead market coupling: Operations (incidents and corrective measures), Performance Monitoring (performance indicators, including paradoxically rejected blocks and social welfare), Scalability and R&D.

As regards to the scalability concern and the growing use of block order and complex conditions, the first report, published in November 2019 [48], confirmed the expectation is for the use of both complex and block orders to keep growing in the coming years. Figure 6, from said report, shows the expected usage of complex and block orders in 2021 (expressed as percentage of 2018 usage), with the use of complex orders and linked block orders expected to grow to 170% of its 2018 value. Those estimates did not take into account the potential impact of the forthcoming 15/30-min products, for which proper data sets and specifications were still missing.

EU power exchanges have traditionally limited the amount of block orders submitted by portfolio (i.e., by market agent and market area), as shown in

Table 4. Limits to block orders in main European power exchanges [49,50].

	EPEX SPOT							NORDPOOL
Area	AT, DE-LU	FR	BE, NL	DK, FI	NO, SE	CH	GB	NO, SE, FI, DK, EE, LV, LT	AT, DE-LU, FR, BE, NL	GB
Block order
Max. volume (MW)	600	600	400	500	500	150	500	500	800	500
Max. Blocks/Portfolio	100	40	40	100	100	40	80	50	100	80
Linked block order
Max. Levels	7	7	7	7	7	7	7	3	7
Max. Children	6	6	6	6	6	6	6	6	6
Max. Blocks/Family	7	7	7	7	7	7	40	7	13
Max. Families/Portfolio	5	5	5	5	5	1	7
Exclusive group block orders
Max. Blocks in group	24	24	24	24	24	24	24	15	24	15
Max. Groups/Portfolio	5	5	5	5	5	5	5	3	5	3
Loop blocks
Max. Volume/Family (MW)	800	800	500	800	800	300	800
Max. Blocks/Family	2	2	2	2	2	2	2
Max. Families/Portfolio	3	3	3	3	3	3	3
Flexible orders
Max. Orders/Portfolio								5		3

. A potential solution would be to further reduce block order limits, but this is clearly not a desirable outcome.

Furthermore, current rules require the clearing algorithm to obtain a solution in less than ten minutes [51]—A much more demanding timeline than USA markets, so the quality of the solution could also be improved by allowing additional time for the clearing process. However, European markets do not incorporate many physical constraints, making additional corrections by Transmission System Operators necessary, so it may not be possible to extend this timeline by a large margin.

Both reducing the number of block orders, and extending the time available to compute the solution, are temporary fixes. In the long term, it is necessary to focus on the root causes. The number of block orders submitted has greatly increased and will keep increasing because no single bidding format properly addresses the needs of market agents, therefore, agents combine orders in an effort to hedge against all the possible market outcomes.

A more permanent alternative would be to create resource-specific bidding formats that would only require one (multi-part) bid per resource. However, since such resource-specific bidding formats would likely be more complex, they should be carefully designed to ensure they indeed reduce the number of orders and overall problem complexity. A currently discussed alternative in this line is the introduction of thermal orders, which is nothing more than multi-part bids like the ones used in ISO markets.

The European Stakeholder Committee of the Price Coupling of Regions [38] suggests that thermal orders can be preferable (from a computational efficiency point of view) if an agent can submit a single multi-part bid replacing multiple block orders. However, in the same stakeholder committee, the EUPHEMIA software provider (N-Side) has also pointed out that including such a bid would need to consider a significant change in the market design, and pricing and clearing rules, which are discussed in the next section.

4.2. Clearing and Pricing Rules

The same fundamental problem—pricing in non-convex markets—has surfaced in different ways in USA and European markets because of differences in clearing rules. The main challenge in USA markets is to reduce uplift payments, which may be a clearer symptom of inefficient pricing, but similar issues should be expected in the European context. The uniform price-based clearing rule in EUPHEMIA relies on marginal pricing, even if the dispatch solution is not fully welfare-maximizing. This means that, in the EU context, inflexible bids cannot set market prices (similarly to block-loaded units in USA markets). As described by Eirgrid et al. [51]:

The effect of defining an order as a block is that the order cannot then be a full price maker. Rather, block orders may impose a bound on the range of prices possible while the price being set would still need to come from the simple order or complex order curves. This is because the decision to execute the order is an integer decision (i.e., the order is executed or not executed) and the decision on whether to accept a block occurs before the price determination sub-problem. The bound created by the last accepted block order would function to affect the price (by limiting possible values) but could not directly set this price.

This was discussed with the PCR ALWG representative, APX, who confirmed that without the blocks setting the price, the price could only be set by other price makers, i.e., simple orders or complex orders, or the price indeterminacy rules of EUPHEMIA.

Another similarity with USA markets is in the way some units may represent their start-up costs using the minimum income condition. This constraint guarantees that the offers of a unit will only be accepted if the price is high enough to compensate a given fixed cost (representing the start-up cost). Although this fixed cost influences the clearing of the market—triggering the rejection of the offer if the price is not high enough—it does not directly set the market price, which is always set by a simple (marginal) bid.

In a scenario where most bids are inflexible, or where they use any non-convex order type, EUPHEMIA can lead to market prices that do not accurately reflect system costs. Note that, even if uniform prices necessarily include all operational costs, there is a nuanced difference between market prices being high enough to cover costs, and market prices being reflective of system costs. This is, in essence, the same problem described for USA markets, where inflexible units could not set the price. Fortunately, this also means that European markets could benefit from the solutions developed for the USA context. However, the only way to allow alternative pricing rules is to modify clearing rules as well.

Clearing rules in European markets are based on uniform pricing, but this is not imposed by bidding formats. Even though market orders always express their constraints with respect to market prices (i.e., if price is below X, reject order), the uniform-price clearing approach is mostly the result of a market that has evolved from a simple auction. The basic information contained in market orders is the necessary remuneration, but said remuneration could include uplift payments if such a policy choice was made. Obviously, this requires a significant change, but current bidding formats do not prevent using a market welfare-maximization clearing approach and an alternative pricing rule.

Indeed, a welfare-maximization clearing approach would greatly simplify the clearing algorithm, helping European markets cope with the current increase in the use of block orders. Additionally, it would enable more complex bidding formats, such as thermal orders; and more complex combinations of block orders, as made necessary by storage resources.

This discussion had a greater momentum back in 2016 [52] and today is not high up on the agenda of NEMOs. However, the possibility has not been completely discarded, and it is still included among the different R&D activities to be explored [53].

5. Conclusions

The penetration of renewable energy resources has significantly altered power systems. In light of these changes, wholesale electricity markets, and, in particular, day-ahead markets, in their role to facilitate planning and operating decisions, require increasingly complex bidding formats. While USA markets already provide detailed multi-part bids to reflect the most relevant constraints of thermal generators, European markets provide a limited choice of block orders and complex conditions. These orders may be falling short to facilitate an efficient participation of all resources into electricity markets, as evidenced by the ever more complex combinations of orders submitted by market agents to achieve an adequate representation of their constraints.

The energy transition will also bring about the introduction of new energy resources, for example, batteries and other types of storage, making it necessary to address their needs and remove barriers for effective competition. The definition of participation models for storage is underway in USA markets, but European markets lack specific bidding formats for these types of resources. Although European markets use abstract bidding formats that should not discriminate any resource (meaning, they are equally limited for all types of resources), storage resources face significant barriers.

Since most difficulties have been identified in the European context, this is where the following conclusions focus. Regarding the limits of current bidding formats to represent both thermal and storage resources, a potential solution is to increase the range and complexity of the orders available. However, European markets are already facing computational difficulties, and this approach would most likely fall into scalability issues.

Therefore, the most sustainable approach in the long term would need to both reduce the computational complexity of the clearing problem, and allow more complex bidding formats. This may seem an impossible puzzle, but there are ways in which it can be achieved. First, resource-specific bidding formats, similar to USA’s multi-part bids, can, in some cases, reduce the computational burden if one multi-part bid substitutes a complex combination of block orders. At the same time, resource-specific bidding formats would remove barriers for small market players (current portfolio bidding is advantageous for large players), and facilitate market monitoring.

However, the primary cause for the limitations of European bidding formats is the clearing approach. European markets are based on uniform prices; clearing the market under uniform-pricing constraints complicates the computation of market programs, so the range of bidding formats available is limited to keep the computational burden under control. Alternatively, USA markets use computationally simpler clearing algorithms based on the maximization of market welfare (without pricing constraints). As reviewed in previous chapters, this approach requires an ex-post price computation, with its own challenges, but it would enable the use of increasingly necessary complex bidding formats in the European context.

In conclusion, resource-specific bidding formats are most advantageous, especially when combined with welfare-maximization clearing rules. However, their design has to be regularly reviewed to ensure no resources are discriminated. This is especially challenging when considering future potential energy resources, of which their technical characteristics are yet unknown. Nonetheless, this cannot be strictly considered a disadvantage over European (resource-independent) bidding formats, since these are equally influenced by current resource needs, and also become outdated. For instance, European-bidding formats present several limitations to represent the constraints of storage resources, so their resource-independence cannot be unequivocally considered a positive feature, unless full generality is achieved.