On the Influence of Renewable Energy Sources in Electricity Price Forecasting in the Iberian Market

Abstract

The mainstream of EU policies is heading towards the conversion of the nowadays electricity consumer into the future electricity prosumer (producer and consumer) in markets in which the production of electricity will be more local, renewable and economically efficient. One key component of a local short-term and medium-term planning tool to enable actors to efficiently interact in the electric pool markets is the ability to predict and decide on forecast prices. Given the progressively more important role of renewable production in local markets, we analyze the influence of renewable energy production on the electricity price in the Iberian market through historical records. The dependencies discovered in this analysis will serve to identify the forecasts to use as explanatory variables for an electricity price forecasting model based on recurrent neural networks. The results will show the wide impact of using forecasted renewable energy production in the price forecasting.

1. Introduction

As the question of climate change mitigation comes into the social and political focus, it becomes more urgent to consider important structural questions in regard to our energy system, which is the origin of most greenhouse gas (GHG) emissions [1]. Hereafter, the decarbonization of the energy sector—meaning the need to abandon fossil fuels as the primary source of energy for any type of activity—has become a basic Leitmotif of energy policy at EU and national level and even beyond EU frontiers [2].

The energy sector is divided into three main areas depending on how the demand is satisfied: electricity, heating and cooling and the transport sector, covering, respectively, approximately 25%, 50% and 25% of the total demand [3]. Non withstanding the fact that most of the energy currently is not electrical, the main trends of EU policies are framed by the idea of electrification and the fundamental concept according to which, the nowadays electricity consumer will turn into the future electricity prosumer (producer and consumer) in markets in which the production of electricity will be more local, renewable and economically efficient. This vision raises many questions since the technical characteristics, demand, pricing structures and supply chains of these markets are quite distinct.

In our work we focus our attention on the role of local communities and policy makers at regional, subregional and urban scale in the planning of their energy system taking into account the policy frameworks that come into play in this decarbonization via electrification pathway. Between the future individual prosumers and the large scale energy markets, the local decision makers will have a key role in the facilitation, acceleration or eventual back away of certain policies due to their closeness to the citizens and their regulatory and planning competences in most of EU countries. Moreover, in countries like Spain, local communities are entitled to act as direct players in the electricity pool (known as “direct electricity consumers” and alternatively as “demand aggregators”) [4], and can also negotiate directly in the electricity market. There are different ways local communities can play a direct role in the market as either producers, public utilities (e.g., Barcelona and Madrid) or even managing local scale distribution system operators (DSOs). To characterize this multifaceted role of local communities in the energy market we could talk in this case more accurate about “pubsumers” rather than prosumers, able to heavily influence the future trends in the electricity markets.

In our research group we have been actively collaborating with different local communities in our area to help decision makers in the planning and transformation of their local energy markets. In this line, we have developed and implemented tools to help them forecast and decide upon the best strategies to decarbonize and maximize their system efficiencies. These tools can be divided in four areas: electricity pricing and demand forecasting, calculation of the impact in terms of GHGe emissions of public and non-public activity sectors, planning tools to assess and develop local renewable energy assets (biomass, solar and geothermal) and finally, dashboard and scoreboard tools to help assess the impact (in terms of emissions) of different decisions towards decarbonization under several technical and non-technical criteria [5].

One key component of a local short-term and medium-term planning tool to help pubsumers and local DSOs to efficiently interact in the electric pool markets is their capacity to predict and decide short-term based on predicted prices. In a local market in which further renewable production will be progressively more important, it is also key to analyze the influence on (one-day-ahead) electricity price forecasting of renewable energy production and electricity price through historical records.

As more and more governmental policies wager on renewable energies like hydraulic power, wind power or solar energy, the energy sector is challenged to utilize all these sources pushing forward new research and development. In fact nowadays the increasing concern about environmental issues has led renewable energy to already have a deep impact in the local power markets of some European countries such as Spain, Germany and Denmark. One of these major open challenges to be seized refers to the volatility of the pricing structure when there is an increased share of renewables. Many decisions rely on the forecasting of the energy supply and demand as well as predictions of the prices of electricity, gas, water, and other renewable energy sources. Increased volatility increases economic risk (a specially sensitive topic referring to local public decision bodies) and disincentives the participation of such actors in the markets.

In this paper, we focus on short-term (one-day-ahead) electricity price forecasting and we analyze the correlation between the renewable energy production and electricity price through historical records. The work in [6] uses the amount of wind energy and hydro energy, the most relevant renewable energy sources in the Iberian Market, to come up with an optimal model for one-day-ahead electricity price forecasting. In the Italian electricity market, which is currently facing a low-carbon transition, electricity prices are highly influenced by the generation from renewable sources [7]. In this latter work, a day-ahead predicted wind generation introduced as a regressor in the model enables to capture the peculiarities of the market and thus improve the forecasting ability of the model.

Renewable energy production is also highly dependent on weather and climate, which can in turn affect the feasibility of future low-carbon energy supply systems. In this sense, climate change not only affects our energy consumption habits but also impacts most renewable resources. We can find several studies that estimate the adaptability of high-elevation hydropower generation to climate warming [8], analyze future changes of wind speed and wind energy potentials considering different global and regional climate model chains [9] or evaluate the impact of climate change in the vulnerability of supply systems of solar photovoltaic power generation [10,11]. The impact of climate change on the renewable resources also affects the energy markets on both electricity demand and supply of electricity [12,13].

Another key aspect to understand the role of renewable energies sources (RES) in the electricity markets is the dispatchability of intermittent RES (iRES) into the grid [14], which makes the price of electricity be dependent on the time and location at which it is produced [15]. The intermittent nature of renewable energies has fostered the emergence of aggregation facilities with the aim to facilitate municipalities and corporations to rely completely on renewable energy and become catalysts of the transition to a carbon free environment. As potential electricity generators, corporations are interested in knowing in advance electricity prices for balancing the decision related to the load to produce and inject into the grid [16]. Hence, accurate day-ahead electricity forecasts are also very valuable for a consumer to be able to operate in the grid.

It is important to note that iRES like solar power, wind power or tidal power may be predictable (length of days, weather patterns, tidal cycles), and that the time period that electricity can be dispatched is limited. Consequently, the temporal variation in the electricity production of intermittent and stochastic renewable energy systems requires of systems to restore the supply-demand balance, which also impacts the energy cost. For that matter, the ability to accurately forecast the output of RES is essential. Since the variability and stochastic variation of RES have a cost, an efficient forecasting of the source fluctuation will largely impact future investments in the RES sector [17]. Interestingly, the dominating view that renewable electricity production increases the price variance has been questioned in other contributions. Authors in [18] put forward a static market model which predicts that small to medium quantities of RES bring about a reduction of the electricity price variance while large quantities produce the opposite effect. In general, the impact of intermittent wind generation on hourly equilibrium prices has shown to lead to below-average prices and to a higher price variance in Great Britain [19], Denmark [20], Texas [21] or New England [22]. In the same line, an analysis of the impact of wind and photovoltaic energy on day-ahead electricity prices at the European Energy Exchange reveals that the introduction of these renewable energies enhances extreme price changes and decreases market spot prices [23]. Likewise, price spikes are higher and more frequent in the California system when the photovoltaic capacity is higher [24]. However, the impact of the renewable energy sources on the electricity price in the Dutch electricity market is claimed to be rather modest [25].

It is known for a fact that renewable energies affect electricity prices in one way or another. The impact of RES in different regions is subject to the regulations of the electricity market and the energy production. In this paper we are particularly interested in determining whether the impact of RES is accurately accounted for in the price electricity forecasts in the Iberian Market, a market that is characterized by an important wind, hydro and, to a lesser extent, solar energy production. In order to address this task, we present a non-linear model for price electricity forecasting and we analyze the sensitiveness of the price forecasting to iRES infeeds by studying the relationship between the renewable energy production and electricity price through historical records. More specifically, we will study whether the introduction of forecasted generation of wind and solar energy in the prediction model affects the error rate in the electricity price forecasting. Ultimately, we seek to answer the following questions:

• can we find a correlation between renewable production and spot price? If so, can we find the same correlation with renewables forecasts?
• how does forecast of renewables affect prediction of price? is it a distorting variable or an aligned component for electricity price forecasting?

This paper is structured as follows. The next section presents a brief overview of the most relevant approaches to electricity price forecasting, stressing particularly models that draw on neural networks. Section 3 summarizes the two neural network architecture that we will use to design our prediction model. Section 4 presents a comprehensive analysis of historical data to uncover the potential correlation of electricity price with RES. Subsequently, we introduce the dataset from the Spanish day-ahead market and come up with our proposed model for predicting prices in Section 5. Section 6 shows the results when comparing the performance of various models with and without consideration of forecasted renewables as explanatory variables. Finally, we present a discussion on the potential impact of the obtained results on the energy policies and last section summarizes our contribution.

2. Related Work

In this section we summarize the most relevant approaches to day-ahead and short-term electricity price forecasting, stressing the studies that employ renewable energies in the design of the prediction models.

The literature on electricity price forecasting (EPF) is extensive and diverse. The excellent review work presented in [26] classifies methods of EPF into five groups, among which statistical models and non-linear models stand out from the rest. The former group makes use of classical statistical models and transformations traditionally applied to time-series analysis. Classical methods include many variations of the ARIMA family of models [27,28,29], where the price is modeled as a function of the prices of previous days and the residuals of the forecasts. These models can be augmented to include exogenous variables (ARIMAX) [30]. GARCH models have been used to explain the behaviour of the residuals, specially when price spikes are present [31]. In contrast, methods based on non-linear models put the emphasis on techniques from computational intelligence and machine learning. Different approaches propose using classical machine learning methodologies such as neural networks [32,33,34,35], support vector machines [36], random forests [37] and k-closest neighbours [38]. Genetic algorithms have sometimes been used in order to select the hyper-parameters of those models.

The neural network revolution that is dominating much of the research in artificial intelligence has also proven very successful in EPF. Recurrent neural networks (RNN) have extensively been used in EPF due to their capacity to model long term dependencies. An early investigation that proposes using an Elman network for EPF shows a good capability for predicting price spikes in the Spanish and New York real-world electricity markets [39]. An hybrid combination of RNN and coupled excitable systems have shown to allow close approximation of spiky time series [40]. In a more recent work, authors claim that RNN are the preferred method for time series type problems due to their efficient memory management of previous instances, a crucial factor for estimating electricity prices of the day-ahead market [41]. The combination of RNN with convolutional neural networks (CNN) has also shown to successfully predict the electricity price of the next hour based on the prices of the previous 24 h. Particularly, the combination of RNN and CNN outperforms other traditional machine learning methods like support vector machines, decision trees, random forests or a stand-alone CNN/RNN [42]. Recently, an exhaustive empirical analysis, which compares 27 state-of-the-art methods for predicting electricity prices in an extensive benchmark, concludes that deep learning models achieve a predictive accuracy that is statistically significantly better than any other model [43]. It is also worth noting that this latter work concludes that machine learning methods produce, in general, a greater accuracy than statistical models as well as that RNN-based models are better for predicting prices at some specific hours.

We can safely argue that NN-based models, particularly variants of deep NN, are the currently dominating technology for EPF as these methods have overall shown to outperform statistical methods. However, in rare cases one can see the use of renewable energy sources in the benchmarks used for computing the EPF models. In general, there is common agreement that RES raise several concerns regarding their influence on electricity prices and grid stability, claiming that the high dependency of renewables on weather conditions brings about the volatility in electricity prices. Despite there is no doubt regarding the contribution of RES to build a more sustainable world, the increasing integration of RES makes the electricity market become naturally more unpredictable, which typically leads EPF models to neglect RES forecasts as explanatory variables. It is also the case that RES forecasts are not always available in electricity markets. In this regard, albeit the Iberian market does provide data of renewable energy forecasting, only a few works have actually deemed RES predictions, particularly hourly wind power generation forecasts [35,44,45].

3. Preliminaries

Deep learning is a programming paradigm consisting of a multitude of machine learning algorithms that allows computers to learn from observational data. The models underlying these techniques are known as artificial neural networks (ANN), given that they are loosely inspired by the biology of the brain. A neural network is a collection of interconnected nodes, known as neurons, that process the data they receive and transmit their output to other nodes. ANNs are data driven systems that learn to perform tasks by inferring rules from large amounts of training examples.

In the following, we will describe the two types of ANNs used in this work: feedforward neural networks, and recurrent neural networks.

3.1. Feedforward Neural Networks

Feedforward neural networks (FFNN) are the most common neural networks. In FFNN, neurons are organized in layers, with one input layer, one or more hidden layers, and one output layer. FFNN receive this name because the output of one layer is used as input to the next layer, meaning that information flows only in one direction, from the input layer to the output layer.

In a FFNN, each neuron receives input from neurons in the preceding layer, computes its activation state and outputs it to all the neurons in the next layer. A neuron can be understood as a vector-to-scalar function. Given the input vector $\mathtt{x}=(x_1,\dots ,x_{D_x})$, the activation state of a neuron is computed as:

$$s=f({\displaystyle \sum _{k=1}^{D_x}}{w}_k{x}_k+w_0)$$

(1)

where $w_i$ is the weight associated to input $x_i$, $w_0$ is the bias and f is an activation function.

We can then understand a FFNN as a composition of functions, some of which (usually those in the hidden layers) are non-linear. The purpose of a FFNN is to approximate some function. In fact, FFNNs are considered universal function approximators, since any function can be arbitrarily approximated using enough neurons.

3.2. Recurrent Neural Networks

Neural networks can also include feedback connections, in which case they are named recurrent neural networks (RNN) [46]. These feedback connections are represented by loops in the network that indicate the influence of the present state of a neuron on its own state and on the states of other neurons in a future time step. The influence over future inputs can be seen as a memory which allows recurrent networks to learn long-term dependencies. This feature makes RNNs ideal for processing sequences of vectors $\langle {\mathtt{x}}^{\left(1\right)},\dots ,{\mathtt{x}}^{\left(T\right)}\rangle$.

Figure 1 shows an example of a recurrent neural network with three neurons in the hidden layer that takes as input a sequence of vectors of dimension 2, $\langle (x_1^{\left(1\right)},x_2^{\left(1\right)})\dots (x_1^T,x_2^T)\rangle$. The presence of feedback connections in this network means that to compute the output at time t, the neurons take into account the current input vector $(x_1^{\left(t\right)},x_2^{\left(t\right)})$ as well as the output of the last time step, $(s_1^{(t-1)},s_2^{(t-1)},s_3^{(t-1)})$.

Hence, the output of a neuron $s_i^{\left(t\right)}$ at time t in a layer with $D_s$ neurons and $D_x$ inputs is computed as:

$$s_i^{\left(t\right)}=f({\displaystyle \sum _{k=1}^{D_x}}{w}_k{x}_k^{\left(t\right)}+{\displaystyle \sum _{k=1}^{D_s}}{u}_k{s}_k^{(t-1)}+w_0),$$

(2)

where $w_i$ is the weight associated to input $x_i$, $u_i$ is the recurrent weight associated to output $s_i^{(t-1)}$, $w_0$ is the bias and f is an activation function.

Recurrent networks usually employ complex memory cells instead of the previously explained neurons. The most popular cell is the long short-term memory (LSTM) [47]. LSTM cells contain an internal self-loop that adds another level of recurrence to the RNN and allows LSTM networks to learn long-term dependencies more easily than with simple neurons.

4. The Day-Ahead Iberian Electricity Market

This section is devoted to delving into the functioning of the day-ahead Iberian electricity market and to analyzing the historical series of electricity prices over the period 2014–2016. We are interested in examining the relationship between the electricity price and the actual values of variables for which the Iberian electricity market provides forecasts, namely electricity demand, and solar and wind energy production. Specifically, our purpose is to gain insight on the influence of these variables in the electricity price so as to determine whether it is positive introducing the forecast for these variables in the hourly-ahead EPF model. Ultimately, we aim to inspect the role of renewable energies in the electricity price and thereby the convenience of using the forecasts of clean energies in the forecast of electricity price.

The section is structured as follows. In Section 4.1, we outline the operation of the Iberian market and we describe OMIE, the Spanish market operator that manages the wholesale electricity market on the Iberian Peninsula. The following subsection provides a comprehensive history of the correlation of the electricity price with respect to electricity demand, solar energy generation and wind energy generation. Moreover, we also studied whether a similar correlation exists between the electricity price and the forecasts for these three variables.

4.1. Market Description

Due to the nature of electricity, markets are held in advance of the actual consumption in energy, in order to allow the grid operator to plan and prepare for any problem or imbalance that might arise. This is why many markets are referred as day-ahead markets, as auction is held one day before the actual electricity is consumed. This specific feature introduces an additional challenge to the forecasting problem, as many of the variables that determine the actual demand and generation of electricity for each trading period of a given day are not known at the time the auction is held. Forecasting systems have to rely on other forecasts of those variables or use lagged values in order to be able to make good predictions about the price. The problem then, consists in forecasting the electricity price for the 24 one-hour periods in a day, using only the information available up to the time of the auction, held in the previous day.

OMIE, the Spanish market operator [4], oversees and manages both day-ahead and intraday electricity markets. These markets are where both generators and consumers of electricity submit their bids in order to be able to trade. The day-ahead market is held every day, and market participants have up to 12 PM to submit their bids for each of the 24 periods (each comprising one hour) of the next day. Participants can submit a bid only stating how much electricity they want to trade and at what price, or they can also attach specific restrictions to their bids such as a minimum income condition (the offer should be considered only if the seller obtains a minimum amount of money). These types of bids are known as simple bids and complex bids, respectively.

Once all bids have been collected, the market operator carries out the procedure of determining the final price. The Spanish market is a type of two-sided auction, where bids are aggregated and sorted by price. For each period, purchasing bids are sorted by descending price (higher prices have preference), while sale bids are sorted by ascending price, with priority given to lower prices. The final price is fixed by the intersection of this two curves, where there is a purchase bid whose price is lower than the remaining sale bids. This price, known as the market clearing price or spot price, is the price to be paid by all quantities of electricity traded, no matter what the original bid was. Therefore, all accepted purchase bids (with an initial bid higher than the market clearing price) will have to pay the same quantity per each energy unit (currently given in MWh) bought, and all sale bids (with an initial bid lower than the market clearing price) will receive the same quantity per each energy unit sold. The finer details of the price coupling algorithm can be consulted on the website of the OMIE [48].

4.2. Analysis of Historical Data

The system operator, Red Eléctrica de España (REE), publishes daily forecasts for wind and solar generation, alongside a demand forecast for the next day. We hypothesize that these forecasted variables will be very useful to our goal of developing an EPF model sensitive to renewable energies. Forecasts of wind and solar are of particular significance as their historical data are less reliable due to the intermittent nature of these energy sources. The purpose of this analysis is to gain some insight on the suitability of these forecasts for our EPF model. To this end, our first step is to verify whether a relationship between solar and wind generation, demand, and electricity price actually exists along the historical data.

The figures presented in this section aim to analyze the correlation between hourly generation/demand and electricity price. If a positive or negative trend is identified, it means that a relationship exists between the two correlated variables, with stronger correlations being represented by more clearly identifiable trends. A positive correlation indicates that both variables increase/decrease in the same direction, while a negative correlation means that when one variable increases the other decreases.

We start the analysis by looking at our two sources of renewable energy, solar and wind. The correlation analysis between wind generation and electricity price (Figure 2 (top)) shows a negative trend, meaning that more wind generation is usually accompanied by lower prices. This correlation is of particular significance given that wind generation represents an important share of the electricity generation, and so the impact is bigger. Solar generation, on the other hand, represents a very small share of the energy generation in the Spanish market. Comparing Figure 2 (top, bottom), we can see that the maximum solar generation is about one third of the maximum wind generation. Furthermore, the highest concentration of hotspots are between 0 and 1000 MWh, meaning that there is a very small amount of solar generation or there is no generation at all (this corresponds to hours of the day with very little sunlight or night hours). Despite all this, it is still surprising that no correlation between the solar production and the electricity price is found.

Apart from power generation, another important factor that affects the price of electricity is the demand. The correlation analysis of Figure 3 shows that, as expected, a higher demand entails higher prices. We find, however, that this is not a particularly strong correlation.

The obtained results led us to consider the definition of a new variable to better capture the dynamics of the offer and demand system in the day-ahead market. Bearing this in mind, we defined a new variable, which we will refer to as ratio of renewable energy production, using the aforementioned studied variables. The ratio is computed by adding the wind and solar generation and dividing the sum by the demand. The correlation for this derived variable (Figure 4) shows a stronger correlation than the correlation previously identified for the single variables. This result indicates that the electricity price is more affected by the composition of the energy generation than by the amount of energy being generated.

Having confirmed the influence of these variables on the electricity price, we now proceed to verify that the forecasted variables provided by REE display a similar behavior. Figure 5 shows a comparative analysis between the real historical data (left) and their forecast counterparts (right). We can see in the figure that the plots are mostly identical, so the conclusions we have drawn from the historical data also apply to the forecasts. Therefore, we conclude that the forecasted variables will be useful input explanatory variables for our EPF model.

5. Methodology

In this section, we present our proposed model for electricity price forecasting. Prior to this, we introduce the dataset from the Spanish day-ahead market that will be used.

5.1. Dataset

We collected a number of explanatory variables to be used as input for our forecasting model. Each entry in the dataset corresponds to an hour h in the two year period for the years 2014 and 2015. The variables collected, summarized in

Table 1. Summary of explanatory variables for a given hour h.

Variable	Category	Description	Range
V1	Day-ahead prices	$p_{h-24}$	0–110 €/MWh
V2	Day-ahead prices	$p_{h-48}$	0–110 €/MWh
V3	Day-ahead prices	$p_{h-168}$	0–110 €/MWh
V4	Time and calendar	Hour	0–23
V5	Time and calendar	Week day	0–6
V6	Time and calendar	National holiday	0–1
V7	Forecasted generation and demand	Forecasted solar generation for h	0–3650.2 MWh
V8	Forecasted generation and demand	Forecasted wind generation for h	277–17,385 MWh
V9	Forecasted generation and demand	Forecasted demand for h	17,599–40,050 MWh
V10	Derived variable	forecasted ratio of renewable energy for h	0–1

, are the following:

• Day-ahead prices: the hourly prices of previous days are the basic input of almost all electricity forecasting systems. Various works have identified correlations between the prices of day d and the previous prices of days $d-1$, $d-2$ and $d-7$, reason why time lagged prices are regularly used in the EPF models presented in the literature [49,50]. Therefore, in order to predict the electricity price for a given hour h, $p_h$, we used the lagged prices $p_{h-24}$, $p_{h-48}$ and $p_{h-168}$. This information was published each day after the day-ahead auction by the market operator, OMIE [4].
• Time and calendar: the hour corresponding to each entry in the dataset was registered using an integer variable $h\in [0,23]$. Additionally, we provided two other variables with calendar information, an integer $d\in [0,6]$ for the day of the week, and a boolean $holiday\in \{0,1\}$ to identify the existence or non-existence of a national holiday that day.
• Forecasted generation and demand: as a result of the findings of Section 4.2, we includeD the hourly forecast solar and wind power generation, as well as the hourly forecast demand. The methodology used for the demand forecast was explained in [51]. All three forecasts were provided by the system operator, REE, and can be obtained through its information system, e·sios (Sistema de Información del Operador del Sistema https://www.esios.ree.es/es).

Additionally, we include as input the forecast ratio of renewable energy computed using the forecasts for wind and solar generation, and demand:

$$\frac{\mathrm{forecasted}\mathrm{wind}\mathrm{generation}+\mathrm{forecasted}\mathrm{solar}\mathrm{generation}}{\mathrm{forecasted}\mathrm{demand}}$$

(3)

5.2. Proposed Model

The proposed model assumes that the price for a certain hour $p_h$ can be defined as the addition of two terms, $p_h={\tilde{p}}_h+{ϵ}_h$, where ${\tilde{p}}_h$ is an estimation of the price, and ${ϵ}_h$ is the error. Following this assumption, we can approximate $p_h$ by estimating ${ϵ}_h$, i.e., $p_h\approx {\tilde{p}}_h+{\widetilde{ϵ}}_h$. Our forecasting model followed this approximation and used two artificial neural networks to compute these two terms (see Figure 6).

The main network was a RNN with a hidden layer of 128 LSTM cells, which was used to compute ${\tilde{p}}_h$. The input for this network was modeled as a sequence of 24 vectors ${\mathtt{x}}^{(h-23)},\dots ,{\mathtt{x}}^{\left(h\right)}$, where each vector ${\mathtt{x}}^{\left(h\right)}$ contains explanatory variables for that one-hour period h. The secondary network, a FFNN with a single hidden layer of 64 neuros, used past residuals of the recurrent network to compute ${\widetilde{ϵ}}_h$. The residuals were computed as ${ϵ}_h=p_h-{\tilde{p}}_h$. This network took as input a vector ${ϵ}_{h-24},\dots ,{ϵ}_{h-168}$ with the last seven residuals corresponding to the same period of the day. The forecast price ${\widehat{p}}_h$ can then be expressed as:

$${\widehat{p}}_h=f_{RNN}({\mathtt{x}}^{(h-23)},\dots ,{\mathtt{x}}^{\left(h\right)})+f_{FFNN}({ϵ}_{h-24},\dots ,{ϵ}_{h-168}).$$

(4)

The addition of the output of these two networks went through a filter that ensured the forecast price fell within the limits established by the electricity market. In the case of the Iberian market, the applied filter was defined as follows:

$${\widehat{p}}_h=\left\{\begin{array}{cc}0\hfill & \mathrm{if}{\tilde{p}}_h+{\widetilde{ϵ}}_h<0\hfill \\ 110\hfill & \mathrm{if}{\tilde{p}}_h+{\widetilde{ϵ}}_h>110\hfill \\ {\tilde{p}}_h+{\widetilde{ϵ}}_h\hfill & \mathrm{otherwise}.\hfill \end{array}\right.$$

(5)

The two-network architecture of our proposed model was inspired by the ARIMA model, an univariate forecasting method where the forecast was given by a linear combination of historical data and a linear combination of past prediction residuals. ARIMA models have a proven track record in time-series forecasting, but are known to struggle in electricity price forecasting. The main reasons behind this behaviour is that ARIMA models do not support time series with a seasonal component, and they are also limited to linear dependencies in the data. Our proposed model addresses the shortcomings of ARIMA models in the following ways:

• Exogenous variables: the proposed model supported an arbitrary number of exogenous variables, i.e., data outside the price time series. This allowed the inclusion of other explanatory variables, such as the ones presented in Section 5.1.
• Non-linearity: neural networks were able to learn nonlinear dependencies between variables.
• Seasonality: LSTM cells were particularly suitable for learning the order dependence of seasonal data.

All these advantages came at the cost of a far more complex system, which required more training data to learn all the additional parameters. Fortunately, EPF datasets contain full years of sampling data available.

6. Experimental Results

In this section we validate the proposed EPF model in the real-world dataset presented in Section 5.1. We are also interested in measuring the improvement that a forecasting model sensitive to renewable energies poses over one that is not, if any. With that aim, we defined three models according to the inputs used: M1 uses only lagged prices and chronological variables, M2 incorporates forecasts of generation and demand, and M3 adds the forecasted ratio of renewable energy. The input configurations of each model are summarized in

Table 2. Input configurations.

Variable	Description	M1	M2	M3
V1	$p_{h-24}$	√	√	√
V2	$p_{h-48}$	√	√	√
V3	$p_{h-168}$	√	√	√
V4	Hour	√	√	√
V5	Week day	√	√	√
V6	National holiday	√	√	√
V7	Forecasted wind generation for h	-	√	√
V8	Forecasted solar generation for h	-	√	√
V9	Forecasted demand for h	-	√	√
V10	Forecasted ratio of renewable energy for h	-	-	√

.

The models were tested on four sample weeks, one for each season in the year 2015. The periods for each week are detailed in

Table 3. Sample testing weeks used for the evaluation.

Week	Period	Season
W1	2 February–8 February	Winter
W2	4 May–10 May	Spring
W3	3 August–9 August	Summer
W4	2 November–8 November	Autumn

. Models were trained using a 16-week rolling window. This means that models were trained once for each day in the testing week, using the prior 16 weeks as training set. Training was stopped for the three models after 200 epochs.

The error of the forecasted prices for the sample weeks achieved by each model was analyzed using the mean absolute percentage error (MAPE) metric, defined as:

$$\mathrm{MAPE}=\frac{100}{N}{\displaystyle \sum _{i=1}^N}\frac{|p_{h_i}-{\widehat{p}}_{h_i}|}{p_{h_i}},$$

(6)

where $p_{h_i}$ is the real price, ${\widehat{p}}_{h_i}$ is the forecasted price, and N is the number of forecasted values (N = 168 if we are forecasting one week).

The results of this error analysis,

Table 4. MAPE results.

Week	M1	M2	M3
W1	24.22	9.99	9.83
W2	24.72	9.93	9.13
W3	7.51	5.07	5.01
W4	12.17	7.23	6.69

, show that both M2 and M3 provide a significant improvement with respect to M1 across all weeks. These results evidence the usefulness of the forecasted variables (V7–V9) in the EPF models. We also find that the inclusion of V10 (forecasted ratio of renewable energy) helps in reducing the error, thus highlighting the importance of knowing the composition of the generation of energy when forecasting prices.

In order to better understand how the additional inputs are helping the EPF models, we have plotted the prices forecasted by each model for the sample weeks (Figure 7, Figure 8, Figure 9 and Figure 10). Starting out with W1 (Figure 7), we found that M2 and M3 usually adjust better than M1, with M3 providing a slightly better refinement over M2. M1 seems to have a harder time forecasting peaks and valleys in the price curve.

In contrast with the fairly uniform price curve of W1, we find a very different picture in Figure 8, where each day is very different from the day before. This heterogeneous curve highlights the limitations of M1 to adapt to sudden changes. M2 and M3, on the other hand, are still able to react and reproduce a price curve that fits much better with the real one.

In W3 (Figure 9), we found a very stable behavior during the weekdays with a sudden drop starting Saturday. It was of no surprise then, that this was the week where all models were able to obtain their best results. In this case, we saw that even M1 can reproduce the behavior of the price curve, but still falls short of M2 and M3. Once again, most of the errors of M1 corresponds to peaks and valleys in the curve.

In the last sample week, W4 (Figure 10), we verified what we have been observing so far. While M2 and M3 mostly overlapped with the price curve, we found M1 often forecasting below or above the real price during the daily maximums and minimums.

Overall, we found that M1 expected a more uniform behavior in the prices and has trouble adapting to changes. M1 also shows difficulties in forecasting prices in the peaks and valleys of the price curve. On the other hand, the EPF models sensitive to renewable energies, M2 and M3, were able to exploit the additional data to react faster to sudden changes and provide a more accurate forecast.

7. Discussion

As our analysis points out, there is a lack of correlation between electricity spot prices and solar PV gross electric energy generation, as shown in Figure 2 (bottom). In contrast, the electricity generated by wind is much more clearly related to prices. It is furthermore clear from the figures that the total amount of wind energy produced is significantly higher than solar, and therefore its expected impact on the markets.

The comparatively lower shares of solar photo-voltaic electricity in Spain has its origin in the unstable regulatory framework that has been a key market factor during the last decade. Before year 2009, the initiative to foster solar PV electricity generation led to a paramount increase in installed capacity—with Spain being no 1 in the EU during several years in this regard. The incentive scheme chosen by the central government, based on a favourable feed-in-tariff for PV, led to a bubble effect, which in year 2008 was retroactively stopped by means of a Royal Decree [52]. This caused a complete stop in new installations and even the shut down of existing plants. In the case of wind, the regulatory framework was more stable, the technology was at that time comparatively more mature and the largest projected plants were in operation at the time regulation stopped new developments. One of the arguments held by responsible officials to justify the Royal Decree was the observation that renewable energies increased electricity prices. Our study shows that this is indeed not the case. Even more, with about 80% energy dependence from foreign fossil fuel sources (gas, petroleum, …) what is clear is that Spain benefits from the price lowering effect of the already installed RES electricity producing infrastructure.

Recently, the government has embarked a new regulation that establishes a simplified mechanism for compensation of self-produced energy and non-used energy [53], thus putting an end to the so-called ‘sun tax’. Since the new Royal Decree approves the collective self-consumption to favour both households and small businesses, a positive effect such as lower prices of the electricity bill are expected. Boosting self-consumption will also benefit the electrical and energy system by augmenting the production of solar energy. Ultimately, we envision that the evolution of the electricity price could show a trend similar to that of the wind generation in Figure 2 (top) once there is a significant increase in net solar PV electricity production.

Apart from the regulation policy of the energy market, our study clearly confirms that the higher the ratio of renewable energy production per demand is, the lower the price of the electricity is, even considering the irregular production of solar energy. When the ratio variable is included in the EPF model, we observe a dramatic improvement in the results of the predictor over a model that disregards the forecast renewable energy production and demand. The conclusions of our study will serve to encourage governments to emit the forecasts of renewables so as to be included in the price forecasting models.

As stated in the Introduction, there is a vast body of literature on the exploration of different approaches for forecasting of electricity prices. EPF models are relevant as they allow participants of the energy market and power pools to maximize profit and make decisions on accurate risk measures. What we put in value in this paper is the role of renewable energies in the electricity price forecasting in the Iberian Market. Our study reveals that the inclusion of such energies not only outputs a significant reduction in the forecast price error but also shows the accuracy of the forecast curves to capture seasonality and price spikes. It is certainly true that the position of the energy markets of different countries on the inclusion of iRES production in the EPF is different, and that not all the markets issue the forecasts of renewable energy production. Nevertheless, the revolution toward the so-called green industry is pushing forward a market demand for renewable energy, aiming to lower energy prices and mitigate the climate change effects.

8. Conclusions

Electricity price forecasting is a key component for making short-term decisions in markets where the production of energy tends to be more local, renewable and economically efficient. This works focuses on the role that renewable energies play in the electricity price, a crucial factor in understanding and enabling the next phase of the energy transition. First, a strong correlation between the generation of renewables, particularly wind energy, as well as electricity demand with the electricity price was found. Additionally, our findings also show an almost identical correlation with the forecast values of renewables production and demand. This led us to hypothesize that using the forecast renewables in the prediction of electricity price would exhibit a good performance. To this end, we designed a NN-based EPF model that relies upon a two-network architecture, a RNN to process the input vectors of explanatory variables and a FFNN that uses the residuals of the RNN to estimate the error. Our work features one of the first attempts in the use of a combination of predicted wind generation, predicted solar generation and predicted demand of electricity to improve the EPF performance.

The results are highly conclusive as to the influence of the renewable energies in the forecasted price. We observed an error reduction of up to 14 points when the forecasted renewables are considered in the model. Moreover, the predictor yields even even slightly better results when using a derived variable that represents the ratio of renewable energy production. All in all, our study reveals that the composition of the energy production plays a crucial role in the determination of the electricity price. We believe the analysis presented in this paper will help decision makers in the planning and transformation of their local energy markets. We must though also point out that the performance of our EPF model is subject to the quality of the forecasted variables, which gives an indication of the importance that governments include robust estimates of renewable energies in their periodic issuances.