Adaptation of High Spatio-Temporal Resolution Weather/Load Forecast in Real-World Distributed Energy-System Operation

Abstract

Despite significant advances in distributed renewable energy systems (DRES), the technology still faces several substantial challenges that prevent the large-scale adoption of these systems into a country’s energy sector. The intermittency of renewables, uncertainties associated with real-time multi-horizon weather and load forecasts, and lack of comprehensive control systems are among the main technical and regulatory challenges for the real-world adoption of DRES. This paper outlines the current state of knowledge in the real-world operation of DRES and also describes pathways and methodologies that enable and facilitate the uptake of DRES in a country’s energy sector.

1. Introduction

Due to the anticipated rapid electrification of industry and transport, and the growing population, recent projections show a significant increase in electricity demand by 2050, e.g., 10–60 TWh in New Zealand [1], and about 50% worldwide reaching 45 trillion kilowatt-hours (kWh) [2]. Renewable sources will play a major role in global electric power generation, providing about 50% of total demand by 2050 [2]. The current centralised infrastructure will not meet this demand and building new power plants and associated transmission and distribution lines is expensive and unsustainable. In addition, “… conventional power grid networks are obsolete due to its difficulty of control and protection when many numbers of distributed generations are used” [3]. Thus, alternative sustainable smart technology is needed to meet future energy demand. Microgrid (MG) technologies can viably replace conventional electricity networks to cope with increases in energy demand while addressing the need for dependability arising from the rapid growth of the power system [3]. The technology of the physical components of MGs has matured in recent years, including photovoltaic systems (PV), natural gas-fuelled combustion turbines, fuel cells and batteries, and the benefits of the integration of MGs into energy systems have been demonstrated [3,4].

The adoption of distributed renewable energy systems (DRES) is a potentially transformative solution to achieve government targets for renewable electricity generation. DRES uses distributed micro-scale solar and wind generation to meet local energy needs with zero emissions. However, there are vital scientific knowledge gaps about micro-scale renewable energy resources and the design and operation of DRES within a country’s national grid. Despite their recognised potential benefits [4], national uptake of MGs has been slow in the real world. The slow uptake is due to a lack of real-time implementation and commercial use of MGs, along with concerns around MG system stability, reliability and operational control [3].

There are considerable barriers to the real-world implementation of DRES [5,6,7,8,9,10]. This situation results from a lack of knowledge about where and how DRES would best be located and operated for optimal integration with national grids and operated under the full range from normal to emergency/contingency conditions [11]. Because they rely on wind and solar generation, efficient DRES design and performance requires accurate knowledge of large-scale and micro-scale climate conditions [12].

This review paper mainly focuses on two main areas:

(a)

How to best employ/integrate high spatiotemporal resolution weather and load forecasts in the design and real-time operation of DRES;

(b)

Embedding comprehensive control systems in DRES that enhance real-world operation, resiliency, and reliability of the microgrid system under normal, emergency, and contingency conditions.

Research in area (a) highlights the methodology and science required to determine DRES potential and operation at microscales. This will demonstrate specifically where and how micro-scale DRES would optimally be incorporated into the national grid. Key components include high-resolution numerical weather prediction (NWP), bioclimate mapping, and control algorithms. These would not only provide design and operation specifications for DRES with various capacities and account for variations in climate and geography settings across a country but also provide input to the control system to optimize the real-time operation of DRES.

The second research area (b) concerns determining practical adoption pathways for DRES in a country’s energy sector. This involves the development of a leading-edge agent-based model (ABM) of the behaviour of regulators, technology developers, distributors, and consumers, incorporating the physics-based model of the electricity system developed in task (a). This advanced ABM will allow:

• The identification of optimal location-based energy system designs, considering current regulations and future changes.
• The simulation of DRES operation within the national grid under various real-world weather, consumer behaviour, and load scenarios and
• An undertaking of cost-benefit analysis, option-analysis and multi-criteria-analysis [13] to qualitatively evaluate new policy and regulatory approaches to accelerate the participation of DRES.

2. Gaps and State-of-Knowledge

To address the barriers to DRES adoption listed in Section 1, most of the recent MG research work e.g., [14,15], has focused on the development of more robust and resilient control systems to enable real-world MG operation, especially given the challenges of tackling variable generation from renewables coupled with demand intermittency.

In Section 2.1, Section 2.2, Section 2.3, Section 2.4 and Section 2.5, we summarise several recent research works [3,16], that have identified major shortcomings that presently prevent large-scale adoption of MG.

2.1. Intermittent Nature of Renewables and Demand Forecasting

Renewable energy sources such as wind and solar power are inherently variable and uncertain, making it challenging to model and predict their behaviour. For the purpose of electricity markets, renewable MG systems (without energy storage systems) are categorised as non-dispatchable units because the power generation is volatile and intermittent [17] and thus not always available for dispatch to the market. This intermittency imposes even larger uncertainties at micro-scales. Both under and over power generation from DRES can have a significant negative impact on reliability [18]. The high penetration of renewable generations over the last few decades has made forecasting and scenario generation necessities to overcome renewables intermittency and mitigate uncertainties in generation/demand forecasts [19]. Accurately modelling the interdependence between renewable energy sources and electricity demand is crucial for effective microgrid planning and operation.

One approach to improve the reliability of volatile and intermittent generation in an MG is the design, optimisation and control of energy storage systems (ESS) [20,21]. MG systems with high penetration of renewable sources would potentially require a very large ESS capacity to alleviate the effects of the intermittent behaviour. This intermittency, or in general, variability, occurs at different timescales or bandwidths. While various ESS technologies exist (to satisfy the capacity and bandwidth requirements), the system designer can determine the optimal ESS combination based on the local and global constraints and specifications of the system [22]. It should be mentioned that although energy storage systems are still relatively expensive, the cost of these systems has been decreasing as technology advances and production scales up. As a result, energy storage is becoming more accessible and affordable, paving the way for a more sustainable energy future [23,24].

In addition, as demonstrated in [25], there are advantages when employing both main MG reactive and proactive operational strategies. The reactive strategy makes operational decisions based on the current operation of the system and does not require any weather/load forecasts. However, the proactive methods rely on forecasts of power generation and demand.

Meteorological conditions determine the weather-reliant renewable resource availability as well as MG users’ demand [12]. Thus, they are essential in establishing optimal operation strategies and energy scheduling. “Despite this, weather information plays a secondary role in most of microgrid studies … because the main focus is on the electrical, computational or economical aspects of the problem” [12]. The unrealistic assumptions in most MG studies from 2014 to 2018 (190 works), such as perfect and determined forecasts, are reviewed in [12]. These studies use synthetic data, historical data, local station measurements and external forecasts. In addition, the lack of including extreme events (high/low temperatures), sufficient and high spatiotemporal resolution weather/load forecasting, and seasonal/annual variations are not assessed and is another gap in the current models.

Reference [19] presented a review of the different approaches used for modelling variable renewable energy sources and highlights the importance of scenario generation in capturing their complementarity and spatial-temporal dependence. These modelling approaches include statistical models, machine learning models, and hybrid models. Although these approaches can improve forecast capabilities in renewable MGs, they rely on available historical data and relatively coarse generation NWP models that are not suitable for capturing the high spatiotemporal variability of renewable generations and energy demand at micro scales. In addition, as concluded in [19], the accuracy and computational efforts of these methods have not yet been evaluated in large power systems.

Therefore, implementing accurate and detailed multi-horizon weather/load forecasts [26,27,28] into the operation of MGs will improve efficiency, and ensure reliable and resilient performance [29]. This will require substantially increased resolution and precision over currently available forecasting methods. The large uncertainties in current approaches are due to their reliance on synthetic climate forecasts derived from historical climatology [12], typically using just one climate variable and, also assuming deterministic (static weather/load patterns derived from historical series which do not change much from day to day) demand and/or forecasts [30].

High-resolution multi-horizon weather/load forecasts have not been used in operational MG control systems and operations. Unlike previous research, both Numerical Weather Prediction (NWP) and the electrical aspects of MGs should be brought together (Figure 1) to maximise the real-world functionality and operability of MGs. Therefore, novel forecast techniques, which incorporate sub-km to very coarse NWP models as well as ensemble (probabilistic) forecast into MG operation, should be employed to minimise the impact of renewable intermittency on MG real-world operation. These techniques are elaborated on in Section 3.1.

2.2. Comprehensive Management Systems

“A significant challenge of MG implementation is developing comprehensive control methods to ensure efficient, stable, and reliable operation” [31].

Although the MG is connected to the main grid, a specific design is required to ensure the reliable and resilient independent operation of the system [31]. Some of the main advantages of MGs, and which incentivise consumers/distributors/regulators to adopt them, are that MGs can provide higher resiliency, reliability and power quality for consumers, lower costs, and carbon-free generation [4,32]. In addition, contingencies and outage causes, such as equipment failures and natural hazards, can significantly impact reliability and resiliency by increasing the average frequency and duration of interruptions.

Therefore, research effort has been directed towards designs for the master comprehensive control, to assist with making the decision of when to switch between interconnected and island modes to minimise the impact on users. Some available methods use short-term linear optimisation of power-plant dispatch, but these methods have limited real-world application because they cannot consider the strategies of boundedly rational actors, for example, prosumers and policymakers [33].

In contrast, some other methods apply long-term optimisation algorithms to determine the energy system dispatch strategy. However, due to a strong assumption of having perfect forecasts of stochastic variables (such as demand or renewable energy sources), this method is also not applicable to real cases [33,34].

2.3. Control of Voltage, Frequency and Power Quality

A growing application of voltage-sensitive loads requires higher power quality to prevent harmonics and voltage issues in the operation of power-system-connected devices. By enabling local control of frequency, voltage, load and rapid response from storage units, MGs offer an efficient solution for addressing power quality concerns [4,7].

Through this aspect of MG research, several compensation techniques for power quality in MGs have been proposed, including the autonomous control method for DC MGs [35], a control and grid-interfacing power quality compensator for single- and three-phase MGs [36], and embedding power quality conditioning capability into voltage inverters of MG using for instance frequency/sequence selective filters [37]. There are several harmonic reduction methods proposed that can be generally categorised as passive (filter high-order harmonics) and active (filter any order harmonics) power filtering techniques [3].

In this investigation it is suggested that the next breakthroughs in this research field will come from the study of wide area harmonics control and also power quality improvement units that can be implemented in real-world power systems.

2.4. Integration in National Grids, Physical Peer-to-Peer Arrangement and Real-World Implementation and Operation

Effective grid integration and power sharing management strategies, through robust peer-to-peer arrangements, play a decisive role in enabling the smooth functioning of MGs either in an island or grid-tied mode [3,38]. If not engineered carefully, MG integration with the main grid can potentially have many adverse system impacts related to protection, control, power quality and reliability, and restoration time after an outage [16,39]. In addition, switching from island to grid-tied mode could cause power imbalances [16].

The integration issues would prevent the otherwise significant operational advantages that microgrids (MGs) could bring to the power sector, such as improved stability of the power network, increased efficiency due to reduced network losses, carbon-free power generation, operation in both grid-tied and autonomous island modes, and increased resilience (MGs provide back-up supply should the main grid fail).

Accordingly, many studies have attempted to address the recognized MG grid integration challenges. However, there are critical simplifications and assumptions that limit the applicability of these studies to MG implementation in real-world conditions.

These idealised and strong simplifying assumptions include neglecting: energy resilience design [40,41], generation analysis [42], new generation sources adaptability [42,43], MG holistic control [44,45], system modelling [46,47] and voltage limit violations [48]. Some of the other idealistic assumptions in previous research work include: ignoring/capping export limits [49]; assuming deterministic demand and/or forecasts [30] and ideal peer-to-peer arrangements by focusing only on either the generation side [50], the load side [51,52] or the generation-demand combination without considering other elements such as batteries and intermittent renewables [53]. Research into the resilient operation of a power system, which includes distributed generation, must account for peer-to-peer transactive arrangements, but this aspect has been missing in most research studies [48].

Thus, considering the vital importance of the real-world operation, an important aspect of the cutting-edge research effort should be the demonstration of the performance of developed control and peer-to-peer algorithms under real-world conditions in the island and grid-tied modes and without idealistic assumptions. However, to date, real-world MG case study demonstrations, such as the CERTS microgrid [54,55] and microgrid in the Illinois Tech Campus [56], have been rare, and even these were limited in scope and operation mode [57].

2.5. Economic Aspects

Much recent research effort, e.g., [58,59], has been directed toward a thorough assessment of the economic benefits of MGs, to provide information that incentivizes consumers and energy sectors to implement them. These studies have shown that generating energy in the proximity of consumers brings significant benefits in terms of eliminating transmission/distribution (T&D) payments and losses, which not only reduces the costs for users of MGs but also could potentially benefit the entire energy sector by reducing the T&D network congestion and enabling a better economic dispatch of available energy sources [4]. Other economic benefits of MGs are achieved through MG scheduling aiming to minimise the operation costs of local MGs.

One aspect of the cutting-edge of research in this field is focussed on economic evaluation and feasibility of MG operation in real-time through transactive peer-to-peer arrangements. Although this requires further investigation, especially in market services [60,61] and also market design [50,62,63,64], addressing the main technical barriers (i.e., Section 2.1, Section 2.2, Section 2.3 and Section 2.4), which enables real-world operation and large-scale adaption of MGs, is a higher priority.

3. Potential Solutions

Novel approaches are required to transform electricity sectors to meet the Government’s target of 100% renewable generation by 2030 [65]. This transformation requires phasing out the electricity generation still coming from fossil fuels. Its replacement is expected to come from growth in wind and solar. However, conventional wind/solar generation requires significant land area and the construction of expensive new transmission lines. Moreover, present transmission and distribution losses make up a considerable fraction of the total electricity generation in a country. For example, in Aotearoa-NZ these losses amount to ~3000 GWh annually, enough to supply electricity to over 415,000 four-person households [66].

Based on the science and technical gaps [3,4,16,67] currently preventing the large-scale adoption of DRES in the energy sector (Section 2), this study argues that research focusing on addressing the below topics could significantly contribute to the large-scale adaption of DRES:

• Intermittent nature of renewables and demand load at micro-scales.
• Lack of efficient and novel comprehensive control systems.
• Integration of DRES into the main grid.

Importantly, a case study demonstration of the performance of the MG system and control algorithms in real-world conditions will contribute to the adoption of MG in both island and grid-tied modes, considering the rarity of MG field demonstration internationally [3]. All these components and scientific advances are interconnected (Figure 1) and are required to be considered simultaneously to achieve a solution for the real-world implementation of DRES.

This research does not focus on the economic aspects of DRES, despite there being a need for more studies in this area. Numerous existing studies have demonstrated significant economic benefits of MGs, not only for the users of MG but also for the whole energy sector. In this study, it is anticipated that by focusing on the three areas listed above, the groundwork will be laid for future economic studies, including comparative cost/benefit analyses of transactive agreements between users and the main network.

Figure 2 shows the layout of the energy system and its interaction with other subsystems. This study considers layers 1, 2 and 3 and recommends accurate analyses of interconnected layers. However, it is vital to model and investigate the effect of other exogenous systems, i.e., socio-economic and policy/regulations, on the electrical system in the design and operations stages.

3.1. Tackle Renewables Intermittency

Most previous renewable resource assessments have been based on observational data from meteorological monitoring [68] (e.g., sodar, masts, and lidar), or rely on synthetic climate forecasts derived from historical records [12]. Even though meteorological stations usually offer valuable and reliable data, their placement is typically far apart from one another, leading to a lack of high-quality spatial representation. Additionally, the scarcity of data, the impact of local topography, and the variation of the regional climate can render spatial interpolation methods unreliable and diminish their significance.

Accurate scenario generation can facilitate effective microgrid planning and operation and enable the development of optimal control strategies to maximise the use of renewable energy sources in the microgrid [19]. Previous approaches for scenario generation modelling rely on statistical and artificial intelligence methods, which are often applied to historical data. Reference [69] proposed a cross-correlated scenario generation approach using implicit generative models to capture the joint probability distribution of renewable energy sources and electricity demand. They emphasised the importance of generating correlated scenarios to capture the interdependence of renewable energy sources and their impact on the operation of the microgrid. Reference [70] developed a comprehensive tool for generating 24-h scenarios of solar irradiance profiles. They used a statistical approach to model solar irradiance based on historical data and employed the Roulette Wheel method to generate an initial set of scenarios.

In this study, the proposed advance is to integrate forecasts from NWP models in DRES to significantly enhance operational performance. This would also require quantifying weather forecast reliability and determining the forecast uncertainties and confidence intervals of the climate variables [67]. Employing high-resolution multiscale NWP forecast as well as ensemble forecasting will assist with scenario generation to enhance the modelling of the spatiotemporal variability of renewable resources.

In New Zealand, by way of example, the national institute of water and atmospheric research (NIWA) operates and has access to the highest, most-reliable NWP forecast models in Aotearoa-NZ. This includes access to the Global Ensemble Forecast System (GEFS), the UK Met Office Global Unified Model [71], the New Zealand Limited-Area Model (NZLAM), the New Zealand Convective-Scale Model (NZCSM) [72] and Auckland’s sub-km model [73]. These models provide forecasts from 30 min to 35 days ahead at 21 km (31 member GEFS [74]), 13 km (Unified Model (UM) Global Atmosphere model [71]), 4.4 km, 1.5 km and 333 m spatial resolutions, respectively.

Although the coarser NWP models (generally with spatial resolutions of >4 km, and more recently convective-scale models with resolutions of about >1.5 km) successfully capture the large-scale climate variations and trends, they are not capable of capturing smaller-scale variations, in particular, urban heat island (UHI) effects, which significantly influence the energy requirements in urban areas [75,76]. Therefore, finer-scale NWP models, such as NIWA’s 333-m Auckland model [73], are required to capture these details. Studies have shown significant forecast improvement of near-surface temperatures [77], wind [73] and other variables [78] when using this very high-resolution model.

The proposed research will not only couple existing NWPs to models of DRES operation (though this itself will be a significant advance), but this will also require delivering key advances for NWP capabilities in the context of DRES operation (Figure 3) as follows:

• Heatwaves and coldwaves can put significant pressure on the energy sector. To investigate the energy requirements in cities under these extremes, coupled NWP/CFD (Computational Fluid Dynamics) simulations need to be conducted for selected historical extreme temperature events [79,80] and also projected future extremes under climate change to understand the operating conditions of DRES in these extreme conditions.
• NWP dynamical downscaling techniques [73,81] with an improved surface energy-balance scheme [82] should be implemented for other major cities. These downscaled models will not only contribute to increasing the understanding of UHIs but also play a crucial role in the design of DRES/storage units and the operation of the system by providing input to the agent-based and control models described in Section 3.2.
• To determine users’ heating/cooling demand, essential for the design of DRES, bioclimate assessments need to be conducted [83]. The bioclimate maps, apart from the design of DRES, will be used for the first time ever in real-time (and forecasted) in the agent-based model to optimise the performance of the generator and storage systems.

3.2. Development of Comprehensive Control System

Comprehensive control systems are required to provide and ensure sufficient, stable and reliable real-world operation [31,68]. However, current control methods have unrealistic assumptions in peer-to-peer trading and scheduling models [84,85], such as the lack of real-time forecast horizons. Scenario-based security assessment methods (e.g., Monte Carlo Simulation [86]) cannot be applied due to the very large number of connections and complex behaviour of the energy system. By way of example, considering peer-to-peer energy trade and also integrating load and generation forecasts into the control strategies. This complexity leads to computationally-expensive calculations that thus cannot be implemented in real-time. Moreover, current methods to design preventive control strategies (such as worst conditions [87]) may lead to higher operational costs for the energy system.

Therefore, it is proposed to develop online security analysis algorithms, based on deep reinforcement architecture [88], to guarantee energy system resiliency and reliability under emergency and contingency conditions, including both low-probable high-impact and high-probable low-impact events. This comprehensive control system will be the first model with a suitable preventative control strategy for the integration of DRES operation into a wider network, which leads to a highly-connected energy system for which risks of cascading blackouts need to be integrated into the operational costs based on load/generation conditions. It should be noted that reinforcement learning implementation can present some challenges, such as the need for extensive computational resources and data processing capabilities to support real-time analysis and decision-making in a highly-connected energy system [89,90]. Moreover, there might be practical limitations and regulatory barriers, such as the need for a large amount of data and computational resources to train the models, which could hinder the adoption and deployment of such systems. Advances in machine learning and computing technology have made it easier to build accurate models and simulate complex systems [91,92]. Moreover, there is a growing interest in developing sustainable energy solutions, which provides an opportunity for the integration of reinforcement learning techniques. As such, with the right approach and investment in research, it is possible to overcome these challenges and fully realize the potential of reinforcement learning in energy systems.

To address the shortcomings in the current MG management systems, mentioned in Section 2.2, a recommended approach in this research is to develop a multi-scale multi-horizon ABM [93,94] to consider the short-term and long-term perspectives of the active players in the energy system. In other words, a multi-scale model will be built to use in different supply/demand scenarios. It is envisioned that this model will accurately and dynamically represent agent decisions, which are usually boundedly rational and yet also be simple enough to allow the longer-term simulations to execute with acceptably fast run-time. It should be noted that the development and implementation of a multi-scale, multi-horizon agent-based modelling is a highly complex computational task, which arises from the need to model a vast range of interactions and decision-making processes between agents at multiple scales and horizons [95]. Simulating such a model requires significant computational power and the use of advanced algorithms to capture the heterogeneity and dynamics of the system. Despite the computational challenges, multi-scale, multi-horizon agent-based modelling is a powerful tool for understanding complex systems and predicting their behaviour under different conditions [94]. With the advancement of computing power, the development of parallel computing techniques, and the optimization of simulation algorithms, the computational demands of agent-based modelling can be addressed to some extent [96]. Furthermore, the use of advanced modelling techniques, such as model reduction and surrogate modelling, can significantly reduce the computational burden of agent-based models [97].

The mentioned studies and ABM development will be followed by a contingency analysis to develop a state-of-the-art comprehensive contingency plan for the energy system to maintain the availability of the energy system to required service levels and limit the risk of cascading blackouts. Contingency analysis and multi-scale agent-based modelling are powerful tools for understanding complex systems and assessing their vulnerability to disruptions. By integrating these tools, a comprehensive understanding of system behaviour can be achieved [98]. Two methods of integration are incorporating contingency analysis results into the agent-based model [99] and using agent-based modelling to inform contingency analysis. The former approach models critical components identified by contingency analysis as agents with unique decision-making processes and behaviours. The latter approach identifies the most critical agents and interactions through simulation and incorporates them into contingency analysis to develop effective mitigation strategies. This integration can optimize system performance and enhance resilience by identifying critical components and interactions and developing effective contingency plans.

3.2.1. Agent-Based Model (ABM)

ABM is a simulation technique that is well-suited for modelling complex and interconnected systems with adaptive agents, which are individual components of the system with their own attributes and behaviour rules. By explicitly modelling the behaviour and properties of these actors and how they interact with each other and their changing environment, ABM can capture the complex relationships between them [100]. The ABM approach, therefore, allows the integration of complex decision rules into system models, which is not given “centrally”. With enough knowledge of these rules and interactions, ABM can help identify possible control strategies and regulation approaches for the system [101].

ABM has been applied to various research fields [102], including energy systems analysis [103,104]. ABM has been linked to multiple benefits [105], including its capability to simulate individual decision-making [106]. For instance, a study focused on the German electricity market employed ABM to model the decision-making processes of different agents, including plant operators of DRES [104]. This research aimed to model decision-making as realistically and comprehensively as possible by representing diverse types of agents. Reference [104] shows that the outcomes of agents’ collective decision-making can determine the overall amount of DRES energy traded in the wholesale power market.

ABM opens the gate to building accurate multidisciplinary models-within-model systems as it can host many different paradigms. Reference [107] shows how psychological decision models can be used when modelling the investment behaviours of solar homeowners. Another example is [108] which proposes an ABM-based approach to create a technology diffusion model and explore the growth of small solar systems based on economic, ecological, and social factors.

ABM can be useful in studying various aspects of policy-making procedures. Numerous policies within the energy sector focus on finding ways to encourage investment in low-carbon technologies and flexibility options at minimal costs [109]. However, when these policies are made and put into practice, it is essential to take the perspective of actors into account for evaluating the effectiveness and efficiency of them [110,111]. ABM is a valuable tool for explicitly modelling and analysing policy effects [101], particularly when dealing with policymakers and markets such as the power market, CO₂ market [112], capacity market [113], and DRES support markets [114]. This modelling is critical for creating operational MG that can interact with these markets in the real world.

Simulating energy trading is another application of ABM in energy systems analysis in which actors could be entities such as energy generators, retailers, or consumers. These actors estimate the profit they could earn by participating in the market and placing corresponding bids. To maximise their profit, actors usually learn to adjust their bidding strategies based on past experiences and market conditions [115]. ABM can also take into account the policy frameworks when developing models of different types of electricity markets, treating the energy system as a complex adaptive system [116]. This leads to identifying the interactions between different actors and factors in the energy system and provides insights into how policies and regulations could affect the behaviour of different agents in the market.

ABM has additional applications in analysing the demand side of energy systems. For instance, it can be used to portray how price-elastic consumers could benefit from demand response [117]. ABM can also help determine the degree of price sensitivity of consumers in a bottom-up approach.

3.3. Pathways to Grid Integration

As explained in Section 2.2, no currently available holistic technique is suitable for real-world MG management in different applications. Offline deterministic modelling with stochastic programming uses limited operation scenarios [118], and thus cannot capture the highly volatile load profile and time-changing conditions expected for DRES. Worst-case condition analysis is overly conservative and uneconomic [119]. Real-time optimisation/scheduling is computationally expensive and has a limited look-ahead window that prevents effective scheduling [120].

The pathway toward grid integration is to embed an ABM (Section 3.2.1) into the operation of the DRES to simulate and manage the behaviours of players in the energy system including regulators, technology developers [121], distributors, prosumers and consumers, incorporating the physics-based model of the electricity system developed in previous steps. Moreover, distributionally robust optimisation (DRO) approaches should be developed to capture the highly volatile operating conditions of DRES.

4. Conclusions

The intermittent nature of renewables, uncertainties associated with real-time multi-horizon weather and load forecasts, and the lack of comprehensive control systems are identified in this paper as the main challenges for the adaptation of DRES in the real world. This discussion paper outlines the current state of knowledge in the real-world operation of DRES and proposes pathways to enable the uptake of DRES in a country’s energy sector. To mitigate and minimise the uncertainties arising from the intermittency of renewables and demand load in real-time, it is recommended to:

• Dynamically downscale current operational NWP models to very fine resolutions, e.g., a few hundred meters, and include improved surface energy balance to capture highly variable urban-area microclimates.
• Conduct high-resolution flow modelling using CFD and coupled NWP-CFD to investigate the effects of extreme temperatures on both energy demand and availability.
• Develop and embed a machine-learning algorithm trained on fine-resolution NWP and CFD results to bypass computationally expensive simulations in real-time and only conduct online updating of the control system.

The lack of a holistic control system is another technical barrier that impedes the large-scale adoption of DRES. A multi-scale agent-based model based on deep reinforcement architecture was suggested as a viable approach to develop a comprehensive control system for normal and contingency conditions. This will consider the short-term and long-term perspectives of the active players in the energy system.