Coordination Model and Digital Twins for Managing Energy Consumption and Production in a Smart Grid

Abstract

Smart grids play an important role for energy management by directly supporting the socio-ecological transition of neighbourhoods. This research provides the design of a coordination model to enable the management and exchange of electrical energy between producers and consumers at a micro-grid level. This model, which derives from the SAPERE coordination model, allows the intelligent digital twins to interact and generate services on the fly to meet different needs in real time. We have designed producer and consumer digital twins, which autonomously generate supply contracts in the form of a transaction, and supervisor digital twins, which regulate energy at the node level, managing threshold violations and proactively avoiding future threshold violations by using predictions. This coordination model allows energy exchanges in a single node and in a micro-grid structure that contains several neighbouring nodes. We have implemented and tested the platform with realistic data, based on the consumption statistics of a real household, and with real data, collected in the living-lab of “Les Vergers” located near Geneva. The results show that the combination of a coordination model and intelligent digital twins actually supports self-adaptive energy management in a smart grid. Such approaches are fundamental to develop efficient and reliable smart grids.

1. Introduction

Since the advent of renewable energies, energy distribution systems between individuals are booming. This contributes to the reduction in energy costs and the improvement of both the life quality of citizens and the quality of the environment.

In this paper, we define smart grids as electricity networks that use computer technologies to adjust the flow of electricity between suppliers and consumers. By collecting information on the state of the network, they contribute to the balance between production, distribution, and consumption. A smart grid is made up of a number of interconnected nodes containing computing units, which are generally connected to electrical devices that consume or produce electricity—except for the highest-level nodes.

Smart grids are part of the new models of distributed, dense, and ubiquitous computing systems that provide services capable of continuously adapting to changing conditions. The objective of such a system is to provide a peer-to-peer exchange platform on which energy producers and consumers connect and interact with each other to perform energy transactions: A consumer can decide to use this or that offer from this or that producing device. The application informs the consumer and the producer of the quantity of energy exchanged and the consumer pays the producer.

However, smart-meter devices, which collect and communicate electrical measurements, do not yet manage energy exchanges in a collaborative and decentralised manner. Indeed, these devices do not have sufficient autonomy to negotiate or exchange energy directly with third-party devices: they only execute directives issued by an external entity. Moreover, it is very difficult to predict the behaviour of renewable energies which are particularly fluctuating in time.

Coordination models (see Definition 1) could fill this gap as they allow to design distributed systems capable of adapting to permanent changes and in which autonomous agents, which represent the smart grid entities, interact with each other.

Researchers have used these models for different applications such as crowd monitoring in event management, and it is possible to adapt them to electricity exchanges. The work presented in this article consists of designing and implementing a coordination model allowing the energy exchange between energy digital twins at the level of a household or a micro-grid. The model also manages over-production or over-consumption in real time using regulation, and in a proactive way, using prediction. This paper discusses the following contributions:

• Implementation of a collaborative energy exchange system composed of a coordination model and digital twins that represent entities in a micro-grid network;
• Implementation of the different algorithms that the digital twins collectively execute to respond to different issues: exchange and regulate energy, manage various crisis situations, and protect personal data;
• Implementation of test simulators and different scenarios: real scenario, based on the data collected in the living-lab “Les Vergers”, realistic scenario based on household consumption statistics and specific scenarios to simulate various crisis situations.

The following contributions are also part of this work, but the paper does not describe them in detail, as it focuses more on the contributions mentioned above:

• Implementation of a thin-client web application to monitor all energy producing and consuming agents in the micro-grid;
• Implementation of a learning model for predicting the energy consumption and production at the house level and the micro-grid level using a decentralised federated learning approach.

We can find full technical details in the following report [1], which has been published in the University of Geneva’s open archive https://archive-ouverte.unige.ch/unige:155369 (accessed on 8 October 2023).

This work is also part of the LASAGNE research project (https://lsds.hesge.ch/digital-framework-for-smart-grid-and-renewable-energie/ (accessed on 8 October 2023)), initiated by various consortia in Switzerland and Sweden, which aims at developing self-adaptive applications for smart grid energy management. These applications cover aspects such as energy negotiation or peak shaving. To address this goal, the LASAGNE project develops an edge-to-cloud infrastructure, relying on a digital platform providing specific services for these applications. This platform instantiates a coordination model, provides digital twins, and supports distributed collaborative Machine Learning (ML) methods.

2. Related Works

Since smart grids involve electricity actors who interact locally with each other and face constantly changing conditions, we are interested in research areas that study distributed applications in which evolving agents can collectively achieve a goal on the fly using dynamic adaptation. These areas are multi-agent systems and coordination models which provide self-adaptation mechanisms. In this article, we focus on experiments carried out in these areas and applied to energy.

2.1. Coordination Models

The proliferation of ubiquitous embedded systems producing huge amounts of data has shown the limits of centralised systems and has led to the search for alternative solutions meeting several requirements [2], such as spatial and temporal dependency, self-adaptation to changing conditions, and device heterogeneity. These changes are having a particular impact on the energy sector [3].

To meet such a challenge, researchers have modelled coordination models in which responsibilities are decentralised (hence the emergence of self-organisation). A coordination model provides a framework composed of an abstraction layer and an associated infrastructure that provides the basic technologies to guarantee interactions between heterogeneous components. Such models offer systems better scalability, better robustness, and a better distribution of computational loads on the different entities. Zambonelli et al. [4] represents it in three layers: the software agents, the environment, and the infrastructure layer, which is necessary to host the agents in the devices. The bio-chemical coordination model adds the notion of logical fragments [5], which are combinations of programs defining ad hoc chemical reactions on the fly that are applied to tuples to manipulate other tuples or to produce new logical fragments. Such models define an environment called tuple space in which the coordination laws are acting on tuples to transform or disseminate tuples, which contain shared knowledge.

To date, very few bio-chemical inspiration models have been used in the field of electricity. There are, however, a few uses of coordination models, but they are limited to biological inspiration as they only apply algorithms inspired by insect colonies or generic reproduction and they do not use transformation mechanisms inspired by chemical reactions.

Peres et al. [6] present a coordination model and optimisation methodologies inspired by ants, bats, and genetic algorithms to solve the problem of tuning stabilisers for electrical systems. The objective of the optimisation presented is to maximise the closed-loop damping ratio that satisfies pre-specified operating conditions. Zhao et al. [7] have proposed a wind power consumption dispatching model using a multi-time scale coordination model to overcome the obstruction problem of large-scale wind power consumption.

Shan et al. [8] have proposed a coordination mechanism based on a particle swarm optimisation algorithm that determines the optimal active and reactive power setpoints, which are then sent to the device level for controlling inverters.

Note that these approaches do not use a coordination medium strictly speaking (such as a tuple space or other type of environment capable of propagating and transforming data asynchronously between autonomous agents).

Saber et al. [9] have designed a coordination model based on a propagation mechanism, to simulate residential prosumers participating in the retail electricity market 15 min in advance to schedule their reactive assets iteratively.

Ben Mahfoudh et al. [10] have adapted the SAPERE [4] model to generate spatial services on the fly. One of the implemented services allows energy exchanges between autonomous agents connected to different locations in a network, considering different criteria in the negotiation process, such as price, quantity, and section bandwidth.

2.2. Agent-Based Approach to Energy Exchange

2.2.1. Using Autonomous Agents for Smart Grids Applications

According to Mahela et al.’s comprehensive review of the smart grid [11], the multi-agent approach is widely used to represent the different entities in the network: these entities are located at different levels and focus on very specific tasks. Autonomous agents have individual responsibilities and focus on very specific issues within an organisation, such as energy marketing, pricing, negotiation, efficiency improvement, reliability, network security, outage management, and fault handling. They make decisions with varying degrees of impact, depending on the hierarchical level of the entity they represent in the smart grid.

Binyamin et al.’s review [12] highlights the different challenges faced by multi-agent systems in smart grids and describes the taxonomy of types of autonomous agents in smart grid applications, as well as the technologies they use. For example, the blockchain [13] technology, which has become popular, improves the security of resources exchanged between smart grid entities by monitoring all changes that occur in the chain.

Other multi-agent approaches focus on peer-to-peer negotiation to exchange energy in a cooperative manner [14]. The proposed strategy is aimed at both personal welfare and distributional fairness. The algorithm for finding the optimal energy contract draws inspiration from the Nash equilibrium (borrowed from game theory) and uses utility functions to consider the personal preferences of each consumer.

Other approaches [15] focus on the implementation of protocols and negotiation mechanisms between agents in real time, with the complete definition of a negotiation language and with the different rules that apply during the different stages, from the issuing of offers to the signing of an agreement. This approach manages the temporal aspects of the negotiation by enriching the FIPA protocol with added time parameters.

2.2.2. Using Holonic Structures

Some approaches have modelled the smart grid using a holonic structure [15,16], in which a ‘holon’ is an entity that can be both a local organisation and an agent participating in a higher-level organisation that encompasses the previous one. Such a structure enables the recursive integration of heterogeneous controls at different network scales (at the level of an electrical appliance, a house, or a neighbourhood). Holonic structures are inherently hierarchical and can thus be very useful in smart grid systems, since electrical grids are hierarchical by nature.

Kansas State University has used the holonic approach to model and implement a smart power distribution system [17]. The university has built an evolutionary and iterative framework suitable for the development of complex, adaptive, and autonomous systems.

Taleb et al. [18] have experimented with scenarios on a grid composed of three levels and 90 holons, to demonstrate that a holonic structure improves flexibility and helps to manage island modes and supply prioritisation.

Wallis et al. [19] have proposed a framework to provide dynamic control for the holonic smart grid, based on a bottom-up approach. The presented framework manages holonic organisation at an atomic level: it improves smart grid capabilities by providing a modular structure for forecasting and optimisation.

2.2.3. Using Intelligent Digital Twin

The joint use of technologies such as IoT, Artificial Intelligence, Cloud Computing, or Virtualisation has favoured the development of digital twins.

VanDerHorn et al. [20] define a Digital Twin as a virtual representation of a physical product containing information about said product, with its origins in the field of product life-cycle management. The virtual and physical entities are permanently inter-connected and exchange data with each other in both directions.

In fact, they are also “intelligent” digital twins, since they interact, make decisions based on the information collected, and have advanced capabilities such as negotiating or learning [21].

Tzanis et al. [22] have implemented a hybrid Digital Twins model for fault identification in a smart grid containing distributed energy resources. This proposed model, which combines machine learning and differential equations, enables the Digital Twin to reduce the calculation time and ensure a sufficient speed to work on real time.

2.2.4. Synthesis

To date, there are no electrical applications using both multi-agent systems and digital twins. In this field, multi-agent systems are widely used and have reached a significant level of maturity, particularly compared to more traditional digital twins. In our work, we leverage the power of multi-agent systems for implementing intelligent digital twins.

3. Materials and Methods

3.1. Design of the Coordination Model

The coordination model (see Definition 1) constitutes the main building block: indeed, our entire approach to managing the energy exchange and regulation is based on coordination between intelligent digital twins that can interact with each other and generate services on the fly.

Definition 1. 

A coordination model provides a coordination medium for sharing data among the coordinated entities, and coordination laws, which are applied on the shared data (transformation, propagation) [23]. We consider coordination laws as self-organising mechanisms inspired by bio-inspired systems. The coordination model uses the stigmergy principle, allowing the exchange of asynchronous information among the coordinated entities that use the coordination medium. Such a service has its own logic, controlled by an autonomous software agent (also called intelligent digital twins).

3.1.1. Sapere Coordination Model

The coordination model used derives from the SAPERE coordination model and SAPERE middleware [24]. Ben Mahfoudh [10] provided the most recent extension. Figure 1 represents the SAPERE coordination model in the form currently used, which draws inspiration from biochemistry. It contains a shared virtual environment, called tuple space (see Definition 2), in which digital twins can share data by submitting or retrieving data properties.

Definition 2. 

A tuple space is a shared virtual space containing all tuples of a node. There is a shared space for each coordination platform (see Definition 5).

Digital twins can thus submit and retrieve LSAs (see Definition 3). LSAs contain tuples of properties provided from other digital twins or generated by the coordination laws (see Definition 4) at any time.

Definition 3. 

A live semantic annotation (LSA) is the tuple of data containing the digital twin properties that travel and evolve in the tuple space. Digital twins can thus submit and retrieve LSAs (tuples of properties) provided from other digital twins or generated by the coordination laws at any time.

Definition 4. 

A coordination law is a mechanism that applies to entities in the environment. Coordination laws are modelled with bio-inspired design patterns [25] and classified into three hierarchical levels:

1.

The bottom layer, which contains the atomic mechanisms;

2.

The middle layer, which contains the mechanisms using those of the lower layer;

3.

The upper layer, which contains the mechanisms using those of the two lower levels.

A coordination model distinguishes itself from a publish/subscribe model as the digital twins interacting in the system do so through the use of the tuple space (a blackboard) to share information in an indirect stigmergy-like manner (submitting LSAs and retrieving LSAs that match the specific criteria they are looking for). A coordination platform (which instantiates the model, see Definition 5) runs in a computational node (see Definition 6).

Definition 5. 

A coordination platform is a system that implements and runs the coordination model. The coordination platform executes within a Grid Edge Device and communicates through the network with other coordination platforms in other Grid Edge Devices (thanks to the Spreading Coordination eco-law, a specific type of broadcasting communication described below). The software agents generate services on the fly by coordinating their activities through the coordination platform: they submit and retrieve data from the platform. The data submitted corresponds to (1) the data generated by an actual physical object linked to the environment (e.g., consumption or production level); (2) the data exchanged among digital twins for managing energy (e.g., requests or offers of energy, peak shaving schedules); or (3) the data provided by other services (possibly processed or transformed dynamically by self-organisation mechanisms).

Definition 6. 

A node is a local computing entity of the environment. It is associated with a network address and a set of direct neighbour nodes. In the context of the paper, a node is associated with a building and is also called a GED (Grid Edge Device).

3.1.2. Coordination Laws

Coordination laws (also called “eco-laws”(as they derive from the SAPERE coordination model) [4]), act like chemical reactions and cause the content of LSAs, in the tuple space, to evolve over time. They transform LSAs and propagate the data between the different nodes of the system. LSAs contain data that are then are submitted to coordination laws, since they themselves are not coordination laws. The coordination laws currently defined are the following:

1.

“Bonding”, which binds (provides the matching of) a digital twin to the data provided by another digital twin (see Definition 7);

2.

“Decay” (evaporation), which periodically decreases the relevance of the data and removes data that has become obsolete;

3.

“Spreading”, which disseminates information within a network across different nodes;

4.

“Aggregation”, which aggregates certain data retrieved from different nodes. We define the aggregation function in the digital twins and give it as a parameter in this coordination law. In this way, the Aggregation coordination law remains generic and independent of the aggregation function implementation.

3.1.3. Intelligent Digital Twins

A digital twin (see Definition 7), which we also call “agents” (a digital twin is essentially a software agent) [26], is generally a detailed virtual replica of a physical object or system, also called the physical twin. A digital twin may also incorporate real-time changes in the physical world based on direct observations and data transmitted by the object itself, most often via sensors or IoT devices. Within the coordination model, digital twins go beyond real-time replicas, since they are equipped with advanced functionalities (e.g., sensing, acting, learning, negotiating, etc.) [21]. They are actual software agents working on behalf of actual devices or users. They coordinate their activities by submitting and retrieving LSAs into and from the LSA tuple space.

Definition 7. 

A digital twin is a software agent that interacts, on the one hand, with end-user applications (or with consumption/production edge devices), and on the other hand, coordinates its actions with other digital twins via the coordination platform. To do so, it submits LSAs into the tuple space or retrieves LSAs from the tuple space. Digital twins serve various purposes:

1.

Interaction with the coordination platform;

2.

Coordination of tasks among digital twins;

3.

Real-time representation of their physical twin counterparts (e.g., edge consumption device data).

3.1.4. Generation of Services on the Fly

The model uses the principle of stigmergy (indirect communication via the environment; in our case, the coordination platform): it allows for the asynchronous transformation and propagation of information provided by the digital twin. The coordination laws ensure the different data transformation operations in the environment. One digital twin can submit data and another digital twin in the same node (or a remote node) can retrieve these same data, either in that form or transformed by the coordination laws. We can consider this data as the output of one digital twin serving as the input to another twin. As a result, this model allows digital twins to perform coordinated actions as well as de facto generating services on the fly [10]. Services arise from the actions of the physical twins in the physical environment.

3.1.5. Coordination Platform, Nodes, and Devices

The coordination platform is the implementation of the coordination model and typically executes in a computational node or (computational) edge device (e.g., computer, raspberry, etc.).

We distinguish computational devices from energy devices, which are electrical devices that consume or produce energy (e.g., solar panel, boiler, etc.).

3.2. Modelling of Intelligent Digital Twins for Smart Grids Applications

We have defined different types of digital twins to model the behaviour of the actors who interact in a micro-grid organisation at the scale of a few houses or buildings. We have thus adapted the coordination model for the needs of electricity exchanges and energy regulation. The modelling of actors (see Figure 2) includes:

• Two types of energy digital twins: the consumers and the producers. Note that the prosumer digital twin, which can consume and supply energy at the same time, is not yet implemented in this version. Further developments will add it as an energy digital twin;
• Two types of supervisory digital twins: the learning twin, which tries to predict consumption and production at the node level, and the regulator twin, which manages crisis situations at the node level.

Energy digital twins participate in negotiations regarding energy exchanges. Negotiations involve energy demands submitted by consumers and energy offers submitted by producers. Section 3.3 provides an explanation of the specific algorithms detailing the establishment of contracts for fulfilling requests based on the received offers. Within a node, there are as many instances of digital energy twins as there are producers and consumers. Supervisory digital twins have only a single instance at the node level and are instantiated when we start the node coordination platform service. In the following, we define the role of each type of digital twin.

For the sake of clarity, we consider the smart grid of Figure 3. The CLEMAP devices are our GED (or nodes, see Definition 6). These devices receive information about the data consumed and produced. Namely, we provide one digital twin for each consumer or producer linked to a node, and one learning and one regulator twin per node, as shown in Figure 2.

We consider digital twins as autonomous pro-active agents that go beyond simulated models of physical devices equipped with real-time information. This corresponds to the notion of the intelligent digital twin, as defined by Di Marzo Serugendo et al. [21].

At the moment, the digital twins actually run in a separate device as shown in Figure 3. In the final version, the CLEMAP device will host a coordination platform and all the corresponding twins.

3.2.1. Producer Digital Twin

A producer digital twin (also called “producer twin”) represents a device producing electrical energy. It performs the following actions:

• It reports any event concerning its activity (start, deactivation, and stop);
• It generates offers to meet the energy requests submitted by consumers;
• After retrieving a new energy supply contract (see Section 3.3), it sends a confirmation to give its approval. It repeats this action until the contract is validated or rejected;
• After retrieving an interrupt command from the regulator twin, it pauses its activity.

When we shut down the node, the twin de-activates itself and the coordination laws remove the twin’s LSA from the tuple space. This LSA cleansing helps reduce the volume of data exchanged and the number of coordination law executions at each cycle.

3.2.2. Consumer Digital Twin

A consumer digital twin (also called “consumer twin”) represents a device that consumes electrical energy. It performs the following actions:

• It reports any event concerning its activity (start, deactivation, and stop);
• As long as it has not retrieved enough offers to meet its needs, the consumer twin verifies if its request resulted in the emission of a new energy offer. If applicable, it checks if all the retrieved offers allow it to satisfy its needs. After retrieving satisfactory offers, it generates a new contract and has it validated by the producers concerned;
• After retrieving confirmation from a producer, it updates the contract status. It stops and removes it immediately in case of disapproval;
• After having the contract validated, it updates its satisfaction indicator to “True”;
• As long as the contract is still valid, the consumer twin has it reconfirmed periodically by the producers concerned;
• After retrieving an interrupt command from the regulator twin, it stops the contract and pauses its activity.

Similarly, when we shut down the device, the digital twin de-activates itself and the coordination laws remove the digital twin’s LSA from the tuple space.

3.2.3. Learning Digital Twin

The learning digital twin (also called “learning twin”) capitalises on the various events occurring at the node and generates prediction data about the state of the node.

• After retrieving an event from an energy twin, it updates the global state of the node, including requested, produced, supplied, consumed, missing, and available powers. The learning twin stores this data for later use, either via the web application to display the history of the node, or by the regulator, to detect possible over-consumption or over-production in the last recorded node state (see the node state definition in Definition 8);
• At each refresh cycle, it updates the global state of the node as well as the node’s training data. The learning twin executes the learning model to regenerate a new prediction of the node’s state at 5, 10, 30, and 60 min horizons. The electric devices we use do not provide predictions, but only provide actual consumption and production data every minute (full data: power and voltage) and every 10 s (only power). The Learning twin has 5, 10, 30, and 60 min horizons, to provide predictions useful for higher-level services and for the regulator twin.

Definition 8. 

A node (also called GED), as expressed in Definition 6, is a computational device. In this paper, we call node state a set of six power variables that characterise the current consumption and production of the node, at time t, taking into account all the electrical devices that are linked to the node. We consider the following six variables:

• Wattage requested: sum of the power demanded by the various electrical devices linked to the node.
• Wattage produced: sum of the power produced by the various electrical devices linked to the node.
• Wattage consumed: sum of the power consumed by the various electrical devices linked to the node. As some devices demanding energy may not be supplied, we obtain the following inequality:

  $$wattageconsumed<=wattagerequested.$$

  (1)
• wattage supplied: sum of the wattages supplied by the various electrical devices linked to the node. As some producer devices may not use all the energy produced for supply, we obtain the following inequality:

  $$wattagesupplied<=wattageproduced.$$

  (2)
• Wattage missing: sum of the power requested and not supplied, for the electrical devices connected to the node.

  $$wattagemissing=wattagerequested-wattageconsumed.$$

  (3)
• Wattage available: sum of the power produced and not used, for the electrical devices connected to the node.

  $$wattageavailable=wattageproduced-wattagesupplied$$

  (4)

• NB: In the case where there are no energy exchanges with the neighbourhood, all producers exclusively supply consumers attached to the same node, and all consumers exclusively use wattage produced locally. The wattages supplied and consumed are therefore equal.

3.2.4. Regulator Digital Twin

The regulator digital twin (also called “regulator twin”) supervises the node to ensure the stability of the grid, i.e., to perform peak shaving (avoiding excess of production or consumption). It performs the following actions:

• At each refresh cycle, it retrieves a complete picture of the node and the twins, so as to detect possible production (or consumption) overruns or various problems that may occur;
• After detecting excess of production or consumption, it sends a deactivation order to some twins to instruct them to return below the threshold;
• After detecting a technical problem, it sends an order to the concerned twin to correct the problem;
• After retrieving a prediction that warns of over-consumption or over-production within the hour, it submits in the coordination platform an LSA that corresponds to an order to postpone certain activities to proactively avoid exceeding the threshold. This LSA is retrieved by the digital twin corresponding to the electrical device that should stop consuming or producing energy.

3.3. Algorithms Used by the Digital Twins

We present here the different algorithms that the digital twins apply collectively to meet different needs in a dynamic way, by continuously adapting to the evolution of the situation. These algorithms use the principle of self-adaptation.

3.3.1. Generation of Contracts

Many situations lead digital twins to regenerate contracts after a sudden stop of the electricity supply. By using self-composition (see Definition 9) and dynamic adaptation (cf. Figure 4), digital twins can generate a new contract: this allows them to fill a new energy gap in a relatively short time.

Definition 9. 

In the context of a distributed application, a  self-composition service is a mechanism that generates a service on the fly without prior intention and without the initiative of a supervising entity. The auto-composition uses the following mechanisms:

• a semantics-based mechanism that determines the most appropriate match using a dataset of service descriptions;
• a scalable and distributed technique for dynamically selecting the most appropriate way to compose a service;
• an evaluation of service composition for which the quality of the composition is based solely on the success of its operation.

Ben Mahfoudh et al. [10] set up such a service to reserve spaces for electric cars on demand at a service station.

Algorithms A1 and A2 represent the actions performed respectively by both the consumer and producer digital twins. Figure 5 represents the contract life cycle in the form of a UML state chart and Figure 6 focuses on the sequential aspects: it represents an example of an interaction between one consumer needing 50 W and two producers with capacities of 13 and 60 W. In this scenario, the second producer has a lower environmental impact and is therefore chosen first when comparing offers.

Table 1. Combinations of producer capacities and resulting supplies, using two or three producers to meet a demand of 50 W. The first row corresponds to the case represented in Figure 6 in which the second producer alone supplies the 50 W. The last line shows a case with three producers.

Producer Capacities of the 2 (or 3) Producers			Resulting Supplies the 2 (or 3) Producers
Producer #1	Producer # 2	Producer # 3	Producer # 1	Producer #2	Producer # 3
13	60		0	50
30	50		20	30
20	35		15	35
30	15	20	15	15	20

displays examples of combinations of producer capacities with the resulting supplies.

The following lists and describes the different stages of interaction that lead to a new contract between producers and consumers:

Step 1: 

sending of the request by the consumer: The consumer twin submits its request via its LSA to report its need for energy supply.

Step 2: 

sending of offers by producers: The producers retrieve the new consumer requests from their LSA. After the requests’ retrieval, each producer then generates new elementary offers to meet the different requests. A producer applies a prioritisation policy to choose the requests to handle first. For example, a policy can classify the requests by decreasing the level of urgency, then by increasing power. The producer generates offers within the limit of its available wattage. As long as its remaining wattage is positive, it generates a new offer, even if it cannot cover all the requested wattage.

Step 3: 

issuance of a new contract by the consumer: The consumer retrieves the offers that have been issued by the producers. Periodically, it checks whether all of its retrieved offers meet the need. Similarly, it applies a policy that determines the offers to choose as a priority. For example, a policy can favour geothermal energies over other types of energy. The consumer then goes through the different offers in an iterative way, following the chosen policy: as soon as the offer aggregated (from the elementary offers) meets the need, the consumer issues a new contract based on the selected offers.

Step 4: 

confirmation of a contract by the producers concerned: To have the contract confirmed by the producers, the consumer twin submits it in its LSA. Then, each producer involved retrieves the contract and checks whether it agrees or not with the terms and then submits its confirmation or refusal in its LSA: in this way, the consumer will retrieve the producer confirmation or refusal and will update the status of the contract validation.

Step 5: 

validation of a contract by the consumer twin: Each time the consumer twin retrieves one confirmation, it updates the contract status to include the producer’s approval (or disapproval) and then submits the new contract content to its LSA so that all stakeholders involved have the updated version. The consumer validates the contract only when all the producers involved have responded positively. If so, all stakeholders involved retrieve a validated contract: the contract then takes effect and the producers can start supplying energy. If any of the stakeholders disapprove the contract, the consumer cancels it and a new bidding cycle begins immediately.

3.3.2. Priority Demands

It may be the case that some energy device consumers should have priority over other energy consumers. For instance, we absolutely need to charge an electric vehicle, and this takes priority before a washing machine. To accommodate such cases, we defined the notion of a priority request.

In such a context, a consumer submits an energy request with a priority flag set to true. After retrieving a priority energy request, a producer ensures that the current supplies do not prevent it from having sufficient availability to meet this request. If necessary, it interrupts the non-priority supplies (by stopping the contracts). It reiterates this action until the availability becomes sufficient. After having stopped the possible supplies, the producer generates and submits an offer to meet the priority demand. Different simulations have shown that new urgent requests with high wattage have a very significant impact on priority management and can even stop all existing supplies according to the demanded power value.

Nota bene: even if the producer’s capacity is not sufficient to meet the demand on its own, it contributes to meeting it and, in this way, it will have acted to respond collectively to the emergency.

3.3.3. Predictions of the Local Node State

The learning twin uses the Markov chains model to predict the trend of each power variable at the node level. This learning model is a stochastic model, which uses transition matrices to define the transition probabilities of all possible states.

Table A1. Definition of Markov states for the node prediction model. This division into states includes five steps of 20% from zero to the maximum power threshold, noted “THRESHOLD”. We also define two additional states: one for value 0 W and one for the values beyond the threshold. For example, it can be set to 10,000 W for one node and 50,000 W for another node.

State	Range in W	Comment
1	{0}	We consider the 0 W as a state in its own right
2	]0,0.2*THRESHOLD [	First 20% range
3	[0.2*THRESHOLD, 0.4*THRESHOLD[	Second 20% range
4	[0.4*THRESHOLD, 0.6*THRESHOLD[	Third 20% range
5	[0.6*THRESHOLD, 0.8*THRESHOLD[	Fourth 20% range
6	[0.8*THRESHOLD, 1.0*THRESHOLD[	Fifth 20% range
7	[1.0*THRESHOLD, +∞ [	Range beyond the alert threshold (over-production or over-consumption)

defines the seven Markov states used in the classification. For each power variable, the learning twin computes a prediction at a target time horizon using the transition matrices. At each iteration step, the model multiplies the probability vector by the relevant transition matrix. The result of the prediction is a seven-component vector containing the probabilities of reaching each Markov state, starting from the initial state (there is one component per Markov state).

After each prediction, it stores the result in the database. Since it stores the actual state of the node at each refresh, it can reconcile a posteriori, which is the prediction generated at time $t_1$ with the actual perceived state at $t_2$. For each prediction, it thus obtains the assumed Markov state and the actual Markov state at prediction horizon $t_2$. The learning twin also stores the actual state to facilitate the evaluation of the predictions.

3.3.4. Peak Shaving

Peak shaving aims at providing stable conditions for the grid. Decisions are taken either based on actual grid conditions or based on predictions.

Management of production and consumption peak overruns: We should note that regulation is carried out entirely by the regulator twin (and not by an external or manual system). This digital twin acts locally and its impact is limited to the other digital twins connected to the same node.

At regular intervals, the regulator twin retrieves the node global state that the learning twin recently stored in the database. From the different totals, the regulator twin detects the production and consumption flows that exceed the alert threshold.

If the total produced or the total consumed electricity exceeds the threshold, the regulator twin applies a policy that implores the digital twin to stop first so that the node can get back under the alert threshold. The regulator twin thus obtains an ordered list of digital twins to stop. For example, the regulator twin can stop by order of priority of the producers with the highest power.

The regulator twin, one by one, stops the digital twins contained in this list, as long as the updated total remains above the alert threshold (see Algorithm 1).

Algorithm 1 Choice of producers to deactivate (run by the regulator)

Avoidance of over-production or over-consumption based on predictions: The regulator twin periodically retrieves the result of the last prediction made by the learning twin (see Figure 7).

When the learning twin predicts a peak within the hour, the regulator determines the list of digital twins to reschedule to reduce the node production (or consumption), so as to return to 20% below the maximum threshold. The regulator therefore chooses a policy to prioritise the digital twins to reschedule, for example, by ranking them by increasing wattage.

After completing the rescheduling table, the regulator submits it through its LSA and then all digital twins retrieve it. The digital twins concerned by the rescheduling then advance their end-of-activity date so that they stop 10 min before the scheduled peak time. By default, they will be able to resume their respective activities only after the end of the shutdown period. For example, in case of over-production, the postponement of production directly concerns certain producer twins. Consequently, it also impacts the consumers supplied by the producers in question, since their production is halted.

From a sequential point of view, the peak avoidance mechanism works as follows:

Step 1: 

the learning twin calculates the probabilities of the threshold crossing risk and submits them in its LSA.

Step 2: 

the regulator retrieves the predictions previously submitted by the learning twin and detects the high probability of overshot.

Step 3: 

the regulator determines the rescheduling table of the energy supplies to postpone. The regulator then submits a rescheduling table in its LSA.

Step 4: 

the concerned digital twins retrieve the re-scheduling directives sent by the regulator: they then defer their consumption or production activity.

3.3.5. Tragedy of the Commons

The “tragedy of the commons” scenario corresponds to the case where the best individual decision for a given digital twin turns out to be the worst-case scenario for them as a group [27]. As an example, let us consider the charging of electrical vehicles (EV) at night when energy prices are less expensive, for instance, between 11:00 pm and 5:00 am. Individually, it is a good policy to start charging the EV from 11:00 pm onward. However, if everybody in a neighbourhood makes this same decision, this leads to a peak of energy consumption at 11:00 pm.

To experiment with such a scenario, we have defined a policy that encourages all digital twins to request electricity at the same time slot. In this way, we test the resilience of the digital twins by straining the available electricity resources. As an example, we have defined a pricing table with a minimum price set at the current time plus 1 min, which encourages all digital twins to request a supply during the time window corresponding to the cheapest price (see the red box in Figure 8). In our simulation, there are 10 consumers and a single producer, which can supply only three digital twins at the same time. As a result, seven demands for electricity will remain unmet (see Figure 9).

To address this supply gap, a first alternative is to modify the pricing policy to consider the real-time demand. To do so, we have defined a price escalation factor that is a function of the remaining availability of the producer in W (see Figure 10). If the producer’s availability decreases, the price increases in real time and the unserved consumers will then review their demand time slot. In this way, we can spread the demand of different consumers over time.

As shown in Figure 11, when applying this indexed pricing policy to the previous example, the seven unsatisfied digital twins then defer their requests to the next time slot. Of the remaining seven requests, only three will become satisfied in the second slot, and similarly, the four remaining unfulfilled digital twins will defer their requests to the next slot. Thus, the next three requests will become satisfied in the third time slot, and the last one will become satisfied in the fourth time slot.

We should note that, at present, digital twins are not yet able to reduce the power demand when the overall supply is insufficient at time t. This last solution would be preferable to cater for all digital twins in a more equitable manner.

3.3.6. Adaptation to Meet a Growing Demand for Electricity

We have introduced a 5% margin on the power demand in an electricity contract, so that the consumer can adjust its consumption by plus or minus 5% without having to modify the contract. The advantage of this flexibility is that most of the contracts avoid stopping when the demand for electricity is constantly fluctuating. However, it has the disadvantage of reducing the producer’s availability (the equivalent of the contract margin), as it cannot supply this 5% margin. Finally, the choice of 5% seems a reasonable compromise to allow contracts to continue without having to solicit a large reserve from the producer. Indeed 1% is not enough to prevent contracts from breaking, while 10% would cause an excessive energy reserve. The digital twins handle the supply margin as follows:

• In case the demand decreases by more than 5%, the contract automatically updates its power range to match the new demand. This on-the-fly adjustment does not require any verification of energy availability as it is a decrease in supply demand;
• In case the demand increases by more than 5%, the consumer automatically issues a new demand for electricity to make up the difference: in this way, the consumer does not have to stop its current contract. The consumer will send a secondary demand to cover the increase and a new “temporary” contract will respond to that demand. As a result, the secondary supply will complement the current supply. As soon as the temporary contract is validated, the consumer has two supply contracts, though, after a few seconds, it merges them automatically into one contract. The automatic merge avoids the proliferation of contracts, as a consumer cannot have more than two contracts at the same time.

Figure 12 represents the generation of a temporary contract to meet an increase in demand of 300 W on an initial contract of 1000 W; the merged contract then has a power of 1300 W.

3.3.7. Privacy

To meet the social acceptance criteria, we apply confidentiality rules to the data exchanged in the tuple space: personal information about a digital twin cannot be visible to its peers. For example, the different supplies made by producer number A are not visible from producer number B.

These confidentiality rules apply to the various data exchanged such as requests, offers, contracts, or producer confirmations: they consist of protecting each data submitted via an LSA individually. Access to “personal” properties is only permitted for the concerned digital twins.

We use the notion of “wrapper” to protect access to sensitive objects (see Figure 13). We therefore declare the sensitive object as a private attribute in a wrapper class which contains it and which only manages access. More precisely, access is managed in a method of type “accessor” from the wrapper class, which requires the authentication of the requesting digital twin as a parameter. In this way, the access method checks whether the digital twin has correctly authenticated to the micro-grid and, if so, it determines the extent of visibility the digital twin has on the requested data.

The method restricts the data to return, according to the obtained visibility, and it sends a security exception if the digital twin has no access rights to the data.

3.3.8. Link with Social Acceptance by Design

In an upcoming version, we will integrate the different criteria that allow us to identify the relevant behaviours at the scale of a micro-grid. To do this, we will define different metrics to evaluate the degree of contribution of participants in a smart grid community—all prosumers shall consider this degree in the decision-making process to prioritise the energy distribution. We will align our algorithms on sociological studies made in the framework of the LASAGNE project [28]. For example, if a prosumer digital twin has an individualistic behaviour, such as profiting from other digital twins without contributing to the community, all other digital twins of the micro-grid will rate that digital twin with a low score. As a result, the “profiting” digital twin will become less of a priority as a potential consumer. We will therefore define a utility function that evaluates the rating of a prosumer. Each digital twin will thus consider the computed rating in decision-making algorithms, as, for example, to rank the requests to serve first or the offers to process first. This modelling draws inspiration from work on the detection and treatment of “free riding” behaviours in peer-to-peer file exchange networks [29].

For example, in its utility function computation, a digital twin can consider the following characteristics of a prosumer twin:

• Energy consumption;
• Energy supplied and purchased from other community members;
• Equity of energy distribution;
• Comparison of energy sale prices to average market prices;
• Financial margin made on the sale of energy;
• Energy stored (and therefore not contributed to the community).

3.4. Software Architecture of the Model

In the implemented model, the coordination platform is the process that runs the coordination model at the node level, i.e., at a single household. It initialises the digital twins that then evolve autonomously in the environment. A client process is a process that sends a request to the node coordination platform to create, update, or delete a digital twin. It can be a test simulator or a web application that monitors the smart grid at the node level (see Figure 14). A node coordination platform service uses the SAPERE core as a coordination model.

The software architecture of the model therefore has three layers:

• The client layer: these are the processes that send requests to create, modify, and delete digital twins;
• The service layer: this is the process that executes the model at the level of the nodes (the digital twins and their environment). It uses the SAPERE kernel;
• The “derived” SAPERE core: this is the library that manages the tuple space and the coordination laws.

We have developed the service layer, the SAPERE core, and the test simulators in Java language. We use the “Spring boot 2.3.2” framework to run a coordination platform service as a Restful API. We have developed the web application using Angular 9, JavaScript, and D3JS. All data exchanges between the client processes and the coordination platform service are in the JSON format and we encapsulate each JSON content in an HTTP request that follows the REST protocol.

3.5. The Evaluation Procedure

3.5.1. Definition of the Metrics Used

The first objective of the contribution is to be able to satisfy a maximum number of consumers in a relatively short period of time, while avoiding threshold overruns in terms of production or consumption. To do this, we evaluate the two following metrics:

Max response time: 

the maximum time during which a request is not satisfied while the power available at the node still permits it to satisfy it (see Formulas (A1) and (A2) in Appendix B).

Failure rate: 

quantifies all unfulfilled requests over the event history since the beginning of the scenario. It considers the energy not supplied during the test period. It is the ratio of the demands not supplied (in kWh), even though there is sufficient energy, divided by the total requested in kWh over the total duration of the test (see Formula (A3) in Appendix B). Note that this ratio does not consider unsatisfiable requests, e.g., unsatisfied requests whose power exceeds the remaining availability of the node.

3.5.2. Definition of Different Producer’s Policies

For a producer twin, the offer policy defines the priority order of the requests to process. During the tests, we applied different policies of request prioritisation to compare their performances:

• The “Random” policy, for which the producer sorts the requests randomly for the same priority level;
• The “Prioritisation” policy, for which the producer sorts the requests by consumer node distance, decreasing urgency level, decreasing alert duration, and increasing power. The alert duration is the time (in seconds) during which the demand remains unsatisfied while the power produced is sufficient;
• The “Hybrid” policy, which uses by default the random policy. It automatically switches to the prioritisation policy when an unmet request reaches the eight-second alert threshold.

3.5.3. Definition of Different Test Scenarios

We have defined and implemented different scenarios to test the coordination model under different conditions. These scenarios identify the challenges that the digital twins may face in meeting power requirements or in regulating the state of the nodes. Some scenarios focus on specific problems while others attempt to replicate realistic conditions. We define the different types of scenarios as follows:

Scenarios for technical tests:

They are ad hoc scenarios that validate the reliability of a particular algorithm by running digital twins with very specific conditions on a short duration: for example, a technical test evaluates the capacity of all digital twins to satisfy simultaneous energy requests. It can also evaluate the ability of digital twins to manage a threshold overrun or to manage a new urgent request that implies stopping certain ongoing supplies. The technical tests also include simulations of over-production, over-consumption, and the tragedy of the commons. During the evaluation, we repeat these tests dozens of times to obtain more meaningful results.

Realistic scenario:

This scenario seeks to replicate actual consumption and production based on household production and consumption statistics [30]. The statistical data includes the average power consumed or produced at each hour of the day and for each category of appliance in a home. The simulator performs periodic processing for each device category:

• Random assignment of a total consumption (or production) target power that is close to statistical averages;
• Adjustments of device powers to reach the total target (by modifying, starting, and stopping some devices).

We should note that, in the “realistic” scenario, we have pushed the granularity to the maximum by using one digital twin per device. This does not necessarily correspond to the configurations commonly used today. However, the model allows for the representation of aggregations of devices by defining a new higher-level category in the reference data.

Realistic scenario in degraded mode:

The “degraded mode” is a variant of the “realistic” scenario: this scenario contributes to generating shortages by constantly reducing producers’ wattage by half compared to the values of the realistic scenario. In this way, it tests the resilience of digital twins in more difficult situations.

Scenario of living-lab:

Since May 2022, we have been collecting measurement data from the living laboratory “Les Vergers”, located in Meyrin near Geneva. Seven Clemap smart meters installed in four different buildings measure the power data of different electrical appliances (see Figure 3). Each smart meter collects power measurements every 10 s and sends more complete measurements every minute (including voltage and amperage). To facilitate the use of the measurement data, we have set up a script that copies and stores the measurement history in a database accessible from the coordination platform. In this way, we can replay any real scenario that has taken place in the living lab since May 2022. For example, we can run the scenario starting on 1 September 2022 at 1:00 pm: the producer and consumer digital twins will retrieve the corresponding demands and productions (in W) that we have registered in the database for the corresponding moment.

We have made the sources of the implementation available in free access, in a GitHub repository (https://github.com/philippe-glass/energy/tree/master/ (accessed on 8 October 2023)). A docker image of the coordination platform is available on docker hub (https://hub.docker.com/repository/docker/philippeglass1/coordination_platform/general/ (accessed on 8 October 2023)).

4. Results

4.1. Results of Technical Scenarios

Table 2. Best, worst, and average results obtained on 10 technical tests performed for 10 min. The prioritisation policy used consists of sorting energy requests as follows: distance from the consumer node (in ascending order, to favour local requests), urgency level (in descending order), warning duration (in descending order), and power requested (in ascending order). For example, for two requests from the same node and with the same urgency level, the one that has been in warning for longer will be given priority.

	Max Response Time (s)			Failures Rates
Policy Used	Best Score	Worst Score	Average	Best Score	Worst Score	Average
Prioritisation policy	9	15	11.75	0.39%	0.84%	0.61%
Hybrid policy	21	17	19.0	1.54%	2.49%	1.84%
Random policy	80	181	141.0	2.51%	8.63 %	5.92%

displays the results obtained with a purely random scenario: this scenario periodically generates digital twins randomly and in large numbers. It favours requested wattage over produced wattage, which also causes energy scarcity at the node and thus makes contract generation more difficult. The difficulty of fulfilling requests also increases with the number of requests arising at the same time.

The prioritisation policy provides the lowest maximal response time and the lowest failure rate. The use of randomness results in very long alert times (average at 141 s, maximum at 181 s). Given the high volume of requests, it is more difficult to provide offers that meet the demand with the highest level of alert. Indeed, it usually takes several offers for the same request (often more than ten) to fulfil the requested amount of energy. In the case of a random policy, the concerned consumer therefore has a lower probability of obtaining sufficient offers when compared to the prioritisation policy.

The prioritisation policy by alert level focuses on the requests to process first. This is the safest strategy, although it sometimes entails longer response times, since the digital twins all focus on one consumer, the most “on alert” at the current moment. On simpler scenarios, the hybrid policy achieves better results because it tries to keep response times very low by partially using randomness, while removing the risk of getting very long times by dynamically switching to the prioritisation policy after a certain alert threshold. This table shows that the hybrid policy also obtains a very degraded performance when the number of digital twins and the difficulty increase.

4.2. Tests on the Three Main Scenarios: Realistic, Degraded, and Living-Lab

Table 3. Results obtained with the following three scenarios: “realistic”, “degraded mode”, and “living-lab”, tested for 60 min. We ran each scenario with the three producer policies: Prioritisation, Random, and Hybrid policies.

	Max Response Time (s) for Each Scenario			Failures Rate Based on KWH, for Each Scenario
Policy Used	Realistic	Degraded Mode	Living-Lab	Realistic	Degraded Mode	Living-Lab
Prioritisation	13	29	11	0.61%	2.19 %	0.27%
Hybrid policy	13	38	37	0.64%	2.48 %	0.62%
Random policy	25	225	71	0.59%	6.12 %	1.37%

compares the results of the maximum response time and failure rates obtained on the three main scenarios described in Section 3.5.3: “realistic”, “degraded”, and ”living lab”. The “prioritisation” and “hybrid” policies seem to obtain comparable response times, while the random policy still obtains worse response times, although they are acceptable compared to the very poor response times obtained in the "random" scenario. In the various tests of the “realistic” scenario, it turns out that shortages are less frequent than in the “random” scenario. Indeed, based on values close to the statistics provided, the quantities produced generally meet the needs. As a result, shortages are less frequent, and the random strategy is less penalised by the difficulty of obtaining a contract when there are multiple demands to satisfy at the same time.

The results we obtained on the “degraded mode” confirmed a very clear deterioration in the performance of the random policy when shortages are frequent or systematic. This deterioration in performance also concerns the hybrid policy, but to a lesser extent.

Our results on the “iving lab” scenario also confirm a significant deterioration for the random and hybrid policies, but more moderate than the deterioration observed in the degraded mode. The tests carried out from a real situation allow us to confirm once again that we have obtained the best performance with the prioritisation policy.

For the different scenarios, we note that the failure rates follow the same trend as the response times.

5. Discussion

The different results show that the choice of the supply management policy has a significant impact. A “pure” random policy can lead to a very poor performance, especially when conditions become more difficult to generate a contract. We should therefore avoid it. A hybrid policy sometimes achieves better results when digital twins are not subject to scarcity constraints. By choosing the most appropriate policy for the situation, it is possible to achieve good response times with failure rates on a kWh basis below 1%. The maximum response times are generally higher than 10 s. This is due to the execution time of the self-composition mechanism:

• There is a delay in the propagation of information between digital twins. Indeed, the coordination law engine waits one second after the start of the previous cycle before executing the next cycle;
• There are a high number of steps required to validate the contract.

6. Conclusions

We provided a coordination model and its implementation as a coordination platform executing in a Grid Edge Device (GED). We developed a series of intelligent digital twins that interact and coordinate their activities to exchange energy and ensure grid stability. To the best of our knowledge, this implementation is an innovative proposal for managing energy in a smart grid. Indeed, on the one hand, it uses multi-agent systems powering the behaviour of the intelligent digital twins. On the other hand, bio-inspired and self-organising principles present in the coordination model provide self-adaptability to the ongoing events in the grid. The test results confirmed the self-adaptation capabilities as it obtains acceptable response times, even in difficult situations, with many requests at the same time and with insufficient produced power. It can also regulate the use of energy, at the present time or a priori using the prediction in the hour. We have looked at critical situation scenarios, such as the tragedy of the commons, and we have proposed an alternative based on dynamic pricing depending on real-time demand. However, we do not yet allow for dynamic demand reduction when there is not enough energy available. It would be a more appropriate solution to face a diminution in availability. Furthermore, digital twins are not yet capable of negotiating with a peer to find the optimal trade-off. Also, we have not yet modelled the notion of a prosumer twin, which can both consume and provide energy, at different moments or at the same time.

As part of the LASAGNE project, future work will concentrate on the following improvements to enable digital twins to meet certain recurring needs:

• Deployment of the model on real edge devices which have constraints in terms of memory space, disk space, CPU usage, and network security;
• Integration of "Gossip" federated and distributed learning by using the platform. This federated learning approach is purely decentralised and allows each node in a micro-grid to participate fully in the training operations of the learning model and in the distribution of model weights with its neighbours;
• Integration of social acceptance by design in the behaviour of digital twins when they have to take some decisions. This would increase the involvement of micro-grid users;
• Staggering supply over time when energy availability is not sufficient to fully meet all demand at a given time;
• Integration of energy pricing and of digital twins’ policies regarding pricing;
• Integration of negotiation between digital twins (peer-to-peer or multi-lateral negotiation using an organised market);
• Integration of energy storage and adapting the behaviour of digital twins for shortage prevention.
• Modelling the prosumer digital twin, who can be both a producer and a consumer at the same time (A prosumer can buy energy from a producer and sell energy to a consumer at a given time). By drawing inspiration on holonic structures, we could make the prosumer more generic: it could represent a device attached to a micro-grid node or a higher-level entity that represents a single micro-grid or a group of micro-grids;
• Generate contracts for needs to happen in the future. This would avoid some shortages;
• Improve the display on the web application, which currently uses tables to represent the node status. One could imagine a more intuitive visualisation that uses graphical elements to represent devices and energy flows.