Ontologies to Enable Interoperability of Multi-Agent Electricity Markets Simulation and Decision Support

Abstract

This paper presents the AiD-EM Ontology, which provides a semantic representation of the concepts required to enable the interoperability between multi-agent-based decision support systems, namely AiD-EM, and the market agents that participate in electricity market simulations. Electricity markets’ constant changes, brought about by the increasing necessity for adequate integration of renewable energy sources, make them complex and dynamic environments with very particular characteristics. Several modeling tools directed at the study and decision support in the scope of the restructured wholesale electricity markets have emerged. However, a common limitation is identified: the lack of interoperability between the various systems. This gap makes it impossible to exchange information and knowledge between them, test different market models, enable players from heterogeneous systems to interact in common market environments, and take full advantage of decision support tools. To overcome this gap, this paper presents the AiD-EM Ontology, which includes the necessary concepts related to the AiD-EM multi-agent decision support system, to enable interoperability with easier cooperation and adequate communication between AiD-EM and simulated market agents wishing to take advantage of this decision support tool.

1. Introduction

The work presented in this paper was initially developed and partially presented in the scope of Gabriel Santos’ MSc Thesis [1]. This paper presents the specific contribution related to the conception, development, and experimentation of a dedicated ontology to enable the interoperability between multi-agent electricity market simulators, smart grid simulators, and decision support systems, as detailed in this manuscript.

The emergence of liberalized electricity markets (EM) completely revolutionized the power sector business. Several challenges have been brought about by the sector’s restructuring process. It required the transformation of the conceptual models that previously dominated the power sector [2]. The market became more competitive but also more complex, posing new challenges to its participants, forcing them to rethink their market strategies and consequently their behavior. The new challenges that EM restructuring produced increased the importance of EM operation study. The raised complexity and competitiveness of the market, together with its unpredictable evolution, hardens the decision-making process [3].

Several models have emerged trying to overcome market challenges. Despite the guidance provided by some pioneer countries’ experience in what regards the implemented market models’ performance, it is still premature to make definitive conclusions [2]. Thereby, the use of tools that allow the study of different market mechanisms and the relationships between market entities becomes essential. The use of simulation tools becomes decisive to study, analyze, and test different alternatives for markets’ structure and evolution, providing entities with decision support tools to address the new challenges.

These tools aim to handle the EM’s constantly evolving reality, ensuring the appropriate solutions for actors to adapt themselves to this new reality, acquiring experience to operate in this changing economic, financial, and regulatory environment. Operators must guarantee a transparent and competitive market while market players try to minimize their costs and maximize their profits [4].

For the analysis of dynamic and complex environments, such as the EM, multi-agent-based simulators are particularly well suited [5]. Multi-agent approaches enable easy inclusion of new market models and mechanisms, new types of market players, and interactions [6]. Some reference modeling tools of this domain are: the Agent-based Modelling of Electricity Systems (AMES) [7], the Electricity Market Complex Adaptive System (EMCAS) [8], the Multi-Agent Simulator of Competitive Electricity Markets (MASCEM) [6,9], and Multi-Agent Smart Grid Simulation Platform (MASGriP) [10,11], developed by the authors’ research team. These are relevant tools but often directed to the perspective of market operators and regulators while not providing adequate decision support to the participating players. In this scope, AiD-EM (Adaptive Decision Support for Electricity Market Negotiations) has been developed [12] to aid players in their negotiation process.

The adequate applicability of multi-agent simulation to the study of EM has already been confirmed. However, there is a lack of interoperability between the various systems. These tools focus only on the study of the different market players and mechanisms and the analysis of the relations between those entities, but still, they do not allow interoperability with heterogeneous tools of the same domain [13], which is a significant limitation.

These systems can benefit significantly from sharing their models with each other, allowing agent-based players from external systems to participate in the same market environment and learn from each other. This paper presents the AiD-EM Ontology, developed to enable semantic modeling of the concepts required for communication between the decision support system and different market simulators. This way, interoperability between the heterogeneous systems is achieved, allowing a strong improvement in the potential for EM studies and development.

Section 2 presents a review of the most relevant work on multi-agent semantic interoperability and existing ontologies for decision support algorithms, while Section 3 overviews the multi-agent-based simulators addressed by this work. Section 4 presents the AiD-EM Ontology, while Section 5 details a case study based on real data from the Iberian EM operator. Section 6 features the final conclusions.

2. Multi-Agent Semantic Interoperability

The Foundation for Intelligent Physical Agents (FIPA) is an IEEE Computer Society standards organization devoted to promoting agent-based technology and the interoperability of its standards with other technologies [14]. FIPA’s agents’ communication model is based on the assumption that two agents wishing to exchange messages share a common ontology for the subject domain, ensuring that agents assign the same meaning to the message’s symbols [15].

FIPA proposes the use of an Ontology Agent (OA) for Multi-agent System (MAS) environments. The OA should grant access to one or more ontology servers which, in turn, provide ontology services to the agent community. Not every MAS needs to include an OA, but if the OA exists in the agent’s community to promote interoperability, it must be compliant with FIPA’s specification as well as the services described in [15].

Designers may decide to develop explicit and declaratively represented ontologies stored publicly or implicitly encoded ontologies that are not available in an ontology service. Furthermore, for a given domain, it is possible to use ontologies implicitly encoded within the agent’s implementation, and full agent communication and understanding is still achievable. However, in these cases, the services provided by the OA do not apply [15].

Therefore, ontologies provide relevant interoperability perspectives between different systems by enabling a correct interpretation of agents’ messages, thus facilitating the interaction between agents of distinct natures and characteristics. It is a crucial requirement for the study of highly complex and dynamic systems, containing numerous different types of agents with a high volume of interactions between them, such as the power system environment and EM in specific.

2.1. Semantic Interoperability

The integration of heterogeneous MAS raises inherent issues of the interconnection of those systems, particularly those using ontologies independently developed [16]. These issues need to be addressed in the power industry to disseminate the development of interoperable MAS [17] and take full advantage of their functionalities. Open standards are needed to provide full interoperability.

There are several proposals for the use of ontologies in the scope of PES [18,19,20,21,22], with emphasis on IBM’s ontology developed under the Insights Foundation for Energy Data Model [23] and also in the field of EM [24,25,26]. Some European projects are also advancing in the reasoning of ontologies, namely the ENERGy and Web of Objects projects. However, most of these ontologies focus on the needs of utilities, and others, such as [24], only consider the domain of the ontology, leaving aside their application scenario, which makes them unusable in realistic application scenarios. Moreover, very few of these ontologies are publicly available, which hardens their reuse and extension.

Catterson [27], alternatively, suggests the use of an upper ontology that defines only general concepts of the domain and ensures a common basis for the representation of those concepts and their relationships between heterogeneous systems. At the same time, it reduces the complexity of ontology mapping. However, bearing in mind that applications must develop lower-level ontologies for all the application-specific concepts. According to Catterson [28], there will be no need to modify existing agents when a new system is integrated if the agents conform to a single upper ontology, as high-level concepts are universally understood, and only these concepts are discussed. The disadvantage of this approach is that defining high-level concepts is a very arduous and complex task, which requires a universal acceptance from all entities involved in the field.

Upper Ontology Overview

An upper, top-level, or more accurately foundational ontology, is a high-level ontology that addresses very general domains (e.g., time, space, inheritance, instantiation, identity, measure, quantity, functional dependence, process, event, attribute, boundary, among others) [29,30] supporting broad semantic interoperability between lower-level ontologies derived from it. It should serve as a common neutral backbone complemented by ontologies of more specific domains, such as medicine, engineering, and geography.

Upper ontologies facilitate interoperability and mutual understanding between people and machines, including comprehension of the reasons for non-interoperability. These motives are sometimes more important than developing an integrated system based on generic shared semantics but conceptually imperfect and unpredictable [29]. The entities (i.e., concepts and relationships) covered by the upper ontology must be abstract enough since these ontologies are supposed to cover all relevant knowledge aspects of each entity’s domain.

2.2. Addressed Solution

Inspired by this last approach, in [6,31], an upper ontology, containing the main concepts required by the entities that participate in electricity markets and power systems simulators, has been proposed. However, considering the definition of upper ontology, the solution formerly suggested in [31] of developing an upper ontology for the EM and power systems domain dropped, approaching the problem from a different but near perspective.

Rather than developing an “electricity markets and power systems upper ontology” as has been initially considered, the authors proposed developing ontologies for the interoperability of EM multi-agent simulation platforms. These ontologies [32,33,34,35,36,37] can be extended to enable the full interoperability with MASCEM [6,9,13]. The developed ontologies are publicly available on MASCEM’s website (http://www.mascem.gecad.isep.ipp.pt/ontologies, accessed on 25 May 2021) so that they can be used by third-party developers wishing to integrate their agent-based tools with MASCEM. The ontologies may also be used and extended for developing new agent-based simulation tools in the wholesale EM context.

In addition to providing MASCEM with semantic interoperability, it’s also essential to supply external players semantic interoperability with AiD-EM, so they can also take advantage of this EM decision-support tool. To this end, we have developed the AiD-EM Ontology.

2.3. Decision-Support Ontologies

Given AiD-EM’s decision-support algorithms (see Section 3.3), a literature search for contemporary and publicly available machine learning and data mining ontologies is mandatory to confirm if any is suitable for reuse and extension.

The OntoDM ontology [38] is a reference and modular ontology for the data mining domain. To ensure compatibility and mappings with other ontologies, it extends the Basic Formal Ontology (BFO) [39] upper-ontology and reuses concepts and relations from the Ontology of Biomedical Investigations (OBI) [40], the Information Artifact Ontology (IAO) [41], and the Software Ontology (SWO) [42]. OntoDM is composed of three modules covering different aspects of data mining, namely the Ontology of Core Data Mining Entities (OntoDM-core) [38], the Ontology of Datatypes (OntoDT) [43], and the Ontology of Data Mining Investigations (OntoDM-KDD) [44]. The OntoDM-core module formalizes the key data mining concepts for representing the mining of structured data. The OntoDT module supports the representation of knowledge about data types. Finally, the OntoDM-KDD module formalizes the knowledge discovery process.

The Exposé ontology [45], in turn, describes machine learning and data mining experiments and workflows in a standardized vocabulary to support collaborative analysis of learning algorithms. It reuses and extends concepts from the BFO, OBI, IAO, and OntoDM ontologies covering experiments in fine detail, including the experiment context, evaluation metrics, performance estimation techniques, datasets, and algorithms. Exposé is a complementary ontology to OntoDM and was designed to be used together with data mining and machine learning experiment databases, enabling a wide variety of querying on machine learning and data mining algorithm performance. Additionally, it supports semantic reasoning, meta-learning, data integration, and logical consistency check.

To the best of the authors’ knowledge, these are currently the only two publicly available ontologies for the data mining and machine learning domains. Section 4 describes the development of the AiD-EM Ontology, which promotes interoperability with the AiD-EM decision support tool, in the scope of the EM.

3. Multi-Agent-Based Simulators Overview

Multi-agent technology is being increasingly used for modeling, representing, studying, and simulating the complex EM [6,7,8,9,10]. MASs are widely chosen as the option for developing complex simulation tools with evolving environments. Two of the main reasons are the facilitated inclusion and enlargement of new and existing models and the potential of representing distinct actors as autonomous software agents with their behaviors, profiles, and goals.

The means that to achieve more complex and advanced simulation, studies are provided by the collaboration between the different MAS. The integration of complementary multi-agent simulators, such as MASGriP or AiD-EM extends MASCEM’s core simulation environment.

3.1. MASCEM

MASCEM [9,13] was developed to study the operation of the competitive and complex EM. It is a simulation tool able to model the main complex market entities and respective interactions. Medium/long-term gathering of data and experience is considered, aiming to support players’ decisions in the market according to their goals and characteristics. Figure 1 illustrates MASCEM’s main features.

MASCEM seeks to model and simulate as many market models, operators, and players as possible, trying to mimic real EM operations. Thus, MASCEM can be used as a decision support tool, considering both short/medium-term purposes and long-term decisions, such as real market regulators.

It is implemented in the Java Agent Development framework (JADE) [47]. JADE “is an open source platform for the development of peer-to-peer agent based applications” that supports and complies with FIPA’s specifications. It simplifies implementing agent-based applications through middleware, providing a set of graphical tools to support debugging and deployment.

MASCEM’s multi-agent model only includes five distinct types of agents, namely the Main Agent, the MIB Agent, the Market Operator, the System Operator, and the Player. The first two agents are not part of the EM environment. Their importance is related to the simulation tool, easing the execution of distributed and simultaneous simulations [13].

MASCEM allows the simulation of several market types from day-ahead to intraday (asymmetric or symmetric, with or without constraints), bilateral contracts, and forward markets. The simulation of hybrid markets is also possible by selecting a combination of the available market models. The simulation of several scenarios simultaneously is also enabled, where each scenario has its agents, i.e., the Market Operator, the System Operator, and the market Players. This way, the execution time of the simulations is optimized, while the study and comparison of various simulation scenarios are accomplished simultaneously.

3.2. MASGriP

MASGriP [10,11] is a MAS that models the internal operation of Smart Grids (SG) and Microgrids (MG). This system models all the typically involved players through software agents capable of representing and simulating their actions.

MASGriP simulates, manages, and controls the most relevant players acting in an SG and MG environment. Additionally, some small players are directly connected to physical installations, providing the means for automatic management of the associated resources. This enables the development of a complex system capable of performing simulations with an agent society that contains both simulated players and real infrastructures, providing the means to test alternative approaches in a realistic simulation environment [11]. To complement simulations with the analysis of the impact of the methods in the energy flows and transmission lines, MASGriP uses real-time simulation [48].

MASGriP provides a simulation platform that allows the test and analysis of different types of models, namely energy resource management methodologies, contract negotiation methods, energy transaction models, and diverse types of Demand Response (DR) programs and events.

Players in MASGriP have been implemented to reflect the real world. Operators, such as the Distribution System Operator (DSO) and the Independent System Operator (ISO), have also been included. However, most players represent energy resources such as different types of consumers (e.g., industrial, commercial, residential) and producers (e.g., wind farms, solar plants, co-generation units); electric vehicles with vehicle-to-grid capabilities, among others.

The interface that allows the interaction between real players (humans) and real hardware (loads, generators units, storage systems, etc.) is achieved by using an interface agent that enables communication with the hardware. Communications are performed using Internet Protocol (IP) to communicate with a Programmable Logic Controller (PLC) and RS-485 to communicate with soft-starters, measurement units, etc.

3.3. AiD-EM

AiD-EM [12] provides decision support to EM players by enabling a contextual adaptation to the competitors’ actions and reactions. AiD-EM uses a multi-agent approach to integrate several specific decision support systems directed to different problems. The combination of such decision-support solutions aims to improve players’ performance in EM negotiations.

AiD-EM’s portfolio optimization model supports players’ decisions when acting in multiple alternative/complementary EM. The energy volume a player should negotiate in each market type is optimized. The expected prices in each market are considered in different contexts, optimizing the expected outcome.

The Adaptive Learning Strategic Bidding System (ALBidS) [5,49] is used for decision support in auction-based markets. ALBidS is a multi-agent-based decision support system that considers a large diversity of alternative market strategies (a quite complete survey about this topic is available in [50]). The description of the included market negotiation strategies can be consulted in [5], including those mentioned in the case study of this paper, e.g., different approaches based on linear regression and simple market price averages that differ depending on the data that is used, and approaches that analyze the historical tendencies of the forecasting error, referred to as Error Theory. The general concept behind ALBidS is the integration of as many distinct market strategies as possible, whose performance is evaluated under different contexts of negotiation. The system then learns which strategies are the most adequate and present the highest chance of success in each different context. The learning process of ALBidS uses reinforcement learning algorithms, namely the Roth–Erev algorithm [51] and an algorithm based on the Bayesian theorem of probability proposed in [52]. Additionally, an Efficiency and Effectiveness (2E) balance management mechanism has been developed to control the balance between the execution time and quality of results of the executed algorithms, according to the needs in each simulation.

The Decision Support for Energy Contracts Negotiation (DECON) [53] is used for decision support in bilateral contract negotiations. DECON integrates decision-support solutions for the pre-negotiation stage of negotiations and also the actual negotiation process. During the pre-negotiation stage, the potential competitor(s) that present the best perspectives of successful negotiation outcomes are identified. The expected limits and target prices of each targeted competitor are also predicted and estimated. The decision support for actual negotiations is based on a set of distinct tactics and strategies, both time-dependent and behavior-dependent. DECON also supports different tactic combinations, allowing players to adapt their tactics strategically.

4. AiD-EM Ontology

Reusing existing ontologies is a good option for systems interoperability. One never knows if others’ work can be extended and refined for its particular domain or task. However, first, the domain and purpose of the ontology must be clearly defined. AiD-EM is a decision support tool designed to assist MASCEM’s players in EM simulation participation. Thus, the AiD-EM’s ontology aims to provide external agents (from MASCEM or other tools) interoperability with AiD-EM’s decision-support agents, making it an application (low level, specific) ontology. Taking a closer look at the publicly available ontologies (see Section 2.3), Exposé models the data mining and machine learning experiments and workflows, which is not suitable nor needed for the AiD-EM tool, while OntoDM describes high-level (abstract) concepts in the data mining domain. However, the way OntoDM structures knowledge and OntoDM’s granularity are unsuitable for reuse, given AiD-EM’s data model. For this reason, AiD-EM Ontology has been developed from scratch.

The AiD-EM Ontology (ADM) (http://www.mascem.gecad.isep.ipp.pt/ontologies/aid-em.owl, accessed on 25 May 2021) has been developed to provide interoperability between AiD-EM and any EM player of any agent-based simulation platform, such as MASCEM. It imports the Electricity Markets Ontology (EMO) [32,33] and extends it, including new classes, object, and data properties. Figure 2 highlights ADM’s new classes (left column), object properties (middle column), and data properties (right column).

By analyzing Figure 2, it is possible to observe a large number of classes, objects, and data properties added, mainly related to AiD-EM’s available strategies and tools. Figure 3 (http://www.mascem.gecad.isep.ipp.pt/ontologies/paper/journal/18/adm.png, accessed on 25 May 2021) illustrates the relations (object properties) and attributes (data properties) of ADM’s concepts (classes). The classes imported from EMO are in yellow. The “EMO:” prefix identifies the concepts and properties imported from EMO.

Figure 3 provides a more detailed vision of ADM’s semantic model. There are three main types of strategies, namely: MarketStrategy, RLAStrategy and PortfolioStrategy. Each one is related to a specific problem, and it may be used by more than a Tool. ADM has expressiveness ALCHIQ(D), similarly to EMO. For further details on ADM’s expressiveness, please see [33,34].

Table 1. AiD-EM Ontology object properties in DL syntax.

Object Properties
hasRequest ⊑ R	⊤ ⊑ ≤ 1 hasRequest
useScenarioAnalysisMethod ⊑ R	⊤ ⊑ ≤ 1 useScenarioAnalysisMethod
hasStrategy ⊑ R	useStrategy ⊑ R
requiresTool ⊑ R	useTool ⊑ R
participateInSession ⊑ R

and

Table 2. AiD-EM Ontology data properties in DL syntax.

Data Properties
a ⊑ ∪	⊤ ⊑ ≤ 1 a
admissibleError ⊑ ∪	⊤ ⊑ ≤ 1 admissibleError
angle ⊑ ∪	⊤ ⊑ ≤ 1 angle
b ⊑ ∪	⊤ ⊑ ≤ 1 b
bidInterval ⊑ ∪	⊤ ⊑ ≤ 1 bidInterval
bidsNumber ⊑ ∪	⊤ ⊑ ≤ 1 bidsNumber
e ⊑ ∪	⊤ ⊑ ≤ 1 e
eInsensitive ⊑ ∪	⊤ ⊑ ≤ 1 eInsensitive
error ⊑ ∪	⊤ ⊑ ≤ 1 error
experimentation ⊑ ∪	⊤ ⊑ ≤ 1 experimentation
firstCost ⊑ ∪	⊤ ⊑ ≤ 1 firstCost
hiddenNodes ⊑ ∪	⊤ ⊑ ≤ 1 hiddenNodes
initQ ⊑ ∪	⊤ ⊑ ≤ 1 initQ
kernelFunction ⊑ ∪	⊤ ⊑ ≤ 1 kernelFunction
learningRate ⊑ ∪	⊤ ⊑ ≤ 1 learningRate
limit ⊑ ∪	⊤ ⊑ ≤ 1 limit
m1 ⊑ ∪	⊤ ⊑ ≤ 1 m1
m2 ⊑ ∪	⊤ ⊑ ≤ 1 m2
m3 ⊑ ∪	⊤ ⊑ ≤ 1 m3
maxProduction ⊑ ∪	⊤ ⊑ ≤ 1 maxProduction
message ⊑ ∪	⊤ ⊑ ≤ 1 message
minProduction ⊑ ∪	⊤ ⊑ ≤ 1 minProduction
numberOfScenarios ⊑ ∪	⊤ ⊑ ≤ 1 numberOfScenarios
offset ⊑ ∪	⊤ ⊑ ≤ 1 offset
r ⊑ ∪	⊤ ⊑ ≤ 1 r
recency ⊑ ∪	⊤ ⊑ ≤ 1 recency
riMaxL ⊑ ∪	⊤ ⊑ ≤ 1 riMaxL
riMaxU ⊑ ∪	⊤ ⊑ ≤ 1 riMaxU
risk ⊑ ∪	⊤ ⊑ ≤ 1 risk
scalingFactorA ⊑ ∪	⊤ ⊑ ≤ 1 scalingFactorA
scalingFactorB ⊑ ∪	⊤ ⊑ ≤ 1 scalingFactorB
secondCost ⊑ ∪	⊤ ⊑ ≤ 1 secondCost
temperature ⊑ ∪	⊤ ⊑ ≤ 1 temperature
totalAmountToNegotiate ⊑ ∪	⊤ ⊑ ≤ 1 totalAmountToNegotiate
trainingLimit ⊑ ∪	⊤ ⊑ ≤ 1 trainingLimit
useSimulatedAnnealing ⊑ ∪	⊤ ⊑ ≤ 1 useSimulatedAnnealing
weight ⊑ ∪	⊤ ⊑ ≤ 1 weight

describe the ADM’s object and data properties, respectively, in description logic (http://www.obitko.com/tutorials/ontologies-semantic-web/owl-dl-semantics.html, accessed on 25 May 2021) (DL) syntax.

In DL syntax, “$property⊑R$” means that property is an object property, “$property⊑\cup$” means that property is a data property, and “$\top ⊑\le 1property$” means that property is functional. A functional property is a property that only allows for a single value for each instance of the class. Good examples of functional properties are the ‘has father’ and ‘has mother’ object properties, where a ‘Child’ instance can only have a (biological) father and a (biological) mother.

Table 3. AiD-EM Ontology root classes in DL syntax.

Classes
Request ⊑ ⊤ ⊓ 1 message
Strategy ⊑ ⊤ ⊓ 1 weight
Tool ⊑ ⊤
ValuePartition ⊑ ⊤
Player ⊑ EMO:Player ⊓ 1 totalAmountToNegotiate ⊓ ∃ hasRequest Request ⊓
∃ participateInSession EMO:Session ⊓ ∃ useStrategy 1 RLAStrategy ⊓
∃ useStrategy MarketStrategy ⊓ ∃ useStrategy PortfolioStrategy ⊓ ∃ hasTool Tool
Session ⊑ EMO:Session ⊓ ∃ useStrategy 1 RLAStrategy ⊓
∃ useStrategy MarketStrategy
Request ⊓ Strategy ⊓ Tool ⊓ ValuePartition ⊓ EMO:Area ⊓ EMO:Operator ⊓
EMO:Period ⊓ EMO:Power ⊓ EMO:Price ⊓ EMO:Market ⊓ EMO:MarketType
⊓ EMO:Bid ⊓ EMO:Session ⊓ EMO:Offer ⊓ EMO:Player
⊓ EMO:BilateralContract = ⊥

presents the DL syntax of ADM’s root classes.

The Request object includes a message, while a Strategy is composed of a weight data property to determine the weight to attribute to the strategy’s results. The Tool concept represents an abstract tool, and the ValuePartition class is a pattern that enables the specification and restriction of certain values that a property can be associated with. A Player is an EMO:Player here redefined to include: the totalAmountToNegotiate; the Request to be sent to AiD-EM; a RLAStrategy, a MarketStrategy and a PortfolioStrategy, if intended; one or more Tools; and also an EMO:Session identifying the session in which the player will participate through the use of the participateInSession object property. Session is subclass of EMO:Session including a RLAStrategy and/or one or more MarketStrategy. Finally, all root classes (including EMO’s classes) are disjoint classes; that is, none of these classes have members in common, or else it will make the ontology inconsistent.

Table 4. AiD-EM Ontology sub-classes in DL syntax.

Classes
MarketStrategy ⊑ Strategy
PortfolioStrategy ⊑ Strategy
RLAStrategy ⊑ Strategy
ALBidS ⊑ Tool ⊓ ∃ hasStrategy MarketStrategy ⊓
∃ hasStrategy RLAStrategy
Bilat ⊑ Tool
PortfolioOptimization ⊑ Tool ⊓ ∃ hasStrategy PortfolioStrategy ⊓
∃ requiresTool 1 ALBidS
ScenarioAnalysisMethod ⊑ ValuePartition

introduces the DL syntax of Strategy’s, Tool, and ValuePartition sub-classes.

The MarketStrategy, PortfolioStrategy and RLAStrategy are subclasses of Strategy. The MarketStrategy refers to the EM negotiation strategies, the PortfolioStrategy is referent to the portfolio optimization methods, and the RLAStrategy is related to the use of reinforcement learning algorithms (RLA) to select the most suitable strategies in each context. ALBidS, Bilat and PortfolioOptimization are subclasses of Tool since they represent AiD-EM’s available tools for decision support. ALBidS may include one or more MarketStrategy and one or more RLAStrategy, using the object property hasStrategy. In turn, the PortfolioOptimization requires the use of ALBidS, through the use of requiresTool object property, including also a PortfolioStrategy by using hasStrategy too, if desired. The ScenarioAnalysisMethod is a subclass of ValuePartition 

Table 5. AiD-EM Ontology MarketStrategy’s sub-classes in DL syntax.

Classes
AdaptiveDerivativeFollowing ⊑ MarketStrategy ⊓ 1 scalingFactorA ⊓
1 scalingFactorB
AMES ⊑ MarketStrategy ⊓ 1 a ⊓ 1 admissibleError ⊓ 1 b ⊓ 1 e ⊓ 1 initQ ⊓ 1 m1 ⊓
1 m2 ⊓ 1 m3 ⊓ 1 maxProduction ⊓ 1 minProduction ⊓ 1 r ⊓ 1 riMaxL ⊓ 1 riMaxU ⊓
1 temperature ⊓ 1 useSimulatedAnnealing
ANN ⊑ MarketStrategy ⊓ 1 hiddenNodes ⊓ 1 trainingLimit
ANNMetalearner ⊑ MarketStrategy
Average1 ⊑ MarketStrategy
Average2 ⊑ MarketStrategy
Average3 ⊑ MarketStrategy
Average4 ⊑ MarketStrategy
ComposedGoalDirect ⊑ MarketStrategy
DeterminismTheory ⊑ MarketStrategy
EconomicAnalysiz ⊑ MarketStrategy ⊓ 1 risk
ErrorTheoryA ⊑ MarketStrategy
ErrorTheoryB ⊑ MarketStrategy
ErrorTheoryC ⊑ MarketStrategy
GameTheory ⊑ MarketStrategy ⊓ 1 bidInterval ⊓ 1 bidsNumber ⊓
1 numberOfScenarios ⊓ ∃ useScenarioAnalysisMethod 1 ScenarioAnalysisMethod
LinearRegression1 ⊑ MarketStrategy
LinearRegression2 ⊑ MarketStrategy
LinearRegression3 ⊑ MarketStrategy
LinearRegression4 ⊑ MarketStrategy
MarketPriceFollowing ⊑ MarketStrategy
SA-QL ⊑ MarketStrategy ⊓ 1 admissibleError ⊓ 1 bidInterval ⊓ 1 bidsNumber ⊓
1 learningRate ⊓ 1 temperature
SimpleMetalearner ⊑ MarketStrategy
STHMetalearner ⊑ MarketStrategy
SVM ⊑ MarketStrategy ⊓ 1 angle ⊓ 1 eInsensitive ⊓ 1 limit ⊓ 1 offset ⊓
(kernelFunction "RBF" ⊔ kernelFunction "eRBF")
WeightedMetalearner ⊑ MarketStrategy

presents the DL syntax of MarketStrategy’s sub-classes.

Table 5 shows that the strategies AdaptiveDerivativeFollowing, AMES, ANN, GameTheory, SA-QL, and SVM extend the MarketStrategy class by adding the needed properties for each case. Although the remaining classes are not adding any property, they are useful to distinguish each available strategy. Finally,

Table 6. AiD-EM Ontology PortfolioStrategy’s sub-classes in DL syntax.

Classes
CaseBaseLearning ⊑ PortfolioStrategy
Deterministic ⊑ PortfolioStrategy
EPSO ⊑ PortfolioStrategy
GA ⊑ PortfolioStrategy
QPSO ⊑ PortfolioStrategy
SimulatedAnnealing ⊑ PortfolioStrategy
StandardPSO ⊑ PortfolioStrategy
TabuSearch ⊑ PortfolioStrategy

introduces the PortfolioStrategy’s sub-classes,

Table 7. AiD-EM Ontology RLAStrategy’s sub-classes in DL syntax.

Classes
BayesianTheorem ⊑ RLAStrategy
RothErev ⊑ RLAStrategy
SimpleRLA ⊑ RLAStrategy

presents the RLAStrategy’s sub-classes, and

Table 8. AiD-EM Ontology ScenarioAnalysisMethod’s sub-classes in DL syntax.

Classes
LearningMethod ⊑ ScenarioAnalysisMethod
OptimisticAnalysis ⊑ ScenarioAnalysisMethod
ScenariosAverage ⊑ ScenarioAnalysisMethod

details the ScenarioAnalysisMethod’s sub-classes in DL syntax.

Analyzing the tables, one can see that none of the remaining sub-classes add properties to extend their parent classes. However, these sub-classes are relevant to clearly identify the instances of each class.

4.1. Application of the Proposed Ontology

After developing the ontology, it is necessary to develop code to interpret the messages exchanged by software agents. For this purpose, the Apache Jena (http://jena.apache.org/, accessed on 25 May 2021) Java framework is used. A specific module has been developed for each type of agent expected to use the ontology, following MASCEM’s architecture [13]. Figure 4 exposes the communications exchanged between the agents of the different MAS. The libraries developed for each type of agent are described briefly in the following subsections. For a more detailed specification, please refer to [1].

4.1.1. MASCEM’s Main Agent

MASCEM’s Main Agent is the agent responsible for the interaction between MASCEM’s agents and users. A library has been developed to convert the user’s input data into the corresponding agents’ knowledge bases (KB). These are sent to each agent after being generated.

4.1.2. MASCEM’s Market Operator

The Market Operator agent is responsible for pool negotiations regulation, including the validation of bids, the determination of the market price and the accepted/refused bids, and the economic dispatch. This agent must be able to: (i) read its KB and generate the respective Call for Proposal (CfP) to be sent to the participating players; (ii) interpret the incoming bids and convert them into the respective objects to execute the market; and (iii) to create the respective responses with the results of each player agent. For this purpose, another library has been developed.

4.1.3. Player

The Player agent represents any market player willing to participate in MASCEM’s negotiations, such as retailers, prosumers, or aggregators. Any player participating in MASCEM is also able to request AiD-EM’s decision support. This agent must be capable of: (i) storing his KB; (ii) interpreting and storing the CfP sent by the market operator; (iii) generating the Request for Support (RfS) to be sent to AiD-EM’s Main Agent; (iv) interpreting the available strategies sent by the AiD-EM’s Manager Agent; (v) responding with the selected ones, including the bid proposal for the current market session; (vi) forwarding the proposal sent by AiD-EM’s Manager Agent to MASCEM’s Market Operator; (vii) interpreting and storing the market results achieved during the pool negotiations. In this case, two libraries were developed: one to deal with MASCEM’s interactions and another to deal with AiD-EM’s.

4.1.4. AiD-EM’s Main Agent

The AiD-EM’s Main Agent has a similar role to MASCEM’s Main Agent but is related to the AiD-EM platform. Besides being responsible for the system’s interaction with the users, it is also responsible for directing external agents to their manager agents for decision support. To this end, a library has been developed to (i) convert the user’s input data into its KB; and (ii) interpret the received RfS and redirect it to a dedicated manager agent.

4.1.5. AiD-EM’s Manager Agent

AiD-EM’s Manager Agent is responsible for supporting a player agent during the decision support event, i.e., until the end of the market’s session. The library developed for this agent (i) generates the list of the available tools and strategies to send to the Player agent; (ii) interprets the selection of strategies and tools sent by the Player agent; (iii) and generates a market proposal with the prices determined by the decision support tools.

5. Case Study

This section introduces a case study aiming at demonstrating the applicability of the proposed ontology for the interoperability between heterogeneous systems. It demonstrates the interoperability between MASCEM, MASGriP, and AiD-EM, by exploring the use of AiD-EM’s decision support by a MASGriP player that participates in a MASCEM’s market simulation, as illustrated by Figure 5.

The scenario was created using the Realistic Scenario Generator (RealScen) [54], a tool developed to automatically generate EM simulation scenarios based on data extracted by the automatic data extraction tool introduced in [55]. This simulation scenario considers real data of all Portuguese and Spanish market participants, a total of 1428 players. With this information, it is possible to represent the key players of the Iberian market, MIBEL [56].

The representation of MIBEL is done by considering 110 players. From Player 1 to Player 55, buyer agents are represented, being the remaining seller agents. In this scenario, only Player 56 (a seller agent from MASGriP) uses AiD-EM’s decision support in an attempt to maximize its profits, and only the day-ahead (or spot) market is considered, consisting of 24 hourly periods, for the day 18 February 2015. Since this scenario only considers one simulation day, the advantage of using AiD-EM’s decision support is not highlighted in the present case study, once its learning is only effective from the second simulation day.

At the beginning of the simulation, MASCEM’s Main Agent reads the input data and generates the knowledge bases (KB)—represented in the Resource Description Framework (https://www.w3.org/RDF/, accessed on 25 May 2021) (RDF)—of each agent. After the creation of the agents, each one receives a message with its respective KB. Since the remaining communications have already been demonstrated in previous works [32,33,34], they are not presented here. This case study highlights the communications process between MASGriP’s and AiD-EM’s agents only.

After the market operator sends the call for proposals to all the registered players, players gather the necessary information, by querying their KB, to submit their proposals to the market operator. Player 56 also prepares its proposal, but before submitting it to the market operator, it requests AiD-EM’s support by sending a request message to AiD-EM’s Main Agent. Figure 6 (http://www.mascem.gecad.isep.ipp.pt/ontologies/paper/journal/18/rfs.rdf, accessed on 25 May 2021) presents the request for a support message sent by Player 56 to the AiD-EM’s Main Agent.

As it is possible to observe from Figure 6, the request message sent by Player 56 imports concepts from the aid-em.owl ontology (from line 15 to 18) which, in turn, imports concepts from the electricity-markets.owl ontology. The sent message identifies the player requesting support (between lines 10 and 14) and also includes a simple text message of type aid-em.owl#Request (lines 19 to 22). This type of message informs the AiD-EM’s Main Agent that Player 56 is requesting support.

After receiving the call for support, the AiD-EM’s Main Agent redirects Player 56’s request to a dedicated AiD-EM’s Manager Agent. This will be the agent that, from this point on, will aid Player 56 on his needs for support in the market.

The AiD-EM’s Manager Agent starts by informing Player 56 about which types of market and strategies are available for decision support. AiD-EM provides strategies for the decision support of spot markets (day-ahead/intraday) and bilateral contracts. It also offers the portfolio optimization methodology, which, at every moment, chooses the most advantageous market types for the player to negotiate.

Figure 7 shows a snippet of the message sent by the AiD-EM’s Manager Agent to Player 56. The complete version is available online (http://www.mascem.gecad.isep.ipp.pt/ontologies/paper/journal/18/rfs-response.rdf, accessed on 25 May 2021).

Analyzing Figure 7, it is possible to verify the description of some of the various strategies available, such as GameTheory (from line 19 to line 21), SimpleMetalearner (between lines 34 and 36) and SVM (from line 37 to 39). After receiving the available strategies for decision support, Player 56 chooses the ones that best fits their needs.

In this case, Player 56 is only interested in using the spot market strategies since they will not participate in bilateral contacts. For this reason, they also ignore portfolio optimization. Therefore, Player 56 chooses to use only ALBidS [5,49] with the market strategies: Average 1, Average 2, Average 3, Average 4, Linear Regression 1, Linear Regression 2, Linear Regression 3, Linear Regression 4, ANN and Weighted Metalearner; and the reinforcement learning algorithm (RLA) SimpleRLA.

Player 56 sends both the selected strategies and the amount of energy to bid for each hourly period to AiD-EM’s Manager Agent. Figure 8 exposes a snippet of the options sent by Player 56 to AiD-EM’s Manager Agent. The complete RDF content is publicly available online (http://www.mascem.gecad.isep.ipp.pt/ontologies/paper/journal/18/strategies.rdf, accessed on 25 May 2021).

In a brief analysis of Figure 8, it is possible to verify the choice of the market strategy LinearRegression3 (from line 29 to line 31), together with the definition of available offers for the day-ahead session of the daily market (the remaining lines). The LinearRegression3 is just one of the strategies selected by this player. The remaining are available on the internet location mentioned above.

The AiD-EM Manager agent receives the strategies selection and the proposal from Player 56. After running the selected algorithms, it answers Player 56 with the price suggestions, taking into account the simulation date and type of session. Figure 9 shows a snippet of the answer sent to Player 56. A full version of the message content is available online (http://www.mascem.gecad.isep.ipp.pt/ontologies/paper/journal/18/proposal.rdf, accessed on 25 May 2021). In the fragment below, it is possible to see the price suggested for the hourly period 8 (between lines 28 and 32).

After receiving the action proposal from AiD-EM’s Manager Agent, Player 56 sends its proposal with the suggested prices to the Market Operator. After validating all incoming proposals, the Market Operator executes the market session and sends the results to the participant players. As stated before, only the RDF concerning the interoperability of the MASGriP agent (Player 56) with AiD-EM is supplied in this case study; on the one hand, this is to prevent the excessive extension of the document, and on the other hand, examples of the remaining communications have already been provided in previous works.

The outcomes of Player 56 are analyzed to demonstrate the impact of using AiD-EM’s decision support. This player takes advantage of AiD-EM’s services, trying to influence the market price to achieve higher profits. Figure 10 illustrates the results of Player 56 in the day-ahead market session of the Iberian market.

As it is possible to observe in Figure 10, Player 56 was able to sell all their available energy.

Table 9. Comparison between the price offered by Player 56 and the market price.

Period	Offered Price (€)	Market Price (€)
1	29.10	53.00
2	44.46	50.06
3	39.75	40.53
4	40.25	40.53
5	40.51	40.53
6	40.50	40.53
7	20.50	51.02
8	53.24	54.13
9	57.10	57.88
10	51.71	58.64
11	63.29	63.51
12	59.77	60.09
13	47.58	63.51
14	50.98	63.51
15	46.13	58.64
16	57.76	58.64
17	57.51	57.88
18	23.80	57.88
19	54.53	55.03
20	54.42	55.03
21	54.45	55.03
22	59.61	60.09
23	59.72	60.09
24	53.35	57.88

compares the prices presented by Player 56 with the resulting market prices.

When comparing the established market price with the prices offered by this player, one can see that Player 56’s price offers are all below the achieved market price. That is essential to guarantee the sale while keeping the bid price close to the market price, contributing to preventing the market price from lowering too much, which would result in a lower selling price.

Using the publicly available AiD-EM Ontology, any player participating in a spot electricity market or bilateral contract can request AiD-EM’s assistance taking advantage of its decision support in the prices’ definition for the negotiation. Being an independent MAS, AiD-EM supports not only players from MASCEM but also players from other wholesale EM simulators.

6. Conclusions

This work presents the AiD-EM Ontology (ADM), a module extended from EMO related to the AiD-EM decision support tool. It is public and available online (http://www.mascem.gecad.isep.ipp.pt/ontologies/aid-em.owl, accessed on 25 May 2021) to be easily accessed, (re)used, or extended by Ontology Engineers or MAS developers in the EM domain. ADM enables the interoperability between MASCEM, AiD-EM, and MASGriP, thus facilitating the integration and co-simulation between different independent MAS in the EM domain.

Results show that the presented ontology enables the interoperability between AiD-EM and a market player from MASGriP, participating in an EM simulation of MASCEM. Player 56 (from MASGriP) requested AiD-EM’s decision support for the definition of prices for the day-ahead market of MIBEL, using AiD-EM’s public ontology. Using the proposed and publicly available ADM, this independent MAS enables the interoperability with any agent from any MAS that may request decision support in the definition of prices within the scope of competitive EM.

The achieved interoperability allows knowledge exchange and promotes adopting common semantics, enabling the communication between heterogeneous systems. In this way, by opening the simulation environment to other systems, the opportunity of integrating different market models is created, allowing agents from other systems with very distinct characteristics to be able to interact in joint simulations. Following the proposed ontologies, the messages exchanged by the involved agents can be correctly interpreted by all.

As future work, semantic approaches must be extended to the systems’ intrinsic rules and constraints. Besides enabling the systems’ interoperability by representing the required concepts to support communications, it will also allow the automated execution of the market models and decision-support algorithms without re-programming the systems whenever a change is needed. The development of semantic agent services catalogs is also suggested, enabling the registration of new agents with new capabilities and roles, which can be easily found and approached by the remaining agents present in the multi-agent environment.