Neuro-Evolutionary Synthesis of Game Models of Control under Uncertainty Based on Distributed Computing Technology ^†

Abstract

The methodology basic principles of the neuro-evolutionary synthesis of multi-object multi-criteria systems control models under conflict and uncertainty in real time are discussed. The proposed methodology includes the following main stages: a hierarchical optimization game model under conflict and uncertainty development; a library development of hierarchical coevolutionary algorithms for multi-criteria optimization under conflict and uncertainty; software implementation of hierarchical coevolutionary algorithms library based on distributed computing technology; and game algorithms of control under uncertainty synthesis based on the technology of neural networks ensembles.

1. Introduction

The report discusses the problem of multi-object multi-criteria control systems (MMS) optimizing under conflict and uncertainty in real time. In modern concepts of system analysis, a strict description of the MMS should take into account various types of uncertain factors: uncertainty of the goal, conflict uncertainty, uncertainty of environmental conditions. As we know, the above types of uncertainties can be most fully taken into account by gaming approaches based on the integration of various conflict optimality principles [1,2,3,4]. In particular, in [1], an approach based on the formation and study of stable-effective gaming compromise properties is being developed. Comparison of different approaches to the formation of stable-effective compromises (STEC) is an important principle of game-theoretic analysis of MMS control models, as well as a source of strict and, at the same time, meaningful reasoning about the motivations of the behavior of conflict participants arising from the structure of conflict models.

When solving applied game problems of STEC search under uncertainty (STECU), a number of problems arise. First, the STECU concept is based on the integration of various game-theoretic principles of optimality. Second, the need for real-time implementation of control algorithms, which often arises in applied problems, requires the representation of control actions in the general case in the form of parameterized program-corrected control laws. Such cases are characterized by a high dimension of criterion space and control parameters space, non-linearity, non-convexity, and the presence of break points of vector effectiveness indicators components of MMS subsystems, which determines the high computational complexity of optimization algorithms. These features, combined with the problem of global optimization, make it difficult or impossible to use well-known optimization methods and algorithms to search for a STECU in real time.

Currently, the Machine Learning Control (MLC) methodology, using machine learning methods to solve control problems of complex technical systems, is actively developing. One of the main paradigms of MLC is the technology of neuro-evolutionary synthesis, which is considered as a promising means of implementing intelligent control algorithms under uncertainty in real time [5,6,7].

At the same time, the effectiveness of the neuro-evolutionary approach to the STECU formation is determined primarily by the capabilities of evolutionary algorithms:

–

Complex accounting of the influence of the totality of these uncertain factors;

–

Integration of the principles of optimality used to solve problems of the specified class;

–

The use of advanced architectures, models and methods of distributed computing.

In [8,9,10,11,12], various aspects of improving of the evolutionary computational technology of multi-criteria optimization efficiency, as well as its development in the direction of coevolutionary algorithms constructing and their hybridization, are discussed. In [13,14], the development of coevolutionary technology for a class of multicriteria optimization problems under uncertainty is proposed. Game evolutionary algorithms are considered in [15,16,17,18]. In [19,20,21], an evolutionary computational technology is being developed that provides the possibility of combining various game-theoretic principles of optimality and takes into account various uncertain factors on a single conceptual and algorithmic basis in the task of MMS control optimization under conflict and uncertainty. This technology is implemented in the form of a library of evolutionary algorithms [20] and has been used to solve practical problems of the evolutionary synthesis of neuro-game algorithms of MMS control in real time based on STECU [22,23,24]. The analysis of the results allows us to draw the following main conclusions: the neuro-evolutionary technology of multicriteria control algorithms under conflict and uncertainty synthesis is effective; at the same time, the algorithms of game MMS control models neuro-evolutionary synthesis have extremely high computational complexity; the practical use of neuro-evolutionary technology for solving problems of the specified class requires its implementation on high-performance distributed computing architectures.

2. Main Methodology Stages of Game MMS Control Models under Uncertainty Neuro-Evolutionary Synthesis

The developed methodology of game MMS control models under uncertainty neuro-evolutionary synthesis is presented in the form of a structural scheme in Figure 1 and includes the following main stages.

Stage 1. Development of a library of hierarchical game models (HGM) of optimization under conflict and uncertainty, implementing the basic principles of conflict optimality for various types of conflict interaction, as well as providing the possibility of their integration in the construction of STECU and coordinated STECU (COSTECU). For this purpose, the problem of the MMS control optimizing under conflict and uncertainty is decomposed into the following problems:

–

Local control under uncertainty (LCU);

–

Distributed control under uncertainty (DCU);

–

Hierarchical control under uncertainty (HCU).

The LCU problem is formalized as a problem of multi-criteria optimization under uncertainty (MCOU). To solve it, the principles of vector minimax and vector minimax regret are used. The hierarchical model of MCOU is considered in [24,25].

The DCU problem integrates game problem statements, that cover a wide range of types of conflict interaction and the corresponding principles of conflict equilibrium:

–

Antagonistic interaction (principles of guaranteed result and saddle point);

–

Non-coalition interaction (scalar equilibrium and vector Nash equilibrium, $\mathrm{\Omega }$—equilibrium);

–

Coalition interaction (coalition equilibrium, equilibrium of threats and counter-threats, active equilibrium);

–

Cooperative interaction (arbitration schemes, Pareto optimality principle).

The hierarchical model of Nash vector equilibrium search under uncertainty is considered in [25].

The hierarchical control problem is formulated as a hierarchical game with the right of the first move. To solve it, the principles of scalar and vector Stackelberg equilibrium are used.

At the same time, all principles of conflict optimality are interpreted, taking into account the uncertainty of environmental conditions.

Stage 2. Development of a library of multi-criteria optimization hierarchical coevolutionary algorithms (HCEA) under conflict and uncertainty. HCEA’s structure corresponds to the structure of hierarchical game models of optimization. This provides a flexible algorithm adjustment for the realization of conflict interaction of various types and conflict optimality principles as well as their integration in the formation of STECU and COSTECU. In addition, the hierarchical structure of coevolutionary algorithms best corresponds to the capabilities of distributed computing technologies. All this together provides a significant synergistic effect.

Stage 3. Development of a software package for the synthesis of neuro-gaming control models based on the HCEA library, Docker and Kubernetes [26,27] platforms, graphics processors [28,29], a library of neural network ensemble learning models (NNE). The Docker platform is used for the development, deployment and launch of container applications. The Kubernetes platform is a tool for scaling, managing and coordinating the functioning of containerized applications in a cluster environment. Figure 2 shows the generalized architecture of the neuro-evolutionary synthesis (NES) hierarchical control models (HCM) software, which includes the following structural and functional components.

Cluster. The Kubernetes cluster includes a master node (master), which provides basic Kubernetes services and orchestrates subordinate (worker) nodes that execute various components of the application. Important components of the master node are the tasks controller, which manages the separation of threads, and the queues broker, which acts as an intermediary between the main and working nodes of the cluster. When solving the problem of integrating the principles of optimality included in the structure of the STECU, the infrastructure of the software environment can be organized in the form of several clusters. To accomplish this, Kubernetes provides a mechanism for organizing a cluster federation.

A Docker container is a set of processes isolated from the main operating system. Applications work only inside containers and do not have access to the main system, except for explicitly connected directories when the container is launched (in this case, GPU—Graphic Processing Unit).

Library of HGM:

$$\mathrm{\Gamma }=\{{\mathrm{\Gamma }}_i,i=\overline{1,\left|\mathrm{\Gamma }\right|}\}.$$

(1)

Image Storage:

$$R=\{R_i,i=\overline{1,\left|R\right|}\}.$$

(2)

Each of HGM ${\mathrm{\Gamma }}_i$ in the image storage corresponds to an image, $R_i$, a set of control instructions, according to which a container is formed from the library of algorithms and data that implements the architecture of ${\mathrm{\Gamma }}_i$.

Description of applications (STECU):

$$S=\{S_i,i=\overline{1,\left|S\right|}\}.$$

(3)

The description of each application $S_i$ characterizes the structure of the corresponding container, the composition and the number of replications that must be performed in parallel when implementing the STECU.

Library of Neural Network Ensemble Training Models (NNE). It is used to solve the problem of synthesis of NNE implementing multi-criteria control game algorithms under conflict and uncertainty in real time. The peculiarity of this library is that it presents single-criteria and multi-criteria statements of NNE training tasks: the least squares method (LSM), the multi-criteria least squares method (MLSM), multi-criteria optimization of a general kind, and multi-criteria optimization under uncertainty. Various models of training sets formation oriented on solving game control optimization problems are also presented.

3. Conclusions

The methodology of neuro-evolutionary synthesis of MMS game control models under conflict and uncertainty has been developed. Within the framework of the developed methodology, the following tasks have been solved.

A library of HGM for optimizing the MMS control under conflict and uncertainty has been developed, which allows us to form various schemes for integrating the principles of conflict optimality when a STECU forms on a unified algorithmic basis.

The HCEA library of multi-criteria optimization under conflict and uncertainty has been developed. The hierarchical structure of evolutionary algorithms best corresponds to the capabilities of distributed computing technologies, and in this sense can be considered as a means for structural meta-optimization of coevolutionary parallel algorithms.

A library of NNE training models oriented on solving of multi-criteria conflict control under uncertainty problems has been developed.