A Hierarchical Model of a Vector Nash Equilibrium Search in a Control Problem under Conflict and Uncertainty ^†

Abstract

A hierarchical model of a vector Nash equilibrium search under uncertainty is developed. The sufficient conditions for a vector Nash equilibrium of a noncooperative game under uncertainty are formulated, which can be used as a criterion to achieve the required degree of nonquilibrium for an acceptable solution to the problem of multi-object multicriteria systems’ control optimization under conflict and uncertainty.

1. Introduction

The technology of the neuroevolutionary synthesis of algorithms for multi-object multicriteria systems’ (MMS) control under conflict and uncertainty is based on the development and widespread use of coevolutionary algorithms for finding a game’s stable-effective compromises (STEC) [1,2,3] and coordinated STEC (COSTEC) [4,5,6]. However, these coevolutionary algorithms have extremely high computational complexity. Therefore, the practical use of neuroevolutionary technology for solving problems of the class in question requires its implementation on high-performance parallel computing architectures. One of the basic principles of optimality used in the formation of STEC and COSTEC is the Nash equilibrium principle. In this report, a hierarchical algorithm for a vector Nash equilibrium search under uncertainty is developed. The hierarchical structure of the developed algorithm best corresponds to the structure of coevolutionary algorithms and the capabilities of parallel computing technologies.

2. The Problem Statement

The problem statement of multi-object system control optimization under conflict and uncertainty is formalized in the form of a noncooperative game with uncertainty with vector indicators of the effectiveness of subsystems–players

$$\Gamma =〈\mathbf{N},\{{\mathbf{U}}_i,i\in \mathbf{N}\},\mathbf{Z},\{{\mathbf{J}}_i(\mathbf{u},\mathbf{z}),i\in \mathbf{N}\},\{{\mathbf{\Omega }}_i,i\in \mathbf{N}\}〉.$$

(1)

In (1), $\mathbf{N}=\left\{\overline{1,n}\right\}$—a set of subsystems–players; ${\mathbf{U}}_i$—a set of admissible strategies of the i-th player ${\mathbf{u}}_i\in {\mathbf{U}}_i\subset {\mathbf{E}}^{r_u}$; $\mathbf{u}=\{{\mathbf{u}}_1,\dots ,{\mathbf{u}}_n\}\in \mathbf{U}={\displaystyle \prod _{\mathbf{i}\in \mathbf{N}}}{\mathbf{U}}_{\mathbf{i}}{\mathbf{E}}^{r_u}$—a situation corresponding to the players’ choice of their strategies ${\mathbf{u}}_i\in {\mathbf{U}}_i,i\in \mathbf{N};\mathbf{Z}\subset {\mathbf{E}}^{{\mathbf{r}}_{\mathbf{z}}}$—a set of uncertainty factor admissible values; $\mathbf{z}\in \mathbf{Z};{\mathbf{J}}_i\left(\mathbf{u}\right)\in {\mathbf{E}}^{m_i}$—i-th player vector indicator of the effectiveness; ${\mathbf{\Omega }}_i={\mathbf{E}}_{\le }^{m_i}$—a convex polyhedral dominance cone defining a binary strict preference relation for the i-th player on the set of achievable vector estimates ${\mathbf{J}}_i(\mathbf{U},\mathbf{Z})={\displaystyle \underset{\mathbf{z}\in \mathbf{Z}}{\bigcup _{\mathbf{u}\in \mathbf{U}}}}{\mathbf{J}}_i(\mathbf{u},\mathbf{z})$ and formalizing the requirement to minimize the components of a vector indicator of effectiveness ${\mathbf{J}}_i(\mathbf{u},\mathbf{z})$.

To solve problem (1), the vector Nash equilibrium principle under uncertainty is used. When implementing the computational procedure for finding the optimal solution, the principle of vector Nash $\mathit{\epsilon }$-equilibrium under uncertainty is used.

Definition 1. 

Vector estimation ${\mathbf{V}}_i\left(\mathbf{u}\right)\in {\mathbf{E}}^{m_i}$ is called the point of extreme pessimism for the i-th player in a situation $\mathbf{u}\in \mathbf{U}$ on a set ${\mathbf{J}}_i(\mathbf{u},\mathbf{Z})={\displaystyle \bigcup _{\mathbf{z}\in \mathbf{Z}}}{\mathbf{J}}_i(\mathbf{u},\mathbf{Z})$, if it has the following properties:

$$1){\mathbf{J}}_i(\mathbf{u},\mathbf{Z})\subset {\mathbf{V}}_i\left(\mathbf{u}\right)+{\mathbf{\Omega }}_i;$$

(2)

2) for any $\tilde{\mathbf{V}}\ne {\mathbf{V}}_i\left(\mathbf{u}\right)$, such that $\mathbf{J}(\mathbf{u},\mathbf{Z})\subset \tilde{\mathbf{V}}+{\mathbf{\Omega }}_{\mathbf{i}}$,

$${\mathbf{V}}_i\left(\mathbf{u}\right)-\tilde{\mathbf{V}}\in {\mathbf{\Omega }}_i.$$

(3)

Definition 2. 

Vector estimation ${\mathbf{V}}^T\left(\mathbf{u}\right)=[{\mathbf{V}}_1^T\left(\mathbf{u}\right),\dots ,{\mathbf{V}}_n^T\left(\mathbf{u}\right)]\in {\mathbf{E}}^m$ is called the point of extreme pessimism for the i-th player in a situation $\mathbf{u}\in \mathbf{U}$ on a set $\mathbf{J}(\mathbf{u},\mathbf{Z})={\displaystyle \bigcup _{\mathbf{z}\in \mathbf{Z}}}\mathbf{J}(\mathbf{u},\mathbf{z})$, where ${\mathbf{J}}^T(\mathbf{u},\mathbf{z})=[{\mathbf{J}}_1^T(\mathbf{u},\mathbf{z}),\dots ,{\mathbf{J}}_n^T(\mathbf{u},\mathbf{z})]\in {\mathbf{E}}^m$, $m={\displaystyle \sum _{i\in \mathbf{N}}}{m}_i$.

Definition 3. 

An admissible solution ${\mathbf{u}}^e\in \mathbf{U}$ is called a vector Nash equilibrium under uncertainty in problem (1) if, for any admissible ${\mathbf{u}}_i\in {\mathbf{U}}_i$$i\in \mathbf{N}$, we have:

$$({\mathbf{V}}_i({\mathbf{u}}^e\parallel {\mathbf{u}}_i)-{\mathbf{V}}_i\left({\mathbf{u}}^e\right))\notin {\mathbf{\Omega }}_i,$$

(4)

where we have the situation ${\mathbf{u}}^e\parallel {\mathbf{u}}_i=\{{\mathbf{u}}_1^e,\dots ,{\mathbf{u}}_{i-1}^e,{\mathbf{u}}_i,{\mathbf{u}}_{i+1}^e,\dots ,{\mathbf{u}}_n^e\}.$

Definition 4. 

An admissible solution ${\mathbf{u}}^{e\mathit{\epsilon }}\in \mathbf{U}$ is called a vector Nash ε-equilibrium under uncertainty in problem (1), where ${\mathit{\epsilon }}^T=[{\mathit{\epsilon }}_1^T,\dots ,{\mathit{\epsilon }}_n^T]\in {\mathbf{E}}^m,{\mathit{\epsilon }}_i\in {\mathbf{E}}^{m_i},i\in \mathbf{N}$, if for any admissible ${\mathbf{u}}_i\in {\mathbf{U}}_i$, we have:

$${\mathbf{V}}_i({\mathbf{u}}^{e\mathit{\epsilon }}\parallel {\mathbf{u}}_i)-({\mathbf{V}}_i\left({\mathbf{u}}^{e\mathit{\epsilon }}\right)-{\mathit{\epsilon }}_i)\notin {\mathbf{\Omega }}_i,i\in \mathbf{N}.$$

(5)

3. Representation of the Nash Vector Equilibrium Search Problem under Uncertainty in the Form of a Hierarchical System

We present the problem of searching for a vector Nash equilibrium in a noncooperative game under uncertainty (1) in the form of a hierarchical system, as shown in Figure 1.

In relation to the specified hierarchical structure, an algorithm for a vector Nash equilibrium searching under uncertainty can be represented as a sequence of the following stages.

Stage 1. The first move is made by the coordinating subsystem $C_0$ of the upper level, which forms and communicates its strategy $\mathbf{u}\in \mathbf{U}$ to the subsystems of the lower level. Each subsystem of the lower level $C_i$ estimates the control $\mathbf{u}\in \mathbf{U}$ nonequilibria degree by the corresponding component ${\mathbf{u}}_i\in {\mathbf{U}}_i$. To do this, for a fixed $\mathbf{u}\in \mathbf{U}$, the MOU problem ${\Gamma }_i$ of the following type is solved:

$${\Gamma }_i=〈\mathbf{u}\parallel {\mathbf{U}}_i,\mathbf{Z},{\mathbf{J}}_i(\mathbf{u}\parallel {\mathbf{u}}_i,\mathbf{z}),{\mathbf{\Omega }}_i〉i\in \mathbf{N}.$$

(6)

In (6), it is required to minimize the vector criteria components ${\mathbf{J}}_i(\mathbf{u},\mathbf{z})$ on the set $(\mathbf{u}\parallel {\mathbf{U}}_i)$ under uncertainty $\mathbf{z}\in \mathbf{Z}$. To solve problem (6), the vector minimax principle and the coevolutionary MOU algorithm are used [4,5]. The result of problem (6) is a vector minimax set ${\mathbf{V}}_i^{{\mathbf{\Omega }}_i}(\mathbf{u}\parallel {\mathbf{U}}_i)={\mathbf{V}}_i(\mathbf{u}\parallel {\mathbf{U}}_i^{{\mathbf{\Omega }}_i})\subset {\mathbf{E}}^{m_i}$.

Stage 2. The second step is estimation by the coordinating subsystem $C_0$ of the coordinating control $\mathbf{u}\in \mathbf{U}$ effectiveness. Based on the results of problem (6), an area is formed for each $i\in \mathbf{N}$:

$${\widehat{\mathbf{U}}}_i^{{\mathbf{\Omega }}_i}=\left\{\begin{array}{c}{\mathbf{u}}_i\in {\mathbf{U}}_i^{{\mathbf{\Omega }}_i},\hfill \\ ({\mathbf{V}}_i(\mathbf{u}\parallel {\mathbf{u}}_i)-{\mathbf{V}}_i\left(\mathbf{u}\right))\in {\mathbf{\Omega }}_i.\hfill \end{array}\right.$$

(7)

We construct the objective function:

$${\Phi }_i(\mathbf{u},{\mathbf{u}}_i)=\parallel {\mathbf{V}}_i(\mathbf{u}\parallel {\mathbf{u}}_i)-{\mathbf{V}}_i\left(\mathbf{u}\right))\parallel ,$$

(8)

and we formulate the optimization problem:

$${\Phi }_i(\mathbf{u},{\mathbf{u}}_i)\to \underset{{\mathbf{u}}_i\in {\widehat{\mathbf{U}}}_i^{{\mathbf{\Omega }}_i}}{max}.$$

(9)

Let ${\mathbf{u}}_i^{*}\in {\widehat{\mathbf{U}}}_i^{*}$ be the optimal solution to problem (9). Then, the vector ${\mathit{\epsilon }}_i^{*}\left(\mathbf{u}\right)={\mathbf{V}}_i(\mathbf{u}\parallel {\mathbf{u}}_i^{*})-{\mathbf{V}}_i\left(\mathbf{u}\right))$ characterizes the maximum degree of control of the $\mathbf{u}\in \mathbf{U}$ nonequilibrium by component ${\mathbf{u}}_i\in {\mathbf{U}}_i$ for the player–subsystem $C_i$, where $\parallel {\mathit{\epsilon }}_i^{*}\left(\mathbf{u}\right)\parallel ={\Phi }_i(\mathbf{u}\parallel {\mathbf{u}}_i^{*})$.

We construct a vector indicator of the coordinator’s effectiveness:

$$\mathbf{F}\left(\mathbf{u}\right)={[f_1\left(\mathbf{u}\right),\dots ,f_n\left(\mathbf{u}\right)]}^T$$

(10)

with components of the form:

$$f_i\left(\mathbf{u}\right)=\parallel {\mathit{\epsilon }}_i^{*}\left(\mathbf{u}\right)\parallel ,i\in \mathbf{N}.$$

(11)

The efficiency indicator (10) and (11) characterizes the degree of nonequilibrium of permissible control $\mathbf{u}\in \mathbf{U}$ relative to all subsystems–players $C_i$.

Stage 3. The problem of optimal coordination is solved at the subsystem $C_0$ level. To do this, we construct an objective function of the form:

$$\Psi \left(\mathbf{F}\left(\mathbf{u}\right)\right)=\underset{i\in \mathbf{N}}{max}\left\{f_i\left(\mathbf{u}\right)\right\},$$

(12)

and we formulate the optimization problem:

$$\Psi \left(\mathbf{F}\left(\mathbf{u}\right)\right)\to \underset{\mathbf{u}\in \mathbf{U}}{min}.$$

(13)

The following statement is true.

Theorem 1. 

Let ${\mathbf{u}}^{*}\in \mathbf{U}$ be the optimal solution to problem (13), and $\Psi \left({\mathbf{u}}^{*}\right)={\mathbf{e}}^{*}$. Then, there exists a vector ${\mathit{\epsilon }}^T=[{\mathit{\epsilon }}_1^T,\dots ,{\mathit{\epsilon }}_n^T]\in {\mathbf{E}}^m$ having the properties:

a) $\parallel {\mathit{\epsilon }}_i\parallel =e^{*},i\in \mathbf{N}$;

b) ${\mathbf{u}}^{*}$ is a vector Nash ε-equilibrium of a noncooperative game under uncertainty (1).

4. Conclusions

A hierarchical model of a vector Nash equilibrium search under uncertainty was developed, which is the basis for constructing parallel hierarchical coevolutionary algorithms for the MMS control optimization under noncooperative conflict interaction and uncertainty.

The sufficient conditions for a vector Nash $\mathit{\epsilon }$-equilibrium of a noncooperative game under uncertainty were formulated, which can be used as a criterion to achieve the required degree of nonequilibrium of an acceptable solution in hierarchical coevolutionary algorithms of multicriteria conflict optimization.

The proposed hierarchical model is a component of the neuro-evolutionary technology of control algorithms and information processing synthesis in MMS under conflict and uncertainty in real time.