Sustainable Cooperation in a Bicriteria Game of Renewable Resource Extraction

Abstract

We study a multi-objective finite-horizon game model of renewable, common resource extraction where the players have two separate objectives (one is economic success; the other describes the players’ environmental concern). We derive the cooperative strategy and the subgame-perfect Pareto equilibrium in linear-state non-stationary feedback strategies by employing the dynamic programming approach. Since the utility is transferable only based on the economic criterion, we need to revise the concept of time consistency and the payoff-distribution procedure to provide a mechanism for sustainable long-term cooperation. All the results are illustrated with a numerical example.

1. Introduction

We consider a dynamic competitive model of renewable resource extraction as a finite-horizon multi-objective game with a feedback information structure. Dynamic models of the resource extraction management have been an active research area for few decades (see, e.g., surveys [1,2]). The most famous example is the “great fish war game” (see, e.g., [3,4,5,6,7,8,9,10,11,12,13,14,15,16]). The general assumption in those models is that every player seeks to maximize some economic objective (utility, profit, revenue, etc.); hence, the unicriterion (scalar) game framework is used, and the environmental concern is taken into account only through the resource stock dynamics or the terminal payoff, which may depend on the terminal stock of the resource. Although the multi-objective approach (see, e.g., [17,18,19,20]) could better simulate the competitive behavior in real-life applications, there are still few attempts (such as [21,22,23,24,25,26]) to apply a multicriteria game framework to dynamic resource-extraction management.

A multicriteria extension of the great fish war model was introduced in [23] by adding the environmental objective in particular to all players’ specifications. The non-cooperative and cooperative solutions in stationary strategies were derived for an infinite-horizon bicriteria game with almost symmetric players (the possible players’ asymmetry was allowed only by using different weighting coefficients (see [17,20,21,24,27]), which the players assign to the objectives.

In the paper, we consider a finite-horizon version of the bicriteria multistage model [23] and take into account two additional sources of the players’ asymmetry: different discount factors and different coefficients which the players use to evaluate the residual resource stock. The aim of the research was to derive optimal non-cooperative (namely, feedback subgame-perfect Pareto equilibrium) strategies and cooperative behavior for this finite-horizon bicriteria model and study the problem of sustainability of the long-term cooperative solution. The substantial distinction from the infinite-horizon game is that now the equilibrium and cooperative strategies are non-stationary. Since the utility function is assumed to be transferable (between the players) only for the economic criterion, and the cooperative solution for every subgame along the cooperative trajectory depends on both (economic and environmental) objectives, we need to revise the time consistency (see, e.g., [2,5,28,29,30,31,32]) property and the payoff-distribution procedure (see, e.g., [2,14,33,34,35]) before applying these concepts to ensure the sustainability of the cooperative agreement in the multi-objective game under consideration.

Our approach is similar to that of B. Crettez, N. Hayek and P. Kort [21,22], namely, we also use two separate objectives and apply a classical approach introduced in [20] to define and compute the non-cooperative (Pareto equilibrium) and cooperative solutions. However, we consider a resource-management problem rather than a pollution-control problem, consider the finite-horizon dynamic game, apply a different solution method to derive a feedback subgame-perfect Pareto equilibrium [27,36,37] and the cooperative solution and provide a mechanism for sustainable cooperation. The paper is related to, but also differs from the papers by A. Rettieva, who has introduced an original solution concept based on the Nash bargaining scheme and applied this approach in [24,25,26] to multicriteria fishery management models.

The contributions of the paper are twofold:

• We derived in explicit form non-cooperative (namely, a subgame-perfect Pareto equilibrium) feedback strategies and the cooperative behavior for a bicriteria finite-horizon multistage game of renewable resource extraction with environmentally concerned asymmetric players.
• We revised the concepts of time-consistency and payoff-distribution procedure for multi-objective game when the utility was assumed to be transferable in some but not all criteria and provide a mechanism to guarantee sustainable long-term cooperation.

The rest of the paper is organized as follows. In Section 2, we specify the bicriteria finite-horizon game. Using a weighted game approach, we derive a non-cooperative solution (subgame-perfect Pareto equilibrium in feedback strategies) and study its properties in Section 3. Cooperative behavior is studied in detail in Section 4. In Section 5, we determine a single-point cooperative solution and derive the payoff-distribution procedure which satisfies time consistency, and hence provides an efficient tool to ensure sustainable cooperation in dynamic multi-objective game. We provide a numerical example in Section 6 and briefly conclude in Section 7.

2. The Model

Consider a multi-objective finite-horizon model of renewable resource extraction in discrete time. Suppose $N=\{1,\dots ,n\}$ players exploit a common renewable resource. Denote by $s\left(t\right)$ the resource stock at time $t=0,1,\dots ,T$ and by $e_j(t,s\left(t\right))$ the extraction level of player $j\in N$ in that period.

Following [2,10,14,24,26], we assume the linear dynamics of the resource stock evolution, that is,

$$s(t+1)=\alpha ·s\left(t\right)-\sum _{j=1}^n{e}_j(t,s\left(t\right)),s\left(0\right)=s_0,$$

(1)

where $\alpha \ge 1$ is the natural growth rate. We adopt in the paper the feedback information structure—that is, $e_j(·)=e_j(t,s\left(t\right)),j\in N,t=0,\dots ,T-1.$

To keep it simple, while still being able to highlight the multicriteria setting issues, we assume that each player has two objectives to optimize. The first (economic) objective function has the form

$$u_j^1(·)=\sum _{\tau =0}^{T-1}{\delta }_j^{\tau }·ln{e}_j(\tau ,s\left(\tau \right))+{\delta }_j^T·R_j^1·lns\left(T\right),j\in N,$$

(2)

where ${\delta }_j\in (0,1)$ is the discount factor and $R_j^1>0$ is a parameter that describes the player j’s valuation of the resource residual stock at time T. A positive salvage value in the economic objective function can be seen as an additional way to limit the total current extraction level (say, the annual fish supply on the market) to support the reasonable market price (see, e.g., [6]). On the other hand, the last term in (2) takes into account the opportunity to continue the resource extraction in the future.

The second (environmental or ecological) performance criterion

$$u_j^2(·)=\sum _{\tau =0}^{T-1}{\delta }_j^{\tau }·lns\left(\tau \right)+{\delta }_j^T·R_j^2·lns\left(T\right),j\in N,$$

(3)

takes into account player $j^{\prime }s$ concern of the resource level maintenance. Again, the parameter $R_j^2>0$ indicates the special role of the ultimate resource amount in the player’s environmental concern. To save on parameters, we set $R_j^1=R_j^2=R_j,j\in N$.

Note that following [21,26], we assume that an environmental objective has a similar form for different players. However, different values of ${\delta }_j$ and $R_j$ (and different weighting coefficients—see, e.g., [17,21,27]) may describe the players’ asymmetry—in particular, the different levels of the environmental concern. Some sources and aspects of the players’ asymmetry are discussed in [4,10,16,25,38]. Cabo and Tidball [38] suppose that players have asymmetric valuation of the cleaner environment and asymmetric responsibility for the state of the environment when studying time-consistent environmental agreements. We allow different levels of ${\delta }_j$ and $R_j$ and different relative weights ${\lambda }_j$ of the economic objective in the players’ performance criteria to describe the players’ asymmetry when studying non-cooperative behavior in Section 3.

Reynolds [39] and Crettez, Hayek and Kort [22] paid special attention to the so-called “partially symmetric NE” assuming that the players are symmetric except with regard to the choices of the relative weight associated with the environmental objective. We employ a similar approach when studying cooperative behavior in Section 4—namely, we consider almost symmetric players but with asymmetric valuation of the residual resource stock (i.e., we assume different values of $R_j$).

Denote by $G^{1,2}(n,s\left(0\right),T)$ the bicriteria multistage (namely, $(T+1)$-stage) n-player game starting at $t=0$ and $s\left(0\right)$ with the resource dynamics (1), feedback information structure, and vector objective functions with components (2) and (3). Each intermediate state $s\left(t\right)$, $t=1,\dots ,T-1$, generates the $(T-t+1)$-stage subgame $G^{1,2}(n,s\left(t\right),T-t)$, starting at time t and current state $s\left(t\right)$ with the vector objective functions ${\tilde{u}}_j^t(·)=\left(u_j^{1t}(·),u_j^{2t}(·)\right),j\in N,$ where

$$u_j^{1t}(·)=\sum _{\tau =t}^{T-1}{\delta }_j^{\tau -t}·ln{e}_j(\tau ,s\left(\tau \right))+{\delta }_j^{T-t}·R_j·lns\left(T\right),$$

(4)

$$u_j^{2t}(·)=\sum _{\tau =t}^{T-1}{\delta }_j^{\tau -t}·lns\left(\tau \right)+{\delta }_j^{T-t}·R_j·lns\left(T\right).$$

(5)

3. Non-Cooperative Solution

Following [20,21,27,36,37], we use the concepts of Pareto equilibria and subgame-perfect Pareto equilibria as non-cooperative solutions in a multiobjective dynamic game. For any vectors $a,b\in R^m$ and $m⩾2$, let $a\ge b$ mean that $a_j⩾b_j$ and $j=1,\dots ,m$, and at least one inequality is strict.

Definition 1.

A feedback strategy profile $e^P(·)=(e_1^P(·),\dots ,e_n^P(·))$ is a Pareto equilibrium ($PE$) in $G^{1,2}(n,s\left(0\right),T)$, if for each player $j\in N$ there is no such feedback strategy $e_j(·)$ that

$${\tilde{u}}_j(e_j(·),e_{-j}^P(·))\ge {\tilde{u}}_j\left(e^P(·)\right).$$

(6)

Let $PE\left(G^{1,2}(n,s\left(0\right),T)\right)$ denote a set of all Pareto equilibriums in $G^{1,2}(n,s\left(0\right),T)$.

Definition 2.

A feedback strategy profile $e^P(·)\in PE\left(G^{1,2}(n,s\left(0\right),T)\right)$ is called a subgame-perfect Pareto equilibrium ($SPPE$) in $G^{1,2}(n,s\left(0\right),T)$, if for each $t=1,2,\dots ,T-1$ and current state $s\left(t\right)$ the restriction of $(e_1^P(·),\dots ,e_n^P(·))$ onto the subgame $G^{1,2}(n,s\left(t\right),T-t)$ forms a Pareto equilibrium in this subgame.

Following [20,21,23,27], we consider a weighted (unicriterion) game $G(n,s\left(0\right),T,{\left({\lambda }_j^1,{\lambda }_j^2\right)}_{j\in N})$, with dynamics (1) and feedback information structure but with scalar objective functions

$$\begin{array}{c}{\widehat{u}}_j={\lambda }_j^1·u_j^1(·)+{\lambda }_j^2·u_j^2(·)=\hfill \\ ={\lambda }_j^1\left({\displaystyle \sum _{\tau =0}^{T-1}}{\delta }_j^{\tau }ln{e}_j(\tau ,s\left(\tau \right))+{\delta }_j^T{R}_{j}lns\left(T\right)\right)+{\lambda }_j^2\left({\displaystyle \sum _{\tau =0}^{T-1}}{\delta }_j^{\tau }lns\left(\tau \right)+{\delta }_j^T{R}_{j}lns\left(T\right)\right),j\in N,\hfill \end{array}$$

(7)

where ${\lambda }_j^1,{\lambda }_j^2$ are some non-negative coefficients (weights the player j assigns to corresponding objectives); ${\lambda }_j^1+{\lambda }_j^2=1$. As was firstly proved in [20], each Pareto equilibrium in multicriteria game $G^{1,2}(n,s\left(0\right),T)$ could be obtained as a Nash equilibrium [40] in a weighted game $G(n,s\left(0\right),T,{\left({\lambda }_j^1,{\lambda }_j^2\right)}_{j\in N})$ for some weights ${\left({\lambda }_j^1,{\lambda }_j^2\right)}_{j\in N}$. The same correspondence holds for $SPPE$ in the multicriteria game and the subgame-perfect equilibrium in a weighted game (see, e.g., [27]).

Henceforth, we consider a truly bicriteria game where each player takes into account both economic and environmental objectives with positive weights, that is, $0<{\lambda }_j^1<1$, $j\in N$. To save on parameters, let ${\lambda }_j=\frac{{\lambda }_j^1}{{\lambda }_j^2}=\frac{{\lambda }_j^1}{1-{\lambda }_j^1}$ denote the relative weight of the economic objective in the performance criterion of player j. Note that ${\lambda }_j>0$, and ${\lambda }_j>1$ (${\lambda }_j<1$) means that the economic objective is more (less) important for player j than the environmental one.

Moreover, a weighted game $G(n,s\left(0\right),T,{\left({\lambda }_j^1,{\lambda }_j^2\right)}_{j\in N})$ with objective function (7) is strategically equivalent to the game $G(n,s\left(0\right),T,{\left({\lambda }_j\right)}_{j\in N})$ with the so called "environmentally normalized" objective functions

$$\begin{array}{c}{u}_j(·)=\frac{1}{{\lambda }_j^2}·{\widehat{u}}_j(·)=\hfill \\ ={\lambda }_j·\left({\displaystyle \sum _{\tau =0}^{T-1}}{\delta }_j^{\tau }ln{e}_j(\tau ,s\left(\tau \right))+{\delta }_j^T{R}_{j}lns\left(T\right)\right)+{\displaystyle \sum _{\tau =0}^{T-1}}{\delta }_j^{\tau }lns\left(\tau \right)+{\delta }_j^T{R}_{j}lns\left(T\right),j\in N.\hfill \end{array}$$

(8)

Similar correspondence is valid for the subgames $G(n,s\left(t\right),T-t,{\left({\lambda }_j\right)}_{j\in N})$ starting at $(t,s(t\left)\right)$, $t=1,\dots ,T-1$, with environmentally normalized subgame objective functions

$$\begin{array}{cc}{u}_j^t(·)\hfill & ={\lambda }_j·\left({\displaystyle \sum _{\tau =t}^{T-1}}{\delta }_j^{\tau -t}ln{e}_j(\tau ,s\left(\tau \right))+{\delta }_j^{T-t}{R}_{j}lns\left(T\right)\right)+\hfill \\ & +{\displaystyle \sum _{\tau =t}^{T-1}}{\delta }_j^{\tau -t}lns\left(\tau \right)+{\delta }_j^{T-t}{R}_{j}lns\left(T\right),j\in N.\hfill \end{array}$$

(9)

We applied the dynamic-programming method to derive the feedback-equilibrium strategies (see, e.g., [29]) in a two-player game $G(n=2,s\left(0\right),T,{\lambda }_1,{\lambda }_2)$ which forms $SPPE$ in a multicriteria game $G^{1,2}(n=2,s\left(0\right),T)$ with asymmetric players (coefficients ${\delta }_j$, $R_j$ and relative weights ${\lambda }_j$ describe different aspects of the players’ asymmetry).

Proposition 1.

Assuming an interior solution, the linear form of the strategies and log-linear form of the value functions, the unique feedback-$SPPE$ strategy profile is given by

$$\left\{\begin{array}{c}{e}_1^{SPPE}\left(s\left(t\right)\right)={\displaystyle \frac{\alpha {\delta }_2}{{\lambda }_2}}·{\displaystyle \frac{A_2(t+1)}{\Delta (t+1)}}·s\left(t\right)\hfill \\ e_2^{SPPE}\left(s\left(t\right)\right)={\displaystyle \frac{\alpha {\delta }_1}{{\lambda }_1}}·{\displaystyle \frac{A_1(t+1)}{\Delta (t+1)}}·s\left(t\right)\hfill \end{array}\right.,t=0,\dots ,T-1,$$

(10)

where

$$A_j\left(t\right)=1+{\lambda }_j+{\delta }_j{A}_j(t+1),t=0,\dots ,T-1;A_j\left(T\right)=R_j(1+{\lambda }_j),$$

(11)

$$\Delta (t+1)={\displaystyle \frac{{\delta }_1}{{\lambda }_1}}{A}_1(t+1)+{\displaystyle \frac{{\delta }_2}{{\lambda }_2}}{A}_2(t+1)+{\displaystyle \frac{{\delta }_1{\delta }_2}{{\lambda }_1{\lambda }_2}}·A_1(t+1)A_2(t+1).$$

(12)

The $SPPE$ state trajectory is given by

$$s^{SPPE}(t+1)={\displaystyle \frac{\alpha {\delta }_1{\delta }_2}{{\lambda }_1{\lambda }_2}}·{\displaystyle \frac{A_1(t+1)A_2(t+1)}{\Delta (t+1)}}·s\left(t\right)={\mu }^{SPPE}\left(t\right)·s\left(t\right),t=0,\dots ,T-1.$$

(13)

The environmentally normalized value functions $V_j^{SPPE}(t,s\left(t\right))$ which correspond to the $SPPE$ payoffs in the subgames $G(n=2,s\left(t\right),T-t,{\lambda }_1,{\lambda }_2)$ are

$$V_j^{SPPE}(t,s\left(t\right))=A_j\left(t\right)·lns\left(t\right)+B_j\left(t\right),$$

(14)

where

$$\begin{array}{cc}{B}_j\left(t\right)\hfill & ={\lambda }_{j}ln\left({\displaystyle \frac{\alpha {\delta }_{3-j}}{{\lambda }_{3-j}}}·{\displaystyle \frac{A_{3-j}(t+1)}{\Delta (t+1)}}\right)+{\delta }_j{A}_j(t+1)·ln\left\{{\displaystyle \frac{\alpha {\delta }_j{\delta }_{3-j}}{{\lambda }_j{\lambda }_{3-j}}}·{\displaystyle \frac{A_j(t+1)A_{3-j}(t+1)}{\Delta (t+1)}}\right\}+\hfill \\ & +{\delta }_j{B}_j(t+1),t=0,\dots ,T-1;B_j\left(T\right)=0.\hfill \end{array}$$

(15)

Proof. 

Assuming that strategies are linear in $s\left(t\right)$ and taking into account the structure of the objective functions (8) and (9), one can guess the log-linear form (14) of the value functions. Then, the Bellman equations take the following form:

$$\begin{array}{c}{V}_j^{SPPE}(t,s\left(t\right))=A_j\left(t\right)lns\left(t\right)+B_j\left(t\right)=\underset{e_j(t,s\left(t\right))}{max}\{{\lambda }_{j}ln{e}_j(t,s\left(t\right))+lns\left(t\right)+\hfill \\ +{\delta }_j·\left[A_j(t+1)·ln\left(\alpha s\left(t\right)-e_j(t,s\left(t\right))-e_{3-j}^{SPPE}(t,s\left(t\right))\right)+B_j(t+1)\right]\},j=1,2.\hfill \end{array}$$

(16)

By solving the first-order conditions, we derive the $SPPE$ strategies (10). Then, by substituting (10) into the state dynamics Equation (1), we get the $SPPE$ trajectory (13).

Finally, one can substitute functions (10) and (13) in (16) and compare coefficients in the LHS and RHS to derive $A_j\left(t\right)$ and $B_j\left(t\right)$ in the recursive forms (11) and (15). □

Let us establish some properties of the non-cooperative ($SPPE$) behavior in the bicriteria game $G^{1,2}(n=2,s\left(0\right),T)$.

Remark 1.

Function $A_j\left(t\right)$ in (11) is decreasing in t if

$${\delta }_j>1-{\displaystyle \frac{1}{R_j}}.$$

(17)

The result follows from straightforward solving inequalities $A_j(t-1)>A_j\left(t\right)$ backwards in $t=T,T-1,\dots ,1$.

Remark 2.

The $SPPE$ state transition function ${\mu }^{SPPE}\left(t\right)$ is increasing in $A_1(t+1)$ and in $A_2(t+1)$.

One can estimate the sign of partial derivative ${\displaystyle \frac{\partial \mu }{\partial A_j}}$, and the results follow. The next property follows from Remarks 1 and 2.

Remark 3.

Function ${\mu }^{SPPE}\left(t\right)$ decreases in time t.

Proposition 2.

Assuming that ${\delta }_1={\delta }_2=\delta$ and ${\lambda }_1={\lambda }_2=\lambda$, the resource stock $s\left(t\right)$ is non-decreasing in t along the $SPPE$ state trajectory (13) for all $t=0,\dots ,T$, if one of the following conditions holds:

$$R_1+R_2-(\alpha -1)\delta R_1{R}_2\le 0,$$

(18)

$$\left\{\begin{array}{c}{R}_1+R_2-(\alpha -1)\delta R_1{R}_2>0,\hfill \\ \lambda \le {\overline{\lambda }}^{SPPE}={\displaystyle \frac{(\alpha -1)\delta R_1{R}_2}{R_1+R_2-(\alpha -1)\delta R_1{R}_2}}.\hfill \end{array}\right.$$

(19)

Proof. 

Since ${\mu }^{SPPE}\left(t\right)$ is decreasing in time t, we need to solve inequality ${\mu }^{SPPE}\left(t\right)\ge 1$ only for $t=T-1$. Then,

$$\frac{\alpha {\delta }^2}{{\lambda }^2}{A}_1\left(T\right)A_2\left(T\right)\ge \Delta \left(T\right)=\frac{\delta }{\lambda }\left(A_1\left(T\right)+A_2\left(T\right)\right)+\frac{{\delta }^2}{{\lambda }^2}{A}_1\left(T\right)A_2\left(T\right).$$

By substituting (11), we get

$$\lambda (R_1+R_2-(\alpha -1)\delta R_1{R}_2)\le (\alpha -1)\delta R_1{R}_2,$$

and the results follow. □

4. Cooperative Solution

There are several possible ways to define a cooperative behavior in a multi-objective game. Crettez and Hayek [21] and Crettez, Hayek and Kort [22] suppose that in cooperation, the firms maximize the sums of their economic objectives (profits), and the environmental objective remains the same as for an individual player in a non-cooperative setting. Rettieva [25,26] introduced a specific approach to define cooperative behavior (called “multicriteria cooperative equilibria”) based on maximizing the sum (among the players) of the Nash products (among the criteria) of the partial criterion excess over its non-cooperative value. In the paper, we adopt for simplicity the classical approach (see, e.g., [24] ) to define a cooperative regime when the players fully coordinate their strategies to maximize the sum of their vector payoff functions:

$$\tilde{u}(·)=\sum _{j\in N}\left(u_j^1(·),u_j^2(·)\right)=\left(u_C^1(·),u_C^2(·)\right).$$

(20)

Although the solution to this problem is Pareto optimal, it is worth noting, that the players need to maximize the weighted sum ${\gamma }_1\left(u_1^1(·),u_1^2(·)\right)+\dots +{\gamma }_n\left(u_n^1(·),u_n^2(·)\right),{\gamma }_1+\dots +{\gamma }_n=1,{\gamma }_j\ge 0$, if they seek to get all the Pareto optimal cooperative solutions. The introduced approach how to derive the cooperative solution and then ensure its sustainability is appropriate for the case of almost symmetric players and could be extended to general case.

We assume, again for simplicity, that under cooperative scenario the players assign the same relative weights to the economic objective, that is ${\lambda }_j=\lambda$, $j\in N$, and apply the same discount factors ${\delta }_j=\delta$, $j\in N$. Note, that (20) implies summing of the payoffs across the players in the same criterion but not across different criteria (see [24] for other approach). Then, the environmentally normalized cooperative objective function for bicriteria game $G(n=2,s(0),T,\lambda )$ takes form

$$\begin{array}{cc}{u}_C(·)\hfill & =\lambda ·u_C^1(·)+u_C^2(·)=\hfill \\ & ={\displaystyle \sum _{\tau =0}^{T-1}}{\delta }^{\tau }\left[\lambda (2lne(\tau ,s(\tau \left)\right)+M(\theta \left)\right)+2lns\left(\tau \right)\right]+{\delta }^T(1+\lambda )(R_1+R_2)lns\left(T\right),\hfill \end{array}$$

(21)

where $e(\tau ,s\left(\tau \right))=e_1(·)+e_2(·)=\theta ·e(·)+(1-\theta )e(·)$, $\theta \in (0,1)$, $M\left(\theta \right)=ln\left(\theta \right(1-\theta \left)\right)$.

Proposition 3.

Assuming an interior solution, the linear form of cooperative strategy $e(·)$ and log-linear of the cooperative value function, the cooperative strategy is given by

$$\overline{e}\left(s\left(t\right)\right)=\frac{2\lambda \alpha }{2\lambda +\delta A(t+1)}·s\left(t\right),t=0,\dots ,T-1,$$

(22)

where

$$A\left(t\right)=2(1+\lambda )+\delta A(t+1),t=0,\dots ,T-1,A\left(T\right)=(1+\lambda )(R_1+R_2).$$

(23)

The cooperative state trajectory is given by

$$\overline{s}(t+1)=\frac{\alpha \delta A(t+1)}{2\lambda +\delta A(t+1)}·s\left(t\right)={\mu }^{COOP}\left(t\right)·s\left(t\right),t=0,\dots ,T-1.$$

(24)

The environmentally normalized cooperative value function $V^{COOP}(t,s\left(t\right))$ which corresponds to the cooperative payoff in the subgame $G(n=2,s(t),T-t,\lambda )$ is

$$V^{COOP}(t,s\left(t\right))=A\left(t\right)·lns\left(t\right)+B\left(t\right),$$

(25)

where

$$\begin{array}{c}B\left(t\right)=2\lambda ln{\displaystyle \frac{2\lambda \alpha }{2\lambda +\delta A(t+1)}}+\lambda M\left(\theta \right)+\delta A(t+1)ln{\displaystyle \frac{\alpha \delta A(t+1)}{2\lambda +\delta A(t+1)}}+\hfill \\ +\delta B(t+1),t=0,\dots ,T-1;B\left(T\right)=0.\hfill \end{array}$$

(26)

Proof. 

Again, using the dynamic programming technique and assuming that the cooperative strategy is linear in $s\left(t\right)$, we guess the log-linear form (25) of the cooperative value functions. The Bellman equation is

$$\begin{array}{c}{V}^{COOP}(t,s\left(t\right))=A\left(t\right)lns\left(t\right)+B\left(t\right)=\underset{e(t,s(t\left)\right)}{max}\{\lambda ·2lne(t,s(t\left)\right)+\lambda ·M\left(\theta \right)+2lns\left(t\right)+\hfill \\ +\delta ·\left[A(t+1)·ln\left(\alpha s\left(t\right)-e\left(t\right)\right)+B(t+1)\right]\},t=0,\dots ,T-1.\hfill \end{array}$$

(27)

By solving the first-order condition, one obtains the cooperative strategy (22) and corresponding state trajectory (24). By substituting functions (22) and (24) into the Bellman equation (27), we derived coefficients $A\left(t\right)$ and $B\left(t\right)$ in the recursive form: (23) and (26).

Note that the environmentally normalized cooperative value function in (27) takes its maximal value if $\theta =0,5$ for the case of the symmetric players under consideration. □

Remark 4.

By solving inequalities $A(t-1)>A\left(t\right)$ backwards in $t=T,T-1,\dots ,1$, one can conclude that function $A\left(t\right)$ is decreasing in t.

Transition coefficient ${\mu }^{COOP}(A(t+1),\alpha ,\delta ,\lambda )$ increases in $A(t+1)$, since ${\displaystyle \frac{\partial \mu }{\partial A(t+1)}}>0$.

Hence, ${\mu }^{COOP}\left(t\right)$ decreases in time t.

Proposition 4.

The resource stock $s\left(t\right)$ is non-decreasing in t along the cooperative state trajectory (24) for all $t=0,\dots ,T$, if one of the constraints on the parameters values holds:

$$2-(\alpha -1)·\delta ·(R_1+R_2)\le 0,$$

(28)

$$\left\{\begin{array}{c}2-(\alpha -1)·\delta ·(R_1+R_2)>0,\hfill \\ \lambda \le {\overline{\lambda }}^{COOP}={\displaystyle \frac{(\alpha -1)\delta (R_1+R_2)}{2-(\alpha -1)\delta (R_1+R_2)}}.\hfill \end{array}\right.$$

(29)

Proof. 

Taking Remark 4 into account, one needs to solve inequality ${\mu }^{COOP}\left(t\right)\ge 1$ only for $t=T-1$. Then,

$$\frac{\alpha \delta (1+\lambda )(R_1+R_2)}{2\lambda +\delta (1+\lambda )(R_1+R_2)}\ge 1\iff \lambda (2-(\alpha -1)\delta (R_1+R_2))\le (\alpha -1)\delta (R_1+R_2)$$

and the results (28) and (29) follow. □

Propositions 2–4 take into account the players’ asymmetry only through different levels $R_j$ of the residual stock valuation.

Remark 5.

Straightforward verification reveals that ${\overline{\lambda }}^{SPPE}<{\overline{\lambda }}^{COOP}$ given (19), (29) and $R_1=R_2$. Thus, there exists (at least for close values $R_1$ and $R_2$) a nonempty domain in the parameter space, namely, $\lambda \in ({\overline{\lambda }}^{SPPE},{\overline{\lambda }}^{COOP})$, for which the $SPPE$ behavior in $G(n=2,s\left(0\right),T,{\lambda }_1={\lambda }_2=\lambda )$ implies that the resource stock starts to decrease (from some intermediate time instant $t=0,\dots ,T-1$), and the cooperative behavior ensures that the resource stock is non-decreasing for all $t=0,\dots ,T$.

A finite-horizon dynamic game of renewable resource extraction with linear dynamics (1) where the players have only an economic objective (2) was studied in [10]. Though the introduced bicriteria model with environmentally concerned players retains the main outcome of a standard unicriterion model (namely, that cooperation is good for the environment), it allows one to evaluate how multicriteria setting would affect the players’ behavior. Whereas formulae in Propositions 1 and 3 are too cumbersome to compare analytically all the properties of the solutions for the bicriteria model and its unicriterion counterpart, we present some results of such a comparison based on numerical simulation in Section 6.

5. Sustainability of a Cooperative Agreement

Following Proposition 3, let $\overline{e}\left(t\right)$, $t=0,\dots ,T-1$, denote the cooperative strategy (total extraction level under cooperation), and $\overline{s}\left(t\right)$, $t=0,\dots ,T$, denote the cooperative trajectory (24).

Since we assume that under cooperative regime in $G(n=2,s(0),T,\lambda )$, both players seek to maximize the sum of their (environmentally normalized) objective functions. By comparing (16) and (27), we can conclude that

$$\sum _{j=1}^2{V}_j^{SPPE}\left(\overline{s}\left(t\right)\right)\le V^{COOP}\left(\overline{s}\left(t\right)\right),t=0,\dots ,T-1.$$

(30)

One can regard the ratio ${\displaystyle \frac{V^{COOP}\left(\overline{s}\left(t\right)\right)-{\displaystyle \sum _{j=1}^2}{V}_j^{SPPE}\left(\overline{s}\left(t\right)\right)}{{\displaystyle \sum _{j=1}^2}{V}_j^{SPPE}\left(\overline{s}\left(t\right)\right)}}$ as a total relative benefit from cooperation in the subgame $G(n=2,\overline{s}\left(t\right),T-t,\lambda )$, $t=0,\dots ,T-1$, along the cooperative trajectory $\overline{s}\left(t\right)$.

A vector $({\phi }_1\left(t\right),{\phi }_2\left(t\right))$ such that

$$\sum _{j=1}^2{\phi }_j\left(t\right)=V^{COOP}\left(\overline{s}\left(t\right)\right)$$

(31)

determines some sharing rule regarding how to distribute the total cooperative environmentally normalized subgame “payoff” between the players and could be considered as a single-point cooperative solution for the subgame $G(n=2,\overline{s}\left(t\right),T-t,\lambda )$.

The (non-negative) value ${\displaystyle \frac{{\phi }_j\left(t\right)-V_j^{SPPE}\left(\overline{s}\left(t\right)\right)}{V_j^{SPPE}\left(\overline{s}\left(t\right)\right)}}$ could be considered as a relative benefit from cooperation (RBC) for player $j\in N$ in the subgame $G(n=2,\overline{s}\left(t\right),T-t,\lambda )$, $t=0,\dots ,T-1$. We adopt in the paper the so-called maxmin RBC approach (see [10]) to specify the cooperative solution $\phi \left(t\right)$, which is the solution of the optimization problem

$$\underset{\phi \left(t\right)}{max}\underset{j\in N}{min}{\displaystyle \frac{{\phi }_j\left(t\right)-V_j^{SPPE}\left(\overline{s}\left(t\right)\right)}{V_j^{SPPE}\left(\overline{s}\left(t\right)\right)}},$$

(32)

where $\phi \left(t\right)$ meets (31). For a 2-player game, problem (32) takes the simpler form:

$$\left\{\begin{array}{c}{\displaystyle \frac{{\phi }_1\left(t\right)-V_1^{SPPE}\left(\overline{s}\left(t\right)\right)}{V_1^{SPPE}\left(\overline{s}\left(t\right)\right)}}={\displaystyle \frac{V^{COOP}\left(\overline{s}\left(t\right)\right)-{\phi }_1\left(t\right)-V_2^{SPPE}\left(\overline{s}\left(t\right)\right)}{V_2^{SPPE}\left(\overline{s}\left(t\right)\right)}}\hfill \\ V_1^{SPPE}\left(\overline{s}\left(t\right)\right)\le {\phi }_1\left(t\right)\le V^{COOP}\left(\overline{s}\left(t\right)\right)-V_2^{SPPE}\left(\overline{s}\left(t\right)\right)\hfill \end{array},\right.$$

(33)

where ${\phi }_2\left(t\right)=V^{COOP}\left(\overline{s}\left(t\right)\right)-{\phi }_1\left(t\right)$, $t=0,\dots ,T-1$.

To determine the cooperative solution $({\phi }_1\left(t\right),{\phi }_2\left(t\right))$ for $t=T$, suppose that

$${\phi }_j\left(T\right)=(1+\lambda )·R_{j}ln\overline{s}\left(T\right),j\in N.$$

(34)

Hence, one can calculate the single-point cooperative solution $({\phi }_1\left(t\right),{\phi }_2\left(t\right))$, which specifies the sharing rule for environmentally normalized cooperative value $V^{COOP}\left(\overline{s}\left(t\right)\right)$ in the subgame $G(n=2,\overline{s}\left(t\right),T-t,\lambda )$, $t=0,\dots ,T$.

Remark 6.

Note that the introduced cooperative solution (31), (33) and (35) could be considered as a “self-enforcing agreement” in the sense of [41] at each subgame $G(n=2,\overline{s}\left(t\right),T-t,\lambda )$, $t=0,\dots ,T-1$, since no player $j\in N$ can increase her environmentally normalized subgame payoff by unilateral deviation from sharing rule $\phi \left(t\right)$.

Considering (21) and (5), let

$${\phi }_j\left(t\right)=\lambda ·{\phi }_j^1\left(t\right)+u_j^{2t}\left(\overline{e}(·)\right),$$

(35)

where

$$u_j^{2t}\left(\overline{e}(·)\right)=\sum _{\tau =t}^{T-1}{\delta }^{\tau -t}·ln\overline{s}\left(\tau \right)+{\delta }^{T-t}·R_j·ln\overline{s}\left(T\right),$$

(36)

and ${\phi }_j^1\left(t\right)$ is the “optimal subgame payoff” which the player j should get (in economic criterion) in accordance with the cooperative solution $\phi \left(t\right)$.

Thus, using (35), (36), (33) and Proposition 3, one can calculate values of ${\phi }_j^1\left(t\right)$, $j=1,2$ for each subgame $G(n=2,\overline{s}\left(t\right),T-t,\lambda )$, $t=0,\dots ,T$, along the cooperative trajectory. Since the payoffs in economic criterion are assumed to be transferable between the players (in contrast to the environmental criterion), we can adjust the payoff-distribution-procedure-based approach (see, e.g., [2,14,33,34,35]) to ensure sustainable long-term cooperation in a bicriteria game.

Definition 3.

A vector ${\beta }^1\left(\overline{s}\left(t\right)\right)=({\beta }_1^1\left(\overline{s}\left(t\right)\right),{\beta }_2^1\left(\overline{s}\left(t\right)\right)),\tau =0,\dots ,T$ is a payoff-distribution procedure (PDP) in the first (economic) criterion for multistage bicriteria game $G^{1,2}(n=2,s_0,T)$, if

$$\sum _{\tau =0}^T{\delta }^{\tau }{\beta }_j^1\left(\overline{s}\left(\tau \right)\right)={\phi }_j^1\left(0\right),j=1,2.$$

(37)

The value ${\beta }_j^1\left(\overline{s}\left(\tau \right)\right)$ means the actual current payoff in the economic criterion which the player j obtains in time instant $\tau$ under cooperative scenario in accordance with the cooperative solution $\phi \left(t\right)$ and PDP ${\beta }^1$.

Note that in addition to the so-called efficiency constraint (37) a PDP ${\beta }^1$ may satisfy some other good properties (see, e.g., [2,33,34]). We focus in the paper on the time consistency property and the strict balance constraint.

Definition 4.

A PDP ${\beta }^1$ is time consistent in multistage bicriteria game $G^{1,2}(n=2,s_0,T)$ if for every $t=1,\dots ,T$

$$\sum _{\tau =0}^{t-1}{\delta }^{\tau }{\beta }_j^1\left(\overline{s}\left(\tau \right)\right)+{\delta }^t·{\phi }_j^1\left(t\right)={\phi }_j^1\left(0\right),j=1,2.$$

(38)

Definition 5.

A PDP ${\beta }^1$ satisfies the strict balance constraints (in economic criterion) if

$$\left\{\begin{array}{c}{\displaystyle \sum _{j\in N}}{\beta }_j^1\left(\overline{s}\left(t\right)\right)=2ln\overline{e}\left(\overline{s}\left(t\right)\right)+M\left(\theta \right),t=0,\dots ,T-1,\hfill \\ {\displaystyle \sum _{j\in N}}{\beta }_j^1\left(\overline{s}\left(T\right)\right)=(R_1+R_2)·ln\overline{s}\left(T\right),\hfill \end{array}\right.$$

(39)

that is, at each current and terminal stage t, the players redistribute exactly what they receive at this stage economically under the cooperative regime.

Proposition 5.

An incremental PDP ${\beta }^1$ such that

$$\left\{\begin{array}{c}{\beta }_j^1\left(\overline{s}\left(t\right)\right)={\phi }_j^1\left(t\right)-\delta {\phi }_j^1(t+1),t=0,\dots ,T-1,\hfill \\ {\beta }_j^1\left(\overline{s}\left(T\right)\right)=R_j·ln\overline{s}\left(T\right)\hfill \end{array}\right.$$

(40)

meets the time-consistency condition and strict balance constraints for the bicriteria dynamic game $G^{1,2}(n=2,s_0,T)$.

Proof. 

Considering (34), (35) and (36), we get

$$\sum _{\tau =0}^{T-1}{\delta }^{\tau }({\phi }_j^1\left(\tau \right)-\delta {\phi }_j^1(\tau +1))+{\delta }^T{R}_{j}ln\overline{s}\left(T\right)={\phi }_j^1\left(0\right),j=1,2.$$

Hence, condition (37) holds, and the rule (40) is a PDP.

Similarly, for every $t=1,\dots ,T$

$$\sum _{\tau =0}^{t-1}{\delta }^{\tau }\left\{{\phi }_j^1\left(\tau \right)-\delta {\phi }_j^1(\tau +1)\right\}+{\delta }^t{\phi }_j^1\left(t\right)={\phi }_j^1\left(0\right),$$

that is, PDP $\beta$ meets the time-consistency property.

Taking (21), (31), (35) and (36) into account, it follows that

$$\sum _{j=1}^2{\phi }_j^1\left(t\right)-\delta {\phi }_j^1(t+1)=2ln\overline{e}\left(\overline{s}\left(t\right)\right)+M\left(\theta \right)$$

for every $t=0,\dots ,T-1$.

Lastly, ${\displaystyle \sum _{j=1}^2}{\beta }_j^1\left(\overline{s}\left(T\right)\right)=(R_1+R_2)·ln\overline{s}\left(T\right)$. Hence, the incremental PDP (40) satisfies strict balance constraints, and the results follow. □

6. Numerical Example

To demonstrate the above theoretical results with a numerical example, let us consider a two-player bicriteria game of renewable resource extraction with the following parameters values: $T=5(t=0,1,\dots ,5)$, $\alpha =1.2$ and $\delta =0.9$. Suppose that the players use different coefficients to value the resource residual stock at time T, namely, $R_1=4$ and $R_2=3$; the relative weight of the economic objective $\lambda =1.5$; and $\theta =0.5$. Let the initial state (measure of the resource stock) $s_0$ be equal to 100.

The $SPPE$ strategies of the players are given by formulae (10)–(12). The cooperative strategy $\overline{e}\left(s\left(t\right)\right)$ is defined by (22)–(24). Optimal non-cooperative and cooperative strategies for given parameters are presented in

Table 1. SPPE strategies and cooperative strategy.

t	0	1	2	3	4
$e_1^{SPPE}$	$10.6986$	$11.0283$	$11.2842$	$11.4185$	$11.3454$
$e_2^{SPPE}$	$11.9967$	$12.6700$	$13.3951$	$14.1959$	$15.1272$
$e^{COOP}$	$12.4961$	$14.5075$	$16.9573$	$20.0038$	$23.9085$

.

Since the vector of the parameters meets (29) but does not satisfy (18) and (19), according to Propositions 2 and 4 and Remark 5, the resource stock under the cooperative regime is non-decreasing, and the SPPE behavior implies the decreasing resource stock. Cooperative (blue) and non-cooperative (red) trajectories are presented in

Table 2. $SPPE$ trajectory and cooperative trajectory.

t	0	1	2	3	4	5
$s^{SPPE}\left(t\right)$	100	$97.3047$	$93.0674$	$87.0015$	$78.7874$	$68.0723$
$s^{COOP}\left(t\right)$	100	$107.5039$	$114.4972$	$120.4393$	$124.5234$	$125.5196$

and Figure 1.

To answer the question of how the multicriteria setting affects the players behavior, let us present in

Table 3. $NE$ strategies and $NE$ state trajectory for a unicriterion game.

t	0	1	2	3	4	5
$e_1^{NE}$	$15.8345$	$14.3771$	$12.8219$	$11.1506$	$9.3364$	–
$e_2^{NE}$	$17.7558$	$16.5174$	$15.2205$	$13.8629$	$12.4486$	–
$s^{NE}\left(t\right)$	100	$86.4097$	$72.7972$	$59.3142$	$46.1635$	$33.6112$

the $NE$ strategies of the players and corresponding non-cooperative state trajectory which have been calculated for unicriterion game with only an economic objective (2) (see [10]) for the same parameters values.

Comparing non-cooperative strategies and state trajectory in a bicriteria game (with environmentally concerned players), and $SPPE$ strategies and the corresponding trajectory in its unicriterion counterpart, one can observe the following:

• The resource stock $s^{SPPE}\left(t\right)$ in a bicriteria game is larger at every stage, and the difference $s^{SPPE}\left(t\right)-s^{NE}\left(t\right)$ increases in t.
• The total extraction level in a bicriteria game is lower for the first stages. However, due to the larger resource stock mentioned above, the players in a bicriteria game have an opportunity to increase the total extraction level, and starting from some intermediate stage, this level becomes larger than the total $NE$ extraction level in a unicriterion game.

Similar conclusions based on the results of numerical simulation are valid for cooperative behavior.

Note that the environmentally normalized cooperative value function (27) in a bicriteria game is greater than the sum of the players’ $SPPE$ value functions at every subgame $G(n=2,\overline{s}\left(t\right),T-t,\lambda )$, $t=0,\dots ,T-1$ along the cooperative trajectory (see

Table 4. The players’ $SPPE$ payoffs versus cooperative payoff in the subgames along the cooperative trajectory.

t	0	1	2	3	4	5
$V_1^{SPPE}(t,\overline{s}\left(t\right))$	$58.2334$	$57.1478$	$55.6460$	$53.6935$	$51.2591$	$48.3246$
$V_2^{SPPE}(t,\overline{s}\left(t\right))$	$53.0916$	$51.0803$	$48.4770$	$45.2032$	$41.1644$	$36.2435$
${\displaystyle \sum _{j\in N}}{V}_j^{SPPE}(t,\overline{s}\left(t\right))$	$111.3250$	$108.2280$	$104.1230$	$98.8966$	$92.4236$	$84.5681$
$V^{COOP}(t,\overline{s}\left(t\right))$	$114.5232$	$110.9068$	$106.2302$	$100.3738$	$93.2035$	$84.5681$

).

Applying (31), (33) and (34), we get cooperative solution $\phi \left(t\right)=({\phi }_1\left(t\right),{\phi }_2\left(t\right))$ (see

Table 5. Cooperative solution for the subgames $G(n=2,\overline{s}\left(t\right),T-t,\lambda )$.

t	0	1	2	3	4	5
${\phi }_1\left(t\right)$	$59.9064$	$58.5622$	$56.7721$	$54.4954$	$51.6917$	$48.3246$
${\phi }_2\left(t\right)$	$54.6168$	$52.3445$	$49.4581$	$45.8783$	$41.5118$	$36.2435$

), which specifies the sharing rule for the environmentally normalized cooperative value $V^{COOP}\left(\overline{s}\left(t\right)\right)$ in the subgame $G(n=2,\overline{s}\left(t\right),T-t,\lambda )$.

The corresponding “optimal subgame payoffs” ${\phi }_j^1\left(t\right)$ which the player j should receive in economic criterion in accordance with the cooperative agreement are given in

Table 6. Optimal subgame payoffs the players get for economic criterion.

t	0	1	2	3	4	5
${\phi }_1^1\left(t\right)$	$19.4529$	$19.6920$	$19.8135$	$19.8034$	$19.6469$	$19.3298$
${\phi }_2^1\left(t\right)$	$17.8289$	$17.6606$	$17.2860$	$16.6682$	$15.7598$	$14.4974$

.

Finally, the incremental payoff-distribution procedure ${\beta }^1$ which satisfies time consistency (38) and strict balance constraints (39) is presented in

Table 7. Payoff distribution procedure (PDP) in the first (economic) criterion.

	${\mathit{\beta }}_{\mathit{j}}^1\left(\overline{\mathit{s}}\left(0\right)\right)$	${\mathit{\beta }}_{\mathit{j}}^1\left(\overline{\mathit{s}}\left(1\right)\right)$	${\mathit{\beta }}_{\mathit{j}}^1\left(\overline{\mathit{s}}\left(2\right)\right)$	${\mathit{\beta }}_{\mathit{j}}^1\left(\overline{\mathit{s}}\left(3\right)\right)$	${\mathit{\beta }}_{\mathit{j}}^1\left(\overline{\mathit{s}}\left(4\right)\right)$	${\mathit{\beta }}_{\mathit{j}}^1\left(\overline{\mathit{s}}\left(5\right)\right)$
$j=1$	$1.7301$	$1.8599$	$1.9904$	$2.1212$	$2.2500$	$19.3298$
$j=2$	$1.9344$	$2.1032$	$2.2847$	$2.4844$	$2.7121$	$14.4974$

. In the example, all the payments to the players are non-negative. Moreover, the longer the players adhere to the cooperative agreement, the more payments they receive according to the incremental PDP (with the highest payments at the final time T).

7. Concluding Remarks

We studied a finite-horizon bicriteria game model of renewable resource extraction, which has several advantageous properties. First, compared to classical (uni-criterion) models with only an economic objective, the introduced bicriteria model allows one to take into account the environmental concern of (asymmetric) players. Second, the finite horizon is more adequate for real-life problems of the resource extraction management. Although the formulae and calculations are more complicated in a multi-objective case and especially for a finite planning horizon (since the optimal strategies are non-stationary), our model retains the main good properties of the (standard) unicriterion infinite-horizon model of renewable resource extraction with linear dynamics. In addition, a multicriteria setting allows one to find out some new aspects of the players behavior (see, e.g., Propositions 2 and 4 and Remark 5).

It turned out that the cooperative regime is better than competitive (subgame-perfect Pareto equilibrium) behavior both from economic and environmental perspectives; therefore, we designed a mechanism to guarantee the sustainability of the cooperative agreement. Since the players can redistribute the payoffs only within the economic criterion, we revised the standard scheme of payoff-distribution procedure and derived appropriate PDP meeting time consistency and strict balance constraints. Hopefully, such an approach could be extended to other classes of dynamic multi-objective games with “non-transferable utilities” within some objectives.

Using Propositions 2 and 4, one can estimate whether the given parameters vector is favorable enough to generate non-decreasing evolution of the resource stock (under non-cooperative or at least under cooperative scenario). Otherwise, the players or the regulator should employ some mechanism to prevent the resource extinction, for instance, a temporal moratorium on extraction activities (see, e.g., [7]) or other appropriate changes in the normal resource extraction regime.