Least Squares in a Data Fusion Scenario via Aggregation Operators

Abstract

In this paper, appropriate least-squares methods were developed to operate in data fusion scenarios. These methods generate optimal estimates by combining measurements from a finite collection of samples. The aggregation operators of the average type, namely, ordered weighted averaging (OWA), Choquet integral, and mixture operators, were applied to formulate the optimization problem. Numerical examples about fitting curves to a given set of points are provided to show the effectiveness of the proposed algorithms.

1. Introduction

Several studies have been carried out on data science. Datasets play an important role in several areas of knowledge, since information can be extracted from them. This information can be used, for example, in decision making, product improvement, process automation, and trend forecasting [1,2,3].

A number of methods and algorithms have been developed in the literature to extract different information from datasets through mathematical and computational methods. In general, these algorithms were developed to model datasets collected from a single source. In this regard, few algorithms have been formulated to solve the problem in a data fusion scenario, that is, in a scenario where data comes from different sources [4].

The least-squares method (LSM) is a widely used technique for data modeling based on the minimization of a quadratic function [4,5,6,7,8,9]. LSM was initially conceived for modeling data from a single source. In [4], an LSM was developed considering a data fusion situation (LSM-DF), that is, a method considering data from different sources. LSM-DF was designed for weighted data fusion.

From a mathematical point of view, the LSM-DF is based on a weighted average of the length of residual vectors of the equations $b_k=A_{k}x+v_k$ with $k=1,2,\dots ,L$, expressed by

$$\sum _{k=1}^L\left|\right|v_k{\left|\right|}_{W_k}^2=\sum _{k=1}^L{v}_k^T{W}_k{v}_k$$

where $W_k$ are the weights, that is, an aggregation of L values with their corresponding weightings. Here, a very interesting question arises: is weighted averaging the best method for aggregating the data in all scenarios? Within this context, the study of different aggregation methods has recently gained prominence.

Aggregation operators constitute a subarea of fuzzy theory that has the characteristic of combining finite datasets of the same nature into a single dataset [1,2,6,7,10,11,12,13,14,15,16,17,18,19]. These operators are basically classified into three categories: mean, conjunctive, and disjunctive. Applications of these operators can be found in medical problems, image processing, decision making, and engineering problems.

$W_k$ weights are directly related to the $\left|\right|v_k{\left|\right|}^2$ length of residual vectors. However, in some situations, it would be interesting to dynamically allocate the weights to the $W_k$ weightings, putting more weight on the more important $\left|\right|v_k{\left|\right|}^2$ values. Thus, considering the above, the aggregation operators can be considered to bw a viable alternative to change the behavior of LSM-DF.

This study seeks to optimally combine the least-squares method and the aggregation operators of the average type, more specifically, the ordered weighted averaging (OWA) [3,20,21,22] Choquet integral, [23,24], and mixture [25,26] operators. Furthermore, the aim of this study is to formulate and solve appropriate least-squares methods to model finite collections of datasets of the same nature. An important goal of these algorithms is to generate optimal estimates that aggregate data of different sources. This is necessary for situations that involve systems that can operate under different failure conditions. A numerical example is presented to show the effectiveness of the proposed algorithm.

This paper is organized as follows: in Section 2, preliminary results are related with an admissible order for matrices, aggregation operators, and LSM. In Section 3, LSM-DF via aggregation operators are deduced. In Section 4, a numerical example is shown.

2. Preliminaries

This section addresses topics that form the theoretical basis for the development of LSM-DF via aggregation operators. Initially, the admissible order for matrices is discussed, followed by the aggregation operators of the average type and the classical least-squares method.

2.1. Admissible Order for Matrices

In this section, we present the concept of admissible order for matrices based on [2,16,27]. This is a special way to consider total orders on the set of all matrices of order $m\times n$ with scalar in $\mathcal{R}$ (set of real numbers) denoted by ${\mathcal{R}}^{m\times n}$.

Let $A,B\in {\mathcal{R}}^{m\times n}$. It is clear that $A{\le }_{M}B$ given by

$$A{\le }_{M}B\mathrm{if}\mathrm{and}\mathrm{only}\mathrm{if}{a}_{ij}\le b_{ij},\forall i,j$$

is a partial order on ${\mathcal{R}}^{m\times n}$.

Considering a matrix $A\in {\mathcal{R}}^{m\times n}$ as a vector of columns, i.e., $A=[A_1,A_2,\dots ,A_n]$ where $A_i$ are the columns of A ($i\in \{1,2,\dots ,n\}$), then ≤ can be defined as

$$A{\le }_{M}B\mathrm{if}\mathrm{and}\mathrm{only}\mathrm{if}{A}_i{\le }_M{B}_i,\forall i\in \{1,2,\dots ,n\}.$$

One can extend that partial order for a total order by considering the concept of admissible order as follows.

Definition 1. 

A total order ≼ on ${\mathcal{R}}^{m\times n}$ is admissible if, for each $A,B\in {\mathcal{R}}^{m\times n}$ we have that $A\preccurlyeq B$ whenever $A{\le }_{M}B$.

Example 1. 

Let be A and B column matrices on ${\mathcal{R}}^{m\times 1}$ and ${\pi }_i\left(A\right)=a_{i1}$ the projection on the i-th line of A. Then,

$$A{\preccurlyeq }_{c}B\iff \exists k\in \{1,2,\dots ,m\}s.t.{\pi }_k\left(A\right)<{\pi }_k\left(B\right)and\forall i,1\le i<k,{\pi }_i\left(A\right)={\pi }_i\left(B\right)$$

is an admissible order.

Therefore, one can generalize an admissible order on ${\mathcal{R}}^{m\times n}$ by considering the following definition: Let $A,B\in {\mathcal{R}}^{m\times n}$ such that $A=[A_1,A_2,\dots ,A_n]$ and $B=[B_1,B_2,\dots ,B_n]$. Then

$$A{\preccurlyeq }_{M}B\iff \exists k\in \{1,2,\dots ,m\}s.t.A_k{\preccurlyeq }_c{B}_{k}and\forall i,1\le i<k,A_i=B_i$$

is an admissible order on ${\mathcal{R}}^{m\times n}$.

2.2. Aggregation Operators

Aggregation operators are numeric operators that combine multiple input values into a single output value. In this data fusion process, operators aggregate data from different sources to obtain a single unit of data from the conducted analysis. Next, the operators used in this study are presented: OWA, Choquet integral, and mixture operators.

Definition 2 

([12]).(OWA operator) Providing an n- dimensional weight vector, that is, a $W=(w_1,w_2,\dots ,w_n)$ with ${\sum }_{k=1}^n{w}_k=1$, the ${OWA}_W:{[0,1]}^n\to [0,1]$ function is defined by

$$\begin{array}{c}\hfill OW{A}_W(x_1,x_2,\dots ,x_n)=\sum _{k=1}^n{w}_k{x}_{\left(k\right)}\end{array}$$

(1)

where $(x_{\left(1\right)},x_{\left(2\right)},\dots ,x_{\left(n\right)})$ is the descending order of vector $(x_1,x_2,\dots ,x_n)$ and is named an ordered weighted average function.

Example 2. 

Defining the $w=(w_1,w_2,\dots ,w_n)$ vector of weights, where $w_i=0$ and $w_k=1$, for some fixed $k\in \{1,2,\dots ,n\}$. So $OW{A}_w\left(x\right)=x_k$ is the so-called static OWA operator.

Remark 1. 

As one can see in Definition 2, the sum of all the weights in the OWA aggregation results is 1 (${\sum }_{k=1}^n{w}_k=1$). If the weights are matrices, the sum is given by ${\sum }_{k=1}^L\left|\right|W_k{\left|\right|}_1=1$ where $\left|\right|•{\left|\right|}_1$ is the norm of the matrices given by

$$\begin{array}{c}\hfill {\left|\right|A\left|\right|}_1=ma{x}_{1\le j\le s}\sum _{i=1}^r\left|a_{ij}\right|,whereA\in {\mathcal{R}}^{rxs}.\end{array}$$

(2)

Remark 2. 

The entries in the OWA aggregation must be sorted; if the entries are a matrix, an ordering relation must be used over the set ${\mathcal{R}}^{mxn}$. So, we can consider an admissible order on ${\mathcal{R}}^{mxn}$ as defined in 1.

The next definition is the fuzzy discrete measure, a significant result for the definition of the Choquet integral operator.

Definition 3 

([15]).A discrete fuzzy measure is a function $\mu :2^{\mathcal{N}}\to [0,1]$ where $\mathcal{N}=\{1,2,\dots ,n\}$ and $2^{\mathcal{N}}$ is the group of parts of $\mathcal{N}$, such that:

• $M_1:$$\mu \left(X\right)\le \mu \left(Y\right)$ when $X\subseteq Y$
• $M_2:$$\mu (\varnothing )=0$ and $\mu \left(\mathcal{N}\right)=1$.

Definition 4 

([10]).(Choquet integral operator) $\mu :2^{\mathcal{N}}\to [0,1]$ is a discrete fuzzy measure. The discrete Choquet integral related to the measure μ is the function $C_{\mu }:{[0,1]}^n\to [0,1]$ defined by:

$$\begin{array}{c}\hfill C_{\mu }(x_1,x_2,\dots ,x_n)=\sum _{k=1}^n{x}_{\left[k\right]}[\mu \left(\{j\in \mathcal{N}:x_j\ge x_{\left[k\right]}\}\right)-\mu \left(\{j\in \mathcal{N}:x_j\ge x_{[k+1]}\}\right)]\end{array}$$

(3)

where $(x_{\left[1\right]},x_{\left[2\right]},\dots ,x_{\left[n\right]})=Sort(x_1,x_2,\dots ,x_n)$ is an ascending ordering of the vector $(x_1,x_2,\dots ,x_n)$ and $x_{[n+1]}=2$ by convention.

The Choquet integral operator can also be calculated with the following simplified expression:

$$\begin{array}{c}\hfill C_{\mu }(x_1,x_2,\dots ,x_n)=\sum _{k=1}^n\left[x_{\left[k\right]}-x_{[k-1]}\right]\mu \left(G_k\right)\end{array}$$

(4)

where $x_{\left[0\right]}=0$ and $G_k=\{\left[k\right],[k+1],\dots ,\left[n\right]\}$.

Example 3. 

Considering fuzzy discrete measure

$${\mu }_{\perp }\left(X\right)=\left\{\begin{array}{cc}\hfill 1,& seX=\mathcal{N}\hfill \\ \hfill 0,& otherwise.\hfill \end{array}\right.$$

Thus, the following Choquet integral can be defined by:

$$C_{{\mu }_{\perp }}(x_1,x_2,\dots ,x_n)=[x_{\left[1\right]}-x_{\left[0\right]}]{\mu }_{\perp }\left(G_1\right)+\cdots +[x_{\left[n\right]}-x_{[n-1]}]{\mu }_{\perp }\left(G_n\right).$$

${\mu }_{\perp }\left(G_1\right)=1$ and ${\mu }_{\perp }\left(G_i\right)=0$ for the other values of i; therefore, the result is $C_{{\mu }_{\perp }}(x_1,x_2,\dots ,x_n)=x_{\left[1\right]}=min(x_1,x_2,\dots ,x_n)$.

Definition 5 

([15]).(Mixture Operator) $w_1,w_2,\dots ,w_n:[0,1]\to [0,+\infty )$ are functions called weight functions. The $MI{X}_{w_1,w_2,\dots ,w_n}:{[0,1]}^n\to [0,1]$ function is defined by:

$$\begin{array}{c}\hfill MI{X}_{w_1,w_2,\dots ,w_n}(x_1,x_2,\dots ,x_n)=\frac{{\sum }_{k=1}^n{w}_k\left(x_k\right).x_k}{{\sum }_{k=1}^n{w}_k\left(x_k\right)}\end{array}$$

(5)

is called the mixture function associated with the weight functions $w_1,w_2,\dots ,w_n$.

Example 4. 

Defining

$$w_i\left(x_i\right)=\left\{\begin{array}{cc}\hfill \frac{1}{n},& se{x}_i=0\hfill \\ \hfill x_i,& \mathit{otherwise}.\hfill \end{array}\right.$$

For simplicity, consider $n=3$. In this case, considering that

$$MI{X}_{w_1,w_2,w_3}(x_1,x_2,x_3)=\left\{\begin{array}{cc}\hfill 0,& se{x}_1=x_2=x_3=0\hfill \\ \hfill \frac{{x_1}^2+{x_2}^2+{x_3}^2}{x_1+x_2+x_3},& \mathit{otherwise}\hfill \end{array}\right.$$

is the mixture function determined by the $w_i$ weights defined above.

2.3. Least-Squares Method

LSM is a widely known and applied mathematical optimization method used to solve several problems, including parameter estimation. This method consists of finding an optimal solution to the problem by minimizing the square of a residual vector.

Considering the equation

$$\begin{array}{c}\hfill b=Ax+v\end{array}$$

(6)

where $x\in {\mathcal{R}}^{n\times 1}$ is an unknown vector, $A\in {\mathcal{R}}^{N\times n}$ is a known parameter matrix, $b\in {\mathcal{R}}^{N\times 1}$ is a known vector, and $v\in {\mathcal{R}}^{N\times 1}$ is a vector named residual.

The least-squares problem is to find a solution $\widehat{x}$ that minimizes the length of the residual vector, that is, satisfying the following property:

$$\begin{array}{c}\hfill \left|\right|b-A\widehat{x}{\left|\right|}^2\le \left|\right|b-{Ax\left|\right|}^2\end{array}$$

(7)

for all $x\in {\mathcal{R}}^{n\times 1}$. The $\left|\right|•{\left|\right|}^2$ denotes the square of Euclidean norm

$$\begin{array}{c}\hfill {\left|\right|v\left|\right|}^2=v^{T}v.\end{array}$$

(8)

Therefore, the solution to the least-squares problem consists of solving the optimization problem

$$\begin{array}{c}\hfill mi{n}_{x}J\left(x\right)\end{array}$$

(9)

where the functional cost $J\left(x\right)$ is given by

$$\begin{array}{ccc}\hfill J\left(x\right)& =& \left|\right|b-{Ax\left|\right|}^2\hfill \\ & =& {\left(b-Ax\right)}^T\left(b-Ax\right).\hfill \end{array}$$

(10)

Theorem 1 

([4]).(Least-Squares Method) If matrix A has full rank, then there is a single optimal solution $\widehat{x}$ for least-squares Problem (9) that is given by

$$\begin{array}{c}\hfill \widehat{x}={\left[A^{T}A\right]}^{-1}\left[A^{T}b\right].\end{array}$$

(11)

Moreover, the resulting minimal value of the cost function can be written as

$$\begin{array}{c}\hfill J\left(\widehat{x}\right)=b^{T}b-b^{T}A{\left(A^{T}A\right)}^{-1}{A}^{T}b.\end{array}$$

(12)

3. LSM-DF via Aggregation Operators

In this section, LSM-DF is developed via aggregation operators. LSM-DF via an OWA operator, LSM-DF via a Choquet integral operator, and LSM-DF via a mixture operator are also presented. These LSM-DFs are an alternative to estimation problems in the case of several datasources.

The next result is necessary to the proof of the LSM-DF via aggregation operators.

Lemma 1. 

If matrices $A_k$ have full rank and matrix $W_k$ is symmetric definite-positive with $k=1,2,\dots ,L$, then ${\overline{A}}^T\overline{\mathit{WA}}$ where

$$\overline{A}=\left[\begin{array}{c}{A}_1\\ A_2\\ ⋮\\ A_L\end{array}\right],\overline{W}=\left[\begin{array}{cccc}{W}_1& 0& \dots & 0\\ 0& W_2& \dots & 0\\ ⋮& ⋮& \dots & ⋮\\ 0& 0& \dots & W_L\end{array}\right]$$

(13)

is nonsingular.

Proof. 

Let suppose that ${\overline{A}}^T\overline{\mathit{WA}}$ is singular; then, there must exist a nonzero vector $\lambda$, such that ${\overline{A}}^T\overline{\mathit{WA}}\lambda =0$, which implies that ${\lambda }^T{\overline{A}}^T\overline{\mathit{WA}}\lambda =0$, i.e.,

$${\lambda }^T{\left[\begin{array}{c}{A}_1\\ A_2\\ ⋮\\ A_L\end{array}\right]}^T\left[\begin{array}{cccc}{W}_1& 0& \dots & 0\\ 0& W_2& \dots & 0\\ ⋮& ⋮& \dots & ⋮\\ 0& 0& \dots & W_L\end{array}\right]\left[\begin{array}{c}{A}_1\\ A_2\\ ⋮\\ A_L\end{array}\right]\lambda =0$$

(14)

$$\begin{array}{c}\hfill {\lambda }^T{A}_1^T{W}_1{A}_1\lambda +{\lambda }^T{A}_2^T{W}_2{A}_2\lambda +\dots +{\lambda }^T{A}_L^T{W}_L{A}_L\lambda =0\end{array}$$

(15)

(15) can be rewritten as

$$\begin{array}{c}\hfill \left|\right|A_1{\lambda \left|\right|}_{W_1}^2+\left|\right|A_2{\lambda \left|\right|}_{W_2}^2+\dots +\left|\right|A_L\lambda {\left|\right|}_{W_L}^2=0.\end{array}$$

(16)

$\left|\right|•{\left|\right|}_W^2$ denotes the square of the weighted Euclidean norm

$$\begin{array}{c}\hfill {\left|\right|v\left|\right|}_W^2=v^{T}Wv.\end{array}$$

(17)

As matrices $W_k$ are symmetric definite-positive, it follows from (16) that $\left|\right|A_k\lambda {\left|\right|}_{W_k}^2=0$ so that $A_k\lambda =0$ with $k=1,2,\dots ,L$. This, in turn, means that the columns of $A_k$ are linearly dependent. Hence, $A_k$ is not full-rank.    □

3.1. LSM-DF via OWA Operator

For the deduction of LSM-DF via OWA operator, the following equations should be considered

$$\begin{array}{c}\hfill b_{\left(k\right)}=A_{\left(k\right)}x+v_{\left(k\right)},k=1,2,\dots ,L\end{array}$$

(18)

where $x\in {\mathcal{R}}^{n\times 1}$ is an unknown vector, $A_{\left(k\right)}\in {\mathcal{R}}^{N\times n}$ known parameters arrays, $b_{\left(k\right)}\in {\mathcal{R}}^{N\times 1}$ known vectors, and $v_{\left(k\right)}\in {\mathcal{R}}^{N\times 1}$ vectors named residuals.

A solution to the least-squares problem via operator OWA $\widehat{x}$ must minimize the length of the residual vector, that is, it must satisfy the following property:

$$\begin{array}{c}\hfill \sum _{k=1}^L\left|\right|b_{\left(k\right)}-A_{\left(k\right)}\widehat{x}{\left|\right|}_{W_k}^2\le \sum _{k=1}^L\left|\right|b_{\left(k\right)}-A_{\left(k\right)}x{\left|\right|}_{W_k}^2\end{array}$$

(19)

for all x∈${\mathcal{R}}^{n\times 1}$ and where $W_k$ are a positive-definite symmetric matrices.

Optimal solution $\widehat{x}$ is found by solving the following minimization problem:

$$\begin{array}{c}\hfill mi{n}_x{\mathcal{J}}_{OWA}\left(x\right).\end{array}$$

(20)

Functional ${\mathcal{J}}_{OWA}\left(x\right)$ can be defined as

$$\begin{array}{c}\hfill {\mathcal{J}}_{OWA}\left(x\right):=OW{A}_W(J_1\left(x\right),J_2\left(x\right),\dots ,J_L\left(x\right))\end{array}$$

(21)

where $W=(W_1,W_2,\dots ,W_n)$ are weight matrices and

$$\begin{array}{ccc}\hfill J_k\left(x\right)& :=& \left|\right|v_{\left(k\right)}{\left|\right|}^2\hfill \\ & =& \left|\right|b_{\left(k\right)}-A_{\left(k\right)}x{\left|\right|}^2,k=1,2,\dots ,L.\hfill \end{array}$$

(22)

Therefore, by defining the OWA operator, Function (21) can be rewritten as

$$\begin{array}{ccc}\hfill {\mathcal{J}}_{OWA}\left(x\right)& :=& \sum _{k=1}^L\left|\right|b_{\left(k\right)}-A_{\left(k\right)}x{\left|\right|}_{W_k}^2\hfill \\ & =& \sum _{k=1}^L{\left(b_{\left(k\right)}-A_{\left(k\right)}x\right)}^T{W}_k\left(b_{\left(k\right)}-A_{\left(k\right)}x\right).\hfill \end{array}$$

(23)

The next theorem brings the solution to the least-squares problem via the OWA operator in (20).

Theorem 2. 

(LSM-DF via OWA Operator) If matrices $A_{\left(k\right)}$ with $k=1,2,\dots ,L$ have full rank and $W_k$ are symmetric definite-positive matrices, then there is a unique optimal solution $\widehat{x}$ to the least-squares problem via OWA operator (LSM-DF via OWA operator) that is given by:

$$\begin{array}{c}\hfill \widehat{x}={\left[\sum _{k=1}^L{A}_{\left(k\right)}^T{W}_k{A}_{\left(k\right)}\right]}^{-1}\left[\sum _{k=1}^L{A}_{\left(k\right)}^T{W}_k{b}_{\left(k\right)}\right].\end{array}$$

(24)

The corresponding minimal value of ${\mathcal{J}}_{OWA}\left(x\right)$ is

$$\begin{array}{ccc}& & {\mathcal{J}}_{OWA}\left(\widehat{x}\right)=\sum _{k=1}^L{b}_{\left(k\right)}^T{W}_k{b}_{\left(k\right)}-\sum _{k=1}^L{b}_{\left(k\right)}^T{W}_k{A}_{\left(k\right)}{\left(\sum _{k=1}^L{A}_{\left(k\right)}^T{W}_k{A}_{\left(k\right)}\right)}^{-1}\sum _{k=1}^L{A}_{\left(k\right)}^T{W}_k{b}_{\left(k\right)}.\hfill \end{array}$$

(25)

Proof. 

Consider the cost function

$$\begin{array}{c}\hfill {\mathcal{J}}_{OWA}\left(x\right)=\sum _{k=1}^L{\left(b_{\left(k\right)}-A_{\left(k\right)}x\right)}^T{W}_k\left(b_{\left(k\right)}-A_{\left(k\right)}x\right)\end{array}$$

(26)

$$\begin{array}{ccc}\hfill {\mathcal{J}}_{OWA}\left(x\right)& =& {\left(b_{\left(1\right)}-A_{\left(1\right)}x\right)}^T{W}_1\left(b_{\left(1\right)}-A_{\left(1\right)}x\right)+{\left(b_{\left(2\right)}-A_{\left(2\right)}x\right)}^T{W}_2\left(b_{\left(2\right)}-A_{\left(2\right)}x\right)\hfill \\ & +\dots +& {\left(b_{\left(L\right)}-A_{\left(L\right)}x\right)}^T{W}_L\left(b_{\left(L\right)}-A_{\left(L\right)}x\right)\hfill \end{array}$$

(27)

$${\mathcal{J}}_{OWA}\left(x\right)={\left[\begin{array}{c}\left(b_{\left(1\right)}-A_{\left(1\right)}x\right)\\ \left(b_{\left(2\right)}-A_{\left(2\right)}x\right)\\ ⋮\\ \left(b_{\left(L\right)}-A_{\left(L\right)}x\right)\end{array}\right]}^T\left[\begin{array}{cccc}{W}_1& 0& \dots & 0\\ 0& W_2& \dots & 0\\ ⋮& ⋮& \dots & ⋮\\ 0& 0& \dots & W_L\end{array}\right]\left[\begin{array}{c}\left(b_{\left(1\right)}-A_{\left(1\right)}x\right)\\ \left(b_{\left(2\right)}-A_{\left(2\right)}x\right)\\ ⋮\\ \left(b_{\left(L\right)}-A_{\left(L\right)}x\right)\end{array}\right]$$

(28)

$$\begin{array}{ccc}\hfill {\mathcal{J}}_{OWA}\left(x\right)& =& {\left(\left[\begin{array}{c}{b}_{\left(1\right)}\\ b_{\left(2\right)}\\ ⋮\\ b_{\left(L\right)}\end{array}\right]-\left[\begin{array}{c}{A}_{\left(1\right)}\\ A_{\left(2\right)}\\ ⋮\\ A_{\left(L\right)}\end{array}\right]x\right)}^T\left[\begin{array}{cccc}{W}_1& 0& \dots & 0\\ 0& W_2& \dots & 0\\ ⋮& ⋮& \dots & ⋮\\ 0& 0& \dots & W_L\end{array}\right]\hfill \\ & \times & \left(\left[\begin{array}{c}{b}_{\left(1\right)}\\ b_{\left(2\right)}\\ ⋮\\ b_{\left(L\right)}\end{array}\right]-\left[\begin{array}{c}{A}_{\left(1\right)}\\ A_{\left(2\right)}\\ ⋮\\ A_{\left(L\right)}\end{array}\right]x\right)\hfill \end{array}$$

(29)

that can be rewritten in matrix form as

$$\begin{array}{c}\hfill {\mathcal{J}}_{OWA}\left(x\right)={\left(\overline{b}-\overline{A}x\right)}^T\overline{W}\left(\overline{b}-\overline{A}x\right)\end{array}$$

(30)

where

$$\overline{A}=\left[\begin{array}{c}{A}_{\left(1\right)}\\ A_{\left(2\right)}\\ ⋮\\ A_{\left(L\right)}\end{array}\right],\overline{b}=\left[\begin{array}{c}{b}_{\left(1\right)}\\ b_{\left(2\right)}\\ ⋮\\ b_{\left(L\right)}\end{array}\right],\overline{W}=\left[\begin{array}{cccc}{W}_1& 0& \dots & 0\\ 0& W_2& \dots & 0\\ ⋮& ⋮& \dots & ⋮\\ 0& 0& \dots & W_L\end{array}\right].$$

(31)

Entries $(A_{\left(1\right)},A_{\left(2\right)},\dots ,A_{\left(L\right)})$ and $(b_{\left(1\right)},b_{\left(2\right)},\dots ,b_{\left(L\right)})$ are descending orders of $(A_1,A_2,\dots ,A_L)$ and $(b_1,b_2,\dots ,b_L)$, respectively. $\overline{W}$ is a diagonal positive-definite symmetric matrix with entries $W_k$.

To find the critical point in x, ${\mathcal{J}}_{OWA}\left(x\right)$ must be differentiated and equal to zero

$$\begin{array}{ccc}& & \frac{\partial }{\partial x}\left[x^T{\overline{A}}^T\overline{\mathit{WA}}x-x^T{\overline{A}}^T\overline{W}\overline{b}-{\overline{b}}^T\overline{\mathit{WA}}x+{\overline{b}}^T\overline{W}\overline{b}\right]=0\hfill \\ & \Rightarrow & x^T{\overline{A}}^T\overline{\mathit{WA}}-{\overline{b}}^T\overline{\mathit{WA}}=0.\hfill \end{array}$$

(32)

Via Lemma 1, matrix ${\overline{A}}^T\overline{\mathit{WA}}$ is invertible. Therefore,

$$\begin{array}{c}\hfill \widehat{x}={\left[{\overline{A}}^T\overline{\mathit{WA}}\right]}^{-1}\left[{\overline{A}}^T\overline{W}\overline{b}\right].\end{array}$$

(33)

Replacing (31) into (33), the solution can be rewritten as

$$\begin{array}{c}\hfill \widehat{x}={\left[\sum _{k=1}^L{A}_{\left(k\right)}^T{W}_k{A}_{\left(k\right)}\right]}^{-1}\left[\sum _{k=1}^L{A}_{\left(k\right)}^T{W}_k{b}_{\left(k\right)}\right].\end{array}$$

(34)

In fact, for the Hermitian matrix to be defined as positive

$$\begin{array}{c}\hfill \frac{{\partial }^2{\mathcal{J}}_{OWA}\left(x\right)}{\partial x^T\partial x}={\overline{A}}^T\overline{\mathit{WA}}=\sum _{k=1}^L{A}_{\left(k\right)}^T{W}_k{A}_{\left(k\right)}>0\end{array}$$

(35)

${\mathcal{J}}_{OWA}\left(x\right)$ in (30) must be a strictly convex function; therefore, $\widehat{x}$ is a unique global minimum.

The minimal cost ${\mathcal{J}}_{OWA}\left(\widehat{x}\right)$ can be expressed as

$$\begin{array}{ccc}\hfill {\mathcal{J}}_{OWA}\left(\widehat{x}\right)& =& \sum _{k=1}^L\left|\right|b_{\left(k\right)}-A_{\left(k\right)}\widehat{x}{\left|\right|}_{W_k}^2\hfill \\ & =& {\left(\overline{b}-\overline{A}\widehat{x}\right)}^T\overline{W}\left(\overline{b}-\overline{A}\widehat{x}\right)\hfill \\ & =& {\overline{b}}^T\overline{W}\overline{b}-{\overline{b}}^T\overline{\mathit{WA}}\widehat{x}-{\widehat{x}}^T{\overline{A}}^T\overline{W}\overline{b}+{\widehat{x}}^T{\overline{A}}^T\overline{\mathit{WA}}\widehat{x}\hfill \end{array}$$

(36)

Replacing (33) into (36) results in

$$\begin{array}{c}\hfill {\mathcal{J}}_{OWA}\left(\widehat{x}\right)={\overline{b}}^T\overline{W}\overline{b}-{\overline{b}}^T\overline{\mathit{WA}}{\left({\overline{A}}^T\overline{\mathit{WA}}\right)}^{-1}{\overline{A}}^T\overline{W}\overline{b}.\end{array}$$

(37)

Replacing (31) into (37), the optimal cost can be rewritten as

$$\begin{array}{c}\hfill {\mathcal{J}}_{OWA}\left(\widehat{x}\right)=\sum _{k=1}^L{b}_{\left(k\right)}^T{W}_k{b}_{\left(k\right)}-\sum _{k=1}^L{b}_{\left(k\right)}^T{W}_k{A}_{\left(k\right)}{\left(\sum _{k=1}^L{A}_{\left(k\right)}^T{W}_k{A}_{\left(k\right)}\right)}^{-1}\sum _{k=1}^L{A}_{\left(k\right)}^T{W}_k{b}_{\left(k\right)}.\end{array}$$

(38)

□

Remark 3. 

Applying $k=1$ in Theorem (2), the LSM-DF via OWA operator reduces to the classical LSM in Theorem (1).

3.2. LSM-DF via Choquet Integral Operator

The deduction of the LSM-DF via the Choquet integral operator follows from the equations

$$\begin{array}{c}\hfill b_{\left[k\right]}=A_{\left[k\right]}x+v_{\left[k\right]},k=1,2,\dots ,L\end{array}$$

(39)

where $x\in {\mathcal{R}}^{n\times 1}$ is an unknown vector, $A_{\left[k\right]}\in {\mathcal{R}}^{N\times n}$ known parameters matrices, $b_{\left[k\right]}\in {\mathcal{R}}^{N\times 1}$ known vectors, and $v_{\left[k\right]}\in {\mathcal{R}}^{N\times 1}$ vectors named residuals.

A solution to the least-squares problem via the Choquet integral operator $\widehat{x}$ must minimize the length of the residual vector, that is, it must satisfy the following property:

$$\begin{array}{c}\hfill \sum _{k=1}^L\left|\right|b_{\left[k\right]}-A_{\left[k\right]}\widehat{x}{\left|\right|}_{I\mu \left(G_k\right)}^2\le \sum _{k=1}^L\left|\right|b_{\left[k\right]}-A_{\left[k\right]}x{\left|\right|}_{I\mu \left(G_k\right)}^2\end{array}$$

(40)

for all x∈${\mathcal{R}}^{n\times 1}$ and where $I\mu \left(G_k\right)$ is a matrix identity multiplied by discrete fuzzy measure.

The optimal solution $\widehat{x}$ is found by solving the following minimization problem:

$$\begin{array}{c}\hfill mi{n}_x{\mathcal{J}}_{C_{\mu }}\left(x\right)\end{array}$$

(41)

Functional ${\mathcal{J}}_{C_{\mu }}\left(x\right)$ can be defined as

$${\mathcal{J}}_{C_{\mu }}\left(x\right):=C_{\mu }({\overline{J}}_1\left(x\right),{\overline{J}}_2\left(x\right),\dots ,{\overline{J}}_L(x))$$

(42)

where

$$\begin{array}{ccc}\hfill {\overline{J}}_k\left(x\right)& :=& \left|\right|v_{\left[k\right]}-v_{[k-1]}{\left|\right|}^2\hfill \\ & =& \left|\right|(b_{\left[k\right]}-A_{\left[k\right]}x)-(b_{[k-1]}-A_{[k-1]}x){\left|\right|}^2\hfill \\ & =& \left|\right|\left(b_{\left[k\right]}-b_{[k-1]}\right)-\left(A_{\left[k\right]}-A_{[k-1]}\right)x{\left|\right|}^2,k=1,2,\dots ,L.\hfill \end{array}$$

(43)

Therefore, by defining the Choquet integral operator, Function (42) can be rewritten as

$$\begin{array}{ccc}\hfill {\mathcal{J}}_{C_{\mu }}\left(x\right)& :=& \sum _{k=1}^L\left|\right|\left(b_{\left[k\right]}-b_{[k-1]}\right)-\left(A_{\left[k\right]}-A_{[k-1]}\right)x{\left|\right|}_{I\mu \left(G_k\right)}^2\hfill \\ & =& \sum _{k=1}^L\left[(b_{\left[k\right]}-b_{[k-1]}\right)-{\left(A_{\left[k\right]}-A_{[k-1]})x\right]}^{T}I\mu \left(G_k\right)\hfill \\ & •& \left[\left(b_{\left[k\right]}-b_{[k-1]}\right)-\left(A_{\left[k\right]}-A_{[k-1]}\right)x\right].\hfill \end{array}$$

(44)

where $I\mu \left(G_k\right)$ is a positive-definite symmetric matrix.

The next theorem brings the solution to the least-squares problem via the Choquet integral operator in (41).

Theorem 3. 

(LSM-DF via Choquet Integral Operator) If the $A_{\left[k\right]}-A_{[k-1]}$ matrices with $k=1,2,\dots ,L$ have a full rank and $I\mu \left(G_k\right)$ are symmetric definite-positive matrices, then there is a single optimal solution $\widehat{x}$ for the least-squares problem via Choquet integral operator (LSM-DF via Choquet integral operator) that is given by:

$$\begin{array}{ccc}\hfill \widehat{x}& =& {\left[\sum _{k=1}^L{\left(A_{\left[k\right]}-A_{[k-1]}\right)}^{T}I\mu \left(G_k\right)\left(A_{\left[k\right]}-A_{[k-1]}\right)\right]}^{-1}\hfill \\ & •& \left[\sum _{k=1}^L{\left(A_{\left[k\right]}-A_{[k-1]}\right)}^{T}I\mu \left(G_k\right)\left(b_{\left[k\right]}-b_{[k-1]}\right)\right].\hfill \end{array}$$

(45)

The corresponding minimal value of ${\mathcal{J}}_{C_{\mu }}\left(x\right)$ is

$$\begin{array}{ccc}\hfill {\mathcal{J}}_{C_{\mu }}\left(\widehat{x}\right)& =& \sum _{k=1}^L{\left(b_{\left[k\right]}-b_{[k-1]}\right)}^{T}I\mu \left(G_k\right)\left(b_{\left[k\right]}-b_{[k-1]}\right)\hfill \\ & -& \sum _{k=1}^L{\left(b_{\left[k\right]}-b_{[k-1]}\right)}^{T}I\mu \left(G_k\right)\left(A_{\left[k\right]}-A_{[k-1]}\right)\hfill \\ & •& {\left(\sum _{k=1}^L{\left(A_{\left[k\right]}-A_{[k-1]}\right)}^{T}I\mu \left(G_k\right)\left(A_{\left[k\right]}-A_{[k-1]}\right)\right)}^{-1}\hfill \\ & •& \sum _{k=1}^L{\left(A_{\left[k\right]}-A_{[k-1]}\right)}^{T}I\mu \left(G_k\right)\left(b_{\left[k\right]}-b_{[k-1]}\right).\hfill \end{array}$$

(46)

Proof. 

Consider functional cost

$$\begin{array}{ccc}\hfill {\mathcal{J}}_{C_{\mu }}\left(x\right)& =& \sum _{k=1}^L{\left[\left(b_{\left[k\right]}-b_{[k-1]}\right)-\left(A_{\left[k\right]}-A_{[k-1]}\right)x\right]}^{T}I\mu \left(G_k\right)\hfill \\ & •& \left[\left(b_{\left[k\right]}-b_{[k-1]}\right)-\left(A_{\left[k\right]}-A_{[k-1]}\right)x\right]\hfill \end{array}$$

(47)

Using the matrices, this can be rewritten as

$$\begin{array}{c}\hfill {\mathcal{J}}_{C_{\mu }}\left(x\right)={\left(b^{\prime }-A^{\prime }x\right)}^T{W}^{\prime }\left(b^{\prime }-A^{\prime }x\right)\end{array}$$

(48)

where

$$\begin{array}{ccc}\hfill A^{\prime }& =& \left[\begin{array}{c}{A}_{\left[1\right]}-A_{\left[0\right]}\\ A_{\left[2\right]}-A_{\left[1\right]}\\ ⋮\\ A_{\left[L\right]}-A_{[L-1]}\end{array}\right],b^{\prime }=\left[\begin{array}{c}{b}_{\left[1\right]}-b_{\left[0\right]}\\ b_{\left[2\right]}-b_{\left[1\right]}\\ ⋮\\ b_{\left[L\right]}-b_{[L-1]}\end{array}\right],\hfill \\ \hfill W^{\prime }& =& \left[\begin{array}{cccc}I\mu \left(G_1\right)& 0& \dots & 0\\ 0& I\mu \left(G_2\right)& \dots & 0\\ ⋮& ⋮& \dots & ⋮\\ 0& 0& \dots & I\mu \left(G_L\right)\end{array}\right].\hfill \end{array}$$

(49)

Entries $(A_{\left[1\right]},A_{\left[2\right]},\dots ,A_{\left[L\right]})$, and $(b_{\left[1\right]},b_{\left[2\right]},\dots ,b_{\left[L\right]})$ are ascending orders of $(A_1,A_2,\dots ,A_L)$, and $(b_1,b_2,\dots ,b_L)$, respectively. $W^{\prime }$ is a diagonal symmetric definite-positive matrix with entries $I\mu \left(G_k\right)$.

On the basis of Function (48) and the solution of LSM-DF via the OWA operator presented in Theorem (2), the solution to Optimization Problem (41) is given by

$$\widehat{x}=\left[{A^{\prime }}^T{W}^{\prime }{A}^{\prime }\right]-1\left[{A^{\prime }}^T{W}^{\prime }{b}^{\prime }\right].$$

(50)

which, through Matrices (49), can be rewritten as

$$\begin{array}{ccc}\hfill \widehat{x}& =& {\left[\sum _{k=1}^L{\left(A_{\left[k\right]}-A_{[k-1]}\right)}^{T}I\mu \left(G_k\right)\left(A_{\left[k\right]}-A_{[k-1]}\right)\right]}^{-1}\hfill \\ & •& \left[\sum _{k=1}^L{\left(A_{\left[k\right]}-A_{[k-1]}\right)}^{T}I\mu \left(G_k\right)\left(b_{\left[k\right]}-b_{[k-1]}\right)\right].\hfill \end{array}$$

(51)

Similar to the procedure performed in Theorem (2), the minimal cost ${\mathcal{J}}_{C_{\mu }}\left(\widehat{x}\right)$ can be expressed as

$${\mathcal{J}}_{C_{\mu }}\left(\widehat{x}\right)={b^{\prime }}^T{W}^{\prime }{b}^{\prime }-{b^{\prime }}^T{W}^{\prime }{A}^{\prime }{\left({A^{\prime }}^T{W}^{\prime }{A}^{\prime }\right)}^{-1}{A^{\prime }}^T{W}^{\prime }{b}^{\prime }.$$

(52)

Replacing (49) into (52), the optimal cost can be rewritten as

$$\begin{array}{ccc}\hfill {\mathcal{J}}_{C_{\mu }}\left(\widehat{x}\right)& =& \sum _{k=1}^L{\left(b_{\left[k\right]}-b_{[k-1]}\right)}^{T}I\mu \left(G_k\right)\left(b_{\left[k\right]}-b_{[k-1]}\right)\hfill \\ & -& \sum _{k=1}^L{\left(b_{\left[k\right]}-b_{[k-1]}\right)}^{T}I\mu \left(G_k\right)\left(A_{\left[k\right]}-A_{[k-1]}\right)\hfill \\ & •& {\left(\sum _{k=1}^L{\left(A_{\left[k\right]}-A_{[k-1]}\right)}^{T}I\mu \left(G_k\right)\left(A_{\left[k\right]}-A_{[k-1]}\right)\right)}^{-1}\hfill \\ & •& \sum _{k=1}^L{\left(A_{\left[k\right]}-A_{[k-1]}\right)}^{T}I\mu \left(G_k\right)\left(b_{\left[k\right]}-b_{[k-1]}\right).\hfill \end{array}$$

(53)

□

Remark 4. 

$A_{\left[0\right]}$ is the null matrix and $b_{\left[0\right]}$ is the null vector by convention.

Remark 5. 

By applying $k=1$ in Theorem (3), the LSM-DF via Choquet integral operator reduces to the classical LSM in Theorem (1).

3.3. LSM-DF via Mixture Operator

For the deduction of the LSM-DF via the mixture operator, it is necessary to adapt the mixture operator presented in Definition (5).

The weight functions that are dynamic in the mixture operator uses were previously calculated and became constant (static) weight functions. Thus, the adapted mixture operator is calculated in two steps. In the first step, the weights are calculated and fixed. In the next step, aggregations are carried out. The next definition brings the adapted mixture operator.

Definition 6. 

(Adapted Mixture Operator) The adapted MIX function can be calculated using the following steps:

• Step 1: weight functions $w_k\left(x_k\right)$ with $k=1,2,\dots ,n$ can be calculated and fixed as follows:

  $$\begin{array}{c}\hfill w_1\left(x_1\right)=w_1,w_2\left(x_2\right)=w_2,\dots ,w_n\left(x_n\right)=w_n.\end{array}$$

  (54)
• Step 2: with the fixed weight functions, the MIX function can be calculated as follows:

  $$\begin{array}{c}\hfill MI{X}_{w_1,w_2,\dots ,w_n}(x_1,x_2,\dots ,x_n)=\frac{{\sum }_{k=1}^n{w}_k{x}_k}{{\sum }_{k=1}^n{w}_k}.\end{array}$$

  (55)

Now, the LSM-DF via the mixture operator must be deduced. The following equation must be considered:

$$\begin{array}{c}\hfill b_k=A_{k}x+v_k,k=1,2,\dots ,L\end{array}$$

(56)

where $x\in {\mathcal{R}}^{n\times 1}$ is an unknown vector, $A_k\in {\mathcal{R}}^{N\times n}$ known parameters matrices, $b_k\in {\mathcal{R}}^{N\times 1}$ known vectors, and $v_k\in {\mathcal{R}}^{N\times 1}$ vectors named residuals.

A solution to the least-squares problem via the mixture operator must minimize the length of the residual vector, that is, it must satisfy the following property:

$$\begin{array}{c}\hfill \frac{{\sum }_{k=1}^L\left|\right|b_k-A_k\widehat{x}{\left|\right|}_{W_k}^2}{{\sum }_{k=1}^L\left|\right|W_k{\left|\right|}^2}\le \frac{{\sum }_{k=1}^L\left|\right|b_k-A_{k}x{\left|\right|}_{W_k}^2}{{\sum }_{k=1}^L\left|\right|W_k{\left|\right|}^2}\end{array}$$

(57)

for all x∈${\mathcal{R}}^{n\times 1}$ and where $W_k$ is a positive-definite symmetric matrix.

Optimal solution $\widehat{x}$ is found by solving the following minimization problem:

$$\begin{array}{c}\hfill mi{n}_x{J}_{MIX}\left(x\right)\end{array}$$

(58)

Functional ${\mathcal{J}}_{MIX}\left(x\right)$ can be defined as

$$\begin{array}{c}\hfill {\mathcal{J}}_{MIX}\left(x\right):=MI{X}_{W_1,W_2,\dots ,W_L}\left({\underline{J}}_1\left(x\right),{\underline{J}}_2\left(x\right),\dots ,{\underline{J}}_L\left(x\right)\right)\end{array}$$

(59)

where

$$\begin{array}{ccc}\hfill {\underline{J}}_k\left(x\right)& :=& \left|\right|v_k{\left|\right|}^2\hfill \\ & =& \left|\right|b_k-A_{k}x{\left|\right|}^2,k=1,2,\dots ,L.\hfill \end{array}$$

(60)

By defining Mixture Operator (59), the function can be rewritten as

$$\begin{array}{ccc}\hfill {\mathcal{J}}_{MIX}\left(x\right)& :=& \frac{{\sum }_{k=1}^L\left|\right|b_k-A_{k}x{\left|\right|}_{W_k}^2}{{\sum }_{k=1}^L\left|\right|W_k{\left|\right|}^2}\hfill \\ & =& \frac{{\sum }_{k=1}^L{\left(b_k-A_{k}x\right)}^T{W}_k\left(b_k-A_{k}x\right)}{{\sum }_{k=1}^L\left|\right|W_k{\left|\right|}^2}.\hfill \end{array}$$

(61)

The next theorem brings the solution to the least-squares problem via the mixture operator in (58).

Theorem 4. 

(LSM-DF via Mixture Operator) If the $A_k$ matrices with $k=1,2,\dots ,L$ have a full rank and $W_k$ are symmetric definite-positive matrices, then there is a single optimal solution $\widehat{x}$ to the least-squares problem via the mixture operator (LSM-DF via mixture operator) (58) that is given by:

$$\begin{array}{c}\hfill \widehat{x}={\left[\sum _{k=1}^L{A}_k^T{W}_k{A}_k\right]}^{-1}\left[\sum _{k=1}^L{A}_k^T{W}_k{b}_k\right].\end{array}$$

(62)

The corresponding minimal value of ${\mathcal{J}}_{MIX}\left(x\right)$ is

$$\begin{array}{ccc}\hfill {\mathcal{J}}_{MIX}\left(\widehat{x}\right)& =& \sum _{k=1}^L{b}_k^T{W}_k{b}_k-\sum _{k=1}^L{b}_k^T{W}_k{A}_k{\left(\sum _{k=1}^L{A}_k^T{W}_k{A}_k\right)}^{-1}\sum _{k=1}^L{A}_k^T{W}_k{b}_k.\hfill \end{array}$$

(63)

Proof. 

Consider the function

$$\begin{array}{c}\hfill {\mathcal{J}}_{MIX}\left(x\right)=\left[\frac{{\sum }_{k=1}^L{\left(b_k-A_{k}x\right)}^T{W}_k\left(b_k-A_{k}x\right)}{{\sum }_{k=1}^L\left|\right|W_k{\left|\right|}^2}\right]\end{array}$$

(64)

that can be rewritten as

$$\begin{array}{c}\hfill {\mathcal{J}}_{MIX}\left(x\right)=\alpha {\left(\beta -\mathcal{A}x\right)}^T\mathcal{W}\left(\beta -\mathcal{A}x\right)\end{array}$$

(65)

where

$$\begin{array}{ccc}\hfill \mathcal{A}& =& \left[\begin{array}{c}{A}_1\\ A_2\\ ⋮\\ A_L\end{array}\right],\beta =\left[\begin{array}{c}{b}_1\\ b_2\\ ⋮\\ b_L\end{array}\right],\mathcal{W}=\left[\begin{array}{cccc}{W}_1& 0& \dots & 0\\ 0& W_2& \dots & 0\\ ⋮& ⋮& \dots & ⋮\\ 0& 0& \dots & W_L\end{array}\right],\hfill \\ \hfill \alpha & =& \frac{1}{{\sum }_{k=1}^L\left|\right|W_k{\left|\right|}^2}\hfill \end{array}$$

(66)

where $\mathcal{W}$ is a diagonal positive-definite symmetric matrix with entries $W_k$.

To find the solution to optimization problem $\widehat{x}$, $J\left(x\right)$ must be differentiated in (65) and equal to zero. On the basis of the theorem, the solution of the derivative is given by

$$\begin{array}{ccc}& & \frac{\partial }{\partial x}\left[\alpha {\left(\beta -\mathcal{A}x\right)}^T\mathcal{W}\left(\beta -\mathcal{A}x\right)\right]=0\hfill \\ & & \alpha \frac{\partial }{\partial x}\left[{\left(\beta -\mathcal{A}x\right)}^T\mathcal{W}\left(\beta -\mathcal{A}x\right)\right]=0.\hfill \end{array}$$

(67)

On the basis of Theorem (2), the solution of the derivative is given by

$$\begin{array}{c}\hfill \alpha \left[x^T{\mathcal{A}}^T\mathcal{W}\mathcal{A}-{\beta }^{T}W\mathcal{A}\right]=0.\end{array}$$

(68)

Therefore,

$$\begin{array}{c}\hfill \widehat{x}={\left[{\mathcal{A}}^T\mathcal{W}\mathcal{A}\right]}^{-1}\left[{\mathcal{A}}^T\mathcal{W}\beta \right].\end{array}$$

(69)

Through Matrices (66), the solution can be rewritten as

$$\begin{array}{c}\hfill \widehat{x}={\left[\sum _{k=1}^L{A}_k^T{W}_k{A}_k\right]}^{-1}\left[\sum _{k=1}^L{A}_k^T{W}_k{b}_k\right].\end{array}$$

(70)

Minimal cost ${\mathcal{J}}_{MIX}\left(\widehat{x}\right)$ can be expressed as

$$\begin{array}{c}\hfill {\mathcal{J}}_{MIX}\left(\widehat{x}\right)={\beta }^T\mathcal{W}\beta -{\beta }^T\mathcal{W}\mathcal{A}\widehat{x}-{\widehat{x}}^T{\mathcal{A}}^T\mathcal{W}\beta +{\widehat{x}}^T{\mathcal{A}}^T\mathcal{W}\mathcal{A}\widehat{x}\end{array}$$

(71)

replacing(69) into (71), the result is

$$\begin{array}{c}\hfill {\mathcal{J}}_{MIX}\left(\widehat{x}\right)={\beta }^T\mathcal{W}\beta -{\beta }^T\mathcal{W}\mathcal{A}{\left({\mathcal{A}}^T\mathcal{W}\mathcal{A}\right)}^{-1}{\mathcal{A}}^T\mathcal{W}\beta .\end{array}$$

(72)

Replacing (66) into (72), the optimal cost can be rewritten as

$${\mathcal{J}}_{MIX}\left(\widehat{x}\right)=\sum _{k=1}^L{b}_k^T{W}_k{b}_k-\sum _{k=1}^L{b}_k^T{W}_k{A}_k\left(\sum _{k=1}^L{A}_k^T{W}_k{A}_k\right)-1\sum _{k=1}^L{A}_k^T{W}_k{b}_k.$$

(73)

□

Remark 6. 

The optimal solution of the LSM-DF via a mixture operator reduces to the LSM-DF in [4].

4. Illustrative Example

In this section, we present artificially created (by authors) datasets in order to illustrate the behavior, effectiveness, and the relationship between the proposed methods for finding the best fitting curve to a given set of points from a mathematical point of view.

Table 1. Simulated datasets about income and consumption.

Income (${\mathit{x}}_1$)	Consumption (${\mathit{y}}_1$)	Income (${\mathit{x}}_2$)	Consumption (${\mathit{y}}_2$)
139	122	140	123
126	114	129	117
90	86	92	89
144	134	145	136
163	146	163	147
136	107	138	109
61	68	64	68
62	117	63	119
41	71	43	73
120	98	122	100

shows two simulated datasets about income and consumption.

First, the LSM was separately applied to the datasets, and the following results were found:

$$\begin{array}{c}\hfill {\widehat{y}}_1=0.49x_1+52.69,\end{array}$$

(74)

$$\begin{array}{c}\hfill {\widehat{y}}_2=0.49x_2+53.65.\end{array}$$

(75)

The MSEs between ${\widehat{y}}_1$ with $y_1$ and ${\widehat{y}}_2$ with $y_2$ were $211.52$ and $221.67$, respectively. Model (74) was more accurate than Model (75).

Second, the LSM-DF via OWA, Choquet integral, and mixture operators were calculated in the two datasets, and the following weighting matrices were used in the simulation: $W_1=0.7\ast diag\left(10\right)$ and $W_2=0.3\ast diag\left(10\right)$; more weight was given to $W_1$ than to $W_2$. The following results were found:

$$\begin{array}{c}\hfill {\widehat{y}}_O=0.49x+53.34,\end{array}$$

(76)

$$\begin{array}{c}\hfill {\widehat{y}}_C=0.49x+52.65,\end{array}$$

(77)

$$\begin{array}{c}\hfill {\widehat{y}}_M=0.49x+52.95.\end{array}$$

(78)

The MSEs between ${\widehat{y}}_O$, ${\widehat{y}}_C$ and ${\widehat{y}}_M$ with $y_1$ were $211.18$, $211.57$, $211.28$, respectively. The MSEs between ${\widehat{y}}_O$, ${\widehat{y}}_C$ and ${\widehat{y}}_M$ with $y_2$ were $222.14$, $223.87$ and 223 respectively.

Table 2. Sample with regard to $x_1$ of Equations (76)–(78).

Income (${\mathit{x}}_1$)	Consumption (${\widehat{\mathit{y}}}_{\mathit{O}}$)	Consumption (${\widehat{\mathit{y}}}_{\mathit{C}}$)	Consumption (${\widehat{\mathit{y}}}_{\mathit{M}}$)
139	121.45	120.76	121.06
126	115.08	114.39	114.69
90	97.44	96.75	97.05
144	123.90	123.21	123.51
163	133.21	132.52	132.82
136	1119.98	119.29	119.59
61	83,23	82.54	82.84
62	83.72	83.03	83.33
41	73.43	72.74	73.04
120	112.14	111.45	111.75

and

Table 3. Sample with regard to $x_2$ of Equations (76)–(78).

Income (${\mathit{x}}_2$)	Consumption (${\widehat{\mathit{y}}}_{\mathit{O}}$)	Consumption (${\widehat{\mathit{y}}}_{\mathit{C}}$)	Consumption (${\widehat{\mathit{y}}}_{\mathit{M}}$)
140	121.94	121.25	121.55
129	116.55	115.86	116.16
92	98.42	97.73	98.03
145	124.39	123.70	124
163	133.21	132.52	132.82
138	120.96	120.27	120.57
64	87.40	84.01	84.31
63	84.21	83.52	83.82
43	74.41	73.72	74.02
122	113.12	112.43	112.73

compare samples with regard to $x_1$ and $x_2$, respectively, of Equations (76)–(78).

Table 4. Sample of $y_1$ and samples generated with Equations (74), (76)–(78).

${\mathit{y}}_1$	${\widehat{\mathit{y}}}_1$	${\widehat{\mathit{y}}}_{\mathit{O}}$	${\widehat{\mathit{y}}}_{\mathit{C}}$	${\widehat{\mathit{y}}}_{\mathit{M}}$
122	120.80	121.45	120.76	121.06
114	114.43	115.08	114.39	114.69
86	96.79	97.44	96.75	97.05
134	123.25	123.90	123.21	123.51
146	132.56	133.21	132.52	132.82
107	119.33	119.98	119.29	119.59
68	82.58	83,23	82.54	82.84
117	83.07	83.72	83.03	83.33
71	72.78	73.43	72.74	73.04
98	111.49	112.14	111.45	111.75

compares the samples of $y_1$ to the samples generated by Equations (74), (76)–(78).

Table 5. Sample of $y_2$ and the samples generated by Equations (76)–(78).

${\mathit{y}}_2$	${\widehat{\mathit{y}}}_2$	${\widehat{\mathit{y}}}_{\mathit{O}}$	${\widehat{\mathit{y}}}_{\mathit{C}}$	${\widehat{\mathit{y}}}_{\mathit{M}}$
123	122.25	121.94	121.25	121.55
117	116.86	116.55	115.86	116.16
89	98.73	98.42	97.73	98.03
136	124.70	124.39	123.70	124
147	133.52	133.21	132.52	132.82
109	121.27	120.96	120.27	120.57
68	85.01	87.40	84.01	84.31
119	84.52	84.21	83.52	83.82
73	74.72	74.41	73.72	74.02
100	113.43	113.12	112.43	112.73

compares the samples of $y_2$ with the samples generated with Equations (76)–(78).

MSE shows that Models (76)–(78) were more accurate than Model (74). The LSM-DF via OWA, Choquet integral, and mixture operators outperformed the LSM.

5. Conclusions

In this paper, the LSM-DF was studied through aggregation operators in order to explore different ways to aggregate data. More specifically, the LSM-DF via an OWA operator, the LSM-DF via a Choquet integral operator, and the LSM-DF via a mixture operator were defined. These operators were particularly chosen due to their efficiency when applied to other methods in different areas of knowledge [12,13,22,24,26]. These new methods provide a theoretical framework with variations of the classic least square, which may be more suitable in certain applications. For instance, LSM-DF via OWA operator could be chosen for situations where one wants to place greater weights on the first data entries.

The main objective of developing these methods is to estimate an optimal parameter for situations involving more than one dataset, and to show how it can be changed for different types of data. The methods were mathematically demonstrated by applying aggregation operators of the average type to optimization problem. The illustrate example was set up to demonstrate the mathematical behavior of these procedures trough fitting curves in comparison with an approach that does not incorporate the aggregation operators in its formulation.

In future studies, we want to explore some applications that can show the advantages and disadvantages of each method, and set up LSM for other aggregation operators such as a weighted OWA (WOWA) operator and a Sugeno integral operator. Furthermore, these methods will be extended to models subject to parametric uncertainties.