Robust Total Least Squares Estimation Method for Uncertain Linear Regression Model

Abstract

In data analysis and modeling, least squares and total least squares are both mathematical optimization techniques. It is noteworthy that both the least squares method and the total least squares method are used to deal with precise and random data. However, when the given data are not random, or when the data are imprecise, and only the range of the data is available, the traditional linear regression method cannot be used. This paper presents an uncertain total least squares estimation method and an uncertain robust total least squares linear regression method based on uncertainty theory and total least squares method. The uncertain total least squares estimation can fully consider errors in the given data and the uncertain robust total least squares linear regression method can effectively eliminate outliers in the data. It is possible to obtain a more reasonable fitting effect with both of these methods, as well as to solve the predicted value and the confidence interval with these two methods. In terms of robust total least squares linear regression estimation, both uncertain total least squares regression estimation and uncertain robust total least squares regression estimation are feasible based on numerical examples. There are more accurate fitting equations and more reliable results with uncertain robust least squares linear regression estimation.

1. Introduction

Linear regression is a widely used statistical method in mathematical statistics, which reveals the mutual dependence between two or more variables through regression analysis. This method is particularly useful for determining the relationship between variables and predicting future trends or values based on historical data, playing a crucial role in research and the solution of practical problems. It is one of the problems encountered in many engineering applications. The least squares method is a widely used mathematical optimization technique in data analysis and modeling. It aims to minimize the sum of squares of errors to establish the relationship between variables and derive the most accurate fitting function for predicting and inferring future data. The least squares method is one of the common methods of linear fitting. In machine learning, especially regression models, least squares is often used [1]. Although the least square method can be used to solve the regression equation easily and minimize the sum of squares of errors between the obtained data and the actual data, it only takes into account the errors of the response variable. When the explanatory variable and the response variable are selected differently, the results of linear regression are often different. In order to solve this problem, many scholars have conducted in-depth studies and proposed the total least squares method, which can simultaneously consider errors in explanatory variables and response variables and obtain good results [2]. But the total least squares linear fitting does not take into account the possible gross errors or outliers in the data, which are actually objective in the measurement. Some experts and scholars derive a robust total least squares linear fitting [3,4]. For the traditional total least squares method and the robust total least squares method mentioned above, they can perform better linear regression on random data and obtain better fitting functions. In practice, however, the data we obtain are not necessarily random or not accurate values or even rough ranges. In this case, traditional total least squares and robust total least squares cannot be used for linear regression. What to do? The uncertain regression analysis proposed by Liu [5] can help us solve these problems.

The uncertainty theory established by Liu [5,6,7,8] is a new interdisciplinary and comprehensive discipline. Its research purpose is to establish a set of axiomatic mathematical systems to research uncertainty and provide rigorous mathematical tools, theoretical basis and method guidance for dealing with uncertain information in the real world. The basic content of uncertainty theory is the three concepts of uncertain measure, uncertain variable and uncertainty distribution and the two algorithms of uncertain operation laws and uncertain expected value. The uncertainty theory is effective in addressing real-world uncertainty problems. Extensive research has been conducted by numerous experts and scholars, who have also widely applied this theory [9,10,11]. In 2010, Liu [5] initiated research into uncertain statistics. He introduced the least squares principle as a means to estimate unknown parameters in uncertain distributions. This groundbreaking work has since inspired many experts and scholars to delve into the field of uncertainty statistics [12,13,14]. Uncertain statistics is a mathematical tool to collect, analyze and process data using uncertainty theory. Uncertain regression analysis is an important research topic and it is a technical tool to research the relationship between variables by using uncertainty theory. In 2018, Yao and Liu [15] proposed least squares estimates for unknown parameters of uncertain regression equations and Lio and Liu [16] further suggested an approach to estimate the uncertainty distribution of the disturbance term in an uncertain regression model. In 2020, Wang, Li and Guo [17] proposed a new uncertain regression model and Liu, Yang [18] proposed the least absolute deviation estimation for uncertain regression with imprecise observations. Wang et al. [19,20] proposed new uncertain linear regression models.

Deterministic models and uncertain models are symmetric. So far, we have many methods to solve the uncertain regression model, but most of these methods only consider the error of the dependent variable, which is obviously one-sided and the accuracy and reliability of the regression equation will be affected. In 2022, Shi et al. [21] proposed an uncertain total least squares estimation. The uncertain total least squares estimation improves the accuracy of the regression equation and has a good effect. However, when there are many variables, the solution process of this method is very tedious. Based on previous studies [2] and uncertain statistics, this paper first proposes an uncertain total least squares estimation method, which applies the traditional singular value decomposition to uncertain regression analysis and takes into account the errors of independent variables and dependent variables. The regression equation obtained is more reasonable. In fact, gross errors or outliers in given data are objective and unavoidable and the total least squares estimation does not take into account the possible gross errors or outliers in the data. Therefore, based on the robust total least squares estimation [3,4], we propose a robust uncertain total least squares estimation when considering gross errors or outliers in the observed data. This method not only considers the errors in all the observed values, but also can eliminate the gross errors or outliers in the data, with higher accuracy and more reliable results.

In this paper, we propose an uncertain total least squares estimation model for linear regression and then derive uncertain robust population least squares estimation when considering the existence of gross errors or outliers in the data. The main organizational structure of this paper is as follows: In Section 2, this paper introduces the uncertain regression model, which is an important part of uncertain statistics and also the basis of the model proposed in this paper. In Section 3, we propose an uncertain total least squares estimation model, which fully considers the errors of independent variables and dependent variables and results in better regression equations. In Section 4, because there are often gross errors or outliers in the data, this paper proposes an uncertain robust total least squares estimation model, which is more stable and reliable. In Section 5, a numerical example verifies the feasibility of uncertain total least squares estimation and uncertain robust total least squares estimation and compared with uncertain least squares estimation the data analysis shows that the regression equation of the model proposed in this paper is better. Finally, the proposed model is discussed and summarized.

2. Uncertain Regression Model

In 2007, Liu [6] founded the uncertainty theory. Uncertain statistics is an important part of uncertainty theory and can better deal with non-random or imprecise data. Readers who are interested in uncertainty theory can refer to Reference [8] for further information. This section focuses on introducing the uncertain regression model.

Assume that the vector of explanatory variables is ($x_1$, $x_2$, ⋯, $x_n$) and the response variable is y. The functional relationship between ($x_1$, $x_2$, ⋯, $x_n$) and y can be used to express it as in Equation (1)

$$\begin{array}{c}\hfill y=f(x_1,x_2,\cdots ,x_n\mid \mathit{\beta })+\epsilon ,\end{array}$$

(1)

where $\mathit{\beta }$ is an unknown vector of parameters and $\epsilon$ is a disturbance term [8]. The model is called an uncertain regression model.

In particular, Liu [8] called

$$\begin{array}{c}\hfill y={\beta }_0+{\beta }_1{x}_1+{\beta }_2{x}_2+\cdots +{\beta }_n{x}_n+\epsilon \end{array}$$

(2)

a linear regression model.

Now, suppose that we have a set of imprecise data as in Equation (3)

$$({\tilde{x}}_{i1},{\tilde{x}}_{i2},\cdots ,{\tilde{x}}_{in},{\tilde{y}}_i),i=1,2,\cdots ,m,$$

(3)

where ${\tilde{x}}_{i1}$, ${\tilde{x}}_{i2}$, ⋯, ${\tilde{x}}_{in}$, ${\tilde{y}}_i$ are independent uncertain variables with regular uncertainty distributions ${\mathrm{\Phi }}_{i1}$, ${\mathrm{\Phi }}_{i2}$, ⋯, ${\mathrm{\Phi }}_{in}$, ${\mathrm{\Psi }}_i$, i = 1, 2, ⋯, m, respectively. The following formulas are defined and derived from this premise.

Yao-Liu [15] introduced the least squares estimation method for the unknown parameter $\mathit{\beta }$ in a linear regression model. This parameter can be obtained by solving the following minimization problem

$$\underset{\mathit{\beta }}{min}\sum _{i=1}^{n}E\left[{({\tilde{y}}_i-f({\tilde{x}}_{i1},{\tilde{x}}_{i2},\cdots ,{\tilde{x}}_{in}\mid \mathit{\beta }))}^2\right].$$

(4)

The operator symbol E in Equation (4) is the symbol of the uncertain expected value, which is the average value of the uncertain variable in the sense of uncertain measure [8].

If the minimization solution is ${\mathit{\beta }}^{*}$, then the regression equation that best fits the data is given by $y=f(x_1,x_2,\cdots ,x_n\mid {\mathit{\beta }}^{*})$. For each index i ($i=1,2,\cdots ,m$), the term

$${\tilde{\epsilon }}_i={\tilde{y}}_i-f({\tilde{x}}_{i1},{\tilde{x}}_{i2},\cdots ,{\tilde{x}}_{in}\mid {\mathit{\beta }}^{*})$$

(5)

is called the i-th residual.

Let the disturbance term $\epsilon$ be an uncertain variable, whose expected value and variance can be estimated as

$$\widehat{e}=\frac{1}{n}\sum _{i=1}^{m}E\left[{\tilde{\epsilon }}_i\right],$$

(6)

and

$${\widehat{\sigma }}^2=\frac{1}{n}\sum _{i=1}^{m}E\left[{({\tilde{\epsilon }}_i-\widehat{e})}^2\right],$$

(7)

where ${\tilde{\epsilon }}_i$ are the i-th residual, i = 1, 2, ⋯, m, respectively [16].

Given a new vector of explanatory variables ($x_1$, $x_2$, ⋯, $x_n$), the forecast uncertain variable of response variable $\widehat{y}$ is

$$\widehat{y}=f(x_1,x_2,\cdots ,x_n\mid {\mathit{\beta }}^{*})+\mathcal{N}(\widehat{e},\widehat{\sigma }).$$

(8)

According to Lio-Liu [16], the forecast value is defined as the expected value of the uncertain variable $\tilde{y}$, which means

$$\widehat{u}=f(x_1,x_2,\cdots ,x_n\mid {\mathit{\beta }}^{*})+\widehat{e}.$$

(9)

Taking $\alpha$ (e.g., 95%) as the confidence level [16], the confidence interval of $\tilde{y}$ is

$$\widehat{u}\pm \frac{\widehat{\sigma }\sqrt{3}}{\pi }\ln \frac{1+\alpha }{1-\alpha }.$$

(10)

3. Uncertain Total Least Squares Estimation

Based on the traditional total least squares method [2], combined with uncertain statistics, this paper proposes uncertain total least squares estimation. This method can deal with imprecise data well and is an important linear regression model for uncertain statistics. As an extension of the traditional total least squares method, it expands the application field of linear regression models and it can produce good fitting results.

In this study, we consider a dataset consisting of imprecise data $({\tilde{x}}_{i1},{\tilde{x}}_{i2},\cdots ,{\tilde{x}}_{im},{\tilde{y}}_i)$ for $i=1,2,\cdots ,n$. Each ${\tilde{x}}_{ij}$ and ${\tilde{y}}_i$ represents an independent uncertain variable, with corresponding regular uncertainty distributions ${\mathrm{\Phi }}_{i1}$, ${\mathrm{\Phi }}_{i2}$, ⋯, ${\mathrm{\Phi }}_{im}$, ${\mathrm{\Psi }}_i$, where i ranges from 1 to n.

We assume that uncertain variables $({\tilde{x}}_{i1},{\tilde{x}}_{i2},\cdots ,{\tilde{x}}_{im})$$(i=1,2,\cdots ,n)$ and ${\tilde{y}}_i$$(i=1,2,\cdots ,n)$ have the following linear function relationship,

$$\begin{array}{cc}\hfill {\tilde{y}}_1& ={\beta }_0+{\beta }_1{\tilde{x}}_{11}+{\beta }_2{\tilde{x}}_{12}+\cdots +{\beta }_m{\tilde{x}}_{1m},\hfill \\ \hfill {\tilde{y}}_2& ={\beta }_0+{\beta }_1{\tilde{x}}_{21}+{\beta }_2{\tilde{x}}_{22}+\cdots +{\beta }_m{\tilde{x}}_{2m},\hfill \\ \hfill {\tilde{y}}_3& ={\beta }_0+{\beta }_1{\tilde{x}}_{31}+{\beta }_2{\tilde{x}}_{32}+\cdots +{\beta }_m{\tilde{x}}_{3m},\hfill \\ \hfill & \cdots \hfill \\ \hfill {\tilde{y}}_n& ={\beta }_0+{\beta }_1{\tilde{x}}_{n1}+{\beta }_2{\tilde{x}}_{n2}+\cdots +{\beta }_m{\tilde{x}}_{nm},\hfill \end{array}$$

(11)

where ${\beta }_0,{\beta }_1,{\beta }_2,\cdots ,{\beta }_m$ are unknown parameters. The model represented by Equation (11) is referred to as a multiple linear regression model. In Equation (11), $({\tilde{x}}_{i1},{\tilde{x}}_{i2},\cdots ,{\tilde{x}}_{im})$$(i=1,2,\cdots ,n)$ and ${\tilde{y}}_i$$(i=1,2,\cdots ,n)$ are uncertain variables.

According to the uncertainty theory, we can calculate the expected values of both sides of Equation (11) and we obtain the following equations with constant coefficients

$$\begin{array}{cc}\hfill E\left({\tilde{y}}_1\right)& ={\beta }_0+{\beta }_{1}E\left({\tilde{x}}_{11}\right)+{\beta }_{2}E\left({\tilde{x}}_{12}\right)+\cdots +{\beta }_{m}E\left({\tilde{x}}_{1m}\right),\hfill \\ \hfill E\left({\tilde{y}}_2\right)& ={\beta }_0+{\beta }_{1}E\left({\tilde{x}}_{21}\right)+{\beta }_{2}E\left({\tilde{x}}_{22}\right)+\cdots +{\beta }_{m}E\left({\tilde{x}}_{2m}\right),\hfill \\ \hfill E\left({\tilde{y}}_3\right)& ={\beta }_0+{\beta }_{1}E\left({\tilde{x}}_{31}\right)+{\beta }_{2}E\left({\tilde{x}}_{32}\right)+\cdots +{\beta }_{m}E\left({\tilde{x}}_{3m}\right),\hfill \\ \hfill & \cdots \hfill \\ \hfill E\left({\tilde{y}}_n\right)& ={\beta }_0+{\beta }_{1}E\left({\tilde{x}}_{n1}\right)+{\beta }_{2}E\left({\tilde{x}}_{n2}\right)+\cdots +{\beta }_{m}E\left({\tilde{x}}_{nm}\right).\hfill \end{array}$$

(12)

According to matrix theory, Equation (12) can be expressed as the following matrix equation.

$$\begin{array}{c}\hfill \left[\begin{array}{c}E\left({\tilde{y}}_1\right)\\ E\left({\tilde{y}}_2\right)\\ E\left({\tilde{y}}_3\right)\\ ⋮\\ E\left({\tilde{y}}_n\right)\end{array}\right]=\left[\begin{array}{ccccc}1& E\left({\tilde{x}}_{11}\right)& E\left({\tilde{x}}_{12}\right)& \cdots & E\left({\tilde{x}}_{1m}\right)\\ 1& E\left({\tilde{x}}_{21}\right)& E\left({\tilde{x}}_{22}\right)& \cdots & E\left({\tilde{x}}_{2m}\right)\\ 1& E\left({\tilde{x}}_{31}\right)& E\left({\tilde{x}}_{32}\right)& \cdots & E\left({\tilde{x}}_{3m}\right)\\ \cdots & \cdots & \cdots \cdots & \cdots \\ 1& E\left({\tilde{x}}_{n1}\right)& E\left({\tilde{x}}_{n2}\right)& \cdots & E\left({\tilde{x}}_{nm}\right)\end{array}\right]\left[\begin{array}{c}{\beta }_0\\ {\beta }_1\\ {\beta }_2\\ ⋮\\ {\beta }_m\end{array}\right].\end{array}$$

(13)

Assuming that

$${\beta }_{(m+1)\times 1}={\left[\begin{array}{c}{\beta }_0,{\beta }_1,{\beta }_2,\cdots ,{\beta }_m\end{array}\right]}^T,$$

(14)

$$\begin{array}{c}\hfill {\tilde{Y}}_{n\times 1}={[E\left({\tilde{y}}_1\right),E\left({\tilde{y}}_2\right),E\left({\tilde{y}}_3\right),\cdots ,E\left({\tilde{y}}_n\right)]}^T,\end{array}$$

(15)

$${\tilde{X}}_{n\times (m+1)}=\left[\begin{array}{ccccc}1& E\left({\tilde{x}}_{11}\right)& E\left({\tilde{x}}_{12}\right)& \cdots & E\left({\tilde{x}}_{1m}\right)\\ 1& E\left({\tilde{x}}_{21}\right)& E\left({\tilde{x}}_{22}\right)& \cdots & E\left({\tilde{x}}_{2m}\right)\\ 1& E\left({\tilde{x}}_{31}\right)& E\left({\tilde{x}}_{32}\right)& \cdots & E\left({\tilde{x}}_{3m}\right)\\ \cdots & \cdots & \cdots & \cdots & \cdots \\ 1& E\left({\tilde{x}}_{n1}\right)& E\left({\tilde{x}}_{n2}\right)& \cdots & E\left({\tilde{x}}_{nm}\right)\end{array}\right].$$

(16)

Then, Equation (13) can be expressed as

$${\tilde{Y}}_{n\times 1}={\tilde{X}}_{n\times (m+1)}{\beta }_{(m+1)\times 1}.$$

(17)

where, ${\tilde{Y}}_{n\times 1}$ is the observed value vector, ${\beta }_{(m+1)\times 1}$ is the unknown parameter, ${\tilde{X}}_{n\times (m+1)}$ is the coefficient matrix. Considering the existence of errors in the imprecise data, we establish the following EIV (error-in-variable) model

$${\tilde{Y}}_{n\times 1}+{\epsilon }_{n\times 1}=({\tilde{X}}_{n\times (m+1)}+E_{n\times (m+1)}){\beta }_{(m+1)\times 1},$$

(18)

where, ${\epsilon }_{n\times 1}$ is the random error matrix of ${\tilde{Y}}_{n\times 1}$ and $E_{n\times (m+1)}$ is the random error matrix of coefficient matrix ${\tilde{X}}_{n\times (m+1)}.$

The uncertain total least squares estimation method is extended based on the least squares estimation, considering the error of observation vector and coefficient matrix. The criterion of the uncertain total least squares estimation is

$$min={\epsilon }_{n\times 1}^T{\epsilon }_{n\times 1}+vec{\left(E_{n\times (m+1)}\right)}^{T}vec\left(E_{n\times (m+1)}\right),$$

(19)

where $vec(·)$ represents the matrix column vectorization operator.

Equation (18) is transformed into

$$\left[\begin{array}{cc}{\tilde{X}}_{n\times (m+1)}+E_{n\times (m+1)}& {\tilde{Y}}_{n\times 1}+{\epsilon }_{n\times 1}\end{array}\right]·\left[\begin{array}{cc}& {\beta }_{(m+1)\times 1}\\ & -1\end{array}\right]=\mathbf{0}$$

(20)

and

$$\begin{array}{cccc}\left(\right[{\tilde{X}}_{n\times (m+1)}& {\tilde{Y}}_{n\times 1}]& [E_{n\times (m+1)}& {\epsilon }_{n\times 1}\left]\right)\end{array}·\left[\begin{array}{cc}& {\beta }_{(m+1)\times 1}\\ & -1\end{array}\right]=\mathbf{0}.$$

(21)

We use matrix singular value decomposition (SVD) [22] to calculate the unknown parameters of the total least squares estimation. By singular value decomposition of the augmented matrix C, we can obtain

$$\begin{array}{c}\hfill C=\begin{array}{cc}[{\tilde{X}}_{n\times (m+1)}& {\tilde{Y}}_{n\times 1}]\end{array}=U\sum V^T.\end{array}$$

(22)

where $U=[u_1,u_2,\cdots ,u_n]$ is an orthogonal matrix composed of n eigenvalue vectors of the matrix $C{C}^T$. $V=[v_1,v_2,\cdots ,v_{m+1}]$ is an orthogonal matrix composed of $m+1$ eigenvalue vectors of the matrix $C^{T}C$. $\sum =diag[{\sigma }_1,{\sigma }_2,\cdots ,{\sigma }_{m+1}]$ is a diagonal matrix of order $n\times (m+1)$, where the elements on the diagonal are the singular value $({\sigma }_{ii}={\sigma }_i,i=1,2,\cdots ,m+1)$ and the elements on the nondiagonal are all 0.

By singular value decomposition of the augmented matrix $\widehat{C}$, we can obtain

$$\begin{array}{c}\hfill \widehat{C}=\begin{array}{cccc}\left(\right[{\tilde{X}}_{n\times (m+1)}& {\tilde{Y}}_{n\times 1}]& [E_{n\times (m+1)}& {\epsilon }_{n\times 1}\left]\right)\end{array}=U\left[\begin{array}{cc}\sum ^{\prime }& \mathbf{0}\\ \mathbf{0}& 0\end{array}\right]V^T,\end{array}$$

(23)

where ${\sum }^{\prime }=diag({\sigma }_1,{\sigma }_2,\cdots ,{\sigma }_m).$

The left and right singular vector matrices in Equation (23) can be decomposed into

$$\begin{array}{cc}\hfill & U=\left[\begin{array}{c}{U}_1,U_2\end{array}\right],\hfill \\ \hfill & V=\left[\begin{array}{c}{V}_1,V_2\end{array}\right]\hfill \\ \hfill & U_1=[u_1,u_2,\cdots ,u_{n-1}],U_2=\left[U_n\right],\hfill \\ \hfill & V_1=[v_1,v_2,\cdots ,v_m],V_2=\left[v_{m+1}\right].\hfill \end{array}$$

(24)

Equation (21) is transformed into

$$\widehat{C}\left[\begin{array}{cc}& {\beta }_{(m+1)\times 1}\\ & -1\end{array}\right]=\mathbf{0}.$$

(25)

Then, the total least squares estimate of the unknown parameter is

$${\beta }_{TLS}=-V_2{V}_{m+1,m+1}^{-1}.$$

(26)

The residual matrix is

$$\begin{array}{c}\hfill E_{min}=\left[\begin{array}{cc}{E}_{n\times (m+1)}& {\epsilon }_{n\times 1}\end{array}\right]=-{\sigma }_{m+1}{U}_2{V}_2^T.\end{array}$$

(27)

In the derivation of the uncertain total least squares estimation, we applied the matrix singular value decomposition to calculate the least squares solution of unknown parameters of the regression equation. Singular value decomposition (SVD) is an important matrix decomposition in linear algebra. It is an extension of the unitary diagonalization of normal matrices in matrix analysis. It has important applications in signal processing, statistics and other fields. This method is theoretical and requires readers to have a strong knowledge of matrix theory.

4. Uncertain Robust Total Least Squares Estimation

Linear fitting is one of the problems encountered in many engineering applications. It can be described as follows: for given data $({\tilde{x}}_i,{\tilde{y}}_i),(i=1,2,\cdots n)$, we need to find a best fit line $y=ax+b$ that passes through or is as close to these points as possible. We can use the total least squares estimator proposed in this paper to obtain estimates of parameters a and b.

To obtain the most accurate estimates of the linear parameters a and b in total least squares estimation, it is crucial to eliminate any gross errors or outliers present in the observed data. Building upon the principles of total least squares estimation, this paper introduces the uncertain robust total least squares estimation. The algorithm steps for this approach are outlined below:

Step 1. Calculate the initial values of a and b according to the total least squares estimation.

Step 2. Based on the calculated values of a and b, the distance $d_i$ from each point to the fitted line is calculated.

$$d_i=\frac{|a{x}_i-y_i+b|}{\sqrt{a^2+1}}.$$

(28)

Step 3. Calculate the standard deviation of distance $d_i$ according to the following equation

$$\gamma =\sqrt{\frac{{(d_i-\overline{d})}^T(d_i-\overline{d})}{n-1}},$$

(29)

where

$$\overline{d}=\frac{1}{n}\sum _{i=1}^n{d}_i.$$

(30)

Step 4. In general, we specify a distance threshold of 2 standard deviations, that is, when $d_i>2\gamma$, the point is considered an outlier and is removed. Otherwise, it is retained.

Step 5. Recalculate a and b using all the remaining points.

Step 6. Repeat steps 2 to 5 until $d_i$ of all remaining points is within the specified threshold; that is, $d_i$ is less than two times the standard deviation.

Step 7. Calculate the final values of parameters a and b based on the uncertain total least squares estimation.

5. Numerical Example

In order to verify the feasibility and superiority of total least squares estimation and robust total least squares estimation, we provide an example of imprecise data and compare them with least squares estimation. Data analysis shows that total least squares estimation and robust total least squares estimation have better effect and higher accuracy. In addition, we also analyze the estimated expected value and variance of the disturbance term of the regression equation and calculate the forecast value and confidence interval.

In

Table 1. Imprecise data (linear uncertainty distribution).

i	1	2	3	4
$x_i$	$\mathcal{L}$(2,4)	$\mathcal{L}$(3,5)	$\mathcal{L}(4,6)$	$\mathcal{L}(5,6)$
$y_i$	$\mathcal{L}(4,6)$	$\mathcal{L}$(6,8)	$\mathcal{L}$(3,5)	$\mathcal{L}$(10,12)
i	5	6	7	8
$x_i$	$\mathcal{L}\left(\mathrm{6,8}\right)$	$\mathcal{L}(7,9)$	$\mathcal{L}(8,10)$	$\mathcal{L}(9,10)$
$y_i$	$\mathcal{L}$(12,14)	$\mathcal{L}$(14,16)	$\mathcal{L}$(22,24)	$\mathcal{L}$(18,20)
i	9	10	11	12
$x_i$	$\mathcal{L}(10,12)$	$\mathcal{L}(11,13)$	$\mathcal{L}(12,14)$	$\mathcal{L}(13,15)$
$y_i$	$\mathcal{L}$(20,22)	$\mathcal{L}$(22,24)	$\mathcal{L}$(24,26)	$\mathcal{L}$(26,28)

, we assume that the imprecise data ($x_i$,$y_i$), $i=1,2,\cdots ,12$ are independent uncertain variables with regular linear uncertainty distributions.

Table 2. The linear regression equations.

Model	Regression Equations
least squares estimation	$y=-1.8428+2.1298x$
total least squares estimation	$y=-1.363+2.0455x$
robust total least squares estimation	$y=-0.9815+2.0093x$

shows the linear regression equations derived using least squares estimations, total least squares estimations and robust total least squares estimations.

From Table 2, it can be seen that there is little difference between the coefficients of regression equations for least squares estimation, for total least squares estimation and for robust total least squares estimation and that the method proposed here is a feasible one.

Table 3. The expected value and variance of the disturbance term.

Model	Expected Value	Variance
least squares estimation	$0.0000$	$59.8156$
total least squares estimation	$0.0000$	$9.9545$
robust total least squares estimation	$0.0000$	$0.8889$

shows a comparison between the expected value and variance of the disturbance terms when least squares estimation, total least squares estimation and robust total least squares estimation are undertaken.

In Table 3, it is shown that the expected value of the disturbance term in regression equations of least squares estimation, total least squares estimation and robust total least squares estimation are all the same, so we are able to say that all three methods produce the same regression results. With robust total least squares estimation, the disturbance term in the regression equation has the smallest variance, the best effect and the highest level of accuracy compared to all other estimation methods.

Forecast values and confidence intervals can also be calculated using robust total least squares estimation. Assume that $x\sim \mathcal{L}(14,15)$ is imprecise data and according to [16], we obtain a forecast value of $\widehat{u}$ as $28.1534.$

The confidence level is assumed to be $\alpha =95\%$ and the interference term is assumed to have an uncertain normal distribution $\mathcal{N}(\widehat{e},\widehat{\sigma })$. According to Reference [16], the confidence interval of variable y is

$$28.1534\pm 1.9053.$$

Uncertain total least squares estimation takes into account the errors of independent variables and dependent variables, which makes the linear regression equation more efficient. Uncertain robust total least squares estimation removes the outliers from the data on the basis of total least squares estimation, so the regression equation has better effect and higher accuracy. In the example, compared with the data results of the existing methods, robust total least squares estimation has a better regression effect than other methods.

6. Conclusions

There are many practical applications for linear regression and we always expect to obtain better linear fitting equations when we apply linear regression techniques. The problem can be solved using the mathematical statistics and probability theory when the given data are random. In cases where the data are not random, we can use the estimation method that has been proposed in this paper to estimate the data. It is demonstrated in this paper that total least squares estimation takes into account both the errors of the explanatory variables as well as errors of the response variables, which makes it more effective. It is also important to note that a robust total least squares linear estimation method has also been proposed in order to address the issue of outliers not being taken into account in previous linear fitting methods. With the help of this method, gross errors or outliers in the data can be removed and the regression equation obtained is more accurate and more reliable than those obtained using other estimation methods. Robust total least squares linear estimation is more stable in practice.

In the derivation of the uncertain robust total least squares estimation method, this paper uses the singular value decomposition method of the matrix. In spite of the fact that this method is widely used and has great effectiveness, it is difficult to understand and requires a certain level of knowledge on linear algebra and matrix theory.