# A New Computational Method for Estimating Simultaneous Equations Models Using Entropy as a Parameter Criteria

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Definition of the Model and Methods for Estimating a SEM Problem

#### 2.1. Definition of the Model

#### 2.2. Methods for Estimating an SEM Problem

#### 2.2.1. Two Stage Least Squares (2SLS)

#### 2.2.2. Bayesian Method of Moments ($BMOM$)

#### 2.2.3. Bayesian Approach in Two Stages ($Baye{s}_{2S}$)

#### 2.2.4. Markov Chain Monte Carlo (MCMC)

## 3. The Proposed Estimation Method: Optimized BMOM Method (${\mathit{Bmom}}_{\mathit{OPT}}$)

## 4. Entropy as an Information Parameter Criteria

## 5. Experimental Design and Results

#### 5.1. Experimental Design

#### 5.2. Experimental Results

## 6. Conclusions and Future Work

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## References

- Gujarati, D.; Porter, D. Econometría, 5th ed.; México, D.F., Ed.; McGraw-Hill Interamericana Editores SA: Ciudad de México, México, 2004. [Google Scholar]
- Zellner, A. The finite sample properties of simultaneous equations’ estimates and estimators Bayesian and non-Bayesian approaches. J. Econom.
**1998**, 83, 185–212. [Google Scholar] [CrossRef] - Chao, J.C.; Phillips, P.C. Jeffreys prior analysis of the Simultaneous Equations Model in the case with n+1 endogenous variables. J. Econom.
**2002**, 111, 251–283. [Google Scholar] [CrossRef] [Green Version] - Geweke, J. Bayesian Reduced Rank Regression in Econometrics. J. Econ. Trics
**1996**, 75, 121–146. [Google Scholar] [CrossRef] - Kleibergen, F.; Dijk, H.V. Bayesian simultaneous equation analysis using reduced rank structures. Econom. Theory
**1998**, 14, 701–743. [Google Scholar] [CrossRef] [Green Version] - Gelman, A.; Carlin, J.B.; Stern, H.S.; Dunson, D.; Vehtari, A.; Rubin, D.B. Bayesian Data Analysis; Chapman and Hall: New York, NY, USA, 2015. [Google Scholar]
- Siburian, M.E. Fiscal decentralization and regional income inequality: Evidence from Indonesia. Appl. Econ. Lett.
**2019**, 1–4. [Google Scholar] [CrossRef] - Steele, F.; French, R.; Bartley, M. Adjusting for Selection Bias in Longitudinal Analysis Using Simultaneous Equations Modeling. The Relationship Between Employment Transitions and Mental Health. Epidemiology
**2013**, 24, 703–711. [Google Scholar] [CrossRef] - Zhao, H.; Lin, B. Impact of foreign trade on energy efficiency in China’s textile industry. J. Clean. Prod.
**2019**, 245, 118878. [Google Scholar] [CrossRef] - Adewuyi, A.O.; Awodumi, O.B. Biomass energy consumption, economic growth and carbon emi-ssions: Fresh evidence from West Africa using a simultaneous equation model. Energy
**2017**, 119, 453–471. [Google Scholar] [CrossRef] - King, T. Using simultaneous equation modelling for defining complex phenotypes. BMC Genet.
**2003**, 4, S10. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Pitfield, D.; Caves, R.; Quddus, M. Airline strategies for aircraft size and airline frequency with changing demand and competition: A simultaneous-equations approach for traffic on the north Atlantic. J. Air Transp. Manag.
**2009**, 16, 151–158. [Google Scholar] [CrossRef] [Green Version] - Olmeda, N.G.; Martinez, I.B. Application of simultaneous equation models to temporary disability prescriptions in primary healthcare centres. Int. J. Comput. Math.
**2014**, 91, 252–260. [Google Scholar] [CrossRef] - Strathe, A.; Jørgensen, H.; Kebreab, E.; Danfær, A. Bayesian simultaneous equation models for the analysis of energy intake and partitioning in growing pigs. J. Agric. Sci.
**2012**, 150, 764–774. [Google Scholar] [CrossRef] [Green Version] - Zhou, W.; Duan, W. An empirical study of how third-party websites influence the feedback mechanism between online Word-of-Mouth and retail sales. Decis. Support Syst.
**2015**, 76, 14–23. [Google Scholar] [CrossRef] - Yang, S.; Narayan, V.; Assael, H. Estimating the Interdependence of Television Program Viewership Between Spouses: A Bayesian Simultaneous Equation Model. Mark. Sci.
**2006**, 25, 336–349. [Google Scholar] [CrossRef] - Park, H.; Manchanda, P. When Harry Bet with Sally: An Empirical Analysis of Multiple Peer Effects in Casino Gambling Behavior. Mark. Sci.
**2015**, 2, 179–194. [Google Scholar] [CrossRef] - Chen, C.; Chang, K.; Lin, L.; Lee, J. Health risk perception and betel chewing behavior. The evidence from Taiwan. Addict. Behav.
**2013**, 38, 2714–2717. [Google Scholar] [CrossRef] - Cohen, D.P.; Benadjaoud, M.A.; Lestaevel, P.; Lebsir, D.; Benderitter, M.; Souidi, M. Effects of repetitive Iodine Thyroid Blocking on the Development of the Foetal Brain and Thyroid in rats: A Systems Biology approach. bioRxiv
**2019**. [Google Scholar] [CrossRef] [Green Version] - Akaike, H. Information Theory and an Extension of the Maximum Likelihood Principle; Springer: New York, NY, USA, 1998. [Google Scholar]
- Keerativibool, W. New Criteria for Selection in Simultaneous Equations Model. Thail. Stat.
**2012**, 10, 163–181. [Google Scholar] - Hurvich, C.; Tsai, C. Regression and time series model selection in small samples. Biometrika
**1989**, 76, 297–397. [Google Scholar] [CrossRef] - Schwarz, G. Estimating the Dimension of a Model. Ann. Stat.
**1978**, 6, 461–464. [Google Scholar] [CrossRef] - Findley, D.F. Counterexamples to parsimony and BIC. Ann. Inst. Stat. Math.
**1991**, 43, 505–514. [Google Scholar] [CrossRef] - Hannan, E.J.; Quinn, B.G. The Determination of the Order of an Autoregression. J. R. Stat. Soc. Ser. B (Methodol.)
**1979**, 41, 190–195. [Google Scholar] [CrossRef] - Keerativibool, W.; Jitthavech, J. Model Selection Criterion Based on Kullback-Leibler’s Symmetric Divergence for Simultaneous Equations Model. Chiang Mai J. Sci.
**2015**, 42, 761–773. [Google Scholar] - Cover, T.M.; Thomas, J.A. Elements of Information Theory; John Wiley and Sons, Inc.: Hoboken, NJ, USA, 1991. [Google Scholar]
- Zhou, R.; Cai, R.; Tong, G. Applications of Entropy in Finance: A Review. Entropy
**2013**, 15, 4909–4931. [Google Scholar] [CrossRef] - Cui, H.; Sivakumar, B.; Singh, V. Entropy Applications in Environmental and Water Engineering. Entropy
**2019**, 20, 598. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Purvis, B.; Mao, Y.; Robinson, D. Entropy and its Application to Urban Systems. Entropy
**2019**, 21, 56. [Google Scholar] [CrossRef] [Green Version] - Oruç, Ö.E.; Kuruoglu, E.; Gündüz, A. Entropy Applications for Customer Satisfaction Survey in Information Theory. Front. Sci.
**2011**, 1, 1–4. [Google Scholar] [CrossRef] - Amigó, J.M.; Balogh, S.G.; Hernández, S. A Brief Review of Generalized Entropies. Entropy
**2018**, 20, 813. [Google Scholar] [CrossRef] [Green Version] - Qayyum, Z.; Hasan, S.S. K-Class estimators-a Review. Int. J. Math. Trends Technol.
**2017**, 50, 104–107. [Google Scholar] [CrossRef] - Theil, H. Repeated Least Squares Applied to Complete Equation Systems; Central Planning Bureau: The Hague, The Netherlands, 1953.
- Basmann, R.L. A Generalized Classical Method of Linear Estimation of Coefficients in a Structural Equation. Econometrica
**1957**, 25, 77–83. [Google Scholar] [CrossRef] - Clausius, R. The Mechanical Theory of Heat; Macmillan: New York, NY, USA, 1879. [Google Scholar]
- Shannon, C.E. A mathematical theory of communication. Bell Syst. Tech. J.
**1948**, 27, 379–423. [Google Scholar] [CrossRef] [Green Version] - Tsallis, C. Possible generalization of Boltzmann-Gibbs statistics. J. Stat. Phys.
**1988**, 52, 479–487. [Google Scholar] [CrossRef] - Lombardi, O.; Holik, F.; Vanni, L. What is Shannon information? Synthese
**2016**, 193, 1983–2012. [Google Scholar] [CrossRef] [Green Version]

**Table 1.**K1 and K2 parameters proposed to minimize loss function. Bayesian Method of Moments (BMOM).

Loss Function | BMOM Approach | |
---|---|---|

Goodness of fit | ${L}_{g}={({y}_{1}-{Z}_{1}{\widehat{\delta}}_{1})}^{{}^{\prime}}({y}_{1}-{Z}_{1}{\widehat{\delta}}_{1})$ | ${K}_{1}=1-k/(n-k)$, ${K}_{2}=1$ |

Precision of estimation | ${L}_{p}={({\delta}_{1}-{\widehat{\delta}}_{1})}^{{}^{\prime}}{Z}_{1}^{{}^{\prime}}{Z}_{1}({\delta}_{1}-{\widehat{\delta}}_{1})$ | ${K}_{1}={K}_{2}=1-k/(n-k)$ |

**Table 2.**Average and standard deviation of 50 simulations of ${D}_{\delta ,\widehat{\delta}}$, $AI{C}_{real}$, $AIC$, $H\left(e\right)$, and execution time in seconds. Markov Chain Monte Carlo (MCMC). Sigma $0.1$.

m | k | n | 2SLS | BMOM | ${\mathit{Bmom}}_{\mathit{OPT}}$ | ${\mathit{Bayes}}_{\mathbf{2}\mathit{S}}$${}^{\mathit{a}}$ | MCMC ${}^{\mathit{b}}$ | ||
---|---|---|---|---|---|---|---|---|---|

Goodness of Fit | Precision of Estimation | ||||||||

${D}_{\delta ,\widehat{\delta}}$ | 10 | 20 | 100 | 27.670 ${}^{7.778}$ | 40.914 ^{7.538} | 40.966 ^{7.546} | 20.647^{8.826} | 33.673 ^{12.084} | 71.410 ^{10.491} |

10 | 40 | 100 | 40.927 ^{9.104} | 58.635 ^{8.039} | 58.932 ^{8.100} | 26.769^{8.209} | 56.340 ^{13.721} | 91.076 ^{4.918} | |

20 | 60 | 100 | 115.852 ^{8.537} | 141.029 ^{6.294} | 140.906 ^{6.250} | 94.999^{12.498} | 146.640 ^{5.904} | 163.771 ^{6.092} | |

10 | 20 | 400 | 16.563 ^{11.257} | 27.508 ^{6.955} | 27.534 ^{6.974} | 10.619^{5.868} | 22.233 ^{17.026} | 70.449 ^{7.899} | |

10 | 40 | 400 | 15.199 ^{4.366} | 30.538 ^{6.467} | 30.494 ^{6.446} | 7.923^{2.576} | 26.009 ^{22.856} | 90.357 ^{5.568} | |

10 | 40 | 1000 | 7.394 ^{2.830} | 17.218 ^{5.450} | 17.229 ^{5.458} | 5.130^{1.944} | 9.233 ^{7.540} | 95.210 ^{10.424} | |

$AI{C}_{real}$ | 10 | 20 | 100 | 1361.764 ^{781.683} | 1895.361 ^{801.349} | 1896.600 ^{800.971} | 1156.465^{792.277} | 1993.402 ^{831.727} | 4598.074 ^{395.783} |

10 | 40 | 100 | 1915.270 ^{651.931} | 2345.388 ^{636.213} | 2351.286 ^{635.582} | 1718.000^{626.334} | 2531.052 ^{715.689} | 4547.361 ^{286.781} | |

20 | 60 | 100 | 5941.844 ^{779.559} | 6458.904 ^{762.773} | 6456.303 ^{763.063} | 5692.839^{793.354} | 6732.681 ^{777.638} | 10,172.018 ^{206.247} | |

10 | 20 | 400 | 4438.187 ^{3271.180} | 7319.586 ^{3449.337} | 7323.685 ^{3450.560} | 3409.573^{3232.656} | 7517.898 ^{5184.224} | 22875.120 ^{1641.293} | |

10 | 40 | 400 | 4645.919 ^{2358.815} | 6989.900 ^{2744.681} | 6983.853 ^{2744.491} | 3877.930^{2299.173} | 7382.701 ^{4943.642} | 22,057.334 ^{917.641} | |

10 | 40 | 1000 | 6824.562 ^{8001.673} | 11,738.582 ^{8294.696} | 11,742.499 ^{8294.949} | 5913.439^{8123.421} | 9046.596 ^{8850.159} | 63,247.410 ^{2586.679} | |

$AIC$ | 10 | 20 | 100 | 2168.030 ^{854.334} | 1784.122 ^{808.851} | 1783.602^{808.150} | 2419.415 ^{893.327} | 2391.887 ^{785.620} | 4413.688 ^{421.952} |

10 | 40 | 100 | 2009.636 ^{639.153} | 1850.753 ^{626.141} | 1850.716^{626.059} | 2372.355 ^{692.930} | 2254.874 ^{728.737} | 4348.852 ^{256.390} | |

20 | 60 | 100 | 3866.102 ^{1051.585} | 3647.221 ^{1056.690} | 3645.439^{1056.018} | 4543.780 ^{1054.504} | 4119.416 ^{1128.524} | 9856.712 ^{241.454} | |

10 | 20 | 400 | 15,027.648 ^{3347.331} | 13,448.372 ^{3161.253} | 13,446.459^{3161.039} | 15,524.231 ^{3459.419} | 15626.142 ^{3279.198} | 22,160.759 ^{1516.667} | |

10 | 40 | 400 | 12,849.587 ^{2699.761} | 11,990.079^{2509.827} | 11,991.659 ^{2509.734} | 13,479.303 ^{2744.344} | 13,606.665 ^{2961.619} | 21,421.728 ^{858.468} | |

10 | 40 | 1000 | 37,879.770 ^{9621.598} | 36,438.251 ^{9459.390} | 36,437.331^{9459.167} | 38,479.010 ^{9605.284} | 38,035.942 ^{9483.024} | 61,720.600 ^{2138.345} | |

$H\left(e\right)$ | 10 | 20 | 100 | 4.074 ^{0.013} | 4.081 ^{0.012} | 4.081 ^{0.012} | 4.074^{0.013} | 4.084 ^{0.018} | 4.096 ^{0.016} |

10 | 40 | 100 | 4.076 ^{0.012} | 4.080 ^{0.010} | 4.080 ^{0.010} | 4.075^{0.014} | 4.084 ^{0.011} | 4.087 ^{0.014} | |

20 | 60 | 100 | 4.086^{0.008} | 4.087 ^{0.009} | 4.087 ^{0.009} | 4.086^{0.009} | 4.086^{0.009} | 4.088 ^{0.009} | |

10 | 20 | 400 | 5.579^{0.005} | 5.579^{0.005} | 5.579 ^{0.005} | 5.579^{0.005} | 5.587 ^{0.021} | 5.609 ^{0.008} | |

10 | 40 | 400 | 5.590^{0.005} | 5.590^{0.005} | 5.590^{0.005} | 5.590^{0.005} | 5.593 ^{0.014} | 5.602 ^{0.007} | |

10 | 40 | 1000 | 6.530 ^{0.003} | 6.531 ^{0.003} | 6.531 ^{0.003} | 6.530^{0.004} | 6.532 ^{0.013} | 6.568 ^{0.004} | |

Time (s) | 10 | 20 | 100 | 0.073 ^{0.145} | 0.732 ^{0.246} | 0.732 ^{0.246} | 258.227 ^{98.562} | 0.056^{0.017} | 294.453 ^{17.949} |

10 | 40 | 100 | 0.143 ^{0.047} | 1.022 ^{0.395} | 1.022 ^{0.395} | 274.088 ^{345.929} | 0.140^{0.050} | 499.554 ^{68.116} | |

20 | 60 | 100 | 0.314^{0.047} | 2.495 ^{0.395} | 2.495 ^{0.395} | 748.504 ^{345.929} | 0.407 ^{0.145} | 1435.791 ^{0.145} | |

10 | 20 | 400 | 0.125 ^{0.031} | 4.265 ^{0.864} | 4.265 ^{0.864} | 2586.186 ^{1005.068} | 0.109^{0.030} | 328.571 ^{18.174} | |

10 | 40 | 400 | 0.235 ^{0.037} | 4.533 ^{0.765} | 4.533 ^{0.765} | 2281.255 ^{804.186} | 0.214^{0.032} | 507.791 ^{38.694} | |

10 | 40 | 1000 | 0.426 ^{0.066} | 21.385 ^{1.595} | 21.385 ^{1.595} | 14,534.080 ^{21,689.539} | 0.376^{0.107} | 524.904 ^{26.564} |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Pérez-Sánchez, B.; González, M.; Perea, C.; López-Espín, J.J.
A New Computational Method for Estimating Simultaneous Equations Models Using Entropy as a Parameter Criteria. *Mathematics* **2021**, *9*, 700.
https://doi.org/10.3390/math9070700

**AMA Style**

Pérez-Sánchez B, González M, Perea C, López-Espín JJ.
A New Computational Method for Estimating Simultaneous Equations Models Using Entropy as a Parameter Criteria. *Mathematics*. 2021; 9(7):700.
https://doi.org/10.3390/math9070700

**Chicago/Turabian Style**

Pérez-Sánchez, Belén, Martín González, Carmen Perea, and Jose J. López-Espín.
2021. "A New Computational Method for Estimating Simultaneous Equations Models Using Entropy as a Parameter Criteria" *Mathematics* 9, no. 7: 700.
https://doi.org/10.3390/math9070700