# Assessing the Interpretability–Performance Trade-Off of Artificial Neural Networks Using Sentinel Fish Health Data

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

#### The Interpretability–Performance Trade-Off

## 2. Materials and Methods

#### 2.1. Trout-Perch Data

#### 2.2. Water Quality Data

#### 2.3. Amalgamated Data

#### 2.4. Scaling Endpoints

#### 2.5. Data Partitioning

#### 2.6. Analysis Techniques

`glmnet`library [25], and the neural networks were developed using

`Keras`and

`tensorflow`libraries [26,27].

#### Neural Networks

#### 2.7. Hyperparameter Optimization

`rBayesianOptimization`package available in R [37]. We ran 100 iterations of the algorithm for both the standard NN and the hybrid NN for each of the three response variables to identify optimal parameters for a total of six unique sets of hyperparameters. Resulting model architectures are presented in Tables S1 and S2 (Supplementary Materials).

#### 2.8. Model Assessments

## 3. Results

#### 3.1. Temporal Split Results

#### 3.2. Random Split Results

## 4. Discussion

## 5. Conclusions

## Supplementary Materials

## Author Contributions

## Funding

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## Abbreviations

OSR | Oil Sands Region. |

JOSM | Joint Canada–Alberta Oils Sands Monitoring Program. |

NN | Neural Network. |

EN | Elastic Net. |

K | Condition Factor. |

GSI | Gonadosomatic Index. |

LSI | Liver Somatic Index. |

ELU | Exponential Linear Unit. |

PI | Probability of Improvement. |

MSE | Mean Square Error. |

MSPE | Mean Square Prediction Error. |

RF | Random Forest. |

CART | Classification and Regression Tree. |

## References

- Environment Canada. Joint Canada/Alberta Implementation Plan for Oil Sands Monitoring; Technical Report En84-89/2013E-PDF; Environment Canada: Gatineau, QC, Canada, 2012. [Google Scholar]
- Stewart-Oaten, A.; Bence, J.R. Temporal and spatial variation in environmental impact assessment. Ecol. Monogr.
**2001**, 71, 305–339. [Google Scholar] [CrossRef] - Underwood, A. On beyond BACI: Sampling designs that might reliably detect environmental disturbances. Ecol. Appl.
**1994**, 4, 3–15. [Google Scholar] [CrossRef] - Anderson, M.J.; Thompson, A.A. Multivariate control charts for ecological and environmental monitoring. Ecol. Appl.
**2004**, 14, 1921–1935. [Google Scholar] [CrossRef] - Kilgour, B.W.; Somers, K.M.; Barrett, T.J.; Munkittrick, K.R.; Francis, A.P. Testing against “normal” with environmental data. Integr. Environ. Assess. Manag.
**2017**, 13, 188–197. [Google Scholar] [CrossRef] [PubMed] - Arciszewski, T.J.; Munkittrick, K.R. Development of an adaptive monitoring framework for long-term programs: An example using indicators of fish health. Integr. Environ. Assess. Manag.
**2015**, 11, 701–718. [Google Scholar] [CrossRef] [PubMed] - Cook, C.N.; de Bie, K.; Keith, D.A.; Addison, P.F. Decision triggers are a critical part of evidence-based conservation. Biol. Conserv.
**2016**, 195, 46–51. [Google Scholar] [CrossRef] - Arciszewski, T.J. A Review of Control Charts and Exploring Their Utility for Regional Environmental Monitoring Programs. Environments
**2023**, 10, 78. [Google Scholar] [CrossRef] - Wiklund, J.A.; Hall, R.I.; Wolfe, B.B.; Edwards, T.W.; Farwell, A.J.; Dixon, D.G. Use of pre-industrial floodplain lake sediments to establish baseline river metal concentrations downstream of Alberta oil sands: A new approach for detecting pollution of rivers. Environ. Res. Lett.
**2014**, 9, 124019. [Google Scholar] [CrossRef] - McMillan, P.G.; Feng, Z.Z.; Deeth, L.E.; Arciszewski, T.J. Improving monitoring of fish health in the oil sands region using regularization techniques and water quality variables. Sci. Total Environ.
**2022**, 811, 152301. [Google Scholar] [CrossRef] - Krizhevsky, A.; Sutskever, I.; Hinton, G. ImageNet classification with deep convolutional neural networks. Commun. ACM
**2017**, 60, 84–90. [Google Scholar] [CrossRef] - Biganzoli, E.; Boracchi, P.; Mariani, L.; Ettore, M. Feed forward neural networks for the analysis of censored survival data: A partial logistic regression approach. Stat. Med.
**1998**, 17, 1169–1186. [Google Scholar] [CrossRef] - Liang, F.; Li, Q.; Zhou, L. Bayesian Neural Networks for Selection of Drug Sensitive Genes. J. Am. Stat. Assoc.
**2018**, 113, 955–972. [Google Scholar] [CrossRef] [PubMed] - Montavon, G.; Samek, W.; Müller, K.R. Methods for interpreting and understanding deep neural networks. Digit. Signal Process.
**2018**, 73, 1–15. [Google Scholar] [CrossRef] - Zou, H.; Hastie, T. Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. Stat. Methodol.
**2005**, 67, 301–320. [Google Scholar] [CrossRef] - Mannix, A.E.; Dridi, C.; Adamowicz, W.L. Water availability in the oil sands under projections of increasing demands and a changing climate: An assessment of the Lower Athabasca Water Management Framework (Phase 1). Can. Water Resour. J.
**2010**, 35, 29–52. [Google Scholar] [CrossRef] - Kelly, E.N.; Schindler, D.W.; Hodson, P.V.; Short, J.W.; Radmanovich, R.; Nielsen, C.C. Oil sands development contributes elements toxic at low concentrations to the Athabasca River and its tributaries. Proc. Natl. Acad. Sci. USA
**2010**, 107, 16178–16183. [Google Scholar] [CrossRef] [PubMed] - Whitfield, C.J.; Watmough, S.A. Acid deposition in the Athabasca Oil Sands Region: A policy perspective. Environ. Monit. Assess.
**2015**, 187, 1–12. [Google Scholar] [CrossRef] - Hornik, K. Approximation capabilities of multilayer feedforward networks. Neural Netw.
**1991**, 4, 251–257. [Google Scholar] [CrossRef] - Environment and Climate Change Canada. Joint Canada-Alberta Oil Sands Environmental Monitoring Information Portal. 2020. Available online: https://www.canada.ca/en/environment-climate-change/services/oil-sands-monitoring.html (accessed on 4 March 2020).
- Kilgour, B.W.; Munkittrick, K.R.; Hamilton, L.; Proulx, C.L.; Somers, K.M.; Arciszewski, T.; McMaster, M. Developing Triggers for Environmental Effects Monitoring Programs for Trout-Perch in the Lower Athabasca River (Canada). Environ. Toxicol. Chem.
**2019**, 38, 1890–1901. [Google Scholar] [CrossRef] - Packard, G.C.; Boardman, T.J. The Misuse of Ratios, Indices, and Percentages in Ecophysiological Research. Physiol. Zool.
**1988**, 61, 1–9. [Google Scholar] [CrossRef] - Jackson, D.A.; Somers, K.M. Adjusting Mercury Concentration for Fish-Size Covariation: A Multivariate Alternative to Bivariate Regression. Can. J. Fish. Aquat. Sci.
**1993**, 50, 2388–2396. [Google Scholar] - R Core Team. R: A Language and Environment for Statistical Computing; R Foundation for Statistical Computing: Vienna, Austria, 2023. [Google Scholar]
- Friedman, J.; Hastie, T.; Tibshirani, R. Regularization paths for generalized linear models via coordinate descent. J. Stat. Softw.
**2010**, 33, 1. [Google Scholar] [CrossRef] [PubMed] - Allaire, J.; Chollet, F. keras: R Interface to ‘Keras’, R package version 2.11.1; R Foundation for Statistical Computing: Vienna, Austria, 2023.
- Allaire, J.; Tang, Y. tensorflow: R Interface to ‘TensorFlow’, R package version 2.11.0; R Foundation for Statistical Computing: Vienna, Austria, 2022.
- Ying, X. An overview of overfitting and its solutions. J. Phys. Conf. Ser.
**2019**, 1168, 022022. [Google Scholar] [CrossRef] - Nakkiran, P. Learning rate annealing can provably help generalization, even for convex problems. arXiv
**2020**, arXiv:2005.07360. [Google Scholar] - Li, Y.; Wei, C.; Ma, T. Towards explaining the regularization effect of initial large learning rate in training neural networks. Adv. Neural Inf. Process. Syst.
**2019**, 32, 1–12. [Google Scholar] - Yao, Y.; Rosasco, L.; Caponnetto, A. On Early Stopping in Gradient Descent Learning. Constr. Approx.
**2007**, 26, 289–315. [Google Scholar] [CrossRef] - Bergstra, J.; Bengio, Y. Random Search for Hyper-Parameter Optimization. J. Mach. Learn. Res.
**2012**, 13, 281–305. [Google Scholar] - Bergstra, J.; Bardenet, R.; Bengio, Y.; Kégl, B. Algorithms for Hyper-Parameter Optimization. In Proceedings of the 25th Annual Conference on Neural Information Processing Systems (NIPS 2011), Granada, Spain, 12–15 December 2011; Volume 24. [Google Scholar]
- Snoek, J.; Larochelle, H.; Adams, R.P. Practical Bayesian Optimization of Machine Learning Algorithms. In Proceedings of the 25th International Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA, 3–6 December 2012; Volume 2, pp. 2951–2959. [Google Scholar]
- Brochu, E.; Cora, V.M.; de Freitas, N. A Tutorial on Bayesian Optimization of Expensive Cost Functions, with Application to Active User Modeling and Hierarchical Reinforcement Learning. arXiv
**2010**, arXiv:1012.2599. [Google Scholar] - Hoffman, M.; Brochu, E.; De Freitas, N. Portfolio Allocation for Bayesian Optimization. In Proceedings of the UAI, Barcelona, Spain, 14–17 July 2011; pp. 327–336. [Google Scholar]
- Yachen, Y. rBayesianOptimization: Bayesian Optimization of Hyperparameters, R package version 1.2.0; R Foundation for Statistical Computing: Vienna, Austria, 2021.
- Opitz, D.; Maclin, R. Popular ensemble methods: An empirical study. J. Artif. Intell. Res.
**1999**, 11, 169–198. [Google Scholar] [CrossRef] - Arciszewski, T.J.; Hazewinkel, R.R.; Munkittrick, K.R.; Kilgour, B.W. Developing and applying control charts to detect changes in water chemistry parameters measured in the Athabasca River near the oil sands: A tool for surveillance monitoring: Control charts and water chemistry parameters. Environ. Toxicol. Chem.
**2018**, 37, 2296–2311. [Google Scholar] [CrossRef] - Taylor, L.N.; McGeer, J.C.; Wood, C.M.; McDonald, D.G. Physiological effects of chronic copper exposure to rainbow trout (Oncorhynchus mykiss) in hard and soft water: Evaluation of chronic indicators. Environ. Toxicol. Chem.
**2000**, 19, 2298–2308. [Google Scholar] [CrossRef] - Bhullar, A.; Nadeem, K.; Ali, R.A. Simultaneous multi-crop land suitability prediction from remote sensing data using semi-supervised learning. Sci. Rep.
**2023**, 13, 6823. [Google Scholar] [CrossRef] [PubMed] - Sun, A.Y.; Scanlon, B.R. How can Big Data and machine learning benefit environment and water management: A survey of methods, applications, and future directions. Environ. Res. Lett.
**2019**, 14, 073001. [Google Scholar] [CrossRef] - Arciszewski, T.J.; Munkittrick, K.R.; Scrimgeour, G.J.; Dubé, M.G.; Wrona, F.J.; Hazewinkel, R.R. Using adaptive processes and adverse outcome pathways to develop meaningful, robust, and actionable environmental monitoring programs. Integr. Environ. Assess. Manag.
**2017**, 13, 877–891. [Google Scholar] [CrossRef] [PubMed]

**Figure 1.**Ensemble performance of the standard and hybrid neural networks over 100 iterations for the adjusted body weight response under the temporal split scenario.

**Figure 2.**Ensemble performance of the standard and hybrid neural networks over 100 iterations for the adjusted gonad weight response under the temporal split scenario.

**Figure 3.**Ensemble performance of the standard and hybrid neural networks over 100 iterations for the adjusted liver weight response under the temporal split scenario.

**Figure 4.**Ensemble performance of the standard and hybrid neural networks over 100 iterations for the adjusted body weight response under the random split scenario.

**Figure 5.**Ensemble performance of the standard and hybrid neural networks over 100 iterations for the adjusted gonad weight response under the random split scenario.

**Figure 6.**Ensemble performance of the standard and hybrid neural networks over 100 iterations for the adjusted liver weight response under the random split scenario.

**Table 1.**Median values performance measures of each model by response for the temporal split scenario.

Response | Technique | MSPE | MSE |
---|---|---|---|

Adjusted Body Weight | EN | 2.83 × 10^{−3} | 1.09 × 10^{−3} |

NN | 2.34 × 10^{−3} | 6.41 × 10^{−4} | |

Hybrid | 2.36 × 10^{−3} | 7.00 × 10^{−4} | |

Adjusted Gonad Weight | EN | 1.02 × 10^{−1} | 5.05 × 10^{−2} |

NN | 6.96 × 10^{−2} | 3.30 × 10^{−2} | |

Hybrid | 7.28 × 10^{−2} | 2.13 × 10^{−2} | |

Adjusted Liver Weight | EN | 2.29 × 10^{−2} | 2.28 × 10^{−2} |

NN | 4.83 × 10^{−2} | 1.41 × 10^{−2} | |

Hybrid | 3.10 × 10^{−2} | 1.41 × 10^{−2} |

**Table 2.**Median values performance measures of each model by response for the random split scenario.

Response | Technique | MSPE | MSE |
---|---|---|---|

Adjusted Body Weight | EN | 1.95 × 10^{−3} | 1.43 × 10^{−3} |

NN | 1.65 × 10^{−3} | 7.52 × 10^{−4} | |

Hybrid | 1.58 × 10^{−3} | 7.79 × 10^{−4} | |

Adjusted Gonad Weight | EN | 7.13 × 10^{−2} | 5.24 × 10^{−2} |

NN | 7.62 × 10^{−2} | 2.90 × 10^{−2} | |

Hybrid | 7.94 × 10^{−2} | 2.55 × 10^{−2} | |

Adjusted Liver Weight | EN | 2.67 × 10^{−2} | 2.21 × 10^{−2} |

NN | 2.53 × 10^{−2} | 1.35 × 10^{−2} | |

Hybrid | 2.33 × 10^{−2} | 1.36 × 10^{−2} |

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |

© 2024 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

McMillan, P.G.; Feng, Z.Z.; Arciszewski, T.J.; Proner, R.; Deeth, L.E.
Assessing the Interpretability–Performance Trade-Off of Artificial Neural Networks Using Sentinel Fish Health Data. *Environments* **2024**, *11*, 94.
https://doi.org/10.3390/environments11050094

**AMA Style**

McMillan PG, Feng ZZ, Arciszewski TJ, Proner R, Deeth LE.
Assessing the Interpretability–Performance Trade-Off of Artificial Neural Networks Using Sentinel Fish Health Data. *Environments*. 2024; 11(5):94.
https://doi.org/10.3390/environments11050094

**Chicago/Turabian Style**

McMillan, Patrick G., Zeny Z. Feng, Tim J. Arciszewski, Robert Proner, and Lorna E. Deeth.
2024. "Assessing the Interpretability–Performance Trade-Off of Artificial Neural Networks Using Sentinel Fish Health Data" *Environments* 11, no. 5: 94.
https://doi.org/10.3390/environments11050094