# Live Weight Prediction of Cattle Based on Deep Regression of RGB-D Images

^{1}

^{2}

^{3}

^{4}

^{5}

^{*}

## Abstract

**:**

## 1. Introduction

- Efficient preprocessing of RGB images and depth maps, as well as creating a color RGB projection and 2.5D depth map for subsequent live weight prediction based on image regression with deep learning, are proposed;
- A method for 3D augmentation of color projection and 2.5D depth map using rigid transformations in the form of three-dimensional rotations, scaling, and translation is proposed, which significantly increases the limited dataset and improves the efficiency of live weight prediction in the presence of variations in the posture and position of the animal;
- An efficient model for predicting live weight based on image regression with deep learning is proposed.

## 2. Related Works

## 3. Materials and Methods

#### 3.1. Datasets

#### 3.2. Preprocessing of Data

#### 3.3. Denoising of Data

#### 3.4. Removing the Background from a Point Cloud

#### 3.5. Pose Normalization and Lines of Symmetry Calculation

- Construction of an axis-aligned box bounding the animal in the point cloud. The algorithm is implemented in PCL, and it is equivalent to taking minimum/maximum values at each coordinate of the point cloud;
- Place the origin of the coordinate system at the center of gravity of the point cloud;
- Estimation of the initial symmetric plane $ax+by+cz=0$ using the PCA algorithm;
- The covariance matrix of the point cloud is calculated, and its eigenvalues and normalized eigenvectors are obtained;
- Calculation of the center of gravity $\left({g}_{x},{g}_{y},{g}_{z}\right)$ as follows:$${g}_{x}=\frac{1}{n}{\sum}_{i=1}^{n}{p}_{x}^{i},{g}_{y}=\frac{1}{n}{\sum}_{i=1}^{n}{p}_{y}^{i},{g}_{z}=\frac{1}{n}{\sum}_{i=1}^{n}{p}_{z}^{i}$$
- An exhaustive search of symmetry planes passing through the center of gravity $\left({g}_{x},{g}_{y},{g}_{z}\right)$ relative to the initial symmetry plane in order to find the optimal symmetry plane in terms of the modified Hausdorff metric:
- (a)
- splitting the point cloud into two smaller clouds ${C}_{R}$ and ${C}_{L}$ with the help of the initial symmetry plane $ax+by+cz=0$ as follows:$$\begin{array}{c}\left\{\begin{array}{l}p\in {C}_{R},a{p}_{x}+b{p}_{y}+c{p}_{z}\le 0,\hfill \\ p\in {C}_{L},a{p}_{x}+b{p}_{y}+c{p}_{z}0\hfill \end{array}\right\},\end{array}$$
- (b)
- construction of the mirror reflection $C{\prime}_{R}$ of the point cloud ${C}_{R}$ as follows:$$\begin{array}{c}{{p}^{\prime}}_{x}=\left(1-2{a}^{2}\right){p}_{x}-\left(2ab\right){p}_{y}-\left(2ac\right){p}_{z},\end{array}$$$$\begin{array}{c}{{p}^{\prime}}_{y}=\left(1-2{b}^{2}\right){p}_{y}-\left(2ab\right){p}_{x}-\left(2bc\right){p}_{z},\end{array}$$$$\begin{array}{c}{{p}^{\prime}}_{y}=\left(1-2{b}^{2}\right){p}_{y}-\left(2ab\right){p}_{x}-\left(2bc\right){p}_{z},\end{array}$$
- (c)
- calculation of the Hausdorff metric ${d}_{H}$ between $C{\prime}_{R}$ and ${C}_{L}$ using the average distance as follows:$${d}_{H}\left({C}_{R},{C}_{L}\right)=\mathrm{max}\left(\frac{1}{\left|{{C}^{\prime}}_{R}\right|}{\displaystyle \sum}_{x\in {{C}^{\prime}}_{R}}\underset{y\in {C}_{L}}{\mathrm{min}}d(x,y),\frac{1}{\left|{C}_{L}\right|}{\displaystyle \sum}_{y\in {C}_{L}}\underset{x\in {{C}^{\prime}}_{R}}{\mathrm{min}}d(x,y)\right),$$

#### 3.6. Calculation of Depth Map Projection (2.5D Depth Map)

#### 3.7. Color Projection

#### 3.8. Image Preprocessing for Neural Networks

#### 3.8.1. Image Resize

#### 3.8.2. Signal Range Normalization

#### 3.9. Deep Learning Models

#### 3.10. Data Augmentation

#### 3.11. Transfer Learning

#### 3.12. Performance Evaluation of Models

## 4. Results

## 5. Conclusions

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Conflicts of Interest

## References

- Pezzuolo, A.; Guarino, M.; Sartori, L.; Gonzalez, L.A.; Marinello, F. On-barn pig weight estimation based on body measurements by a Kinect v1 depth camera. Comput. Electron. Agric.
**2018**, 148, 29–36. [Google Scholar] [CrossRef] - Wang, Z.; Shadpour, S.; Chan, E.; Rotondo, V.; Wood, K.M.; Tulpan, D. ASAS-NANP SYMPOSIUM: Applications of machine learning for livestock body weight prediction from digital images. J. Anim. Sci.
**2021**, 99, skab022. [Google Scholar] [CrossRef] [PubMed] - Ruchay, A.; Kober, V.; Dorofeev, K.; Kolpakov, V.; Miroshnikov, S. Accurate body measurement of live cattle using three depth cameras and non-rigid 3-D shape recovery. Comput. Electron. Agric.
**2020**, 179, 105821. [Google Scholar] [CrossRef] - Kuzuhara, Y.; Kawamura, K.; Yoshitoshi, R.; Tamaki, T.; Sugai, S.; Ikegami, M.; Kurokawa, Y.; Obitsu, T.; Okita, M.; Sugino, T.; et al. A preliminarily study for predicting body weight and milk properties in lactating Holstein cows using a three-dimensional camera system. Comput. Electron. Agric.
**2015**, 111, 186–193. [Google Scholar] [CrossRef] - Sawanon, S.; Boonsaen, P.; Innurak, P. Body Measurements of Male Kamphaeng Saen Beef Cattle as Parameters for Estimation of Live Weight. Kasetsart J.-Nat. Sci.
**2011**, 45, 428–434. [Google Scholar] - Wangchuk, K.; Wangdi, J.; Mindu, M. Comparison and reliability of techniques to estimate live cattle body weight. J. Appl. Anim. Res.
**2017**, 46, 349–352. [Google Scholar] [CrossRef][Green Version] - Vanvanhossou, F.; Diogo, R.; Dossa, L. Estimation of live bodyweight from linear body measurements and body condition score in the West African Savannah Shorthorn Cattle in North-West Benin. Cogent Food Agric.
**2018**, 4, 1549767. [Google Scholar] [CrossRef] - Huma, Z.; Iqbal, F. Predicting the body weight of Balochi sheep using a machine learning approach. Turk. J. Vet. Anim. Sci.
**2019**, 43, 500–506. [Google Scholar] [CrossRef] - Hempstalk, K.; Mcparland, S.; Berry, D. Machine learning algorithms for the prediction of conception success to a given insemination in lactating dairy cows. J. Dairy Sci.
**2015**, 98, 5262–5273. [Google Scholar] [CrossRef][Green Version] - Miller, G.A.; Hyslop, J.J.; Barclay, D.; Edwards, A.; Thomson, W.; Duthie, C.A. Using 3D Imaging and Machine Learning to Predict Liveweight and Carcass Characteristics of Live Finishing Beef Cattle. Front. Sustain. Food Syst.
**2019**, 3, 30. [Google Scholar] [CrossRef][Green Version] - Milosevic, B.; Ciric, S.; Lalic, N.; Milanovic, V.; Savic, Z.; Omerovic, I.; Doskovic, V.; Djordjevic, S.; Andjusic, L. Machine learning application in growth and health prediction of broiler chickens. World’s Poult. Sci. J.
**2019**, 75, 401–410. [Google Scholar] [CrossRef] - Weber, V.; Weber, F.; Gomes, R.; Junior, A.; Menezes, G.; Belete, N.A.; Abreu, U.; Pistori, H. Prediction of Girolando cattle weight by means of body measurements extracted from images. Rev. Bras. De Zootec.
**2020**, 49. [Google Scholar] [CrossRef][Green Version] - Tasdemir, S.; Urkmez, A.; Inal, S. Determination of body measurements on the Holstein cows using digital image analysis and estimation of live weight with regression analysis. Comput. Electron. Agric.
**2011**, 76, 189–197. [Google Scholar] [CrossRef] - Song, X.; Bokkers, E.; van der Tol, P.; Koerkamp, P.G.; van Mourik, S. Automated body weight prediction of dairy cows using 3-dimensional vision. J. Dairy Sci.
**2018**, 101, 4448–4459. [Google Scholar] [CrossRef][Green Version] - Ranganathan, H.; Venkateswara, H.; Chakraborty, S.; Panchanathan, S. Deep active learning for image regression. In Deep Learning Applications; Springer: Berlin/Heidelberg, Germany, 2020; pp. 113–135. [Google Scholar]
- Bezsonov, O.; Lebediev, O.; Lebediev, V.; Megel, Y.; Prochukhan, D.; Rudenko, O. Breed Recognition and Estimation of Live Weight of Cattle Based on Methods of Machine Learning and Computer Vision. East.-Eur. J. Enterp. Technol.
**2021**, 6, 64–74. [Google Scholar] - Ruchay, A.; Dorofeev, K.; Kalschikov, V.; Kolpakov, V.; Dzhulamanov, K.; Guo, H. Live weight prediction of cattle using deep image regression. In Proceedings of the 2021 IEEE International Workshop on Metrology for Agriculture and Forestry (MetroAgriFor), Trento-Bolzano, Italy, 3–5 November 2021; pp. 32–36. [Google Scholar]
- Ruchay, A.; Kober, V. Impulsive Noise Removal from Color Images with Morphological Filtering. In Proceedings of the Analysis of Images, Social Networks and Texts, Moscow, Russia, 27–29 July 2017; Springer: Berlin/Heidelberg, Germany, 2018; pp. 280–291. [Google Scholar]
- Ruchay, A.; Kober, V.; Dorofeev, K.; Kolpakov, V.; Dzhulamanov, K.; Kalschikov, V.; Guo, H. Comparative analysis of machine learning algorithms for predicting live weight of Hereford cows. Comput. Electron. Agric.
**2022**, 195, 106837. [Google Scholar] [CrossRef] - Ruchay, A.; Gritsenko, S.; Ermolova, E.; Bochkarev, A.; Ermolov, S.; Guo, H.; Pezzuolo, A. A Comparative Study of Machine Learning Methods for Predicting Live Weight of Duroc, Landrace, and Yorkshire Pigs. Animals
**2022**, 12, 1152. [Google Scholar] [CrossRef] - OZKAYA, S. The prediction of live weight from body measurements on female Holstein calves by digital image analysis. J. Agric. Sci.
**2013**, 151, 570–576. [Google Scholar] [CrossRef] - Tasdemir, S.; Ozkan, I.A. ANN approach for estimation of cow weight depending on photogrammetric body dimensions. Int. J. Eng. Geosci.
**2018**, 4, 36–44. [Google Scholar] [CrossRef][Green Version] - Rudenko, O.; Megel, Y.; Bezsonov, O.; Rybalka, A. Cattle Breed Identification and Live Weight Evaluation on the Basis of Machine Learning and Computer Vision. Available online: https://openarchive.nure.ua/handle/document/18827 (accessed on 7 October 2022).
- Mortensen, A.K.; Lisouski, P.; Ahrendt, P. Weight prediction of broiler chickens using 3D computer vision. Comput. Electron. Agric.
**2016**, 123, 319–326. [Google Scholar] [CrossRef] - Nir, O.; Parmet, Y.; Werner, D.; Adin, G.; Halachmi, I. 3D Computer-vision system for automatically estimating heifer height and body mass. Biosyst. Eng.
**2018**, 173, 4–10. [Google Scholar] [CrossRef] - Na, M.H.; Cho, W.H.; Kim, S.K.; Na, I.S. Automatic Weight Prediction System for Korean Cattle Using Bayesian Ridge Algorithm on RGB-D Image. Electronics
**2022**, 11, 1663. [Google Scholar] [CrossRef] - Pezzuolo, A.; Milani, V.; Zhu, D.; Guo, H.; Guercini, S.; Marinello, F. On-Barn Pig Weight Estimation Based on Body Measurements by Structure-from-Motion (SfM). Sensors
**2018**, 18, 3603. [Google Scholar] [CrossRef][Green Version] - He, H.; Qiao, Y.; Li, X.; Chen, C.; Zhang, X. Automatic weight measurement of pigs based on 3D images and regression network. Comput. Electron. Agric.
**2021**, 187, 106299. [Google Scholar] [CrossRef] - Gjergji, M.; de Moraes Weber, V.; Otavio Campos Silva, L.; da Costa Gomes, R.; Luis Alves Campos de Araujo, T.; Pistori, H.; Alvarez, M. Deep Learning Techniques for Beef Cattle Body Weight Prediction. In Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, UK, 19–24 July 2020; pp. 1–8. [Google Scholar]
- Jun, K.; Kim, S.J.; Ji, H.W. Estimating pig weights from images without constraint on posture and illumination. Comput. Electron. Agric.
**2018**, 153, 169–176. [Google Scholar] [CrossRef] - McPhee, M.J.; Walmsley, B.J.; Cafe, L.M.; Oddy, V.H.; McPhee, M.J.; Alempijevic, A.; Skinner, B.; Littler, B.; Siddell, J.P.; Wilkins, J.F. Live animal assessments of rump fat and muscle score in Angus cows and steers using 3-dimensional imaging. J. Anim. Sci.
**2017**, 95, 1847–1857. [Google Scholar] [CrossRef] [PubMed] - Lu, J.; Guo, H.; Du, A.; Su, Y.; Ruchay, A.; Marinello, F.; Pezzuolo, A. 2-D/3-D fusion-based robust pose normalisation of 3-D livestock from multiple RGB-D cameras. Biosyst. Eng.
**2021**, in press. [Google Scholar] [CrossRef] - Ruchay, A.; Kolpakov, V.; Kosyan, D.; Rusakova, E.; Dorofeev, K.; Guo, H.; Ferrari, G.; Pezzuolo, A. Genome-Wide Associative Study of Phenotypic Parameters of the 3D Body Model of Aberdeen Angus Cattle with Multiple Depth Cameras. Animals
**2022**, 12, 2128. [Google Scholar] [CrossRef] [PubMed] - Ruchay, A. Available online: https://github.com/ruchaya/CowDatabase2 (accessed on 6 October 2022).
- Bochkovskiy, A.; Wang, C.Y.; Liao, H.Y.M. YOLOv4: Optimal Speed and Accuracy of Object Detection. arXiv
**2020**, arXiv:2004.10934. [Google Scholar] - Hu, Y.; Luo, X.; Gao, Z.; Du, A.; Guo, H.; Ruchay, A.; Marinello, F.; Pezzuolo, A. Curve Skeleton Extraction from Incomplete Point Clouds of Livestock and Its Application in Posture Evaluation. Agriculture
**2022**, 12, 998. [Google Scholar] [CrossRef] - Ruchay, A.; Dorofeev, K.; Kalschikov, V. A novel switching bilateral filtering algorithm for depth map. Comput. Opt.
**2019**, 43, 1001–1007. [Google Scholar] [CrossRef] - Rusu, R.B.; Cousins, S. 3D is here: Point cloud library (PCL). In Proceedings of the 2011 IEEE International Conference on Robotics and Automation, Shanghai, China, 9–13 May 2011; pp. 1–4. [Google Scholar]
- Tan, M.; Le, Q. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. In Proceedings of the Proceedings of the 36th International Conference on Machine Learning, Long Beach, CA, USA, 9–15 June 2019; Volume 97, pp. 6105–6114. [Google Scholar]

**Figure 3.**Areas of the body, thigh, and head of the animal are marked in red, blue, and yellow, respectively.

**Figure 5.**Point cloud with the animal, point cloud with the background, and the resultant cloud after removing the background.

**Table 1.**Live weight prediction results for cattle using the proposed MRGBDM, MRGB, and MDM models and the pre-trained EfficientNet (ENET) model on training and test datasets.

Input to CNN | Model | Training Data | Test Data | ||||
---|---|---|---|---|---|---|---|

MAE | MAPE | Accuracy | MAE | MAPE | Accuracy | ||

Raw RGB images and depth maps | MRGBDM | 37.9 | 9.1 | 90.9 | 40.1 | 9.6 | 90.4 |

MRGB | 46.9 | 11.1 | 88.9 | 50.3 | 11.9 | 88.1 | |

MDM | 40.5 | 9.5 | 90.5 | 43.5 | 10.2 | 89.8 | |

ENET | 41.1 | 9.8 | 90.2 | 43.6 | 10.4 | 89.6 | |

Color and depth map projections | MRGBDM | 34.2 | 8.1 | 91.9 | 35.5 | 8.4 | 91.6 |

MRGB | 42.5 | 10.1 | 88.9 | 45.6 | 10.8 | 89.2 | |

MDM | 37.6 | 8.9 | 91.1 | 39.7 | 9.4 | 90.6 | |

ENET | 38.9 | 9.2 | 90.8 | 41.8 | 9.9 | 90.1 |

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Ruchay, A.; Kober, V.; Dorofeev, K.; Kolpakov, V.; Gladkov, A.; Guo, H.
Live Weight Prediction of Cattle Based on Deep Regression of RGB-D Images. *Agriculture* **2022**, *12*, 1794.
https://doi.org/10.3390/agriculture12111794

**AMA Style**

Ruchay A, Kober V, Dorofeev K, Kolpakov V, Gladkov A, Guo H.
Live Weight Prediction of Cattle Based on Deep Regression of RGB-D Images. *Agriculture*. 2022; 12(11):1794.
https://doi.org/10.3390/agriculture12111794

**Chicago/Turabian Style**

Ruchay, Alexey, Vitaly Kober, Konstantin Dorofeev, Vladimir Kolpakov, Alexey Gladkov, and Hao Guo.
2022. "Live Weight Prediction of Cattle Based on Deep Regression of RGB-D Images" *Agriculture* 12, no. 11: 1794.
https://doi.org/10.3390/agriculture12111794