# A Track Geometry Measuring System Based on Multibody Kinematics, Inertial Sensors and Computer Vision

^{1}

^{2}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. Description of the TGMS

- Two video cameras.
- Two laser line-projectors.
- An IMU.
- A rotary encoder.

- It is capable to measure track alignment, vertical profile, cross-level, gauge, twist and rail-head profile using non-contact technology.
- It can be installed in in-service vehicles. It is compact and low cost. Provided that the equipment sees the rail heads when the vehicle is moving, it can be installed in any body of the vehicle: at the wheelsets level, above primary suspension (bogie frame) or above the secondary suspension (car body).

## 3. Kinematics of the Irregular Track and the Railway Vehicle

#### 3.1. Frames of Reference

- The inertial and global frame (GF) $<X,Y,Z>$. It is a frame fixed in space.
- The track frame (TF) $<{X}^{t},{Y}^{t},{Z}^{t}>$. It is not a single frame but a field defined for each value of the arc-length coordinate along the track s. The position ${\mathbf{R}}^{t}\left(s\right)$ and orientation matrix ${\mathbf{A}}^{t}\left(s\right)$ of the TF with respect to the GF are functions of an arc-length coordinate s along the center line of the design track (without irregularities). These functions are implemented computationally in a track preprocessor.
- The body frame (BF) $<{X}^{i},{Y}^{i},{Z}^{i}>$ of each body i. It is a frame rigidly attached to the body. In this document, the body i is the TGMS. The body frame of the TGMS is denoted as $<{X}^{tgms},{Y}^{tgms},{Z}^{tgms}>$
- The rail profile frames. Left rail-profile frame (LRP), $<{X}^{lrp},{Y}^{lrp},{Z}^{lrp}>$, and right-profile frame (RRP), $<{X}^{rrp},{Y}^{rrp},{Z}^{rrp}>$. These frames are not unique frames but fields defined for each value of the arc-length coordinate along the track s. These frames are rigidly attached to the rail-heads.

#### 3.2. Kinematics of the Design Track Centerline

- Horizontal curvature: ${\rho}_{h}$.
- Vertical curvature: ${\rho}_{v}$.
- Twist curvature: ${\rho}_{tw}$.
- Spatial-derivative of horizontal curvature: ${{\rho}_{h}}^{\prime}$.
- Vertical slope: ${\alpha}_{v}$.

#### 3.3. Kinematics of the Irregular Track

- Alignment: $\hspace{1em}\hspace{1em}\hspace{1em}{\xi}_{al}=({y}^{lir}+{y}^{rir})/2$
- Vertical profile: $\hspace{1em}\hspace{1em}{\xi}_{vp}=({z}^{lir}+{z}^{rir})/2$
- Gauge variation: $\hspace{1em}{\xi}_{gv}={y}^{lir}-{y}^{rir}$
- Cross level: $\hspace{1em}\hspace{1em}\hspace{1em}{\xi}_{cl}={z}^{lir}-{z}^{rir}$

#### 3.4. Kinematics of a Body Moving along the Track

## 4. Kinematics of the Computer Vision

**P**is $3\times 4$. Matrix ${\mathbf{M}}^{int}$ is $3\times 3$ and it is called matrix of intrinsic parameters of the camera, and matrix ${\mathbf{M}}^{ext}$ is $3\times 4$, it is called matrix of extrinsic parameters of the camera, and it is given by:

## 5. Detecting the Rail Cross-Section from a Camera Frame

## 6. Equations for Geometry Measurement

- (1)
- The calculation of the relative track irregularities (${\xi}_{gv}$ and ${\xi}_{cl}$), as shown in Equation (24), needs as an input the output of the computer vision ${\widehat{\mathbf{u}}}_{Olrp}^{tgms}\phantom{\rule{0.166667em}{0ex}}\phantom{\rule{0.166667em}{0ex}}\mathrm{and}\phantom{\rule{0.166667em}{0ex}}\phantom{\rule{0.166667em}{0ex}}{\widehat{\mathbf{u}}}_{Orrp}^{tgms}$ and the roll angle of the TGMS with respect to the track ${\phi}^{tgms}$.
- (2)
- The calculation of the absolute track irregularities (${\xi}_{al}$ and ${\xi}_{vp}$), as shown in Equation (27), needs, in addition, the relative trajectory ${\overline{\mathbf{r}}}^{tgms}$ of the TGMS with respect to the TF.

## 7. Measurement of TGMS to TF Relative Orientation

- The gyroscope provides three signals that are proportional to the components of the angular velocity vector in the sensor frame (the sensor frame is assumed to be parallel to the TGMS frame), as follows:$${\omega}^{imu}={\widehat{\omega}}_{abs}^{tgms}$$These components are non-linearly related to the coordinates that define the TGMS orientation and their time-derivatives, as shown later.
- The accelerometer signals are proportional to the components of the absolute acceleration in the local frame plus the absolute gravity vector field, as follows:$${\mathbf{a}}^{imu}={\widehat{\ddot{\mathbf{R}}}}^{tgms}+{\left({\mathbf{A}}^{tgms}\right)}^{T}{\left[\begin{array}{ccc}0& 0& g\end{array}\right]}^{T}$$This is the absolute acceleration in the sensor frame, plus the gravitational constant g, that is assumed to act in the absolute Z direction.
- The magnetometer signals are proportional to the components of the Earth’s magnetic field in the local frame. This information can be used to find the direction of the Earth’s magnetic north.

## 8. Odometry Algorithm

## 9. Measurement of TGMS to TF Relative Motion

- The accelerometer data ${\mathbf{a}}^{\mathit{imu}}$.
- The instantaneous forward velocity V and acceleration $\dot{V}$ of the vehicle. This is obtained from the encoder data.
- The position ${s}^{tgms}$ of the TGMS along the track. This is the output of the odometry algorithm explained in previous section. The position ${s}^{tgms}$ is used as an entry to the track preprocessor to get the track design cant angle ${\phi}^{t}$ and the curvatures ${\rho}_{tw},\phantom{\rule{0.166667em}{0ex}}\phantom{\rule{0.166667em}{0ex}}{\rho}_{v}\phantom{\rule{0.166667em}{0ex}}\mathrm{and}\phantom{\rule{0.166667em}{0ex}}\phantom{\rule{0.166667em}{0ex}}{\rho}_{tw}$.
- The relative orientation of the TGMS with respect to the TF that is calculated in Section 7.

- Using a digital signal processing approach for the integration that combines double-integration and high-pass filtering of the signal [3].
- Using a Kalman filter approach that adds to the system dynamic equations, Equation (46), a set of measurements. These measurements are virtual sensors that in practice always provide a zero value of the TGMS to TF relative motion. The assumed covariance of these measurements is the expected covariance of the track irregularities. This method has been successfully applied in [23] to eliminate the drift in the results while keeping a good accuracy.

## 10. Experimental Setup

## 11. Computer Implementation and Comparison of TGMS Measurement with Reference Irregularity

- Pre-process: Camera calibration module. It finds the cameras’ intrinsic and extrinsic parameters using pictures of a checkerboard pattern [19].
- Pre-process: Track pre-processor module. It is set to provide the design geometry of the track.
- Process: Computer vision module. It finds the position and orientation of the rail cross-sections using the recorded frames and the method described in Section 5.
- Process: Odometry module. As described in Section 8, it finds the position, velocity and acceleration of the TGMS every time instant.
- Process: TGMS orientation module. It finds the orientation of the TGMS using the method described in Section 7.
- Process: TGMS trajectory module. It finds the relative trajectory of the TGMS with respect to the TF using the method described in Section 9.
- Process: Irregularity module. It calculates the track irregularities with the method of Section 6.

## 12. Summary and Conclusions

## Author Contributions

## Funding

## Data Availability Statement

## Conflicts of Interest

## Abbreviations

TGMS | Track geometry measurement system. |

RTT | Rail track trolleys. |

TRV | Track recording vehicles. |

VMS | Versine measurement systems. |

IMS | Inertial measurement systems. |

$<X,Y,Z>$ | Global frame (GF). |

$<{X}^{t},{Y}^{t},{Z}^{t}>$ | Track frame (TF). |

$<{X}^{i},{Y}^{i},{Z}^{i}>$ | Body frame i (BF). |

$<{X}^{tgms},{Y}^{tgms},{Z}^{tgms}>$ | TGMS frame. |

$<{X}^{lrp},{Y}^{lrp},{Z}^{lrp}>$ | Left rail-profile frame (LRP). |

$<{X}^{rrp},{Y}^{rrp},{Z}^{rrp}>$ | Right rail-profile frame (RRP). |

$\overrightarrow{R}$ | Position vector with respect to the GF. |

${\overrightarrow{R}}^{t}$ | Position vector of the TF with respect to the GF. |

$\overrightarrow{r}$ | Position vector with respect to the TF. |

$\overrightarrow{u}$ | Position vector with respect to the BF, LRP or RRP. |

${\overrightarrow{u}}_{P}^{i}$ | Position vector of point P that belongs to body i. |

$\mathbf{v}$ | Components of a generic vector $\overrightarrow{v}$ in the GF. |

$\overline{\mathbf{v}}$ | Components of a generic vector $\overrightarrow{v}$ in the TF. |

$\widehat{\mathbf{v}}$ | Components of a generic vector $\overrightarrow{v}$ in the BF, LRP or RRP. |

${\mathbf{A}}^{t}$ | Rotation matrix of the TF with respect to the GF. |

${\mathbf{A}}^{i}$ | Rotation matrix of the BF of body i with respect to the GF. |

${\mathbf{A}}^{t,i}$ | Rotation matrix from the BF of body i to the TF. |

${\rho}_{h}$, ${\rho}_{v}$, ${\rho}_{tw}$ | Horizontal, vertical and twist curvatures of the track centerline. |

${{\rho}_{h}}^{\prime}$ | Spatial-derivative of horizontal curvature. |

${\alpha}_{v}$ | Vertical slope of the track centerline. |

${\psi}^{t}$, ${\theta}^{\phantom{\rule{0.166667em}{0ex}}t}$, ${\phi}^{\phantom{\rule{0.166667em}{0ex}}t}$ | Euler angles that describe the orientation of the TF with respect to |

the GF. | |

${\psi}^{i}$, ${\theta}^{\phantom{\rule{0.166667em}{0ex}}i}$, ${\phi}^{\phantom{\rule{0.166667em}{0ex}}i}$ | Euler angles that describe the orientation of the body i with respect to |

the TF. | |

${\psi}_{abs}^{\phantom{\rule{0.166667em}{0ex}}i}$, ${\theta}_{abs}^{\phantom{\rule{0.166667em}{0ex}}i}$, ${\phi}_{abs}^{\phantom{\rule{0.166667em}{0ex}}i}$ | Euler angles that describe the orientation of the body i with respect to |

the GF. | |

s | Arc-length coordinate. |

V, $\dot{V}$ | Forward velocity and acceleration. |

${\overrightarrow{\omega}}^{t}$, ${\overrightarrow{\alpha}}^{t}$ | Absolute angular velocity and acceleration vectors of the TF. |

${\overrightarrow{\omega}}^{i}$, ${\overrightarrow{\alpha}}^{i}$ | Absolute angular velocity and acceleration vectors of body i. |

${\overrightarrow{\omega}}^{t,i}$, ${\overrightarrow{\alpha}}^{t,i}$ | Angular velocity and acceleration vectors of body i with respect to |

the TF. | |

${\overrightarrow{r}}^{lir}$ = [ 0, ${y}^{lir}$, ${z}^{lir}$] | Irregularity vector of the left rail. |

${\overrightarrow{r}}^{rir}$ = [ 0, ${y}^{rir}$, ${z}^{rir}$] | Irregularity vector of the right rail. |

${\xi}_{al},{\xi}_{vp},{\xi}_{gv},{\xi}_{cl}$ | Alignment, vertical profile, gauge variation and cross level. |

## References

- Grassie, S.L. Measurement of railhead longitudinal profiles: A comparison of different techniques. Wear
**1996**, 191, 245–251. [Google Scholar] [CrossRef] - Aknin, P.; Chollet, H. A new approach for the modelling of track geometry recording vehicles and the deconvolution of versine measurements. Veh. Syst. Dyn.
**1999**, 33, 59–70. [Google Scholar] [CrossRef] - Lewis, R.B. Track-recording techniques used on British Rail. IEE Proc. B Electr. Power Appl.
**1984**, 131, 73–81. [Google Scholar] [CrossRef] - Escalona, J.L. Vertical Track Geometry Monitoring Using Inertial Sensors and Complementary Filters. In Proceedings of IDETC/CIE 2016; ASME: Charlotte, NC, USA, 2016. [Google Scholar]
- Weston, P.; Ling, C.S.; Roberts, C.; Goodman, C.J.; Li, P.; Goodall, R.M. Monitoring vertical track irregularity from in-service railway vehicles. Proc. Inst. Mech. Eng. Part F: J. Rail Rapid Transit
**2007**, 221, 75–88. [Google Scholar] [CrossRef] - Weston, P.; Ling, C.S.; Goodman, C.J.; Roberts, C.; Li, P.; Goodall, R.M. Monitoring lateral track irregularity from in-service railway vehicles. Proc. Inst. Mech. Eng. Part F: J. Rail Rapid Transit
**2007**, 221, 89–100. [Google Scholar] [CrossRef] - Weston, P.; Roberts, C.; Yeo, G.; Stewart, E. Perspectives on railway track geometry condition monitoring from in-service railway vehicles. Veh. Syst. Dyn.
**2015**, 53, 1063–1091. [Google Scholar] [CrossRef] - Seok Lee, J.; Choi, S.; Kim, S.S.; Park, C.; Guk Kim, Y. A Mixed Filtering Approach for Track Condition Monitoring Using Accelerometers on the Axle Box and Bogie. IEEE T. Instrum. Meas.
**2012**, 61, 749–758. [Google Scholar] - Tsai, H.C.; Wang, C.Y.; Huang, N. Fast Inspection and Identification Techniques for Track Irregularities Based on HHT Analysis. Adv. Adapt. Data Anal.
**2012**, 4. [Google Scholar] [CrossRef] - Tsai, H.C.; Wang, C.Y.; Huang, N.; Kuo, T.W.; Chieng, W.H. Railway track inspection based on the vibration response to a scheduled train and the Hilbert-Huang transform. Proc. Inst. Mech. Eng. Part F: J. Rail Rapid Transit
**2014**, 229. [Google Scholar] [CrossRef] - Tsunashima, H.; Naganuma, Y.; Kobayashi, T. Track geometry estimation from car-body vibration. Veh. Syst. Dyn.
**2014**, 52, 207–219. [Google Scholar] [CrossRef] - Tsunashima, H. Condition Monitoring of Railway Tracks from Car-Body Vibration Using a Machine Learning Technique. Appl. Sci.
**2019**, 9, 2734–2746. [Google Scholar] [CrossRef] [Green Version] - Tsunashima, H.; Hirose, R. Condition monitoring of railway track from car-body vibration using time-frequency analysis. Veh. Syst. Dyn.
**2020**, 1–18. [Google Scholar] [CrossRef] - Muñoz, S.; Aceituno, J.F.; Urda, P.; Escalona, J.L. Multibody model of railway vehicles with weakly coupled vertical and lateral dynamics. Mech. Syst. Signal Process.
**2019**, 115, 570–592. [Google Scholar] [CrossRef] - Escalona, J.L.; Aceituno, J.F. Multibody simulation of railway vehicles with contact lookup tables. Int. J. Mech. Sci.
**2019**, 155, 571–582. [Google Scholar] [CrossRef] - Escalona, J.L.; Aceituno, J.F.; Urda, P.; Balling, O. Railway multibody simulation with the knife-edge-equivalent wheel-rail constraint equations. Multibody Syst. Dyn.
**2020**, 48, 373–402. [Google Scholar] [CrossRef] - Escalona, J.L. A methodology for the measurement of track geometry based on computer vision and inertial sensors. arXiv
**2020**, arXiv:2008.03763v1. [Google Scholar] - Zhang, Z. A Flexible New Technique for Camera Calibration; Technical Report MSR-TR-98-71 for Microsoft Research; Microsoft Corporation: Redmond, WA, USA, 1998. [Google Scholar]
- Escalona, J.L. Kinematics of motion tracking using computer vision. arXiv
**2020**, arXiv:2008.00813v1. [Google Scholar] - Szelinski, R. Computer Vision. Algorithms and Applications; Springer: Berlin/Heidelberg, Germany, 2011. [Google Scholar]
- Madgwick, S.; Harrison, A.; Vaidyanathan, A. Estimation of IMU and MARG orientation using a gradient descent algorithm. In Proceedings of the IEEE International Conference on Rehabilitation Robotics, Zurich, Switzerland, 29 June–1 July 2011. [Google Scholar]
- Valenti, R.; Dryanovski, I.; Xiao, J. Keeping a good attitude: A quaternion-based orientation filter for IMUs and MARGs. Sensors
**2015**, 15, 19302–19330. [Google Scholar] [CrossRef] [PubMed] [Green Version] - Muñoz, S.; Ros, J.; Escalona, J.L. Estimation of lateral track irregularity through Kalman filtering techniques. arXiv
**2020**, arXiv:2003.05222. [Google Scholar] - Urda, P.; Muñoz, S.F.; Aceituno, J.; Escalona, J. Application and Experimental Validation of a Multibody Model with Weakly Coupled Lateral and Vertical Dynamics to a Scaled Railway Vehicle. Sensors
**2020**, 20, 3700. [Google Scholar] [CrossRef] [PubMed]

**Figure 1.**Kinematics of the Track Geometry Measuring System (TGMS) installed in a vehicle moving along the track.

**Figure 13.**Scaled track at the School of Engineering, University of Seville. (

**a**) Aerial view, (

**b**) detail of track supports.

**Figure 14.**Scale vehicle, (

**a**) global view of instrumented scale vehicle, (

**b**) detail showing video cameras and laser beam.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Escalona, J.L.; Urda, P.; Muñoz, S.
A Track Geometry Measuring System Based on Multibody Kinematics, Inertial Sensors and Computer Vision. *Sensors* **2021**, *21*, 683.
https://doi.org/10.3390/s21030683

**AMA Style**

Escalona JL, Urda P, Muñoz S.
A Track Geometry Measuring System Based on Multibody Kinematics, Inertial Sensors and Computer Vision. *Sensors*. 2021; 21(3):683.
https://doi.org/10.3390/s21030683

**Chicago/Turabian Style**

Escalona, José L., Pedro Urda, and Sergio Muñoz.
2021. "A Track Geometry Measuring System Based on Multibody Kinematics, Inertial Sensors and Computer Vision" *Sensors* 21, no. 3: 683.
https://doi.org/10.3390/s21030683