# Haptic Devices Based on Real-Time Dynamic Models of Multibody Systems

^{*}

## Abstract

**:**

## 1. Introduction

## 2. State-of-the-Art

#### 2.1. Multibody Formalisms

#### 2.2. Hardware Implementation

## 3. Multibody Formalism

#### 3.1. Modeling MBS Using Relative Coordinates

#### 3.2. Symbolic Multibody Model

- significantly speed up the computation of MBS models; this represents a very valuable asset for applications requiring real-time computing, such as those targeted here, and
- easily couple multibody models—being encapsulated in symbolic files—to other disciplines (such as optimization, control, electromechanical dimensioning) at software but also at hardware levels.

- the constraints and their resolution at position, velocity, and acceleration levels;
- the external forces and torques (interfaced with possible external user constitutive equations);
- the dynamics of the restored tree-like MBS; and
- the reduction to an ODE system and its resolution with respect to the generalized accelerations ${\ddot{u}}_{u}$.

## 4. Hardware Framework

#### 4.1. Specifications

#### 4.2. Human-in-the-Loop Haptics with ROBOTRAN

#### 4.3. ROS-ROBOTRAN Coupling

**ROSbotran**—has been created, that is able to publish/subscribe to the data of a ROS network. It contains the multibody model running in real-time, which allows to simulate the mechanical system and to forecast, analyze, and control its behavior. Inside this package, for our applications of Section 5, this node uses the multi-threads approach to parallelize the calculations.

**ROScom**—has simply a listener–receiver role, containing two nodes: Listener and Talker. Two topics are used to exchange kinematic and dynamic information between the ROSbotran and ROScom packages, see Figure 8. The communication is ensured through the SSH protocol and physically via an ethernet cable.

## 5. Applications

#### 5.1. Haptic Piano Key

#### 5.1.1. Previous Realizations

#### 5.1.2. Mechatronic Implementation

#### 5.1.3. Model Validation

^{®}action (Figure 13). This homemade binocular element contains four strain gauges glued to the sides, connected through a Wheatstone bridge. This setup allows to precisely measure the normal force between the whippen and the key pilot, without influencing the action dynamic behavior.

#### 5.1.4. Real-Time Sensors Validation

#### 5.1.5. Haptic Key Results

^{®}demonstrator for which these values have been physically modified.

^{®}(left) and the haptic prototype (right). The parameter is the height of the let-off button [43], see also Figure 9. In the legend, Normal means that the setting value is nominal and Low (resp. High) that the let-off button is lower (resp. higher) than its nominal height, by approximately 0.5 mm.

#### 5.2. Haptic Driving Simulator

#### 5.3. Other Implementations

## 6. Conclusions and Prospects

## Author Contributions

## Funding

## Institutional Review Board Statement

## Informed Consent Statement

## Data Availability Statement

## Acknowledgments

## Conflicts of Interest

## Abbreviations

CPU | Central Processing Unit |

DAE | Differential-Algebraic Equation |

DOF | Degree of Freedom |

FPGA | Field Programmable Gate Area |

GPU | Graphics Processing Unit |

HIL | Human-in-the-loop |

IMU | Inertial Measurement Unit |

MBS | Multibody Systems |

NTV | Narrow Tilting Vehicles |

ODE | Ordinary Differential Equation |

OS | Operating System |

RAM | Random Access Memory |

ROS | Robot Operation System |

SSH | Secure Shell |

YARP | Yet Another Robot Platform |

## References

- Lin, M.C.; Otaduy, M. (Eds.) Haptic Rendering: Foundations, Algorithms, and Applications, 1st ed.; A K Peters—CRC Press: Boca Raton, FL, USA, 2008. [Google Scholar] [CrossRef]
- Sreelakshmi, M.; Subash, T. Haptic Technology: A comprehensive review on its applications and future prospects. Mater. Today Proc.
**2017**, 4, 4182–4187. [Google Scholar] [CrossRef] - Wittenburg, J. Dynamics of Systems of Rigid Bodies; Teubner Verlag: Berlin, Germany, 1977. [Google Scholar] [CrossRef]
- Haug, E. Computer-Aided Kinematics and Dynamics of Mechanical Systems Volume-I: Basics Methods; Allyn and Bacon: Boston, MA, USA, 1989; Volume 1, ISBN 0-205-11669-8. [Google Scholar]
- Nikravesh, P. Computer-Aided Analysis of Mechanical Systems; Prentice-Hall Int.: Hoboken, NJ, USA, 1988; ISBN 0-13-162702-3. [Google Scholar]
- Geradin, M.; Cardona, A. Flexible Multibody Dynamics: A Finite Element Approach; Wiley–Blackwell: Hoboken, NJ, USA, 2001; ISBN 978-0471489900. [Google Scholar]
- Roberson, R.; Schwertassek, R. Dynamics of Multibody Systems; Springer: Berlin/Heidelberg, Germany, 1988. [Google Scholar] [CrossRef]
- Samin, J.; Fisette, P. Symbolic Generation of Multibody Systems; Springer: New York, NY, USA, 2003; ISBN 978-1402016295. [Google Scholar]
- Garcia de Jalon, J.; Bayo, E. Kinematic and Dynamic Simulation of Multibody Systems: The Realtime Challenge; Springer: New York, NY, USA, 2011; ISBN 978-1461276012. [Google Scholar]
- Docquier, N.; Poncelet, A.; Fisette, P. ROBOTRAN: A powerful symbolic gnerator of multibody models. Mech. Sci.
**2013**, 4, 199–219. [Google Scholar] [CrossRef] [Green Version] - Pucheta, M.; Cardona, A.; Preidikman, S.; Hecker, R.E. Multibody Mechatronic Systems (Papers from the MuSMe Conference in 2020); Mechanisms and Machine Science Series; Springer International Publishing: Berlin/Heidelberg, Germany, 2021; ISBN 978-3-030-60371-7. [Google Scholar]
- Docquier, N.; Lantsoght, O.; Dubois, F.; Brüls, O. Modelling and simulation of coupled multibody systems and granular media using the non-smooth contact dynamics approach. Multibody Syst. Dyn.
**2020**, 49, 181–202. [Google Scholar] [CrossRef] - Escalona, J.L.; Urda, P.; Muñoz, S. A Track Geometry Measuring System Based on Multibody Kinematics, Inertial Sensors and Computer Vision. Sensors
**2021**, 21, 683. [Google Scholar] [CrossRef] - Risaliti, E.; Tamarozzi, T.; Vermaut, M.; Cornelis, B.; Desmet, W. Multibody model based estimation of multiple loads and strain field on a vehicle suspension system. Mech. Syst. Signal Process.
**2019**, 123, 1–25. [Google Scholar] [CrossRef] - Cuadrado, J.; Michaud, F.; Lugrís, U.; Soto, M.P. Using Accelerometer Data to Tune the Parameters of an Extended Kalman Filter for Optical Motion Capture: Preliminary Application to Gait Analysis. Sensors
**2021**, 21, 427. [Google Scholar] [CrossRef] - Hashemi, S.; Friedrich, H.; Bobach, L.; Bartel, D. Validation of a thermal elastohydrodynamic multibody dynamics model of the slipper pad by friction force measurement in the axial piston pump. Tribol. Int.
**2017**, 115, 319–337. [Google Scholar] [CrossRef] - Zierath, J.; Rachholz, R.; Rosenow, S.E.; Bockhahn, R.; Schulze, A.; Woernle, C. Modal testing on wind turbines for validation of a flexible multibody model. In Proceedings of the ECCOMAS Thematic Conference Multibody Dynamics, Prague, Czech Republic, 19–22 June 2017. [Google Scholar]
- Torres-Moreno, J.; Blanco-Claraco, J.; Giménez-Fernández, A.; Sanjurjo, E.; Naya, M. Online Kinematic and Dynamic-State Estimation for Constrained Multibody Systems Based on IMUs. Sensors
**2016**, 16, 333. [Google Scholar] [CrossRef] [Green Version] - Khadim, Q.; Kaikko, E.P.; Puolatie, E.; Mikkola, A. Targeting the user experience in the development of mobile machinery using real-time multibody simulation. Adv. Mech. Eng.
**2020**, 12, 168781402092317. [Google Scholar] [CrossRef] - Rodríguez, A.J.; Pastorino, R.; Carro-Lagoa, Á.; Janssens, K.; Naya, M.Á. Hardware acceleration of multibody simulations for real-time embedded applications. Multibody Syst. Dyn.
**2020**. [Google Scholar] [CrossRef] - Pastorino, R.; Cosco, F.; Naets, F.; Desmet, W.; Cuadrado, J. Hard real-time multibody simulations using ARM-based embedded systems. Multibody Syst. Dyn.
**2016**, 37, 127–143. [Google Scholar] [CrossRef] - Ros, J.; Plaza, A.; Iriarte, X.; Pintor, J.M. Symbolic multibody methods for real-time simulation of railway vehicles. Multibody Syst. Dyn.
**2017**, 1. [Google Scholar] [CrossRef] [Green Version] - Rivera, Z.B.; Simone, M.C.D.; Guida, D. Unmanned Ground Vehicle Modelling in Gazebo/ROS-Based Environments. Machines
**2019**, 7, 42. [Google Scholar] [CrossRef] [Green Version] - Munawar, A.; Wang, Y.; Gondokaryono, R.; Fischer, G.S. A Real-Time Dynamic Simulator and an Associated Front-End Representation Format for Simulating Complex Robots and Environments. In Proceedings of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China, 3–8 November 2019. [Google Scholar] [CrossRef]
- Habra, T.; Dallali, H.; Cardellino, A.; Natale, L.; Tsagarakis, N.; Fisette, P.; Ronsse, R. Robotran-YARP Interface: A Framework for Real-Time Controller Developments Based on Multibody Dynamics Simulations. In Computational Methods in Applied Sciences; Springer International Publishing: Berlin/Heidelberg, Germany, 2016; pp. 147–164. [Google Scholar] [CrossRef]
- Xia, P. New advances for haptic rendering: State of the art. Vis. Comput.
**2018**, 34, 271–287. [Google Scholar] [CrossRef] - Paris, J.N.; Archut, J.L.; Hüsing, M.; Corves, B. Haptic simulation and synthesis of mechanisms. Mech. Mach. Theory
**2020**, 144, 103674. [Google Scholar] [CrossRef] - Gillespie, B.; Cutkosky, M. Interactive Dynamics with Haptic Display. In Proceedings of the 2nd Annual Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, ASME/WAM, New Orleans, LA, USA, November 1993; p. 55-1. Available online: http://www-personal.umich.edu/~brentg/Web/Conference/asme93.pdf (accessed on 12 July 2021).
- Dialynas, G.; Happee, R.; Schwab, A.L. Design and hardware selection for a bicycle simulator. Mech. Sci.
**2019**, 10, 1–10. [Google Scholar] [CrossRef] - Timmermans, S.; Dehez, B.; Fisette, P. Multibody-Based Piano Action: Validation of a Haptic Key. Machines
**2020**, 8, 76. [Google Scholar] [CrossRef] - Angeli, A.; Desmet, W.; Naets, F. Deep learning for model order reduction of multibody systems to minimal coordinates. Comput. Methods Appl. Mech. Eng.
**2021**, 373. [Google Scholar] [CrossRef] - Perrelli, M.; Cosco, F.; Carbone, G.; Lenzo, B.; Mundo, D. On the Benefits of Using Object-Oriented Programming for the Objective Evaluation of Vehicle Dynamic Performance in Concurrent Simulations. Machines
**2021**, 9, 41. [Google Scholar] [CrossRef] - Jahnke, M.D.; Cosco, F.; Novickis, R.; Rastelli, J.P.; Gomez-Garay, V. Efficient Neural Network Implementations on Parallel Embedded Platforms Applied to Real-Time Torque-Vectoring Optimization Using Predictions for Multi-Motor Electric Vehicles. Electronics
**2019**, 8, 250. [Google Scholar] [CrossRef] [Green Version] - Hayward, V. A Brief Overview of the Human Somatosensory System. Music. Haptics
**2018**. [Google Scholar] [CrossRef] [Green Version] - Verrillo, R.T. Vibration sensation in humans. Music. Percept. Interdiscip. J.
**1992**, 9, 281–302. [Google Scholar] [CrossRef] - Papetti, S.; Järveläinen, H.; Giordano, B.L.; Schiesser, S.; Fröhlich, M. Vibrotactile sensitivity in active touch: Effect of pressing force. IEEE Trans. Haptics
**2017**, 10, 113–122. [Google Scholar] [CrossRef] - Barrea, A.; Delhaye, B.P.; Lefèvre, P.; Thonnard, J.L. Perception of partial slips under tangential loading of the fingertip. Sci. Rep.
**2018**, 8. [Google Scholar] [CrossRef] [Green Version] - MacLean, K.E. Haptic interaction design for everyday interfaces. Rev. Hum. Factors Ergon.
**2008**, 4, 149–194. [Google Scholar] [CrossRef] - O’Modhrain, S.; Gillespie, R.B. Once More, with Feeling: Revisiting the Role of Touch in Performer-Instrument Interaction. In Musical Haptics; Springer: Berlin/Heidelberg, Germany, 2018; pp. 11–28. [Google Scholar] [CrossRef]
- Hayward, V.; Astley, O.R.; Cruz-Hernandez, M.; Grant, D.; Robles-De-La-Torre, G. Haptic interfaces and devices. Sens. Rev.
**2004**, 24, 16–29. [Google Scholar] [CrossRef] - Hayward, V.; MacLean, K.E. Do it yourself haptics: Part I. IEEE Robot. Autom. Mag.
**2007**, 14. [Google Scholar] [CrossRef] - Bokiau, B.; Ceulemans, A.E.; Fisette, P. Historical and dynamical study of piano actions: A multibody modelling approach. J. Cult. Herit.
**2017**, 27, S120–S130. [Google Scholar] [CrossRef] - Timmermans, S.; Ceulemans, A.E.; Fisette, P. Upright and grand piano actions dynamic performances assessments using a multibody approach. Mech. Mach. Theory
**2021**, 160, 104296. [Google Scholar] [CrossRef] - Oboe, R.; De Poli, G. Multi-instrument virtual keyboard—The MIKEY project. In Proceedings of the Conference on New Instruments for Musical Expres, Dublin, Ireland, 24–26 May 2002. [Google Scholar]
- Lozada, J.; Hafez, M.; Boutillon, X. A novel haptic interface for musical keyboards. In Proceedings of the 2007 IEEE/ASME International Conference on Advanced intelligent mechatronics, Zurich, Switzerland, 4–7 September 2007; pp. 1–6. [Google Scholar]
- Horváth, P. Towards to Haptic Keyboard: Modeling the Piano Action. In Mechatronics 2013; Springer: Berlin/Heidelberg, Germany, 2014; pp. 49–55. [Google Scholar]
- Leonard, J.; Cadoz, C. Physical Modelling Concepts for a Collection of Multisensory Virtual Musical Instruments. In Proceedings of the New Interfaces for Musical Expression 2015, Baton Rouge, LA, USA, 31 May–3 June 2015; Available online: https://hal.archives-ouvertes.fr/hal-01262132 (accessed on 12 July 2021).
- Adamou, D.; Chin, C.; Rovelli, D.; Szafián, M.; Wood, M.G.; Yanchev, B.; Bailey, N.; Muir, D. Analysis and Reproduction of Keyboard Instrument Touch. In Proceedings of the 8th International Scientific Meeting for the Study of Sound and Musical Instruments–Organological Congress, Belmonte, Portugal, 20–22 September 2019; pp. 20–22. [Google Scholar]
- Persson, J.; Blanc, C.; Nguyen, V.; Perriard, Y. Sensorless position estimation of linear voice-coil transducers. In Proceedings of the Conference Record of the 2001 IEEE Industry Applications Conference 36th IAS Annual Meeting (Cat. No.01CH37248), Chicago, IL, USA, 30 September–4 October 2001; Volume 1, pp. 70–74. [Google Scholar] [CrossRef]
- Dülk, I.; Kovácsházy, T. A sensorless method for detecting spool position in solenoid actuators. Carpathian J. Electron. Comput. Eng.
**2013**, 6, 36. [Google Scholar] - Savioz, G.; Perriard, Y. Towards self-sensed drives in linear haptic systems. In Proceedings of the 2009 International Conference on Electrical Machines and Systems, Tokyo, Japan, 15–18 November 2009; pp. 1–5. [Google Scholar] [CrossRef]
- Timmermans, S.; Desclee, Q.; Paillot, G.; Fisette, P.; Dehez, B. Application and Validation of a Linear Electromagnetic Actuator within a Haptic Piano Key. In Proceedings of the 12th International Symposium on Linear Drives for Industry Applications (LDIA), Neuchatel, Switzerland, 1–3 July 2019. [Google Scholar] [CrossRef]
- Miedema, W. Active Haptic Feedback within a Musical Keyboard. Master’s Thesis, University of Twente, Enschede, The Netherlands, 2016. [Google Scholar]
- Wynne, R.A.; Beanland, V.; Salmon, P.M. Systematic review of driving simulator validation studies. Saf. Sci.
**2019**, 117, 138–151. [Google Scholar] [CrossRef] - Docquier, Q. Dynamic Analysis and Control of Narrow Track Vehicles via a Multibody Modeling Approach. Ph.D. Thesis, UCLouvain-Université Catholique de Louvain, Ottignies-Louvain-la-Neuve, Belgium, 2020. [Google Scholar]

**Figure 1.**Multibody applications range from small piano action, through road and railway vehicles, to huge port cranes (UCLouvain).

**Figure 7.**HIL haptic principle with ROBOTRAN. The light blue box, in this case, corresponds to the Direct Dynamics of Figure 4.

**Figure 9.**Main components of the grand piano action for the MBS model: mobile bodies (resp. other elements) are indicated by letters (resp. numbers). Circular arrows represent the DOFs (adapted from [43]).

**Figure 10.**Piano action multibody model force feedback principle, adapted from the work in [30].

**Figure 11.**Mechatronic design of one key haptic feedback device, enhanced from the work in [52].

**Figure 23.**Piano haptic key: measurements of the force felt by the pianist ${\mathrm{F}}_{\mathrm{haptic}}$ through an external linear actuator for the Renner

^{®}demonstrator (

**up**) and the haptic prototype (

**bottom**) [30].

**Figure 24.**Haptic piano key force validation: prototype comparison with the Renner

^{®}demonstrator (results confirming those obtained in [30]).

**Figure 25.**Haptic piano key force validation: variation of the button height, physically on the Renner

^{®}demonstrator (

**left**) and virtually on the haptic key prototype (

**right**).

**Figure 26.**Haptic piano key force validation: variation of the hammer mass, physically on the Renner

^{®}demonstrator (

**left**) and virtually on the haptic key prototype (

**right**).

**Figure 27.**Haptic steering wheel: experimental setup with its real-time visualization and its corresponding block diagram below.

**Figure 31.**NTV demonstrator and its two main sensors: a rotating potentiometer and a binocular strain gauge for torque measurement.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |

© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Docquier, N.; Timmermans, S.; Fisette, P.
Haptic Devices Based on Real-Time Dynamic Models of Multibody Systems. *Sensors* **2021**, *21*, 4794.
https://doi.org/10.3390/s21144794

**AMA Style**

Docquier N, Timmermans S, Fisette P.
Haptic Devices Based on Real-Time Dynamic Models of Multibody Systems. *Sensors*. 2021; 21(14):4794.
https://doi.org/10.3390/s21144794

**Chicago/Turabian Style**

Docquier, Nicolas, Sébastien Timmermans, and Paul Fisette.
2021. "Haptic Devices Based on Real-Time Dynamic Models of Multibody Systems" *Sensors* 21, no. 14: 4794.
https://doi.org/10.3390/s21144794