Automated and Controlled System for Analysis of Residual Limbs Thermograms of Transtibial Amputees

Abstract

This work describes the development of a controlled cabin for capturing and analyzing thermal images. The motivation of such a device is to aid in the thermal image acquisition process within a confined space. The thermograms generated provide helpful information for analyzing the residual human limb in subjects with transtibial amputation. Such a study proposes a non-intrusive method to study the thermal activity on the amputee residual limb and seek a correlation to the quality of the socket. The proposed cabin ensures the repeatability of the thermograms acquisition process and provides an isolated workspace, thus improving the quality of the samples. The methodology consists of the design of the mechanical elements and parts of the system on computer-aided design software, the electronic instrumentation, a graphic user interface, and the control algorithm based on a barrier Lyapunov function to solve the trajectory tracking for the camera movements, and numerical simulations to illustrate the functionality and the manufacture of a prototype. The results obtained by implementing the control design on the automated cabin reveal that the thermal image acquisition process is completed following the desired trajectory with a mean squared tracking error of 0.0052. In addition, an example of the thermal images of two subjects and the results processing this class of pictures using the designed interface is shown.

1. Introduction

The current design and development methodology implemented by prosthetists to create prosthetic sockets is inconsistent regarding the quality of the socket produced, as a result of such methodology being a largely handmade procedure [1,2,3]. It has been noted that poor socket fit has a negative effect on the quality of life for the patient; despite its importance, proper socket fit can be troublesome to attain [4]. The study conducted by Hagberg and Branemark found that the most frequent hassle related to the use of prostheses and the life quality of the user were the excessive heat and perspiration, as reported by 97 trans-femoral amputee subjects in a survey [5].

Other works on prosthetic sockets research techniques to improve or assess quality for the socket and, therefore, improve the patient’s daily life [3,6,7]. For example, Peery et al. researched to develop a device for manufacturing new prostheses. These novel prostheses aim to appropriately distribute the different thermal loads arising from daily life activities by implementing a three-dimensional (3D) thermal model of a transtibial residual limb and prosthesis. The results show that high temperatures zones are near muscle tissue, and the proposed representation can be applied to design prosthetic systems focused on adjusting the thermal conditions in the prosthesis [8].

Aguila et al. proposed a socket interface pressure system to make a comparison of the different pressure encounters in six regions of interest (ROIs) during the gait cycle between two types of prosthetic sockets for a transtibial amputee subject and found that uneven distribution of pressure between the residual limb and the prosthetic socket can be associated with the perception of the comfort of the amputee [9].

It is essential to highlight that thermography is an imaging technique used to determine the temperature and temperature variations of an object of interest [10], which, in this case, is the residual human limb. Such technology is widely employed in both commercial and industrial environments. In the medical field, thermography is used in different analysis, such as diagnosis of brown adipose tissue, diagnosis for diabetic foot, breast cancer, and analysis method for knee osteoarthritis screening, among others [11,12]. The thermographic analysis is a non-invasive, low-cost method to assess skin temperature of great importance in medicine.

Some authors have proposed a method using thermography to analyze how the compression and friction rub caused by the socket on the lower residual limb affects the temperature of certain ROI after walk activity. The main goal is to detect zones of discomfort by analyzing thermal images of the lower residual limb to aid in the design process of a personalized socket [13,14].

The aforementioned studies serve as inspiration and starting point for this particular work. In such studies, gathering the thermograms is a nonautomated process, sometimes conducted without a specific controlled environment, where the thermal camera is moved from one point to the next, guided by the marks previously placed on the floor. In this process, the environmental noise represents a problem. Other objects and changes of illumination are caught during the process of the thermogram snapshot, decreasing the quality of the results and increasing the time of image processing.

Based on those mentioned above, this work proposes the automation and control of a system or cabin to capture and analyze the thermograms. The study aims to develop a functional and manufacturable system to improve image processing results for thermographic analysis on the residual human limb for transtibial amputee subjects. With such development is expected a reduction of thermal and luminance noise from the environment and the definition of a standardized method to obtain the pictures within a fixed period. In addition, this system contributes to obtaining more precise temperature gradients on the ROIs, which helps evaluate the prosthetic socket adjustment.

Other devices with similar mechanism are used to generate 3D models using photogrammetry, a method used in various industries such as aeronautics, geology, meteorology, entertainment, and many others, such as the medical field 3D surface imaging and data acquisition [15,16]. Devices used in photogrammetry consist of a camera rotating around an object to generate a collection of 2D images from an object later processed by software to generate the 3D model.

The proposed system for thermography is not limited to it; for example, with a minor modification in the algorithm, the system can be used in photogrammetry. The main difference is that the current thermography protocol needs to capture an image every 90 degrees with only four thermal images. In addition, the application requires an isolated workspace to avoid thermal noise in the thermograms.

The main contributions of the proposed work are:

• The design of a portable structure that allows the analysis of residual human limb of transtibial amputee subjects with an adequate workspace by thermal imaging technology.
• A graphic user interface (GUI) for the automated acquisition of thermal images and the selection of the operation mode of the device.
• The control algorithm design considering partial state constraints. The design uses a barrier Lyapunov function to ensure the tracking trajectory of the mobile camera platform for each operation mode. As some model elements are considered uncertain, the estimation of the parameters is included in the analysis. Moreover, the use of Nussbaum functions helps with the uncertain control gains.
• The electronic instrumentation, including sensors, power devices, microcontrollers, actuators, and all the electronic components selected for the system automation.
• A virtual prototype based on the CAD model and the model for the actuator to test the functionality of the complete proposed design with the proposed control algorithm.

Based on the contributions mentioned above, the novelties are the design of the proposed system because, to the author’s knowledge, there are no automated devices for the same application, including the anthropometric measures of transtibial amputees and the user interface designed specifically for thermal analysis. On the other hand, the proposed controller does not require the exact parameters of the model and can be used in multiple applications where restrictions in the trajectory tracking error are required.

The outline of this manuscript is presented as follows: Section 2 gives the problem formulation encompassed for the design of the automated cabin; in this section are described the system requirements. Section 3 contains a detailed description of the methodology, including the computed assisted design, the assembly, the instrumentation, and the user interface to regulate the device. Furthermore, the control algorithm to ensure the tracking of a predefined trajectory for the camera mobile platform and the virtual prototype used for numerical simulations to test the tracking and functionality are also explained. Section 4 shows the results of each stage proposed in the methodology. Finally, Section 5 gives some final remarks and perspectives for future research with the proposed design.

2. Problem Statement

The current process for capturing thermal images to analyze the skin temperature in transtibial residual limbs is nonautomated and mostly manual, and the workspace is not isolated. To the authors’ knowledge, there are not commercial devices for the automation of this specific task. The development of an automated cabin to capture and analyze the thermal images is proposed to improve the actual process. For the development of such a device, the following points were considered:

• The first step is to set the requirements for developing the system, including hardware, software, system, and design criteria.
• The second step is to validate the requirements, then, the CAD design for the structure and mechanisms.
• Parallel to the CAD, the electronic instrumentation should be considered.
• Another important issue is to obtain the mathematical representation or to validate that some general model can represent the robotic system to design and implement an algorithm to solve the trajectory tracking of the thermal camera, i.e., generally, robotic systems can be represented by ordinary differential equations (ODEs) using the Euler–Lagrange approach [17]. Such general equation can be described as follows,

  $$F_1\left(z\right)\ddot{z}+F_2(z,\dot{z})\dot{z}+F_3\left(z\right)+F_4(z,\dot{z},t)=Q,$$

  (1)

  where $z\in {\mathbb{R}}^n$, $\dot{z}\in {\mathbb{R}}^n$, and $\ddot{z}\in {\mathbb{R}}^n$ represent the position, velocity, and acceleration for the n degrees of freedom, respectively, $F_1:{\mathbb{R}}^n\to {\mathbb{R}}^{n\times n}$ is the inertia matrix, $F_2:{\mathbb{R}}^n\times {\mathbb{R}}^n\to {\mathbb{R}}^{n\times n}$ is the Coriolis matrix, the viscous friction vector is denoted with $F_3:{\mathbb{R}}^n\to {\mathbb{R}}^n$, $F_4:{\mathbb{R}}^n\times {\mathbb{R}}^n\times {\mathbb{R}}^{+}\to {\mathbb{R}}^n$ represents bounded perturbations and model uncertainties that might affect the system, and $Q\in {\mathbb{R}}^n$ represents the generalized external forces. Once the mathematical representation is validated, the following issue is the design of the control algorithm.
• The input for (1) should be designed in such a way that the angular position of the camera follows a desired trajectory; this can be achievd with an input Q ensuring $\underset{t\to \infty }{lim}\parallel \Delta \left(t\right)\parallel \to 0$, where $\Delta =z-z^{\ast }$ is the tracking error and $z^{\ast }:{\mathbb{R}}_{+}\to {\mathbb{R}}^n$ describes the desired reference for the states z.
• The path for the mobile platform should be smooth and adjusted with the user selection of steps.
• Once the capture of the image in all the steps is completed, the processing to obtain the analysis of the thermograms for the residual limb is required.
• Finally, the integration of all the steps and some class of validation are required.

The following subsection describes the first step in the development of the system. This step defines a list of critical requirements that must be satisfied by the final design. This practice helps define, understand, and visualize the overall scope of the development process.

System Requirements

To set the requirements for the cabin, first, we review the actual process for the capture. Figure 1a depicts the markers on the floor to fix a tripod with the thermal camera, take the picture, and then change it to another position.

This task is time-consuming and depends on the user collocation of the camera, which affects the thermograms. Figure 1b shows the manual thermal snapshot of the residual limb of an amputee. In the picture, the background noise can be noticed.

The actual thermal image processing and analysis of the residual limb are part of a set of tests performed to obtain a comfort factor and evaluate the quality of sockets for transtibial prostheses. The selection of participants for the trials has an inclusion criterion to have a homogeneous population. The selection and all the tests, including thermography, are part of a clinical trial approved by an ethics committee (PR2021-01). Two male subjects were the participants to test the image processing and analysis proposed for the automated system; Figure 2 shows one of the subjects in the manual process for thermography.

Based on the actual issues for the manual process, the following requirements were established.

(1) General system requirements:(a) process time, the translation of the camera from start to end and the capture time of all samples is not greater than two minutes; (b) positions, the camera takes a clean capture every 90 degrees at least and has the possibility to take more steps if necessary; (c) control, the system has a microcontroller to manage the camera trajectory and the image capture; (d) isolation, the cabin reduces noise from the environment.

(2) Design requirements:(a) exterior, the device has an outer surface that prevents light from getting inside without increasing the inner temperature; (b) portability, the system can be deployed with ease in a time of five minutes approximately by two adults, and in the same way it can be undeployed; (c) radial distance, a constant radial distance of 45 cm through the entire process to avoid loss of focus; (d) interface, the cabin has an orifice in the middle of the upper part where the residual human limb is placed; (e) height, the height of the camera can be adjusted to compensate height variance due to the seat or the proportions of the participant.

(3) Hardware requirements:(a) light, the system has a light that indicates that the process of capture is in progress; (b) power button, the system has a button that energizes the device; (c) power source, the system has a power source that supplies the necessary voltage in direct current for sensors, actuators, and microcontroller; (d) actuator, the cabin has a DC motor to move the mobile platform with the camera through a rail.

(4) Software requirements:(a) camera movement, camera movement in the mobile platform is smooth, avoiding balance loss and sudden movements; (b) velocity, the platform of the camera moves with a smooth velocity, (c) image capture, image capture is performed every step of the moving platform: for the actual process at least every 90 degrees, but if the number of steps changes, the automated capture in each step is required; (d) image processing and analysis, the processing of the thermal pictures to obtain the average skin temperature of the subjects and the standard deviation; (e) interface, the system has an interface to adjust the movement and position of the camera, making desirable the automated and manual modes for movement and capture, (f) programming language, the software for the functionality of the device does not need the execution of additional software.

After defining the system requirements, it is much easier to visualize a sketch design of the cabin. In the following section, the mechanical design of the automated cabin is described.

3. Methodology

This section describes the proposed methodology for the development of the system. Figure 3 shows a diagram with the main steps.

First, after the system requirements are set, the structural and mechanical design to achieve the camera’s movement is proposed. Then the instrumentation and design of a control algorithm are solved, including the proposition of a smooth function for the trajectory.

The software implementation is another stage of the project. The designed software includes the development of the user interface for the functionality of the prototype. This interface coordinates the movements with the proposed control design, the capture of the image in each step, and then the image processing for the thermogram analysis. In the last part, we include numerical simulations in a virtual system to verify the performance of the system with the designed trajectories and controller.

3.1. Mechanical Design

Essential features are defined following the points stated before. For example, the requirement specifying the radial distance from the center of the cabin to the camera limits the size of the rail, and the overall structure is selected based on this. Another requirement specifies that the cabin must have an outer surface to prevent thermal and luminescence noise from the environment. Thus, the need for a type of structure capable of providing support to such surface while considering the portability resulted in the selection of a type of deployable pole.

In this stage, all the mechanisms and the structure for the device were designed. We divide the design into two main elements. The first element is the structure with the rail for the circular movement and the second element is the mobile platform supporting the thermal camera. Many commercial camera rails are straight and use a timing belt since they have two ends. To solve this in the design, we consider a circular rail. The platform’s design was dependent on the design of the rail and the selection of the actuators for the movement.

The design considered all the materials and the instrumentation with all the electronic elements. The design was generated using Solidworks${}^{\text{®}}$. Each part is made individually and included in an assembly model of the prototype. The assembly drawing includes identifiers for each part and depicts the assembly process. Along with the mechanical design, selecting electronic components is another essential part of the methodology. This is described in the following subsection.

3.2. Electronic Instrumentation

The instrumentation stage sets the appropriate conditions for implementing the control system, meaning that the correct use of sensors, actuators, and power requirements must be considered. Figure 4 shows a diagram that depicts the functionality of the proposed electronic elements for the automated system.

The main electronic components for the system functionality include the power supply, an encoder to measure position and velocity, an optocoupler to isolate the digital and power stages, and an H-bridge to drive the actuator, which was selected as a DC motor and the microcontroller. The communication protocol between the microcontroller and the computer allows controlling the automated device using a graphical user interface (GUI). With the GUI, the user can capture the thermal images automatically or by manual control of the camera and synchronize the snapshot with the movement of the mobile platform. A brief description of the user interface for the PC is described in the following subsection.

3.3. Software Implementation

To control the thermal image capture process, a GUI was developed with the help of the Matlab${}^{\text{®}}$ Appdesigner toolbox. This GUI makes the process user-friendly. First, the user sets the information of the amputee participant and then selects a working directory. After that, the user can choose between performing the process manually or automatically, and after the process is completed, the user has the option to perform another set of captures. This GUI is portable and can be installed in a personal computer without the Matlab${}^{\text{®}}$ software. Figure 5 depicts the process considered for the design of the interface.

Once the user has captured and saved all the desired images, the following step is the image processing and analysis.

Thermographic Images Analysis

For the automated system we consider the following analysis performed on the thermal images:

• Average temperature computation of the stump considering each view.
• Calculation of the standard deviation of the stump per view.

In order to test the designed software and to avoid background useless information, ten thermal captures for each participant with a transtibial amputation are taken using the cabin. An example of one of these photos is shown in Figure 1b, which can be compared with Figure 1b of a thermographic image without the use of the cabin.

The thermographic camera used during this study was an FLIR C3 thermal camera with WIFI. This camera requires a minimum focus distance of 0.15 m, has a thermal sensitivity of less than 0.10 °C, and a measurement accuracy of +/−2 °C or +/−2% at 25 °C. It also has a set of parameters that can be tuned for specific tasks, for instance, in this study we set the emissivity value to 0.98, which is the accepted emissivity value for human skin found in literature. In addition, a target distance was set to 0.45 m, which is the distance between the camera and the center of the cabin. Proper camera calibration needs to be performed by an authorized center; however, before the capture of the thermal images, the FLIR C3 temperature range was set from 20 to 40 °C, aiming to detect the temperature of the stump within this range in such a way that color map variations can be used for a qualitative analysis. This temperature range was fixed by setting an environment temperature of 20 °C and placing a glass of water at 40 °C, 45 cm away from the camera. During the image processing stage, no reflected temperature compensation nor measurement errors were eliminated resulting from non-axial measurement of the distance of the camera from the object; however, these will be considered for further improvements in order to achieve a more precise analysis. Changes in ambient temperature, e.g., background objects, were not considered at first. Once they were found to impair processing, the proposal of an automated cabin was suggested to isolate the stump from air movements and background objects with different emissivity values in order to improve processing performance. The selection of participants needs to satisfy the clinical trial protocol approved by an ethics committee (PR2021-01). All participants in this study must have experienced at least one year since their amputation surgery.

Two sets of five thermal images each are taken for each participant. The first set is captured when the participant arrives at the laboratory (Instance 1), while the second set is captured after a 10 min walk (Instance 2) (Example of this image can be seen in Figure 6). A total of ten thermographic images per thermographic analysis are shot, covering the positions lateral, frontal, medial, inferior, and posterior of the stump.

To capture the images for the analysis, the mobile platform follows a fixed trajectory. The following subsection describes the control solution to achieve the desired path.

3.4. Control Solution

The design of a control algorithm first needs the mathematical representation of the system to regulate, or at least a valid general class of model. Then, according to the specific problem, the input should be selected in such a way that the goal is met. For this work, there is a trajectory tracking problem for a nonlinear system that can be modeled with the Euler–Lagrange approach and represented by an ODE as (1). The first subsection for the control solution describes a simple dynamic model for the automated cabin.

3.4.1. Plant Description

In Figure 7, the free-body diagram representing the system of the automated cabin is depicted.

Table 1. Parameters for the mathematical model of the cabin.

Parameter	Unit	Value	Description
r	m	0.033	Ratio of the wheel
l	m	0.551	Distance between the center of the rail and the center of the wheel
R	m	0.990	Ratio external circle of the rail
$\theta$	rad	NA	Angular position of the car in the rail
$\alpha$	rad	NA	Angular position of the wheel
${\tau }_1$	Nm	NA	Torque exerted for the car
${\tau }_0$	Nm	NA	Torque of the wheel motor (input)
N	N	NA	Normal force in the wheel and rail
g	m/${\mathrm{s}}^2$	9.8	Gravity acceleration
$m_1$	Kg	1.68	Mass of the car
$m_2$	Kg	0.077	Mass of the wheel

summarizes the variables and constants for the system.

First, the position and velocity in the Cartesian coordinates are computed, such that

$$\begin{array}{c}\left\{\begin{array}{c}x=lsin\left(\theta \right)\\ y=lcos\left(\theta \right)\end{array}\right.,\left\{\begin{array}{c}\dot{x}=lcos\left(\theta \right)\dot{\theta }\\ \dot{y}=-lsin\left(\theta \right)\dot{\theta }\end{array}\right.\end{array}$$

The quadratic resulting velocity is given by $v^2=l^2\dot{\theta }$. As it is shown in the detailed view of the diagram (Figure 7), there exists an angular restriction generated for the wheel rolling without slipping in the circular surface. Such restriction is given by

$$(R+r)\theta =r\alpha ,R>r.$$

(2)

The kinetic and potential energy are computed as follows:

$$\begin{array}{c}V=(m_1+m_2)glcos\left(\theta \right),\\ K=\frac{1}{2}(m_1+m_2)l^2{\dot{\theta }}^2+\frac{1}{2}{I}_{m_2}{\dot{\alpha }}^2,\end{array}$$

where $I_{m_2}$ is the inertia for the wheel, which is considered as a solid disk; thus, the inertia can be obtained as $I_{m_2}=m_2{r}^2$, where V is the potential energy, K is the kinetic energy, and the Lagrangian is described as

$$\begin{array}{c}L=\frac{1}{2}(m_1+m_2)l^2{\dot{\theta }}^2+\frac{1}{2}{m}_2{r}^2{\dot{\alpha }}^2-(m_1+m_2)glcos\left(\theta \right).\end{array}$$

Considering the wheel normal force N, the restriction (2), and the relation between the torques ${\tau }_1$ and ${\tau }_0$ given by ${\tau }_1=R{r}^{-1}{\tau }_0$, the Euler–Lagrange equation ${\displaystyle \frac{d}{dt}}\left({\displaystyle \frac{\partial L}{\partial \dot{\theta }}}\right)-{\displaystyle \frac{\partial L}{\partial \theta }}={\tau }_1$ is given by

$$(m_1+2m_2)l^2\ddot{\theta }+(m_1+m_2)lgsin\left(\theta \right)=R{r}^{-1}{\tau }_0.$$

(3)

The following equation describes the electrical model of the DC motor actuating over the wheel:

$$R_{m}I+L_m\dot{I}+K_m\dot{\alpha }=u,$$

(4)

where $R_m$ is the constant that represents the motor coil resistance, $L_m$ is the constant for the motor coil inductance, $I\in \mathbb{R}$ is the current of the motor, $K_m$ is the constant for the electromotive force constant, and $u\in \mathbb{R}$ is the input voltage.

3.4.2. Tracking Trajectory Problem

The dynamic model representing the automated cabin (3), including the motor actuating the wheel (4), can be rewritten using a state variable representation as follows:

$$\dot{q}=f\left(q\right)+Bu,$$

where $q={\left[q_1,q_2,q_3\right]}^{\top }$, $q_1=\theta$, $q_2=\dot{\theta }$, and $q_3=I$, considering the relation between the angles $\alpha$ and $\theta$ given in (2), and considering that the motor torque ${\tau }_0=K_{\tau }I$, with $K_{\tau }$ as the torque constant, the function $f:{\mathbb{R}}^3\to {\mathbb{R}}^3$ and constant vector $B\in {\mathbb{R}}^3$ are described as

$$f=\left[\begin{array}{c}{q}_2\\ {\varphi }_{1}sin{q}_1+{\varphi }_2{q}_3\\ {\varphi }_3{q}_2+{\varphi }_4{q}_3\end{array}\right],B=\left[\begin{array}{c}0\\ 0\\ {\varphi }_5\end{array}\right].$$

where ${\varphi }_1=-{\displaystyle \frac{\left(m_1+m_2\right)g}{\left(m_1+2m_2\right)l}}$, ${\varphi }_2={\displaystyle \frac{R{K}_{\tau }}{r{l}^2(m_1+2m_2)}}$, ${\varphi }_3=-{\displaystyle \frac{K_{m}l}{r{L}_m}}$, ${\varphi }_4=-{\displaystyle \frac{R_m}{L_m}}$ and ${\varphi }_5={\displaystyle \frac{1}{L_m}}$.

The control strategy considers partial state constraints on the tracking error $\Delta ={[{\Delta }_1,{\Delta }_2]}^{\top }$, ${\Delta }_1=q_1-q_1^{\ast }$, ${\Delta }_2=q_2-q_2^{\ast }$, where $q_1^{\ast }$ and $q_2^{\ast }$ are the state variables to represent the desired path with the following dynamics ${\dot{q}}_1^{\ast }=q_2^{\ast }$, ${\dot{q}}_2^{\ast }=h\left(q^{\ast }\right)$, where $h:{\mathbb{R}}^2\to \mathbb{R}$ is a smooth function. Thus, the extended dynamics considering the tracking error and the electrical model of the motor can be represented as

$$\begin{array}{c}{\dot{\Delta }}_1={\Delta }_2,\\ {\dot{\Delta }}_2={\varphi }_{1}sin{q}_1+{\varphi }_2{q}_3-h\left(q^{\ast }\right),\\ {\dot{q}}_3={\varphi }_3{q}_2+{\varphi }_4{q}_3+{\varphi }_{5}u.\end{array}$$

(5)

Here, the following assumptions are considered.

Assumption 1.

The reference path $q_1^{\ast }$ is smooth, bounded, and its derivatives are bounded, i.e.,

$$\begin{array}{c}|q_1^{\ast }|\le q_1^{+},|q_2^{\ast }|\le q_2^{+},\left|h(q\ast )\right|\le h^{+},\forall t\in {\mathbb{R}}^{+},\end{array}$$

where $q_1^{+}$, $q_2^{+}$ and $h^{+}$ are positive constants.

Assumption 2.

The parameters ${\varphi }_i$, $i=\{1,\cdots ,5\}$ are unknown constants, and are bounded in the following sense:

$$|{\varphi }_i|\le {\varphi }_i^{+},$$

where ${\varphi }_i^{+}$ are positive constants.

The control design uses a Nussbaum class of function; this type of function satisfies the following definition.

Definition 1

([18]). A function $N:\mathbb{R}\to \mathbb{R}$ is called a Nussbaum-type function if it satisfies

$$\begin{array}{c}\underset{s\to \infty }{lim\; sup}{\displaystyle \underset{s_0}{\overset{s}{\int }}}N\left(\psi \right)d\psi =+\infty ,\underset{s\to \infty }{lim\; inf}{\displaystyle \underset{s_0}{\overset{s}{\int }}}N\left(\psi \right)d\psi =-\infty .\end{array}$$

For this work, we use the Mittag–Leffler function $N\left(\psi \right)=-k_N{\psi }^{{\alpha }_N}$, ${\alpha }_N\in (2,3]$, and $k_N>0$ [19].

Taking the above-mentioned Nussbaum function, the following lemma is an important preliminary for the control presented here.

Lemma 1

([18]). Let $U(·)$ be a Lyapunov function, $N(·)$ be a Nussbaum function, and $\psi (·)$ be a smooth function defined in $[0,t_f)$. For $t\in [0,t_f)$, if the following inequality is valid:

$${\displaystyle U\left(t\right)\le c_0+e^{-c_{1}t}{\int }_0^t\left(\upsilon N\left(\psi \right)+1\right)\dot{\psi }{e}^{c_1\tau }d\tau ,\forall t\in [0.t_f),}$$

with υ as a nonzero constant, and $c_0$ and $c_1$ as positive constants,

then $U\left(t\right)$, $\psi \left(t\right)$ and $e^{-c_{1}t}{\int }_0^t\left(\upsilon N\left(\psi \right)+1\right)\dot{\psi }{e}^{c_1\tau }d\tau$ are bounded $\forall t\in [0,t_f)$.

The following theorem summarizes the result of the proposed controller. The control solution considers the methods proposed in [18,20,21,22,23], in such a way that the control ensures the restriction fulfillment for the tracking error, the ultimate boundedness for all the states to the equilibrium point, and the asymptotic convergence of the tracking error.

Theorem 1.

Consider Assumptions 1 and  2, and the system  (5) in closed-loop with the following control law,

$$\begin{array}{c}u=N\left({\zeta }_u\right){\alpha }_u,\\ {\alpha }_u=(\gamma +{\left({\varphi }_2^{+}\right)}^2/2){\overline{\Delta }}_3+{\widehat{\Phi }}_3^{\top }{q}_{2,3}-{\dot{\nu }}_2,\end{array}$$

(6)

where $N:\mathbb{R}\to \mathbb{R}$ satisfies Definition 1, the variable ${\zeta }_u$ satisfies ${\dot{\zeta }}_u={\overline{\Delta }}_3{\alpha }_u$, with γ as a positive constant. Consider ${\widehat{\Phi }}_3^{\top }=[{\widehat{\varphi }}_3,{\widehat{\varphi }}_4]$ the vector for estimated parameters ${\varphi }_3$ and ${\varphi }_4$, the vector $q_{2,3}={[q_2,q_3]}^{\top }$. The auxiliary variables $\overline{\Delta }={[{\overline{\Delta }}_2,{\overline{\Delta }}_3]}^{\top }$, ${\overline{\Delta }}_2={\Delta }_2-{\nu }_1$, ${\overline{\Delta }}_3=q_3-{\nu }_2$, with ${\nu }_1$ and ${\nu }_2$ as stabilizing inputs for the two steps applying the back-stepping method,

$$\begin{array}{c}{\nu }_1=-\gamma {\Delta }_1,{\nu }_2=N\left({\zeta }_{\nu }\right){\alpha }_{\nu },\\ {\alpha }_{\nu }={\gamma }_{\varphi }{\overline{\Delta }}_2+{\widehat{\varphi }}_{1}sin\left(q_1\right)-h\left(q^{\ast }\right)-{\dot{\nu }}_1+{\displaystyle \frac{{\Delta }_1}{{\Delta }^{+}-{\Delta }_1^2}},\end{array}$$

(7)

where ${\gamma }_{\varphi }=\gamma +1/2$, ${\zeta }_{\nu }$ satisfies ${\dot{\zeta }}_{\nu }={\overline{\Delta }}_2{\alpha }_{\nu }$, and ${\widehat{\varphi }}_1$ is the estimated parameter for ${\varphi }_1$. The estimated parameters have the following adaptive law:

$$\begin{array}{c}\dot{\widehat{\Phi }}=H\left(q\right)\overline{\Delta }-{\gamma }_{\Phi }\widehat{\Phi },\end{array}$$

with $H\left(q\right)=\left[\begin{array}{cc}sin\left(q_1\right)& 0\\ 0& q_2\\ 0& q_3\end{array}\right]$, $\widehat{\Phi }={\left[{\widehat{\varphi }}_1,{\widehat{\Phi }}_3^{\top }\right]}^{\top }$.

If the initial condition for $q_1$ satisfies $q_1\left(0\right)\in \left(q^{\ast }\left(0\right)-{\left({\Delta }^{+}\right)}^{1/2},q^{\ast }\left(0\right)+{\left({\Delta }^{+}\right)}^{1/2}\right)$, then, the tracking error ${\Delta }_1^2\le {\Delta }^{+}$, $\forall t\in {\mathbb{R}}^{+}$, ${\Delta }_1$ will asymptotically converge, and $\overline{\Delta }$ and the estimation error for the parameters will be ultimately bounded.

Proof. 

Consider the Lyapunov function candidate

$$V_c={\displaystyle \frac{1}{2}}ln\left({\displaystyle \frac{{\Delta }^{+}}{{\Delta }^{+}-{\Delta }_1^2}}\right)+{\displaystyle \frac{1}{2}}\parallel \overline{\Delta }{\parallel }^2+{\displaystyle \frac{1}{2}}{\parallel \tilde{\Phi }\parallel }^2.$$

(8)

The full time derivative of Equation (8) is given by

$${\dot{V}}_c={\displaystyle \frac{{\Delta }_1{\dot{\Delta }}_1}{{\Delta }^{+}-{\Delta }_1^2}}+{\overline{\Delta }}^{\top }\dot{\overline{\Delta }}-{\tilde{\Phi }}^{\top }\dot{\widehat{\Phi }}.$$

(9)

Taking the derivatives ${\dot{\Delta }}_1={\overline{\Delta }}_2+{\nu }_1$ and $\dot{\overline{\Delta }}=[{\dot{\overline{\Delta }}}_2,{\dot{\overline{\Delta }}}_3]$, we can express (9) as follows:

$$\begin{array}{c}{\dot{V}}_c={\displaystyle \frac{{\Delta }_1\left({\overline{\Delta }}_2+{\nu }_1\right)}{{\Delta }^{+}-{\Delta }_1^2}}+{\overline{\Delta }}_2\left({\dot{\Delta }}_2-{\dot{\nu }}_1\right)+{\overline{\Delta }}_3\left({\dot{q}}_3-{\dot{\nu }}_2\right)-{\tilde{\Phi }}^{\top }H\left(q\right)\overline{\Delta }+{\gamma }_{\Phi }{\tilde{\Phi }}^{\top }\widehat{\Phi }.\end{array}$$

(10)

Considering $q_3={\overline{\Delta }}_3+{\nu }_2$ and substituting ${\dot{\Delta }}_2$, ${\dot{q}}_3$ from (5) and the controls (6) and (7) into (10) yields

$$\begin{array}{c}{\dot{V}}_c=-\gamma {\displaystyle \frac{{\Delta }_1^2}{{\Delta }^{+}-{\Delta }_1^2}}+{\displaystyle \frac{{\Delta }_1{\overline{\Delta }}_2}{{\Delta }^{+}-{\Delta }_1^2}}\\ +{\overline{\Delta }}_2\left(({\varphi }_{1}sin{q}_1+{\varphi }_2({\overline{\Delta }}_3+N\left({\zeta }_{\nu }\right){\alpha }_{\nu })-h\left(q^{\ast }\right))-{\dot{\nu }}_1\right)\\ +{\overline{\Delta }}_3\left(({\varphi }_3{q}_2+{\varphi }_4{q}_3+{\varphi }_{5}N\left({\zeta }_u\right){\alpha }_u)-{\dot{\nu }}_2\right)-{\tilde{\Phi }}^{\top }H\left(q\right)\overline{\Delta }+{\gamma }_{\Phi }{\tilde{\Phi }}^{\top }\widehat{\Phi }.\end{array}$$

Now, adding and subtracting ${\dot{\zeta }}_{\nu }$ and ${\dot{\zeta }}_u$ we have

$$\begin{array}{c}{\dot{V}}_c=-\gamma {\displaystyle \frac{{\Delta }_1^2}{{\Delta }^{+}-{\Delta }_1^2}}+{\varphi }_2{\overline{\Delta }}_2{\overline{\Delta }}_3+{\dot{\zeta }}_{\nu }-{\gamma }_{\varphi }{\overline{\Delta }}_2^2+{\overline{\Delta }}_2{\tilde{\varphi }}_{1}sin{q}_1+{\varphi }_{2}N\left({\zeta }_{\nu }\right){\dot{\zeta }}_{\nu }\\ +{\varphi }_{5}N\left({\zeta }_u\right){\dot{\zeta }}_u+{\dot{\zeta }}_u+{\overline{\Delta }}_3{\tilde{\Phi }}_3^{\top }{q}_{2,3}-(\gamma +{({\varphi }_2^{+}/2)}^2){\overline{\Delta }}_3^2-{\tilde{\Phi }}^{\top }H\left(q\right)\overline{\Delta }+{\gamma }_{\varphi }{\tilde{\Phi }}^{\top }\widehat{\Phi }.\end{array}$$

(11)

Taking the following inequality, $-\frac{\phi }{1-\phi }\le -ln\left(\frac{1}{1-\phi }\right),$ which is valid $\forall \phi \in [0,1)$[24], eliminating terms and considering that $\tilde{\Phi }=\Phi -\widehat{\Phi }$, the derivative (11) satisfies the following inequality:

$$\begin{array}{c}{\dot{V}}_c\le -\gamma ln\left({\displaystyle \frac{{\Delta }^{+}}{{\Delta }^{+}-{\Delta }_1^2}}\right)+{\dot{\zeta }}_{\nu }-{\gamma }_{\varphi }{\overline{\Delta }}_2^2+{\varphi }_{2}N\left({\zeta }_{\nu }\right){\dot{\zeta }}_{\nu }\\ +{\varphi }_{5}N\left({\zeta }_u\right){\dot{\zeta }}_u+{\dot{\zeta }}_u-(\gamma +{\left({\varphi }_2^{+}\right)}^2/2){\overline{\Delta }}_3^2-{\gamma }_{\varphi }{\tilde{\Phi }}^{\top }\tilde{\Phi }+{\gamma }_{\varphi }{\tilde{\Phi }}^{\top }\Phi +{\varphi }_2{\overline{\Delta }}_2{\overline{\Delta }}_3.\end{array}$$

(12)

Applying Young’s inequality for the last two elements in (12) and considering the value for ${\gamma }_{\varphi }$, such that

$$\begin{array}{c}{\dot{V}}_c\le \gamma \left(ln\left({\displaystyle \frac{{\Delta }^{+}}{{\Delta }^{+}-{\Delta }_1^2}}\right)+\parallel \overline{\Delta }{\parallel }^2+{\parallel \tilde{\Phi }\parallel }^2\right)+{\dot{\zeta }}_{\nu }+{\varphi }_{2}N\left({\zeta }_{\nu }\right){\dot{\zeta }}_{\nu }\\ +{\varphi }_{5}N\left({\zeta }_u\right){\dot{\zeta }}_u+{\dot{\zeta }}_u+{\gamma }_{\varphi }^2{\parallel \Phi \parallel }^2.\end{array}$$

Then ${\dot{V}}_c\le -2\gamma V_c+\beta$,

with $\beta =\left({\varphi }_{2}N\left({\zeta }_{\nu }\right)+1\right){\dot{\zeta }}_{\nu }+\left({\varphi }_{5}N\left({\zeta }_u\right)+1\right){\dot{\zeta }}_u+3{\gamma }_{\varphi }^2\underset{k=1,3,4}{max}\left\{{\left({\varphi }_k^{+}\right)}^2\right\}$.

Finally, multiplying by $e^{2\gamma t}$ both sides and integrating over $(0,t)$, yields to

$${\displaystyle V\left(t\right)\le V\left(0\right)e^{-2\gamma t}+{\int }_0^t\beta \left(\tau \right)e^{-2\gamma (t-\tau )}d\tau .}$$

Considering the definition of $\beta$, Lemma 1, and Barbalat’s Lemma [25], the proof is completed.    □

3.4.3. Path Design

In order to fulfill the assumption made in the control design, the path should be smooth. This smooth function was composed of sigmoid functions, which have been used to design paths for other classes of robots [26,27]. Sigmoid functions satisfy the following definition:

Definition 2.

A sigmoid function is bounded, continuously differentiable, and has a non-negative derivative. In this work, we define the sigmoid as follows; $s:{\mathbb{R}}^{+}\to \mathbb{R}$ is defined as

$$s_{a,c}\left(t\right)={\left(1+e^{-a\left(t+c\right)}\right)}^{-1},$$

where $a\in {\mathbb{R}}^{+}$ modifies the slope of the function and $c\in {\mathbb{R}}_{+}$ set the time inflection point.

The GUI includes the trajectory to regulate a chosen number of steps in a fixed time. Four steps are needed for the actual thermographic analysis in residual limbs. However, to satisfy the design requirements, it was desirable that the number of steps could be changed for the user. Algorithm 1 describes the steps to obtain the smooth trajectory based on sigmoid functions.

Algorithm 1 Algorithm to adjust the number of steps in the trajectory.

1: Start 2: Initialization 3: $s_t\leftarrow \mathrm{number}\mathrm{of}\mathrm{steps}$; 4: $T\leftarrow \mathrm{selected}\mathrm{time}\mathrm{in}\sec \mathrm{onds}$; 5: $t\leftarrow \mathrm{input}\mathrm{time}$ 6: $\delta =T/s_t$; 7: $A=2\pi /s_t$; 8: $b=2$; 9: for $i=0:st$ do 10:     $au{x}_i=\delta /2+\delta \ast i$; 11:     $y_i=A{s}_{b,au{x}_i}\left(t\right)$; 12:     $y=y+y_i$; 13: end for 14:     $\mathrm{New}\mathrm{path}=\mathrm{y}$; 15: Stop

In Algorithm 1, the function $s_{b,au{x}_i}:{\mathbb{R}}^{+}\to \mathbb{R}$ satisfies the form of Definition 2. Figure 8 depicts a designed trajectory with four steps.

The following subsection briefly describes the modeling for the system to perform a numerical simulation to test the tracking trajectory with the designed path before it can be tested in the actual system.

3.5. Virtual Prototype

The virtual prototype of the cabin was developed with Matlab${}^{\text{®}}$ and the Simscape${}^{\text{®}}$ in Simulink${}^{\text{®}}$ toolbox using the the assembly CAD model from Solidworks${}^{\text{®}}$. Such a prototype served to test the proposed control in a system with similar parameters to the actual cabin.

Figure 9 shows the Simulink workspace of the virtual system.

In the following section, the results of each stage presented in the methodology are described.

4. Results

The results of the mechanical design, electronic instrumentation, control implementation, the GUI, and the image analysis are presented in this section.

4.1. Mechanical Design

In Figure 10, a rendered picture of the CAD is depicted. This assembly shows all the structural elements used to fulfill the design requirements.

The cabin uses an aluminum U channel to hide the round aluminum tubes that support the fabric used to isolate the interior. The aluminum U channels are joined together by medium-density fiberboard (MDF) joints to hold the parts and screws to keep the assembly. A rectangular aluminum tube is also used to elevate the three-layer MDF rail. The rectangular tube and the rail are joined with the rest of the structure by MDF supports that are screwed under the U aluminum channel. The aluminum legs are screwed to the bottom of the structure. Those can be folded to store the structure when not in use.

We use a direct current (DC) motor with an encoder to control the platform’s position in the circular rail. Figure 11a shows an exploded view of the platform while Figure 11b shows the final design.

The final design consists of three roller trackers that help the platform to move along the rail. Four elements need to be joined together to insert the platform into the rail. These four sections form the whole platform, are made from MDF, and are fixated by fastening the camera mount in place with screws.

4.2. Electronic Instrumentation

The main electronic components proposed for the cabin are the DC motor as the actuator, with a gear ratio of 4.4:1, maximum torque of 1.7 kg, maximum RPM of 2250, unload current of 300 mA, maximum current of 5600 mA, and dimensions of 25 × 75 mm. This actuator is a metal gear-motor with a 4.1:1 relation with an encoder, and H-bridge, an optocoupler, a board STM32F103C8T6, and an encoder. The encoder serves to estimate the angular position and velocity of the mobile platform. Such selection fulfills the system requirements for velocity and has enough torque to allow the movement with the mobile platform weight and the external forces of the system.

The communication protocol between the interface and the other system elements is implemented on the main-board and the computer and it was selected as universal asynchronous receiver/transmitter (UART) communication protocol.

As the main-board, we select the STM32F103C8T6 from STMicroelectronics${}^{\text{®}}$ microcontroller; this board has been used for a wide range of applications [28]. To separate the high power consumption elements from the logic devices, we use optocouplers, and for the power stage, we use the module HR0112, which has an H-bridge L298N with the required components to drive the selected actuator.

4.3. Control Implementation

In order to test the proposed control, we use as a reference a sinusoidal function $\pi /2sin\left(0.5t\right)$ to compare the designed algorithm with a classical PD controller. The designed path with sigmoid functions (Algorithm 1) was not used for the comparison in order to better demonstrate the results using both controllers.

The gains for both controllers were selected as $\gamma =K_d=K_p=850$, where $K_p$ and $K_d$ are the proportional and derivative gains in the PD algorithm. The bound was ${\Delta }^{+}=1.02$ and the initial conditions were $q_1\left(0\right)=-1$, $q_2\left(0\right)=1$ and $q_3\left(0\right)=0.01$. In Figure 12, the comparison of the position evolution with the desired trajectory using both controllers is shown. The trajectory with the controller based on BLF converges faster than the trajectory with the PD algorithm.

Figure 13 shows the comparison of the tracking error evolution. The error with the PD controller violates the restriction.

In Figure 14, the convergence of the states ${\Delta }_1$, ${\overline{\Delta }}_2$ and ${\overline{\Delta }}_3$ implementing the back-stepping methodology is depicted.

The implementation of the proposed controller considered the estimation of some of the unknown parameters. Figure 15 shows the real parameter ${\varphi }_1$ used for the system in simulation and the estimated parameter ${\widehat{\varphi }}_1$ with the proposed algorithm.

Finally, Figure 16 shows the response of the controller when the tracking error is close to the imposed boundary. For this test, we set the restriction with the value ${\Delta }^{+}=0.5$.

The control algorithm will be implemented on the GUI to regulate the actual device. The following subsection shows the outcomes of the software implementation.

4.4. Software Implementation Results

Figure 17 shows the actual layout of the GUI. Figure 17a is the main window, Figure 17b is the window after selecting the settings button, and a popup window is shown to introduce the data of the participant. Finally, Figure 17c depicts when the data are set and the working directory is selected. The control to regulate the tracking trajectory is embedded in the GUI code.

The code embedded in the GUI includes the analysis of the thermographic images. The results of this analysis are described in the following subsection.

Thermographic Images Analysis Results

The average temperature and standard deviation of the stump of two participants were computed for each view and are presented in

Table 2. Thermographic image analysis: Participant 1 arriving at laboratory.

View	L	F	M	I	P
Average Temperature °C	28.4	29.3	29.5	27.8	32.6
Standard Deviation	1.0	1.0	0.8	0.5	1.8

,

Table 3. Thermographic image analysis: Participant 2 arriving at laboratory.

View	L	F	M	I	P
Average Temperature °C	31.8	31.5	31.3	30.7	31.2
Standard Deviation	1.3	1.5	1.4	1.0	1.8

,

Table 4. Thermographic images analysis: Participant 1 after 10 min walk.

View	L	F	M	I	P
Average Temperature °C	29.8	30.2	30.3	32.8	32.7
Standard Deviation	1.1	1.0	0.7	1.3	1.4

and

Table 5. Thermographic images analysis: Participant 2 after 10 min walk.

View	L	F	M	I	P
Average Temperature °C	31.9	30.9	31.4	30.8	31.4
Standard Deviation	1.5	1.3	1.7	0.9	1.7

. The data in Table 2 and Table 3 correspond to the images of Participant 1 and 2, respectively, arriving to the laboratory. Notation L, F, M, I, and P denote the lateral, frontal, medial, inferior, and posterior views, respectively.

The data in Table 4 and Table 5 correspond to the analysis of the thermal images after the participant walks for 10 min.

The data mentioned above will be used to assess the quality of prosthetic sockets. In addition, the final average and standard deviation can be used as metrics to evaluate the quality and compare different sockets. Such data are obtained with an automated and controlled process that improves the thermographic capture pipeline and previous analysis made in the lab.

5. Conclusions

This paper presents the design of an automatic system to capture thermal pictures of the residual limbs of amputees. The motivation of such a device is to aid in the thermal image acquisition process within a confined space. The thermograms generated provide helpful information for analyzing the residual human limb in subjects with transtibial amputation. Such a study proposes a non-intrusive method to study the thermal activity on the amputee’s residual limb and seek a correlation to the quality of the socket. The proposed cabin ensures the repeatability of the thermograms acquisition process and provides an isolated workspace, thus, improving the quality of the samples. The methodology consists of the design of the mechanical elements and parts of the system on computer-aided design software, the electronic instrumentation, a graphic user interface (GUI), and the control algorithm based on a barrier Lyapunov function to solve the trajectory tracking for the camera movements, and numerical simulations to illustrate the functionality and the manufacture of a prototype. The GUI was designed to operate the device and store and analyze the obtained data. The results obtained by implementing the control design on the automated cabin reveal that the thermal image acquisition process is completed following the desired trajectory with a mean squared tracking error of 0.0052. With this result, it is evident that the algorithm successfully solved the tracking trajectory problem, allowing the mobile platform to follow a smooth path with a maximum error setting, and it did not need the exact value of the system parameters. In addition, an example of the thermal images of two subjects and the results of processing this class of pictures using the designed interface is shown. In further research, the system will be used in a clinical protocol to compare different sockets.