Lessons Learned from Investigating Robotics-Based, Human-like Testing of an Upper-Body Exoskeleton

Abstract

Assistive devices like exoskeletons undergo extensive testing not least because of their close interaction with humans. Conducting user studies is a time-consuming process that demands expert knowledge, and it is accompanied by challenges such as low repeatability and a potential lack of comparability between studies. Obtaining objective feedback on the exoskeleton’s performance is crucial for developers and manufacturers to iteratively improve the design and development process. This paper contributes to the concept of using robots for objective exoskeleton testing by presenting various approaches to a robotic-based testing platform for upper-body exoskeletons. We outline the necessary requirements for realistically simulating use cases and evaluate different approaches using standard manipulators as robotic motion generators. Three approaches are investigated: (i) Exploiting the anthropomorphic structure of the robotic arm and directly placing it into the exoskeleton. (ii) Utilizing a customized, direct attachment between the robot and exoskeleton. (iii) Attaching a human arm dummy to the robot end effector to simulate a more realistic interface with the exoskeleton. Subsequently, we discuss and compare the results against the aforementioned requirements of a systematic testing platform. Our conclusion emphasizes that achieving objective and realistic testing necessitates highly specialized hardware, algorithms, and further research to address challenging requirements.

1. Introduction

Exoskeletons, wearable mechanical systems designed to enhance human physical capabilities [1], have seen substantial growth in various domains over the past few years. The global exoskeleton market is expected to grow significantly, reaching 14.67 billion USD in 2030 from 0.9 billion USD in 2022, reflecting a compound annual growth rate of 42.2% according to Fortune Business Insights [2]. The healthcare and manufacturing sectors are the primary drivers of this growth [2]. In healthcare, exoskeletons aid individuals with physical disabilities by reestablishing motor functions and supporting rehabilitation [3], for instance [4,5]. In manufacturing, they assist in physically demanding tasks, enhance endurance, and reduce the risk of physical stress [6].

Despite significant efforts in developing exoskeleton devices [7], there remains a need for more standardized testing [8]. Currently, testing procedures predominantly occur within the development cycle, tailored to the specific characteristics of individual exoskeleton designs [4,9]. However, systematic testing is crucial, specifically due to the close human–machine interaction and various wearing effects [10,11]. Additionally, systematic testing could contribute to benchmarking exoskeletons against each other, offering insights for improvement and supporting iterative development.

User studies, while essential, suffer drawbacks for systematic testing. Conducting studies requires a significant number of participants, making them time-consuming and reliant on expert knowledge [12]. Test-retesting reliability criteria are challenging to meet [13]. Technical measurements within user studies, like measuring contact forces, present challenges with special equipment requirements [14] and potential side effects. Specifically, installing the required sensor at the contact location can cause inaccurate results or change the original behavior of the test subject.

In this context, a promising approach evidently is robotics-based testing that mimics human body dimensions and motion behaviors, inspired by test benches in automotive or aviation. This technical evaluation enhances comparability between different systems or designs, addressing the variation in test setups such as test samples, tasks, execution, and environments [15,16]. User feedback offers essential insights into the exoskeleton’s performance with regard to different measures like comfort, usability and human acceptance. These qualities are hard to measure objectively, which is why user studies are conducted in many areas of human-machine interactions, for instance in [17,18,19]. Robotics-based testing rather offers a complementary tool to address the limitations inherent in user studies and, in combination, offers a more comprehensive testing toolkit. In the literature, the first approaches utilizing humanoid robots as test platforms have been documented, e.g., the humanoid robot “PETMAN” for testing chemical clothing [20]. Regarding exoskeletons, several approaches [21,22,23,24,25] primarily focus on back-support systems, with shoulder support systems yet to be thoroughly explored.

Proposed Approach for Systematic Testing of Exoskeletons

To address the absence of objective testing, our research investigates how to systematically test exoskeletons using robots; this paper presents the lessons we have learned through this approach. It covers conducted approaches to create a testing platform for upper-body exoskeletons utilizing standard robotic manipulators.

In our previous work by Wehrle et al. [26], we have outlined how three interconnected research domains interact in implementing the test platform, as illustrated in Figure 1. The first domain, Template Models and Interaction Simulation addresses the challenge of creating simplified abstractions of the interaction between the robot and the exoskeleton. These template models are crucial for gaining insights into complex interaction dynamics, developing theoretical approaches, and facilitating simulation. This primarily involves modeling the dynamics of the exoskeleton’s actuator, essential for control and design processes.

Achieving proper testing of exoskeletons with robots necessitates human-like motion generation, a challenge addressed in the second domain: Human Motion Capture and Learning. This domain is further divided into two subproblems: Human Motion Capture and Human Motion Learning. Human Motion Capture involves recording representative human movements and force profiles. As reported in [8], due to the diverse real-life applications of exoskeleton devices, selecting a reduced yet representative set of test cases becomes essential. With a representative dataset, Human Motion Learning focuses on encoding motion and force primitives through machine learning techniques.

The third research domain, the Dedicated Robotic Test Platform, tackles challenges associated with robotic hardware. The robotic hardware must execute human-like motions based on the motion and force primitives acquired from the Human Motion Learning domain. This poses significant challenges, imposing requirements on the robot platform in terms of kinematics, dynamics, morphology, and control. The ability to perform human-like motions is critical for the effectiveness of the test platform.

The subsequent sections include a detailed description of the scope of robotic testing, problem analysis, and requirements. Different approaches and experiments towards a concrete robotics-based testing platform are then provided and discussed. Finally, emerging challenges and lessons learned are thoroughly discussed and summarized.

2. Materials

This section delineates the scope of the paper, focusing on the evaluated approaches within. The subsequent discussion entails a comprehensive problem analysis, emphasizing the need for a harmonized test platform. By formulating specific requirements, it also addresses crucial aspects such as manipulators, interfaces, and data collection.

2.1. Scope and Used Exoskeleton Hardware

In pursuit of initial insights, this work employs a standard robotic manipulator to explore the early stages of a hardware platform prototype to test the upper-body exoskeleton “Lucy” [27]. Positioned on the lower back and secured by a shoulder belt, Lucy incorporates two passive degrees of freedom (DoF) for human shoulder movement. It is equipped with a pneumatic actuator designed to facilitate arm-lifting motions as well as holding elevated arm positions, as illustrated in Figure 2. A force is applied to the upper arm, varying in a parabolic fashion relative to the shoulder angle $\alpha$ (see Figure 2). This support begins at zero when the upper arm faces down vertically and progressively increases, reaching its maximum at a shoulder angle of approximately 90°.

Manufacturers typically design upper-body exoskeletons for overhead tasks, see Figure 2. There is a diverse range of use cases, as listed in reviews [28,29]. Starting with simple reaching and transportation tasks, for instance, package handling tasks. Another big area of use is assembly tasks like automotive undercarriage, cab and hydraulic assembly tasks. Further use cases are process tasks, like welding, painting, bending and twisting. Therefore, an effective robotic testing hardware platform must adeptly represent the critical aspects of these diverse use cases. The primary considerations revolve around interactions with the environment, processing forces, and the key motion patterns essential for various use cases, especially lifting motions and holding elevated arm positions.

2.2. Problem Analysis and Requirements

The primary goal of evaluating exoskeletons is to assess how well they fulfill user needs across various use cases [30]. Evaluation metrics and methods in this context encompass kinematic behavior, performance, usability, workload, and ergonomics [8].

Metrics related to behavior include degrees of freedom, kinematic structure, supportive effect and dexterity, providing insights into the exoskeleton’s workspace capabilities. Performance metrics, such as task productivity, range of motion, mobility, and load capacity describe a user’s ability to perform tasks while wearing an exoskeleton. Ergonomics metrics, covering aspects like muscle activity, joint angle, and interaction force, focus on the physical aspects associated with task execution. Finally, usability metrics, including usage complexity, comfort, and human-exoskeleton alignment assess cognitive load and compatibility between the exoskeleton and the user.

Evaluation methods fall into three categories: physical model-based, human-based, and biomechanical model-based methods, each with its advantages and disadvantages [31]. Physical model-based methods stand out for their objectivity, low human life risk, high repeatability, applicability to various test conditions, and provision of standardized test procedures. For upper-body exoskeletons, evaluating their supportive effect involves measuring generated joint torques, such as in the shoulder, to reduce workload [32]. Analyzing assistive torque profiles aids in understanding how to choose an appropriate profile based on task requirements and user anthropometry, contributing to the evaluation of exoskeleton benefits [33].

To ensure an objective and standardized test procedure, objective metrics, particularly force-based metrics are preferred over subjective and human-effected metrics. Interaction force metrics, including normal and shear forces, overall interaction force, and peak and average contact force, contribute to assessing exoskeleton ergonomics and comfort [34]. Interaction torque, derived from interaction forces, provides insights into load transfer and is typically measured at joints to calculate torque transmitted from the exoskeleton’s interface to the user [30,34]. Interaction pressure, calculated from interaction force over the contact area is a critical and measurable metric impacting exoskeleton ergonomics. Metrics include maximum pressure and pressure distribution [34]. Given the complexity of anticipating shoulder motion through electromyography signals and the challenge of calculating forces in human joints, evaluating the impact of exoskeletons on joint moments, muscle forces, and joint contact forces is crucial [35,36].

2.2.1. Robotic Manipulator

The robot serves as the motion generator responsible for replicating human-like behavior, necessitating the consideration of various aspects. Primarily, the robot must generate trajectories aligning with the desired use cases, imposing requirements on positioning, velocity, and acceleration. Whether the robot is directly connected to the exoskeleton or indirectly via other objects, its workspace must enable trajectory following while avoiding singularities and collisions.

The dynamics of the manipulator introduce additional requirements. Firstly, robot motors must generate the desired accelerations while compensating for forces from the exoskeleton, as well as counteracting gravity and inertia forces from potential tools. Secondly, for contact tasks, the robot must create desired contact wrenches while interacting with and moving the exoskeleton. From a control perspective, dynamics are intriguing. Research indicates that humans can adjust their endpoint stiffness under externally applied forces [37,38] and even learn models of unstable dynamics [39,40]. To realistically capture human dynamics, a torque-controllable robot is a natural choice.

2.2.2. Exoskeleton Interface

The interfaces that connect the human body and the exoskeleton are crucial for defining the kinematic chain and facilitating their interaction, significantly influencing operational efficiency. Simulating and analyzing these interfaces is vital for evaluating control effectiveness and mechanical motion transmission. While various approaches for digital simulation and numerical analysis have been explored [41,42,43], physical simulations are essential to gain insights into the actual interaction, despite inherent simplifications in these methods.

Figure 3 illustrates the highlighted interfaces of the exoskeleton “Lucy” (left) and the coupling regions on a streamlined human body (right). The highlighted interfaces, such as arm shells on the upper arm and belts encircling the shoulders and hips are essential for understanding the configured kinematic chain, the transmitted support load via the coupling, and the maintenance of stability, safety, and coordinated movements. The primary focus is on the interface where supporting forces are applied, particularly at the arm shells marked with dotted lines.

One critical aspect to consider is that the human arm is not rigidly fixed to the exoskeleton. In typical upper-body exoskeletons, the human arm is positioned in an arm shell fixed with a strap (Figure 2). During application, the human arm can potentially move slightly inside the arm shell, necessitating consideration of sliding motions and rotations in interface modeling or motion generation. The softness of human tissue adds another layer of complexity. Soft tissue deformation under applied forces results in parts of the interaction force not contributing to the lifting motion. Furthermore, shear forces appear due to relative motion at the coupling. Realistically modeling a human arm with soft tissue is challenging due to variations in dimensions and softness among different subjects [44,45].

Additionally, the spatially and temporally changing interaction pressure distribution of the force poses challenges. The exoskeleton applies force as pressure on the contact surface between the human arm and the arm shell. With movement and soft tissues, the distribution may change significantly within different movements. Friction forces between the human arm and the interface, along with effects like softness and movement, must be considered in the design of the test station. Clothing worn by the individual inside the arm shell also introduces effects as soft objects that deform, move, and alter friction, clothes contribute to the complexities of the interaction, alongside the pure human arm.

The relevance of these considerations extends beyond the main arm shell interface to all interfaces connecting the human to the exoskeleton. Counterforces resulting from assistive forces on the upper limbs transmit through the exoskeleton, impacting all interfaces. Consequently, examining interactions at the lower back and shoulder becomes crucial for comprehensive investigations and understanding the flow of forces.

2.2.3. Sensors and Data Collection

To measure the objective metrics outlined earlier, various sensors are necessary. A key requirement for these sensors is to minimize the introduction of side effects. Interaction forces or wrenches can be measured using force/torque (F/T) sensors, either as standalone devices or by leveraging a robotic manipulator. Special consideration is required for the evaluation point of the interaction force, which may not be fixed given the potential movement of the human arm within the arm shell. Therefore, pressure sensors could be applied at the interfaces to measure the spatial-temporal pressure distribution, independent of any specific evaluation point.

Similar considerations apply to measuring interaction torque. F/T sensors can be employed, but defining the evaluation point is crucial. For upper-body exoskeletons, the shoulder support in the form of the shoulder moment is of particular interest, requiring a dynamically defined shoulder axis. Kinematic quantities can be measured either by the robot manipulator through forward kinematics or by a motion-tracking system.

For example, analyzing the kinematic chain of the exoskeleton can involve measuring the position of passive joints. This not only helps compare the human likeness of motion generated by the test platform but also provides insights into the nature of the kinematic chain of the exoskeleton.

3. Methods and Results

This section explores intermediate approaches to developing a systematic testing platform for upper-body exoskeletons, utilizing a Franka Panda 7 DoF redundant robotic arm as the robotic platform. In the subsequent experiments, the exoskeleton is firmly attached to the environment by mounting the shoulder-neck element of the exoskeleton to an aluminum profile using screw clamps, as illustrated in Figure 4. More details about the mounting of the exoskeleton can be found in Appendix A. Unlike regular usage scenarios of upper-body exoskeletons, where the flow of forces is established between the arm interface and the lower back interface, this rigid attachment alters the natural force flow. It redirects forces from the arm interface into the environment, representing an oversimplification. Moreover, in practical usage, the exoskeleton would have slight relative movement with the wearer, allowing for sliding of parts like the shoulder belt or lower back. Additionally, the back structure is not rigid and exhibits elastic deformation effects. By rigidly fixing the exoskeleton in place, these effects are disregarded in favor of focusing on the primary interface at the upper limb.

3.1. Approach A: Direct Placement

One initial concept is to utilize an anthropomorphic robotic arm and position it within the exoskeleton interface, with the straightforward idea of having the robotic manipulator emulate the human arm. We refer to this as “Direct Placement” and explore this fundamental idea first.

3.1.1. Idea

This approach encompasses scenarios where the robot manipulator is directly placed in the arm shell of the exoskeleton. The connection between the robot and the exoskeleton is inherently constrained, allowing movement and rotation within the arm shell which is a desirable characteristic. Given the rigid nature of the contact interface and the absence of additional soft tissue modeling, deformable aspects are neglected. Nevertheless, it serves as a basic and straightforward starting point.

3.1.2. Implementation

Initially, we attempted to leverage the anthropomorphic kinematic chain of the Panda to position its “upper arm” (composed of links 2 and 3) into the arm shell of the exoskeleton, as depicted in Figure 5 (left). This configuration would be advantageous because the anthropomorphic structure of the robot arm could be coupled with the exoskeleton while retaining sufficient degrees of freedom to perform tasks with the environment. Motion generation would benefit from this setup since there is a limited yet interpretable mapping from human motion to the kinematic chain. From the interface perspective, the contact would be modeled as rigid but capable of sliding and rotating inside the arm shell and identified as useful in the problem analysis. Moreover, the support of the exoskeleton would be meaningfully applied to the robot chain, allowing for reasonable interpretation of joint torques from the joints contributing to the lifting motion.

However, the dimensions of the robot’s links do not align with the dimensions of the arm shell. Additionally, finding a suitable position to place the robot inside the exoskeleton while ensuring a sufficiently large workspace for reasonable motion creation without collisions, especially with the exoskeleton itself, proved to be difficult. Another issue with the setup in Figure 5 (left) is that the robot only has joints 1 and 2 to move the exoskeleton. This limitation hinders human-like motion generation, as it can only create a spherical elbow motion and does not account for the complex motion generated by the human shoulder. This limitation is inherent in anthropomorphic standard manipulators, as additional degrees of freedom at the base link would be required to replicate the more versatile motions of the human shoulder.

An alternative approach would be to deviate from exploiting the anthropomorphic structure of the robot and to place any other link inside the arm shell, establishing contact with the robotic arm. Figure 5 (middle and right) illustrates the idea of placing the “lower arm” (link 5) into the arm shell. While this approach retains the benefit of allowing contact in the interface to slide and move in the arm shell, the workspace is severely limited, making reasonable motion ranges impractical. The middle and right illustrations in Figure 5 show configurations with the lowest and highest possible lifted positions of the exoskeleton, highlighting the constrained workspace.

Similar challenges arise with different placements of the robot, as additional environmental constraints, such as collision and singularity avoidance, need to be considered. Moreover, interpreting joint torque measurements becomes more challenging, as the topology of the applied support force undergoes significant changes. Consequently, the ability to leverage the anthropomorphic design of the robot for measurement interpretation is compromised.

3.1.3. Results

In summary, we were unable to identify a practical compromise for a robot and exoskeleton setup. The authors believe that the challenges faced are not specific to the robot used but are of a general nature. Utilizing other standard manipulators is unlikely to resolve the issues related to incompatible dimensionality, environmental conditions, or workspace limitations. We also had access to various industrial robots manufactured by Kuka and Fanuc, including the Kuka iiwa. However, these alternatives were even less suitable due to their physical dimensions.

3.2. Approach B: Direct Attachment

To address the challenges linked with directly placing the robot within the exoskeleton, we explored the option of directly attaching the robot to the exoskeleton. This section will first delve into the concept and key considerations based on the requirements outlined in Section 2.2. Subsequently, we will detail and analyze an experiment involving the direct attachment of the robot to the exoskeleton.

The idea is to attach the robot directly to the exoskeleton, which necessitates modifications to the exoskeleton by attaching a specialized hardware interface. To ensure a meaningful experiment, several considerations must be addressed. First, the exoskeleton should support a lifting or rising motion of the robot within a reasonably large workspace. Second, the robot end effector must remain free for interaction with the environment, allowing the study of contact and process wrench effects. It is crucial to select an appropriate attachment point on the robotic chain to achieve these objectives. However, modifications to the exoskeleton introduce changes to the interaction and may vary across different exoskeleton models, posing challenges to systematic testing. While full reproduction of human motion with the robot is unattainable due to the lack of human morphology, it is feasible to define points on the exoskeleton and track their positions based on user-recorded data, aiming for realistic motion representation as described in the requirements in Section 2.2.

3.2.1. Implementation

The key challenge in directly attaching the robot and exoskeleton is to determine a suitable position and configuration for the robot that allows the proper movement of the exoskeleton. The objective is to position the attachment point close to the actual supporting force position in real applications, minimizing unnecessary levers. To achieve this, the arm shell is removed from the exoskeleton, and a 3D-printed part is created to connect the exoskeleton with the robot, as depicted in Figure 6 (top). A successful setup involves attaching the exoskeleton at link 5 of the robot (Figure 6), leaving two degrees of freedom for the robot’s interaction with the environment. However, due to the limited workspace, only a constrained set of configurations adhering to the attachment constraints is possible. Alternative attachments were either unable to move the exoskeleton as intended or introduced large levers with additional side effects.

3.2.2. Idea

To overcome the limitation of the workspace, a sequence of hand-guided lifting and lowering motions of the exoskeleton is recorded. A Dynamic Movement Primitive (DMP) is then employed to learn from the recorded motion data and generate trajectories for executing motion similar to the hand-guided training data. The DMPs are implemented with the “DMP” ROS package from [46].

3.2.3. Results

With the help of the learned DMPs, we executed the same lift and lowering motion ten times, respectively, with the support of the exoskeleton and without the exoskeleton attached to the robot. Figure 7 shows the recorded joint torque profiles for the first five joints in the upper plots, and the last plot shows the z-position of the end effector. Since the attachment is made after joint 5, the remaining joints are not influenced by the support of the exoskeleton and are not considered here. All the torque signals are filtered using a moving average with a window size of 30, equivalent to a 30 ms time window. The graphs in blue represent the mean joint torques without the support and the black ones with the support of the exoskeleton.

In the lifting and lowering motion, joints 2 and 4 contribute significantly (refer to the robot configuration in Figure 6). The impact of exoskeleton support is evident in their respective torque profiles. Joint 2 exhibits consistently lower torque throughout the execution, while joint 4 initially shows higher absolute torque but decreases below the unsupported torque signal as the motion progresses. The sign of the torque difference depends on the direction of “positive” rotation, and notably, joint 4 flips the sign indicating the controller working against the support. Additionally, other joints exhibit changes in torque profiles. Particularly, joints 1 and 5 show increased torque with the exoskeleton attached, contrasting with the near-zero torque without it. This might be attributed to the constraint forces and disturbances induced by the attachment, potentially explaining the higher starting torque at joint 4.

While these measurements provide support for the concept of testing exoskeletons with robots, their interpretation in the context of a systematic testing approach is limited. The recorded measurements are specific to a particular sequence of robot configurations. Changing the robot’s configuration, especially when testing different exoskeletons, may alter the robot joint trajectories. Consequently, comparing torque profiles alone would lack meaning, as torques would be recorded at different robot configurations.

To address this, a criterion independent of the robot’s configuration and joint trajectory is required. One approach could involve evaluating the contact wrench at the attachment point as a metric for the interaction force, thereby generating an assistive profile. However, careful consideration is needed to exclude constraint forces resulting from non-constraint-consistent motion. Regarding the interaction with the environment, potential interaction will be limited due to the coupling with the exoskeleton. However, by creating a specialized environment, it should be possible to study the effects of contact wrenches exerted by the robot end effector. Nevertheless, a lot of simplifications regarding the interface, interaction, and motion have been introduced, making it challenging to justify this approach for a systematic testing framework.

3.3. Approach C: Specialized Human Arm Dummy

As a further improvement, the utilization of a specialized human arm dummy is proposed; it is mounted to the robot’s end effector and is used to imitate the shape of a human arm.

3.3.1. Idea

Our approach involves creating a physical model of a human arm and attaching it to a robotic manipulator that moves it, as depicted in Figure 8. A basic model can be constructed using 3D-printed or specially manufactured parts with appropriate dimensions. Such a model allows for the use of the unchanged exoskeleton interface, as intended by the manufacturer. Building upon this basic model, additional levels of complexity can be introduced to account for various effects.

For example, to simulate the softness of human tissue, the concept can be expanded to incorporate removable soft tissue models with different properties. Integrated pressure pads could sense the interaction with the arm shell, providing valuable information about the interaction pressure. Additionally, various types of clothing could be placed over the model to emulate additional effects. Further complexities may involve incorporating an end effector to interact with the environment, which could be limited or require a complex tool at the arm dummy.

The requirement of human motion reproduction in this approach is primarily a control and modeling problem. For a rigid arm dummy, human motion reproduction is limited to the pose of the arm dummy, representing a simplification of the actual motion. Consequently, only the main contributing motions can be considered.

3.3.2. Implementation

For the experiments, a 3D-printed human arm dummy is created from a 3D scan of a real human upper limb. The used arm model and robotic setup can be seen in Figure 9. For the conducted experiments, a rigid arm dummy is used for the prototype, and effects caused by soft tissue effects are neglected. Additionally, a rigid adapter is designed to attach the arm dummy to an end effector of the robot.

Similar to the previous approaches, identifying an optimal configuration of the robot and exoskeleton to execute the desired pose trajectory is challenging due to workspace constraints and physical dimensions. While changing the robotic hardware might address some issues related to a small workspace and singularity avoidance, improvements in collision avoidance may be limited. Nonetheless, a viable setup was identified for the Panda robot. More details about the experimental setup can be found in Appendix A.

To achieve the human-like movement of the arm dummy, motion data were recorded from various users wearing the exoskeleton. Users donned a motion capture suit with X-base markers based on the Plug-in Gait full-body model from the Vicon motion capture system. Additionally, users wore the upper body exoskeleton, and to align the user’s arm posture with the dummy arm, an elbow angle stabilizer was employed. Users were instructed to execute a straight lifting and lowering movement of the right arm. The recorded data were post-processed and utilized to train a DMP-based motion representation.

Nevertheless, applying these learned motions directly to the robot is not straightforward. Given that the robot-arm dummy-exoskeleton forms a closed-loop system, generated motions must adhere to the constraints imposed by the coupling. Failure to do so may result in additional constraint forces affecting the torque profiles. Specifically, the intricate shoulder motion of a human arm and the corresponding relative motion between the human arm and the exoskeleton must be accurately captured in the motion planning to satisfy all the constraints. In our experiments employing user-recorded data, we were not able to avoid configurations of the human arm dummy that introduced significant constraint forces. However, achieving constraint-consistent motion generation in this setting poses a complex optimization problem, requiring compromising the reproduction of recorded and learned motion data with satisfying constraints introduced by the coupling. Given these challenges, we opted to record a hand-guided motion of an arm lift from the robotic setting directly and trained a corresponding DMP, following a procedure similar to the one used in the “Direct Attachment” approach. As in the previous experiment, the DMPs function as trajectory generators for our controller.

3.3.3. Results

The trained DMPs were employed to execute ten sequences of hand-guided lifting and lowering motions, both with the support of the exoskeleton and without the exoskeleton attached to the arm dummy. Throughout the arm lift motion, the robot maintains a similar configuration, avoiding a null-space self-motion of the elbow. Figure 10 displays the recorded mean joint torque profiles, with the support of the exoskeleton represented in black and without support in blue. All torque signals are filtered with a moving average of window size 50, equaling a 50 ms time window.

The robot’s configuration at the start of each motion is depicted in Figure 9. Observing this configuration reveals that joints 2, 4, and 6 directly contribute to an upward motion of the robot end effector. Therefore, evaluating the torque profiles of these three joints is of particular interest. The sign of the difference between the curves (comparing joints 2 and 4) depends on the direction of “positive” rotation. For all three joints of interest, it is evident that the absolute torque required to lift the arm is reduced. Without the exoskeleton, joint 2 is needed to lift the arm upwards. With the support, however, the required torque is temporarily completely alleviated. There is even a torque in the opposite direction applied between the 9–13 s mark, indicating that the robot needs to compensate for the exoskeleton’s support to remain at the desired configuration.

Joints 1, 3, and 5 demonstrate the need for a different torque profile to follow the desired trajectory. While a relatively constant torque is applied without the exoskeleton, the interaction force creates torsion that needs compensation, particularly around the base (joint 1) and before the elbow (joint 3). Steps in the joint torque diagrams in Figure 10 are noticeable around the 16–20 s interval, potentially originating from the exoskeleton’s pneumatic actuator releasing air abruptly during the downward motion of the arm.

This approach offers a more realistic interface with the human arm model compared to the previous approaches discussed above. However, it still has shortcomings related to softness, potential interaction with the environment, and achieving truly human-like motion. While challenges associated with softness and human-like motion might be solvable with further research, the interaction with the environment may always be limited by the human arm dummy.

The recorded joint profiles support the idea of robotic-based testing for exoskeletons. However, torque profiles alone are not suitable as evaluation metrics due to their configuration-dependent nature. Therefore, more sophisticated metrics are needed, such as the interaction pressure measured by a pressure pad between the exoskeleton interface and the human arm dummy. Additionally, it might be possible to leverage the realistic dimensions of the human arm model to calculate interaction torques in the shoulder of the human arm dummy and generate an assistive shoulder torque profile. This requires an accurate model of the human arm and realistic human motion generation, considering that the shoulder frame moves within the execution. By addressing this challenge, a reasonable, transferable metric could be developed to evaluate different exoskeletons.

3.4. Comparative Analysis

In this subsection, the approaches’ advantages and disadvantages are compared in

Table 1. Comparison of different intermediate approaches towards a robotic-based testing station.

	Advantages	Disadvantages
Approach A	+ no modifications to the exoskeleton + exploiting anthropomorphic structure possible * + robot end effector remains free *	− standard manipulator might not fit into constrained workspace − rigid contact interface and no human-like shape − appropriate hardware needed (i) for dexterous motion generation and (ii) to fit into the workspace − motion retargeting to kinematic chain necessary − different exoskeleton devices might introduce infeasible workspace constraints if hardware is not appropriate
Approach B	+ robot end effector remains free + standard manipulators feasible + supporting wrench acts at defined contact point	− no realistic contact interface − specialized hardware needed to connect exoskeleton and robot − potentially strong modifications to exoskeleton − only motion of exoskeleton is imitable − different exoskeleton devices need different attachment devices
Approach C	+ more realistic modeling of contact interface (softness + shape) + no modifications to the exoskeleton; usage of contact interface as intended + motion retargeting to arm dummy + relatively free choice of robotic hardware + approach works for different exoskeleton devices	− no interaction with the environment possible − design of human arm (specialized hardware)

* with appropriate hardware

and a discussion of the applicability to testing scenarios is presented. The following discussion is made under the assumption, that an implementation of Approach A is possible, e.g., with specialized hardware. The advantages and disadvantages are based on the requirements discussed in Section 2.2, and therefore, have a different significance.

For a systematic testing platform, a wide range of different shoulder support exoskeleton devices need to be able to be tested. Testing a variety of exoskeleton devices with Approach C should be possible, as the arm dummy generally can be expected to fit into the devices’ arm shells. Potential workspace constraints depend more on the robotic hardware, which is simply interchangeable in this approach. This is not generally true for Approach B, on the other hand, since a special device needs to be created to attach the exoskeleton to the robot. With these attachment devices and different modifications to the exoskeleton devices, comprehensive systematic testing of exoskeletons is significantly more challenging. Further, Approach B falls short in terms of keeping the interaction interface as realistic as possible, as modifications to the interface need to be implemented. Positively, the interaction can be modeled as defined point contact, simplifying an evaluation of the performance. Yet, Approaches A and C maintain the exoskeleton interface and hence are more desirable. In the context of modeling the interface realistically, Approach C exceeds Approach A since human shape and tissue characteristics can be taken into account.

From the perspective of creating human-like motion, motion retargeting must be solved differently for the approaches. For Approach B the motion retargeting needs to be conducted in terms of motions of the exoskeleton, e.g., tracking motions of markers on the exoskeleton. On the other hand, upper-arm configurations can be captured and retargeted for Approaches A and C. In Approach C the motion data can be mapped to the arm dummy pose, assuming an arm dummy without additional degrees of freedom. For Approach A, the captured motion can be retargeted to the kinematic chain of the robot. The different retargeting targets lead to different motion aspects, which are presented in the approaches. For Approach B the motion of the exoskeleton is represented, and for Approach C mainly the pose of the human upper arm and motions from the shoulder are considered. For Approach A, the motion retargeting depends on the kinematic chain of the robot, but for a more comprehensive approach, the aspects of the complete motion of the human upper arm and shoulder should be taken into account.

The concepts of Approaches A and B provide the opportunity to interact with the environment since, with appropriate hardware, the end effector remains free. The interaction capability of Approach C depends on the design of the human arm dummy.

In terms of workspace constraints and robotic hardware, Approach C offers the most flexibility, only requiring moving the arm dummy within the workspace. Therefore, any robotic hardware that allows collision and singularity-free motion can be used. For Approaches A and B, the requirements for the robotic hardware are more demanding since they need to fit into the constrained workspace.

4. Lessons Learned

The previous section introduced three approaches (A: Direct Placement, B: Direct Attachment, and C: Human Arm Dummy) for creating a robotic testing platform for exoskeletons. The experiments conducted using these approaches revealed various challenges and considerations that are discussed in this section specifically in relation to the requirements outlined in Section 2, along with essential aspects for designing future specialized robotic hardware for testing exoskeletons.

4.1. Mounting of the Exoskeleton and Interface

In all conducted experiments, the exoskeleton was rigidly fixed to the environment. While this may suffice for initial experiments, it neglects critical effects arising from relative motion, elastic effects, and changes in the flow of forces within the exoskeleton. A clear lesson is that creating a realistic human body dummy is crucial to wearing the exoskeleton more realistically, considering more device-specific effects. Utilizing a human body dummy could establish the natural flow of forces in the exoskeleton, enabling a better understanding of force transfer and distribution to the human body and of the effects on other interfaces, such as the lower back. While we expected initially that putting the robot within the exoskeleton would yield useful insights despite the simplifications and approximations implied, this did not turn out to be very fruitful.

Creating a realistic interface between the exoskeleton, the robot, and the environment is another key challenge. The fixed attachment of the robot to the exoskeleton is not promising for systematic testing. Introducing a human arm and torso dummy could address this issue. However, while more realistic, this approach presents design challenges of its own, including creating softness, appropriate friction, and shaping. Additionally, designing a dedicated robotic platform requires a realistic interface similar to a human arm dummy. A potential research direction could combine Approaches A and C, involving a specially designed upper-body torso with arm chains featuring the required characteristics. For example, upper arm links could accommodate interchangeable arm models to simulate different human subjects with varied arm shapes, dimensions, and soft tissues. While this paper primarily focused on the upper arm-exoskeleton interface, similar principles could be applied to other interfaces like the lower back and shoulder areas.

4.2. Kinematics and Human-like Motion Generation

Systematic testing of exoskeletons requires human-like movement capabilities. The kinematic structure of the robot must align with the specified workspace influenced by use-case requirements. However, our experiments revealed the challenge of achieving this with standard robot manipulators. Any form of connection between the robot and the exoskeleton establishes a parallel system, significantly constraining motion. This limitation is mediated when the exoskeleton is not rigidly attached to the robot, introducing deformations and additional degrees of freedom. These constraints must be carefully considered in platform design or motion planning, demanding extra effort to ensure accurate testing conditions.

Every approach faced distinct limitations due to the inherent parallel nature of the interaction. For instance, if an anthropomorphic robotic arm is placed inside the exoskeleton, the ability to move the exoskeleton is restricted by the joints until the attachment point, as discussed in Section 3.1.2. This constraint necessitates having sufficient degrees of freedom to create motions closely resembling those a human would generate. Similarly, the “Direct Attachment” approach faces limitations as the joints before the attachment point define the available degrees of freedom to move the exoskeleton, potentially conflicting with the requirement for enough remaining degrees of freedom to interact with the environment.

Additional challenges arise with the “Human Arm Dummy” approach. In addition to requiring human-like arm trajectories and satisfying constraints imposed by the parallel mechanism, controlling the sliding and motion of the arm within the arm shell introduces a significant control problem. Moreover, the generated motion must be constraint-consistent to prevent the generation of constraint forces that could compromise interaction measurements and protect the robotic hardware from exceeding limits. Existing human motion imitation methods for robots frequently do not address these complex environmental constraints. For instance, in [47] the motion imitation problem is formulated as an optimization problem and only considers hardware limits of the humanoid. As [48] pointed out in their review, constraints in general and avoidance of obstacles specifically is in general an under-researched problem and might only feature simple static and dynamical obstacles. Nevertheless, a dedicated robotic platform would need to account for all of these challenges, not only from the hardware point of view.

A crucial insight gained from this exploration is the intricate connection between human-like motion generation, the design of the hardware platform, and the associated control challenges. It became evident that relying on a simplistic “learn-from-the-human” approach using standard methods like Dynamic Movement Primitives (DMP) falls short, highlighting the necessity for more advanced methods to address these complexities effectively. This motivates future research of special motion algorithms that need to account for the learned motion primitives and satisfy the constraints introduced by the coupling to the exoskeleton. Optimization-based algorithms could be a promising research direction to solve the motion retargeting problem in constrained environments, with the inherent challenge of maintaining the human-likeness of machine-learned motion primitives. The lesson learned is the importance of a nuanced and sophisticated approach to achieve the desired human-like motion in the testing context.

4.3. Benchmarking and Evaluation Metrics

As outlined in Section 2.2, identifying suitable evaluation metrics involves considering interaction forces, torques, and pressures. Evaluating interaction forces at a specific point in space on the human side is challenging due to the dynamic nature of the interaction. A potential solution is to assess forces at the exoskeleton, although this necessitates further research on accurate measurement methods. For approaches without fixed attachments, interaction pressure emerges as a more viable metric, measurable through pressure pads, providing deeper insights into interaction dynamics. Models incorporating human morphology (e.g., approaches A and C) offer the potential to evaluate interaction torques within the modeled shoulder frame, serving as an effective metric for assistive shoulder torque. However, the torque measurement results presented, even as they are very preliminary and in a constraint setting, provide already interesting insights such as taking care that the exoskeleton does not work against the desired movement. In summary, the path to comprehensive benchmarking involves further research utilizing a dedicated robotic testing platform. However, the lesson learned from this work is the encouraging demonstration of the feasibility of measuring interaction effects even within a limited testing setting.

4.4. Interaction with Environment for Real Tasks

Throughout the prototype creation process, a primary focus was developing a setup that enables the robot to interact with the environment following the ansatz of mimicking human-like tasks for the study of the effects of assistance in this interaction. The authors still emphasize the importance of this aspect as also discussed with respect to the kinematics above, considering that typical exoskeleton use cases involve interactions with the environment and the processing of forces. Achieving this goal with a human arm dummy appears challenging, as it would require an effector to interact with the environment, introducing limitations and additional complexity. From the presented work, the most effective approach in this context seems to be placing an anthropomorphic robot platform inside the exoskeleton, providing full human-like motion capabilities and allowing the arm end effectors to interact with the environment.

This finding presents a notable contradiction, considering the relative success of the human-arm dummy, which provides a simpler option for creating a realistic contact situation at the interaction point or arm-hold surface. Achieving realism and required versatility poses a high standard for a dedicated robotic platform to demonstrate its utility. The lesson learned here emphasizes the significant trade-offs between design complexity, associated costs, functionality, human likeness, and the efforts required for motion control in resulting complex interacting systems. Finding the optimal platform that satisfies these considerations might not be easily achievable, or such a platform might not even exist.

5. Conclusions

This paper explores different approaches to creating a systematic testing platform for exoskeletons, exemplified by the upper-body exoskeleton “Lucy”. Starting with an analysis and formulation of requirements, it sketches out a respective research program. It reports on different approaches to realize the first robotic test platform prototype based on a standard industrial 7 DoF manipulator—a Franka Emica Panda.

Significant challenges emerged from this work and can be summarized in some of the lessons learned about crucial issues. The mounting of the exoskeleton and the hardware interface design proved essential. Rigidly fixing the exoskeleton to the environment, as observed in all experiments, neglects critical effects arising from relative motion, elastic effects, and forces within the exoskeleton. An improvement might involve a realistic human body dummy, allowing for a better understanding of force transfer, but the kinematics design and human-like motion generation interact in complex ways and require careful consideration of constraints in motion planning. Whether directly putting the robot inside the exoskeleton, attaching it rigidly to it, or utilizing a human-arm dummy, every approach had distinct limitations in achieving the required degrees of freedom for human-like motions. Additional research will enhance our understanding of the intricate trade-offs between design complexity, functionality, and human likeness necessary for the development of an advanced robotic test platform.

In conclusion, achieving a systematic approach to testing upper-body exoskeletons requires a highly specialized robotic platform, potentially in the form of an upper-body torso with fully articulated arms, addressing the complexity of generating necessary complex motions, particularly at the arm interface. Existing robotic platforms, including commercial manipulators and humanoid robots, lack this required degree of dexterity for comprehensive testing, highlighting the need for further development efforts. The conducted experiments demonstrate the promise of this approach, revealing valuable insights into the applied forces in exoskeleton assistance. They also highlight the intricate interplay between robot design, exoskeleton attachment, and human-like motion control, emphasizing the need for careful consideration during the design phase through template and simulation models. Additionally, tailoring the platform to the specific tasks under investigation is likely necessary for optimal results. Despite these challenges, the authors advocate for this investment, believing it is essential for systematically advancing the quality of exoskeleton technology and its positive impact on human well-being.