Next Article in Journal
Trihedral Lattice Towers Optimization with a Limitation on the Resonant Vortex Excitation Occurrence
Previous Article in Journal
Manufacturing Constraints in Topology Optimization for the Direct Manufacturing of Extrusion-Based Additively Manufactured Parts
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Design of Medical Devices with Usability in Mind: A Theoretical Proposal and Experimental Case Study Using the LEPRE Device

1
Department of Mechanical and Industrial Engineering, University of Brescia, Via Branze 38, 25123 Brescia, Italy
2
Polibrixia s.r.l., Via Branze 43, 25123 Brescia, Italy
3
Domus Salutis Rehabilitation Clinic, Teresa Camplani Foundation, Via Lazzaretto 3, 25123 Brescia, Italy
*
Authors to whom correspondence should be addressed.
Submission received: 25 November 2022 / Revised: 27 December 2022 / Accepted: 3 January 2023 / Published: 6 January 2023
(This article belongs to the Section Bioengineering Design)

Abstract

:
Usability is a critical product feature and is required for widespread market adoption. Standards on usability are highly focused on evaluation procedures and specific aspects, such as software issues or human–machine interaction, whereas the relative scientific literature is very normative oriented. The few methodological works dealing with usability either consider it as one of the many attributes that a particular project must satisfy or present very general methods. No design methods systematically oriented toward the integration of usability and usability-related constraints have been developed to date. This paper presents a usability-oriented model for the design of medical devices and its application to the design of LEPRE, a medical device for upper- and lower-limb robotic rehabilitation. Two methods were used to assess the device’s usability: interviews with experts to outline qualitative evaluations and System Usability Scale (SUS) questionnaires on eight physiotherapists, two physiatrists, and 12 patients, enabling a quantitative assessment. Results support the intention of providing an integrated methodological approach to be applied from the early stages of the project, thus saving time and costs, leading to a more linear product development for this application.

1. Introduction

The usability of a product or service, in conjunction with its safety, is a critical feature required for widespread market adoption of the good itself [1], and even more in the field of medical devices. Nonetheless, few scientific works and regulations address usability methodologically, considering it one of the many attributes that the project must satisfy [2], or otherwise proposing very general methods [3].
According to International Organization for Standardization (ISO), usability is defined as “the extent to which a system, product or service can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” [3]. The concept of usability is, therefore, extremely broad, and the scientific literature includes further attributes to the list of ISO parameters, interpreting and offering additional shades of meaning to the ISO definition, such as: ease of use, learnability, flexibility, attitude, and memorability [2,4]. Furthermore, since usability is applicable to both software and hardware, it encompasses a wide range of applications, including apps, websites, computer mice, smartphones and machinery. For this paper, usability is meant as described by the ISO definition.
The scientific literature on usability, particularly in the medical field, is very normative-oriented [5,6,7]. Themes such as usability risk assessment [8], usability reports [9], documentation [10] and protocols [11] are of particular interest. Moreover, most of the articles are software-related, with an emphasis on the human–machine interface [12,13,14], evaluation procedures [15,16,17,18], and numerous but very specific application cases [19,20,21]. For instance, Pei et al. [22] described the usability of a novel robotic bilateral arm rehabilitation device for patients with stroke [22], Surma-aho et al. [23] analyzed the usability issues of operating room devices [23]. Heinemann et al. [24] focused on the usability of medical devices for patients with diabetes who are visually impaired or blind [24]. As the application cases cited above demonstrate, in addition to being circumstantial to a particular medical field (stroke, operating room and diabetes), they often do not cover the entire relative range of design possibilities. For instance, regarding stroke technologies, the first application example [22] is restricted to a robotic bilateral arm rehabilitation device, while the case regarding diabetes [24] is confined to patients who are visually impaired or blind. Apart from the application cases, all the usability-related literature mainly concerns software applications and the human–machine interface so it does not extend beyond what is outlined in the standards. Most of all, it does not address the topic of usability with a methodological approach.
For the European market, the normative framework that governs the usability domain of devices in the medical field is comprised of three references: (i) IEC 62366-1:2015 “Application of usability engineering to medical devices” [25], which describes a process for a manufacturer to analyze, specify, develop, and evaluate the usability of a medical device in terms of safety; (ii) ISO/TR 16982:2002 “Usability methods supporting human-centered design” [26], which provides information on human-centered usability methods that can be used for design and evaluation; and (iii) ISO 9241-11:2018 “Ergonomics of human–system interaction—Part 11: Usability: Definitions and concepts” [3], which specifies a framework for understanding the concept of usability and applies to situations where people use interactive systems, products (including industrial and consumer products) and services. As the last reference provides a global perspective on usability, the first two are primarily concerned with the software domain.
In other words, the normative framework analyzes usability almost as a final output of the design process for a medical device, providing indications, for instance, on its post-hoc evaluation. On the contrary, fewer indications are provided about how to actively manage usability-related aspects during the design evolution itself. In addition, the fulfillment of usability-based requirements could strongly affect the final design of a device, especially in the medical field, and it is a given that every unexpected change applied to a process introduces additional costs to a project, e.g., personnel hours, economic costs, or resources.
Accordingly, several works describe design techniques and methodologies devoted to identifying device requirements [27,28], as well as detecting or anticipating, if possible, project criticalities. Among them, design-for-X [29] and open innovation approaches [30] especially emerged, although not specifically devoted to the integration of usability in the design process.
Other approaches, such as Design Thinking and Human-Centered Design, represent valuable, complementary design tools [31,32], even if they do not properly consider design models, as they are not contextualized into all the various stages of product design. In fact, Design Thinking is a new paradigm for dealing with problems in many professions, most notably Information Technology (IT) and Business [33,34], but does not result in either specifically to the medical field or referable to one or more specific stages of the product design. Likewise, Human-Centered Design is an approach to interactive systems development that aims at making systems usable and useful by focusing on the users, their needs and requirements, and by applying human factors/ergonomics and usability knowledge and techniques [35,36]. Although Human-Centered Design does provide guidance on the key aspects to be considered throughout the entire design process, significantly affecting the definition of requirements, this tool is broad and does not define specific engineering tasks. Therefore, according to the authors’ knowledge, no design methods systematically oriented towards the integration of usability and usability-related constraints at all the phases of the design process have been developed up to date.
In this context, the objective of this study is to propose a usability-oriented model for the design of medical devices based on an extension of the concept of usability, which is intended not only as an outcome, but more significantly, as an integrated methodological approach to be applied during the early stages of the project. Therefore, this approach embraces all the stages of the design process. Involves a more linear product development and allows for reducing time and costs
Furthermore, the innovative aspect of this process is also grounded in its practicality and broad applicability, distinguishing it, for example, from the generic iterative approach proposed by the standards [35].
The usability-oriented model will be discussed by demonstrating its application in the design process of the LEPRE medical device for robotic rehabilitation of the lower and upper limbs. This robotic device has an end-effector-based architecture and can perform rehabilitation treatment in different modalities: passive (CPM), active-assisted, or active with biofeedback. It is composed of three main groups, i.e., the frame, the main body, and the monitor. A compact differential system in the main body enables the implementation of any motion profile in the desired plane. The extended description of the device is given in Section 2.2: Case Study: LEPRE Rehabilitation Device.

2. Method

2.1. Usability-Oriented Model for the Design of Medical Devices

The Usability-Oriented Model for the Design of Medical Devices is represented in Figure 1. This results from more than 15 years of research and analysis of scientific literature, regulatory framework, and practical applications. It is the outcome of two implementation steps that transformed and incrementally improved an original model. This original model, called the “soft open innovation model” [37], has been applied in the last 10 years to the design of the five medical devices depicted in previous literature by the authors [37] and was implemented from two points of view: (a) through a more detailed specification of the actors involved in the various stages of the design process, and (b) with the integration of the regulatory aspect [38]. The result of this first improvement is the model named “Open innovation-inspired model for the design process of medical devices” [39]. From this second model, the role of usability was investigated and contextualized within the design process, obtaining the approach proposed in this work.
According to the Usability-Oriented Model for the Design of Medical Devices, shown in Figure 1, the design process begins with a specific unmet need that requires innovation and progresses through the following phases to the final prototype: “Concept Definition”, “Simulations”, “Prototype”, “On-Field Evaluation”, and “Final Prototype”.
In some cases, the emergence of usability issues itself may stimulate the launch of a new innovative project. Referring to Figure 1, the “Goal” is not included in the usability area (green dashed box) because this is not a condition that always occurs. Nevertheless, usability desiderata can be the reason for defining the “Goal”, i.e., the model’s starting point. For example, suppose products on the market in a given area of use have poor usability with regard to a specific function. In that case, an industrial company could cover that market segment by developing a new innovative solution that satisfies users in terms of usability.
Focusing communication on usability is critical during the “Concept Definition” and, in particular, when identifying “Functional Specifications” of the project by addressing with stakeholders the main features that are involved, such as functionality, ease of use, comfort, flexibility and learnability. Interviews are important for eliciting usability requirements [40], particularly during the “Concept Definition” and subsequent phases. Interviews are more effective when conducted in groups [41,42] because conflicting requirements are reduced. These are primarily used in the search for requirements rather than for a posteriori evaluation, emphasizing the usability paradigm shift once more.
The detailed design and early production phases, represented in the scheme by “Simulations” and “Prototype”, remain primarily engineering-related tasks. As a result, usability improvements are less likely to emerge at these stages. However, there are times when technicians become aware of issues during the detailed design stages of a project, like in the case of previously unknown conflicts of requirements. These issues must be discussed with stakeholders, and as a result, new usability requirements may emerge.
Once the prototype has been created, it must pass a test before being used in the field. This check reveals “unexpressed requirements”, or requirements deemed necessary by a stakeholder but not expressed until that point. In most cases, these requirements are related to usability.
If the prototype passes the test, the next phase is the “On-Field Evaluation”. This is the most important phase after the “Concept Definition” because it validates the prototype’s usability. Typically, if the previous steps of the design process were correctly carried out using the usability-oriented approach; in this case, the usability requirements emerge that do not imply design revisions but rather improvements to the prototype under evaluation.
The “Final Prototype” is the result of passing the “On-field evaluation” check, although the implementation of further minor design changes could still occur also after the “Final Prototype” is completed.
The Beginning-of-Life (BOL) phase concludes with the production of the “Final Prototype”. The usability-validated product is then allowed to proceed through the remaining phases of the product life cycle [43]: the Middle-of-Life (MOL), which includes use and maintenance, and the End-of-Life (EOL), which comprises disposal or recycling.
Devices intended for the European market are partially guided to be compliant with usability since they require the set of mandatory procedures of the CE marking to be freely commercialized within the European market itself. Nevertheless, additional usability requirements that lead to improvements may also emerge at subsequent stages, like in after-sales phases, for example, during the post-market surveillance procedures required by the normative framework

2.2. Case Study: LEPRE Rehabilitation Device

The LEPRE Rehabilitation Device is an end-effector-based robotic device for upper- and lower-limb rehabilitation. It is characterized by a compact differential system, with two degrees of freedom that enable the implementation of any motion profile in the desired plane [44], and can work either through pre-set exercises or with exercises customized ad hoc by the clinician. The device is suitable for rehabilitation in passive modality (Continuous Passive Motion, CPM), active assisted, and active modality with biofeedback. Figure 2 depicts the current CE-marked version of the device, validated for use in clinical practice [45,46].
Rehabilitation of both upper and lower limbs is due to the device’s central body’s ability to tilt and thanks to the interchangeability of accessories, allowing for two setups: upper-limb configuration (Figure 3a) and lower-limb configuration (Figure 3b).
The device can also perform rehabilitation treatment in different modalities: passive (CPM), active-assisted or active with biofeedback.

2.3. Suitability Analysis of Usability Evaluation Methods

Several tools exist to elicit requirements. Especially in relation to the usability field, the standard ISO/TR 16982:2002 [26] is fundamental. This technical report provides an overview of existing usability methods that can be used on their own or in combination to support design and evaluation. Methods presented in the standard include the observation of users, performance-related measurements, critical incidents analysis, questionnaire, interviews, thinking aloud, collaborative design and evaluation, creativity methods, document-based methods, model-based approaches, expert evaluation and automated evaluation. In addition, the standard provides guidelines for choosing the optimal method based on the life-cycle process. Specifically, the standard divides the Life Cycle of a project into three macro phases: acquisition and supply processes, development process, and operation and maintenance processes. Since the model presented in this paper concerns the development phase, only the macro phase of development was considered for choosing the most suitable usability evaluation method. In turn, the development process is divided by the standard into three stages: requirements analysis, architectural design, and qualification testing.
In fact, no linear correspondence between the model presented in this paper and the model proposed by the standard can be established. In fact, the latter defines the stages of architectural design and qualification testing as follows:
  • Architectural design: “During the design phases, usability methods will be implemented to confirm, modify or refine the previous findings”;
  • Qualification testing: “Is the activity where usability methods are applied to test the match with the requirements”.
In the Usability-Oriented Model for the Design of Medical Devices, on the other hand, there is no clear distinction between these two phases because both are evaluated in the various testing phases: the fulfillment of initial requirements is provided by the qualification testing phase of the standard, and the architectural design phase provides the possibility of changes and improvements.
Figure 4 synthesizes the correspondence between the different phase nomenclatures of the standard and the model presented in this paper.
For each of the three stages of development, the standard indicates which usability evaluation methods are most appropriate (see Table 1 for an extract from the standard).
For the requirement analysis phase, the standard identifies the following methods as most recommended (i.e., methods marked “++”): observation of users, questionnaires, interviews, and thinking aloud. The observation of users, in the sense of a collection of information about the behavior and the performance of users, is not feasible in our case as both clinical actors and patients cannot fully interact with the device as a functioning prototype has not yet been produced at this early stage of defining the specifications. For the same reason, thinking aloud, intended as the verbalization by users of their ideas, beliefs, expectations, doubts, and discoveries during their use of the system under test, is not applicable as there is no complete prototype. Moreover, valid questionnaires in these early stages are qualitative and aimed at eliciting opinions on the device. Consequently, interviews are the best tool for evaluating usability in the requirements analysis phase as they allow for greater flexibility than the qualitative questionnaires and enable unexpressed opinions and requirements to be more easily revealed.
Since, for the model presented in this paper, the two phases of architectural design and qualification testing are not separate, a combined evaluation was carried out to understand the best usability evaluation method. The method that would be most suitable for these phases is the performance-related measurements, as it has the higher mark (“++”) for both phases. The commonly used quantifiable performance measurements include time spent to complete a task, number of errors, and number of tasks that can be completed within a predefined duration. It was decided not to adopt this method for the evaluation of usability in these phases because the large number of tasks that can be performed with the device would have made a complete evaluation difficult and because this tool does not allow an evaluation of the overall device but in terms of human–machine interface only.
Consequently, the methods most suited by the standard to assess usability at these stages are questionnaires, interviews, thinking aloud and document-based methods (i.e., those marked with a “++” and a “+”). The latter method was not considered, as it involves pre-existing documentation, which is, however, not available as the first prototype is being evaluated. Interviews were continually held with healthcare personnel, who also reported the patients’ feelings about the device in their care. Interviews also made it possible to replace and supplement the thinking aloud, avoiding specific tests and leaving the clinical actors free to use the device according to instructions for use and clinical practice. Questionnaires are mostly employed in this phase, as they allow a quantitative evaluation of usability concerning the whole device and can be applied to both clinical actors and patients.
Following the instructions of the standard, the tools that emerged as the best ones to support the design and evaluate usability were interviews, especially in the initial stages indicated by the standard under the heading of “Development–Requirements Analysis,” and questionnaires, especially in the final stages indicated by the standard under the heading of “Development–Architectural Design” and “Development–Qualification Testing”. Therefore, in the present work, the Usability-Oriented Model for the Design of Medical Devices has been applied to the LEPRE Rehabilitation Device (Polibrixia s.r.l., Padova, Italy), and interviews and questionnaires were used to evaluate the usability of the device.

2.4. Usability Evaluation Protocol

Two methods available in the scientific literature [6,15] and the standards [26] were used to evaluate the obtained usability of the LEPRE Rehabilitation Device: interviews to outline a first qualitative evaluation and questionnaires to enable a quantitative assessment. The subjects provided the proper consent for the data treatment.
The interviews were conducted with more than 15 physiatrists (at least 45 years old) and physiotherapists (at least 35 years old) with at least 10 years of experience from different rehabilitation clinical facilities, treating patients with acute-phase impairments with orthopedic and neurological genesis by the same trained operator. The interviews were conducted throughout the different stages of the device design. In particular, as the phases progressed, the interview questions went from being open-ended to increasingly specific ones. For example, in the early stages of “Concept Definition”, typical questions were: “What are the rehabilitation modalities that you expect from the device?” or “What are the trajectories that the device should be able to perform?”. Differently, at the end of the prototyping phases, the usual questions were: “Which usability criticalities do you think can be present with the implemented structural shape?” or “Can the usability of the 3D mouse be improved by modifying its position?”. A multidisciplinary team of three engineers analyzed the answers provided by the experts to capture suggestions or desiderata aimed at improving the device’s usability. In the current work, usability desiderata were considered new design constraints and translated into device requirements.
For the quantitative analysis, the System Usability Scale (SUS) [48,49,50] was used, i.e., a survey composed of ten statements with a final score ranging from 0 to 100, which is the most valid and widely used questionnaire, both scientifically and industrially [51]. In particular, the applied SUS totally coincides with the original version proposed by Brooke [48,49,50], but for the translation of the statements in the Italian language. Appendix A presents the detail of the questions used and the formula to evaluate the SUS value. Generally, a usability rating is considered fully acceptable when it exceeds the SUS value of 70: Table 2 collects the adopted convention for interpreting the ratings, compliant with the values proposed by Bangor et al. in [52].
In particular, the SUS questionnaire tool was used to assess the usability of the device at three distinct stages of the development of LEPRE. The device usability was evaluated for the first time after the first prototype was created: this moment is indicated in the following elaborations as T0. As it was impossible to assess patient usability because the device had not been CE marked yet, the questionnaire was distributed to eight physiotherapists and two physiatrists. After the device was CE-marked, a clinical investigation in accordance with ISO 14155:2020 [53] was carried out to evaluate the LEPRE device’s usability on patients. At that moment, as indicated in the following elaborations as T1, the questionnaire was submitted to 12 post-stroke patients, both ischemic (50%) and hemorrhagic (50%), and once again to the clinical operators: the same eight physiotherapists and the same two physiatrists. The SUS evaluation was then repeated on the same patients at the end of the second therapy session with the device to capture possible evolution in the evaluation. This moment is indicated in the following elaborations as T2. For both the questionnaires, the SUS values were analyzed with descriptive statistics, and in particular, computing minimum and maximum values (min and max), as well as mean and standard deviation (SD) values, of the data samples (combined in mean ± SD). In addition to the exposed parameters, the p-value calculated according to the Mann–Whitney U-test with a level α = 0.05 is also reported to analyze statistical consistency.
In addition to the overall SUS values as a function of groups of participants (physiotherapists, physiatrists, and patients), which represent the primary endpoints, SUS results were also analyzed by considering the two components of which it is composed: Usability (SUS(U)) and Learning (SUS(L)). In fact, among the 10 questions of which the SUS questionnaire is constituted, in accordance with what is reported in the literature [22], eight questions (Q1, Q2, Q3, Q5, Q6, Q7, Q8, Q9) are intended to express an evaluation in the usability domain (SUS(U)) and two (Q4, Q10) in the learning domain (SUS(L)). Appendix A presents the detail of the formula to evaluate the value of the SUS components: Usability (SUS(U)) and Learning (SUS(L)).
Moreover, the Mann–Whitney U-test and single-sample t-test with significance level α = 0.05 are applied to compare the physiotherapists’ scores to the average SUS score obtained by Sauro [54], who used a population of 5000 different products that have been sold in the market.
Regarding the quantitative analyses, the flowcharts of the activities conducted to obtain the resulting SUS values are presented in Figure 5 and Figure 6.
Table 3 summarizes the descriptive data of the participants involved in the study.

3. Results

The Usability-Oriented Model for the Design of Medical Devices was adopted to develop the LEPRE Rehabilitation Device (Polibrixia s.r.l., Padova, Italy).
The following are examples of changes, improvements, and implementations that occurred in the LEPRE design process due to implementing the usability-oriented method. These are usability requirements, in addition to those prescribed by regulations, that results from the extension of the concept of usability foreshadowed by the model’s paradigm change. The following examples are graphically depicted in Figure 5: Carter Shape, Structural Shape, Mirror Function, and Mouse 3D Height. The figure also indicates the suitability of the proposed model with respect to the project timeline, indicating at which stage of the LEPRE design process the usability changes were performed.
Furthermore, usability evaluation tests were conducted at various stages of the device development to assess the usability achieved thanks to the application of the usability-oriented model. The moments when usability measurements were performed, i.e., T0, T1 and T2, are shown in blue in Figure 7.

3.1. Carter Shape

The original LEPRE Carter was designed to have the shape depicted in Figure 8a to maximize the internal spaces. However, during the early stages of the “Concept Definition”, the following usability requirement emerged from interviews with doctors and physiotherapists: (u1) the device must allow patients to lean on its frontal part (circled in green in the figure) during the rehabilitation therapy on the upper limbs. This necessity does not arise from a specific pathology but might arise because of a particular physical condition, prominent breasts, or because of the set law of motion, which involves reaching the device workspace most distal from the patient’s trunk (a condition to be avoided in the exercise setting).
The newly designed LEPRE carter (Figure 8b) presents a radius of curvature of 90 mm, in contrast to the initially planned 8 mm. In this way, through simple geometric considerations, it turns out that the reduction of the interior space (considering the four corners of the crankcase) results to be about 6900 mm2.
The carter usability requirement impacted aesthetics, safety, mechanical and electronic structure and device market share.
The first affected aspect is the aesthetics. For the carter to be useful, the radii of curvature must be much more pronounced.
Instead, safety is strengthened on two fronts. First, mechanical safety improves, as more pronounced radii of curvature allow for safer and more comfortable support of the patient’s trunk (Figure 9). Above all, the new usability requirement necessitates the relocation of the emergency push bottom (element in red in Figure 8), which would otherwise be difficult to access by the operator (physician or physiotherapist) in the event of an emergency, especially when the device is in the lower-limb rehabilitation therapy configuration.
Furthermore, the new usability requirement affected the mechanical and electronic structure within the carter, particularly in terms of overall dimensions. More pronounced curvature radii, the carter’s longitudinal dimensions being equal, imply a reduction in the space available for the internal mechanical and electronic structure.
Finally, thanks to the possibility of the carter supporting the trunk, the inclusion criteria of the device are expanded, allowing even patients who need frontal support to perform upper-limb rehabilitation therapies with the device.

3.2. Structural Shape

The Structural Shape of the LEPRE chassis was initially designed, as represented in Figure 10a, to maximize the device stability. After the first prototype was built, the primary usability requirement from clinician interviews was: (u2) the ability to perform onoliteral rehabilitation therapies laterally with respect to the device’s central body. This unexpressed requirement was translated into the possibility of positioning the patient on the wheelchair at the side of the device. For example, if the patient needed to perform rehabilitation exercises only on the right limb, they would position themselves to the left of the device’s central body. This functionality was not possible with the original structural form. The structure currently implemented in the CE-marked device (Figure 10b) was designed to fulfill the new requirement.
The new usability requirement for the LEPRE structural shape influenced aesthetics, mechanical structure, safety, and electronic and software features.
Due to the revision of the structural shape of the chassis, the device aesthetics have been drastically altered.
New load analyses and structural simulations were required from a mechanical structural standpoint to develop a new reliable, and robust solution.
Furthermore, the device’s stability must be tested to ensure safety according to standards [55]. As a result, new stability tests were required.
However, the new requirement had the greatest impact on electronics and software functionality. In fact, it was necessary to implement the device and the accessories of new electronic components capable of recognizing the working side of the device and implementing all relevant software features. It was mandatory, for example, to prevent the setting of a bilateral rehabilitation exercise when only one tool is mounted or to set a rehabilitation exercise on the incorrect side.

3.3. Mirror Function

During the “on-field evaluation”, the following new usability requirement emerged: (u3) the clinical operator must be able to perform the device movements when setting the trajectory foreseen by the rehabilitation exercise according to their position (relative reference system) rather than the device absolute reference system. The mirror function (Figure 11) was implemented to meet this requirement. This function allows users to mirror motion commands, invert the clockwise and anticlockwise sense, and change the horizontal translation direction on both the Mouse 3D and the virtual console.
The new usability requirement for the Mirror Function has changed how the software operates and the device’s safety.
The first domain affected by the new requirement was undoubtedly software, as it was necessary to implement the previously unanticipated Mirror Feature.
However, the most significant impact was on device safety, particularly in the medical field. The new function implies that the clinical operator better manages the fundamentals for the setting of robotic rehabilitation therapies. The new function prevents the device from unwanted and potentially dangerous motions that could harm the patient.

3.4. Mouse 3D Height

Although the device was successfully operating at full capacity in the hospital after receiving the CE mark, the periodic interviews revealed a new usability requirement: (u4) Mouse 3D Height must be increased by a few centimeters. Therefore, a minor design review was required (Figure 12), based on which the height of the Mouse 3D was raised by 40 mm.
The new usability requirement for Mouse 3D Height affected both the mechanical aspect and the ease of use.
A minor design review was performed to meet the requirement, resulting in mechanical modifications to the monitor support.
However, this new usability requirement’s major impact was improving the ease and comfort of using the Mouse 3D.

3.5. Usability Evaluation

As expected, the performed interviews did not reveal quantitative values to be analyzed but rather qualitative suggestions and requirements regarding functionality and usability to be implemented in the device as modifications, improvements, or new functions. In particular, it was possible to elicit requirements that involved: the modification of the Carter Shape (u1) and the Structural Shape (u2); the improvement of the usability of the Mouse 3D (u4); and the implementation of the new Mirror Function (u3). In addition, the involved changes impact multiple areas of influence, from Aesthetics to Safety.
Concerning the quantitative analyses, Appendix B presents the collected data’s details. Instead, the SUS results elaborated for groups of participants (physiotherapists, physiatrists, and patients), i.e., the primary endpoints, are shown in Table 4.
In addition, secondary endpoints are shown in Table 5: that is, the components of SUS, Usability and Learning.
The mean SUS score incurs a statistically significant change (p = 0.028) from 75.00 ± 11.48 to 85.00 ± 9.35 for patients and from 64.38 ± 12.30 to 78.13 ± 9.98 for physiotherapists. In contrast, for physiatrists, the change in mean SUS score from 85.00 ± 3.54 to 86.25 ± 1.77 is not statistically significant (p = 0.667).
The mean value of the usability component SUS(U) always increased, but the change is statistically significant only for patients and physiotherapists. Indeed, SUS(U) increased from 77.34 ± 10.92 to 86.56 ± 8.87 (p = 0.039) for patients, from 63.67 ± 11.80 to 78.91 ± 8.64 (p = 0.010) for physiotherapists, and from 82.81 ± 2.21 to 90.63 ± 4.42 (p = 0.333) for physiatrists. On the other hand, the mean value of the learning component SUS(L) increased only for patients and physiotherapists, and in all cases, the changes were not statistically significant. The SUS(L) increased from 65.63 ± 17.78 to 79.17 ± 15.39 (p = 0.114) for patients, from 67.19 ± 22.10 to 75.00 ± 21.13 (p = 0.442) for physiotherapists and decreased from 93.75 ± 8.84 to 68.75 ± 8.84 (p = 0.333) for physiatrists.
In addition, the Mann–Whitney U-test and single-sample t-test comparing the physiotherapists’ score to the average SUS score obtained by Sauro revealed a significant difference: p_Mann = 0.007, p_Ttest = 0.005, with the LEPRE mean SUS 12.07 higher.

4. Discussion

The presented Usability-Oriented Model for the Design of Medical Devices addresses usability methodologically, overcoming the limitations revealed by the analysis of scientific literature and regulations. To apply this method, no specific conditions must be a priori met, so we expect that the model could be easily extended to other contexts or, in general, to medical product development, provided the proper fine adaptations. Indeed, the involved stakeholders could greatly vary, depending on the product’s function and the main entailed analysis aspects. For surgical devices, the focus will be, for instance, on the grip and/or the movement ability, whereas the elements evaluated in the case of the LEPRE device are expected to be shared among electromedical devices for rehabilitation. Nonetheless, the continuous interaction provided by the proposed method allows for the integration of the different visions of the various stakeholders, identifying solutions that represent a trade-off between the technical needs indicated by designers and the demands of medical stakeholders.
As the proposed model emphasizes, the usability aspect encompasses the entire development process and regulation framework. The paradigm shift introduced by this model corresponds to the extension of the notion of usability, intended as an integrated methodological approach to be applied since the early stages of the project rather than as an outcome or a simple attribute, as it is commonly seen in literature or in normative contexts. Furthermore, within this model, the concept of usability has been broadened to all elements of the product or service being designed rather than being limited to software aspects or the human–machine interface.
Finally, the scientific literature and standards cover the topic of usability in a broad sense. For example, ISO 9241-210:2019 proposes a general design approach based on iterative repetition of four steps: (i) understanding and specifying the context of use; (ii) defining user requirements; (iii) producing design solutions; and (iv) evaluating the design. In contrast, the usability-oriented model proposed in this paper contextualizes and specifies the influence of usability in the typical phases of a medical device design process.
Furthermore, the introduced paradigm shift requires that usability represents both the goal to be achieved and the guideline as well of the whole design process and product development. In these terms, compared to the models in the literature for both design process, such as the model of Pahl and Beitz [56], and product development, like the one described by Ulrich and Eppinger [57], the proposed usability-oriented model presents two main peculiar characteristics: (a) details the phases of the design process and product development for the medical field, showing the involved actors (Users, Physicians, Operators and Engineers), and (b) emphasizes the need to focus throughout the development of the device, in addition to the regulations, on usability. Moreover, the application of the usability-oriented model enables the identification of all the usability requirements in reference to a particular functional group but does not exclude the possibility of applying the approaches indicated by the literature in the process of design and development of its constituent components. For example, once the shape of the Carter has been determined thanks to the usability requirements that emerged, the design and development of its components can follow the phases indicated by the method of Pahl and Beitz (Concept, Embodiment and Detail) or those indicated by Ulrich and Eppinger (Planning, Concept Development, System-Level Design, Detail Design, Testing and Refinement). However, without the definition of the usability requirements resulting from the application of the usability-oriented model, the approaches of the literature would generate products that work and perform well but could reveal to be usability non-compliant.
Compared to other design support tools existing in the scientific literature, such as Design Thinking and Codesign, the proposed model presents strong affinities and, in some aspects, embodies them. In fact, the five stages of the design thinking defined by the Hasso Plattner Institute of Design at Stanford [58] (empathize, define, ideate, prototype, and test) or the four ones of the typical design thinking framework known as Double Diamond [59] (Discover, Define, Develop and Deliver) reasonably reflect the macro phases of the proposed usability-oriented model (“Concept Definition”, “Simulations”, “Prototype”, “On-Field Evaluation”, and “Final Prototype”). At the same time, the principle of involving end users in the design established by the Codesign paradigm [60] finds correspondence in the usability-oriented model in making explicit the actors involved in the individual stages and in dedicating extensive attention to constructive confrontations, e.g., through interviews and questionnaires, involving various stakeholders throughout the design process.
However, both Design Thinking and Codesign are not contextualized to the various stages of product design. Furthermore, even the integration of the two paradigms [61] does not resolve the weaknesses of the two individual approaches since it results in the introduction of co-design sessions in models composed of generic steps (seeing, knowing, thinking, acting and reflecting) that are not related to the stages that occur in industrial practice. In contrast, the presented usability-oriented model overcomes the two paradigms, although it implements their principles. In fact, whereas the frameworks described by Design Thinking are general, the proposed model details the specific stages; for example, by distinguishing between functional and technical specifications in the “Concept Definition” or by differentiating the “Simulations” from the “Prototype” in the detail stage.
Moreover, in contrast to Codesign, the usability-oriented model not only indicates the importance of end-user involvement but also delineates the stakeholders involved in the individual stages of the design process. For example, there are phases, such as in the definition of “Functional Specifications”, where the involvement of patients (users), operators, and physicians is essential. In contrast, other phases, such as “Simulations” or “Prototype,” remain primarily engineering-related tasks. In addition, compared to the two paradigms, the presented model explicitly introduces the influence of usability and regulations affecting all stages of design, anticipating various critical issues and encouraging the arising of unexpressed requirements.
Therefore, the novelty and usefulness of the presented usability-oriented model consist of the extension of the concept of usability applied to a detailed design model in which the various actors involved in the different stages are also made explicit. More specifically, the novelty lies in the paradigm shift of considering usability not as an outcome but systematically throughout the various stages of the design process. The usefulness lies in the contextualization of this paradigm shift using a practical and specific design model that presents the various phases pursued in the medical design practice. Table 6 summarizes the comparison with respect to the presented paradigms and methods.
With specific reference to the LEPRE medical device application case, the adoption of the Usability-Oriented Model for the Design of Medical Devices highlighted several benefits. The qualitative investigation performed through informal interviews with experts emphasized suggestions for design improvement, as the analyzed usability requirements.
Quantitative and statistical analyses could be useful tools to assess the changes and improvements that have been described. However, these evaluations cannot be applied if no additional measurable data is introduced. In our case study, the Carter Shape improvement, where the numerical values of the radii of curvature were given, and the Mouse 3D Heigh modification, where the specific value of the uplift was given, were the only cases suitable for analyses based on the numerical evaluation. In fact, the case of the Structural Shape resolves in a radical change in shape; therefore, no values are meaningful for numerical comparison, while the Mirror Function is a newly introduced function, which thus has no comparison term. In addition, it would have been useful to punctually compare results in terms of usability evaluation before and after each change to understand its impact through differentials. However, this was not possible because the best design support and usability assessment tool, according to the standard ISO/TR 16982:2002, was in the early stages of the device development interviews. Nonetheless, these tools did not provide quantitative results to be analyzed but rather qualitative suggestions and requirements regarding functionality and usability to be implemented in the device as modifications, improvements, or new functions.
Table 7 summarizes the areas affected by these requirements, detailing the nature of the modification required for each field.
Table 6 shows for each usability requirement the impacted areas, specifying their nature where appropriate. For example, focusing on the “Mechanical field”, the first two usability requirements ((u1) and (u2)) affected the “Mechanical Structure”, while the modification regarding the Mouse 3D (u4) introduced “Minor Design Review”. At the same time, in terms of the “Electronic field”, the revision regarding the Carter (u1) changed the “Electronic Structure”, while the Structural Shape (u2) concerned “Electronic Features”. Similarly, in the “Other fields,” the usability requirements related to the Carter (u1) expanded the “Market Share”, while changing the height of the Mouse 3D improved the “Ease of Use”. For “Aesthetic field”, “Safety field” and “Software field”, no further detail about the impact area was provided as it would not carry further relevant information. In addition to specifying the impacted areas, the table also provides an overview of which specific fields were impacted. Table 6 also depicts the broadening of the concept of usability since the presented usability requirements are not limited to influencing the software area or the human–machine interface
As a general outcome, time-to-market and project costs are significantly reduced, as critical design issues can be anticipated and emerging usability requirements promoted thanks to the usability paradigm shift. Table 6 reveals that the earlier emerging usability requirements are also the most impactful in terms of the number of affected fields and relative impact on the overall development process.
As a result, the number of pre-validation prototypes is reduced, and the design process follows a linear path, as evidenced by the increase in SUS score recorded for all categories of participants involved. The analysis of SUS results shows, as summarized in Table 4 that for physiotherapists, an increase from 64.38 ± 12.30 to 78.13 ± 9.98 (from T0 to T1), for physiatrists from 85.00 ± 3.54 to 86.25 ± 1.77 (from T0 to T1), and for patients from 75.00 ± 11.48 to 85.00 ± 9.35 (from T1 to T2). In particular, for physiotherapists and patients, there is statistical consistency as the p-values recorded, in both cases, p = 0.028, are such as to reject the null hypothesis. On the contrary, there is no statistical consistency in the case of physiatrists as the p-value is found to be p = 0.667 and, therefore, not such as to reject the null hypothesis.
These last statistical results are besides strongly affected by the small sample size available for physiatrists, which corresponds to only two participants. In addition, Table 4 also reveals that for all three categories of participants, standard deviation decreases and minimum recorded values increase between consecutive measures of SUS. These are two very significant results since a reduction in standard deviation indicates greater agreement in usability assessment. The increase in the minimum recorded values means that an increasing number of participants consider the device to be at least acceptable in terms of usability. In fact, for physiotherapists, the standard deviation decreased from 12.30 to 9.98 (from T0 to T1), for physiatrists from 3.54 to 1.77 (from T0 to T1), and patients from 11.48 to 9.35 (from T1 to T2).
On the other hand, in terms of minimum value, it increased from 40.00 to 60.00 for physiotherapists (from T0 to T1), shifting from a usability between “Poor” and “OK” to a usability between “OK” and “Good”, increased from 82.50 to 85.00 for physiatrists (from T0 to T1), but still reg at an almost “Excellent” level of usability, and increased from 60.00 to 75.00 for patients (from T1 to T2), upgrading from a usability between “OK” and “Good” to a usability between “OK” to “Excellent”.
After implementing all the improvements that emerged from the application of the presented usability model, the CE-marked device (at instant T1) shows a usability level between “Good” and “Excellent” for the three categories of participants, which is definiable. Moreover, the SUS value emerging for physiotherapists and patients is rather close, whereas it is significantly higher in the case of physiatrists—this is the reason for the different uses of the device by the categories. Indeed, different users have different roles; thus, their usability scores provide different indications. In the case of the LEPRE device, physiatrists are primarily responsible for setting up the rehabilitation protocol, physiotherapists deal with setting up the exercise (defining the trajectories and the remaining parameters), whereas the patients perform the exercises guided by the on-screen instructions and feedback from the device. Therefore, SUS values at T1 indicate that the usability of the CE-marked device is acceptable with respect to rehabilitation protocol and exercise setting and exercise execution.
The results of the SUS components always increase their value, except in the case of physiatrists. In fact, the SUS(U) increased from 82.81 ± 2.21 to 90.63 ± 4.42; in contrast, the SUS(L) decreased from a value of 93.75 ± 8.84 to 68.75 ± 8.84 in the case of physiatrists. Therefore, the device is perceived as more complex to use. Nevertheless, the SUS score measured for physiatrists grew due to the increase of the SUS(U) component.
Ultimately, the SUS measures involving clinical stakeholders (physiotherapists and physiatrists) indicate the influence on the device usability evaluation of the changes introduced through the application of the usability-oriented model, whereas the SUS measures on patients reveal how much using a device multiple times affects the overall device usability evaluation. Given the numerical results, the introduced changes have improved the device’s usability, which grows further at a second session.
Furthermore, the results of the Mann–Whitney U-test and single-sample t-test comparing the physiotherapists’ score to the average SUS score obtained by Sauro indicate that the usability of the LEPRE device is statistically different from those of most market products, with LEPRE mean SUS higher of 12.07. Moreover, implementing the usability-oriented model has allowed for improvements or the introduction of new functions, influencing the mechanical, electronic, and software aspects and enabling the increase of the potential device market share.
More specifically, at the beginning of the project, three prototypes were planned to be realized before achieving the final one. However, a single, final prototype was produced thanks to the application of this model, whose changes impacted economically as one prototype. Therefore, the estimated economic saving compared to the forecast is 33%, corresponding to the saving of an intermediate prototype. On the other hand, the greatest advantage is in the product development timeline since this method saves time. Although no specific evaluation has been performed to quantify the overall time saving, this aspect can be concretely assessed, considering that only one prototype was made thanks to the early modifications. Half of the time required to design and make a prototype is estimated to be the impact of the early modifications. Consequently, the estimated time saving compared to the forecast is 50%, corresponding to the saving of one-and-a-half intermediate prototypes. The reported percentages are estimations, as there is no comparison term; that is, no comparison device is designed without the aid of the Usability-Oriented Model. Furthermore, the proposed estimates are for the specific case under consideration. However, economic savings of 10% to 40% and time-saving of 20% to 60% are estimated through the application of the model. These estimations could be demonstrated and further investigated in a specific study on the design of different devices and possibly in different fields of application.

5. Conclusions

The Usability-Oriented Model for the Design of Medical Devices presented in the current paper reduces project time and cost but also leads to linear product development. The novelty introduced by the model consists in the extension of the notion of usability, intended as an integrated methodological approach to be applied since the early stages of the project rather than as an outcome or a simple attribute, and in its practicality and broad applicability.
The LEPRE design applicative case demonstrates some of its potentiality. In fact, through the usability paradigm shift, many criticalities of the project have been anticipated, and several unexpressed requirements have emerged. For the application case, the efficiency of the model emerges from the economic-temporal saving values (33% in economic terms and 50% in temporal ones).
Future research could focus on analyzing the influence of usability in the post-CE-marking phases; in this work, it was only mentioned as the focus was on the design process. Specific studies could also be developed to better investigate and validate the reported values of time-economic saving resulting from the application of the proposed model with different application cases and in different application areas.
The various components that constitute SUS could also be investigated in future research, introducing and investigating new metrics, even more specifically, such as measuring usefulness, perceived benefit, comprehensiveness, interpretability and ease of use of the presented usability-oriented methodology.
Finally, although the presented model is centered on medical device design, its applicability could be easily extended to other fields, or further investigations could be performed within the same sector by broadening the involved stakeholders.

Author Contributions

Conceptualization, A.B., C.A. and R.F.; methodology, A.B., C.A. and R.F.; validation, L.B., M.M. and R.F.; formal analysis, A.B., C.A. and R.F.; investigation, M.M., L.B. and R.F.; resources, M.M., L.B. and R.F.; data curation, A.B., M.M. and R.F.; writing—original draft preparation, A.B., C.A. and R.F.; writing—review and editing, A.B. and C.A.; project administration, C.A., M.M. and L.B. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki and approved by the Ethics Committee of the Brescia province (np 3123–Studio LEPRE).

Data Availability Statement

Not applicable.

Acknowledgments

RF acknowledges Polibrixia Srl for the partial funding of the PhD scholarship.

Conflicts of Interest

A.B. and M.M. declare themselves shareholders of Polibrixia s.r.l.

Appendix A

This appendix contains the statements of the SUS questionnaire in the English version and the Italian translation (Table A1). The Italian translation complies with previous application examples and the suggestions provided by the local authority for the public health system, Regione Lombardia. The subject must answer each question with an integer number between 1 and 5.
Table A1. Statements of the SUS questionnaire.
Table A1. Statements of the SUS questionnaire.
NumberLanguageStatement
S1EnglishI think that I would like to use this system frequently
ItalianPenso che mi piacerebbe utilizzare questo Sistema frequentemente
S2EnglishI found the system unnecessarily complex
ItalianHo trovato il sistema complesso senza che ce ne fosse bisogno
S3EnglishI thought the system was easy to use
ItalianHo trovato il sistema molto semplice da usare
S4EnglishI think that I would need the support of a technical person to be able to use this system
ItalianPenso che avrei bisogno del supporto di una persona già in grado di utilizzare il sistema
S5EnglishI found the various functions in this system were well integrated
ItalianHo trovato le varie funzionalità del sistema bene integrate
S6EnglishI thought there was too much inconsistency in this system
ItalianHo trovato incoerenze tra le varie funzionalità del sistema
S7EnglishI would imagine that most people would learn to use this system very quickly
ItalianPenso che la maggior parte delle persone potrebbero imparare ad utilizzare il sistema facilmente
S8EnglishI found the system very cumbersome to use
ItalianHo trovato il sistema molto macchinoso da utilizzare
S9EnglishI felt very confident using the system
ItalianHo avuto molta confidenza con il sistema durante l’uso
S10EnglishI needed to learn a lot of things before I could get going with this system
ItalianHo avuto bisogno di imparare molti processi prima di riuscire ad utilizzare al meglio il sistema
The formula for calculating the amount of SUS from the values of individual responses to the statements is presented in relation (A1).
S U S = n = 0 4 S 2 n + 1 1 + n = 1 5 5 S 2 n · 2.5
In this relation, n is the counter for the question number, and Sn is the value assigned by the subject as answer to the specific question. The formula does not merely sum the results of all the questions in the same way, since the numerical values assume a different meaning for odd and even questions: for odd statements, the higher the value, the better the feedback, and the other way around for the even questions.
The formulas for calculating the amount of the Usability component, SUS(U), and of the Learning one, SUS(L), of the SUS from the values of individual responses to the statements, are presented in relations (A2) and (A3), respectively.
S U S U = n = 0 4 S 2 n + 1 1 + n = 1 5 5 S 2 n 5 S 4 + 5 S 10 · 2.5 · 10 8
S U S L = 5 S 4 + 5 S 10 · 2.5 · 10 2

Appendix B

In this appendix, the detail of the results of the applications of the SUS questionnaire to the LEPRE device are presented.
In particular, they collect the evaluations from 8 physiotherapists at T0 (Table A2) and T1 (Table A3), 2 physiatrists at T0 (Table A4) and T1 (Table A5), 12 post-stroke patients at T1 (Table A6) and T2 (Table A7). Color scales are added to the original values to ease the data interpretation. Legend for the adopted color scales is provided at the end of each table.
Table A2. Results of the application of the SUS questionnaire to eight physiotherapists at T0.
Table A2. Results of the application of the SUS questionnaire to eight physiotherapists at T0.
S1S2S3S4S5S6S7S8S9S10SUSSUS(U)SUS(L)
P0133252343234043.7525
P023113211121555075
P0332434132427071.87562.5
P04423342423267.568.7562.5
P05414132313177.571.875100
P0642321141317065.62587.5
P0731323222336059.37562.5
P0851434352327578.12562.5
Legend for Color Scales:
S2n12345
S2n+112345
SUS0102030405060708090100
Table A3. Results of the application of the SUS questionnaire to eight physiotherapists at T1.
Table A3. Results of the application of the SUS questionnaire to eight physiotherapists at T1.
S1S2S3S4S5S6S7S8S9S10SUSSUS(U)SUS(L)
P0142243242436065.62537.5
P02424341224172.571.87575
P0342324151337578.12562.5
P0431524142518584.37587.5
P0551515141439093.7575
P06525142425187.584.375100
P07414332424272.57562.5
P08424132525182.578.125100
Legend for Color Scales:
S2n12345
S2n+112345
SUS0102030405060708090100
Table A4. Results of the application of the SUS questionnaire to two physiatrists at T0.
Table A4. Results of the application of the SUS questionnaire to two physiatrists at T0.
S1S2S3S4S5S6S7S8S9S10SUSSUS(U)SUS(L)
P01414141414187.584.375100
P02413242525182.581.2587.5
Legend for Color Scales:
S2n12345
S2n+112345
SUS0102030405060708090100
Table A5. Results of the application of the SUS questionnaire to two physiatrists at T1.
Table A5. Results of the application of the SUS questionnaire to two physiatrists at T1.
S1S2S3S4S5S6S7S8S9S10SUSSUS(U)SUS(L)
P01515151414487.593.7562.5
P0251424252528587.575
Legend for Color Scales:
S2n12345
S2n+112345
SUS0102030405060708090100
Table A6. Results of the application of the SUS questionnaire to 12 post-stroke patients at T1.
Table A6. Results of the application of the SUS questionnaire to 12 post-stroke patients at T1.
S1S2S3S4S5S6S7S8S9S10SUSSUS(U)SUS(L)
P0152433342446571.87537.5
P0242334233336062.550
P034242424242757575
P045151515151100100100
P05524433324362.568.7537.5
P064242424242757575
P07515252325382.587.562.5
P08514343524277.581.2562.5
P09433343434262.562.562.5
P1051434241518587.575
P1152424352528081.2575
P124242424242757575
Legend for Color Scales:
S2n12345
S2n+112345
SUS0102030405060708090100
Table A7. Results of the application of the SUS questionnaire to 12 post-stroke patients at T2.
Table A7. Results of the application of the SUS questionnaire to 12 post-stroke patients at T2.
S1S2S3S4S5S6S7S8S9S10SUSSUS(U)SUS(L)
P01524243514377.581.2562.5
P02515342524282.587.562.5
P035151515151100100100
P045151515151100100100
P0552523242418078.12587.5
P064242424242757575
P07515252325382.587.562.5
P08514343524277.581.2562.5
P09425142414282.581.2587.5
P105151515151100100100
P1152424352528081.2575
P12425241414282.584.37575
Legend for Color Scales:
S2n12345
S2n+112345
SUS0102030405060708090100

References

  1. Vincent, C. Product development—Safety and usability of medical devices. In Contemporary Ergonomics and Human Factors 2013; Taylor & Francis: London, UK, 2013; pp. 127–130. [Google Scholar]
  2. Shackel, B. Usability—Context, framework, definition, design and evaluation. Interact. Comput. 2009, 21, 339–346. [Google Scholar] [CrossRef]
  3. ISO 9241-11:2018; Ergonomics of Human-System Interaction—Part 11: Usability: Definitions and Concepts. ISO: Geneva, Switzerland, 2018.
  4. Becerril, L.; Stahlmann, J.-T.; Beck, J.; Lindemann, U. Usability of processes in engineering design. In Proceedings of the 21st International Conference on Engineering Design (ICED17), Design Processes, Design Organisation and Management, Vancouver, BC, Canada, 21–25 August 2017; Volume 2. [Google Scholar]
  5. Roma, M.S.G.; de Vilhena Garcia, E. Medical device usability: Literature review, current status, and challenges. Res. Biomed. Eng. 2020, 36, 163–170. [Google Scholar] [CrossRef]
  6. Bitkina, O.V.; Kim, H.K.; Park, J. Usability and user experience of medical devices: An overview of the current state, analysis methodologies, and future challenges. Int. J. Ind. Ergon. 2020, 76, 102932. [Google Scholar] [CrossRef]
  7. Braun, S. Usability for medical devices. In Proceedings of the IEEE Symposium on Product Safety Engineering, 2005, Schaumburg, IL, USA, 3–4 October 2005; pp. 16–22. [Google Scholar] [CrossRef]
  8. Ravizza, A.; Lantada, A.D.; Sánchez, L.I.B.; Sternini, F.; Bignardi, C. Techniques for usability risk assessment during medical device design. In Proceedings of the BIODEVICES 2019—12th International Conference on Biomedical Electronics and Devices, Proceedings; Part of 12th International Joint Conference on Biomedical Engineering Systems and Technologies, BIOSTEC 2019, Prague, Czech, 22–24 February 2019; pp. 207–214. [Google Scholar] [CrossRef]
  9. Bevan, N.; Carter, J.; Earthy, J.; Geis, T.; Harker, S. New ISO standards for usability, usability reports and usability measures. In Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Springer: Berlin/Heidelberg, Germany, 2016; Volume 9731, pp. 268–278. [Google Scholar] [CrossRef]
  10. Femberg, S.G.; Feinberg, B.N. The role of usability testing and documentation in medical device safety. In Proceedings of the 2001 Conference Proceedings of the 23rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Istanbul, Turkey, 25–28 October 2001. [Google Scholar]
  11. Schmettow, M.; Schnittker, R.; Schraagen, J.M. An extended protocol for usability validation of medical devices: Research design and reference model. J. Biomed. Inform. 2017, 69, 99–114. [Google Scholar] [CrossRef] [PubMed]
  12. Bras Da Costa, S.; Beuscart-Zéphir, M.C.; Bastien, J.M.C.; Pelayo, S. Usability and safety of software medical devices: Need for multidisciplinary expertise to apply the IEC 62366: 2007. In Studies in Health Technology and Informatics; IOS Press: Amsterdam, The Netherlands, 2015; Volume 216, pp. 353–357. [Google Scholar] [CrossRef]
  13. Hegde, V. Role of human factors usability engineering in medical device design. In Proceedings of the Proceedings Annual Reliability and Maintainability Symposium (RAMS), Orlando, FL, USA, 28–31 January 2013; pp. 1–5. [Google Scholar] [CrossRef]
  14. Ivory, M.Y.; Hearst, M.A. The State of the Art in Automating Usability Evaluation of User Interfaces. ACM Comput. Surv. 2001, 33, 470–516. [Google Scholar] [CrossRef]
  15. Keogh, A.; Argent, R.; Anderson, A.; Caulfield, B.; Johnston, W. Assessing the usability of wearable devices to measure gait and physical activity in chronic conditions: A systematic review. J. NeuroEng. Rehabil. 2021, 18, 138. [Google Scholar] [CrossRef]
  16. Kim, T. Factors influencing usability of rehabilitation robotic devices for lower limbs. Sustainability 2020, 12, 598. [Google Scholar] [CrossRef] [Green Version]
  17. Donald, D.; Saliba, M.A. Addressing simplicity, dexterity and usability of compact, multi-degree-of-freedom mechatronic devices. In Proceedings of the 2016 IEEE International Conference on Advanced Intelligent Mechatronics (AIM), Banff, AB, Canada, 12–15 July 2016. [Google Scholar]
  18. Kushniruk, A.W.; Patel, V.L. Cognitive and usability engineering methods for the evaluation of clinical information systems. J. Biomed. Inform. 2004, 37, 56–76. [Google Scholar] [CrossRef] [Green Version]
  19. Kwak, H.; Oh, H.; Cha, B.; Kim, J.M. The assessment of usability of pain medical device by physiatrists and physiotherapists A Delphi survey. Medicine 2021, 100, e27245. [Google Scholar] [CrossRef]
  20. Tosi, F.; Rinaldi, A. Design and Usability of the Next Medical Devices for the Home Care. Des. J. 2017, 20, S2033–S2043. [Google Scholar] [CrossRef] [Green Version]
  21. Campoe, K.R. Medical Device Usability Analyses. Proc. Int. Symp. Hum. Factors Ergon. Health Care 2013, 2, 123–130. [Google Scholar] [CrossRef]
  22. Pei, Y.C.; Chen, J.L.; Wong, A.M.K.; Tseng, K.C. An evaluation of the design and usability of a novel robotic bilateral arm rehabilitation device for patients with stroke. Front. Neurorobotics 2017, 11, 36. [Google Scholar] [CrossRef] [Green Version]
  23. Surma-aho, A.; Hölttä-Otto, K.; Nelskylä, K.; Lindfors, N.C. Usability issues in the operating room—Towards contextual design guidelines for medical device design. Appl. Ergon. 2020, 90, 103221. [Google Scholar] [CrossRef] [PubMed]
  24. Heinemann, L.; Drossel, D.; Freckmann, G.; Kulzer, B. Usability of Medical Devices for Patients with Diabetes Who Are Visually Impaired or Blind. J. Diabetes Sci. Technol. 2016, 10, 1382–1387. [Google Scholar] [CrossRef] [PubMed]
  25. IEC 62366-1:2015; Medical Devices—Part 1: Application of Usability Engineering to Medical Device. IEC: Geneva, Switzerland, 2015.
  26. ISO/TR 16982:2002; Ergonomics of Human-System Interaction—Usability Methods Supporting Human-Centred Design. ISO/TR: Geneva, Switzerland, 2002.
  27. Jiang, L.; Eberlein, A. Selecting requirements engineering techniques based on project attributes—A case study. In Proceedings of the 14th Annual IEEE International Conference and Workshops on the Engineering of Computer-Based Systems (ECBS’07), Tucson, AZ, USA, 26–29 March 2007. [Google Scholar]
  28. Amici, C.; Pellegrini, N.; Tiboni, M. The Robot Selection Problem for Mini-Parallel Kinematic Machines: A Task-Driven Approach to the Selection Attributes Identification. Micromachines 2020, 11, 711. [Google Scholar] [CrossRef] [PubMed]
  29. Huang, G.Q. (Ed.) Design for X: Concurrent Engineering Imperatives, 1st ed.; Springer: Dordrecht, The Netherland, 1996. [Google Scholar] [CrossRef]
  30. Chesbrough, H.W. Open Innovation: The New Imperative for Creating and Profiting from Technology; Harvard Business School Press: Boston, MA, USA, 2003. [Google Scholar]
  31. Saidi, T.; Mutswangwa, C.T.; Douglas, T.S. Design Thinking as a Complement to Human Factors Engineering for Enhancing Medical Device Usability. Eng. Stud. 2019, 11, 34–50. [Google Scholar] [CrossRef]
  32. Hehn, J.; Mendez, D.; Uebernickel, F.; Brenner, W.; Broy, M. On Integrating Design Thinking for Human-Centered Requirements Engineering. IEEE Softw. 2020, 37, 25–31. [Google Scholar] [CrossRef] [Green Version]
  33. Dorst, K. The core of “design thinking” and its application. Des. Stud. 2011, 32, 521–532. [Google Scholar] [CrossRef]
  34. Denning, P.J. Design thinking. Commun. ACM 2013, 56, 29–31. [Google Scholar] [CrossRef]
  35. ISO 9241-210:2019; Ergonomics of Human-System Interaction—Part 210: Human-Centred Design for Interactive Systems. ISO: Geneva, Switzerland, 2019.
  36. Norman, D.A. The Design of Everyday Things, Human Factors and Ergonomics in Manufacturing; Basic Books: New York, NY, USA, 2013. [Google Scholar]
  37. Amici, C.; Faglia, R.; Taveggia, G.; Mor, M. Development of user-oriented mechatronic devices for post-stroke rehabilitation: The experience of UniBS H&W. In Proceedings of the Proceeding of R&D Management Conference 2016, Cambridge, UK, 3–6 July 2016. [Google Scholar]
  38. Formicola, R.; Ragni, F.; Mor, M.; Bissolotti, L.; Amici, C. Design Approach of Medical Devices for Regulation Compatibility: A Robotic Rehabilitation Case Study. In Proceedings of the ICT4AWE 2021, Online, 24–26 April 2021; pp. 146–153. [Google Scholar] [CrossRef]
  39. Formicola, R.; Ragni, F.; Borboni, A.; Amici, C. Design process of medical devices for robotic rehabilitation: An open innovation-inspired approach. In Robotics, Machinery and Engineering Technology for Precision Agriculture. Smart Innovation, Systems and Technologies; Springer: Berlin/Heidelberg, Germany, 2021. [Google Scholar] [CrossRef]
  40. Boser, Q.A.; Dawson, M.R.; Schofield, J.S.; Dziwenko, G.Y.; Hebert, J.S. Defining the design requirements for an assistive powered hand exoskeleton: A pilot explorative interview study and case series. Prosthetics Orthot. Int. 2020, 45, 161–169. [Google Scholar] [CrossRef]
  41. Garmer, K.; Ylvén, J.; Karlsson, I.C.M.A. User participation in requirements elicitation comparing focus group interviews and usability tests for eliciting usability requirements for medical equipment: A case study. Int. J. Ind. Ergon. 2004, 33, 85–98. [Google Scholar] [CrossRef]
  42. Caplan, S. Using focus group methodology for ergonomic design. Ergonomics 1990, 33, 527–533. [Google Scholar] [CrossRef]
  43. Stark, J. Product Lifecycle Management; Springer International Publishing: Cham, Switzerland, 2015. [Google Scholar] [CrossRef]
  44. Ceresoli, F.; Aggogeri, F.; Amici, C.; Borboni, A.; Faglia, R.; Pellegrini, N.; Tiboni, M.; Antonini, M.; Fausti, D.; Mor, M.; et al. Differential system for limb rehabilitation. In New Trends in Medical and Service Robotics (MESROB 2018); Springer: Berlin/Heidelberg, Germany, 2018. [Google Scholar] [CrossRef]
  45. Amici, C.; Ghidoni, M.; Ceresoli, F.; Gaffurini, P.; Bissolotti, L.; Mor, M.; Fausti, D.; Antonini, M.; Ragni, F.; Tiboni, M. Preliminary Validation of a Device for the Upper and Lower Limb Robotic Rehabilitation. In Proceedings of the 2019 23rd International Conference on Mechatronics Technology (ICMT), Salerno, Italy, 23–26 October 2019; pp. 1–6. [Google Scholar] [CrossRef]
  46. Amici, C.; Ragni, F.; Ghidoni, M.; Fausti, D.; Bissolotti, L.; Tiboni, M. Multi-Sensor Validation Approach of an End-Effector-Based Robot for the Rehabilitation of the Upper and Lower Limb. Electronics 2020, 9, 1751. [Google Scholar] [CrossRef]
  47. Polibrixia, S.R.L. Polibrixia Biomedical Engineering. Available online: https://www.polibrixia-bioeng.it/ (accessed on 19 May 2022).
  48. Brooke, J. SUS: A Retrospective. J. Usability Stud. 2013, 8, 29–40. [Google Scholar] [CrossRef]
  49. Brooke, J. SUS: A quick and dirty usability scale. Usability Eval. Ind. 1996, 189, 4–7. [Google Scholar]
  50. Bangor, A.; Kortum, P.T.; Miller, J.T. An empirical evaluation of the system usability scale. Int. J. Human–Computer Interact. 2008, 24, 574–594. [Google Scholar] [CrossRef]
  51. Lewis, J.R.; Sauro, J. Item Benchmarks for the System Usability Scale. J. Usability Stud. 2018, 13, 158–167. [Google Scholar]
  52. Bangor, A.; Kortum, P.; Miller, J. Determining What Individual SUS Scores Mean: Adding an Adjective Rating Scale. J. Usability Stud. 2009, 4, 114–123. [Google Scholar]
  53. ISO 14155:2020; Clinical Investigation Of Medical Devices For Human Subjects—Good Clinical Practice. ISO: Geneva, Switzerland, 2020.
  54. Sauro, J. A Practical Guide to the System Usability Scale: Background, Benchmarks & Best Practices; CreateSpace Independent Publishing Platform: Denver, CO, USA, 2011. [Google Scholar]
  55. IEC 60601-1-8:2006; Medical Electrical Equipment—Part 1–8: General Requirements for Basic Safety and Essential Performance—Collateral Standard: General Requirements, Tests and Guidance for Alarm Systems in Medical Electrical Equipment and Medical Electrical Systems. IEC: Geneva, Switzerland, 2006.
  56. Pahl, G.; Beitz, W.; Feldhusen, J.; Grote, K.H. Engineering Design: A Systematic Approach, 3rd ed.; Springer: Berlin/Heidelberg, Germany, 2006. [Google Scholar]
  57. Ulrich, K.; Eppinger, S. Product Design and Development, 5th ed.; McGraw-Hill Education: New York, NY, USA, 2011. [Google Scholar]
  58. Siang, Y.T. Design Thinking. Interaction Design Foundation. Available online: https://www.interaction-design.org/literature/topics/design-thinking (accessed on 14 November 2022).
  59. Mesa, D.; Renda, G.; Kuys, B.; Cook, S.M. Implementing a Design Thinking Approach to De-Risk the Digitalisation of Manufacturing SMEs. Sustainability 2022, 14, 14358. [Google Scholar] [CrossRef]
  60. Sanders, E.B.-N.; Stappers, P.J. Co-creation and the new landscapes of design. CoDesign 2008, 4, 5–18. [Google Scholar] [CrossRef] [Green Version]
  61. Paay, J.; Kuys, B.; Taffe, S. Innovating product design through university-industry collaboration: Codesigning a bushfire rated skylight. Des. Stud. 2021, 76, 101031. [Google Scholar] [CrossRef]
Figure 1. Usability-Oriented Model for the Design of Medical Devices.
Figure 1. Usability-Oriented Model for the Design of Medical Devices.
Designs 07 00009 g001
Figure 2. LEPRE Rehabilitation Device (Polibrixia s.r.l., Brescia, Italy).
Figure 2. LEPRE Rehabilitation Device (Polibrixia s.r.l., Brescia, Italy).
Designs 07 00009 g002
Figure 3. From the left (modified with permission from [47], person’s face obscured): (a) LEPRE Rehabilitation Device in the upper-limb setup: central body in upper position and accessories that allow hand grip, and (b) LEPRE Rehabilitation Device in the lower-limb setup: central body in lower position and accessories that allow the anchorage of the feet.
Figure 3. From the left (modified with permission from [47], person’s face obscured): (a) LEPRE Rehabilitation Device in the upper-limb setup: central body in upper position and accessories that allow hand grip, and (b) LEPRE Rehabilitation Device in the lower-limb setup: central body in lower position and accessories that allow the anchorage of the feet.
Designs 07 00009 g003
Figure 4. Correspondences between the Usability-Oriented Model for the Design of Medical Devices and the development process defined by the ISO ISO/TR 16982:2002 standard.
Figure 4. Correspondences between the Usability-Oriented Model for the Design of Medical Devices and the development process defined by the ISO ISO/TR 16982:2002 standard.
Designs 07 00009 g004
Figure 5. Flowcharts of clinical operators: (a) physiotherapists’ flowchart; (b) physiatrists’ flowchart.
Figure 5. Flowcharts of clinical operators: (a) physiotherapists’ flowchart; (b) physiatrists’ flowchart.
Designs 07 00009 g005
Figure 6. Patients’ flowchart.
Figure 6. Patients’ flowchart.
Designs 07 00009 g006
Figure 7. LEPRE Case Study: Application of the Usability-Oriented Model for the Design of Medical Devices.
Figure 7. LEPRE Case Study: Application of the Usability-Oriented Model for the Design of Medical Devices.
Designs 07 00009 g007
Figure 8. From the left: (a) The Original LEPRE Carter; (b) the carter of the LEPRE Rehabilitation Device in the CE marked version.
Figure 8. From the left: (a) The Original LEPRE Carter; (b) the carter of the LEPRE Rehabilitation Device in the CE marked version.
Designs 07 00009 g008
Figure 9. Illustrative detail of the contact between the patient’s trunk and the carter in correspondence with the radius of curvature.
Figure 9. Illustrative detail of the contact between the patient’s trunk and the carter in correspondence with the radius of curvature.
Designs 07 00009 g009
Figure 10. From the left: (a) the First LEPRE Structural Shape, (b) the Structural Shape of the LEPRE Rehabilitation Device in the CE-marked version.
Figure 10. From the left: (a) the First LEPRE Structural Shape, (b) the Structural Shape of the LEPRE Rehabilitation Device in the CE-marked version.
Designs 07 00009 g010
Figure 11. Detail of the Mirror Function switch on the LEPRE virtual console.
Figure 11. Detail of the Mirror Function switch on the LEPRE virtual console.
Designs 07 00009 g011
Figure 12. Mouse 3D Height.
Figure 12. Mouse 3D Height.
Designs 07 00009 g012
Table 1. Usability evaluation methods recommended based on the development stage.
Table 1. Usability evaluation methods recommended based on the development stage.
Development StagesUsability Evaluation Methods
Observation of Users Performance-Related MeasurementsCritical Incident AnalysisQuestionnairesInterviewsThinking AloudCollaborative Design and EvaluationCreativity MethodsDocument-Based MethodsModel-Based MethodsExpert EvaluationAutomated Evaluation
Requirements analysis+++++++++++++++
Architectural design+++ ++++++++++++
Qualification testing++++++++++ ++++
Legend
++Recommended
+Appropriate
empty cellNeutral
Table 2. Adopted convention for the interpretation of the SUS values as Adjective Ratings. The interpretation complies with Bangor et al.’s classification [52].
Table 2. Adopted convention for the interpretation of the SUS values as Adjective Ratings. The interpretation complies with Bangor et al.’s classification [52].
AdjectiveSUS Value
Worst Imaginable12.5
Awful20.3
Poor35.7
OK50.9
Good71.4
Excellent85.5
Best Imaginable90.9
Table 3. Study population description.
Table 3. Study population description.
CategoryParameterValue
PatientsAge54.08 ± 9.77
SexM = 8, F = 4
Stroke type (I = Ischemic, H = Hemorrhagic)I = 6, H = 6
PhysiotherapistsAge44.88 ± 7.79
SexM = 5, F = 3
Experience (years)19.63 ± 7.48
PhysiatristsAge48.00 ± 1.41
SexM = 1, F = 1
Experience (years)18.50 ± 2.12
Table 4. SUS results for groups of participants.
Table 4. SUS results for groups of participants.
CategoryT0T1T2
Patients\75.00 ± 11.4885.00 ± 9.35 (p = 0.028)
min = 60.00min = 75.00
Max = 100.00Max = 100.00
Physiotherapists64.38 ± 12.3078.13 ± 9.98 (p = 0.028)\
min = 40.00min = 60.00
Max = 77.50Max = 90.00
Physiatrists85.00 ± 3.5486.25 ± 1.77 (p = 0.667)\
min = 82.50min = 85.00
Max = 87.50Max = 87.50
Table 5. SUS(U) and SUS(L) results for groups of participants.
Table 5. SUS(U) and SUS(L) results for groups of participants.
CategoryT0T1T2
PatientsSUS(U)\77.34 ± 10.9286.56 ± 8.87 (p = 0.039)
min = 62.50min = 75.00
Max = 100.00Max = 100.00
SUS(L)\65.63 ± 17.7879.17 ± 15.39 (p = 0.114)
min = 37.50min = 62.50
Max = 100Max = 100.00
PhysiotherapistsSUS(U)63.67 ± 11.8078.91 ± 8.64 (p = 0.010)\
min = 43.75min = 65.63
Max = 78.13Max = 93.75
SUS(L)67.19 ± 22.1075.00 ± 21.13 (p = 0.442)\
min = 25.00min = 37.50
Max = 100.00Max = 100.00
PhysiatristsSUS(U)82.81 ± 2.2190.63 ± 4.42 (p = 0.333)\
min = 81.25min = 87.50
Max = 84.38Max = 93.75
SUS(L)93.75 ± 8.8468.75 ± 8.84 (p = 0.333)\
min = 87.50min = 62.50
Max = 100.00Max = 75.00
Table 6. Overview of the comparison between the usability-oriented model and the presented paradigms/models (Y: Yes, N: No).
Table 6. Overview of the comparison between the usability-oriented model and the presented paradigms/models (Y: Yes, N: No).
Paradigm/
Method
A Priori
Condition
Different
Vision
Different
Stakeholders
Define
Engineering Tasks
Regulation
Oriented
Usability
in All Stages
Design ProcessYNNYNN
Product DevelopmentYNNYNN
Design ThinkingNYNNNN
Human-Centered DesignNYYNYY
CodesignNYYNNN
Usability-Oriented ModelNYYYYY
Table 7. Overview of fields affected by usability requirements that emerged in the illustrative case.
Table 7. Overview of fields affected by usability requirements that emerged in the illustrative case.
Impact AreasUsability Requirements
(u1) Carter Shape(u2) Structural Shape(u3) Mirror Function(u4) Mouse 3D Height
Aesthetic fieldAestheticsAestheticsxx
Safety fieldSafetySafetySafetyx
Mechanical fieldMechanical StructureMechanical StructurexMinor Design Review
Electronic fieldElectronic StructureElectronic Featuresxx
Software fieldxSoftware FeaturesSoftware Featuresx
Other fieldsMarket SharexxEase of use
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Formicola, R.; Amici, C.; Mor, M.; Bissolotti, L.; Borboni, A. Design of Medical Devices with Usability in Mind: A Theoretical Proposal and Experimental Case Study Using the LEPRE Device. Designs 2023, 7, 9. https://doi.org/10.3390/designs7010009

AMA Style

Formicola R, Amici C, Mor M, Bissolotti L, Borboni A. Design of Medical Devices with Usability in Mind: A Theoretical Proposal and Experimental Case Study Using the LEPRE Device. Designs. 2023; 7(1):9. https://doi.org/10.3390/designs7010009

Chicago/Turabian Style

Formicola, Raffaele, Cinzia Amici, Maurizio Mor, Luciano Bissolotti, and Alberto Borboni. 2023. "Design of Medical Devices with Usability in Mind: A Theoretical Proposal and Experimental Case Study Using the LEPRE Device" Designs 7, no. 1: 9. https://doi.org/10.3390/designs7010009

Article Metrics

Back to TopTop