From Human-Human to Human-Machine Cooperation in Manufacturing 4.0

Abstract

Humans are currently experiencing the fourth industrial revolution called Industry 4.0. This revolution came about with the arrival of new technologies that promise to change the way humans work and interact with each other and with machines. It aims to improve the cooperation between humans and machines for mutual enrichment. This would be done by leveraging human knowledge and experience, and by reactively balancing some complex or complicated tasks with intelligent systems. To achieve this objective, methodological approaches based on experimental studies should be followed to ensure a proper evaluation of human-machine system design choices. This paper proposes an experimental study based on a platform that uses an intelligent manufacturing system made up of mobile robots, autonomous shuttles using the principle of intelligent products, and manufacturing robots in the context of Manufacturing 4.0. Two experiments were conducted to evaluate the impact of teamwork human-machine cooperation, performance, and workload of the human operator. The results showed a lower level of participants’ assessment of time demand and physical demand in teamwork conditions. It was also found that the team working improves the subjective human operator Know-how-to-cooperate when controlling the autonomous shuttles. Moreover, the results showed that in addition to the work organization, other personal parameters, such as the frequency of playing video games could affect the performance and state of the human operator. They raised the importance of further analysis to determine cooperative patterns in a group of humans that can be adapted to improve human-machine cooperation.

1. Introduction

Human is currently experiencing the fourth industrial revolution called Industry 4.0. Industry 4.0 aims at creating new factories based on several advanced technologies, such as cyber-physical systems, cloud computing, digital twins, and the Internet of things [1]. Manufacturing 4.0 translates the new abilities of Industry 4.0 manufacturing systems to become more intelligent. A manufacturing 4.0 system can monitor processes and support decisions using real-time communication and computing technologies [2]. Manufacturing 4.0 promises to change the way humans work and interact with each other and with machines and aims to improve the cooperation between human and machine for mutual enrichment [3]. This would be done by leveraging human knowledge and experience, and by reactively balancing some complex or complicated physical or mental tasks between the human, being operator or supervisor, and the manufacturing systems [4]. An intelligent manufacturing system (IMS) is a Manufacturing 4.0 system composed of autonomous artificial entities (e.g., intelligent products, smart production resources, autonomous automated guided vehicles, and robots, …) able to cooperate with humans and exploit digital technologies, such as digital twin technologies, to mirror the reality and to test different strategies relevant to manufacturing activities to reach production objectives (production rate, quality level, cost, energy consumption…) [4]. These objectives are known to be conflictual [5].

This research aims to examine whether a new cooperative human-machine organization improves the control and supervision of IMS task performance. Precisely, this study inspects whether in this organization the human supervisors can effectively perform two complementary tasks at two cooperative layers: the tactical layer dealing with high decision level about machines, products, and resources organization, and the operational layer dealing with decisions close to the control level of the process, have to cooperate to meet each other needs and to adapt their respective tasks [6]. The tasks are as follows: (1) to plan and manage production; (2) to supervise and control mobile robots. This article is a follow-up to a previous study presented in [7]. It presents a complementary experiment and provides results about the human performance and workload with and without assistance systems when the human was controlling a complex IMS alone. The main results highlighted the need to improve cooperation between human and artificial entities involved in the control of the process, but also cooperation between artificial entities. These results are shown in the study and experiments presented in this paper.

The outlines of this paper are the following. In Section 2, a review of the literature in the field of human-machine cooperation in the context of Industry 4.0. In Section 3, the IMS designed within the framework of the HUMANISM project is presented. In Section 4, the experiments and the results are presented. In Section 5, a discussion is led, and finally, in Section 6, conclusions and perspectives are provided.

2. Review of the Literature and Presentation of the Research Objective

Both manual and highly skilled labor forces remain crucial in smart factories, especially in high precision and complex processes or customized production [8]. At the same time, as induced by Industry 4.0, the technological possibilities to support manual work are increasing and point out more intelligent and complex industrial systems [9]. Therefore, it would be difficult for the human worker of the future (supervisor or operator) to understand the behaviors of these systems and interact effectively with them [10]. In this context, recently, more and more studies in Industry 4.0 emphasize the importance of closer cooperation between humans and machines and are interested in the design of this new type of interaction [11,12]. Jones et al. propose a new way to model human interactions with cyber-physical Systems and Industry 4.0 technologies [13]. They support the idea of modeling agents (humans and machines) as joint cognitive systems that remove the separation between them. According to this view, Romero et al. define a symbiotic industrial framework, in which human workers and intelligent systems dynamically adapt to each other and cooperate to reach the common goals [14]. To support the evolution of human-machine symbiosis, Hadorn et al. propose a holistic system modeling in which the system is considered as a whole, where all-important entities, such as human workers and technical artifacts are fully integrated [15].

Today, many questions remain to be answered to improve the integration of operator 4.0 in the intelligent factory of the future. Indeed, ethical risks have been identified, especially regarding the wish of designers dealing with human-machine symbiosis [16]. One of the main risks is the dependency of humans upon machines’ ability and capacity, and therefore humans will be unable to complete tasks without machine support when necessary. In the manufacturing 4.0 domain, a recent study had pointed out the complexity of carrying out the task of controlling and supervising an IMS by a single human operator [7]. It was reported that the analysis of the spatial representation of the IMS was difficult for humans because they had to permanently split their attention between several screens. The study also highlighted the necessity to better share human activities. Operator 4.0 has real-time access to large amounts of data and information that would impose a large cognitive workload. In this sense, this paper focuses on investigating a new cooperative organization for operators 4.0, in which the production and logistics tasks are performed by two human operators instead of one. This organization shares the tasks among operators to optimize the overall process and improve human-machine cooperation by reducing the cognitive workload of one human operator. This study is relevant because it merges human-human cooperation with human-machine one, instead of concentrating on the development and implementation of technologies like most Industry 4.0 research.

The question of evaluating teams of operators has been raised for years. For example, Pacaux-Lemoine et al. conducted an analysis of the cooperation that arises from human operators and introduced the concept of patterns, which describes how humans cooperate with each other and with machines [17]. Previous studies have confirmed the effectiveness of cooperative work on a complex task [18]. Other studies have identified some problems that can occur in group work, such as cognitive conflicts [19]. Clear organization and sharing of tasks between people in the same group make teamwork more meaningful. Just like in an orchestra [20], everyone needs to know their place, their role, and the tasks they have to perform. If one musician plays out of tune or at a different tempo, the whole orchestra suffers. To ensure good integration and organization of human operators, it is necessary to study upstream that integration and organization. Cognitive task modeling was previously used to design the human-intelligent manufacturing system. The effectiveness of this method in deepening the analysis of task allocation strategies has been demonstrated in the literature [7].

Our work aims to study to what extent human-human cooperation can be studied to define efficient human-machine cooperation, in the general context of Industry 4.0 systems. In this paper, a new experiment, for which a set of tasks is shared and performed in-group, have been led and are described and analyzed. A study of the results helped us to draw some conclusions that may be useful for researchers working on the efficient design of the human-machine cooperation system for Industry 4.0. More precisely, the proposed experimental set-up consists of production and logistics tasks to be performed by two human operators in the context of manufacturing 4.0. Both operators had to communicate and cooperate for better work quality and performance. The experimental set-up has been constructed on the Humanism IMS presented in the next section.

3. The Humanism Intelligent Manufacturing System

The intelligent manufacturing system used in this work is part of the French ANR project “HUMANISM” and was developed to facilitate the integration and evaluation of the human industry 4.0 context. It exploited the SUCRé project platform [21] and an existing real educational cell through one emulator and one simulator (SMART cell at UPHF, Valenciennes) (Figure 1a). HUMANISM platform uses a digital twin, integrating a digital shadow of the SMART flexible cell using Arezzo [22]. The digital twin processes information about the smart cell and control. The HUMANISM IMS consists of manufacturing robots linked by conveyors on which shuttles transport products from one manufacturing robot to another in real-time (cf. Figure 1b).

In recent years, intelligent manufacturing and logistics have motivated the development and use of mobile robots [23]. In this research, real ground mobile robots are used to virtually supply the manufacturing robots and unload the finished products (cf. Figure 2a). To link the virtual world of the shuttles and manufacturing robots to the real world of the mobile robots, a projection of the production cell is made on a wooden structure and transmitted to the human operator via an interface (cf. Figure 2b). Thus, mobile robots could move around the cell area and give the human operator a realistic view of the IMS (cf. Figure 2c). The mobile robots were programmed in RobotC and the production cell in Netlogo and interfaced with JAVA. The platform was designed in such a way as to avoid developing new versions for each new research project. The architecture adopted for this platform is flexible, allowing modules to be added and/or modified, but also to operate in the same way under real and simulated conditions. For details about the technical developments of the HUMANISM platform, the reader is referred to [24].

The HUMANISM project analyses human-machine cooperation according to the three decisional levels: strategic, tactical, and operational [25]. In our case, a production plan detailing the required number and types of products to manufacture is provided by an industrial manager from the strategic level. At the tactical level, the human supervisor has to analyze this plan, update if needed and send orders to the operational level. However, he/she also has to manage unexpected events. At this level, key performance indicators and information provided by the operational level are used by the human supervisor to analyze the system performance and adapt the decision in real-time. This work focuses on the interaction between the tactical and operational levels. The “intelligence” of the flexible cell resides in the ability of the “intelligent products” to self-organize according to the events that occur and the manufacturing operations that the robots have to carry out. The behavior of these intelligent products is based on the principle of potential fields [10], which is a digital signal that enables production robots to dynamically attract “intelligent products” that can sense the fields emitted by resources. The value of the field is set according to the current queue of products to proceed and state for each robot. The more this value, the more attractive the robots and the fastest the operation shall be realized. This leads to self-organization among intelligent products and robots composing the manufacturing 4.0 intelligent manufacturing system, whose global behavior provides a powerful mechanism to react in real-time to various unexpected events. However, it is remarked that this approach does not provide sufficient guarantee to achieve production target, especially in case of disturbances. To address this, the HUMANISM project suggests asking for human help, as he/she can analyze the intelligent products and robots activities. Moreover, the human supervisor can react and adapt to unpredictable events, for example, due to his/her expertise and experience. The human supervisor ensures that manufacturing goals are met while taking into account constraints, which are hardly implementable in the self-organized system, such as global energy consumption limits. Nevertheless, given the complexity of the self-organized system, the human supervisor must remain focused and have a good situation awareness to make effective decisions at the right time.

4. Study and Tested Assumptions

In this study, three product types (denoted “L”, “T”, and “I”) were to be manufactured and required different sets of operations (1 to 7). The same operation could be requested multiple times for a single product type. The “L” and “I” products are similar but generate different workloads for the manufacturing robots. “T” product requires a specific operation that takes a long time to proceed and can be performed by only one manufacturing robot. The primary task (production control) of the human operator was to complete the production plan that included the required operations and quantity for each type of product. This task was to be performed while monitoring the consumption of the manufacturing robots and maintaining general power consumption under a given limit. The second task (logistics control) was to supply the manufacturing robots with raw materials and two mobile robots. The mobile robots can be either autonomous or remotely controlled by the human operator. Built on a previous study, two experiments were designed to gather empirical data on the impact of a new cooperative human-machine organization in the control and supervision of an IMS and to enable relative comparison between the two. In the first experiment (Organization 1), only one participant handled both tasks. In the second experiment (Organization 2), both tasks were performed by three participants, one participant per task. In this second experiment, we added a third task (Analysis task). During this task, the human operator had to analyze and evaluate the performance of the production and logistics tasks by writing his/her remarks on a given paper.

Three research questions were formulated about the impact of the working organization (organization 1 and organization 2) on various performance indicators. The first question concerns the impact of the work organization on the effectiveness and efficiency in controlling and supervising the IMS. The second question aims to analyze the impact of work organization on the workload of the human operator. The third question concerns the impact of the work organization on human-machine cooperation and the usability of an assistance system that was provided. Thus, the following assumptions (predictions) were constructed and tested:

Hypothese 1 (H1).

The “Organization 2” outperforms the “Organization 1” for the control and supervision of the human-intelligent manufacturing system;

Hypothese 2 (H2).

The “Organization 2” describes an effective way for the control and supervision of the human-intelligent manufacturing system;

Hypothese 3 (H3).

The “Organization 2” describes an efficient way for the control and supervision of the human-intelligent manufacturing system;

Hypothese 4 (H4).

The “Organization 2” is an effective organization to reduce the human operator’s workload;

Hypothese 5 (H5).

The “Organization 2” improves the human-machine cooperation and the assistance system usability.

5. Evaluation Method

The experimental method used to test the previous hypotheses is presented in the following sub-sections.

5.1. Participants

Forty-seven ungraduated students from the Polytechnic University of Hauts-de-France, with similar levels of academic ability, took part in the experiments. All were required to perform the tests as part of their coursework. Then, they were trained to conduct the experiment tasks. Little information has been requested from students except for their habits with mobile robot use and video games. We did not record and take into account physical-motor and psychological abilities during the experiments. We suppose there was no interaction between participants except verbal communications and through the experimental platform. Indeed, participants were far from each other, far from robots, and they were all students with no hierarchical dependence. We did not control the conditions in the experimental room either (temperature, noise, humidity, light, smoke…), but the experimental room was comfortable, with no disturbances, and was the same for all the participants.

Twenty-three of them carried out the first experiment. The data of three participants were not usable and were excluded from the analysis because of burned data. The selected participants for the first experiment were composed of 2 women and 18 men aged between 20 and 23 years old (mean 20.9; SD: 0.85). The remaining 24 participants carried out the second experiment, 4 women and 20 men aged between 20 and 27 years old (mean: 21.16; SD: 8.90).

5.2. Apparatus

As introduced, the experiments were carried out using the HUMANISM IMS defined in Section 4. Human supervisor’s workplaces, two mobile robots, and the HUMANISM facility with seven manufacturing robots and six self-organized shuttles were used for the experiments. One manufacturing robot is dedicated to loading the products to be manufactured on the shuttles and unloading the finished products, while the others are responsible for manufacturing the products. In the first experiment, only one workplace was used to perform both production and logistics tasks, because one human operator had to perform all these tasks. Figure 3 shows that workplace.

In the second experiment (based on organization 2), production and logistics workplaces were set up. The production workplace (cf. Figure 4) includes planning and control interfaces that allow the human operator to prepare and manage the production and a bird’s eye view that offers a general view of the real cell.

The logistics workplace (cf. Figure 5) comprises a display of the bird’s eye view of the production cell, the control, and tactical interfaces through which the human operator can manage the mobile robots.

The mobile robots can either move autonomously or be teleoperated. A joystick was used to manipulate the direction in which the selected mobile robot moved when in the remote control level of automation. Under the autonomous level of automation, the mobile robot navigates autonomously toward an objective (geographical point) sent by the human operator by clicking on the map of the robot’s environment on the tactical interface, where the robot’s positions are displayed in real-time.

The experimentation supervision workplace (cf. Figure 6) is added in the second experiment. It has been installed so that the experimenter can manage the experimentation through an interface that displays an overall view of the experimental environment. The experimentation supervision and production workplaces are connected, so that random failure can be triggered by the experimenter, who was also in charge of validating the loading and unloading of products. In addition, he/she could start, stop or pause the platform.

As introduced in the previous section, an assistance system was provided to support humans in the control of the IMS. This assistance system can be used to simulate new potential field amplitudes and improve the self-organization of products. It also helps to manage overconsumption by reducing the amplitude of potential fields of high consumption production robots, as well as by postponing the manufacture of products requiring these production robots. This assistance system can be used when desired by the human operator.

5.3. Experimental Design and Measures

Both experiments were run as a between-subjects study design that splits participants into groups, which tests one organization each (organization 1 or organization 2). The measures collected during the experiment were

• Objective measures (production performance):
  • Effectiveness: effectiveness concerns the objective “to what extent the task was accomplished?” This is evaluated in our experiments by counting the number of products produced at the end of the experiment (accomplishment of the production plan) and the score performance (the sum of the achieved products and a bonus minus the overconsumption, the consumption, and a penalty). The bonus is assisted based on energy savings and compliance with consumption limits. The penalty is fixed according to the collisions of mobile robots in the environment.
  • Efficiency: efficiency concerns the objective “to what extent the means were used and how the tasks were accomplished?” This is evaluated in our experiment by counting the energy consumption and over-consumption (the exceeding of the energy consumption limit by the manufacturing robots) during the experiment.

For information on how these measures were calculated precisely, the reader is referred to [7]. However, the main information are presented below.

The recorded data are used to compute a final score representing how well the participants performed in the control of the IMS. Factors used to compute are separated into two families:

• the positive points represent every factor that contributes to the completion of the objectives, also known as what is expected from the IMS and the human supervisor (bonus):

  ◦

  average consumption per product

  ◦

  low cumulated overconsumption

  ◦

  no request to the experiment team on topics not related to technical issues

  ◦

  faced important technical perturbations

• the negative points represent both the infractions to the constraints and elements inducing a difference in production cost (penalty).

  ◦

  late delivery

  ◦

  major incident with the ground robots (damage or physical assistance to recover the robot)

Negative points relating to the total consummation are attributed linearly, while overconsumption is sanctioned using steps with incremental factors.

• Subjective measures:
  • Workload: that is “what was the workload perceived by the participants during the experiment?” This is evaluated using the answers of the participants to the NASA-TLX questionnaire [26].
  • Human-machine cooperation: that is “how was the cooperation between participants and the system?” This is evaluated using the answers of the participants to a questionnaire based on the human-machine cooperation model [27].

5.4. Procedure

Upon arrival, the participants received a 30 min self-paced training PowerPoint tutorial showing the elements of the experimental platform, the score, and its main factors: the accomplishment of the production plan, consumption, overconsumption, bonus, and penalty. The participants have been informed that a ranking will be based on their performance in the experiment. After the tutorial, the participants were introduced to the workplace of the platform and its major elements. Then, the participants were trained on the tasks for 15 min assisted by the experimenter to ensure correct training. To counteract the learning effect, the scenario used for the training was different from the experimental scenario and no disturbances were caused. Once the participant had correctly completed the training session, the 10-min testing session started. The experimenter was out of the direct line of sight of the participant so as not to cause any disturbance. However, the experimenter had a direct view of the participant as well as video feeds to monitor the good running of the platform. The participant was allowed to ask questions in case of problems. The experimenter could only give answers and was not to influence the participant. In case of a technical problem, the experimenter could try to solve the problem remotely using specific commands, or could remotely access the digital twin to simulate part of the system operation to ensure the continuity of the experiment.

During the experiment runs, participants’ tasks were to use the platform to:

• supervise the whole production system to check the state of the robots, consumption, and production,
• use the assistance system to simulate new potential field amplitudes and improve the self-organization of products,
• manage overconsumption by reducing the amplitude of potential fields of high consumption production robots, as well as by postponing the manufacture of products requiring these production robots,
• manage production robot breakdowns by rescheduling production with an adapted selection of products requiring these robots.

Once the 10 min have elapsed, the experiment was stopped and considered complete. At the end of the experimental session, the participants filled out the questionnaires.

6. Results

The results obtained from the objective and subjective measures collected during the experiments are presented in the following subsections. The statistical effect of the independent variable (here, the organization) on the dependent one (here, the performance indicators, perceived workload, and perceived human-machine cooperation) are presented respectively in the next tables.

6.1. Performance Indicators

Based on the results presented in

Table 1. Average of the participants’ performance per group and the result of the significance test. The energy consumption and overconsumption are in kWh.

Dependent Variables (Performance Indicators)	Average (Std Dev.)		p-Value (Confidence Level of 95%)
	Organization 1 Group, n = 20	Organization 2 Group, n = 24
Number of products produced	6.80 (1.58)	7.04 (2.01)	0.273
Score performance (1 × 104)	1.72 (0.652)	1.76 (0.821)	0.442
Energy consumption (1 × 104)	2.20 (0.325)	2.13 (0.355)	0.259
Overconsumption	623.60 (499.57)	418.08 (442.48)	0.106

, it is possible to list the trends noted when comparing the averages of each performance indicator for each experimental condition (organization 1 and organization 2). The trends are the following:

• The number of products produced during the experiment in organization 2 is higher than in organization 1.
• The score performance is slightly higher in organization 2 than in organization 1.
• Organization 2 leads to lower energy consumption than organization 1.
• Over-consumption is higher in organization 1 than in organization 2.

6.2. Workload Assessment

The weighted workload score obtained by the TLX questionnaire consists of two parts [26]. The first part is the raw scores for each scale (mental demand, physical demand, temporal demand, performance, effort, and frustration), as recorded by the participants at the end of the experiment. The second part is the weighted workload score, which combines the raw values into a single value. The steps followed in measuring the workload using the NASA-TLX method are:

• Raw ratings of each scale
• Weightings
• Weighted rating of each scale: (weight × raw)
• Weighted workload score: ∑ $\frac{\mathrm{Weighted}\mathrm{scales}}{15}$

Results reveal that organization 1 has the highest subjective temporal demand and frustration, while organization 2 has the highest subjective physical demand; performance and effort (cf.

Table 2. Averages of the perceived workload measures and the result of the significance test.

Dependent Variables (NASA-TLX Workload)	M (S. D.)		p-Value (Confidence Level of 95%)
	Organization 1 Group, n = 20	Organization 2 Group, n = 24
Workload	65.90 (9.47)	64.13 (18.75)	0.344
Weighted Mental Demand	308.00 (110.15)	310.42 (108.33)	0.942
Weighted Physical Demand	8.50 (19.27)	43.75 (69.71)	0.034
Weighted Temporal Demand	246.50 (111.23)	190.00 (90.26)	0.070
Weighted Performance	168.50 (70.36)	180.42 (114.46)	0.687
Weighted Effort	110.50 (72.73)	124.00 (110.38)	0.642
Weighted Frustration	146.50 (97.67)	113.33 (151.36)	0.404

).

The mental demand average rating is approximatively equal under both organizations. Statistical tests of each scale with factor “organization” were conducted to test whether the differences in weighted scale ratings between organizations are statistically significant. A confidence level of 95% and a tendency to a significance level of 98% were considered. The results show a significant difference in only physical demand (p-value = 0.004) and a tendency to be significantly different in temporal demand (p-value = 0.07). The average NASA-TLX workload score in organization 2 tends to be lower than in organization 1 and thus in the hypothesized direction, but the size of the difference does not reach statistical significance. Furthermore, the pie chart plot of weightings for each scale of both conditions (organization 1 and organization 2) reveals the scale mental demand to be the most important subjective source of workload (27%), and the scale physical demand the less important one (4%) (cf. Figure 7).

6.3. Human-Machine Cooperation

The human-machine cooperation is evaluated according to the answers of the participants to the questionnaire based on the human-machine cooperation model. A questionnaire has been developed using a seven-point Linkert scale where a value of 1 corresponded to “Not at all” and value of 7 corresponds to “Totally”. The questions were related to the participant and shuttle Know-How (KH) and Know-How-to-Cooperate (KHC) levels [27]. The results reveal a significant effect of the organization on the human operator KHC regarding the control of the shuttles (p-value = 0.049). The question asked was: to what extent do you feel in control of the shuttles? Organization 2 improves significantly the participants’ sense of control over the shuttles that qualifies a function of participants’ KHC. This means organization 2 improves the participants’ KHC to some extent. Based on

Table 3. Averages of the perceived Human-Machine cooperation measures and the significant results.

Dependent Variables (Human-Machine Cooperation Questionnaire)	Average (Std Dev.)		p-Value (Confidence Level of 95%)
	Organization 1 Group, n = 20	Organization 2 Group, n = 24
Perception 1	5.05 (1.146)	4.92 (1.283)	0.602
Information Analysis 1	4.95 (1.191)	4.71 (1.628)	0.629
Decision Making 1	4.65 (1.348)	4.29 (1.459)	0.797
Action Implementation 1	6.00 (1.076)	5.42 (1.501)	0.909
Information gathering 2	5.55 (1.356)	5.42 (1.613)	0.544
Conflict detection 2	4.90 (1.518)	4.79 (1.414)	0.567
Conflict management 2	4.15 (1.496)	4.58 (1.442)	0.167
Control of the shuttles 2	4.05 (1.432)	4.83 (1.606)	0.049
Conflict detection 3	5.20 (1.361)	4.63 (1.345)	0.956
Conflict management 3	4.80 (1.436)	4.75 (1.648)	0.452
Authority management 3	5.45 (1.276)	5.29 (1.160)	0.702

¹ Shuttle KH functions. ² Human operator KHC functions. ³ Shuttle KHC functions.

, other trends in the direction of the assumptions can also be observed.

7. Further Results

Given the non-significance of some of the previous results, which could be caused by the small sample of the population, it was decided to extend the analysis by creating classes of participants to identify possible clusters concerning the participants’ performance and their responses to the questionnaires. This classification is carried out by the K-means method using SPSS software [28]. This is a statistical method of data processing, called “clustering” that allows the elements under study (in our case the participants) to be organized into similar groups. According to defined criteria, the elements of a group should be as similar as possible and different from those of other groups. The creation of two clusters was chosen because of the limited sample size of the population. The parameters entered in the SPSS software are the number of clusters and how to label the observations (hereby the participants’ ID).

7.1. Clusters from Objective Measures

Two clusters were created using the score performance and the use of the simulation (assistance system) variables.

Table 4. ANOVA result of the clustering according to score and the number of users of the assistance system variables, and the final cluster centers.

	ANOVA
	Cluster		Error				Cluster Centers
	Mean Square	dof	Mean Square	dof	F	Sig.	Cluster 1	Cluster 2
Score	1,450,453,042.50	1	21,620,446.899	42	67.087	<0.001	11,114	22,645
Use of assi.	47.728	1	19.623	42	2.432	0.126	1	3

presents the ANOVA result of the clustering and the centers of the final clusters. The significant difference between the classes in their answers to the questionnaires was tested statistically and the significant results are presented in

Table 5. Averages of the participants’ performance per group and the result of the significance test.

Dependent Variables	Average (Std. Dev.)		p-Value (Confidence Level of 95%)
	Cluster 1, n = 20	Cluster 2, n = 24
Workload	47.9 (14.3)	71.3 (9.54)	<0.001
Weighted Mental Demand	247.5 (122.0)	332.5 (93.98)	0.018
Weighted Performance	113.3 (67.9)	198.1 (95.80)	0.008
Weighted Frustration	50.8 (99.2)	157.5 (128.59)	0.005
Sufficient training	5.50 (1.17)	4.19 (1.203)	0.002
Environment complexity	3.67 (1.37)	4.69 (1.447)	0.037

.

The results showed that participants who scored high and used the “simulator” assistance system more frequently (cluster 2), were those who reported a higher workload, mental demand, and frustration. It can also be noted that the participants in this same cluster under-rated their performance in comparison with the participants in cluster 1. In addition, these participants felt that they had been sufficiently trained and the experimental environment was not very complex in comparison to those in cluster 1 (cf. Table 5 and Figure 8).

The responses of the participants to personal questions showed that cluster 2 (high score and use of assistance system) contains more frequent gamers (46% of the participants often/always play video games) than cluster 1 (45% of the participants never/occasionally play video games).

The second clusters were created according to the consumption and over-consumption (cf.

Table 6. ANOVA result of the clustering according to consumption (cons.), over-consumption (over-cons.) variables, and the final cluster centers.

	ANOVA
	Cluster		Error				Cluster Centers
	Mean Square	dof	Mean Square	dof	F	Sig.	Cluster 1	Cluster 2
Cons.	354,135,224.575	1	3,346,472.711	42	105.823	<0.001	23.777	17.879
Over-cons.	3,793,647.098	1	140,768.188	42	26.950	<0.001	733	123

and Figure 9). Cluster 2 with the lowest consumption contains more participants in organization 2 (69%) than in organization 1 (31%), as opposed to cluster 2, which contains more participants in organization 1 (54%) than in organization 2 (46%). The significant difference between the participants of each cluster in their answers to the questionnaires was tested statistically but no significant results were obtained.

7.2. Clusters from Subjective Measures

Depending on the answers of the participants to the Human-Machine cooperation questionnaire, 3 × 2 clusters were created (cf.

Table 7. ANOVA result of the clustering according to Human-Machine cooperation questionnaire and the final cluster centers.

	ANOVA
	Cluster		Error				Cluster Centers
	Mean Square	dof	Mean Square	dof	F	Sig.	Cluster 1	Cluster 2
Perception 1	28.219	1	0.828	42	34.098	<0.001	6	4
Analysis 1	50.037	1	0.917	42	54.574	<0.001	6	4
Decision 1	53.201	1	0.755	42	70.468	<0.001	5	3
Action 1	10.370	1	1.599	42	6.484	0.015	6	5
Info. Gathering 2	40.268	1	1.303	42	30.913	<0.001	4	6
Conflict detection 2	52.517	1	0.890	42	59.024	<0.001	3	6
Conflict managt 2	52.953	1	0.940	42	56.334	<0.001	3	5
Control of shuttles 2	32.317	1	1.730	42	18.680	<0.001	3	5
Conflict detection 3	41.613	1	0.924	42	45.023	<0.001	3	6
Conflict managt 3	53.062	1	1.159	42	45.795	<0.001	3	5
Authority managt 3	15.016	1	1.123	42	13.371	<0.001	4	6

¹ Shuttle KH functions. ² Human operator KHC functions. ³ Shuttles KHC functions.

). Cluster 1 is the group of participants who better evaluate their own KHC and that of the shuttles, and cluster 2 is the one who better evaluates the KH of the shuttles. Figure 10, Figure 11 and Figure 12 give a graphical representation of the clusters obtained from the shuttles KH, the HO KHC and the shuttles KHC functions variables, respectively.

According to the KH of the shuttles, the result shows a significant difference between cluster 1 and cluster 2 in the score achieved. Participants who had better evaluated the KH of the shuttles are those who obtained the highest score. The participants who better evaluate their own KHC and that of the shuttles are those who were those who felt that the workload was less, that they thought they were sufficiently trained, and that the experimental environment was not too complex. In addition, regarding the KHC shuttle, the participants who gave a better evaluation were those who used the assistance system (simulation support) the most (

Table 8. Averages of the participants’ performance per group and the result of the significance test.

Dependent Variables	Average (Std. Dev.)		p-Value (Confidence Level of 95%)
	Cluster 1	Cluster 2
Score	19,640 (77.11)	14,721 (61.86)	0.026
Workload	71.72 (7.18)	62.09 (16.68)	0.011
Sufficient training	3.85 (1.21)	4.84 (1.27)	0.028
Environment complexity	5.08 (1.44)	4.13 (1.43)	0.044
Use of assistance	0.667 (1.23)	3.21 (5.301)	0.019

).

The descriptive statistics reveal that most participants (58%) in cluster 1 of shuttles KH functions had access to the assistance system and 60% of participants in cluster 2 did not. Most participants in cluster 1 are frequent gamers (46%) and most of them in cluster 2 are not (45%). Cluster 1 related to the human operator KHC functions has 54% of participants that are from organization 1 and 58% in cluster 2 are from organization 2. Most of the participants in cluster 1 are not frequent gamers (54%), while 52% in cluster 2 are.

In addition, most participants in Cluster 1, which relies on KHC shuttles, did not have access to the support system (73%), which is in contrast to Cluster 2, where 62% of participants had access to the support system.

Additional clusters were created based on responses to the NASA-TLX questionnaire. The significant difference between clusters was in the assessment of time demand, effort, frustration, and overall workload. The results show that the participants in cluster 1 felt the most time demand, frustration, and workload, and participants in cluster 2 felt they put in the most effort (cf.

Table 9. ANOVA result of the clustering according to weighted NASA-TLX scales and overall workload, and the final cluster centers.

	ANOVA
	Cluster		Error				Cluster Centers
	Mean Square	dof	Mean Square	dof	F	Sig.	Cluster 1	Cluster 2
Mental demand	5595.01	1	11,782.96	42	0.475	0.495	300.37	323.53
Physical demand	12.379	1	3151.43	42	0.004	0.950	28.15	27.06
Temporal demand	50165.82	1	9693.184	42	5.175	0.028	188.89	258.23
Performance	11409.80	1	9202.148	42	1.240	0.272	187.78	154.70
Effort	35399.12	1	8269.430	42	4.281	0.045	140.37	82.12
Frustration	484020.4	1	5623.052	42	86.078	<0.001	45.18	260.58
Workload	2153.030	1	182.565	42	11.793	0.001	59.38	73.75

). Figure 13 is a graphical representation of the clusters obtained from the weighted subjective rating of the NASA-TLX scales and the overall workload.

The result of the significance test shows that participants in cluster 1 had the highest rating of the level of training and the lowest rating of the complexity of the experimental environment (cf.

Table 10. Average of the participants’ performance per group and the result of the significance test.

Dependent Variables	Average (Std. Dev.)		p-Value (Confidence Level of 95%)
	Cluster 1, n = 27	Cluster 2, n = 17
Sufficient training	5.50 (1.17)	4.19 (1.20)	0.002
Environment complexity	3.67 (1.37)	4.69 (1.45)	0.037

).

Descriptive statistics show that cluster 1 with the highest assessment of physical demand, performance, and effort has more frequent gamers (55%) than cluster 2 (18%). Most participants in cluster 1 are from organization 2 (63%) and most participants in cluster 2 are from organization 1 (59%). In addition, (59%) of participants in cluster 1 had access to the assistance system (simulator), and only (35%) in cluster 2 did.

8. Discussion and Lesson Learnt

This study was focused on the control and supervision of an IMS in an Industry 4.0 context. Two experiments were conducted to study the performance, workload, and cooperation of the human operator under two work organizations (Organization 1, one human operator performs the task individually. Organization 2, the task is shared between two human operators). Contrarily to our expectations, the results indicated that cooperative work on the control and supervision of an IMS task does not significantly affect the effectiveness and efficiency of the task. This may be related to other factors. During cooperative work, individual differences in personality characteristics are likely to influence a person’s behavior. Thus, team performance can be directly influenced by personality factors such as conscientiousness and extraversion [29]. In this sense, during the tests, we noticed that some participants did not have a communicative personality, which disrupted the quality of the discussion and provided less help between group members. In addition, it was found that participants’ assessment of time demand was affected by the type of organization. These participants had a higher level of time demand when they had to perform all tasks individually (organization 1) and they also had a significantly lower physical demand when they had to work in groups (organization 2). This result could be related to the fact that in teamwork, the human operator has to listen and respond to the other operator, which increases the physical activity required for the task, as the participants found. With regard to the aspects of human-machine cooperation, the level of feeling of control over the shuttles was higher in organization 2 than in organization 1. In teamwork, the task performed by each participant (organization 2) was less complex than in multitasking (organization 1). Referring to [30], multitasking and task complexity can lead to low performance and thus low cooperation. Lack of training on tasks, processes, and teamwork could also have an impact on participants’ teamwork behaviors and the team’s performance. Previous studies in real-life healthcare, aviation, military, and university settings have proven the effectiveness of teamwork training in promoting teamwork and performance [31]. For instance, it has been estimated that about 70% of adverse events in the medical field are not caused by individual technical errors, but by failures in teamwork [32]. Therefore, it is essential to ensure that training for teams is effective as it has a considerable impact on individual performance and teamwork effectiveness.

Table 11. Summary of the results obtained.

		Clustering Variables
		Score + Assistance		Cons. + Over-cons.		Shuttles KH		HO 1 KHC		Shuttles KHC		WL 2 + TD 3 + Frustration		PD 4 + Effort + Performance
		QCL_1(−)	QCL_2(+)	QCL_1(+)	QCL_2(−)	QCL_1(+)	QCL_2(−)	QCL_1(−)	QCL_2(+)	QCL_1(−)	QCL_2(+)	QCL_1(−)	QCL_2(+)	QCL_1(+)	QCL_2(−)
Factors (Independent variables)	Organization 2			−	+			−	+			+	−	+	−
	Assistance					+	−			−	+	+	−	+	−
	Gaming	−	+			+	−	−	+			+	−	+	−
Dependent variables	Training	+	−					−	+			+	−	+	−
	Env 5. Complexity	−	+					+	−			−	+	−	+
	Score					+	−
	Workload	−	+					+	−
	Mental demand	−	+
	Performance	−	+
	Frustration	−	+
	Use of assistance									−	+

¹ Human Operator. ² Workload. ³ Temporal Demand. ⁴ Physical Demand. ⁵ Environment.

summarizes the results obtained from the clustering method. The columns of the table represent the clustering variables and the rows the dependent and independent variables. The independent variables include organization 2, the presence of a support system (“assistance”), and number of frequent gamers (“gaming”). They allow for describing to some extent the population of each cluster. Under each clustering variable, there are two clusters (QCL_1 and QCL_2). Under each cluster, a (+) or (−) sign is written to indicate the highest and lowest cluster average. The relationship between the clusters, the independent and dependent variables are indicated by a − or + sign. For example, if a + is found in the cell joining the QCL_1 column and the organization 2 row, and a - in the cell joining the QCL_2 column and the organization 2 row, this means that cluster 1 contains more participants from organization 2 than cluster 2. A sign in the row of a dependent variable shows which of the two clusters has the higher and lower average.

In summary, the results showed that in addition to the work organization, other personal parameters, such as the frequency of playing video games could affect the performance and state of the operator. The presence of a support system also tends to have an effect. Patterns of cooperation can be identified by analyzing the activity of the participants during organization 2 as proposed in [33].

Consequently, from our experimental work, it is possible to list some principles, yet to be confirmed by complementary studies, that could be useful to design effective human-machine team cooperation in the context of Manufacturing 4.0:

• Ability in video games influences significantly the use of an assistance system by the participants, the participants’ performance, the self-assessment of their ability to cooperate, their perceived workload, temporal demand, frustration, physical demand, effort, and performance.
• Team working improves performance by reducing energy consumption and over-consumption. It enhances the participants’ self-assessment of their KHC, which tends to reduce the perceived workload, temporal demand, and physical demand. Nevertheless, teamwork can also increase physical demand and effort.
• The participants who performed better and made the most use of the assistance system were those who felt they had not been sufficiently trained. In addition, they overstated the complexity of the experimental environment.
• The participants who rated their KHC best were those who thought they had a good level of training and were those who underestimated the complexity of the experimental environment.
• The participants who experienced lower workload, temporal demand, and frustration were those who felt they were sufficiently trained and that the environment was not highly complex. However, these participants experienced a greater physical demand and felt they had put in more effort, and were less satisfied with their performance in accomplishing their goals (as introduced by the NASA-TLX questionnaire, the performance scale is reversed. In other words, a high value corresponds to low satisfaction).

9. Conclusions

This paper aimed to test the hypotheses according to which teamwork in performing a complex task in industry 4.0 would improve the effectiveness and efficiency of the task, reduce the workload of the human operator and improve cooperation. The results showed a lower level of participants’ assessment of time demand and physical demand in teamwork conditions. It was also found that teamwork improves the subjective human operator Know-how-to-cooperate when controlling autonomous entities such as shuttles. In some statistical tests, the significance is not reached, which did not allow us to confirm all of the hypotheses. Nevertheless, the results showed a trend in line with our expectations, and their analysis allowed us to speculate on the impact of other factors on teamwork effectiveness and performance, such as individual personality, and lack of training. In further work, complementary analyses of participants’ activities from voice and screen recordings will be carried out to identify other factors that have an impact on task completion and cooperation between participants. This would allow the extraction of patterns and get some possible structures of human-human cooperation.