Multimodal Technol. Interact., Volume 7, Issue 4 (April 2023) – 8 articles

A new approach to learning is mobile learning (m-learning), which makes use of special features of mobile devices in the education sector. M-learning is becoming increasingly common in higher education institutions all around the world. The use of mobile devices for education and learning has also gained popularity in Jordan. Unlike studies about Jordan, there are many studies that thoroughly analyze the situation of m-learning in other countries. Thus, it is important to understand the current situation of m-learning at Jordanian universities, especially in light of the COVID-19 pandemic. While there have been some studies conducted prior to COVID-19 and a few studies after COVID-19, there is a need for a comprehensive study that provides an in-depth exploration of the current situation, student adoption, benefits, disadvantages, and challenges, particularly following COVID-19. Therefore, this study utilizes a sequential exploratory mixed research method to investigate the current state of the art of m-learning in Jordanian higher education with a particular focus on student adoption, benefits, disadvantages, and challenges. Firstly, the study explores the existing literature on m-learning and conducts 15 interviews with educators and learners in three Jordanian universities to gain insights into their experiences with m-learning. The study then distributes a survey to students at four Jordanian universities, representing both public and private universities, to generalize the results from the qualitative study. Additionally, the study investigates the relationship between student enrollment in public/private universities and the adoption of m-learning. The study came to the conclusion that students have a positive opinion of m-learning and are also willing to use it. However, there are a number of disadvantages and challenges to its adoption. Additionally, there is a relationship between student enrolment in public/private universities and the adoption of m-learning. These findings have important implications for institutions that want to incorporate m-learning into their undergraduate and graduate degree programs, as they aid decision-makers in these universities in creating frameworks that may be able to meet the needs of m-learning. Full article

This paper discusses the potential benefits of using augmented reality (AR) technology to enhance human–robot collaborative industrial processes. The authors describe a real-world use case at Siemens premises in which an AR-based authoring tool is used to reduce cognitive load, assist human workers in training robots, and support calibration and inspection tasks during assembly tasks. The study highlights the potential of AR as a solution for optimizing human–robot collaboration and improving productivity. The article describes the methodology used to deploy and evaluate the ARContent tool, which demonstrated improved usability, reduced task load, and increased efficiency in the assembly process. However, the study is limited by the restricted availability of workers and their knowledge of assembly tasks with robots. The authors suggest that future work should focus on testing the ARContent tool with a larger user pool and improving the authoring tool based on the shortcomings identified during the study. Overall, this work shows the potential for AR technology to revolutionize industrial processes and improve collaboration between humans and robots. Full article

Stimulus Equivalence (SE) is a behavioural phenomenon in which organisms respond functionally to stimuli without explicit training. SE provides a framework in the experimental analysis of behaviour to study language, symbolic behaviour, and cognition. It is also a frequently discussed matter in interdisciplinary research, linking behaviour analysis with linguistics and neuroscience. Previous research has attempted to replicate SE with computational agents, mostly based on Artificial Neural Network (ANN) models. The aim of this paper was to analyse the effect of three Training Structures (TSs) on stimulus class formation in a simulation with ANNs as computational agents performing a classification task, in a matching-to-sample procedure. Twelve simulations were carried out as a product of the implementation of four ANN architectures on the three TSs. SE was not achieved, but two agents showed an emergent response on half of the transitivity test pairs on linear sequence TSs and reflexivity on one member of the class. The results suggested that an ANN with a large enough number of units in a hidden layer can perform a limited number of emergent relations within specific experimental conditions: reflexivity on B and transitivity on AC, when pairs AB and BC are trained on a three-member stimulus class and tested in a classification task. Reinforcement learning is proposed as the framework for further simulations. Full article

In this paper, we investigate the impact of avatar personalization on perceived emotions. Avatar embodiment is a crucial aspect of collaborative and social virtual reality (VR) systems. Previous research found that avatar appearance impacts the acceptability of the virtual body and changes users’ behavior. While virtual embodiment has been extensively investigated, we know very little about how embodiment affects users’ experienced emotions. In a user study (N = 40), we applied an autobiographical recall method to evoke happiness and investigated the influence of different types of avatar embodiment (personalized same-gender, personalized opposite-gender, non-personalized same-gender, and non-personalized opposite-gender) on participants’ perceived emotion. We recorded both self-reported assessments and physiological data to observe participants’ emotional responses resulting from the emotions elicited by the use of different avatars. We found significant differences in happiness with the personalized same-gender avatar and the personalized opposite-gender avatar. We provide empirical evidence, demonstrating the influence of avatar personalization on emotions in VR. We conclude with recommendations for users and designers of virtual reality experiences. Full article

In this paper, we investigate the effect of distractions and hesitations as a scaffolding strategy. Recent research points to the potential beneficial effects of a speaker’s hesitations on the listeners’ comprehension of utterances, although results from studies on this issue indicate that humans do not make strategic use of them. The role of hesitations and their communicative function in human-human interaction is a much-discussed topic in current research. To better understand the underlying cognitive processes, we developed a human–robot interaction (HRI) setup that allows the measurement of the electroencephalogram (EEG) signals of a human participant while interacting with a robot. We thereby address the research question of whether we find effects on single-trial EEG based on the distraction and the corresponding robot’s hesitation scaffolding strategy. To carry out the experiments, we leverage our LabLinking method, which enables interdisciplinary joint research between remote labs. This study could not have been conducted without LabLinking, as the two involved labs needed to combine their individual expertise and equipment to achieve the goal together. The results of our study indicate that the EEG correlates in the distracted condition are different from the baseline condition without distractions. Furthermore, we could differentiate the EEG correlates of distraction with and without a hesitation scaffolding strategy. This proof-of-concept study shows that LabLinking makes it possible to conduct collaborative HRI studies in remote laboratories and lays the first foundation for more in-depth research into robotic scaffolding strategies. Full article

Virtual Reality (VR) technologies have the potential to be applied in a clinical context to improve training and rehabilitation for individuals with hearing impairment. The introduction of such technologies in clinical audiology is in its infancy and requires devices that can be taken out of laboratory settings as well as a solid collaboration between researchers and clinicians. In this paper, we discuss the state of the art of VR in audiology with applications to measurement and monitoring of hearing loss, rehabilitation, and training, as well as the development of assistive technologies. We review papers that utilize VR delivered through a head-mounted display (HMD) and used individuals with hearing impairment as test subjects, or presented solutions targeted at individuals with hearing impairments, discussing their goals and results, and analyzing how VR can be a useful tool in hearing research. The review shows the potential of VR in testing and training individuals with hearing impairment, as well as the need for more research and applications in this domain. Full article

To improve the perception of haptic feedback, materials and sense-modifier effects should be examined. Teflon, Nylon mesh, and Silicone overlays were tested in combination with lateral vibrations to study their impact on the tactile sense. A feelable point moving along a line was implemented through the use of a dynamically moving interference maximum generated via the offset actuation of four haptic exciters affixed to corners of a Gorilla Glass surface. This feedback was presented to eight participants in a series of randomized experiments. Both the Nylon mesh and Teflon covering revealed a statistically significant (p < 0.05) impact of improvement to the user performance in the task of dynamic haptic virtual straight lines localization. While Silicone covering, having three times greater friction than Gorilla Glass, has less or no impact on both decision time, the number of task repetitions, and error rate (p > 0.05). The lateral vibration modifier (60 Hz) can also successfully be used with an increase in performance by about twofold, at least that was demonstrated for both the Nylon mesh and Teflon covering. Full article

This study attempted to model the stag and hare hunting behaviors of students using their interaction data in a mobile computer-supported collaborative learning application for Grade 5 mathematics. Twenty-five male and 12 female Grade 5 students with an average age of 10.5 years participated in this study. Stag hunters are more likely to display personality dimensions characterized by Openness while students belonging to hare hunters display personality dimensions characterized by Extraversion and Neuroticism. Students who display personality dimensions characterized by Agreeableness and Conscientiousness may tend to be either hare or stag hunters, depending on the difficulty, types of arithmetic problems solved, and the amount of time spent solving arithmetic problems. Students engaged in a stag hunting behavior performed poorly in mathematics. Decision tree modeling and lag sequential analysis revealed that stag and hare hunting behaviors could be identified based on personality dimensions, types of arithmetic problems solved, difficulty level of problems solved, time spent solving problems, and problem-solving patterns. Future research and practical implications were also discussed. Full article