Next Article in Journal
Can AI-Oriented Requirements Enhance Human-Centered Design of Intelligent Interactive Systems? Results from a Workshop with Young HCI Designers
Next Article in Special Issue
Developing Usability Guidelines for mHealth Applications (UGmHA)
Previous Article in Journal / Special Issue
A Review of Design and Evaluation Practices in Mobile Text Entry for Visually Impaired and Blind Persons
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
MTI
Volume 7
Issue 3
10.3390/mti7030023
mti-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Comparison of Using an Augmented Reality Learning Tool at Home and in a Classroom Regarding Motivation and Learning Outcomes

by
Aldo Uriarte-Portillo
1,
María Blanca Ibáñez
2,*,
Ramón Zatarain-Cabada
1 and
María Lucía Barrón-Estrada
1
1
Tecnológico Nacional de México Instituto Tecnológico de Culiacán, Guadalupe, Culiacán Rosales 80220, Mexico
2
Department of Telematics Engineering, Carlos III University of Madrid, 28911 Leganés, Spain
*
Author to whom correspondence should be addressed.
Multimodal Technol. Interact. 2023, 7(3), 23; https://doi.org/10.3390/mti7030023
Submission received: 10 February 2023 / Revised: 20 February 2023 / Accepted: 22 February 2023 / Published: 23 February 2023
(This article belongs to the Special Issue Feature Papers in Multimodal Technologies and Interaction—Edition 2023)
Browse Figure
Versions Notes

Abstract

:
The recent pandemic brought on considerable changes in terms of learning activities, which were moved from in-person classroom-based lessons to virtual work performed at home in most world regions. One of the most considerable challenges faced by educators was keeping students motivated toward learning activities. Interactive learning environments in general, and augmented reality (AR)-based learning environments in particular, are thought to foster emotional and cognitive engagement when used in the classroom. This study aims to compare the motivation and learning outcomes of middle school students in two educational settings: in the classroom and at home. The study involved 55 middle school students using the AR application to practice basic chemistry concepts. The results suggested that students’ general motivation towards the activity was similar in both settings. However, students who worked at home reported better satisfaction and attention levels compared with those who worked in the classroom. Additionally, students who worked at home made fewer mistakes and achieved better grades compared with those who worked in the classroom. Overall, the study suggests that AR can be exploited as an effective learning environment for learning the basic principles of chemistry in home settings.

1. Introduction

The emergency caused by the recent pandemic showed the need to implement effective pedagogical approaches to keep students engaged and motivated during long periods of online learning [1]. To prevent a lack of motivation among students, educational institutions needed to integrate new modalities of teaching and learning into existing pedagogical resources through interactive technologies [2].
Previous studies in the field of education have provided solid evidence regarding the positive impact of interactive technologies for teaching and learning, including virtual reality (VR) and augmented reality (AR). These technologies are potentially effective in promoting affective and cognitive learning due to their capabilities to allow students the possibilities of changing visual perspective when observing educational objects and of interacting in real time with these objects [3,4,5,6]. In the case of AR, students have an additional possibility to interact both with real and digital objects integrated into the learning environment, which can lead to greater involvement in the learning tasks [7]. There is broad agreement among researchers that AR-supported teaching fosters motivation toward learning activities, which contributes to improved learning outcomes [8,9,10].
Despite the growing interest in using AR-supported teaching, little is known about whether this technology has an impact on students’ motivation and learning outcomes when used at home [11]. In this paper, we provide evidence from the study designed to evaluate the effectiveness of using AR during learning activities at home or in the classroom in terms of students’ motivation and learning outcomes.
This paper provides an overview of school and home learning environments, the effects of motivation in education, and the impact of AR in education (Section 2). Section 3 presents an overview of the AR-based learning application designed for this study, while Section 4 describes the research methodology. Section 5 presents the results of the system evaluation, and in Section 6 we discuss the findings. Finally, Section 7 summarizes the conclusions and contributions of this article.

2. Background

Success in education is influenced by numerous factors, with the learning environments at home and school being particularly significant. These environments can significantly affect a student’s motivation, which is a crucial element in education. Motivation impacts a student’s academic performance. Another emerging trend in education is the use of augmented reality (AR), which has the potential to transform how students learn and interact with the world around them. In this section, we provide an overview of these topics and examine their impact on modern education.

2.1. School and Home Learning Environments

Most distance learning is delivered via the Internet, which students can access from the comfort of their own homes or from public libraries [12]. Distance learning can be separated into two categories: synchronous and asynchronous delivery. Synchronous distance learning entails that students and their teachers have real-time interactions. Meanwhile, asynchronous distance learning occurs when the teacher and student engage in different places and at different times [13].
Several authors have contrasted the benefits and drawbacks of synchronous remote learning compared with face-to-face strategies. In terms of interaction, the school setting fosters relationships between teachers and students. While teacher-to-student interaction is critical for passing on information and providing feedback, student-to-student interaction occurs naturally at school. Both types of interactions are determinants of student perceived learning and satisfaction [14,15]. In the context of remote learning, this feeling of connection with teacher and peers must be mediated through technology. Furthermore, distractions are more difficult to control in distance learning, where interaction with teacher and peers decreases compared with in-class settings [12]. In at-home settings, student self-regulation is vital to deal with distractions [16]. Finally, insufficient Internet access, a lack of appropriate digital equipment, and a low degree of digital competency are all challenges faced during synchronous distant learning [17].
The limitations associated with distance learning have underscored the need to deploy instructional learning strategies adapted to at-home environments. These strategies should provide learning content to students, issue learning activities that foster motivation and engagement, implement necessary scaffolding support for students who lack self-regulation strategies, and facilitate interaction with the professor, peers, and learning content [18,19].

2.2. The Effects of Motivation in Education

Motivation is an internal state that drives a person to act in a certain way to achieve a specific goal. Motivation is seen as a critical component of learning performance. Students that are highly motivated are more likely to put more effort into their studies, resulting in better learning outcomes [20,21,22].
This goal-oriented behavior has been assessed by some educators through self-determination theory (SDT) [23]. According to SDT, motivation can be divided into two categories; intrinsic and extrinsic, both of which represent strong forces in determining who we are and how we act [24]. Extrinsic motivation refers to behaviors that result in outcomes such as rewards or avoided punishments but have a limited level of perceived autonomy. Intrinsic motivation is understood as an inner drive that propels a person to pursue an activity because the activity is aligned with their values or interests, they perceive a freedom of choice, and they believe that they can execute the activity successfully.
On the other hand, Keller’s theory [25] has recently been used to examine how a learning environment such as AR affects students’ motivation [26,27]. Keller’s framework considers four factors: attention, relevance, confidence, and satisfaction (ARCS). In this context, the theory states that the design process for a learning environment should aim to grab and maintain learners’ attention, then ensure that learning activities are matched with learners’ personal objectives and needs so that they are viewed as relevant by the student. Learners will gain confidence only once they feel in charge of and satisfied with the completion of learning activities.

2.3. Augmented Reality in Education

AR is a technology that merges the real and virtual worlds in real time, augmenting the real environment with computer-generated virtual items [28]. Since The Horizon Report [29] anticipated that AR would have a positive impact on learning, creativity, and education, AR has become one of the top educational technology trends. However, at the time, the technology was not completely available on consumer devices. Circumstances have changed in recent years. For instance, AR no longer requires specialized equipment and can be easily utilized on mobile devices; additionally, AR now allows multiple modes of interaction, making it potentially valuable in the design of various learning experiences. Consequently, educational researchers have increasingly explored potential opportunities for teaching and learning using AR [30,31]. The humanities and arts are among the most popular educational environments in which AR technology is used [32]; eHealth [26,33] has been successfully used to reproduce medical situations in controlled environments, and STEM (science, technology, engineering, and mathematics) subjects such as biology [34,35], physics [36], chemistry [37,38], and mathematics [39] have been useful mainly to simulate experiments.
The way AR is planned, deployed, and integrated into formal and informal learning environments has a direct impact on its educational usefulness [31]. High-quality learning experiences can be scarce, and interactions with the learning environment should aid rather than hinder the teaching-learning process. Learners’ emotional states, on the other hand, contribute to boosting students’ cognitive processes and learning outcomes after high-quality interactions with the learning environment are accomplished [29]. AR-based learning environments have been tested to determine their capacities to assist students in knowledge acquisition when it comes to learning affordances of AR technology [40,41]; knowledge retention [42]; enhancing spatial skills [43,44]; helping with spatial reasoning [45]; and solving given STEM problems [35,46]. Furthermore, there is growing interest in determining the influence of AR learning environments on student motivation. Some studies have used SDT motivation theory to investigate the impact of AR-based learning environments on student motivation [34,47]. Finally, some works explore the impact of AR on students’ motivation toward science learning [35,48,49]. Despite the large number of studies that examine the impact of AR in education, few have been evaluated in educational contexts other than classrooms, laboratories, and museums [50,51,52]. Therefore, the level of educational effectiveness that this technology can have in at-home settings is unknown.
The aforementioned studies were considered to set up the aim of this article, which is to analyze whether AR learning activities can keep students’ motivation and learning achievement at similar levels in two different learning settings: in classroom and at home.

3. Learning Application

We developed ReAQ, which is a marker-based AR learning application to practice basic chemistry principles, including chemical bonds and chemical reactions. The application was developed by applying the successive approximation model 2 (SAM2) [53,54] with the active participation of teachers from the educational institutions involved in this study. ReAQ was developed using Vuforia Augmented Reality Software Development Kit [55] combined with Unity 3D game engine [56]. With an Internet connection and a camera, ReAQ may be accessed from a PC or a mobile device. ReAQ was specifically designed to be user-friendly for children, whether they used it in the classroom or at home.
One of the most important requirements of the ReAQ application is to manage the progress of students when solving the exercises and give appropriate feedback on their performance. ReAQ’s graphical interface must be user-friendly and easy to use and must be adaptable to any screen resolution of a mobile device.
ReAQ was structured around three topics: (1) making up a chemical compound; (2) producing a chemical reaction; and (3) identifying chemical elements. These activities required students’ manipulation of 3D shapes (25 markers), each one representing a different chemical element, like hydrogen, chlorine, oxygen, etc. Students must solve several problems in each topic. When a problem is posed, students must choose the marker representing the chemical elements required, collide them if necessary, and either observe the simulated chemical reaction when answering correctly or receive feedback otherwise. At all times, the “Help button” is available for students to ask their instructor for clarification.
Figure 1 shows students using ReAQ to solve exercises. In the upper-left and the lower-right image, the goal is to form a metallic bond. The student makes a collision of two markers (carbon and iron) to form the steel metallic bond. When the edge of both markers is detected, the 3D element is activated and enlarged on the screen. Similarly, in the upper-right image, an explosive effect is triggered by the collision of two markers representing an acid and a base. Finally, the lower-left image shows the collision of two non-metallic elements (oxygen with hydrogen) to form water. ReAQ also displays some information on the screen revealing the electronegativity level (electrons) between the two elements (less than 1.7).

4. Research Design

The study uses an image-based AR application for practicing basic chemistry principles. The application was designed by the authors of this study following general principles stated by researchers exploring the impact of AR on education [35,49], with the aim of fostering students’ motivation toward interactive learning activities. The tool was enhanced with additional information of the activities’ learner-content interaction, with the aim of fostering students’ satisfaction and success [15]. Finally, the tool includes a log module, useful to assess the interactive behavior of the student with the tool.
The study compared the use of an AR-based learning tool in two learning settings: in class and at home. The learning setting (in class, at home) was used as an independent variable in a quantitative quasi-experimental/control group design.
The effect of the learning setting on students’ learning outcomes, as indicated by a knowledge exam on basic chemical topics, was investigated using a pretest/post-test approach. Logs containing the number of errors, calls for help, and time spent were compared to evaluate the differences in students’ involvement with the AR-based learning tool in the two learning contexts. Students completed the IMMS survey to investigate the impact of the learning environment on students’ motivation in terms of attention, relevance, confidence, and satisfaction with the learning activities given by the AR-based learning tool [57]. Finally, utilizing the learning environment as an independent variable, the study investigated the impact of motivation on students’ interactions with the AR-based learning tool.
The main research questions in this study try to explore whether AR technology can support guided learning activities at home by maintaining student motivation and learning outcomes at similar levels to those obtained by performing the same activities in a classroom:
RQ1: Is there a difference in the students’ learning outcomes based on the physical learning spaces that they used?
RQ2: Is there any difference in students’ motivation influenced by which of the physical learning spaces were used?
RQ3: Is there any difference in students’ interaction with the AR-based learning environment depending on which of the physical learning spaces were used?
RQ4: How does student motivation relate to students’ interaction with the AR-based learning environment?

4.1. Participants

Participants in this study were ninth grade students from various schools in Mexico. All schools use the same syllabus for the ninth grade chemistry course. In each school, participants were enrolled in the same chemistry course. The sample consists of 55 students (ages 13–15, M = 14.25, SD = 0.48). Of these, 24 were part of a control group (13 male, 11 females, ages 13–15, M = 14.25, SD = 0.509); 25 were part of the experimental group (12 males, 13 females, ages 13–15, M = 14.04, SD = 0.510), and six students were not considered in the analysis because they did not complete the tests. Whereas the control group used the ReAQ tool as a face-to-face activity, the experimental group used the ReAQ tool as a remotely guided activity.
All students and their parents received thorough information detailing the investigation’s goal as well as their right to remain in the study or withdraw at any moment. All participants who agreed to participate in the study signed a consent letter.

4.2. Measurement Instruments

4.2.1. Questionnaires

We completed knowledge pretests and post-tests to evaluate the success of the interventions on learners’ comprehension of chemical bonds and chemical reaction concepts. Both pretest and post-test contained 10 multiple-choice questions, valued at 1 point each. The researchers designed the test and teachers examined and validated their content. The following represents an example of a question from one of these exams.
Besides Hydrogen, what other element is part of Ammonia?
(a)
Carbon
(b)
Oxygen
(c)
Nitrogen
(d)
Phosphorus

4.2.2. Motivation Questionnaire

We evaluated the students’ motivation as they read the instructional material. For the evaluation, we used an IMMS survey [57]. IMMS contains 36 questions with 5-point Likert-scale items to measure motivation aspects in the ARCS model, which are: attention (12 items), relevance (9 items), confidence (9 items), and satisfaction (6 items).

4.2.3. Interaction Logs

Three types of learners’ interactions with the ReAQ tool were stored in log files: (1) the number of errors by each student while performing the learning activities; (2) the number of times each student requested clues within the learning tool; and (3) the time spent performing the required learning activities.

4.3. Procedure

A week before the study kickoff, all the students participating in the intervention received a lecture from their teachers regarding the topics in the activity. All the subjects in this study followed the same chemistry curriculum for Mexican students. Thereafter, all the students answered the pretest questionnaire. In the subsequent week, students in the control group received a mobile device with the application installed. Meanwhile, the students of the experimental group downloaded the ReAQ application to their own mobile devices from Google store or via a Web link.
The students received a file with instructions to install ReAQ on the device before they began any interaction with the app. The interventions lasted 20 min. Throughout this time, researchers were present in the control group while the students interacted with the application and responded directly when they needed technical help. For students in the experimental group, the application was installed on the device they used for remote learning. Students, teachers, and researchers all participated in the virtual session for the intervention. Students who needed technical or procedural help asked questions via videoconference or chat using the online platform. At the end of the interventions, students answered a 20-min knowledge post-test. After that, students had approximately 20 min to respond the motivation survey.

5. Results

This section presents the study’s results related to the research questions. The research questions were formulated to analyze whether AR learning activities could maintain students’ motivation and learning achievement at similar levels in two distinct learning settings: in class and at home.

5.1. Samples Equivalence

The Shapiro–Wilk test was used to confirm the normal distribution of the data collected in the pretest for the control group (n = 24, w = 0.159, M = 5.75, SD = 1.70) and the experimental group (n = 25, w = 0.163, M = 6.24, SD = 1.392). These findings suggested that the difference in scholars’ prior knowledge might be described by a normally distributed population. As a result, parametric tests might be employed to assess the remaining analyses.
An independent sample test was used to examine the prior knowledge of students in the control (classroom) and experimental (home) groups. Results (t [49] = −1.106, p = 0.275) demonstrated that there was no statistically significant difference between students in the control and experimental groups. Therefore, before the intervention, the students of both groups had similar previous knowledge about the topic of chemical bonds.

5.2. Research Question 1. Is There a Difference in the Students’ Learning Outcomes Based on the Physical Learning Spaces That They Used?

To ensure a normal distribution in the data from the control group, we applied a Shapiro–Wilk test for both the pretest (M = 5.75, SD = 1.700, w = 0.159) and post-test (M = 6.96, SD = 1.459, w = 0.492), with findings which described a normal data distribution. Based on the data from the pretest and post-test, a t-test paired sample was used to compare the learning results of students in the control group. The result revealed a statistically significant difference between the students’ knowledge before and after the intervention using the AR learning tool (t [24] = −4.608, p < 0.001). The effect size is d = 0.76, indicating that using the AR learning tool in face-to-face mode enhanced learning outcomes for students in the control group. This is considered a medium-effect size.
Data for the experimental group was also evaluated using the Shapiro–Wilk test, both for the pretest (M = 6.24, SD = 1.393, w = 0.163) and post-test (M = 7.88, SD = 1.424, w = 0.101) scores, indicating a normal data distribution. The t-test paired sample was conducted to compare the learning outcomes of the students of the experimental group based on the data from the pretest and post-test. The result (t [25] = −10.119, p < 0.001) when comparing the students’ knowledge before and after the intervention using the AR learning tool was statistically significant. The effect size is d = 1.09, highlighting that those students in the experimental group had improvements in the learning outcome by using the AR learning tool at home. This is considered a large-effect size.
A Shapiro–Wilk test was applied to determine the normal distribution of the data generated in the post-test, for the control group (n = 24, w = 0.492, M = 6.96, SD = 1.459) and the experimental group (n = 25, w = 0.101, M = 7.88, SD = 1.424), identifying a normal distribution. As a result, the variation in prior knowledge across students could be due to a population with a normal distribution. Based on the post-test data, an independent samples t-test was used to compare students’ learning improvements between the control (classroom) and experimental (home) groups. The result (t [49] = −2.238, p = 0.03) showed that there was a statistically significant difference when comparing the post-test data among students in the control and experimental groups, favoring the experimental arm of the study, with an effect size of d = 0.63, which is considered a medium-effect size.
The adoption of the ReAQ application technology resulted in an overall increase in knowledge, although the subjects in the experimental group had a significantly improved learning outcome compared with subjects in the control group.

5.3. Research Question 2. Is There Any Difference in Students’ Motivation Influenced by Which of the Physical Learning Spaces Were Used?

The instrument IMMS has a score range of 36 to 180 using a response Likert´s scale from 1 to 5. In the control group, the total scores ranged from 118 to 169, compared with 122 to 173 in the experimental group. With these results, we assume that the motivation of students of both groups was moderately high. Because the difference in motivation is from a normal distribution, we applied a Shapiro–Wilk test to assess the distribution of the difference in motivation between the two groups (W = 0.956, p-value = 0.062). As the results show a normal data distribution, we proceeded with the use of parametric tests for the remainder of the analyses.
The results of the two independent samples t-tests are shown in Table 1. Results from this comparison highlight a numerically higher degree of motivation in students in the experimental group (4.17) compared with those in the control group of the study (4.07); however, this difference did not reach statistical significance (t = −0.823, p < 4.15).
Table 2 shows descriptive data regarding motivation based on the ARCS model’s four subscales. The mean values for the attention and satisfaction subscales were significantly higher among students in the experimental group compared with those in the control group.
We used a Shapiro–Wilk test to investigate the difference between two teaching scenarios for each ARCS factor, concluding that the difference is due to a normal distribution: attention (W = 0.955, p-value = 0.13), relevance (W = 0.958, p-value = 0.081), confidence (W = 0.965, p-value =0.156), and satisfaction (W = 0.956, p-value = 0.064). We used an independent samples t-test to compare the training scenarios across the four parameters (see Table 2).
The results of the intervention revealed that both groups of students were motivated in the same way by the AR-based learning environment. When looking at the motivating elements examined by the IMMS instrument, statistically significant variations in attention and satisfaction were identified, favoring the experimental group in both cases.

5.4. Research Question 3. Is There Any Difference in Students’ Interaction with the AR-Based Learning Environment Depending on Which of the Physical Learning Spaces Were Used?

Table 3 illustrates descriptive statistics for the three types of interactions considered: number of errors, amount of help requested, and time spent on learning activities. The control group’s mean values for the three measures were found to be considerably higher when compared with the experimental group.
We ran a Shapiro–Wilk test on each of the three interactions to see if there was a difference when the two instructional scenarios were used. The difference between each factor in both teaching scenarios comes from a normal distribution. Independent samples t-tests were used to compare the teaching scenarios across the three types of interactions (Table 3).
The results of the intervention suggest that students from the experimental group made fewer errors, asked for fewer helping clues, and needed less time to complete their work compared with students from the control group.

5.5. Research Question 4. How Does Student Motivation Relate to Students’ Interaction with the AR-Based Learning Environment?

A Pearson product–moment correlation coefficient demonstrated a negative correlation between the attention and satisfaction factors and the number of errors students made in the experimental group. A high positive correlation was detected between attention and satisfaction factors and the time spent by students to finish their learning activities; there was also a slight correlation between relevance and confidence factors with the time variable. There was no correlation for any motivation factor with the amount of help students requested during the intervention (see Table 4).
A Pearson product–moment correlation coefficient demonstrated a negative correlation between confidence and satisfaction factors and the number of errors students made in the control group. There was no correlation for any motivation factor with the amount of help students requested during the intervention, nor with the time students spent completing the intervention (see Table 5).
Results suggest that, for the experimental group, attention toward the learning activities is higher when the students spend more time in the activities and make fewer mistakes. Similarly, results suggest that students who feel more satisfied with the learning activities tend to spend more time on the activities and make fewer mistakes.
Results in the control group show that the number of errors made by students is inversely correlated with their confidence and level of satisfaction.

6. Discussion

In this study, ReAQ, an AR application designed to learn basic chemistry principles, was presented and evaluated. The study was planned to compare the use of the same AR-based learning tool in two physical learning settings: in a face-to-face class and at home in a virtual class. The study aimed to identify possible differences in terms of motivation, learning outcomes, and students’ interactions with the AR-based learning tool. The main findings of this study, as well as their implications, are discussed below.
Results from this study show that the use of an AR tool in both physical and virtual learning environments increases student motivation. This finding is aligned with other studies which have used the IMMS instrument to measure motivation towards a significant number of AR-based learning environments, many of which have identified a positive impact on motivation [35,42]. However, our results revealed that there were substantial differences in the attention and satisfaction factors between groups, favoring the experimental group. These results partially contradict findings reported by recent studies which indicate low levels of satisfaction among students using online platforms [58,59]. To provide solutions to this problem, some researchers conducted empirical analysis to determine factors influencing student satisfaction in online educational environments. Several of these studies agree that teacher–student interaction is a determining factor for student satisfaction [59,60,61,62]. Regarding the attention factor, some researchers claim that in online environments students fail to maintain high levels of attention. [63,64] suggest that learning environments with high cognitive load and home distractions are among the main causes of the lack of attention in these educational settings [65,66]. Some researchers have suggested that teacher supervision in online environments avoids student distractions [67]. In our case, the effort to guide the students both at home and in the classroom could have contributed to the measures obtained in the factors of attention and satisfaction in the experimental group, which had a home setting. Further studies are needed to identify why this teaching support had a greater impact at home compared with in the classroom.
Our findings suggest that working at home with an AR-based learning tool confers a direct benefit in terms of learning outcomes compared to working in class with the same learning tool. Several authors have argued that students performing learning activities at home can have modestly better outcomes compared with those receiving face-to-face instruction at school [68,69]. However, other authors maintain a neutral position and affirm that factors such as teachers’ presence, the quality of content, or students’ social conditions are responsible for the impact of educational settings on learning outcomes [70,71]. Our results suggest that students working at home performed significantly better, made fewer errors, required less help, and needed less time to complete the activities compared with students who worked in a classroom setting. We also identified a positive correlation between the attention factor and the time spent on the activities and a negative correlation between the satisfaction factor and the number of errors made. These results suggest that motivational factors had a positive impact on learning outcomes. Further studies are required to establish whether our findings are due to learning affordances of AR technology, the learning setting used, or a combination of both.
Finally, we point out some of the study’s limitations: (1) the study was designed to evaluate short-term retention of basic chemistry concepts, and a long-term retention evaluation was not included, but we believe it would be highly valuable to assess the effectiveness of the AR-based application; and (2) collected data were self-reported.

7. Conclusions and Future Work

The results presented in this study lead us to conclude that the setting where learning activities are performed has an impact on different motivational factors and on learning outcomes. It seems that efforts to develop AR-based learning tools that attract students’ attention and help them feel satisfied will motivate students to spend more time on their activities and to complete them more carefully. This opens the possibility of conducting online guided learning activities that are at least as effective as those that are performed in a classroom. Future studies should explore the reproducibility of these findings across other STEM disciplines and with students of different academic levels.

Author Contributions

All authors contributed to the study, conceptualization, and methodology of this work; the software was developed by A.U.-P.; the validation and the formal analysis by M.B.I. and A.U.-P.; the investigation and resources by M.B.I., R.Z.-C., and M.L.B.-E.; writing—original draft preparation by A.U.-P. and M.B.I.; writing—review and editing by all the authors; supervision by M.B.I., R.Z.-C., and M.L.B.-E. All authors have read and agreed to the published version of the manuscript.

Funding

The authors acknowledge the eMadrid Network, which is funded by the Madrid Regional Government (Comunidad de Madrid) with grant No. P2018/TCS-4307. This work also received partial support from FEDER/Ministerio de Ciencia, Innovación y Universidades—Agencia Estatal de Investigación through project Smartlet (TIN2017-85179-C3-1-R) and received partial support from MCIN/AEI/10.13039/501100011033. The publication is part of the I + D+i project “H2O Learn” (PID2020-112584RB-C31). We also acknowledge the PRODEP program in Mexico that supported this work (511-6/2019-8474).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Acknowledgments

We value the help provided by teachers and directors of Escuela Secundaria SNTE Seccion 53, located in Culiacan, México.

Conflicts of Interest

The authors declare that they agree to make the results of the research a variable for scientific purposes only, without any restrictions. Moreover, the authors declare that the tests and surveys were conducted confidentially and that they alone had access to the results while performing the research. There was no violation of any personal rights of the people involved in the research. The authors have no conflicts of interest to declare that are relevant to the content of this article.

References

  1. Huang, R.; Tlili, A.; Chang, T.-W.; Zhang, X.; Nascimbeni, F.; Burgos, D. Disrupted classes, undisrupted learning during COVID-19 outbreak in China: Application of open educational practices and resources. Smart Learn. Environ. 2020, 7, 1–15. [Google Scholar] [CrossRef]
  2. Almaiah, M.A.; Al-Khasawneh, A.; Althunibat, A. Exploring the critical challenges and factors influencing the E-learning system usage during COVID-19 pandemic. Educ. Inf. Technol. 2020, 25, 5261–5280. [Google Scholar] [CrossRef] [PubMed]
  3. Brown, M.; McCormack, M.; Reeves, J.; Brook, D.C.; Grajek, S.; Alexander, B.; Bali, M.; Bulger, S.; Dark, S.; Engelbert, N.; et al. 2020 Educause Horizon Report Teaching and Learning Edition; EDUCAUSE: Louisville, CO, USA, 2020. [Google Scholar]
  4. Çetin, H.; Türkan, A. The Effect of Augmented Reality Based Applications on Achievement and Attitude towards Science Course in Distance Education Process. Educ. Inf. Technol. 2022, 27, 1397–1415. [Google Scholar] [CrossRef]
  5. Low, D.Y.S.; Poh, P.E.; Tang, S.Y. Assessing the impact of augmented reality application on students’ learning motivation in chemical engineering. Educ. Chem. Eng. 2022, 39, 31–43. [Google Scholar] [CrossRef]
  6. Martins, N.C.; Marques, B.; Alves, J.; Araújo, T.; Dias, P.; Santos, B.S. Augmented reality situated visualization in decision-making. Multimed. Tools Appl. 2022, 81, 14749–14772. [Google Scholar] [CrossRef]
  7. Dede, C. Immersive Interfaces for Engagement and Learning. Science 2009, 323, 66–69. [Google Scholar] [CrossRef] [Green Version]
  8. Lampropoulos, G.; Keramopoulos, E.; Diamantaras, K.; Evangelidis, G. Augmented Reality and Virtual Reality in Education: Public Perspectives, Sentiments, Attitudes, and Discourses. Educ. Sci. 2022, 12, 798. [Google Scholar] [CrossRef]
  9. Marques, B.; Silva, S.S.; Alves, J.; Araujo, T.; Dias, P.M.; Santos, B.S. A Conceptual Model and Taxonomy for Collaborative Augmented Reality. IEEE Trans. Vis. Comput. Graph. 2021, 28, 5113–5133. [Google Scholar] [CrossRef]
  10. Ens, B.; Lanir, J.; Tang, A.; Bateman, S.; Lee, G.; Piumsomboon, T.; Billinghurst, M. Revisiting collaboration through mixed reality: The evolution of groupware. Int. J. Hum. Comput. Stud. 2019, 131, 81–98. [Google Scholar] [CrossRef]
  11. Iqbal, M.Z.; Mangina, E.; Campbell, A.G. Current Challenges and Future Research Directions in Augmented Reality for Education. Multimodal Technol. Interact. 2022, 6, 75. [Google Scholar] [CrossRef]
  12. Sadeghi, M. A Shift from Classroom to Distance Learning: Advantages and Limitations. Int. J. Res. Engl. Educ. 2019, 4, 80–88. [Google Scholar] [CrossRef] [Green Version]
  13. Baxter, G.; Hainey, T. Remote learning in the context of COVID-19: Reviewing the effectiveness of synchronous online delivery. J. Res. Innov. Teach. Learn. 2022. ahead-of-print. [Google Scholar] [CrossRef]
  14. Baber, H. Determinants of Students’ Perceived Learning Outcome and Satisfaction in Online Learning during the Pandemic of COVID-19. J. Educ. e-Learn. Res. 2020, 7, 285–292. [Google Scholar] [CrossRef]
  15. Moore, J. Effects of Online Interaction and Instructor Presence on Students’ Satisfaction and Success with Online Undergraduate Public Relations Courses. J. Mass Commun. Educ. 2014, 69, 271–288. [Google Scholar] [CrossRef]
  16. Yasynska, E. Modern Look at the Advantages and Disadvantages of Distance Learning. Sci. Herit. 2020, 49, 29–30. [Google Scholar]
  17. Manca, S.; Delfino, M. Adapting educational practices in emergency remote education: Continuity and change from a student perspective. Br. J. Educ. Technol. 2021, 52, 1394–1413. [Google Scholar] [CrossRef]
  18. Aydemir, M.; Özkeskin, E.E.; Akkurt, A.A. A Theoretical Framework on Open and Distance Learning. Procedia Soc. Behav. Sci. 2015, 174, 1750–1757. [Google Scholar] [CrossRef] [Green Version]
  19. Burdina, G.M.; Krapotkina, I.E.; Nasyrova, L.G. Distance Learning in Elementary School Classrooms: An Emerging Framework for Contemporary Practice. Int. J. Instr. 2019, 12, 1–16. [Google Scholar] [CrossRef]
  20. Csikszentmihalyi, M. Flow and Education. NAMTA J. 1997, 22, 2–35. [Google Scholar]
  21. Keller, J.M. Motivation and instructional design: A theoretical perspective. J. Instr. Dev. 1979, 2, 26–34. [Google Scholar] [CrossRef]
  22. Pintrich, P.R. A Motivational Science Perspective on the Role of Student Motivation in Learning and Teaching Contexts. J. Educ. Psychol. 2003, 95, 667–686. [Google Scholar] [CrossRef] [Green Version]
  23. Deci, E.L.; Ryan, R.M. Intrinsic Motivation and Self-Determination in Human Behaviour; Plenum: New York, NY, USA, 1985. [Google Scholar]
  24. Deci, E.L.; Ryan, R.M. Self-determination theory: A macrotheory of human motivation, development, and health. Can. Psychol. Can. 2008, 49, 182–185. [Google Scholar] [CrossRef] [Green Version]
  25. Keller, J.M. Motivational Design for Learning and Performance: The ARCS Model Approach; Springer Science & Business Media: Berlin/Heidelberg, Germany, 2009. [Google Scholar]
  26. Khan, T.; Johnston, K.; Ophoff, J. The Impact of an Augmented Reality Application on Learning Motivation of Students. Adv. Human-Computer Interact. 2019, 2019, 7208494. [Google Scholar] [CrossRef] [Green Version]
  27. Wei, X.; Weng, D.; Liu, Y.; Wang, Y. Teaching based on augmented reality for a technical creative design course. Comput. Educ. 2015, 81, 221–234. [Google Scholar] [CrossRef]
  28. Azuma, R.T. A Survey of Augmented Reality. Presence Teleoperators Virtual Environ. 1997, 6, 355–385. [Google Scholar] [CrossRef]
  29. Johnson, L.; Levine, A.; Smith, R.; Stone, S. The 2010 Horizon Report; New Media Consortium: Austin, TX, USA, 2010. [Google Scholar]
  30. Pellas, N.; Fotaris, P.; Kazanidis, I.; Wells, D. Augmenting the learning experience in primary and secondary school education: A systematic review of recent trends in augmented reality game-based learning. Virtual Real. 2019, 23, 329–346. [Google Scholar] [CrossRef]
  31. Wu, H.-K.; Lee, S.W.-Y.; Chang, H.-Y.; Liang, J.-C.; Lai, A.-F.; Chen, C.-H.; Lee, G.-Y.; Hsu, Y.-S.; Lin, Y.-H.; Yang, B. Current Status, Opportunities and Challenges of Augmented Reality in Education. Comput. Educ. 2013, 62, 41–49. [Google Scholar] [CrossRef]
  32. Cabero-Almenara, J.; Roig-Vila, R. The Motivation of Technological Scenarios in Augmented Reality (AR): Results of Different Experiments. Appl. Sci. 2019, 9, 2907. [Google Scholar] [CrossRef] [Green Version]
  33. Marinou, E.-A.; Tselios, C.; Theocharakis, P. On the Relief of Phantom Limp Pain Using Augmented Reality and Edge Computing. In Proceedings of the 2020 IEEE 25th International Workshop on Computer Aided Modeling and Design of Communication Links and Networks (CAMAD), Pisa, Italy, 14–16 September 2020; pp. 1–6. [Google Scholar]
  34. Erbas, C.; Demirer, V. The effects of augmented reality on students’ academic achievement and motivation in a biology course. J. Comput. Assist. Learn. 2019, 35, 450–458. [Google Scholar] [CrossRef]
  35. Georgiou, Y.; Kyza, E.A. Relations between student motivation, immersion and learning outcomes in location-based augmented reality settings. Comput. Hum. Behav. 2018, 89, 173–181. [Google Scholar] [CrossRef]
  36. Fidan, M.; Tuncel, M. Integrating augmented reality into problem-based learning: The effects on learning achievement and attitude in physics education. Comput. Educ. 2019, 142, 103635. [Google Scholar] [CrossRef]
  37. Acosta, J.L.B.; Navarro, S.M.B.; Gesa, R.F.; Kinshuk, K. Framework for designing motivational augmented reality applications in vocational education and training. Australas. J. Educ. Technol. 2019, 35. [Google Scholar] [CrossRef]
  38. Chen, S.-Y.; Liu, S.-Y. Using augmented reality to experiment with elements in a chemistry course. Comput. Hum. Behav. 2020, 111, 106418. [Google Scholar] [CrossRef]
  39. Chen, Y.-C. Effect of Mobile Augmented Reality on Learning Performance, Motivation, and Math Anxiety in a Math Course. J. Educ. Comput. Res. 2019, 57, 1695–1722. [Google Scholar] [CrossRef]
  40. Cakmak, Y.O.; Daniel, B.K.; Hammer, N.; Yilmaz, O.; Irmak, E.C.; Khwaounjoo, P. The Human Muscular Arm Avatar as an Interactive Visualization Tool in Learning Anatomy: Medical Students’ Perspectives. IEEE Trans. Learn. Technol. 2020, 13, 593–603. [Google Scholar] [CrossRef]
  41. Cen, L.; Ruta, D.; Al Qassem, L.M.M.S.; Ng, J. Augmented Immersive Reality (AIR) for Improved Learning Performance: A Quantitative Evaluation. IEEE Trans. Learn. Technol. 2019, 13, 283–296. [Google Scholar] [CrossRef]
  42. Cai, S.; Chiang, F.-K.; Sun, Y.; Lin, C.; Lee, J.J. Applications of augmented reality-based natural interactive learning in magnetic field instruction. Interact. Learn. Environ. 2017, 25, 778–791. [Google Scholar] [CrossRef]
  43. Cuendet, S.; Bonnard, Q.; Do-Lenh, S.; Dillenbourg, P. Designing augmented reality for the classroom. Comput. Educ. 2013, 68, 557–569. [Google Scholar] [CrossRef] [Green Version]
  44. Martín-Gutiérrez, J.; Saorín, J.L.; Contero, M.; Alcañiz, M.; Pérez-López, D.C.; Ortega, M. Design, and validation of an augmented book for spatial abilities development in engineering students. Comput. Graph. 2010, 34, 77–91. [Google Scholar] [CrossRef]
  45. Rossano, V.; Lanzilotti, R.; Cazzolla, A.; Roselli, T. Augmented Reality to Support Geometry Learning. IEEE Access 2020, 8, 107772–107780. [Google Scholar] [CrossRef]
  46. Kamarainen, A.M.; Metcalf, S.; Grotzer, T.; Browne, A.; Mazzuca, D.; Tutwiler, M.S.; Dede, C. EcoMOBILE: Integrating augmented reality and probeware with environmental education field trips. Comput. Educ. 2013, 68, 545–556. [Google Scholar] [CrossRef] [Green Version]
  47. Buchner, J.; Zumbach, J. Promoting Intrinsic Motivation with a Mobile Augmented Reality Learning Environment. In Proceedings of the 14th International Conference Mobile Learning, Lisbon, Portugal, 14–16 April 2018. [Google Scholar]
  48. Kirikkaya, E.B.; Başgül, M. The Effect of the Use of Augmented Reality Applications on the Academic Success and Motivation of 7th Grade Students. J. Balt. Sci. Educ. 2019, 18, 362–378. [Google Scholar] [CrossRef]
  49. Sahin, D.; Yilmaz, R.M. The effect of Augmented Reality Technology on middle school students’ achievements and attitudes towards science education. Comput. Educ. 2020, 144, 103710. [Google Scholar] [CrossRef]
  50. Garzón, J.; Pavón, J.; Baldiris, S. Systematic review and meta-analysis of augmented reality in educational settings. Virtual Real. 2019, 23, 447–459. [Google Scholar] [CrossRef]
  51. Maas, M.J.; Hughes, J.M. Virtual, augmented, and mixed reality in K–12 education: A review of the literature. Technol. Pedagog. Educ. 2020, 29, 231–249. [Google Scholar] [CrossRef]
  52. Scavarelli, A.; Arya, A.; Teather, R.J. Virtual reality, and augmented reality in social learning spaces: A literature review. Virtual Real. 2021, 25, 257–277. [Google Scholar] [CrossRef]
  53. Allen, M. Leaving ADDIE for SAM: An Agile Model for Developing the Best Learning Experiences; Association for Talent Development: Alexandria, VA, USA, 2012; ISBN 9781562867119. [Google Scholar]
  54. Jung, H.; Kim, Y.; Lee, H.; Shin, Y. Advanced Instructional Design for Successive E-Learning: Based on the Successive Approximation Model (SAM). Int. J. E Learn. 2019, 18, 187327. [Google Scholar]
  55. Vuforia. Available online: https://developer.vuforia.com/ (accessed on 15 June 2021).
  56. Unity. Available online: https://unity.com/es (accessed on 15 June 2021).
  57. Keller, J.M. Strategies for stimulating the motivation to learn. Perform. Instr. 1987, 26, 1–7. [Google Scholar] [CrossRef]
  58. Maqableh, M.; Alia, M. Evaluation online learning of undergraduate students under lockdown amidst COVID-19 Pandemic: The online learning experience and students’ satisfaction. Child. Youth Serv. Rev. 2021, 128, 106160. [Google Scholar] [CrossRef]
  59. Kornpitack, P.; Sawmong, S. Empirical analysis of factors influencing student satisfaction with online learning systems during the COVID-19 pandemic in Thailand. Heliyon 2022, 8, e09183. [Google Scholar] [CrossRef]
  60. Eom, S.B.; Ashill, N. The Determinants of Students’ Perceived Learning Outcomes and Satisfaction in University Online Education: An Update. Decis. Sci. J. Innov. Educ. 2016, 14, 185–215. [Google Scholar] [CrossRef]
  61. Conrad, C.; Deng, Q.; Caron, I.; Shkurska, O.; Skerrett, P.; Sundararajan, B. How student perceptions about online learning difficulty influenced their satisfaction during Canada’s COVID-19 response. Br. J. Educ. Technol. 2022, 53, 534–557. [Google Scholar] [CrossRef] [PubMed]
  62. Turk, M.; Heddy, B.C.; Danielson, R.W. Teaching and social presences supporting basic needs satisfaction in online learning environments: How can presences and basic needs happily meet online? Comput. Educ. 2022, 180, 104432. [Google Scholar] [CrossRef]
  63. Esra, M.E.Ş.E.; Sevilen, Ç. Factors Influencing EFL Students’ Motivation in Online Learning: A Qualitative Case Study. J. Educ. Technol. Online Learn. 2021, 4, 11–22. [Google Scholar]
  64. Mukhtar, K.; Javed, K.; Arooj, M.; Sethi, A. Advantages, Limitations and Recommendations for online learning during COVID-19 pandemic era. Pak. J. Med. Sci. 2020, 36, S27–S31. [Google Scholar] [CrossRef] [PubMed]
  65. Lemay, D.J.; Bazelais, P.; Doleck, T. Transition to online learning during the COVID-19 pandemic. Comput. Hum. Behav. Rep. 2021, 4, 100130. [Google Scholar] [CrossRef]
  66. Lin, C.-H.; Wu, W.-H.; Lee, T.-N. Using an Online Learning Platform to Show Students’ Achievements and Attention in the Video Lecture and Online Practice Learning Environments. Educ. Technol. Soc. 2022, 25, 155–165. [Google Scholar]
  67. Chen, C.-M.; Wang, J.-Y.; Yu, C.-M. Assessing the attention levels of students by using a novel attention aware system based on brainwave signals. Br. J. Educ. Technol. 2017, 48, 348–369. [Google Scholar] [CrossRef]
  68. Kirtman, L. Online versus In-Class Courses: An Examination of Differences in Learning Outcomes. Issues Teach. Educ. 2009, 18, 103–116. [Google Scholar]
  69. Means, B.; Toyama, Y.; Murphy, R.; Baki, M. The Effectiveness of Online and Blended Learning: A Meta-Analysis of the Empirical Literature. Teach. Coll. Rec. 2013, 115, 1–47. [Google Scholar] [CrossRef]
  70. Bernard, R.M.; Abrami, P.C.; Lou, Y.; Borokhovski, E.; Wade, A.; Wozney, L.; Wallet, P.A.; Fiset, M.; Huang, B. How Does Distance Education Compare with Classroom Instruction? A Meta-Analysis of the Empirical Literature. Rev. Educ. Res. 2004, 74, 379–439. [Google Scholar] [CrossRef] [Green Version]
  71. Khlaif, Z.N.; Salha, S.; Kouraichi, B. Emergency remote learning during COVID-19 crisis: Students’ engagement. Educ. Inf. Technol. 2021, 26, 7033–7055. [Google Scholar] [CrossRef] [PubMed]
Mti 07 00023 g001 550
Figure 1. Student using the ReAQ learning application.
Figure 1. Student using the ReAQ learning application.
Mti 07 00023 g001
Table
Table 1. Learning motivation among students in the control and experimental arms of the study.
Table 1. Learning motivation among students in the control and experimental arms of the study.
VariableGroupnMeanSDt
MotivationControl group244.070.389−0.823
Experimental group254.170.435
Table
Table 2. t-test results of the four factors of learning motivation for the two groups.
Table 2. t-test results of the four factors of learning motivation for the two groups.
VariableGroupnMeanSDt
AttentionControl group244.090.53−2.138 *
Experimental group254.370.37
RelevanceControl group244.170.481.462
Experimental group253.960.51
ConfidenceControl group243.870.63−0.344
Experimental group253.930.59
SatisfactionControl group244.080.44−2.286 *
Experimental group254.460.40
* p < 0.05.
Table
Table 3. The interaction between the two groups of students was tested using a t-test.
Table 3. The interaction between the two groups of students was tested using a t-test.
MeasureGroupMeanSDt
Number of ErrorsControl group5.712.1162.676 **
Experimental group4.281.595
Help requestsControl group5.291.9224.534 ***
Experimental group3.041. 541
TimeControl group14.751.2193.452 ***
Experimental group13.581.145
*** p < 0.01. ** p < 0.03.
Table
Table 4. Correlation results between motivational factors and students’ interaction with the AR-based learning environment in the experimental group.
Table 4. Correlation results between motivational factors and students’ interaction with the AR-based learning environment in the experimental group.
MeasureMotivation Factorrdfp-Value
ErrorAttention−0.621250.001 ***
Relevance−0.148250.481
Confidence−0.192250.357
Satisfaction −0.521250.008 ***
HelpsAttention−0.237250.253
Relevance−0.015250.943
Confidence−0.173250.407
Satisfaction 0.213250.308
TimeAttention0.510250.009 ***
Relevance0.448250.025 *
Confidence0.458250.021 *
Satisfaction 0.612250.001 ***
*** p < 0.01. * p < 0.05.
Table
Table 5. Correlation results between motivational factors and students’ interaction with the AR-based learning environment in the control group.
Table 5. Correlation results between motivational factors and students’ interaction with the AR-based learning environment in the control group.
MeasureMotivation Factorrdfp-Value
ErrorAttention−0.261240.218
Relevance−0.381240.066
Confidence−0.132240.540 *
Satisfaction −0.436240.033 *
HelpsAttention−0.255240.230
Relevance0.004240.984
Confidence0.014240.947
Satisfaction −0.275240.193
TimeAttention0.044240.839
Relevance0.093240.665
Confidence−0.014240.948
Satisfaction 0.254240.230
* p < 0.05.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Uriarte-Portillo, A.; Ibáñez, M.B.; Zatarain-Cabada, R.; Barrón-Estrada, M.L. Comparison of Using an Augmented Reality Learning Tool at Home and in a Classroom Regarding Motivation and Learning Outcomes. Multimodal Technol. Interact. 2023, 7, 23. https://doi.org/10.3390/mti7030023

AMA Style

Uriarte-Portillo A, Ibáñez MB, Zatarain-Cabada R, Barrón-Estrada ML. Comparison of Using an Augmented Reality Learning Tool at Home and in a Classroom Regarding Motivation and Learning Outcomes. Multimodal Technologies and Interaction. 2023; 7(3):23. https://doi.org/10.3390/mti7030023

Chicago/Turabian Style

Uriarte-Portillo, Aldo, María Blanca Ibáñez, Ramón Zatarain-Cabada, and María Lucía Barrón-Estrada. 2023. "Comparison of Using an Augmented Reality Learning Tool at Home and in a Classroom Regarding Motivation and Learning Outcomes" Multimodal Technologies and Interaction 7, no. 3: 23. https://doi.org/10.3390/mti7030023

Article Metrics

Back to TopTop