Supporting Students’ Visualization of Multivariable Calculus Partial Derivatives via Virtual Reality

Abstract

Multivariable calculus is a subject undertaken by engineering students as a core module at the freshman level. One of the intended learning outcomes (ILOs) in multivariable calculus is to gain an intuition for visualizing three-dimensional surfaces and deducing their properties. For students to visualize more complex multivariable calculus concepts, a virtual reality (VR) application has been created. Tapping on existing infrastructures, we investigate the effectiveness of visualization through VR usage vis-à-vis a two-dimensional digital screen. We have conducted a controlled trial on a group of $N=119$ students across two groups. The first group (control group) comprises students who participated in an online quiz (as a baseline test). The second group (treatment group) is given two sets of tests, the first is the same baseline test that the control group participated in, before administering the test questions on the VR platform (termed the treatment test) to the same group of students. Our analysis reveals that students, in general, perform better on questions pertaining to the identification of the sign of partial derivatives in the treatment test, but for other intended learning outcomes linked to other questions, students have performance similar to the baseline test. Furthermore, low-progress students in the treatment group exhibited improvement after the treatment. Our work here has the potential to be developed into a future-ready smart classroom through VR usage.

1. Introduction

In higher education, multivariable calculus is a foundational course taken by mathematics, physical sciences, and engineering students. For single-variable functions, it is common to visualize them using graphs. Consequently, geometric features of the graph, and by extension, analytic properties of the function can be studied simultaneously. However, many students have difficulty visualizing objects in three dimensions. This forms a steep learning curve that may hinder their conceptual understanding of this foundational subject [1,2].

Proper scaffolding is necessarily required to help students make this cognitive leap from single-variable to multivariable functions [3,4]. Modern multimedia tools [5,6,7] have facilitated the learning process by taking steps to substitute illustrations, such as those that are used on lecture slides or in textbooks, with animated figures. It is generally agreed that animated or interactive computer software plots allowing for a three-dimensional object to be viewed from user-controlled angles and pitches, are better tools for visualization in calculus. VR technology advances this by allowing information to be in the form of text, images, sound, or objects placed in virtual environments so that students can participate in a variety of situational simulations or visual activities while understanding the main concepts from the lesson [8]. Consequently, with smart devices becoming increasingly ubiquitous, coupled with the improvements to computing power in modern computers, virtual and augmented reality have been tested as potential technology-enhanced learning tools to meet the same purpose or even redefine the possibility of teaching and learning multivariable calculus [9,10,11].

However, both virtual and augmented reality have their own strengths and weaknesses, and the choice between them will depend on the specific learning objectives, the available resources, and the preferences of the teacher and students. Orozco et al. created an augmented reality (AR) application that gave the classroom teacher control over the input that students were receiving on their own learning devices [12]. Even though the students were unable to change the graphs, the teacher’s guided inputs allowed for the visualization to be tailored to the topic of the class. There are also studies utilizing the AVRAM (Remote Virtual Environments for the Learning of Mathematics) tool for augmented reality visualization of graphs [13,14]. However, few studies [15,16] (especially those involving empirical investigations) have been dedicated to the use of VR for immersive visualization, and this remains a critical gap in the literature.

With the increased call for blended teaching and learning methods, the objective of this study is to evaluate the effectiveness of VR as a medium to visualize the partial derivatives of two-variable functions. We also wish to investigate what learning objectives are suited for the virtual environment. The introduction of VR in higher vocational education has facilitated access to learning materials while improving the interaction between the instructor, materials, and students. Flipped classroom amalgamates all these aspects of pedagogy, and has shown some benefits to teaching and learning [17]. Thus, the main purpose of using VR technology in this work is to investigate the possibilities of supplementing or replacing parts of the standard classroom environment with some aspects of a flipped classroom. We designed a virtual environment where students can interact with three-dimensional virtual objects to explore the geometrical meaning of partial derivatives. By comparing the performance of students using the VR platform to students who use traditional self-study materials on a two-dimensional screen, it is possible to make some conclusions on whether the VR environment can match or exceed the learning that takes place in a traditional lecture or classroom setting without VR. Tapping on the existing infrastructure [15], we ask the following research questions through this iterative study:

• Is there a benefit of employing VR as a medium to visualize the partial derivatives of two-variable functions?
• What learning objectives, for partial differentiation, are suited for the virtual environment?

Classes for these foundational subjects are commonly conducted in cohort classrooms. Each cohort lesson is conducted by three instructors—two faculty members supported by one upper-year (or a postgraduate) student serving as a teaching assistant. A typical lesson will start with a high-level overview of key concepts based on the flipped-classroom format followed by group discussions of problem sets and questions. Every student attended a weekly one-hour pre-recorded lecture video, followed by a two-and-a-half-hour cohort class in seminar style [18]. An essential component of our experimental study is letting students learn independently without teacher supervision throughout the experiment in order to mimic some aspects of a flipped classroom. In this study, all students ($N=119$) underwent a self-study material before attempting an online baseline quiz. Due to the nature of the VR study being conducted outside classroom hours during the COVID-19 pandemic, we have a treatment group of $N_{TR}=34$. These students are then exposed to a similar quiz on the VR medium after the online quiz. The remaining students who formed the control group ($N_{CO}=85$) only participated in the first online quiz. We will term the quiz conducted on the online platform as the baseline test, and the quiz conducted on the VR platform as the treatment test.

2. Experimental Setup

2.1. VR Application

As this VR platform was previously used for a similar purpose [15], an initial technical feasibility check was performed to match its purpose with our current study. Unreal Engine 4 (UE4), an open-source three-dimensional gaming engine, was used to create the VR application. The Oculus Rift, a consumer VR headset with motion controllers and six degrees of freedom tracking, was our target platform. The final design only used one controller because it was adequate for the controls of modifying the three-dimensional graph and interacting with the menu. This has greatly simplified the use of the controllers for these students who are mostly using the Oculus Rift for the first time. To avoid confusion at this early stage, we have decided against the use of the capacitive touch sensors. We also decided against including some of the more intricate interactions because they could add much complexity to the whole session.

An iterative design cycle was then employed to establish the framework for this VR platform. There was an additional briefing at the beginning of the VR session that led the user through the various ways they can interact with the three-dimensional surface in order to acquaint users of the VR platform with the user interface and tactile controls. Data were then captured from students on the VR application.

The experiment was conducted on $N=119$ volunteer students from the 2021 and 2022 Term 2 cohorts. Students participated in this study in Week 3 of the course after going through three 1-hour lectures, six 2.5 h sections, followed by a self-study session and the baseline quiz, where no time limits were imposed on the students. The volunteers were informed that their participation could be used as part of a research study (consent was obtained online before the students could access the questions). There were two groups: the control group ($N_{CO}=85$) and the treatment group ($N_{TR}=34$). Participants from the treatment group were required to attend an additional in-person VR session after completing the online baseline quiz, whereas participants who did not attend the in-person VR session were assigned to the control group by default. Due to the national and institutional restrictions put in place on any in-person teaching and research activities during the COVID-19 pandemic (2021–2022), during which the data for the current study were collected, we could not recruit as many participants for the treatment group ($N_{TR}=34$) as the control group ($N_{CO}=85$). A baseline test was conducted for all the students, where similarly, no time limits were imposed. The baseline test was in the form of an online quiz consisting of three questions segmented into ten multiple-choice sub-questions. The questions on the baseline test were from the same learning objectives as the pre-test self-study materials. Then, the students from the treatment group participated in the treatment test. The students were given ample time to familiarize themselves with the Oculus headset and controls before participating in the same quiz, this time on the VR platform with a familiarization tutorial. Both tests were completed in the order mentioned within the same week without any instructor intervention in supplementing any academic assistance on the topics during the duration of the experiment, which is in stark contrast to an earlier study [15]. The scores for both tests were recorded and analyzed to answer the research questions.

2.2. Pre-Test Self Study

The pre-test self-study material consists of a set of slides, with illustrations taken from the course textbook [19]. The students were expected to be familiar with certain contents and examples from the textbook, which were reproduced in their weekly materials (with acknowledgement) for self-reading purpose. The intended learning outcomes (ILOs) from this self-study material are:

• Identify the sign of the partial derivatives for a function of two variables;
• Identify whether the partial derivative is increasing, decreasing, or unchanging for a function of two variables;
• Interpret the changes to the partial derivative by studying a contour plot.

2.3. Baseline Test

Students in both the control and treatment groups participated in the baseline test conducted on Google Forms. The baseline test comprises three questions, separated into ten multiple-choice sub-questions. The test items targeted the learning objectives that were focused in the pre-test self-study material. The first two questions are basic questions that can reveal students’ learning based on the first ILOs. The first question also seeks to evaluate students’ performance on the second ILOs. Finally, question 3 directly refers to students’ learning of the third ILOs.

2.4. VR Familiarization Self Study

The VR laboratory was used for the students from the treatment group. In addition to allowing students to visualize and manipulate the surfaces and contour maps in three dimensions on the VR platform, the VR application also allowed students to learn at their own pace and with the same concepts as the pre-test self-study material. On average, the students spent about 1.5 h in the VR laboratory.

The VR laboratory is a physical space on campus that has twenty Oculus sets for use (see Figure 1 and Figure 2). Due to the ongoing COVID-19 pandemic, it is mandatory to wear a mask in the VR laboratory with safe distancing in place, which has limited certain level of participation. Students were grouped into various shifts outside of class contact hours to participate in this VR experience. The students were guided by the laboratory technician to set up the VR application with instructions on the usage, as this was the students’ first time doing so. Following this, the students did not receive any help pertaining to the materials. Students were allowed to seek technical assistance as and when required, and most were able to go about completing the tasks at their own pace.

Students could gesture to any specific point on the three-dimensional surfaces, and the corresponding slope of both the $f_x$ and $f_y$ partial derivatives would be displayed. The students could also use the triggers on the hand-held controls to activate the zoom functions to enlarge or minimize the surfaces as well as rotate them (see Figure 3 and Figure 4). The perceived benefit of these capabilities is that students were now able to view the partial derivative slopes at the selected point, and view the projection of the slope onto a contour graph with simple gestures. The students had the opportunity to start the quiz in the VR environment whenever they deemed that they were familiar enough with the controls and the concepts from the self-study material.

2.5. Treatment Test

In the quiz, the students had to complete the same ten questions as used in the baseline test. The quiz format was mostly the same, although surfaces were projected as three-dimensional surfaces in the VR environment. The option to view the partial derivative slope was disabled during the quiz. Students could move between questions and revisit questions in any order that they prefer, similar to the baseline test. A comparison between the delivery of a question in the baseline test and treatment test is shown in Figure 5.

3. Results and Discussion

We worked with the assumption that the students assigned to the control and treatment groups did not collaborate, thus not affecting each other in their attempt of either test. The answers for both the control and treatment groups, by question, are recorded in

Table 1. Recorded answers for the both the control and treatment groups. Correct answers are in bold. See Supplementary Information for labelling convention of multiple choices.

	Control Group (CO)			Treatment Group (TR)
	Baseline Test (BT)			Baseline Test (BT)			Treatment Test (TT)
Qn	−1	0	1	−1	0	1	−1	0	1
1ai	24	1	60	3	2	29	5	3	26
1aii	59	3	23	27	2	5	29	1	4
1bi	13	6	66	5	5	24	3	7	24
1bii	6	70	9	4	27	3	0	31	3
2a	61	9	15	18	7	9	28	0	6
2b	4	11	70	5	8	21	2	0	32
3a	3	0	82	3	0	31	1	0	33
3b	3	1	81	3	0	31	0	0	34
3c	1	0	84	4	1	29	2	0	32
3d	78	0	7	30	1	3	32	1	1

, with score distribution by group and test in Figure 6. We set the respective recorded mean scores according to the format shown in Figure 7. The weighted mean scores are recorded in

Table 2. Recorded weighted mean scores and standard deviation by question and total score (rounded to the nearest 2-decimal place).

	${\mathit{\mu }}_{\mathit{CO}-\mathit{BT}}$ (${\mathit{\sigma }}_{\mathit{CO}-\mathit{BT}}$)	${\mathit{\mu }}_{\mathit{TR}-\mathit{BT}}$ (${\mathit{\sigma }}_{\mathit{TR}-\mathit{BT}}$)	${\mathit{\mu }}_{\mathit{TR}-\mathit{TT}}$ (${\mathit{\sigma }}_{\mathit{TR}-\mathit{TT}}$)
Q1	0.75 (0.30)	0.79 (0.26)	0.81 (0.30)
Q2	0.77 (0.37)	0.57 (0.46)	0.88 (0.25)
Q3	0.96 (0.14)	0.89 (0.21)	0.96 (0.09)
Total	0.83 (0.16)	0.79 (0.22)	0.89 (0.17)

. We performed multiple t-tests to answer the research questions. All t-tests performed are two-tailed, unless otherwise specified.

3.1. Test-Based Analysis

We first performed a two-sample t-test, assuming unequal variance, to investigate whether the mean scores for the baseline test the control and treatment groups participated in, as well as the treatment test the treatment group participated in, were equal. This will allow us to answer the first research question. $H_0$: There is no statistical difference between the sample means of the baseline test and the treatment test. That is, III.A.1: ${\mu }_{BT}-{\mu }_{TT}=0$.

Upon performing the two-sample t-test (see

Table 3. Summary of two-tail t-test performed for baseline and treatment tests. Hypothesis III.A.4 is a paired t-test.

${\mathit{H}}_0$	p-Value	95% CI	Conclusion
III.A.1	6.35 × ${10}^{-2}$	$(-0.131,0.004)$	Fail to reject
III.A.2	1.05 × ${10}^{-2}$	$(0.043,0.296)$	Reject null
III.A.3	$0.632$	$(-0.063,0.103)$	Fail to reject
III.A.4	2.58 × ${10}^{-4}$	$(-0.277,-0.102)$	Reject null

), we fail to reject the null hypothesis with a 95% confidence interval. On closer inspection of the experimental data, we notice a stark difference in mean scores for the students in the 2021 cohort. Namely, the students who took the baseline test in the treatment group (average score of 7.1 out of 10) performed significantly worse than the students in the control group (average score of 8.8 out of 10). Using a two-sample t-test, for the null hypothesis III.A.2: μ_21CO-BT − μ_21TR-BT = 0, we are able to conclude, with 98% confidence (Cohen’s $d\approx 0.900$), that prior to the treatment, the students in the treatment group were weaker than the students in the control group. However, after the treatment, the same students from the treatment group performed equally well with an average score of 9.0 out of 10. A two-sample t-test failed to reject the null hypothesis III.A.3: μ_21TR-TT − μ_21CO-BT = 0. Lastly, a paired t-test, with null hypothesis III.A.4: μ_21TR-BT − μ_21TR-TT = 0, to evaluate the effectiveness of VR as intervention for the treatment group also displays a significant improvement due to the treatment (with more than 99.9% confidence, Cohen’s $d\approx 0.942$).

We find that, for low-progress learners who may have difficulty grasping concepts relating to multivariable functions, the use of VR may bring them to the same competency as their peers. In order to rule out the possibility of a practice effect, as part of our future work, the control group should also be exposed to a second reading of the self-study material and take the second test to equate the total study duration.

3.2. Question-Based Analysis

Next, we performed three two-sample t-tests to investigate whether the mean scores for each question in the baseline test and treatment test were equal. This will allow us to answer the second research question. The respective null hypotheses are as follows, and

Table 4. Summary of two-tail t-test performed for each question.

${\mathit{H}}_0$	p-Value	95% CI	Conclusion
III.B.1	$0.403$	$(-0.163,0.066)$	Fail to reject
III.B.2	$3.90$ × ${10}^{-3}$	$(-0.281,-0.055)$	Reject null
III.B.3	$0.229$	$(-0.069,0.016)$	Fail to reject

summarizes the two-tail t-test performed for each question.

• III.B.1: ${\mu }_{BTQ1}-{\mu }_{TTQ1}=0$;
• III.B.2: ${\mu }_{BTQ2}-{\mu }_{TTQ2}=0$;
• III.B.3: ${\mu }_{BTQ3}-{\mu }_{TTQ3}=0$.

Since question 1(a) tests a similar concept to question 2, a two-sample t-test is conducted to examine the statistical significance of the treatment. In analyzing the data for the null hypothesis ${\mu }_{BTQ1a}-{\mu }_{TTQ1a}=0$, we fail to reject the null hypothesis, with a p-value of $0.281$, and $95\%$ CI of $(-0.209,0.062)$. This is significantly different from the outcome to question 2. Consistently across both 2021 and 2022 cohorts, question 2 was poorly answered for the baseline test. However, for the same question in the treatment test, there are no signs that question 2 posed any difficulty to the respondents. This seems to suggest that the VR platform could be useful to bridge misconceptions pertaining to the first ILOs.

Upon further investigation, we noted that in the baseline test, students were presented with $f(x,y)$ in question 2 together with the functions $f_x(x,5)$ and $f_y(1,y)$. In the treatment test, the students were presented with the same function $f(x,y)$, albeit with some level curves. We hypothesize that the difference in information provided in questions 1 and 2 could have been the resulting factor for the disparity in outcome. However, we do not have any strong evidence to advance this claim. This is a potential gap that needs to be bridged, as our study seems to suggest that students tend to perform better when they understand the concept of level curves.

As expected, question 3 shows no statistically significant difference in the results obtained from the baseline and treatment tests. The purpose of the contour lines is to represent the three-dimensional surfaces on a two-dimensional map. When translated into the VR environment, the contours are still presented as a two-dimensional map. Therefore, there should be no difference in interpreting partial derivatives on a conventional digital screen and on the VR platform.

Lastly, we present some insights into relevant future works. For example, multiple questions testing the same concept can be implemented to validate the effectiveness of the treatment. The following additional information would be helpful in evaluating the VR platform and its viability as an intervention tool:

• Students’ perceived level of confidence in their provided inputs both for the baseline and treatment tests;
• Response from students on how they arrive at the answer for each question in both the baseline and treatment tests;
• Students’ perceived importance of how VR helped them arrive at their answers for each question in the treatment tests;
• Keep track of students’ gaze and head/hand movements via video analytics [20,21,22] for further interpretation of the data.

One of the key pillars of teaching and learning is the taxonomy of learning. At present, this is an iterative study that seeks to tackle only basic and singular concepts in multivariable calculus. Hence, each question in the baseline and treatment tests were designed to cover no more than two ILOs. The ILOs are also chosen such that they cover the lower echelons of cognition and learning. In more advanced topics (for example, vector calculus), it will involve a cognitive demand that can challenge most students. In such cases, VR may reveal itself to be of great benefit to learning.

4. Conclusions

In conclusion, our experimental results suggest that while VR learning does not necessarily translate to better attainment of learning outcomes, it can still match current traditional learning methods to a certain extent. It appears that students perform better on questions relating to identifying the sign of the partial derivative, and for other questions, students have performance similar to the baseline test.

Importantly, for low-progress students, the experiment based on our setup has shown that VR can prove to be beneficial to help these students match up to the required knowledge as their peers. Utilizing VR for more advanced topics within multivariable calculus, for example, vector calculus will involve a cognitive demand that can challenge most students. Under such conditions, VR may reveal itself to be of greater benefit to learning. Investigation on the use of VR on these advanced topics motivates future work. It is our hope that the study here can inspire and encourage experimentation of VR for other topics within the multivariable calculus classroom, if resources permit.