3D-Printed Teeth with Multicolored Layers as a Tool for Evaluating Cavity Preparation by Dental Students

Abstract

Accurate assessment of dental student performance during preclinical operative mannequin courses is an essential milestone within the educational process. Training on novel, multicolored 3D-printed teeth resulted in higher performances of the students in comparison to training on standard, monochromatic plastic teeth. However, low reliability of students’ grading using standard, monochromatic plastic teeth was reported. The aim of this study was to verify whether the use of 3D multicolored teeth can (1) provide better inter- and intra-examiner reliability, and (2) assess the effect of instructors’ experience on their reliability. The novel tooth analogs consisted of digitally planned and 3D-printed plastic teeth containing green, yellow, and red stratifications according to increasing depths of preparation. Thirty-seven dental students performed three Class I preparations on the 3D-printed teeth, and these underwent blind evaluation by two examiners of varied experience at two timepoints. The data were compared with preparations done on conventional (monochromatic) plastic teeth. Results indicated excellent inter-examiner reliability on 3D-printed teeth (0.768 < ICC < 0.929), but only moderate reliability with conventional plastic teeth (0.314 < ICC < 0.672). The examiner having more experience was found to show higher intra-examiner reliability (ICC = 0.716 and 0.612 using 3D-printed teeth and conventional teeth, respectively) than the less experienced examiner (ICC = 0.481 and 0.095 using 3D-printed teeth and conventional teeth, respectively). The novel, multicolored 3D-printed teeth can provide more objective evaluation of cavity preparation compared with conventional plastic teeth.

1. Introduction

Dentistry is a medical profession that requires fine motor skills [1], hand–eye coordination [2], and exceptional spatial perception [3]. Furthermore, perceptual learning is needed in order to develop indirect visualization using a mirror [4,5]. These prerequisites have led dental educators to develop simulators in order to assist in the teaching of these skills [6].

The dental mannequin is the traditional simulator used to prepare dental students for treating patients; it is an anatomic construction of a human head and jaws containing plastic duplicates of each tooth arranged in a normal occlusion, on which the novice dental student is taught to perform basic dental procedures. Dental students have been shown to be particularly stressed during their preclinical studies [7]—more so when being taught cavity preparations and endodontic treatment [8]. Evaluating student performance under these conditions is further complicated because of poor examiner reliability due to human subjectivity, compounded by a paucity of objective parameters. Studies have shown that the conventional clinical evaluating systems—“glance and grade”, which provides an overall mark; and the “analytical method”, which utilizes specific criteria and checklists—have low reliability and validity due to high inter- and intra-examiner variability [9,10,11]. Sharaf et al. (2007) [9] concluded that both evaluation methods showed disagreement and high variability between examiners regarding the evaluation of operative dental procedures on plastic teeth. Jenkins et al. (1996) [10] demonstrated that instructors grading preclinical operative procedures using a glance and grade method have a high degree of inter-examiner variability, and highlighted the need for a more comprehensive system of assessing preclinical performance. Salvendy et al. (1973) [12] found not only inter-examiner variability, but also wide intra-examiner variation when the same instructor evaluated the same cavity preparation on a second occasion.

These realizations, coupled with applied digital technologies, have prompted the development of more objective and reliable methods of teaching and evaluating outcomes [13,14,15]. For example, PrepCheck (Dentsply Sirona, Wals, Austria) makes use of CAD/CAM (computer-assisted design and computer-assisted manufacturing) technology by comparing scans of cavity preparations with a standardized “master” preparation [16]. In addition, the Dental Teacher system (KaVo Dental, Biberach, Germany) includes a 3D scanner, PC, and three software modules for preparation validation [15].

Studies of these newer methods of evaluation have shown them to be more effective than traditional methods, primarily in self-assessment and self-directed learning of manual skills [14]. Other studies have deduced that these methods do not improve student comprehension of the skills, because of the unrealistic goal of manually reproducing any preparation exactly according to predefined values [15,17]. Therefore, obtaining an objective perspective for preparation evaluation remains a challenge.

An attempt to attain this goal using a novel, 3D-printed tooth model composed of multicolored layers was recently introduced [18]. This 3D multicolored tooth model, replacing the monocolored standard plastic teeth, was shown to be effective in the self-learning of dental students during the process of acquiring new manual skills [18]. It was concluded that this unique design provides real-time augmented visual feedback during the teaching exercise [18]. Therefore, it was assumed that the color scheme will make students’ preparation evaluation by clinical instructors more objective, which could potentially reduce inter- and intra-examiner variability.

The aims of the present study were (1) to compare students’ clinical assessment of cavity preparations using the multicolored 3D model to the conventional plastic teeth by evaluating inter- and intra-examiner variability, and (2) to evaluate the effect of instructors’ clinical experience on the reliability of their evaluations of clinical performance.

2. Materials and Methods

2.1. Participants

Thirty-seven dental students (20 females, 17 males, mean age 25.7 ± 1.81 years, range 23–29 years) in the 2018 academic year participated in the present study. The dental students were in their fourth (15 students, mean age 25.2 ± 1.94 years), fifth (10 students, mean age 25.7 ± 2.11 years), and sixth (12 students, mean age 26.5 ± 1.16 years) years of dentistry studies on a six-year program. All participants signed informed consent approved by the Ethics Committee of Tel Aviv University.

2.2. Dental Training Models

2.2.1. Conventional Plastic Teeth

(Figure 1): Typically, dental students acquire manual dexterity by practicing on replicas of human teeth formed from plastic material (Nissin Dental Products Inc., Nakagoku, Japan) arranged within a mannequin head (Columbia, Dentalez, Malvern, PA, USA model AH-1-BP), which acts as a simulation of clinical conditions. These plastic teeth are made of a single acrylic resin material and are of a single homogeneous color (ivory), and simulate arch traits (maxilla or mandible) and type traits (anatomically correct crown shapes of each tooth).

2.2.2. 3D-Printed Teeth with Multicolored Layers

(Figure 1): The analog models of regular plastic teeth described above were digitally scanned using an Einscan-SP 3D Scanner (Shining 3D^®, Hangzhou, China), the files of which were uploaded into FUSION 360 CAD/CAM 2017 software 3D-Builder programs (AUTODESK, San Rafael, CA, USA) and MySolidWorks 2017–2018 (SOLIDWORKS, Waltham, MA, USA), intended for planning models for 3D printing. After attaining the Standard Tessellation Language (STL) files, a master model of each tooth—containing multicolored layers analogous to the traffic light array (i.e., green, yellow, and red) as internal layers—was designed for each tooth. The precise outer/enveloping layers, as well as the cavity preparation of each model (both in white color), were planned according to specific wall forms, depths, and marginal forms, owing to the principles of ideal cavity preparation for amalgam restoration.

The design for the tooth model was planned for Class I cavity preparation, according to Black’s classification [19], as if for an amalgam restoration including central fossa and pits and fissures on the occlusal surface; it included a 0.5-mm-thick green enveloping layer underlying the white margin, under which there was a 0.5-mm-thick yellow layer, with the deepest being an inner red layer. The concept of the layered, 3D-color-printed tooth design was to designate the cavity preparation with colored limit zones; as such, preparation of the cavity within the green zone was to be considered good preparation, preparation of the cavity within the yellow zone incomplete but not a failure, and preparation that exposes the red layer—representing the pulp chamber and pulp horns—as a clinical failure. These demarcations do not exist in conventional analog plastic teeth.

The teeth were printed using an OBJET J750 printer (Stratasys Ltd., Eden Prairie, MN, USA) utilizing Polyjet^TM jetting technology, which prints models in multicolored layers with a resolution of 17 microns. The models were printed with Vero photopolymer material, creating custom-made RGB using: Black (OBJ-03286), Cyan VIVID (OBJ-03296), Magenta VIVID (OBJ-03299), Yellow VIVID (OBJ-03302), Pure White (OBJ-03327), and Support 706 (OBJ-03326) (Stratasys Ltd., Eden Prairie, MN, USA). These materials offer the following mechanical properties: modulus of elasticity = 2000–3000 MPa; flexural strength = 75–110 MPa; and tensile strength = 60–70 MPa.

2.3. Design and Method

The study evaluated the preparations of 37 dental students performing Class I amalgam cavity preparation on the mandibular left first molar as part of a complete dentition oriented in a dental mannequin head (Columbia, Dentalez, Malvern, PA, USA model AH-1-BP).

All participants performed three Class I amalgam cavity preparations on multicolored 3D-printed teeth, after undergoing a half-hour didactic lecture regarding the concept of doing so using the unique, 3D-printed, multicolored oriented teeth. Twelve participants were also randomly selected to perform the same task on two conventional plastic teeth. All preparations were performed using a micromotor handpiece (NSK, Tokyo, Japan) and straight fissure bur (330 bur, Strauss, Palm coast, FL, USA) according to the accepted preparation parameters. Participants were seated during the procedure and made use of a unit light source to simulate the actual clinical environment.

All cavity preparations were evaluated blind by two independent examiners using an explorer hand instrument without magnification; they were provided a checklist with specific criteria to result in standardized data for analysis (Appendix A). Both instructors are instructors in the operative course at the phantom laboratory. Examiners 1 and 2 have 7 and 3 years of experience as clinical instructors, respectively, although both were 10 years post-graduation.

Each preparation was initially evaluated using the prescribed checklist by each examiner; this procedure was performed a second time after 3 months under identical conditions. Examiners were blinded as to participant identity at all times.

Performance evaluation was regimented using analytical evaluation methods (checklist and criteria). The criteria examined were: outlines of cavity preparation, proportion of cavity walls, cavity depth, line angles of the cavity, directionality of the gingival floor and axial walls, injury to adjacent tooth, and finish (Appendix A). Grades were on a continuous scale of 0–100, where the passing grade was 60. Measured success of tasks was judged by depth of cavity (0.2 mm inside the dentine–enamel junction, or 1.5 mm as measured from the depth of the central groove), a marginal configuration of 90°, maintaining as much unprepared tooth structure as possible (preserving cusps and the marginal ridge), having pulpal and gingival walls perpendicular to occlusal planes, and maintaining round internal preparation angles.

2.4. Statistics

Assumptions of normality of dependent variables (students’ grades) were assessed using the Kolmogorov–Smirnov test, and via visual inspections of histograms with a normal curve.

In order to assess the intra- and inter-reliability of the two examiners, an intraclass correlation coefficient (ICC) test was used [20,21].

A two-way ANOVA was used to estimate the differences in grades with the independent variables—examiners (Examiner 1, Examiner 2), and type of training model (multicolored 3D-printed teeth and regular plastic teeth)—at the first and second evaluation times.

All analyses were performed using SPSS version 20 (IBM Corp, Armonk, NY, USA). Significant differences were considered as p < 0.05.

3. Results

A Kolmogorov–Smirnov test performed on students’ grades indicated a normal distribution (p > 0.05).

Table 1. Mean, SD, range, and percentiles (P25, P50, and P75) of the preparations on multicolored 3D-printed teeth (N = 111) and preparations on conventional plastic teeth (N = 24) by two instructors, at the first and second evaluation times.

		First Evaluation				Second Evaluation
		Mean (±SD) (Range)	P25	P50	P75	Mean (±SD) (Range)	P25	P50	P75
multicolored 3D-printed teeth (N = 111)	Examiner 1	51.6 (±23.6) (12.5–97)	30	55	72.5	44.8 (±22.4) (8.75–98)	25	41.2	63.7
	Examiner 2	47.5 (±24.4) (5–93.7)	26.2	46.2	67.5	32.7 (±18.2) (8.75–75)	18.7	27.5	45
Conventional plastic teeth (N = 24)	Examiner 1	49.7 (±28.7) (4–95)	23.7	51.8	71.8	45.4 (±21.9) (20–93)	30	36.8	65.2
	Examiner 2	31.6 (±15.9) (8–73)	20	30	35	27 (±16.2) (6–68)	14	26.2	32.1

shows the mean, standard deviation (SD), range, and percentiles (P25, P50, and P75) of the cavity preparations’ grades on multicolored 3D-printed teeth and on conventional plastic teeth by the two examiners, at the first and second evaluations.

3.1. Intra-Examiner Reliability

The intraclass correlation coefficient (ICC) is a measure (between 0 and 1) of the reliability of data to be collected as a group; the higher the value, the higher the reliability. A confidence interval (CI) of 95% was chosen to compare the means of the group’s data.

The ICC for the intra-examiner reliability of Examiner 1 was significant and good on multicolored 3D-printed teeth preparations (0.716, p = 0.0005, 95% CI (0.61, 0.79)) and moderate on conventional plastic teeth preparations (0.612, p = 0.001, 95% CI (0.28, 0.89)). For examiner 2 it was significant and moderate on multicolored 3D-printed teeth preparations (0.481, p = 0.0005, 95% CI (0.32, 0.61)) and not significant and low on conventional plastic teeth (0.095, p = 0.326, 95%CI (−0.31, 0.47)).

3.2. Inter-Examiner Reliability

The ICC for inter-examiner reliability was significant and excellent between examiners for multicolored 3D-printed teeth preparations at the first (0.929, p = 0.0005, 95% CI (0.89, 0.95)) and second (0.768, p = 0.0005, 95% CI (0.67, 0.83)) evaluation times (Figure 2). For conventional plastic teeth it was not significant and low for the first evaluation (0.314 p = 0.063, 95% CI (−0.94, 0.63)) and significant and moderate for the second evaluation (0.672 p = 0.0005, 95% CI (0.376, 0.843)).

3.3. Effect of Examiners and Type of Training Model

A two-way ANOVA examined the effects of the examiners and the type of training model on students’ grades in each evaluation time.

At the first evaluation, there was no significant interaction between the effects of the examiners and the type of training on the grades (p = 0.066). Simple main effects analysis showed that Examiner 1’s grades were statistically significantly higher than Examiner 2’s grades (p = 0.004), and the grades on multicolored 3D-printed teeth were statistically significantly higher than the grades on conventional plastic teeth (p = 0.019).

At the second evaluation, there was no significant interaction between the effects of the examiners and the type of training on the grades (p = 0.323). Simple main effects analysis showed that Examiner 1’s grades were statistically significantly higher than Examiner 2’s grades (p = 0.0005), but there were no differences between the type of training on grades (p = 0.428).

4. Discussion

The purpose of preclinical dental courses is to simulate as closely as possible the actual circumstances that patient care entails. The use of mannequins as representative of human head anatomy is the classic method of presenting dental students with these parameters. Included in this model are replicas of the jaws and teeth, on which instruction is given by skilled educators as to the steps required to perform appropriate dental preparations [6].

The most common mode of this type of education is to observe the instructor as they perform and explain the needed tasks. Furthermore, said instructors are required to evaluate each student’s endeavors in order to gauge their progress. However, the majority of instructors are solo practitioners, which tends to produce uneven standards in technique and insular scales of evaluation. Previous studies have described the educational shortcomings of evaluating preclinical dental student performance due to inconsistency within and among examiners [9,10,11]. In addition, the use of magnifications (i.e., loupes) for evaluating cavity preparation might further enhance the procedure [22].

The present study is an effort to alleviate the issues of the reliability and accuracy of the evaluation of clinical performance by instructors. The former refers to the consistency of earned marks regardless of who administers them or when [20]. To fulfill this definition, we developed a novel analog model of human teeth with multicolored layers using 3D digital printing. These are designed to follow the margins of an ideal cavity preparation (border of white and green) in all dimensions, with strata of green, yellow, and red beneath an outer ivory surface [19]. The removal of material from these analogs provides both student and instructor with direct visual feedback as to the performance of the given procedure. The ability to discern even minor deviations in this manner is superior to and more objective than the traditional method using monochromatic plastic teeth.

The results of this study show that, by using the novel multicolored 3D-printed teeth, both intra- and inter-examiner reliability was improved. Cicchetti (1994) [23] described commonly cited ranges for the qualitative rating of agreement based on ICC values, i.e., “good” for values between 0.6 and 0.74, and “excellent” for values between 0.75 and 1. In our study, the ICC was used to measure the inter-examiner reliability in both the multicolored 3D-printed and the conventional plastic teeth. The agreement between examiners was found to be excellent for both the first (ICC = 0.929), and second (ICC = 0.768) evaluations on multicolored 3D-printed teeth. When conventional plastic teeth were used to test this, it was found to be low at the first (ICC = 0.314) and moderate at the second (ICC = 0.672) evaluations.

These findings are different from those reported by Lee et al. (2018) [21], who reported that inter-examiner reliability was excellent for both methods of digital assessment by CEREC software and conventional visual assessment of cavity preparation on regular plastic teeth (ICC from 0.77–0.87). However, Zou et al. (2016) [24], who compared a conventional visual assessment with a computerized laser-scanning cavity preparation skill evaluation system (CPSES) of class I cavity preparation by clinical instructors, reported that the CPSES gave consistent inter-examiner reliability and excellent intra-examiner reliability (ICC 1), in contrast to the conventional method (ICC from 0.56 to 0.77).

The use of digital tools in modern dentistry permits the rapid scanning of student cavity preparation and comparison with standardized cavity preparation models. This can generate 3D feedback data using computer-based evaluation systems such as PrepCheck [15], CEREC [21], and CPSES [24]. Previous studies have indicated that this method provides more consistent and accurate grading [25,26]; however, they require significant training before usage, standardization of variation among independent instructors now needing to alter individual variability in teaching habits, increased time needed to obtain digital feedback and high costs. Consequently, digital assessment should not replace conventional methods; rather, it can overlap these as a supplemental method of evaluation.

The aim of developing the novel, multicolored 3D-printed teeth was to replace the conventional monochromatic plastic teeth in order to facilitate a more objective and accurate evaluation method, while still using the traditional assessment method by both students and instructors. Moreover, the conventional assessment method more closely follows actual clinical practice. Hence, the novel multicolored 3D-printed tooth analogs enable assessment in the same manner as with conventional plastic teeth, while providing more objective and accurate evaluation of cavity preparation.

Intra-examiner reliability is determined by comparing the grades given by an examiner of the same object at one point in time with those given at another significantly separated point in time. In the present study, this was done 3 months apart. Furthermore, the examiners who participated in the present study were of different levels of expertise (Examiner 1 was considered to have more expertise), because previous studies have reported that experience improved intra-examiner reliability, and less experienced examiners have been shown to be more inconsistent [20]. Our findings are consistent with those previously reported, as Examiner 1 displayed more intra-examiner reliability (ICC = 0.716 on multicolored teeth and 0.612 on regular teeth) than Examiner 2 (ICC = 0.481 on multicolored teeth and 0.095 on regular teeth).

Our findings suggest that examiner/instructor influence is the main factor correlated to the grades given to dental student cavity preparations. Furthermore, it was found that the novel, multicolored 3D-printed tooth analogs increase the level of consistency of the less experienced examiner. The explanation for the improvement by the less experienced examiner with the 3D model is that it provides an alternative means to assess student performance over the conventional method of assessment. The multicolored 3D-printed teeth provided additional visual feedback on performance success and errors according to the various color layers exposed inside the cavity. These make the dimensions, depth, and location of the tooth preparation more clear for both student and teacher, thus facilitating more objective and accurate evaluation of tooth preparation.

5. Conclusions

• The novel, multicolored 3D-printed tooth analogs can provide a more objective evaluation of cavity preparation compared with conventional plastic teeth. The novel, multicolored 3D-printed teeth could improve the current evaluation method for assessing undergraduate skills by improving examiner consistency.
• Although intra-examiner unreliability persists, the use of this novel, multicolored 3D-printed tooth analog might reduce it, especially with less experienced instructors.