Keep It Brief and Targeted: Driving Performance Feedback Report Features to Use with Novice Drivers
- Numeracy: More numerate participants (those better able to understand numbers and probability ) will find the reports easier to use and easier to understand; they will also have higher motivation to improve than less numerate participants.
- Summary presence: Reports with a summary present will be easier to understand and lead to higher motivation to improve than a summary absent.
- AP length: Reports with a short AP will be easier to use, easier to understand, and lead to higher motivation to improve than a long AP.
- Report length interactions: Reports with a summary absent and a short AP will be the easiest to use, easiest to understand, and lead to highest motivation to improve than any other combination.
- AP order: Reports with AP ordered best-to-worst will be easier to use, easier to understand, and lead to higher motivation to improve than by worst-to-best or by importance.
- AP grading: Reports with the letter-number combination grading will be easier to use, easier to understand, and lead to higher motivation to improve than with numbers only or letters only.
- Peer comparison presence: Reports with peer comparison present will be easier to use, easier to understand, and lead to higher motivation to improve than with peer comparison absent.
2. Materials and Methods
2.2. Driver Performance Feedback Report Designs
2.2.1. Report Design Features
- Driving performance summary: This bulleted list provided an overall summary of driving performance, such as “You managed your speed well” and “You followed vehicles too closely, increasing your chances of rear-ending them”. The presence or absence of the summary was manipulated across conditions; participants with summary present received identical information.
- Action plan (AP) with results: This table of skills feedback included domains of safe driving, their definitions, and a grade with suggestions for improvement and links to training materials. The number of domains shown, order of appearance, and grading system varied. In the long AP, eight domains were shown (speed management, road positioning, gap selection, managing blind spot, hazard anticipation and response, attention maintenance, communication/right of way, and vehicle control), while the short AP had four domains (speed management, gap selection, managing blind spot, and attention maintenance). For the order of appearance, this was by grade best-to-worst, by grade worst-to-best, or by order of domain importance (i.e., gap selection always first). Grading was either by letter grade only, numerical grade only, or a combination of number and letter grade and were the same or equivalent for each domain (e.g., speed management was always A, 96, or 96-A).
- Peer comparison: This sentence appeared next to the overall grade and improvement opportunity and stated, “Of all the drivers who completed the virtual driving test in your peer group, 65% drove the virtual driving test safer than you”. The presence or absence of the peer comparison was manipulated across conditions; participants with peer comparison present received identical information.
2.2.2. Experimental Conditions
2.3.1. Demographics Questions
2.3.2. Feedback Report Questions
- Easy to use: These questions checked the information was clearly displayed and the report was easy enough to use they would recommend for others (e.g., “The information in this report was easy to understand”). The questions were on a 7-point Likert-type scale (i.e., strongly disagree to strongly agree) with median responses used.
- Easy to understand: These questions checked participants could navigate the form and find critical information (e.g., “Which of these driving skills does the report indicate the driver needs the most improvement on?”). The questions were multiple-choice questions transformed into correct/incorrect answers.
- Motivation to improve: These questions checked the report made the participants reflect on their driving skills (e.g., “This report makes me want to be a safer driver”). The questions were on a 7-point Likert-type scale (i.e., strongly disagree to strongly agree) with median responses used.
2.3.3. Numeracy Scale
3.1. Sample Descriptives
3.2. Report Descriptives
3.3. Report Length Analyses
3.4. AP Order
3.5. AP Grading
3.6. Peer Comparison
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
- Brand, P.L.; Jaarsma, A.D.; van der Vleuten, C.P. Driving lesson or driving test? A metaphor to help faculty separate feedback from assessment. Perspect. Med. Educ. 2021, 10, 50–56. [Google Scholar] [CrossRef] [PubMed]
- Beanland, V.; Goode, N.; Salmon, P.M.; Lenné, M.G. Is there a case for driver training? A review of the efficacy of pre- and post-license driver training. Saf. Sci. 2013, 51, 127–137. [Google Scholar] [CrossRef]
- Roberts, I.G.; Kwan, I. School-based driver education for the prevention of traffic crashes (Review). Cochrane Database Syst. Rev. 2001, 3, CD003201. [Google Scholar]
- Shell, D.F.; Newman, I.M.; Córdova-Cazar, A.L.; Heese, J.M. Driver education and teen crashes and traffic violations in the first two years of driving in a graduated licensing system. Accid. Anal. Prev. 2015, 82, 45–52. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Walshe, E.A.; Romer, D.; Wyner, A.J.; Cheng, S.; Elliott, M.R.; Zhang, R.; Gonzalez, A.K.; Oppenheimer, N.; Winston, F.K. Licensing examination and crash outcomes postlicensure in young drivers. JAMA Netw. Open 2022, 5, e228780. [Google Scholar] [CrossRef] [PubMed]
- Mayhew, D.; Marcoux, K.; Wood, K.; Simpson, H.; Valnaar, W.; Lonero, L.; Clinton, K. Evaluation of Beginner Driver Education Programs: Studies in Manitoba and Oregon (Report); AAA Foundation for Traffic Safety: Washington, DC, USA, 2014. [Google Scholar]
- Alharbi, M.A.; Alghammas, A. Teacher written vs audio feedback on undergraduates’ written assignments. Theory Pract. Lang. Stud. 2021, 11, 1562–1570. [Google Scholar] [CrossRef]
- Grundmann, F.; Scheibe, S.; Esptude, K. When ignoring negative feedback is functional: Presenting a model of motivated feedback disengagement. Curr. Dir. Psychol. Sci. 2021, 30, 3–10. [Google Scholar] [CrossRef]
- Walshe, E.A.; Elliott, M.R.; Romer, D.; Cheng, S.; Curry, A.E.; Seacrist, T.; Oppenheimer, N.; Wyner, A.J.; Grethlein, D.; Gonzalez, A.K.; et al. Novel use of a virtual driving assessment to classify driver skill at the time of licensure. Transp. Res. Part F Traffic Psychol. Behav. 2022, 87, 313–326. [Google Scholar] [CrossRef]
- Walshe, E.A.; Romer, D.; Kandadai, V.; Winston, F.K. A novel health-transportation partnership paves the road for young driver safety through virtual assessment. Health Aff. 2020, 39, 1792–1798. [Google Scholar] [CrossRef]
- Farah, H.; Musican, O.; Shimshoni, Y.; Toledo, T.; Grimberg, E.; Omer, H.; Lotan, T. Can providing feedback on driving behavior and training on parental vigilant care affect male teen drivers and their parents? Accid. Anal. Prev. 2014, 69, 62–70. [Google Scholar] [CrossRef]
- Mirman, J.H.; Curry, A.E.; Elliott, M.R.; Long, L.; Pfeiffer, M.R. Can adolescent drivers’ motor vehicle crash risk be reduced by pre-licensure intervention? J. Adolesc. Health 2018, 62, 341–348. [Google Scholar] [CrossRef] [PubMed]
- Simons-Morton, B.G.; Bingham, C.R.; Ouimet, M.C.; Pradhan, A.K.; Chen, R.; Barretto, A.; Shope, J.T. The effect on teenage risky driving of feedback from a safety monitoring system: A randomized controlled trial. J. Adolesc. Health 2013, 53, 21–26. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Curry, A.E.; Pfeiffer, M.R.; Durbin, D.R.; Elliott, M.R. Young driver crash rates by licensing age, driving experience, and license phase. Accid. Anal. Prev. 2015, 80, 243–250. [Google Scholar] [CrossRef]
- Fatality Facts 2020: Teenagers. Available online: https://www.iihs.org/topics/fatality-statistics/detail/teenagers (accessed on 28 August 2022).
- Curry, A.E.; Hafetz, J.; Kallan, M.J.; Winston, F.K.; Durbin, D.R. Prevalence of teen driver errors leading to serious motor vehicle crashes. Accid. Anal. Prev. 2011, 43, 1285–1290. [Google Scholar] [CrossRef] [PubMed]
- Khattak, A.J.; Ahmad, N.; Wali, B.; Dumbaugh, E. A taxonomy of driving errors and violations: Evidence from the naturalistic driving study. Accid. Anal. Prev. 2021, 151, 105873. [Google Scholar] [CrossRef] [PubMed]
- Seacrist, T.; Maheshwari, J.; Sarfare, S.; Chingas, G.; Thirkill, M.; Loeb, H.S. In-depth analysis of crash contributing factors and potential ADAS interventions among at-risk drivers using the SHRP2 naturalistic driving study. Traffic Inj. Prev. 2021, 22, S68–S73. [Google Scholar] [CrossRef]
- Saricaoglu, A. The impact of automated feedback on L2 learners’ written causal explanations. ReCALL 2019, 31, 189–203. [Google Scholar] [CrossRef]
- Núñez-Peña, M.I.; Bono, R.; Suárez-Pellicioni, M. Feedback on students’ performance: A possible way of reducing the negative effect of math anxiety in higher education. Int. J. Educ. Res. 2015, 70, 80–87. [Google Scholar] [CrossRef][Green Version]
- Cheng, A.; Overly, F.; Kessler, D.; Nadkarni, V.M.; Lin, Y.; Doan, Q.; Duff, J.P.; Tofil, N.M.; Bhanji, F.; Adler, M.; et al. Perception of CPR quality: Influence of CPR feedback, just-in-time CPR training and provider roles. Resuscitation 2015, 87, 44–50. [Google Scholar] [CrossRef]
- Chen, Y.; Zhang, Y.; Zhang, F. Personalized path generation and robust H∞ output-feedback path following control for automated vehicles considering driving styles. IET Intell. Transp. Syst. 2021, 15, 1582–1595. [Google Scholar] [CrossRef]
- Mullen, J.; Ryan, S.R.; Mathias, C.W.; Dougherty, D.M. Feasibility of a computer assisted alcohol screening, brief intervention and referral to treatment program for DWI offenders. Addict Sci. Clin. Pract. 2015, 10, 25. [Google Scholar] [CrossRef] [PubMed]
- Guskey, T.R. Grades versus comments: Research on student feedback. Phi Delta Kappan 2019, 101, 42–47. [Google Scholar] [CrossRef]
- Peters, E.; Dieckmann, N.; Dixon, A.; Hibbard, J.H.; Mertz, C.K. Less is more in presenting quality information to consumers. Med. Care Res. Rev. 2007, 64, 169–190. [Google Scholar] [CrossRef] [PubMed]
- Mumm, J.; Mutlu, B. Designing motivational agents: The role of praise, social comparison, and embodiment in computer feedback. Comput. Human Behav. 2011, 27, 1643–1650. [Google Scholar] [CrossRef]
- Peters, E.; Hibbard, J.; Slovic, P.; Dieckmann, N. Numeracy skill and the communication, comprehension, and use of risk-benefit information. Health Aff. 2007, 26, 741–748. [Google Scholar] [CrossRef][Green Version]
- Peters, E.; Dieckmann, N.; Västfjäll, D.; Mertz, C.K.; Slovic, P.; Hibbard, J.H. Bringing meaning to numbers: The impact of evaluative categories on decision. J. Exp. Psychol. Appl. 2009, 15, 213–227. [Google Scholar] [CrossRef][Green Version]
- Peters, E.; Västjäll, D.; Slovic, P.; Mertz, C.K.; Mazzocco, K.; and Dickert, S. Numeracy and decision making. Psychol. Sci. 2006, 17, 407–413. [Google Scholar] [CrossRef]
- Chandler, J.; Rosenzweig, C.; Moss, A.J.; Robinson, J.; Litman, L. Online panels in social science research: Expanding sampling methods beyond Mechanical Turk. Behav. Res. Methods 2019, 51, 2022–2038. [Google Scholar] [CrossRef][Green Version]
- Harris, P.A.; Taylor, R.; Thielke, R.; Payne, J.; Gonzalez, N.; Conde, J.G. Research electronic data capture (REDCap)—A metadata-driven methodology and workflow process for providing translational research informatics support. J. Biomed. Inform. 2009, 42, 277–381. [Google Scholar] [CrossRef][Green Version]
- Racial and Ethnic Categories and Definitions for NIH Diversity Programs and for Other Reporting Purposes. Available online: https://grants.nih.gov/grants/guide/notice-files/not-od-15-089.html (accessed on 15 November 2020).
- Cokely, E.T.; Galesic, M.; Schulz, E.; Ghazal, S.; Garcia-Retamero, R. Measuring risk literacy: The Berlin Numeracy Test. Judgm. Decis. Mak. 2012, 7, 25–47. [Google Scholar]
- Weller, J.A.; Dieckmann, N.F.; Tusler, M.; Mertz, C.K.; Burns, W.J.; Peters, E. Development and testing of an abbreviated numeracy scale: A Rasch analysis approach. J. Behav. Decis. Mak. 2013, 26, 198–212. [Google Scholar] [CrossRef] [PubMed]
- Dogan, E.; Steg, L.; Delhomme, P.; Rothengatter, T. The effects of non-evaluative feedback on drivers’ self-evaluation and performance. Accid. Anal. Prev. 2012, 45, 522–528. [Google Scholar] [CrossRef] [PubMed]
- Wilson, J.; Potter, A.; Cruz Cordero, T.; Myers, M.C. Integrating goal-setting and automated feedback to improve writing outcomes: A pilot study. Innov. Lang. Learn. Teach. Available online: https://www.tandfonline.com/doi/full/10.1080/17501229.2022.2077348 (accessed on 15 November 2020).
- Horswill, M.S.; Garth, M.; Hill, A.; Watson, M.O. The effect of performance feedback on drivers’ hazard perception ability and self-ratings. Accid. Anal. Prev. 2017, 101, 135–142. [Google Scholar] [CrossRef] [PubMed]
- Brooke, J. SUS: A “quick and dirty” usability scale. In Usability Evaluation in Industry; Jordan, P.W., Thomas, B., Weerdmeester, B.A., McClelland, A.L., Eds.; Taylor and Francis: London, UK, 1996. [Google Scholar]
- Ghanbari, S.; Ramezankhani, A.; Montazeri, A.; Mehrabi, Y. Health Literacy Measure for Adolescents (HELMA): Development and psychometric properties. PLoS ONE 2016, 11, e0149202. [Google Scholar] [CrossRef]
- Manser, M.P.; Noble, A.M.; Machiani, S.G.; Shortz, A.; Klauer, S.G.; Higgins, L.; Ahmadi, A. Driver Training Research and Guidelines for Automated Vehicle Technology (Report); Virginia Tech Transportation Institute (VTTI): Blacksburg, VA, USA, 2019. [Google Scholar]
|Condition||Summary||AP Length||AP Order||AP Grading||Peer Comparison|
|Non-binary or Other||17||3.26|
|License endorsement 1||23||4.41|
|Highest Education Completed|
|Less than high school||4||0.77|
|High school degree||85||16.31|
|Some college/trade school||199||38.20|
|More than 4-year degree||34||6.53|
|Native American/Alaskan Native||0||0.00|
|Native Hawaiian/Other Pacific Islander||2||0.38|
|More than one race||45||8.64|
|Condition||n||Motivation to Improve||Easy to Use||Easy to Understand|
|mean of medians||mean of medians||mean accuracy|
|Feature||Conditions||n||Motivation to |
|Easy to Use||Easy to |
|Mean 1||p||Mean 1||p||Mean 1||p|
|Report Length 2|
|Summary Present||1 + 2||237||0.02||0.02 *||0.06||0.90||−0.01||0.45|
|Summary Absent||3 + 4||−0.32||−0.33||0.01|
|AP Short||2 + 4||−0.19||0.65||−0.21||0.01 *||0.04||0.002 *|
|AP Long||1 + 3||−0.12||−0.06||−0.03|
|Summary * AP||NA||0.007 *||NA|
|AP Order 3|
|AP Grading 3|
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Ward McIntosh, C.M.; Walshe, E.A.; Cheng, S.; Winston, F.K.; Peters, E. Keep It Brief and Targeted: Driving Performance Feedback Report Features to Use with Novice Drivers. Adolescents 2022, 2, 448-458. https://doi.org/10.3390/adolescents2040035
Ward McIntosh CM, Walshe EA, Cheng S, Winston FK, Peters E. Keep It Brief and Targeted: Driving Performance Feedback Report Features to Use with Novice Drivers. Adolescents. 2022; 2(4):448-458. https://doi.org/10.3390/adolescents2040035Chicago/Turabian Style
Ward McIntosh, Chelsea M., Elizabeth A. Walshe, Shukai Cheng, Flaura K. Winston, and Ellen Peters. 2022. "Keep It Brief and Targeted: Driving Performance Feedback Report Features to Use with Novice Drivers" Adolescents 2, no. 4: 448-458. https://doi.org/10.3390/adolescents2040035