Elevating Academic Advising: Natural Language Processing of Student Reviews

Abstract

Academic advising is often pivotal in shaping students’ educational experiences and choices. This study leverages natural language processing to quantitatively evaluate reviews of academic advisors, aiming to provide actionable insights on key feedback phrases and demographic factors for enhancing advising services. This analysis encompassed a comprehensive evaluation of 1151 reviews of undergraduate students for academic advisors, which were collected within a European University alliance consisting of five universities, offering a diverse pool of feedback from a wide range of academic interactions. Employing sentiment analysis powered by artificial intelligence, we computed compound sentiment scores for each academic advisor’s reviews. Subsequently, statistical analyses were conducted to provide insights into how demographic factors may or may not influence students’ sentiment and evaluations of academic advisory services. The results indicated that advisor’s gender had no substantial influence on the sentiment of the reviews. On the contrary, the academic advisors’ age showed a notable impact, with younger advisors surprisingly receiving more favorable evaluations. Word frequency analyses, both for positive and negative expressions, were also performed to contextualize the language used in describing academic advisors. The prevalent word combinations in reviews of highly rated academic advisors emphasized attributes like empathy, approachability, and effectiveness in guiding students towards achieving their academic goals. Conversely, advisors with less favorable reviews were often perceived as inadequate in addressing students’ concerns related to their academic journey, revealing persistent challenges in the student–advisor interaction that impacted their evaluation. This analysis of academic advisor reviews contributes to the body of literature by highlighting the significance of managing student expectations and enhancing advisor skills and qualities to foster positive interactions and academic success.

1. Introduction

Academic advising is a program or initiative employed within educational systems to offer students guidance and counsel [1]. It is widely respected and extensively employed to support students in educational institutions around the world [2]. The field of academic advising, as a student support service, has undergone significant transformation in recent years, paralleling the broader evolution within the educational landscape. In the realm of higher education, universities have made significant strides in implementing comprehensive evaluation processes for various educational aspects. These include the evaluation of teachers’ performance, assessment of academic programs of study, appraisal of administrative services, and even the scrutiny of extracurricular activities and student support services. However, this progress rarely extends to academic advising [3]. Unlike the robust evaluation mechanisms in place for many academic processes, the assessment of academic advising performance remains relatively unexplored.

In this paper, we utilize natural language processing methods for a quantitative examination of student feedback in an effective and streamlined approach to develop a more profound comprehension of students’ interactions with academic advisors. The core purpose of this study is to evaluate assessments of advisors within the academic advising sphere and explore the demographic influences on students’ sentiments and their evaluations. We employ sentiment analysis, a powerful artificial intelligence technique, to systematically assess the linguistic content of these reviews, thereby offering an impartial and practical evaluation of the academic advising process. By using statistical analyses, we performed word frequency analyses for both positive and negative expressions to provide valuable context to the language used in describing academic advisors. This comprehensive approach offered a nuanced understanding of the dynamics between students and their academic advisors, contributing to the ongoing development of academic advising services.

The research presented in this paper was conducted within the INVEST European University (INnoVations of REgional Sustainability: European UniversiTy Alliance https://www.invest-alliance.eu/, accessed on 1 December 2023), which is an alliance that consists of five universities spread across Europe. This alliance allowed us to collect feedback and insights from students hailing from various European countries and academic backgrounds. By transcending the boundaries of a single institution, our research can offer a more comprehensive understanding of academic advising experiences, incorporating the multifaceted factors that shape students’ perceptions and expectations. Through this collaboration, we aim to provide insights and solutions that can enhance academic advising services, not only for our immediate community but for the broader European higher education landscape, as well. This interconnected approach aligns perfectly with the spirit and goals of the European Universities initiative, making it an essential and timely contribution to the advancement of higher education.

This study aims to delve into the intricate landscape of academic advisor representation through the lens of student reviews by employing natural language processing techniques. The primary objectives include dissecting the multifaceted perceptions expressed in these evaluations and gaining a deeper understanding of the different perspectives portrayed in student feedback. Notably, our research marks a significant step as it is the first study, to our knowledge, to utilize sentiment analysis to acquire insights into the portrayal of academic advisors within student reviews. This unique approach enables us to explore uncharted territory in comprehending the complex dynamics and subtle nuances embedded within these evaluations. The utilization of natural language processing techniques offers a novel vantage point, allowing us to unravel intricate patterns and sentiments that may have remained obscured without such technological processing. By leveraging these analytical tools, we aim to uncover hidden layers within student feedback, providing a more comprehensive understanding of the student–advisor relationship and the factors influencing student evaluations.

The remaining parts of this paper are in the following order: In Section 2, we embark on an exploration of sentiment analysis in education. Following this, Section 3 details our research methodology, covering various aspects including data acquisition and sentiment analysis approach. Section 4 presents our detailed findings, discussing the outcomes of our analyses, including results from linear regression, demographic influences on student sentiment, word analysis insights, and multivariate analysis. In Section 5, we engage in a discussion to contextualize and interpret our findings. Section 6 is dedicated to acknowledging the limitations of this study, while Section 7 concludes the paper by summarizing our results and suggesting potential directions for future research.

2. Sentiment Analysis in Education

Sentiment analysis has proven invaluable in gaining insights into students’ learning behavior and performance. Jena employed sentiment analysis to model students’ emotions based on their data from platforms like Moodle, Twitter, and Facebook [4]. The study’s large dataset of 12,300 tweets, 10,500 Facebook comments, and 8450 Moodle feedback messages allowed for a comprehensive understanding of student sentiment in collaborative learning environments. Santos and Rita analyzed online reviews written by international students to understand the variation in students’ satisfaction toward higher education institutions, shedding light on the different factors that influence learning behavior [5]. These studies emphasize the role of sentiment analysis in understanding students’ feelings, behavior, and experiences.

Moreover, many studies have demonstrated the potential of sentiment analysis to enhance various systems within the academic environment. Sentiment analysis has been incorporated into management systems, online learning platforms, and evaluation systems, enabling real-time analysis of student feedback. For instance, Cobos and Jurado used sentiment analysis to extract user opinions about online courses, thus effectively enhancing the learning materials of these courses [6]. Kandhro and Chhajro developed sentiment analysis models to improve the teaching quality in higher education institutions [7]. These applications in evaluation systems showcase the capacity of these techniques to augment academic platforms for better student experiences. In addition, Balachandran and Kirupananda introduced a system that rates higher education institutions based on sentiment analysis of student reviews from platforms like Facebook and Twitter, providing students with a valuable tool for making informed choices about their education [8].

Several studies have demonstrated the capacity of sentiment analysis to enhance the teaching and learning process. Newman and Joyner explored its significance in analyzing student evaluations of teaching, providing insights into instructor strengths [9]. Balahadia and Fernando developed a teacher’s performance evaluation tool, leveraging opinion mining with sentiment analysis to identify faculty members’ strengths and weaknesses based on student feedback [10]. Dhanalakshmi and Bino compared various algorithms to predict the polarity of student feedback, emphasizing its potential to improve the teaching quality [11]. These findings underscore the role of natural language processing in improving teaching and learning processes by providing valuable insights into instructor performance, student evaluations, and teaching quality.

While previous research demonstrates the power of sentiment analysis in improving teaching quality, student satisfaction, faculty evaluation, and other facets of academia, there is a notable gap in understanding how natural language processing techniques can enhance academic advising. This study aims to bridge this gap by investigating the untapped potential of sentiment analysis in the academic advising domain.

3. Research Methodology

3.1. Data Acquisition

The data collected for this study comprise records of feedback surveys of students who attended the universities in the INVEST European University alliance. This alliance creates a unified European university campus with a substantial student population, through which data collection is conducted more comprehensively and efficiently. The focus of this investigation was undergraduate students in various academic disciplines and semesters who had sought academic advisory services as part of their educational journey. Academic advising at the sampled universities predominantly involved 107 faculty members who assumed advisory roles alongside their teaching responsibilities. While there might be variations in the extent of counseling specialization, faculty members commonly integrate academic advising into their responsibilities, providing guidance on course selections, learning pathways, and academic support. The initial dataset encompassed a total of 1156 students reviews.

Students were requested to complete 5-point Likert-type scales assessing their perceptions of the academic advisor’s skills. These Likert scale items encompassed the following main aspects:

• My academic advisor encouraged me to ask questions.
• My academic advisor was receptive to discussing any academic topic of importance to me.
• My academic advisor provided me with information and recommendations that were easy to comprehend.
• I felt that my academic advisor understood my academic goals, concerns, and needs.
• During your advising session, do you believe you received sufficient relevant information from your academic advisor?

In addition to the Likert rating, each student response included a required textual evaluation of the advising sessions, allowing students to capture their assessments of various facets of the academic advisor’s skills.

The data collection process commenced in the academic year of 2023 and spanned over a period of 3 months, focusing on academic advising interactions from the past 3 years. In this study, students proficient in English completed the survey in English, while those more comfortable in their native languages were provided with the option to respond in their preferred language. To ensure accurate translations, multilingual partners of the INVEST project collaborated on the translation process, rigorously verifying and cross-checking the translated versions for fidelity to the original survey content. These validations were conducted through a meticulous process to guarantee linguistic accuracy and consistency across languages.

During data preprocessing, we incorporated inclusion criteria, ensuring that only responses meeting specified parameters of completeness, relevance, and clarity were included for detailed analysis. Conversely, exclusion criteria involved disregarding responses that lacked substantive content or displayed inconsistencies, ensuring the quality and accuracy of the analyzed dataset. Subsequently, through this vetting process, 5 outliers were identified and excluded from the dataset. These exclusions were based on indications that these particular responses might have been affected by potential misinterpretations of the Likert rating scales or lacked the necessary coherence and relevance required for robust analysis.

3.2. Sentiment Analysis Approach

We obtained sentiment features by utilizing the widely recognized VADER (Valence Aware Dictionary for sEntiment Reasoning) tool for sentiment analysis through the NLTK natural language processing toolkit [12]. VADER computes compound sentiment analysis scores for written text, determining the positivity or negativity of a sentence based on the language used and the context of the text. It relies on a lexicon of word scores, which were crafted by ten independent human raters. Each word in the lexicon was assigned a score, which could range from −4 (indicating an extremely negative sentiment) to +4 (representing a highly positive sentiment), with 0 representing neutrality. VADER utilizes this lexicon as a basis for determining sentiment scores in a given text. It computes these scores and subsequently standardizes the outcome to a range between −1 (indicating a strongly negative sentiment) and +1 (indicating a highly positive sentiment). Its algorithm assigns scores, considering punctuation, capitalization, and word modifiers, providing a comprehensive analysis of the sentiment in a given set of text.

Apart from the intensity of sentiment words, VADER identifies properties and specific attributes within the text, particularly in brief comments, influencing the perceived sentiment score of the text. For instance, an exclamation mark increases the intensity of sentiment orientation. Furthermore, it recognizes various sentiment-infused emoticons as well as sentiment-laden initialisms and acronyms. VADER accommodates negation by reversing the rating’s sign when it is applied to words listed within the VADER lexicon. This enables the algorithm to accurately evaluate phrases like “not friendly” assigning them a negative score to reflect the altered context surrounding the word.

In conjunction with this tool, we initiated the sentiment analysis process by tokenizing the collected textual reviews through NLTK’s sophisticated tokenization methods, thus segmenting the comments into individual words or tokens for further analysis. Stop words—common but noninformative terms such as “the”, “and”, and “or”—were systematically removed from this tokenized corpus, ensuring that only significant, content-rich words contributed to the sentiment analysis. VADER proceeded to evaluate the sentiment scores for the refined set of words. It considered not just the emotional intensity of individual words but also their context within each sentence. Punctuation, capitalization, and word modifiers were all taken into account in the computation of sentiment scores, providing a nuanced analysis that considered the intricacies of language. Moreover, VADER intelligently accounted for various linguistic nuances such as negation, accurately assessing sentiments, even when phrases might reverse the expected sentiment due to negating terms. This meticulous methodology allowed for a comprehensive understanding of sentiment nuances within the student feedback, thus enabling a profound evaluation of perceptions and attitudes toward academic advisors.

Regarding the utilization of lexicon analysis in our study, it is essential to clarify that although lexicon analysis itself is not directly categorized as a machine learning approach, the lexicon used has its roots in AI methodologies. VADER, as a sentiment analysis tool, was developed using machine learning techniques and natural language processing to construct its sentiment lexicon, which our study employed for sentiment analysis.

4. Results

4.1. Linear Regression

To evaluate the effectiveness of our sentiment analysis in gauging the perceptions of students toward academic advisors, we conducted a linear regression analysis. This analysis aimed to explore the relationship between the average sentiment analysis scores generated from student feedback and the corresponding average Likert ratings provided by students.

The results of our linear regression analysis revealed a statistically significant and positive correlation between these two sets of scores (see Figure 1, r² = 0.561, p value < 0.01). This finding indicates a robust agreement in sentiment scores and the Likert ratings assigned by students to their academic advisory interactions. It suggests that the sentiment analysis scores align closely with the reported Likert ratings, indicating that the sentiment analysis effectively captures the sentiments and perceptions of students regarding academic advisor–student interactions. This validation supports the utility and reliability of our approach in quantitatively assessing student sentiment and feedback.

4.2. Demographic Influences on Student Sentiment and Academic Advisor Evaluations

To assess the influence of demographic variables on students’ sentiment towards academic advisors, we conducted a series of statistical tests. Firstly, a student t-test was employed to determine if there was a significant difference between the sentiment analysis scores provided for male and female academic advisors (

Table 1. Comparison of student reviews in relation to the gender of academic advisors.

	Male Average	Female Average	p-Value
Sentiment analysis	+0.61	+0.63	0.112
Likert	4.52	4.41	0.371

). The analysis showed that gender had an insignificant effect on sentiment scores (males = +0.61, females = +0.63; p = 0.112). Similarly, the comparison of mean Likert scores with gender revealed no significant differences (males = 4.52, females = 4.41; p = 0.371).

Additionally, a one-way ANOVA was conducted to explore potential variations in sentiment analysis scores across different age groups of academic advisers. The results revealed a significant distinction among the four age groups, with average sentiment scores of +0.73 for those under 40, +0.58 for individuals aged 40–49, +0.41 for those between 50–59, and +0.37 for those above 59 (p < 0.01). This suggests that in our research, academic advisors of a younger age generally tend to convey more favorable sentiments overall. Similarly, the mean Likert scores for the four age groups exhibited significant differences, with scores of 4.72, 4.31, 4.11, and 3.89, respectively (p < 0.01) (see

Table 2. Comparison of student reviews in relation to the age of academic advisors.

	<40	40–49	50–59	>59	p-Value
Sentiment analysis	+0.73	+0.58	0.41	+0.37	<0.01
Likert	4.72	4.31	4.11	3.89	<0.01

for details).

Finally, we conducted a university analysis by categorizing academic advisors into five geographical subgroups based on their affiliated universities: University of Thessaly, Greece (UTH); Slovak University of Agriculture, Slovakia (SUA); University of Agribusiness and Rural Development, Bulgaria (UARD); Karelia University of Applied Sciences, Finland (KARELIA); and Van Hall Larenstein University of Applied Sciences, Netherlands (VHL). To explore the potential impact of university on the relationships between average Likert ratings and sentiment analysis scores, we employed one-way ANOVA tests. In contrast to age, no statistically significant differences were observed in Likert ratings and sentiment analysis scores across the diverse universities (refer to

Table 3. Comparison of Likert ratings and textual reviews in relation to the universities of the INVEST alliance.

	UTH	SUA	UARD	KARELIA	VHL	p-Value
Sentiment analysis	+0.62	+0.6	0.59	+0.69	+0.61	0.213
Likert	4.55	4.48	4.46	4.51	4.59	0.392

).

These analyses provide insights into how demographic factors may or may not influence students’ sentiments and evaluations of academic advisory services.

4.3. Word Analysis

To gain insight into the most common and impactful words used in student reviews of academic advisors, we conducted a word frequency analysis. The analysis focused on identifying frequently used words that could shed light on the key terms affecting student reviews. To ensure the relevance and academic significance of the words, common high-frequency yet less pertinent terms, such as “great” or “awesome” were excluded from the analysis.

In our analysis of the most positively reviewed academic advisors, we found that students often used words that emphasized qualitative and behavioral attributes in their feedback. Notably, terms like “Understanding”, “Postgraduate”, and “Skills” highlighted positive aspects of advisors’ qualities, while the distinct mention of “Goals” and “Results” signified a strong correlation with addressing students’ academic objectives. Conversely, the descriptions for academic advisors who received the most negative reviews often centered on aspects related to students’ academic “Goals” and “Needs”, highlighting specific challenges in the students’ academic journey. Terms like “Availability”, “Clear”, and “Rude” indicated negative aspects of advisor qualities (see

Table 4. Relevant analysis of word frequencies in reviews.

Top-Rated Reviews		Low-Rated Reviews
Word	Frequency	Word	Frequency
Goals	10.56%	Needs	14.67%
Friendly	10.00%	Goals	1.98%
Postgraduate	7.79%	Clear	1.88%
Skills	5.07%	Availability	1.56%
Understanding	3.30%	Rude	0.67%

). Additionally, we conducted an analysis to identify the most frequently occurring pairs of words.

Table 5. Relevant analysis of word pairs frequencies in reviews.

Top-Rated Reviews		Low-Rated Reviews
Word	Frequency	Word	Frequency
Academic goals	3.26%	Personal needs	1.68%
Communication skills	1.26%	Poor communication	1.58%
Friendly approach	1.19%	Different goals	0.12%
Postgraduate program	1.19%	Low availability	0.91%
Felt comfortable	1.18%	Clear instructions	0.74%

presents specific word pairs that surfaced more frequently in student reviews, indicating positive experiences. Expressions such as “Academic goals”, “Postgraduate program”, “Friendly approach”, “Communication skills”, and “Felt Comfortable” were frequently highlighted, indicating favorable evaluations. On the other hand, in our analysis of low-rated academic advisors, students often used word pairs that reflected challenges and negative experiences. Phrases like “Personal needs”, “Poor communication”, “Different goals”, “Low availability”, and “Clear instructions” were recurrent in these reviews, indicating less favorable assessments (see Table 5).

4.4. Multivariate Analysis

Subsequently, a multiple logistic regression analysis was performed on academically relevant keywords. This analysis aimed to ascertain the probability of academically pertinent words and word pairs being present in a student review with an overall sentiment analysis score exceeding 0.5.

The results of this analysis revealed a positive association between words describing positive academic advisor behaviors, such as “Empathy” (OR = 6.21), “Friendly” (OR = 4.36), and “Accessible” (OR = 2.51), and reviews with positive sentiment outputs (refer to

Table 6. Multiple logistic regression analysis on academically relevant keywords.

Word	Odds Ratio (OR)	95% Confidence Interval (CI)	p-Value
Academic goals	3.11	1.89–5.20	<0.01
Accessible	2.51	1.28–4.20	<0.01
Availability	0.47	0.21–0.97	0.04
Confident	1.41	0.81–3.42	0.02
Digital resources	0.71	0.42–2.17	0.64
Empathy	6.21	2.81–12.42	<0.01
Friendly	4.36	1.25–11.06	0.02
Graduation	3.02	1.56–5.12	<0.01
Postgraduate	3.25	2.10–5.41	<0.01
Clear instructions	0.31	0.24–0.44	<0.01
Virtual meetings	0.42	0.08–3.15	0.53
Well-informed	1.54	1.06–2.33	0.04

).

Furthermore, the results from this analysis underscore the significance of meeting students’ expectations toward their “Academic goals” in shaping academic advisor evaluations. The inclusion of terms like “Availability” and “Clear instructions” was correlated with a decreased likelihood of receiving positive reviews (with respective odds ratios of 0.47 and 0.31). Conversely, the presence of “Graduation”, “Postgraduate”, and/or “Academic goals” in advisor reviews was linked to a threefold rise (odds ratios of three, approximately) in the probability of receiving a favorable review.

5. Discussion

The primary objective of this study was to pinpoint the academic advisor demographics and critical feedback expressions that exhibit the strongest connection with favorable or unfavorable responses from students. An existing similar analysis of 396 undergraduate student reviews revealed that advisor characteristics, including accessibility, availability, encouragement, motivation, and care, significantly influenced the intensity of student ratings [13]. Yet, as far as our research indicates, this is the first study to employ natural language processing to acquire insights into the representation of academic advisors in student reviews, granting us a deeper understanding of the nuanced perceptions expressed in these evaluations. This approach not only facilitated a comprehensive analysis but also unveiled intricate patterns and sentiments that might have remained obscured without the utilization of this AI processing, enhancing the precision and depth of our findings.

Furthermore, to our knowledge, no prior study has examined the correlation between the age of academic advisors and student ratings of their performance. In a related study, Stonebraker et al. conducted a demographic analysis of 3629 university professors and discovered that those with fewer cumulative years of academic experience obtained significantly higher ratings in comparison to their more senior counterparts [14]. In a similar vein, Murray et al., who examined 18,946 records, reported that older tenure-track faculty members tended to receive lower overall ratings [15]. Joye et al. also noted that professors with fewer years of experience garnered more positive reviews [16]. Our analysis, which is distinct from existing studies focused on professors, discovered a similar trend within academic advising, indicating that younger advisors tended to garner higher ratings and receive more positively framed student reviews. This parallel trend echoes findings observed in the broader landscape of professorial evaluations, highlighting a consistent correlation between advisor age and favorable student perceptions.

Moreover, Moghadam et al. observed that the gender of academic advisors had no substantial effect on the ratings of academic advisors [17]. Similarly, in a study based on data from a survey of U.S. graduate students in five distinct disciplines within the natural and social sciences, gender was not consistently identified as a significant factor influencing the positivity of reviews [18]. Our research is consistent with prior studies, as we found that the gender of academic advisors had no noticeable influence on their Likert scores or the textual reviews. This implies that both male and female academic advisors are evaluated similarly by students.

In our examination, we discovered that there were no notable statistical variations observed in Likert ratings and sentiment analysis scores among the five universities of the INVEST alliance. It is essential to highlight that the INVEST European University provides a set of digital resources and tools including a comprehensive academic advising system powered by AI, aiming to support both academic advisors and students in their decision-making processes [19]. Despite these advancements, our findings indicate that the use of digital resources and virtual meetings did not significantly affect the student reviews for the academic advisors. In contrast, prior studies have highlighted that academic advisors who leverage virtual meetings and advanced technology were associated with higher ratings [19,20].

Furthermore, previous studies have shown that academic advisors characterized as empathetic, responsive, and helpful were more inclined to receive elevated ratings from students [21,22]. Likewise, our study indicates that academic advisors whom students perceived accessible, friendly, and empathetic to their academic needs were significantly more likely to receive positive reviews. When these words were incorporated into student reviews, they were 2.5 times, 4.3 times, and 6.2 times more inclined to be positive, respectively. Furthermore, this study underscores the importance of instilling confidence and having a strong knowledge of academic programs, policies, and university resources to provide accurate guidance. Advisors who were described as confident and well-informed consistently received higher reviews than those who were not. As evidenced by this study, students pay close attention to the aforementioned qualities as they significantly impact their experiences with academic advisors.

Students’ academic goals and needs emerged as substantial factors influencing both highly positive and considerably negative student reviews of academic advisors. The most commendable reviews praised advisors for their pivotal role in guiding students toward graduation or postgraduate studies, lauding their aforementioned skills and qualities. In contrast, unfavorable reviews frequently highlighted students’ experiences of unaddressed concerns, revealing persistent challenges in the student–advisor interaction that impacted their evaluations. Soria et al., in their study on students’ perspectives on academic advising, emphasized that academic advising should be directed toward achieving higher retention and graduation rates [23]. Their research underscored that students typically have high expectations for academic advising to support their path to graduation. Our study adds to the growing body of literature on advisees’ expectations. Students often anticipate that academic advisors will play a significant role in assisting them to achieve academic milestones like graduation or postgraduate studies. However, when these expectations are not met, student dissatisfaction tends to intensify, which is often reflected in their feedback. Therefore, this study emphasizes the significance of assessing these expectations before putting academic interventions into practice. Advisors who can dispel any misconceptions about the extent of academic advising’s role in supporting graduation are better positioned to mitigate potential dissatisfaction and effectively address these concerns.

Drawing from the insights gleaned in this study, it is crucial for academic institutions to leverage comprehensive evaluations to further enhance the performance and effectiveness of their academic advisors, ensuring that they meet the diverse needs of today’s students. Historically, academic advisor training and upskilling has not been the norm. However, McGill et al. highlighted the necessity of comprehensive training and continuing professional education and development for academic advisors [24]. The current study reaffirms the significance of integrating these educational initiatives into the development of academic advisors in their academic roles. Enhancing the abovementioned skills and qualities significantly improves the student experience and, consequently, enhances academic advisor evaluations.

6. Limitations

Our research was conducted in the context of the INVEST European University alliance, allowing us to collect a diverse and impactful dataset from different universities spread across Europe. However, it is important to note that these universities may not comprehensively represent the entire countries they are located in, and as such, the findings may be limited in their generalizability beyond the scope of the alliance. Furthermore, in the scope of our study, which was conducted within the university alliance, which offers diverse programs across multiple academic cycles and supplementary educational activities like winter and summer schools, Living Labs, and other extracurricular engagements, students from various countries have the opportunity to pursue learning pathways across different member universities. The academic advisors included in this study were faculty members affiliated with the INVEST University, operating within an environment characterized by substantial academic mobility. As a consequence of this dynamic and widespread educational ecosystem, calculating the precise proportion of academic advisors by subject areas or counseling types poses challenges due to the aforementioned interconnected academic networks.

A significant limitation of this study arises from the potential multifaceted nature of the interaction between student and advisor. This study lacks the capability to determine if unfavorable reviews persist even when academic advisors make sincere efforts to address issues or recommend improvements. In many instances, challenges students face during their academic studies may persist despite extensive and suitable support from academic advisors, depending on various factors such as students’ specific needs or circumstances. Additionally, it is often challenging to discern whether the individuals leaving reviews have had extensive academic advising interactions, including ongoing guidance throughout their academic journey, or if they are basing their reviews on a single office visit.

7. Conclusions

In this study, we utilized sentiment analysis to quantitatively evaluate reviews of academic advisors, aiming to provide actionable insights for enhancing academic advising services. This study primarily focused on academic advisors within the context of a European University alliance, thus aiming to attain a more extensive dataset that encompasses a broader spectrum of student experiences. Our analysis revealed that advisor characteristics, particularly age and specific traits like empathy, knowledgeability, approachability, and the ability to provide academic support toward academic goals, significantly influence the positivity or negativity of student ratings and reviews. This study is, to our knowledge, the first to employ natural language processing to gain insights into how academic advisors are portrayed by their students.

This work underscores the importance of evaluating academic advising processes in higher education. It not only identifies key elements that influence student perceptions but also offers a concrete AI method for assessing and improving academic advising services. By shedding light on the dynamics between academic advisors and students, this study contributes to the ongoing development of academic advising and highlights the path to more effective and student-centered educational experiences.