Next Article in Journal
An Analysis of Food Accessibility of Mountain Cities in China: A Case Study of Chongqing
Next Article in Special Issue
A Personalized Learning Service Compatible with Moodle E-Learning Management System
Previous Article in Journal
Strength, Flexibility and Postural Control of the Trunk and Lower Body in Participants with and without Patellofemoral Pain
Previous Article in Special Issue
A Collaborative Framework for Customized E-Learning Services by Analytic Hierarchy Processing
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Educational AI Chatbots for Content and Language Integrated Learning

Kleopatra Mageira
Dimitra Pittou
Andreas Papasalouros
Konstantinos Kotis
Paraskevi Zangogianni
1 and
Athanasios Daradoumis
Department of Cultural Technology and Communication, University of the Aegean, 81100 Lesvos, Greece
Department of Mathematics, University of the Aegean, 83100 Samos, Greece
Author to whom correspondence should be addressed.
Appl. Sci. 2022, 12(7), 3239;
Submission received: 19 February 2022 / Revised: 15 March 2022 / Accepted: 21 March 2022 / Published: 22 March 2022
(This article belongs to the Collection The Application and Development of E-learning)


Using advanced artificial intelligence (AI) technology in learning environments is one of the latest challenges for educators and education policymakers. Conversational AI brings new possibilities for alternative and innovative Information and Communication Technologies (ICT) tools, such as AI chatbots. This paper reports on field experiments with an AI chatbot and provides insights into its contribution to Content and Language Integrated Learning (CLIL). More specifically, this paper presents an experimental use case of an educational AI chatbot called AsasaraBot, designed to teach high school students cultural content in a foreign language, i.e., English or French. The content is related to the Minoan Civilization, emphasizing the characteristic figurine of the Minoan Snake Goddess. The related chatbot-based educational program has been evaluated at public and private language schools in Greece. The findings from these experiments show that the use of AI chatbot technology for interactive ICT-based learning is suitable for learning foreign languages and cultural content at the same time. The AsasaraBot AI chatbot has been designed and implemented in the context of a postgraduate project using open-source and free software.

1. Introduction

In schools in developed countries, the traditional tutor-centered approach in which students are in a constant effort to keep up, retain information, and cope with the rapid pace of the modern educational process still dominates. However, such an approach cannot adequately address the challenges of the digital era, which requires special knowledge and skills from students and tutors to exploit the capabilities offered by Information and Communication Technologies (ICT). Therefore, tutors should use modern ICT tools to improve, and possibly optimize, the learning experience of their students.
In this context, the use of trending technologies has a new, significant direction potential. An important emerging educational technology is based on conversational AI, implemented as so-called AI chatbots. AI chatbots are intelligent systems/applications that are able to interact with humans in various aspects of daily life using natural language (NL). They can be found mainly as personal assistants in business customer support. In the educational context, AI chatbots can play the role of intelligent tutors by presenting educational material, stimulating dialogue, providing feedback to students, etc. In some cases, AI chatbots can also play a complementary/supportive role to human tutors by answering students’ questions and providing guidance in a 24/7 timespan, something that is evidently impossible or unprofitable to implement with human tutors.
This paper reports on recent research related to the design, implementation, and evaluation of an educational AI chatbot, aiming to support the modern methodological learning approach of Content and Language Integrated Learning (CLIL) [1], i.e., the simultaneous learning of both content (e.g., a cultural subject) and foreign languages (e.g., English and French). Specifically, in a CLIL approach, students are taught a specific subject in a foreign language instead of their mother tongue. Activities based on the CLIL method are mainly designed to allow language and content of a particular subject to be learned at the same time. More specifically, in this paper, a real use case of an educational AI chatbot called AsasaraBot is presented, which complements human tutors in teaching cultural content related to the subject of the Minoan Civilization, emphasizing the characteristic figurine of the Minoan Snake Goddess. The related chatbot-based educational program was evaluated in English and French as a foreign language at a public Lyceum and in two private language schools in Greece. Findings from the related experiments show that the use of AI chatbot technology for interactive ICT-based learning can effectively support the CLIL method for learning a foreign language and cultural subject at the same time. The AsasaraBot has been designed and implemented using free software, i.e., the Snatchbot platform.
The contribution of the work presented in this paper is the following: it introduces AI Chatbot technology to support the modern learning method of CLIL, mainly playing the role of an ICT-based tutor and secondarily the role of an ICT-based assistant of a human tutor. The proposed approach has been implemented and evaluated in English and French as foreign languages for Greek students, reporting on the effectiveness and potential of the approach. To this end, an educational program has been specifically designed and implemented according to the needs of distance ICT-based learning, due to the COVID-19 pandemic.
The structure of this paper is as follows: Section 2 provides background knowledge regarding the utilized technology, i.e., AI chatbots. Section 3 describes the proposed approach, and Section 4 presents its evaluation in a real setting, i.e., in foreign language classes. Section 5 provides a discussion of the research issues and challenges of the chatbot-based educational approach. Finally, Section 6 concludes the paper.

2. Preliminaries and Related Work

Conversational AI refers to AI techniques (e.g., natural language processing, machine learning, knowledge graphs) for engineering intelligent software agents that can be engaged in natural language conversations with humans or other agents. It includes software, such as voice assistants and chatbots. The first generation of this technology focused on short task-oriented dialogue [2], such as music playback (e.g., “Alexa, play music”) or information retrieval (e.g., “Cortana, what is the weather today in Athens?”). The current challenge in this area is a constant, coherent, and appealing dialogue, as current software is still far from being able to have natural everyday conversations with humans [3]. The chat approach is designed in such a way that it gives human users the impression that they are naturally chatting with another human.
Chatbots can play an important role in the field of education because they are an interactive mechanism compared to traditional e-learning systems [4]. Among other ICTs, chatbots are considered safe and accessible learning tools that can bring positive results in learning. One of the reasons that chatbots are popular in various businesses is the increased efficiency of the process. This argument is also used in education as it increases user–student satisfaction by speeding up response times and can be available 24 h a day to answer or clarify any question [5]. This allows teachers, for example, to avoid answering repetitive questions that can be easily answered through chatbots, while also supporting students who have missed one or more lessons [6].
In addition to simply providing the question–answer function or exchanging information between students and the system, a chatbot contributes to tackling the problem of supporting/managing several individual student cases. During a lesson, the teacher usually does not have enough time for formative feedback to each individual student [5]. However, formative feedback during learning processes is, according to Hattie [7], one of the most important factors in increasing students’ performance and motivation. An AI chatbot has the ability to assist each student individually, following different learning paces, to provide them with knowledge according to their individual cognitive level.
In addition, chatbots seem to allow students to receive individual support in a bug-friendly environment, giving them the opportunity to apply their cognitive skills anytime and anywhere. This reduces stress and increases the willingness to learn [8]. Furthermore, a chatbot can be considered an excellent data analysis tool. Shawar and Atwell [9] state that it is important to remember that the teacher is the backbone of the teaching process, and that learning technology can act as an enhancer rather than a substitute. For example, when the chatbot is used to answer students’ questions, the teacher can use the conversation logs and statistics reporting to see where the students are having problems and what their weaknesses are. Thus, the teacher can use this technology to look for problems, while students use it to solve them.
Despite the aforementioned advantages, there are still communication problems between chatbots and humans. For example, limitations are found in the ability to correctly recognize natural language (especially in languages beyond English, such as Greek), difficulties in the flow of dialogue [10], inability to control repetitive sentences, and difficulty in dealing with unknown sentences [11]. Another disadvantage is that chatbots are often unable to answer complex or unexpected questions, as a result of which, they misinterpret what the student says and produce inappropriate dialogue flows [12]. This can result in student frustration, who will then seek other means of learning. Finally, chatbots are considered predictable, redundant, and have a bad memory, as they find it difficult to memorize previous conversations, forcing users to ask the same question repeatedly [13]. This may upset the user due to the greater effort required.
The increasing use of AI chatbot technology has influenced the way modern ICT-based teaching and learning is designed and implemented. Freudbot (, last accessed 15 January 2022) is an example of a chatbot developed for the educational purposes of psychology students, and it is used as a teaching and learning tool in distance and online education. Through Freudbot, students are taught about the theories, concepts, and biographies of the famous psychologist Freud [14]. Another example is Ethnobot (, last accessed 15 January 2022), which plays the role of a human ethnographer who collects ethnographic data and asks users a series of conversational questions [15]. Chatbots have been developed for learning foreign languages, such as Cleverbot (, last accessed 15 January 2022) and CSIEC (, last accessed 15 January 2022) [16,17]. Autotutor [18] is a multidisciplinary chatbot aimed at encouraging learning through conversation with learners. Chatbot [19] is an educational chatbot aimed at promoting the study of computer science. StudyBuddy and SmarterChild [20] are examples of chatbots used in non-formal learning. The NDLtutor [21] has been developed to improve student involvement and reflection. Confucius [22] is used as a supplement in the classroom. The Xbot chatbot [23] aims to encourage high school students to improve their mathematical, programming, and logic skills.
A few chatbots have been created and designed for language learning. CLIVE is an intelligent chatbot for conversational language practice capable of language switching among any of the world’s major languages [24]. Gengobot is a chatbot-based grammar dictionary application for Japanese language learning that gives explanations and meanings in Japanese, Indonesian, and English [25]. The BookBuddy system [26] consists of three sub-chatbots assisting children in selecting the appropriate book for their English language level, supporting reading comprehension by answering vocabulary questions, conversing with them, and delivering individualized feedback. Mondly and Andy English Bot offer several possible responses so that learners can choose the most appropriate one [27]. The CSIEC system generates communicative responses according to user input, dialogue context, user and personality knowledge, common-sense knowledge, and inference knowledge [16].
Haristani [17] points out that related work on development and research on teaching and learning languages other than English is still in its infancy. Huang et al. [28], in their systematic review of chatbot-supported language learning, noted that little research evaluated chatbots compared to other equivalent tools and suggested research involving education levels other than university. AssassaraBot was built from the ground up to help students meet their content and language learning objectives in two languages: English and French. In this research, teaching through AssassaraBot was evaluated compared to a teacher teaching with other ICT tools in secondary education.

3. The Proposed Approach

Chatbots focus on dialogue/conversation, aiming for agent interaction that follows patterns similar to those followed by humans. Through conversation, chatbots must be able to analyze the context and suggest solutions to problems, interpret emotions, and act accordingly or contribute to the learning process [6]. Based on this aim, we have designed and developed an educational chatbot with the purpose of teaching secondary school students (ages between 12–18 years old) the history (cultural content) of the Goddess of Snakes, and at the same time, to teach them a foreign language. A chatbot named AsasaraBot is introduced to the students as the Goddess of Snakes, acting as a tutor who is playing a theatrical role to better engage his/her students. AsasaraBot is teaching without the intervention of a human teacher. Through the whole learning experience of students chatting with the AsasaraBot, the conversation is enriched with proactive verbal techniques that could be used by a teacher to address the student in a personalized and encouraging way. Examples of messages in this conversation form are:
  • Questions facilitate learning in multiple ways. They motivate students to inquire and reflect and cultivate their cognitive and linguistic abilities. In this sense, AsasaraBot addresses certain questions to advance the flow of intercourse (e.g., “What do you see in this picture?” or “Qu’est-ce que tu vois sur la photo?”).
  • Praise and encouragement support knowledge assimilation by motivating students, empowering their self-confidence, and strengthening their efforts toward learning. The AsasaraBot was designed to encourage and motivate with remarks such as, “Oh, why not?”/“Oh, pourquoi pas?” or “Super! You are really good!”/“Superbe! Τu as très bien compris le texte!”.
AsasaraBot is an original educational teaching software that implements the basic idea of learning cultural content and foreign languages (CLIL approach) in an interactive educational learning environment. This software is mainly based on the exchange of messages between a student and a virtual tutor–chatbot, which in our case takes the form (plays the role) of the Goddess of Snakes of the Minoan Civilization. From this role comes the name Asasara, which was chosen for the chatbot, a name that is encountered in inscriptions written in the Linear A script and often seems to be used for the Minoan Goddess of Snakes. AsasaraBot is deployed via different channels (Facebook Messenger, Viber) and an educational web site (, last accessed 15 February 2022), aiming to guide students through dialogue to discover and learn cultural content that is available in two languages, English and French.
The purpose of AsasaraBot determines the functional requirements of the software, i.e., what functionality the software must provide, and how the system will behave while interacting with the users (students). Specifically, AsasaraBot meets the following main functional requirements:
  • The user (student) communicates with AsasaraBot via guided and free (thus, hybrid) chat.
  • The user is able to select the flow of the dialogue through available options (buttons, voicemail, etc.).
  • The chatbot is available (online) through at least three messaging platforms (Webchat, Messenger, Viber).
  • The chatbot has a start and restart button.
  • The system provides the user with a simple and understandable dialogue flow.
  • The system understands the natural language (English or French) of the user.
  • The system informs the user in cases of misunderstanding of the message to take appropriate actions, e.g., repeat the message.
  • The information provided to the user comes from reliable sources, e.g., Wikipedia.
  • The system does not violate the privacy protection policy regarding the recording of data.
  • Each user’s chat history is securely and anonymously stored in the system database.
  • The input and output messages are exchanged via text or audio format.
  • The content of the conversation combines text, image, video, and audio.

3.1. Design of Asasarabot

AsasaraBot is designed using the general architectural structure of a hybrid chatbot model (guided and free-text communication). Therefore, it includes the following components (as depicted in Figure 1): a natural language understanding engine, a dialog manager, and the information retrieval components along with a knowledge base.
  • Natural Language Understanding (NLU) engine: The user’s message goes through the component of the NLU engine, where, through customized NLU models, the chatbot “uncovers” the meaning (intent) of the user’s message and provides the appropriate response.
  • Dialog Manager: This component is responsible for handling/managing all the messages exchanged (questions/requests, answers/responses) in the form of text or audio messages. It is essentially the middleware of the chatbot connecting the NLU engine and the information retrieval components.
  • Information Retrieval and Knowledge Base: In this component, the chatbot searches for the most appropriate response stored in the knowledge base (or externally in the Web) and sends the retrieved related information to the Dialog Manager. Basically, all the information and knowledge that constitutes the ‘brain’ of the chatbot is stored and handled here.
In Figure 1, the general architectural design of the implemented system is depicted.

3.2. Development of Asasarabot

AsasaraBot has been developed on Snatchbot (, accessed on 15 January 2022), a cloud-based platform that provides technological infrastructure for the design, development, and testing of chatbots, without requiring knowledge of software coding. Such a selection of platforms, among other platforms examined (e.g., DialogFlow, Rasa, OpenDialog), was made mainly due to its simplicity and codeless interface, as well as its functionality related to the design requirements of our approach. Thus, although a few limitations of the platform were identified (see later section), the advantages offered were determinant.
The knowledge base (KB) for AsasaraBot was built using a template-driven graphical interface of the platform. For each question designed in the dialogue, predefined answers were manually inserted into the KB. The KB is not automatically updated with new knowledge, since the chatbot does not have the ability to learn by itself. Such an extension/update of the KB must be performed manually by the developers who monitor the dialogue and collect the intents of users from the log files. Furthermore, since the interaction of the users with the chatbot is made anonymous, personalized customization of the dialogue is not supported. However, the customization of the dialogue (and consequently the improvement of interactions) can be performed at the class level by analyzing the interactions of students as a whole (e.g., how many understood the question or not).
During the development, the logic proposed by the Snatchbot platform for the creation of configurable components to be synthesized for the development of AsasaraBot was followed. Thus, effort was put to distinguish the necessary functionality and create components that would meet the requirements of each one of them. As a result, the main part of the chatbot application includes the following components:

3.2.1. Interactions

In the terminology of Snatchbot, interactions are the basic elements that define the behavior of a chatbot. The platform supports 13 types of interactions. The most basic type of interaction, that is, the basic building block of AsasaraBot’s structure, is where the chatbot makes a statement (bot statement) in plain text, and interacts with the user’s response. A total of 52 interactions were developed in AsasaraBot, defined according to the needs of the dialogue and the requirements of the educational scenario. Reformulations of these interactions were inserted between the basic interactions in cases where the user’s message was not understood by AsasaraBot, encouraging the user to try again (e.g., repeat the question).

3.2.2. Connections

The Snatchbot platform provides pre-trained natural language processing (NLP) models that are ready to be used at any time by the developers. The pre-trained “Detect Persons” NLP model already included in the Snatchbot platform was used, and additional custom models were developed from scratch in both languages (English and French), focusing on intents and entities that do not exist in pre-trained models. In particular, new models were developed to detect common phrases that users use when they are unable to answer a question, and models to detect negative and positive human reactions. For instance, the following custom entity-based NLP models have been developed to a) detect general positive/negative emotions, e.g., boredom, happiness, excitement, and b) detect disappointment or excitement from AsasaraBot performance (e.g., “you are not so helpful to me”, “I do not want to talk to you any more”, “you are so bad”). The need for developing new custom models appears when developers want to focus on knowledge (intents or entities) that does not exist in the pre-trained models. In custom models, the developer can manually insert knowledge (words or phrases) or automatically insert large numbers of samples at a time using the “Bulk insert” feature. Within the chatbot paradigm, the term “intent” refers to the goal (intention) the user has in mind when making/typing a question (what he/she really means), while the term “entity” refers to the modifier used by the user to describe an issue/concept. SnatchBot uses emoji and text analysis (NLU) to understand (interpret) emotions. NLP (models) is what allows Snatchbot chatbots to understand the meaning of a user’s statement (context in the model) and act accordingly. NLP models use machine-learning algorithms for training, i.e., to learn how to recognize the intent of users and related entities.

3.2.3. Chat Bubbles

This feature was primarily used to make the conversation more engaging. In particular, it allows the creator to split the dialogue into different bubbles so that a large amount of information is not displayed at once. Using the Instant Response or Delayed Response functions, the creator can specify how long the message bubble will be displayed to the user, offering a more natural conversation experience that mimics human conversation.

3.2.4. Text-to-Speech

Text-to-Speech functionality provides the option to listen to chatbot messages (instead or in addition to written ones). For each interaction selected to use the Text-to-Speech function, an audio file was generated per language.

3.2.5. Prior Responses

The bot’s dialogue reuses responses it has received earlier from the user who is chatting with it. For example, in the very first interaction with a student, AsasaraBot asks his/her name, and uses the response (the responded name) later in the conversation to personalize the interaction.

3.2.6. Quick Replies

This add-on feature offers users simple and guided ways to respond to messages. It has the form of ready-made buttons, where the user can tap as a response to a message. After the selection of a quick reply by a user, all other buttons are hidden, preventing users from tapping on buttons associated with previous interactions.

3.2.7. Custom Variables

Custom variables allow the collection of data/information from user interactions, especially for the execution of numerical functions during the conversation. For example, they were used to collect correct/wrong answers (as positive or negative points) to calculate the final test score for each user.

3.2.8. Arithmetic Operation

This functionality is responsible for performing numerical operations and was used to add or subtract points during the conversation and calculate the final test score. In particular, when a student answers a question correctly or incorrectly, a point is added or subtracted, respectively.

3.2.9. Embedding Media

This feature allows the integration of images, videos, audio, and files in the dialogue. These attachments can either be embedded directly by entering the URL in the chatbot message or via the Rich Cards option. This option is a set of cards, each of which can contain any combination of text, images, and buttons. Through this, it is possible to integrate video, video, and audio into the conversation for the needs of the educational scenario.

3.2.10. Connecting Channels

The AsasaraBot was initially attached to the official website of the proposed educational program. This process was performed simply by copying and embedding special code (provided by the Snatchbot platform) on a webpage. On the website, AsasaraBot appears as a pop-up window. A prerequisite for the integration of AsasaraBot in the Facebook Messenger channel was the creation of a Facebook page. Thus, two pages were developed to host the two versions, English and French, of AsasaraBot. Finally, AsasaraBot has also been deployed from a Viber channel.

3.3. The Dialogue

During the development of the AsasaraBot, a group of people was involved in the continuous testing of the overall process. In particular, these tests were performed by people with software engineering knowledge, by researchers familiar with chatbot technology (Conversational AI experts), by language teachers (Language Education Experts/Teachers), and by ordinary users. These tests significantly contributed to the elimination of bugs that occurred in the early stages of AsasaraBot development.
In addition, efforts were made to determine if AsasaraBot worked smoothly on the three messaging channels: Facebook Messenger, Viber, and the Webchat environment. A few problems were identified when switching platforms, for instance, the lack of STT (Speech-to-Text) functionality on Messenger and Viber platforms, a feature that is only provided in Webchat, the TTS (Text-to-Speech) functionality in Viber, where the users are required to download the voice message, and problems with the importing of external files. To overcome these problems, a number of actions have been tested, such as the use of Rich Cards, making the conversation more user-friendly and visually richer on all three messaging platforms. Finally, language teachers helped to improve the content of the dialogue with comments on detecting syntax and grammar errors, poor vocabulary, spelling mistakes, etc. in English and French dialogues.
After an extended number of experiments and evaluation cycles, the final flow of the dialogue implemented in AsasaraBot has been shaped in the form depicted in Figure 2. As can be seen, the dialogue is composed of a number of questions to be answered, and activities to be completed by the students. The specific flow is based on the general pedagogical aspects of educational programs that have been researched prior to the development of this dialogue.
The activities and questions implemented in the dialogue are depicted in Figure 3, Figure 4, Figure 5 and Figure 6.

3.4. The Platform

Based on the hands-on experience during the development and testing of the AsasaraBot, a number of advantages of the SnatchBot platform have been identified:
  • It supports a large number of online messaging platforms, such as Webchat, Twitter, Facebook Messenger, Skype, WhatsApp, Viber, and Slack, while new ones are constantly being added.
  • It facilitates the collection and analysis of the generated interaction data and provides statistics for each chatbot implementation.
  • It supports natural language understanding (NLU), making a chatbot quite sophisticated.
  • It supports text-to-speech (TTS) and speech-to-text (STT) conversion of messages.
  • It provides a plethora of free chatbot templates that can be used to support developers at their early developing steps.
On the other hand, at the time of our experimentation with the platform, there were some issues and limitations depending on the communication channels chosen for interacting with AsasaraBot. In a few cases, the user was not allowed to send an audio message in the chatbot. Additionally, embedding files such as importing a video, or an audio clip did not work properly, e.g., as occurred while working with the Facebook Messenger channel.

4. AsasaraBot Evaluation

4.1. Experimental Educational Program for Cultural Content and English/French Language Learning

The experiments were conducted online (due to COVID-19 restrictions) on the 16 and 20 of December 2020 through the WebEx and Zoom platforms. The participating classes were a) English and French language classes of the 1st and 2nd grade of the 1st School of Excellence General Lyceum of Mytilene, Greece, and b) two private foreign language schools (French and English, respectively) also in Mytilene, Greece.

4.1.1. Participants

The selected students varied in terms of educational level, language level, gender, and age. A total of 61 students participated in the experiments. From those, 35 were selected from among the English and French language classes of the Lyceum (age 15 to 17 years old), 18 of them were students of English as a foreign language, and 7 were students of French as foreign language. Another group of 26 participants were selected from private foreign language schools. Of these, 10 students attended the school of English as a foreign language, while 16 attended the school of French as a foreign language. Two teaching methods were selected for the experiments: the proposed chatbot-based approach, and another with other ICT-based learning tools. In Table 1, the identity of the experiments is summarized and organized per school, class, language, language level, and teaching method.

4.1.2. Evaluation Method

Two different evaluation methods were used in the experiments: chatbot-based learning and another ICT-based learning approach. The latter was developed using Google Forms, Padlet, and YouTube videos. Padlet (, accessed 15 January 2022) is a digital tool through which one or more “walls” can be created that can accommodate posts for sharing, including content such as videos, pictures, documents, audio, diagrams, and blueprints. It has a collaborative character, allowing for the collaborative engagement of students and teachers. This tool was used to construct a short diagram as an introduction to the educational program and, in particular, to the Minoan Civilization. As an introductory activity for the other-ICT-based learning method, a small diagram is designed using Padlet to answer two questions on the subject of Minoan Civilization, so that the educational process can start with a related discussion. The same questions were used in the chatbot, but this time they were placed by AsasaraBot.
A detailed task-based experimental setting was designed. A total of 35 students of the Lyceum were randomly assigned to two groups: 25 females and 10 males. An experimental group, consisting of 18 students, was involved in a learning scenario using the AsasaraBot learning method, while the members of the control group of 17 students were involved in the other-ICT-based learning method. Both methods involved the same educational content.
A pre-test and a post-test questionnaire-based approach were conducted to assess the content and foreign language knowledge gain of corresponding participants. The 26 students of the two private language schools also participated in the chatbot-based learning method. Overall, 44 students participated in the Chatbot evaluation group, that is, 18 students in the experimental group and another 26 students that used the Asasarabot without participating in the experiment (non-experimental group), as illustrated in Figure 7. All students in the chatbot evaluation group were given a questionnaire in order to evaluate their experience interacting with the chatbot. The students in the Lyceum were given the evaluation questionnaire after taking the post-test. Figure 7 illustrates the flow of the evaluation procedure.

4.1.3. Pre-Test

A pre-test (in the form of a questionnaire) was given to the students before the start of the educational program. This test consisted of six closed-ended question items (see Appendix A). Items A, B, and C assessed the student’s prior knowledge of the cultural content being examined, while question items D, E, and F assessed the knowledge of vocabulary in English and French. The items and the provided answers were collected via Google forms (due to privacy and language issues, they are available upon request).

4.1.4. Post-Test

A post-test (in the form of a questionnaire) was given to the students at the end of the educational program. This test consisted of a total of 11 closed-ended questions organized into two sections, similar to the pre-test. That is, content knowledge, as well as language learning were assessed, in alignment with the pre-test. The aim of this is to test the effectiveness of the educational program, which was taught with the two methods, i.e., with the chatbot-based and other-ICT-based methods. The items and the provided answers were collected via Google forms (due to privacy and language issues, they are available upon request).

4.1.5. AsasaraBot Evaluation Questionnaire

Finally, another questionnaire was given to the students who followed the chatbot-based educational program (see Chatbot evaluation Group in Figure 3) to evaluate the usability of the AsasaraBot environment, as well as the overall experience gained from the educational process. This questionnaire consisted of 18 questions and was structured into three sections. In the first section, information about the profile of the students was collected. In the second section, the questions aimed to evaluate students’ learning experiences in relation to the educational content of AsasaraBot, while the third section included questions for the evaluation of the students’ experiences in relation to the chatbot-as-software. The answers were collected using mainly a five-point Likert scale, ranging from strong agreement to strong disagreement. The items and the provided answers were collected via Google forms (due to privacy and language issues, they are available upon request).

4.2. Learner Experience Evaluation Results

While most participants were familiar with new ICT technologies, 23% (10 out of 44 students) reported that they had interacted with a chatbot in the past. Regarding their experience with AsasaraBot in terms of cultural content, most of the student participants reported that they had a positive impression. That is, 91% of the participants agreed that a student can learn cultural content with the assistance of a chatbot. On the other hand, only 48% of them were positive about using a chatbot for learning a foreign language. The majority of the participants (93%) reported that the conversation with the chatbot was engaging. Regarding their experience with AsasaraBot in terms of usability, 98% of the students reported that it was user-friendly, 54% of the students reported that it did not require any prerequisite knowledge for its use, and 91% of the students reported that the design and development of the dialogue was simple and understandable. These results were also confirmed by an interview that followed (qualitative analysis) at the end of the learning process, where the majority of participants stated that it was an interesting experience, that they liked the AsasaraBot environment quite a lot, and that it was enjoyable as a means for educational purposes. In fact, for many students this experience was unprecedented, and the use of chatbots in the context of teaching was an experience they wanted to have again in future courses.

4.3. Cultural Content and Language Knowledge Gain Evaluation

As mentioned earlier, the members of the experimental and control groups were given a pre-test and a post-test in order to assess knowledge gain when interacting with the AsasaraBot, compared to another technology-based educational environment. The experimental group had a mean performance (mean) of 3.61 and a standard deviation (SD) of 1.61 for N = 18, while the control group had a mean performance of 3.41, with a standard deviation of 1.8 for N = 17.
The Shapiro–Wilk test showed that the performance of both groups in the pre-test did not follow the normal distribution (W = 0.92633, p < 0.05). A non-parametric analysis based on the Kruskal–Wallis test (H(2) = 0.10244, p = 0.7489) showed no statistically significant difference between the performance of the two groups in the pre-test (a = 0.05).
The difference in the performance of the members of both groups in the post- and pre-tests was calculated. More specifically, the overall performance difference was calculated as well as the performance difference pertaining to both cultural content and language content knowledge gain. The results are shown in Table 2.
According to the Kruskal–Wallis non-parametric test, no statistically significant differences were found for the overall performance gain of the two groups (H(2) = 0.10344, p = 0.7477), the performance gain on cultural content learning (H(2) = 0.22496, p = 0.6353), or the gain in language learning (H(2) = 0.25127, p = 0.6162). Furthermore, no significant difference was found between the performance gains of male and female members in the experimental and control groups (H(2) = 1.7185, p = 0.1899).
Concerning descriptive statistics, the gain in performance appeared to be greater for the AssasaraBot group for cultural content learning, while the control group performed better in the question items for language learning. While this finding is not conclusive, it aligns with the previous finding from the evaluation questionnaire, where relatively few participants had a positive attitude toward using the chatbot for language learning, compared to cultural content learning.

5. Discussion

Based on the data collected from the evaluation of the presented approach, we discuss a number of issues that highlight our concerns in the domain of ICT-based learning and in the CLIL approach.
Initially, the use of chatbot technology, as part of the conversational AI paradigm, can enhance traditional teaching in many ways. A student has the opportunity to work at his or her own time, without the fear of failure and error, through a friendly environment, outside the classroom. Participants in the evaluated training program stated that the discussion with AsasaraBot was made in a language they understood in a quite interesting manner, and that they would like to use a similar type of training method again in the near future.
At the same time, a chatbot can be a useful tool for teachers by providing them with assistance during distance-learning courses. Due to the difficult situation of the COVID-19 pandemic, distance-learning courses were applied all over the world at different educational levels. One of the main goals of chatbot-based ICT methods is not to replace human teaching but to provide a support tool/method in the existing/traditional educational process. Based on the data collected from our experiments, it would be rather early to argue that chatbot-based ICT can fully replace traditional teaching, as there are still technical difficulties and obstacles, such as restrictions on natural language understanding, difficulties in the human-like flow of dialogue, and familiarization of teaching staff and students with such new technology.
Regarding the integrated/unified learning of foreign language and (cultural) content using a chatbot, the result is rather positive, although it is possible that the formulation of a small number of questions or the short time frame for completing the questionnaires have slightly affected the result. This argument is based on the student data collected, with a high percentage of correct answers. As the students also argued, the chatbot itself positively contributes to their educational process. In addition, the evaluation shows that in both languages, the data is positive, which proves that adapting the material to different foreign languages aims at similar learning outcomes. In the future, it would also be interesting to study the extent to which the level of language learning (e.g., basic or intermediate) contributes to the educational process, and how chatbot technology can contribute to the learning of language and cultural content at each of these levels.

6. Conclusions

This paper presented experiments with an AI chatbot that provides insights into the contribution of this technology (conversational AI) to the learning of cultural content and a foreign language at the same time (content and language integrated learning). The developed educational AI chatbot, called AsasaraBot, supports human tutors in teaching cultural content related to the Minoan Civilization and the figurne of the Minoan Snake Goddess. The related chatbot-based educational program has been evaluated in English and French at public and private schools in Greece, with encouraging preliminary findings. The use of AI chatbot technology for interactive ICT-based learning can efficiently support the learning of foreign languages and cultural content at the same time. Future plans for this research line include the use and evaluation of this technology as an assistive technology for human teaching, delivering part of a course curriculum with this method. The goal is to identify a gold balance between chatbot-based and other ICT or non-ICT learning methods for individual courses or for the entire teaching curriculum of a school. A future study with a greater number of participants will allow us to draw more conclusive results concerning the pedagogic effectiveness of chatbot-based teaching methodology.

Author Contributions

Conceptualization, K.K. and A.P.; methodology, K.M.; software, D.P.; validation, K.K., K.M. and D.P.; formal analysis, A.P.; investigation, K.M.; resources, P.Z.; data curation, D.P.; writing—original draft preparation, K.M.; writing—review and editing, K.K., A.P., A.D.; visualization, P.Z.; supervision, K.K.; project administration, K.K.; funding acquisition, K.K. All authors have read and agreed to the published version of the manuscript.


This research received no external funding. The APC was funded by the University of the Aegean-Research Unit, Departement of Cultural Technology and Communication, Postgraduate Studies Program.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

Table A1. Question Items.
Table A1. Question Items.
French Language LearningPre-Experimental
Post-Experimental Questionnaire
ITEM A: To check if the students know and, in the process, learned the Greek interpretation of the word “exquis”.10. Read the following sentence in French: “la Crète reserve vis ses visiteurs des plaistirs gastronomiques exquis”. The word “exquis” is interpreted as: (a) excellent, (b) important, (c) strange (Correct Answer: a).11a. The Greek interpretation of the word “exquis” is “exquisite”. Is this sentence correct or wrong? (Correct Answer: Proposal is correct).
ITEM B: To check if the students know the English definition of the word “Figurine” and in the process if they learned its Greek interpretation.13. “Small sculpture in round bosom of an inferior tail à 25 cm”. Which of the following words does the sentence refer to? (a) statue, (b) figurine, (c) busts (Correct Answer: b).11c. The Greek interpretation of the word “figurine” is “statuette”. Is this sentence right or wrong? (Correct Answer: Proposal is correct).
ITEM C: To check if the students know and understand the phrase “doit son nom” and in the process if they finally learned the alternative interpretation of the verb in French “doivent”.15. “Ils doivent leur nom…” Which of the following expressions could be replaced? (a) They have given their name, (b) Their name comes from, (c) ils ont refuse their name (Correct Answer: d).11f. The Greek interpretation of the phrase “doivent leur nom…” is “they got their name…”. Is this sentence right or wrong? (Correct Answer: Proposal is correct).
Learning Cultural ContentPre-Experimental
Post-Experimental questionnaire
ITEM D: To test their knowledge about the name of the archaeologist who discovered the archaeological site of Knossos, and if they learned it after the training program. 5. “The archaeologist who discovered their palace at Knossos was Henry Schliemann.” Do you think this is: (a) True, (b) False (Correct Answer: Wrong)?6. “The archaeologist who discovered the palace of Knossos was Arthur Evans”. Do you consider this proposal to be: (a) True, (b) False (Correct Answer: Correct)?
ITEM E: To test their knowledge about the Goddess of Snakes, and if they learned about her after the end of the training program. 6. Do you know the Goddess of Snakes?
(a) Yes, (b) No, (c) I’m not sure
(Correct Answer: a or b or c).
7. Do you think that after their participation in the educational program, you know about the Goddess of Snakes? (a) Oh yes, (b) No, (c) I’m not sure (Correct Answer: a or b or c).
ITEM F: Students to evaluate their knowledge of the Goddess of Snakes before and after their participation in the educational program.7. If so, on a scale of 1 to 5, how would you rate their knowledge? 1 = Incomplete, 2 = Moderate, 3 = Relatively good, 4 = Very good, 5 = Excellent (Correct Answer: 1 or 2 or 3 or…). 8. If yes, based on the following scale do you consider that your knowledge is: 1 = Incomplete, 2 = Moderate, 3 = Relatively good, 4 = Very good, 5 = Excellent (Correct Answer: 1 or 2 or 3 or…).
English Language LearningPre-experimental
Post-Experimental Questionnaire
ITEM A: To check if the students know and, in the process, learned the Greek interpretation of the word “exquisite”.10. Read the following sentence in English: “Minoan art is characterized by an exquisite naturalness”. The word “exquisite” is interpreted as: (a) exquisite, (b) important, (c) effortless (Correct Answer: a).11a. Their Greek interpretation of the word “exquisite” is “exquisite” Is this sentence right or wrong? (Correct Answer: Proposal is correct).
ITEM B: To check if the students know the English definition of the word “Figurine”, and, in the process, if they learned its Greek interpretation.13. “Male figurines are usually decorated with red paint, have a dagger at the waist, and wear the characteristic loincloth”. Which of the following words would you use to replace the word “figurines”? (a) colossuses, (b) puppets, (c) statuettes (Correct Answer: c).11c. Their Greek interpretation of the word “exquisite” is “exquisite”. Is this sentence right or wrong? (Correct Answer: Proposal is correct).
ITEM C: To check if the students understand their use of passive voice in the sentence, and then if they have learned the alternative interpretation of the verb “call” in English.15. “They were called…” Which of the following expressions could be replaced? (A) They called someone on the phone, (b) Their name is, (c) They named someone (Correct Answer: b).11f. Their Greek interpretation of the phrase “they were called…” is “they were named”. Is this sentence right or wrong? (Correct Answer: Proposal is correct).


  1. British Counsil, “CLIL,” Teaching English. Available online: (accessed on 15 February 2022).
  2. Ram, S.; Gabriel, A.; Cheng, R.; Wartick, M.; Prasad, A.; Liu, R.; Nagar, Q.; Pan, A.; Hwang, Y.; Khatri, G.; et al. Conversational AI: The science behind the Alexa prize. arXiv 2018, arXiv:1801.03604. [Google Scholar]
  3. Levesque, H.J. Common Sense, the Turing Test, and the Quest for Real AI; MIT Press Ltd: Cambridge, MA, USA, 2017. [Google Scholar]
  4. Kiptonui, B.P. Chatbot technology: A possible means of unlocking student potential to learn how to learn. Educ. Res. 2013, 4, 218–221. [Google Scholar]
  5. Winkler, R.; Söllner, M. Unleashing the Potential of Chatbots in Education: A State-Of-The-Art Analysis. In Proceedings of the Academy of Management Annual Meeting (AOM), Chicago, IL, USA, 20 August 2018. [Google Scholar]
  6. Garcia, N.; Brustenga, G.; Alpiste, F. Castells, Briefing Paper: Chatbots in Education; Universitat Oberta de Catalunya: Barcelona, Spain, 2018. [Google Scholar]
  7. Hattie, J. Visible Learning for Teachers: Maximizing Impact on Learning; Routledge: London, UK, 2012. [Google Scholar]
  8. Ayedoun, E.; Hayashi, Y.; Seta, K. A Conversational Agent to Encourage Willingness to Communicate in the Context of English as a Foreign Language. Procedia Comput. Sci. 2015, 60, 1433–1442. [Google Scholar] [CrossRef] [Green Version]
  9. Shawar, B.A.; Atwell, E. Different measurements metrics to evaluate a chatbot system. In Proceedings of the Workshop on Bridging the Gap: Academic and Industrial Research in Dialog Technologies, Rocheester, NY, USA, 26 April 2007; pp. 89–96. [Google Scholar]
  10. Leonhardt, M.; Tarouco, L.M.R.; Vicari, R.; Santos, E.R.; Da Silva, M.D.S. Using Chatbots for Network Management Training through Problem-based Oriented Education. In Proceedings of the Seventh IEEE International Conference on Advanced Learning Technologies, Nigata, Japan, 18–20 July 2007; Institute of Electrical and Electronics Engineers (IEEE): Piscataway, NJ, USA, 2007; pp. 845–847. [Google Scholar]
  11. Neves, A.M.M.; Barros, F.A.; Hodges, C. IAIML: A mechanism to treat intentionality in AIML chatterbots. In Proceedings of the International Conference on Tools with Artificial Intelligence, ICTAI, Washington, DC, USA, 13–15 November 2006; pp. 225–231. [Google Scholar]
  12. Agrafiotou. Conversational Agents in Education: Using MentorChat To Support Students’ Online Dialogue, Mediated by a Partner-Agent; Aristotle University of Thessaloniki: Thessaloniki, Greece, 2012. [Google Scholar]
  13. Roos, R.; Lochan, R. Chatbots in Education: A Passing Trend or a Valuable Pedagogical Tool? Master’s Thesis, Uppsala University, Uppsala, Sweden, 2018. [Google Scholar]
  14. Heller, B.; Procter, M.; Jewell, L.M. Freudbot: An Investigation of Chatbot Technology in Distance Education Evaluation of the Regional Psychiatric Centre’s Fetal Alcohol Spectrum Disorder (FASD) Pilot Project View Project Assessment of High-School Students’ Experiences with and Perceptions; Association for the Advancement of Computing in Education (AACE): Montreal, Canada, 2005. [Google Scholar]
  15. Smutny, P.; Schreiberova, P. Chatbots for learning: A review of educational chatbots for the Facebook Messenger. Comput. Educ. 2020, 151, 103862. [Google Scholar] [CrossRef]
  16. Jia, J. CSIEC: A computer assisted English learning chatbot based on textual knowledge and reasoning. Knowl.-Based Syst. 2009, 22, 249–255. [Google Scholar] [CrossRef]
  17. Haristiani, N. Artificial Intelligence (AI) Chatbot as Language Learning Medium: An inquiry. J. Phys. Conf. Ser. 2019, 1387, 012020. [Google Scholar] [CrossRef]
  18. Graesser, A.; Chipman, P.; Haynes, B.; Olney, A. AutoTutor: An Intelligent Tutoring System With Mixed-Initiative Dialogue. IEEE Trans. Educ. 2005, 48, 612–618. [Google Scholar] [CrossRef]
  19. Benotti, L.; Martinez, M.C.; Schapachnik, F. A Tool for Introducing Computer Science with Automatic Formative Assessment. IEEE Trans. Learn. Technol. 2017, 11, 179–192. [Google Scholar] [CrossRef]
  20. Molnar, G.; Szuts, Z. The Role of Chatbots in Formal Education. In Proceedings of the 2018 IEEE 16th International Symposium on Intelligent Systems and Informatics (SISY), Subotica, Serbia, 13–15 September 2018; pp. 197–202. [Google Scholar] [CrossRef]
  21. Suleman, R.M.; Mizoguchi, R.; Ikeda, M. A New Perspective of Negotiation-Based Dialog to Enhance Metacognitive Skills in the Context of Open Learner Models. Int. J. Artif. Intell. Educ. 2016, 26, 1069–1115. [Google Scholar] [CrossRef]
  22. Hsieh, S.W. Effects of cognitive styles on an MSN virtual learning companion system as an adjunct to classroom instructions. Educ. Technol. Soc. 2011, 14, 161–174. [Google Scholar]
  23. Auccahuasi, W.; Santiago, G.B.; Núñez, E.O.; Sernaque, F. Interactive online tool as an instrument for learning mathematics through programming techniques, aimed at high school students. In Proceedings of the ICIT 2018: 6th International Conference on Information Technology: IoT and Smart City, New York, NY, USA, 29–31 December 2019; pp. 70–76. [Google Scholar]
  24. Zakos, J.; Capper, L. CLIVE—An Artificially Intelligent Chat Robot for Conversational Language Practice. In SETN 2008: Artificial Intelligence: Theories, Models and Applications; Springer: Berlin/Heidelberg, Germany, 2008; pp. 437–442. [Google Scholar]
  25. Haristiani, N.; Danuwijaya, A.A.; Rifai, M.M.; Sarila, H. Gengobot: A chatbot-based grammar application on mobile instant messaging as language learning medium. J. Eng. Sci. Technol. 2019, 14, 3158–3173. [Google Scholar]
  26. Ruan, S.; Willis, A.; Xu, Q.; Davis, G.M.; Jiang, L.; Brunskill, E.; Landay, J.A. Bookbuddy: Turning digital materials into interactive foreign language lessons through a voice chatbot. In Proceedings of the 6th 2019 ACM Conference on Learning at Scale, L@S 2019, Chicago, IL, USA, 24–25 June 2019. [Google Scholar]
  27. Lee, J.H.; Yang, H.; Shin, D.; Kim, H. Chatbots. ELT J. 2020, 74, 338–344. [Google Scholar] [CrossRef]
  28. Huang, W.; Hew, K.F.; Fryer, L.K. Chatbots for language learning—Are they really useful? A systematic review of chatbot-supported language learning. J. Comput. Assist. Learn. 2021, 38, 237–257. [Google Scholar] [CrossRef]
Figure 1. General architectural design of AsasaraBot.
Figure 1. General architectural design of AsasaraBot.
Applsci 12 03239 g001
Figure 2. AsasaraBot implemented dialogue flow (activities and questions).
Figure 2. AsasaraBot implemented dialogue flow (activities and questions).
Applsci 12 03239 g002
Figure 3. Brainstorming task as the starting point of the dialog.
Figure 3. Brainstorming task as the starting point of the dialog.
Applsci 12 03239 g003
Figure 4. Activity 1: Watch a video.
Figure 4. Activity 1: Watch a video.
Applsci 12 03239 g004
Figure 5. Activity 2: Read text.
Figure 5. Activity 2: Read text.
Applsci 12 03239 g005
Figure 6. Activity 3: Listening.
Figure 6. Activity 3: Listening.
Applsci 12 03239 g006
Figure 7. The flow of the evaluation procedure.
Figure 7. The flow of the evaluation procedure.
Applsci 12 03239 g007
Table 1. Experiments’ organization.
Table 1. Experiments’ organization.
DepartmentLanguageLanguage LevelTeaching MethodNumber of Participants
Lyceum (Class B)EnglishBasic (A1–A2), Intermediate (B1–B2), Proficiency (C1–C2)Other ICT-based Learning9
Chatbot-Based Learning9
Lyceum (Class A)FrenchBasic (A1–A2), Intermediate (B1–B2), Proficiency (C1–C2)Other ICT-based Learning8
Chatbot-Based Learning9
English Private Foreign Language School EnglishIntermediate (B1–B2), Proficiency (C1–C2)Chatbot-Based Learning10
French Private Foreign Language SchoolFrenchIntermediate (B1–B2), Proficiency (C1–C2)Chatbot-Based Learning16
Table 2. Experimental results for learning gain.
Table 2. Experimental results for learning gain.
OverallCultural ContentLanguage
AsasaraBot181.441.250.778 1.000.6670.907
Control171.351.660.529 0.9430.8241.07
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Mageira, K.; Pittou, D.; Papasalouros, A.; Kotis, K.; Zangogianni, P.; Daradoumis, A. Educational AI Chatbots for Content and Language Integrated Learning. Appl. Sci. 2022, 12, 3239.

AMA Style

Mageira K, Pittou D, Papasalouros A, Kotis K, Zangogianni P, Daradoumis A. Educational AI Chatbots for Content and Language Integrated Learning. Applied Sciences. 2022; 12(7):3239.

Chicago/Turabian Style

Mageira, Kleopatra, Dimitra Pittou, Andreas Papasalouros, Konstantinos Kotis, Paraskevi Zangogianni, and Athanasios Daradoumis. 2022. "Educational AI Chatbots for Content and Language Integrated Learning" Applied Sciences 12, no. 7: 3239.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop