Gulf Countries’ Citizens’ Acceptance of COVID-19 Vaccines—A Machine Learning Approach

Abstract

The COVID-19 pandemic created a global emergency in many sectors. The spread of the disease can be subdued through timely vaccination. The COVID-19 vaccination process in various countries is ongoing and is slowing down due to multiple factors. Many studies on European countries and the USA have been conducted and have highlighted the public’s concern that over-vaccination results in slowing the vaccination rate. Similarly, we analyzed a collection of data from the gulf countries’ citizens’ COVID-19 vaccine-related discourse shared on social media websites, mainly via Twitter. The people’s feedback regarding different types of vaccines needs to be considered to increase the vaccination process. In this paper, the concerns of Gulf countries’ people are highlighted to lessen the vaccine hesitancy. The proposed approach emphasizes the Gulf region-specific concerns related to COVID-19 vaccination accurately using machine learning (ML)-based methods. The collected data were filtered and tokenized to analyze the sentiments extracted using three different methods: Ratio, TextBlob, and VADER methods. The sentiment-scored data were classified into positive and negative tweeted data using a proposed LSTM method. Subsequently, to obtain more confidence in classification, the in-depth features from the proposed LSTM were extracted and given to four different ML classifiers. The ratio, TextBlob, and VADER sentiment scores were separately provided to LSTM and four machine learning classifiers. The VADER sentiment scores had the best classification results using fine-KNN and Ensemble boost with 94.01% classification accuracy. Given the improved accuracy, the proposed scheme is robust and confident in classifying and determining sentiments in Twitter discourse.

1. Introduction

The COVID-19 pandemic created an unprecedented global health emergency. Over the last two years, almost 248 million people have been diagnosed COVID-19 and it has lead to more than 5 million deaths [1]. In an attempt to prevent the spread of COVID-19, countries were forced to enforce several preventive measures in accordance with recommendations from international and local health organizations. These include social distancing, working remotely, wearing face masks, etc. However, remote access and work from home increased social media use, where people shared their opinions worldwide using social media websites [2]. Tweets about COVID-19 and its vaccination have remained top-trending throughout the previous two years (2020 and 2021) [3]. In addition to implementing restrictions upon COVID-19 spread, governments and pharmaceutical companies dedicated their resources to researching and discovering medication-based solutions [4]. Although the medical domain has been achieving significant improvement day by day in discovering medical treatments [5], hesitancy about approved vaccines exists in the world’s population; this is not only about COVID-19 vaccines—vaccine hesitancy has existed for some decades [6]. Vaccine hesitancy is a term used when vaccines are available but people still refuse to accept them [7,8]. There are certain factors that lead to this hesitancy, such as social [9], cultural [10], psychological [11], and scientific reasons. The actual success of discovered vaccines depends upon the willingness of people to accept them. Therefore, the reasons why people are not getting vaccinated at the desired rate need to be explored and investigated by scholars and researchers from different fields, with the hope of coming to conclusions and finding implications that can subsequently improve the positive perception of the public. Due to its prevalence and inherent features, the data, and especially linguistic data, from the Twitter platform were generally used by many studies to analyze people’s perceptions regarding different vaccines [12,13,14,15]. Such data contain both useful and misleading information [16] regarding different people from different countries, and and also entail real-time open opinions and attitudes from various countries. Moreover, Twitter has 166 million active users who interact using the platform on a daily basis, and this activity has increased during the COVID-19 pandemic [17,18]. Machine learning tools and techniques have been used to analyze positive, negative, and neutral opinions on discourse features obtained from the platform. The Gulf countries were also heavily affected by the COVID-19 pandemic, and have seen speedy vaccination efforts relative to the rest of the world. The Gulf countries have also seen rumors and uncertainty causing vaccination hesitancy. Data from “Our World in Data” regarding vaccination rates in the Gulf countries up to 29 December 2021 are shown in Figure 1.

According to the above data, the population of the United Arab Emirates is getting vaccinated at a relatively higher rate compared to other Gulf countries, where 91% of people are fully vaccinated and 8.3% are partially vaccinated from a total vaccinated population of 99%. Among the Gulf countries, Oman is the least vaccinated [19]. These data show that these countries have varying vaccination rates despite having the same religious beliefs. The Saudi Arabia data of vaccinated people up to 3 November show it is the most vaccinated country among all other Gulf countries. However, the rates of getting vaccinated changed throughout the whole year. The rate of getting vaccinated highly depends upon people’s vaccination hesitancy factors. Therefore, in order to analyze the Gulf countries’ populations’ views regarding different vaccines, a sentiment analysis was performed on Twitter data. In this study, we used Twitter tweet data related to COVID-19 vaccines. Our study makes the following contributions:

• The proposed study highlights the concerns of Gulf countries’ populations regarding COVID-19 vaccines;
• the positive and negative thoughts of text data are encoded and classified using intelligent machine learning methods;
• the vaccine hesitancy of Gulf countries’ populations could be reduced by analyzing and responding to people’s concerns, ultimately contributing to an increase in COVID-19 vaccination rates.
• we propose a method to improve the accuracy of currently used sentiment analysis tools.
  The rest of the article is divided into an overview of related work, a description of the proposed method, the presentation of our results, a discussion, and lastly the conclusions.

2. Related Work

Machine learning techniques and methods are used for the sentiment analysis of Twitter discourse regarding various topics. Similarly, COVID-19 vaccine-related data for different categories and countries could be analyzed. Region-specific data for various types of vaccines have already been examined to check for factors resulting in people’s hesitancy in various regions and nations. Indian citizens’ attitudes towards COVID-19 vaccination were analyzed using Twitter data analysis in two studies [20]. The authors found that people’s views changed as time passed, whereas in another study, they identified major issues regarding people’s vaccine hesitancy. Using their gathered social media data, they found that 47% of people have neutral perceptions about vaccines and that 17% have negative views and suggested ways to solve the issues of vaccination hesitancy. An Indonesian tweet sentiment analysis regarding COVID-19 vaccines was performed in another study [21]. “Vaccine COVID-19” keyword-based tweet data were scraped and collected for one month only. The naïve Bayesian model was used for probability-based classification into positive, negative, and neutral tweets. A manual method of data collection did not suffice, whereas a rapid miner tool used to collect data using the Twitter developer account proved to be key. Finally, using more than 6000 tweets, the authors found that 56% were positive, 39% were negative, and only 1% were neutral.

Global opinions regarding three of the most prominently used vaccines were analyzed in a study using the natural language processing (NLP) and supervised K-nearest neighbor methods. It classified tweets into three classes, namely positive, negative, and neutral perspectives of people. The Pfizer vaccine had the highest positive value (47.29%), the Moderna vaccine had the highest negative value, 40.71%, and the highest neutral value was shown by Pfizer, at 15.21% [22]. A similar study on the same three vaccines was conducted using four months’ data in the English language, to analyze people’s hesitancy towards vaccine acceptance. More than 0.7 million tweets were used for this analysis. A sentiment analysis was performed using a lexicon-based tool, and the authors claimed that sentiment analysis using this tool is relatively easy. The authors found that the positivity regarding the AstraZeneca vaccine was decreasing day by day. The highlighted concerns regarding vaccines could be used to develop strategies to fight vaccine hesitancy [23]. A region-based analysis using four months’ data from the USA and Brazil was carried out in another study. English and Portuguese language-based tweets were used in this study. It used more than 3.3 million English and 3.1 million Portuguese tweets. It raised ten topics of discussion using data gathered from government, private, and open discussions in these countries [24]. The authors in [25] used Reddit platform data from 13 communities. They used six months’ data from December 2020 to May 2021, and showed that these platforms remained statically positive over the examined time-span. The study reported that these communities’ discussions were mainly about vaccines’ side effects. Latent Dirichlet allocation (LDA) topic modeling was used where vaccine hesitancy-based keywords were discussed in this analysis. The authors of [26] used 1.2 million tweets over five weeks. The forecasting model in their study reported that the USA will be fully vaccinated by the end of July 2021. Positive views were more common compared to negative views. English tweets (more than 2.6 million) were taken and then classified into three types of tweets (positive, negative, and neutral). Further sub-types found using topic modeling were discussed, including information, administration, real life, etc. The classified tweets were 42.8% positive, 26.9% neutral, and 30.3% negative [27].

To see the cross-culture and continent-based vaccine perspective difference among people, social media data from the USA and China were used [28]. Twitter data were used for the collection of USA data and Weibo was used to collect Chinese vaccine data. A semantic network of tweets sentiments was proposed where discussions showed that the Weibo-based data showed positive sentiments towards vaccination while Twitter data from the USA showed an anti-vaccine perspective regarding different vaccines. Theme-based quantification regarding COVID-19 vaccines was performed that showed the highlighted discussion of official health authorities, ingredients, and different clinical findings of different vaccines [29]. The study claims that with the passage of time, the opposition regarding vaccines on the Twitter platform was increased by 80% which created serious concerns. Another theme identification study used four nations’ data to analyze and compare their perspectives [30]. It highlighted the topic modeling of India, South Korea, Japan, and the United Kingdom. The topic modeling showed that education, economy, and sports sectors are disturbed. The validation score of 90% from the dataset suggests that the United kingdom has most of the negative sentiments towards vaccination.

NLP is a computer science field in which text analysis is performed. In previous years, deep learning (DL), ML, and NLP have shown a breakthrough via intelligent methods [31]. NLP- and ML-based medical reviews have also been used to check the public behavior of people. In a recent survey [32], ML- and NLP-based medical studies were reviewed and showed the impact of mental illness using text analysis. Sometimes, the use of NLP gave magical outputs by giving sentiments of unexpected and tough text analysis. It uses paraphrase detection, POS tagging, and many other tasks [33]. Online stores’ feedback reviews on various products could be analyzed and studied by business investors to focus on churn analysis and to promote businesses by using people’s interest [34]. This could all be carried out using NLP- and ML-based intelligent analysis.

Most of the recent studies on COVID-19 vaccine data discussed above classified data into three classes, namely, positive, negative, and neutral. Other factors, such as misinformation that includes sarcasm, anecdotal stories, and other negativity, spread information from various celebrities and famous people on the internet [35], which creates a negative hype about vaccines. However, only a few studies generated sub-topics, which are necessary to diagnose the reasons for vaccine hesitancy. We also found that studies used their own scraped data with millions of tweets within a range of 4 to 6 months’ data only. In many studies, English tweets were mostly taken into consideration, with a specific focus on the USA. We found no region-based analysis of the Gulf countries with a focus on Arabic countries’ tweets. This research gap needs to be addressed in order to solve these nations’ vaccine hesitancy.

3. Materials and Methods

Many of the recent studies have explored different real-world problems using deep learning and machine learning methods [36,37,38]. The natural language processing used in many aspects of the field, such as COVID-19 vaccine tweets’ sentiment scores based on deep and machine learning classification methods, is utilized in this study. The basic steps include data collection, their cleaning, and tokenization. After tokenizing the document, the data are discussed in terms of hot topics and trends in Gulf countries. The word-embedding method-based encoded data are used in the proposed architecture of long short-term memory (LSTM), where the 3 different sentiment score datasets are fed into the same LSTM network that includes 3 networks and then deep features are used by different classical machine learning classification methods. These machine learning classifiers enhance the classification in terms of accuracy, F1-score, and other measures. Step-wise discussion is discussed in a later section, and the primary steps are shown in Figure 2.

3.1. Data Collection and Cleaning

Data were collected using the public repository Kaggle, which updates data on a regular basis. The data were collected using Twitter tweet data with #Pfizer, #Sinopharm, #Sinovac, #Moderna, #Oxford/AstraZeneca, #COVAX, #SputnikV and #COVID-Vaccine tags, where data contain the user ID, name, description and location of users, the date of user creation, their tweets, followers, friends, favorites, and whether the user is verified or not. In our study, we only examined Gulf countries’ data to check how residents of these countries expressed their views and discussed various COVID-19 vaccines. The data were collected and then refined by filtering tweets from the countries Bahrain, Kuwait, Oman, UAE, Saudi Arabia, and Qatar. These tweets were then considered for further discussion and analysis in our study. The data were tokenized and filtered out by removing punctuation, repeated words, and URLs. The refined filtered data could be represented as shown in Equation (1).

$${TD}_i=D_i\left(F\right|C_i)$$

(1)

In Equation (1), ${TD}_i$ represents the Twitter data documents and i represents the Gulf countries, numbered 1 to 6. $D_i$ are the tweeted data documents of certain filtered data $\left(F\right|C_i)$ of 6 Gulf countries where F represents filtered data of Gulf country $C_i$. The finalized data were refined and tokenized, yielding 685 tweets, and then further used in sentiment analysis.

3.1.1. Ratio Sentiment Scores

The ratio score analysis works with ratios of positive and negative scores of given documents. In this method, the VADER lexicon is used. However, the ratio is calculated as the value of the positive score to the negative score. If the value is greater than 1, then it gives 1 to that particular document sentiment score. On the other hand, if the negative to positive score ratio is greater than 1, then it returns $-1$ to that particular document. The sentiment lexicon used in this method is based upon VADER contextual lexicon features where other user-specific or defined lexicons could also be used in order to make the lexicon. The ratio score data of this study could be represented as in Equation (2).

$${CS}_1=Ratio\left(S_{{TD}_i}\right)$$

(2)

In Equation (2), the sentiment score data using the ratio score method are shown as compound scores of the 1st method (${CS}_1)$ where the sentiment scores $S_{{TD}_i}$ represent the tweeted data of a particular Gulf country that are filtered with respect to 6 countries.

The word cloud of positive and negative sentiments for all six countries is shown in Figure 3.

In Figure 3, we can see that “COVID-19” and “vaccine” are the most dominant words used in both positive and negative contexts. Similarly, Moderna is a vaccine name that is commonly used in all Gulf countries with both negative and positive aspects, but it is used more in negative feedback. Sinopharm is the Chinese vaccine, and it is used mostly in negative contexts as can be seen in the negative word cloud, where it is more dominant as compared to the positive sentiments on the left side. It is also seen that the word “doses” is used mostly in negative sentiments after “COVID”, “Moderna”, and “vaccine”. However, let us now examine the data of the top tweets to check the validity of these tweet texts.

3.1.2. TextBlob Sentiment Scores

TextBlob is the Python library that uses NLTK and pattern methods to give sentiments to a given text. NLTK is the Natural Language Processing Toolkit that provides various methods of lexicon analysis, tokenization, stemming, semantic meanings, and other classification libraries. It provides powerful methods regarding computational linguistics and natural language processing. Pattern is another type that comprises the TextBlob library, and it provides functional support for natural language, web mining, and many more tasks such as classification, text extraction, etc. However, these libraries make TextBlob a more powerful library for work in computational linguistics. TextBlob-based sentiment scores range from −1 to 1 where a negative number shows negatively scored sentiments, 0 is neutral, and scores above this are considered to be positively scored texts. The tokenized text in this study is used with this method and the sentiment scores could be represented as shown in Equation (3):

$${CS}_2=TextBlob\left(S_{{TD}_i}\right)$$

(3)

In Equation (3), the sentiment score data using the TextBlob method are shown as compound scores of the 2nd method (${CS}_2)$ where the sentiment scores $S_{{TD}_i}$ represent the tweeted data of a particular Gulf country that is filtered with respect to 6 countries.

3.1.3. Valence-Aware Dictionary and Sentiment Reasoning (VADER) Sentiment Scores

The VADER method is generally based upon rule-based score execution. It makes a list of lexical features using combinations of both qualitative and quantitative measures. It basically uses a compound score (CS) that thresholds the negative, positive, and neutral text data. A tweet would be negative if CS $\le -0.05$. Other than this threshold value, the text data remain neutral with a 0 score and positive when >0. The VADER score data of this study could be represented as in Equation (4):

$${CS}_3=VADER\left(S_{{TD}_i}\right)$$

(4)

In Equation (4), the sentiment score data using the VADER method are shown as compound scores of the 3rd method (${CS}_3)$ where the sentiment scores $S_{{TD}_i}$ represent the tweeted data of a particular Gulf country that is filtered with respect to 6 countries.

In our study, we compared eleven different state-of-the-art sentiment analyzers that use ML-based approaches to calculate their sentiment scores. They use entropy, probability, and other support vector machine-type methods to aggregate the sentiment scores. However, the VADER method first uses a lexicon that expresses the contextual intensity of the given text data. Therefore, to utilize the contextually aware lexicon methods, this study also used it for sentiment analysis. To check for promising results and to check the validity of the VADER method, we also used another method of sentiment analysis that is discussed below.

3.2. Gulf Tweet Analysis

The tweet data from the Gulf countries were filtered and a small number of data types were considered in order to analyze the validity of tweet accounts and their tweeted texts. First, as can be seen in

Table 1. Data of Gulf countries’ tweet data about different COVID-19 vaccines.

Countries	Data	User Followers	User Friends	User Favorites	Retweets	Favorites
	Count	35.00	35.00	35.00	35.00	35.00
Bahrain	Mean	3834.9142	408.171429	2569.5714	0.2571	1.6857
	Standard Deviation	9264.6748	354.6651	3688.1505	0.6572	4.0712
	Count	49.00	49.00	49.00	49.00	49.00
Kuwait	Mean	12,991.6938	427.5510	2120.7346	0.9387	5.000
	Standard Deviation	12,657.5279	499.8954	6238.9526	1.3602	6.9251
	Count	21.00	21.00	21.00	21.00	21.00
Oman	Mean	43,965.1904	436.1904	3044.1428	0.6666	3.9047
	Standard Deviation	47,286.9812	325.1511	4527.7635	1.2382	3.7269
	Count	269.00	269.00	269.00	269.00	269.00
Qatar	Mean	44,200.2825	238.6022	1245.3531	0.6728	2.6468
	Standard Deviation	22,251.9688	145.8434	3647.5030	1.2506	2.6524
	Count	55.00	55.00	55.00	55.00	55.00
Saudi Arabia	Mean	269,870.3636	97.8727	1187.80	4.2545	14.6545
	Standard Deviation	150,890.9840	292.777	5422.63	3.7870	22.1820
	Count	35.00	35.00	35.00	35.00	35.00
UAE	Mean	11,694.542	507.1714	1433.2571	0.7428	2.0285
	Standard Deviation	18,937.479	705.4559	4945.575	2.2405	3.8462

, Bahrain had a value of 35 in user followers, user friends, user favorites, retweets, and favorites. This means that a total of 35 tweets were taken after filtering to all data with “vaccine” hashtags. The mean of user followers was 3834, which means that the used Twitter accounts have thousands of followers. However, when we look at the standard deviation value of followers, it shows a very large deviation value, 9264, which indicates that some of the used Twitter accounts have thousands of followers while some of them do not. There was a slight difference between the user friend mean value and the standard deviation value. This allows us to estimate that the majority of followers have about 408 friends, and that some people that are tweeting have similar friends, as there is not a large standard deviation. The retweet and favorite tweet values show that people did not encouragingly like the tweets or may not have been active users.

For Kuwait, the value of tweets was 49. The mean and standard deviation were both high. The mean value of 12,991 indicates that all 49 tweeting accounts have more than 10k followers, whereas the standard deviation value of 12,657 shows that some of the users do not have many followers. However, if we look at the user friends then it is the same situation as discussed for Bahrain. The friend mean value of 427 with a standard deviation of 499 indicates that the users have fewer friends. However, the retweet and favorites mean and standard deviation values again lead to the surprising insight that the followers are not interacting with the tweets in the form of favorites or retweets.

Oman had only 21 tweets that were filtered out from the given Twitter data. The mean and standard deviation were 43,965 and 47,286, respectively, for user followers in the Oman data. By looking into these figures, an average of more than 43k user followers can be seen, which mostly consists of Twitter accounts with thousands of followers, while the standard deviation is also high. Therefore, there was a big difference between 21 tweeting accounts. For user friends, the mean and standard deviation were 436 and 325, respectively. This indicates that most of the tweeting accounts had about 400 friends, with the standard deviation also indicating a large variance in numbers of friends. Retweets and favorites again had lower mean and standard deviation values, which indicates that these texts were not encouraged or liked by Twitter users.

In Qatar, there were 269 user-based tweets, with mean and standard deviation values of user followers of 44,200 and 22,251, respectively. This indicates that, on average, these tweeting accounts had more than 44k followers, whereas the standard deviation value also indicates a big difference, though, in comparison to the deviation values discussed above, it is not very high. Therefore, we can say that these accounts have more followers or are more popular relative to the above countries. Similarly, these accounts’ friends had a standard deviation value of 145, which indicates a small difference between the accounts’ friends. The favorite mean and standard deviation were only 2. This indicates again that the users were not very active in their responses to these popular pages.

Saudi Arabia had 55 tweets in total. Surprisingly, the mean and standard deviation were 269,870 and 150,890, respectively, which were the biggest values found in our study. This indicates that most of these tweeting accounts have 0.2 million followers, whereas the standard deviation value indicates that the tweeting accounts have only 0.15 million followers. However, there was an average of 14 favorites per tweet with 22 as the standard deviation. The slight variation in standard deviation and mean values could be due to followers’ variation, as well as old and new tweeting accounts. However, the favorite mean value of 14 is interesting, as it highlights that the users are at least responding and active.

The UAE had 35 tweets in total. It had a mean value of followers of 11,694 with 18,937 as the standard deviation value. This means that the UAE has a large amount of variation in tweeting accounts followers. The user friend mean and standard deviation values of 507 and 705, respectively, also highlight the big difference between tweeting accounts’ friends. Finally, again, the retweet and favorite mean and standard deviation were 2 and 3, respectively. This also highlights the inactivity and lower responsiveness to all the accounts’ tweets about COVID-19 vaccines.

By analyzing the data of all six countries’ tweets, we see that Qatar had the most tweets, with 269 tweets on the vaccine topic, while its users were not very active even though they had thousands of Twitter followers. However, Saudi Arabia had the largest number of popular tweeting accounts with the largest number of favorites and retweeted texts. However, for an account-based comparison and discussion, we sorted the top tweets for each of these countries and will now discuss their tweeted topics, which will highlight the country-wise interests and feedback regarding different vaccines.

Topic Discussion

Various topics were discussed in each of the Gulf countries, from which various top retweeted topics are highlighted and discussed in

Table 2. Analyzed top topics in Gulf countries regarding COVID-19 vaccines.

Country	Topic Number	Source	Topic Title	Description
Bahrain	T1	Bh News	Saudi Arabia to accept Sinopharm vaccine	Saudi Arabia to accept Sinopharm vaccinated individuals 14 days after receiving a booster dose of Pfizer, Moderna
	T2	Gulf Daily News	Sinopharm acceptance in Bahrain	Authorities have once again affirmed the efficacy of the Chinese vaccine Sinopharm
	T3	Bh News	Booster shot	Bahrain to conduct study on booster shots for fully vaccinated individuals
Kuwait	T1	Arab Times Kuwait	Ministry of Health allowance	MOH does not recognize vaccines taken outside Kuwait
	T2	Dr. Fatima M. Khajah	COVID-19 vaccine consultation	Part of interview about COVID-19 vaccine
	T3	Arab Times Kuwait	Vaccine acceptance	Kuwait expands sites for vaccination, Sinopharm, Sinovac, and Sputnik V vaccinated passengers allowed to enter
Oman	T1	Muscat Daily	MOH postponed 2nd dose of vaccine	Ministry of Health: Postponement of second dose of Pfizer/BioNTech vaccine for those who received the first shot
	T2	Times of Oman	Oman emergency use of Russian and Chinese Vaccines	Oman approves emergency use of Russian, Chinese vaccines
	T3	Times of Oman	Effect of use of different Vaccines	A study has found that alternating doses of the AstraZeneca and Pfizer BioNTech vaccines generates a robust immune response
Qatar	T1	Reunite couples stuck apart in the pandemic	COVAX allowance	“Please, EVERYONE retweet this to support us to get the COVAX approved in all countries”
	T2	Reunite couples stuck apart in the pandemic	Qatar opened flights for vaccinated	Qatar opened to vaccinated visitors from India from July, many people were able to meet their families
	T3	The Peninsula Qatar	Booster shot	Regulators and vaccine developers are looking at whether booster doses are necessary or not
Saudi Arabia	T1	Arab News	Sinopharm acceptance	The UAE will start providing Sinopharm COVID-19 vaccine to children aged 3–17
	T2	Arab News	Side effects of AstraZeneca	Saudi study on AstraZeneca vaccine against COVID-19 says no major side effects were observed
	T3	Saudi Gazette	Moderna acceptance for younger people	Moderna reports that its coronavirus vaccine is effective in those aged between 12 and 17
UAE	T1	WAM English	Sinopharm acceptance for younger people	UAE Ministry of Health approves use of Sinopharm vaccine for ages 3–17
	T2	Nesreen Bakheit	2nd dose approved in UAE	Dubai has approved its 2nd Pfizer vaccine, which will roll out to the public
	T3	Mira Salem	UAE collaboration against COVID-19	The UAE is making global efforts to fight the COVID-19 pandemic

. The tweeted topic-based model could be represented as shown in Equation (5).

$${Topics}_{C_i}=T_j\left(M_k\right)$$

(5)

In Equation (5), the selected and discussed topics are represented as ${Topics}_{C_i}$ where $C_i$ represents the country-wise topics and the i range is from 1 to 6. $T_j$ are the top three discussed topics of each country ($C_i$) where the topics are selected based upon the majority of their retweets ($M_k$), where k varies from country to country.

The retweeted topics from Bahrain were regarding the Sinopharm vaccine, which was about to be approved in Saudi Arabia. This means that people from Bahrain were also interested in the approval of the Chinese vaccine. In Saudi Arabia, a major topic was that people were not allowed to come into Saudi Arabia without their approved vaccines. Another discussed topic in Saudi Arabia was the acceptance of the Chinese vaccine with a booster vaccine from Pfizer or Moderna. Additional top topics also had to do with Sinopharm acceptance. We found that acceptance of the Chinese vaccine was a major issue in all Gulf countries. To further check this finding, we examined the other countries’ retweeted topics. As can be seen in Table 2, a top topic in Kuwait was also the Ministry of Health’s (MOH) allowance of Chinese vaccines.

One more interesting topic that we identified in the Oman tweets is the discussion of the vaccine mix-and-match strategy to enhance immunity against COVID-19. It also seems that Oman has allowed Chinese and Russian vaccines due to the health emergency. If we look at Qatar, all tweets are encouraging and requesting the government to allow the COVAX shots and booster shots to enter the country. People from Asian countries are waiting to re-enter Gulf countries after receiving their vaccinations in their countries. Saudi Arabia considered using the AstraZeneca vaccine to vaccinate its population. However, people aged 12 to 17 were to be vaccinated with Moderna, another vaccine. The side effects of these vaccines are also discussed, which could affect people’s decisions to receive their vaccine shots as well. It is also discussed that the UAE will allow administering the Sinopharm vaccine to its younger nationals. In general, discussion in the UAE revolved around the Sinopharm and Pfizer vaccines, which were allowed by the government.

3.3. Topic Analysis Using Machine Learning

Machine learning methods of analysis are developing and are also used in text data analysis. Features are encoded using various methods that are also used in this study. The feature engineering at various stages is discussed in the coming sections.

3.3.1. Tokenizing

The document texts were taken for each country and then tokenized into different tokens. The Unicode Standard Annex (UAX) [39] was used for text boundary and Unicode-based text segmentation. This standard is used for text segmentation when there are many rules regarding regex and grammatical punctuation. Sentences, words, paragraphs, and other breaking rules are defined in this method, which is used to extract the tokens of documents. Our study used this method to tokenize the tweeted texts and documents.

3.3.2. Data Cleaning

The tokenized documents were cleaned after obtaining each individual text word or character. The stop words, URLs, punctuation, repeating words, and numeric figures were removed and all of the words were shifted to lowercase letters. After cleaning all of the data, the individual pieces of data were unique and distinguished enough to obtain the encodings of their texts. Additionally, this data cleaning led to the need for less computing space and effort in feature computing, as we removed the redundancy and useless information before making features. It also led to more promising results of the sentiment scores.

3.3.3. Tokenized Encodings

The tokenized and cleaned filtered data contain the most useful and unique words, which were used to obtain the actual vocabulary or dictionary of vaccine-related Gulf countries’ data. These encodings were collected out of all of the refined tokenized documents. Finally, with the use of these data, a 1 × 1773 string vector was designed made up of the tweeted data vocabulary. This method provided the indices of a particular word or string that is read out while a document is iterated over these encodings. In this way, a true or false, or 0/1-based, feature vector or sequence could be designed to obtain 2 numeric features for texts.

3.3.4. Document Sequential Data

Sequential data were made to design input words embedded in a layer for further machine learning-based analysis. The cleaned tokenized data-based encoding data of document sequencing were generated, and equal-length sequences were generated. These sequences contained sparsity-type matrices for each input instance or document. The data were split up in a 70/30% ratio. There were 685 instances in total, and after splitting, there were 480 instances for training and 205 instances for testing.

3.3.5. Proposed LSTM

The proposed LSTM contained a nine-layer architecture. All layers are summarized in

Table 3. Proposed LSTM architecture and activations.

Serial Number	Type	Layers	Weights
1	Input	Sequence input	1
2	Words embedding	Word embedding	20 × 1774
3	LSTM	LSTM_1	400 × 20
4	Dropout	Dropout_1	100
5	LSTM	LSTM_2	200 × 20
6	Dropout	Dropout_2	50
7	Fully connected	FC_1	2 × 50
8	Activation	Softmax	2
9	Classification	Classification_Output	2

. The first layer takes the input of sequences of data that were designed using an encoding vector. The sparsity matrix-based sequences are given to the next layer for word embedding, which basically obtains each input sequence as a multiple of the encoding of the designed vocabulary. These weights are the inputs of the next LSTM layer, which repeats the sequence until the weights are updated for their next input. This is called a long short-term memory network, which is based upon a recurrent neural network. It contains long- and short-term memory based upon its recurrence and updated weights.

The next layer is a dropout layer; it is generally used to drop out useless information that is read out by the LSTM layer. It is designed to obtain the probability at first, and in the proposed architecture, 0.5 is taken as the probability value to drop 20%. The next layer of the LSTM has a 20% probability of reducing data; as can be seen, 400 reduced to 200 is the next layer input with the sequence length. However, the next layer of data is taken as an LSTM layer and again the recurrence-based input sequences are iterated by this layer. The dropout is again applied in the next step, which again reduces data weights by 20% by setting 0.5 as the probability value. The refined encoded and distinguished data are passed on to the fully connected layer that is aligned to the vector weights and then fed to the activation-based softmax layer. It designs the probability-type scores that are used by the classification layer to classify the data into negative and positive sentiment scores.

3.3.6. Deep Feature Extraction and Classification

After obtaining the deep learning-based classifiable model, our study used deep features that are currently consistently used in many other machine learning-based applications [37,40,41,42]. The features are extracted using a fully connected layer, where these deep features are used as the machine learning classifier SVM to classify data as positive and negative tweets and data. The extracted features are used to normalize the data. The data are first normalized using a mean and its standard deviation, which produces data in the range of 1 to $-1$. Then, a chi-squared test is performed to obtain rank features where the top 15 are selected for each instance and then fed to the SVM.

4. Results and Discussion

In our basic workflow, we have used sentiment score methods in order to sort our data into negative and positive feedback about COVID-19 vaccines in the Gulf countries. The ratio and VADER sentiment score data were fed into two models, a proposed LSTM and other machine learning classifiers, which returned different prediction results in both sentiment analysis methods. We will now discuss them individually and make the results of this study more robust.

4.1. Ratio Sentiment Score-Based LSTM and Machine Learning Classifier Results

The first main LSTM-based network was designed and applied to sentiment score-based sequential data. The training parameters are shown in

Table 4. Training hyper-parameters of LSTM network.

Parameters	Values
Activation Function	Stochastic Gradient Descent Moment (SGDM)
Batch Size	32
Training Environment	GPU
Gradient Threshold	1
Max Epochs	5000

. It includes the information regarding activation function, training epochs, batch size, and training environment.

It can be seen in

Table 5. Machine learning-based tweet classification results using the ratio sentiment score method.

Methods	Accuracy	Sensitivity	Precision	F1-Score	Cohen’s kappa
LSTM	77.56	91.13	76.35	83.09	50.51
SVM	77.56	91.13	76.35	83.09	50.51
Fine-KNN	93.28	97.35	92.03	94.61	85.72
Ensemble Boost	93.28	97.35	92.03	94.61	85.72
Total Boost	91.24	97.35	89.18	93.09	81.20

that there are various categories of different domains of classification that were applied for prediction; a visual representation is shown in Figure 4.

These sequential data have shown 77.56% accuracy on negative and positive tweet data. However, sensitivity and precision values were 91.13 and 76.35, respectively, which shows the two different perspectives of truly predicted positive tweets.

As for LSTM, it showed a 83.09 F1-score, which could be a more appropriate measure as compared to accuracy. The kappa value showed a weak level of agreement on data for LSTM-based classification. However, the LSTM was trained on 70% of the data and validated on 30% of the data. Therefore, these predictions could have less confidence. However, the trained LSTM-based fully connected layer was used to extract deep features. These deep features were then normalized using the 1, $-1$ range of values with standard deviation and mean formulation. These normalized features were extracted for all positive and negative sentiments. These deep features were then fed to the SVM, Fine-KNN, Ensemble, and Total Boost methods of machine learning classification.

To reduce biases, the 10-fold cross-validation method was used, which further split the data into 10 proper folds of data. The training and testing were carried out properly on these data. However, these normalized deep feature-based classifications are more promising than the LSTM-based sequential data. If we look into the table, the SVM showed the same results as LSTM. However, if we look into the Fine-KNN results, they are more satisfying than both those of LSTM and SVM. Fine-KNN showed a 93.28% accuracy of results for the positive and negative tweet data. The sensitivity and precision level were also improved to 97.35% and 92.03%, respectively, in the case of Fine-KNN. The F1-score was 94.61%, which is also better than both SVM and LSTM. The other category of classification that belongs to the boosting category was also used. Two methods, Ensemble Boost and Total Boost, were used, where Ensemble Boost showed the same results as Fine-KNN. The Total Boost method showed lower results than both of these methods. However, all of the scores of the Total Boost method were higher than those of the SVM and LSTM methods.

4.2. TextBlob Sentiment Score-Based LSTM and Machine Learning Classifier Results

It can be seen by looking at

Table 6. Machine learning-based tweet classification results using the TextBlob sentiment score method.

Methods	Accuracy	Sensitivity	Precision	F1-Score	Cohen’s Kappa
LSTM	78.54	88.00	79.14	88.33	53.43
SVM	93.58	96.39	93.24	94.79	86.43
Fine-KNN	93.58	96.39	93.24	94.79	86.43
Ensemble Boost	93.58	96.39	93.24	94.79	86.43
Total Boost	86.72	89.16	88.94	89.05	72.17

that there are various categories of different domains of classification and prediction that are applied, and the visual representation is shown in Figure 5. The 1st main LSTM-based network is designed and applied on sentiment score-based sequential data. These sequential data have shown 78.54% accuracy on negative and positive tweet data. However, sensitivity and precision values are 88% and 79.14%, showing the two different perspectives of truly predicted positive tweets. The F1-score is an important measure that shows the positive and negative effect over specified wrong predictions only.

As for LSTM, it showed a value of 88.33% that could be a more appropriate measure as compared to accuracy. The kappa value showed a weak level of agreement on data for LSTM-based classification with a 53.43 rate value. However, the LSTM was trained on 70% of data and validated on 30% of data. Therefore, these predictions could have less confidence. However, the trained LSTM-based fully connected layer is used to extract deep features. These deep features are then normalized using the 1, −1 range of values with standard deviation and mean formulation. These normalized features are extracted for all positive and negative sentiments. These deep features are then fed to SVM, Fine-KNN, Ensemble, and Total Boost methods of machine learning classification.

To reduce the bias, the 10-fold cross-validation method is used that further split up the data into 10 proper folds of data. The training and testing was carried out on this data. However, these normalized deep feature-based classification are more promising than the LSTM-based sequential data. If we look into the table, the SVM showed higher results as compared to LSTM. However, if we look into the Fine-KNN results, they are the same as SVM. It showed 93.58% accuracy results over the positive and negative tweet data, more than the ratio-based sentiment score classification results. The sensitivity and precision level is also improved, as it is 96.39% and 93.24% in the case of SVM, Fine-KNN, and Ensemble Boost. The F1-score showed positive and negative effects, with a 94.79% score that is also better than LSTM (88.33%). However, the Total Boost method showed relatively lower results than these three methods but they are better than the actual LSTM results.

4.3. VADER Sentimental Score-Based LSTM and Machine Learning Classifiers Results

The VADER sentimental score-based positive and negative score sequential data features were given to LSTM, and then normalization was performed on those deep features. These deep features were then fed to SVM, Fine-KNN, and two boosting classification methods. The 70–30 data split ratio was used in the case of LSTM after extracting deep features from all data. All other classification methods used the 10-fold validation method. A tabular representation is shown in

Table 7. Machine learning-based tweet classification results using the VADER sentiment score method.

Methods	Accuracy	Sensitivity	Precision	F1-Score	Cohen’s Kappa
LSTM	80.00	88.71	80.29	84.29	56.96
SVM	80.00	88.71	80.29	84.29	56.96
Fine-KNN	94.01	96.62	93.68	95.12	87.38
Ensemble Boost	94.01	96.62	93.68	95.12	87.38
Total Boost	82.48	87.75	85.34	85.54	63.32

, and a visual representation is shown in Figure 6.

The classification results on sequential data from LSTM were more satisfying than the LSTM results of the ratio score-based sentiment sequential data, with an 80% classification accuracy score relative to a 77.56% value. Similarly, the truly predicted positive tweet data-based sensitivity and precision scores were not better. The lower sensitivity score showed that some of the false negative-based instances created different scores when the precision value was better, with 80.29% versus 76.35% in the previous LSTM score. The F1-score was 84.29%, for both the positive and negative composite scores, which is better than the previous LSTM score. As far as the kappa value is concerned, it was again lower in this case. As in the previous case, the deep feature-based SVM showed the same scores, and in this case the SVM also showed the same evaluation values. However, the better results of VADER sentiments also affected the other machine learning classifiers’ performance. The Fine-KNN showed a 94.01% accuracy score, which is by far the best accuracy score among all sentiment and sequential data scores. All other measures, including the kappa constant, showed a perfect level of agreement on the normalized deep feature data for Fine-KNN and the Ensemble Boost method. These methods showed 96.62% and 93.68% scores of sensitivity and precision, respectively, which demonstrated confidence regarding positive tweets’ true predictions. However, the F1-score was 95.12%, which could be a more significant and important measure for the classification of positive and negative tweets. Finally, the Total Boost method showed less improved results than the Fine-KNN and Ensemble Boost methods, but had higher values than the LSTM and SVM methods.

As per the above results, the VADER sentiments were also promising as compared to the ratio method of positive and negative compound scores, where these scores, based on cleaned and encoded sequential data, showed 77.56% and 80% confidence values via LSTM for both methods, respectively. However, the deep transferred features have shown very good and more promising results in the classification of positive and negative tweet data. This validated the robustness of deep features that are used in many other machine learning classification methods nowadays. Furthermore, we can say that the VADER sentiment score-based data classification is more promising than the ratio sentiment method.

4.4. Evaluation Measure

We used certain evaluation measures to validate the results, including accuracy, sensitivity, precision, F1-score, and the statistical measure Cohen’s kappa. The accuracy is calculated as shown in the following equation.

$$Accuracy=\left(\frac{TP+TN}{TP+TN+FN+FN}\right)\times 100$$

(6)

It is a general measure of data prediction based upon four fundamental terms, including true positive (TP), false positive (FP), true negative (TN), and false negative (FN). The accuracy measure indicated how correctly the positive and negative instances have been classified over all four terms. In our study, if a positive tweet is predicted correctly, it is TP. For TN, prediction is negative if it is a negative tweet and is predicted to be negative as well. FP refers to the classification of positive tweets into the negative tweet class. For FN, negative tweets are predicted as positive tweets.

$$Sensitivity=\left(\frac{TP}{TP+FN}\right)\times 100$$

(7)

In our study, the sensitivity is calculated as true predicted positive tweets divided by the sum of truly predicted positive tweets and negative predictions of actual positive tweets. Therefore, it showed the overall ratio of the correctly predicted positive tweets to the correctly and wrongly predicted positive tweets.

$$Precision=\left(\frac{TP}{TP+FP}\right)\times 100$$

(8)

The precision value is calculated as the truly predicted positive tweets divided by the sum of the truly predicted positive tweets and predictions of true positives as negative tweets. Therefore, this measure was used as another perspective regarding correctly predicted positive tweets.

$$F1\text{-}Score=2\times \left(\frac{recall\times precision}{recall+precision}\right)\times 100$$

(9)

The F1-score consists of relationships between the recall and precision values based on actual measures of truly predicted positive and negative tweet data. It could be a more reliable measure relative to accuracy to identify true results, as it is a composite form of true positive and true negative measures.

$$Kappa–Cohen=\left(\frac{\frac{{cm}_1\times {rm}_1}{n}+\frac{{cm}_2\times {rm}_2}{n}}{n}\right)\times 100$$

(10)

Cohen’s kappa is a statistical constant that estimates the reliability of data for classification. The equation explains this measure using a confusion matrix. ${cm}_1$ shows the value of column 1 in the confusion matrix, whereas ${rm}_1$ shows row 1 of the confusion matrix. Similarly, for row 2 and column 2 of the confusion matrix, the margins are taken up for the calculation of the kappa value. Values of 0–20 for kappa indicate no agreement of data, 21–39 indicate a minimal level of agreement, 40–59 indicate a weak agreement, 60–79 indicate a moderate agreement, 80–89 indicate a strong level of agreement, and a value of 90 indicates a perfect level of agreement.

5. Conclusions

COVID-19 vaccine-related Twitter sentiment data for the Gulf countries were collected via Twitter data. The data contained information regarding different behaviors and feedback of Gulf countries’ populations. To examine vaccine hesitancy and the effect of different types of vaccines on it, the current study was conducted. Various hot topics in all six countries, derived from tweets, were discussed, highlighting the concerns of these countries. The collected data were refined using various data cleaning methods. The refined tokenized data were then scored using three sentiment analysis methods: The ratio, TextBlob, and VADER methods. All of these methods produced negative and positive scored data, which were then converted into sequential data using a tokenized data vocabulary. These sequential data were split up into training and testing sets and then fed into LSTM and classified into positive and negative tweet data. Sentiment score data of all three sentiment methods were separately classified using LSTM. However, the results were not very promising, and thus, deep features from LSTM were extracted and fed into other classical machine learning classification methods. The deep feature-based classification methods showed more promising results than the LSTM via all three sentiment method-based features. By looking into all results, the Fine-KNN and Ensemble Boost methods of classification on deep features of the LSTM method both showed a 94.01% accuracy level on VADER sentiment score data.

In the future, additional data from social media platforms could be used in the sentiment analysis of various countries to analyze their concerns, which could help in countering vaccine hesitancy. As far as the classification of positive and negative text data is concerned, various other methods of feature encoding could be used, representing opportunities for classification improvement. The proposed method of LSTM and deep features could be used in different NLP tasks to enhance the accuracy.