Next Article in Journal / Special Issue
The Impact of COVID-19 on Active Living and Life Satisfaction of Rowers
Previous Article in Journal
Antimicrobial Nanomaterials: A Review
Previous Article in Special Issue
A Cross-Sectional Study to Identify Factors for Vaccination Uptake Amongst University Staff and Students in Northern Ireland
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Mandatory Vaccination against COVID-19: Qualitative Analysis of Responses to Open-Ended Questions

by
Konstantinos Giannakou
*,
Eftychia Soulioti
and
Maria Kyprianidou
Department of Health Sciences, School of Sciences, European University Cyprus, Nicosia 1516, Cyprus
*
Author to whom correspondence should be addressed.
Hygiene 2023, 3(3), 291-305; https://doi.org/10.3390/hygiene3030021
Submission received: 29 June 2023 / Revised: 25 July 2023 / Accepted: 3 August 2023 / Published: 4 August 2023
(This article belongs to the Special Issue COVID-19: Health and Hygiene)

Abstract

:
The mandatory COVID-19 vaccination has been suggested as a strategy to increase vaccination uptake; however, there are doubts about its legitimacy as a measure. This study aimed to explore in depth the opinions of the Cypriot population regarding mandatory vaccination against COVID-19. Qualitative analysis of responses to an open-ended question was included as part of an anonymous, online, cross-sectional survey of the general population of Cyprus. A thematic analysis was performed by reading the responses and developing a coding frame. Descriptive analysis was performed on the sociodemographic and COVID-19-vaccine-related information. A total of 311 participants responded to the open-ended question, and the thematic analysis yielded two main themes: “Vaccination Perceptions and Challenges” and “Mandatory Vaccination Policies and Societal Impact.” Within these themes, participants expressed diverse opinions on universal and targeted mandatory vaccination, highlighting concerns about insufficient research, gaps in communication, and potential side effects. Views also varied on topics such as protective measures, interdisciplinary collaboration, vaccine side effects, human rights, and societal divisions. Criticisms encompassed political motives, skepticism towards the pandemic, and considerations of individual freedoms. In summary, the findings highlighted the intricate and varied perspectives surrounding the issue of mandatory COVID-19 vaccination, which can be valuable for policymakers and public health officials in designing targeted vaccination strategies. To optimize vaccination programs, policymakers and public health officials should prioritize trust, communication, and individual autonomy.

1. Introduction

In March 2020, the World Health Organization (WHO) declared coronavirus disease 2019 (COVID-19) as a pandemic. The new zoonotic disease, referred to as COVID-19, caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), spread rapidly in practically all parts of the world [1,2,3]. Various public health measures, such as mask usage, the enforcement of quarantines, and proof of COVID-19 status for traveling and social interactions, have been implemented to reduce the risk of SARS-CoV-2 infection in the community [4,5]. From the end of 2020 onwards, the authorization of vaccines against SARS-CoV-2 became a reality.
Vaccinations are among the most successful public health interventions ever introduced, responsible for reducing, eliminating, and eradicating several diseases. They have also contributed significantly to the overall reduction in mortality of vaccine-preventable diseases over the past two centuries [6,7,8]. The benefits of vaccination as an effective and safe measure to mitigate the impact of COVID-19 are significant, including its efficacy in preventing hospitalizations, reducing mortality rates, and lowering healthcare-related expenses [9,10,11,12]. Despite the extensive evidence supporting the effectiveness and safety of the COVID-19 vaccines, a widespread global vaccination campaign was necessary to persuade the global population to willingly adopt the free vaccination program against the disease.
Vaccine hesitancy, defined as the postponement or refusal to receive vaccines despite their availability, has emerged as a significant concern and threat [13]. Because of this, some countries imposed mandatory COVID-19 vaccination [14,15,16,17]. Existing research indicates a fragmented public stance on vaccine mandates, with diversity observed across different countries [18,19,20,21,22,23]. So far, very little is known about COVID-19 vaccine mandates, and qualitative studies on this topic are rare. In a recently published cross-sectional online study that investigated the attitudes of the general population of Cyprus towards mandatory COVID-19 vaccination, we reported that more than two-thirds of the participants opposed a mandatory policy, possibly suggesting that this policy is perceived as infringing upon personal freedoms and human rights [24]. This study was conducted during the initial stage of general public COVID-19 vaccination in the Republic of Cyprus (late 2021 to early 2022), a period marked by high case rates and a strong emphasis on the importance of vaccination [24]. Upon conducting a more detailed examination of the participants’ responses, we analyzed their additional information and perspectives shared regarding mandatory COVID-19 vaccination, as prompted by an optional open-ended question in the survey. To the best of our knowledge, this study represents the first qualitative assessment exploring the opinions of the general population of Cyprus on this topic.

2. Materials and Methods

2.1. Study Design, Population, and Data Collection

In this paper, a thematic analysis will be conducted on the open-ended optional questions included in the cross-sectional online study that gathered self-reported data on the attitudes of the general population of Cyprus towards mandatory COVID-19 vaccination. Details on the methodology (e.g., study population, data collection, and questionnaire) have been previously published elsewhere [24]. In short, we conducted an online cross-sectional survey between 15 November 2021 and 7 January 2022 using convenience (non-probability) sampling to recruit participants. The study population consisted of adult Greek–Cypriot individuals, both men and women, aged 18 years and above, residing in the five government-controlled municipalities in the Republic of Cyprus, namely, Nicosia, Limassol, Larnaca, Paphos, and Ammochostos.
An online self-administered questionnaire, created in Google Forms and distributed online via instant messaging applications and social media platforms, was used as the research instrument. The survey consisted of a total of 47 questions, both open-ended and close-ended, in the Greek language. These questions covered various aspects such as sociodemographic characteristics (e.g., age, gender, and educational level), health-related characteristics, trust and satisfaction with the healthcare system, utilization of preventive healthcare services (e.g., presence of chronic diseases and use of preventive healthcare services), information related to COVID-19 vaccination, sources of vaccine information, and participants’ general knowledge and attitudes towards mandatory vaccination. In addition to these close-ended items, the final open-ended question of the survey asked participants to provide any additional information or perspective they would like to share regarding mandatory COVID-19 vaccination. Subsequently, qualitative thematic analysis was used to examine the responses to this question.

2.2. Qualitative Data Analysis

In order to maintain methodological rigor, we adhered to the consolidated criteria for reporting qualitative research (COREQ) [25]. This comprehensive 32-item checklist was used to accurately depict the study’s methods, contextual information, findings, analysis, and interpretations [25]. In order to analyze the qualitative content of the open-ended responses, we conducted an inductive thematic analysis with the objective of providing a meaningful and coherent representation of the data’s content [26]. Thematic analysis was used to organize the qualitative data, establishing a coding system where codes were grouped into recurring themes that aligned with the research question [27].
Due to the broad question and the manageable number of responses (n = 311), manual coding of the data was chosen over computer-assisted analysis. All responses were imported into Microsoft Excel (Excel 2021 for Windows) for coding. All responses were independently coded by two researchers (E.S. and K.G.). The analysis began with a thorough reading of all responses to immerse oneself and gain an overall understanding. The data were then examined word by word to identify codes by highlighting specific words or phrases that captured key patterns and differences in ideas and concepts [28]. Throughout this process, labels emerged for codes that encompassed multiple key considerations, often derived directly from the text and forming the initial coding scheme. Afterward, the codes were organized into categories or key ideas at a surface level, capturing the explicit meaning of the data. Subsequently, these categories were critically examined and grouped into themes [28]. To ensure the credibility of the analysis, regular meetings were held with co-authors (E.S. and K.G.) to discuss the codes, categories, and themes. To ensure objectivity and validate the coding decisions, a third researcher (M.K.), who was not involved in the initial development of the themes, independently reviewed the coding. M.K. assessed the plausibility of the identified themes and evaluated the consistency of the analysis, thereby contributing to the rigorous approach [29]. In cases of disagreement, M.K. made the final coding decision. This process ultimately led to the identification of the themes.

2.3. Validity and Reliability

Qualitative rigor was achieved by adhering to the criteria outlined by Guba and Lincoln, which include credibility, transferability, dependability, and confirmability [30]. These criteria served as a guiding framework to ensure the robustness and reliability of the qualitative research process. Credibility, akin to internal validity in quantitative studies, was ensured through comprehensive data collection and analysis, with all researchers thoroughly familiarizing themselves with the collected responses and employing accurate coding techniques. To ensure transferability, similar to external validity, direct quotes were utilized to illustrate the synthesized findings. Dependability, similar to reliability, was maintained by engaging a single coder who had no prior involvement in the theme-development process. Additionally, confirmability was achieved through analyst triangulation, involving the participation of three researchers. The coders independently analyzed the verbatim responses and subsequently cross-validated their findings through discussions and consensus among themselves. Source triangulation was accomplished by collecting responses from participants of diverse ages and genders residing in various geographical areas of Cyprus.

2.4. Statistical Analysis

The Shapiro–Wilk normality test and QQ (quantile–quantile) plots were used to examine the normality of the continuous variables. Participants’ characteristics are presented as mean ± standard deviation (SD) for continuous measures with normal distribution and absolute (n) and relative (%) frequencies for categorical variables. The statistical analysis was conducted using STATA 14.0 software (Stata Corp, College Station, TX, USA).

2.5. Ethical Approval

Participation in the study was completely anonymous, and prior to their involvement, all participants were provided with information regarding the purpose and objectives of the study. Ethical approval was granted from the Cyprus National Bioethics Committee (CNBC) (EEBK ΕΠ. 2021.01.219).

3. Results

3.1. Sociodemographic Characteristics and COVID-19 Vaccination Status

The final sample consisted of 311 participants, with 183 (59.4%) being females. Out of the respondents, 45 (14.5%) were in favor of mandatory COVID-19 vaccination, while 266 (85.5%) were against such a policy. The mean age of the participants was 39.3 years old (SD = 10.5). Most of the participants were employed in the private sector (n = 129, 42.2%), and a significant portion were married or cohabiting (n = 224, 73%). Furthermore, 63.3% (n = 170) had children under the age of 18 living in their households. Most participants had completed undergraduate education (n = 152, 49.5%) and earned an annual income of over EUR 19,500 (n = 156, 51.8%) (Table 1).
The majority of the participants (n = 198, 64.9%) had not received the COVID-19 vaccine. Among the vaccinated individuals, a significant proportion (n = 67, 61.5%) received two doses, with the Pfizer vaccine being the most common choice (n = 76, 69.7%). Furthermore, most participants did not fall into the category of a vulnerable group (n = 257, 84.8%), and most of them expressed no intention to receive the COVID-19 vaccine in the future (n = 173, 86.5%) (Table 2).

3.2. Findings of the Qualitative Data Analysis

Table 3 presents the findings of the qualitative analysis conducted on responses to the opened-ended question. The qualitative analysis identified two thematic areas: “Vaccination Perceptions and Challenges” and “Mandatory Vaccination Policies and Societal Impact”.

3.3. Vaccination Perceptions and Challenges

3.3.1. Universal Mandatory Vaccination

Some respondents expressed the view that mandatory vaccination should be universal. According to them, to effectively combat the pandemic and eradicate it completely, universal mandatory vaccination for the entire population is the most appropriate approach. In the open responses of the survey, some participants chose to write their comments in capital letters.
“Vaccination should be made mandatory for everyone.”
—Male, age 46
“Forced vaccination.”
—Woman, age 25
“Mandatory vaccination for everyone.”
—Woman, age 23

3.3.2. Targeted Mandatory Vaccination

The concept of targeted vaccination, focusing on specific population groups, garnered greater acceptance. They acknowledge that certain vulnerable groups, such as individuals aged 65 and above or those with specific health conditions, should receive the vaccine against COVID-19. Additionally, mandatory vaccination is deemed necessary for healthcare workers who face higher exposure to COVID-19 cases, making them more susceptible to the disease.
“Mandatory vaccination should apply to specific population groups, not to the general population (e.g., above a certain age).”
—Woman, age 26
“I disagree with the way you forced the third dose on people who don’t need it at this stage—forced vaccination! Older people have not been vaccinated with even one dose! Let’s make it mandatory for the over 60s, like other countries have done.”
—Woman, age 42
“Mandatory vaccination should be implemented at least in working groups: health professionals, those who work in hospitals, closed structures, schools, police, fire department, shops, and public services.”
—Woman, age 47

3.3.3. Insufficient Research

Several respondents expressed concern regarding the vaccines administered to protect people from COVID-19, stating that they have not undergone sufficient testing. They felt that the duration of the administration process was too short, leaving potential side effects untested. They referred to it as a “premature vaccine” and expressed particular concern that these vaccines were developed and granted approval within a significantly condensed timeframe. Some respondents called for more research on vaccines, especially regarding their side effects and impacts on children and specific population groups. They mentioned that older vaccines, which have been in use for several years, provide reassurance due to their extensive testing that minimizes potential risks. Another concern expressed by the respondents is that the vaccines in question are mRNA vaccines, classified as gene therapy. They pointed out studies suggesting that vaccines of this type may lack long-term effectiveness and could potentially have side effects. Their concern stems from the insufficient research available on this specific type of vaccine to guarantee the absence of side effects.
“But the much-discussed ‘vaccine’ against ‘SARS-CoV-2′, as we all know, is an early vaccine that has not undergone long-term testing stages (6+ years) …”
—Male, age 29
“The older vaccines have shown improvement over time, and the occurrence of side effects has been extremely rare… The main concern is the rushed administration of the vaccine”
—Woman, age 42
“…for a vaccine to be considered safe, it usually takes 5–6 years… Therefore, personally, I don’t feel confident in the safety of these vaccines…”
—Woman, age 42
“For people with autoimmune diseases, there have been reports of disease flare-ups after completing their vaccination regimen.”
—Female, age 44

3.3.4. Inadequate Communication and Information Discrepancies

This category is intricately linked to the previous one, as the majority of respondents express concerns about the lack of immediate access to up-to-date information on this crucial global issue. They seek more information and answers to questions such as why vaccinated individuals can still become sick, the true effectiveness of the vaccine, and many other queries that arise, contributing to a sense of insecurity among participants. Respondents feel that the competent authorities have not adequately communicated the issue to the population, imposing obligations without providing the necessary clarifications. The communication policy adopted by the government, as perceived by the respondents, failed to convince them about the necessity of COVID-19 vaccination. The respondents expressed the need for comprehensive and convincing information from authoritative sources, but they found the messages conveyed through social media to be contradictory and lacking proper documentation.
“Mandatory vaccination may not be the safest approach to tackle COVID-19… Better information is needed.”
—Male, age 52
“Providing proper information, rather than enforcement, will yield better results.”
—Male, age 48
“The communication strategy throughout the pandemic, in my opinion, was fundamentally flawed! The attitudes displayed by prominent scientists, who were not necessarily the most specialized or experienced in the field, were often contradictory and lacked proper substantiation. This not only failed to inspire trust but also ran counter to the accumulating evidence.”
—Male, age 58
“I believe that if the mass media in Cyprus, particularly television, didn’t repeatedly emphasize the importance of vaccination, more people might have chosen to get vaccinated.”
—Woman, age 42

3.3.5. Protective Measures

In the case of protective measures, several opinions were expressed, which were quite different. Some expressed the opinion that there should be measures for the unvaccinated, while others stated that either the same protection measures should be applied with greater diligence or that measures should be applied to all population groups regardless of whether the unvaccinated were vaccinated. Also, another opinion that was expressed is that there should be a lifting of the measures as they do not, in fact, bring any result. More specifically, the respondents asked for measures to protect the unvaccinated, either by imposing their mandatory confinement or restricting their movements, or even imposing fines and banning them from leaving the country, considering that the unvaccinated are the cause of the spread of the virus. They further believe that people who have not been vaccinated and subsequently become ill with the coronavirus should be denied the right to healthcare.
“Implement a lockdown for the unvaccinated.”
—Woman, age 58
“Implement measures to address the pandemic by restricting the activities of the unvaccinated and considering compulsory vaccination.”
—Male, age 66
“Find a legal mechanism for compulsory vaccination and restrict the travel of unvaccinated individuals, similar to what Israel has done.”
—Female, age 56
“…unvaccinated individuals should not be entitled to ICU care without paying.”
—Male, age 39
Some respondents emphasized the need to increase safeguards. They highlighted those vaccinated individuals, despite being protected from severe illness, can still contract and transmit the virus. Therefore, they believe that frequent antigen tests should be conducted for vaccinated individuals as well. These respondents particularly emphasized the importance of testing for those working in high-occupancy environments, such as large companies, hospitals, and social events like weddings and receptions. Additionally, they stressed the significance of strengthening the body’s defenses through measures such as vitamin supplementation. Furthermore, they emphasized the essential role of strengthening the healthcare system and the involvement of both private and public hospitals.
“…by using other methods to fight the virus, such as vitamin supplementation.”
—Woman, age 23
“…given that vaccinated individuals can still get infected and transmit the virus, it would be beneficial for them to undergo antigen testing as well.”
—Woman, age 23
“There should be weekly free testing available in businesses, especially in hospitals and clinics where nurses come into close contact with patients.”
—Female, age 23
“It should be mandatory for everyone, regardless of vaccination status, to undergo rapid testing if we genuinely prioritize the well-being of all.”
—Male, age 27

3.3.6. Interdisciplinary Collaboration

The collaboration of various scientific disciplines emerged as another important issue highlighted in the participants’ responses. Participants emphasized the need for cooperation with legal experts to ensure that all legal procedures regarding mandatory vaccination have been followed. They stressed the importance of involving a broader range of experts beyond those perceived to be part of the government circle. Additionally, respondents expressed the belief that establishing contact with experts from other European countries would yield more positive outcomes, as they are better informed and experienced in dealing with crises like the COVID-19 pandemic.
“It is important to seek the opinion of lawyers as well.”
—Woman, age 50
“They should consult experts outside of the government team.”
—Man, age 40
“It is crucial to follow the instructions and actions taken by other countries such as France, Germany, and Italy, which have more advanced medical, university, research networks and knowledge.”
—Male, age 56

3.3.7. Lack of State Responsibility

What particularly scared the participants was that the government did not take responsibility for the side effects of the vaccine. This lack of accountability acted as a deterrent to vaccination.
“Even worse is the fact that you do not take responsibility for any side effects.”
—Male, age 45

3.3.8. Natural Immunity

Some participants were proponents of natural immunity. They hold the belief that the coronavirus, like any other virus, should be dealt with through natural means. They believe that the human body has the inherent ability to recognize and develop ways to combat the virus on its own. On the other hand, they view vaccines as acting aggressively on the virus and potentially causing it to mutate.
“I firmly believe that natural immunity against viruses is significantly superior to any vaccine. However, I acknowledge that vaccines may have their usefulness in dealing with other diseases.”
—Male, age 38
“In my opinion, the ultimate solution lies in natural immunity, especially for high-risk individuals.”
—Male, age 37

3.3.9. COVID-19 Denial and Perceived Risk

Very few respondents expressed the belief that the coronavirus does not exist, that it is a conspiracy, and that there is no pandemic. They view all political decisions and precautionary measures implemented by governments as fraudulent, suggesting that it is a global experiment. Additionally, concerns were raised about the perceived level of danger posed by the coronavirus. They consider it to be a simple virus that has received excessive attention.
“If you don’t know, you should learn that there is no such thing as COVID-19…. You are committing the greatest crime in the history of mankind.”
—Male, age 43

3.4. Mandatory Vaccination Policies and Societal Impact

3.4.1. Two-Tiered Society

According to the respondents, the differing approach by governments regarding the measures applied to vaccinated and unvaccinated individuals has led to a division among the people. This division has created tensions and conflicts between individuals. The measures implemented by the government were perceived as punitive towards the unvaccinated rather than serving as an encouragement to get vaccinated.
“… not in the division of people.”
—Male, 45 age years
“Privileges are provided by the government, such as overnight stays for vaccinated people, while other measures are punitive towards the unvaccinated. Personally, this differential treatment instils fear in me.”
—Female, age 39

3.4.2. Human Rights

The main argument against mandatory vaccination is the violation of human rights and a threat to autonomy. Respondents strongly believe that individuals should have the freedom to choose whether to be vaccinated or not. They argue that this decision pertains to personal autonomy and the ability to protect one’s own body.
“…the infringement of human rights”
—Woman, age 44
“Everyone should have the right to decide what goes into their own body…”
—Woman, age 33
“…mandatory vaccination across the entire population could potentially infringe upon individual personal liberties to some extent”
—Male, age 56
“Human rights should never be questioned under any circumstances.”
—Man, age 48

3.4.3. Reactive Response to Obligation

A few participants argue that the mandatory nature of vaccination alone has led citizens to view it with suspicion and report that this approach inadvertently contributes to the emergence of conspiracy theories. The pressure to get vaccinated and the implementation of punitive measures have resulted in increased resistance from people. The respondents believe that if the vaccination campaign had been approached more gently, there would have been a higher number of people willing to get vaccinated.
“…as well as the oppressive way that it is imposed, only create a negative mood in the world…”
—Woman, age 40
“…when something becomes mandatory, it is automatically considered suspicious…”
—Male, age 31

3.4.4. Political and Economical Expediencies

The contradictions among experts regarding pandemic management and the dissemination of information by the media on virus-related issues have greatly concerned the responders. One major worry is the lack of conviction that we are truly experiencing a pandemic, leading to skepticism about political decisions driven by powerful interests. The respondents believe that these interests, particularly pharmaceutical companies collaborating with governments, prioritize financial gains over public welfare. They perceive a pre-planned setup to exploit the world for personal gain.
“I have faith in science, but I no longer trust the financial backers of scientific research and their pursuit of enormous profits. I am certain that they are not concerned about my well-being…”
—Female, age 46
“…the government has failed to persuade me that it is providing the vaccine for my personal protection”
—Woman, age 39

3.4.5. Social Impact Outweighed

Vaccination has been mandated by governments as a matter of public health, according to some respondents. The rationale behind this viewpoint is that one person’s freedom of choice regarding vaccination can pose risks to the health of others. Therefore, mandatory vaccination is deemed necessary. It is seen as the most effective way for people to overcome the current restrictions in their lives more swiftly and return to a state of normalcy.
“Vaccines are considered the sole solution for the greater good, and anyone who refuses to comply should be compelled to do so.”
—Female, age 52
“While mandatory vaccination may encroach upon individual rights, the infringement is deemed less significant compared to the violation of the collective’s right to health and freedom from the restrictions imposed by the pandemic.”
—Male, age 60

4. Discussion

Through this qualitative research, we aimed to conduct a thematic analysis of the optional open-ended question included in our previous cross-sectional study [24]. The qualitative analysis conducted in this study identified two thematic areas related to vaccination perceptions and challenges, mandatory vaccination policies and societal impact. Overall, these results reflect a complex landscape of perspectives and concerns regarding mandatory COVID-19 vaccination. Participants had varying opinions on universal vaccination, with some supporting mandatory vaccination for all and others, suggesting it should be limited to specific groups. Concerns were raised about insufficient research, communication gaps, and potential side effects. Opinions varied on protective measures, with calls for measures to protect the unvaccinated and differing perspectives on their effectiveness. Interdisciplinary collaboration and the involvement of external experts were highlighted. Participants discussed vaccine side effects, human rights and autonomy concerns, and societal divisions arising from vaccination policies. Criticisms included perceived political and economic motivations, skepticism about the pandemic, and concerns about individual freedoms.
In the first thematic section, “Vaccination Perceptions and Challenges”, the analysis uncovered various perspectives among respondents regarding COVID-19 vaccination. Some participants expressed support for universal vaccination, advocating for mandatory vaccination for everyone, while the majority advocated for targeted mandatory vaccination of specific vulnerable groups, such as vulnerable individuals and healthcare workers. The use of capital letters in some responses may indicate strong beliefs or intensity of feeling about the topic. Nevertheless, to establish a definitive interpretation, additional research would be necessary. In alignment with our findings, a prior qualitative study investigating the perspectives of British adults regarding the enforcement of COVID-19 vaccination discovered that the concept of targeted mandatory vaccination was generally well-received, whereas only a small number of participants expressed support for universal mandatory vaccination [23]. Furthermore, concerns were raised about the vaccines’ insufficient testing, particularly due to the shortened timeline of their development and approval. Participants expressed concerns about vaccine side effects and called for more research on vaccine side effects and their impact on specific populations. Similarly to our findings, prior qualitative studies demonstrated that concerns related to the swift development of COVID-19 vaccines and potential side effects were the key drivers of vaccine hesitancy [31,32,33]. Likewise, previous studies found that the fear of possible side effects and concerns over the newness and safety of the vaccine were among the reasons for COVID-19 vaccination refusal [34,35,36]. In our study, participants expressed concerns regarding the lack of clear and comprehensive communication from authoritative sources, resulting in confusion and a sense of insecurity. They sought up-to-date information and answers to their questions regarding vaccine effectiveness and potential side effects. These findings align with a previous qualitative study that identified the main critique of public health measures as the conflicting nature of the measures and the inconsistent messaging from public health authorities [37]. Similarly, another qualitative study revealed that participants emphasized the importance of government messages addressing side effects, utilizing social media to engage younger individuals effectively, and prioritizing personalized one-on-one discussions and household visits to build trust and inform community members [32].
According to our findings, different opinions were expressed regarding protective measures. Some participants called for stricter measures for the unvaccinated, while others questioned the effectiveness of the measures altogether. Some emphasized the importance of frequent testing and strengthening the healthcare system. In line with findings from a previous qualitative study, it was found that some individuals argued it was reasonable to withhold certain privileges from those who, by choosing not to share the societal burden of vaccination, were perceived as selfish [23]. Participants emphasized the need for legal experts to ensure proper procedures are followed in implementing mandatory vaccination policies. They also believed that involving a broader range of experts, beyond those associated with the government, would lead to better outcomes. They suggested collaborating with experts from other countries who have experience in dealing with similar crises. In addition, some respondents felt that the government did not take responsibility for vaccine side effects, which acted as a deterrent to vaccination. Furthermore, some participants believed in the power of natural immunity and preferred relying on the body’s ability to combat the virus without vaccines. Similarly, in line with findings from a previous study, it was noted that some individuals in focus groups believed that widespread infection would result in population immunity and, thus, minimize the necessity for vaccination [31]. In our study, a minority of respondents denied the existence of the coronavirus and expressed disbelief about the severity of the virus. Notably, a previous study observed a significant divergence of opinions among the participants regarding the perceived level of threat posed by the COVID-19 virus. Those opposing mandatory vaccination often argue that the danger of the virus is exaggerated by others, while proponents of such a policy hold the opposite view [18].
In the second theme, “Vaccination Mandatory Policies and Societal Impact”, the issue of creating a two-tiered society was prominently discussed. Respondents argue that citizens of Cyprus were divided into vaccinated and unvaccinated, leading to tensions between people. Respondents felt that the government’s measures were punitive towards the unvaccinated rather than serving as a genuine incentive for vaccination. Consistent with our findings, a prior qualitative study identified a theme referred to as the “two-tier system.” This theme encompassed participants’ perspectives on the perceived punishment imposed on non-vaccinated individuals, such as exclusion from specific sectors of employment, limited access to certain activities and services, or segregation in daily life. Also, a minority of participants raised a counter-argument, suggesting that it was unjust for vaccinated individuals to endure ongoing restrictions due to others’ refusal to get vaccinated [23]. According to our findings, the violation of human rights was a major argument against mandatory vaccination. Participants emphasized the importance of personal autonomy and the freedom to choose whether to be vaccinated or not. Freedoms encompass individuals’ beliefs regarding their behavior, evaluation, and emotions, where any factors that impede an individual’s autonomy can be seen as jeopardizing their freedom [38]. Previous studies also highlighted that the perception of vaccine mandates was a challenge to individual autonomy, freedom of choice, and personal rights [23,33]. Furthermore, participants in our study felt that the mandatory nature of vaccination, along with the implementation of punitive measures, increased resistance and contributed to the emergence of conspiracy theories while also resulting in increased resistance due to the pressure to get vaccinated. A previous qualitative study supports our findings, suggesting that vaccine mandates had the potential to further strengthen opposition among those who were already skeptical [23]. Similarly, in a previous qualitative study, respondents argued that individuals in their community, particularly young people, display vaccine hesitancy as a result of their reluctance to be mandated or coerced into receiving the vaccine, especially by governmental authorities [32]. Moreover, evidence from various countries, including Cyprus, indicates that conspiracy beliefs are a significant predictor of vaccine hesitancy [31,39,40,41]. In the same theme, it was revealed that a possible mandatory imposition of the COVID-19 vaccine by the government was believed to serve economic and political interests. Moreover, in our study, respondents expressed concerns about contradictions among experts, media dissemination of information, and skepticism about political decisions driven by powerful interests. They believed that financial gains were prioritized over public welfare and perceived a pre-planned setup to exploit the world. Some participants viewed mandatory vaccination as necessary for public health, believing that one person’s freedom of choice regarding vaccination could pose risks to others. They saw it to overcome restrictions and return to normalcy.
Based on the findings of this qualitative research, there are important implications for policy, public health, and further research. Policymakers should carefully consider the advantages and challenges of universal vaccination strategies, considering the support expressed by some participants. Addressing concerns about vaccine side effects is crucial, and policymakers should provide accurate and accessible information on safety profiles and potential risks. Communication strategies need improvement to ensure transparent and authoritative information reaches the public. Public health efforts should focus on building trust in healthcare providers and the government to increase vaccine acceptance. Strategies should be developed to protect both vaccinated and unvaccinated populations. Collaboration among different disciplines, including legal experts and scientists from other countries, can provide valuable insights for managing vaccination and public health challenges. Future research should encourage interdisciplinary collaboration to address complex issues related to vaccination and public health. This study highlights that the mandatory vaccination approach led to backlash, infringed upon human rights, and contributed to a divided society. Concerns about vaccine side effects further deter individuals from getting vaccinated. Improved information sharing, research efforts, and interdisciplinary collaboration would promote a greater sense of safety and enhance vaccine uptake among citizens.
It is important to acknowledge the limitations of this research. One significant limitation relates to the analysis of responses to open-ended questions, as it is challenging to ascertain the reasons behind individuals’ choices to answer or not answer a question. Factors like time constraints may influence participants’ decision to respond. Moreover, relying solely on a single question may restrict a comprehensive understanding of the underlying meanings in participants’ perspectives, as a more extensive interview process would yield more nuanced insights. While qualitative studies can be prone to researcher bias, it is worth noting that the current study took measures to address this concern. The analyses conducted in this study were cross-validated by an independent third researcher who was not involved in the original coding process. This external validation helped ensure the credibility and reliability of the findings. The use of online convenience sampling may lead to limitations in representativeness and introduce selection bias, impacting the validity of the findings. While qualitative research aims for in-depth understanding, the generalizability of the results can be affected. Disparities in technology access may also skew the sample, influencing external validity. It is advisable to address these issues in future research.

5. Conclusions

In conclusion, this qualitative research study revealed diverse opinions and concerns regarding mandatory vaccination policy. Participants expressed varying opinions on universal vaccination, with some supporting mandatory vaccination for all and others advocating for targeted vaccination of specific groups. Concerns were raised regarding insufficient research, communication gaps, potential side effects, and the societal impact of vaccination policies. The analysis also highlighted the importance of interdisciplinary collaboration and the involvement of external experts. Criticisms included perceived political and economic motivations, skepticism about the pandemic, and concerns about individual freedoms. Policymakers and public health officials in Cyprus can navigate vaccination programs more effectively while upholding public health and individual rights by strengthening public trust, improving communication strategies, and respecting individual autonomy.

Author Contributions

Conceptualization, K.G.; methodology, K.G., E.S. and M.K.; software, K.G. and M.K.; validation, K.G., E.S. and M.K.; formal analysis, K.G., E.S. and M.K.; investigation, K.G. and M.K.; data curation, K.G., E.S. and M.K.; writing—original draft preparation, K.G. and M.K.; writing—review and editing, K.G., E.S. and M.K.; supervision, K.G.; project administration, K.G. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

This study was conducted according to the Declaration of Helsinki guidelines, and all procedures involving research study participants were approved by the Cyprus National Bioethics Committee (CNBC) (EEBK ΕΠ. 2021.01.219).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The datasets used and/or analyzed during the current study are available from the corresponding author upon reasonable request.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Tsiotas, D.; Tselios, V. Understanding the uneven spread of COVID-19 in the context of the global interconnected economy. Sci. Rep. 2022, 12, 666. [Google Scholar] [CrossRef] [PubMed]
  2. Pullano, G.; Pinotti, F.; Valdano, E.; Boëlle, P.-Y.; Poletto, C.; Colizza, V. Novel coronavirus (2019-nCoV) early-stage importation risk to Europe, January 2020. Eurosurveillance 2020, 25, 2000057. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. World Health Organization. Naming the coronavirus disease (COVID-19) and the virus that causes it. Braz. J. Implantol. Health Sci. 2020, 2, 1–4. [Google Scholar]
  4. Mills, M.C.; Rüttenauer, T. The effect of mandatory COVID-19 certificates on vaccine uptake: Synthetic-control modelling of six countries. Lancet Public Health 2022, 7, e15–e22. [Google Scholar] [CrossRef]
  5. Scott, N.; Saul, A.; Spelman, T.; Stoove, M.; Pedrana, A.; Saeri, A.; Grundy, E.; Smith, L.; Toole, M.; McIntyre, C.R. The introduction of a mandatory mask policy was associated with significantly reduced COVID-19 cases in a major metropolitan city. PLoS ONE 2021, 16, e0253510. [Google Scholar] [CrossRef]
  6. Greenwood, B. The contribution of vaccination to global health: Past, present and future. Philos. Trans. R. Soc. B Biol. Sci. 2014, 369, 20130433. [Google Scholar] [CrossRef] [Green Version]
  7. Kauffmann, F.; Bechini, A.; Bonanni, P.; Casabona, G.; Wutzler, P. Varicella vaccination in Italy and Germany–different routes to success: A systematic review. Expert Rev. Vaccines 2020, 19, 843–869. [Google Scholar] [CrossRef]
  8. Clark-Wright, J.; Hudson, P.; McCloskey, C.; Carroll, S. Burden of selected infectious diseases covered by UK childhood vaccinations: Systematic literature review. Future Microbiol. 2020, 15, 1679–1688. [Google Scholar] [CrossRef]
  9. Chuenkitmongkol, S.; Solante, R.; Burhan, E.; Chariyalertsak, S.; Chiu, N.-C.; Do-Van, D.; Husin, M.; Hwang, K.-P.; Kiertiburanakul, S.; Kulkarni, P.S. Expert review on global real-world vaccine effectiveness against SARS-CoV-2. Expert Rev. Vaccines 2022, 21, 1255–1268. [Google Scholar] [CrossRef]
  10. Utami, A.M.; Rendrayani, F.; Khoiry, Q.A.; Alfiani, F.; Kusuma, A.S.; Suwantika, A.A. Cost-Effectiveness Analysis of COVID-19 Vaccination in Low-and Middle-Income Countries. J. Multidiscip. Healthc. 2022, 15, 2067–2076. [Google Scholar] [CrossRef]
  11. Shrotri, M.; Navaratnam, A.M.; Nguyen, V.; Byrne, T.; Geismar, C.; Fragaszy, E.; Beale, S.; Fong, W.L.E.; Patel, P.; Kovar, J. Spike-antibody waning after second dose of BNT162b2 or ChAdOx1. Lancet 2021, 398, 385–387. [Google Scholar] [CrossRef]
  12. Pormohammad, A.; Zarei, M.; Ghorbani, S.; Mohammadi, M.; Razizadeh, M.H.; Turner, D.L.; Turner, R.J. Efficacy and safety of COVID-19 vaccines: A systematic review and meta-analysis of randomized clinical trials. Vaccines 2021, 9, 467. [Google Scholar] [CrossRef]
  13. Dror, A.A.; Eisenbach, N.; Taiber, S.; Morozov, N.G.; Mizrachi, M.; Zigron, A.; Srouji, S.; Sela, E. Vaccine hesitancy: The next challenge in the fight against COVID-19. Eur. J. Epidemiol. 2020, 35, 775–779. [Google Scholar] [CrossRef] [PubMed]
  14. Dyer, O. COVID-19: Turkmenistan becomes first country to make vaccination mandatory for all adults. Br. Med. J. 2021, 374, n1766. [Google Scholar]
  15. Dyer, O. COVID-19: US imposes mandatory vaccination on two thirds of workforce. BMJ 2021, 374, n2238. [Google Scholar] [CrossRef]
  16. Gravagna, K.; Becker, A.; Valeris-Chacin, R.; Mohammed, I.; Tambe, S.; Awan, F.A.; Toomey, T.L.; Basta, N.E. Global assessment of national mandatory vaccination policies and consequences of non-compliance. Vaccine 2020, 38, 7865–7873. [Google Scholar] [CrossRef] [PubMed]
  17. Wise, J. COVID-19: France and Greece make vaccination mandatory for healthcare workers. Br. Med. J. 2021, 374, n1797. [Google Scholar]
  18. Graeber, D.; Schmidt-Petri, C.; Schröder, C. Attitudes on voluntary and mandatory vaccination against COVID-19: Evidence from Germany. PLoS ONE 2021, 16, e0248372. [Google Scholar] [CrossRef] [PubMed]
  19. Smith, D.T.; Attwell, K.; Evers, U. Support for a COVID-19 vaccine mandate in the face of safety concerns and political affiliations: An Australian study. Politics 2022, 42, 480–491. [Google Scholar] [CrossRef]
  20. Largent, E.A.; Persad, G.; Sangenito, S.; Glickman, A.; Boyle, C.; Emanuel, E.J. US public attitudes toward COVID-19 vaccine mandates. JAMA Netw. Open 2020, 3, e2033324. [Google Scholar] [CrossRef]
  21. Gagneux-Brunon, A.; Botelho-Nevers, E.; Bonneton, M.; Peretti-Watel, P.; Verger, P.; Launay, O.; Ward, J.K. Public opinion on a mandatory COVID-19 vaccination policy in France: A cross-sectional survey. Clin. Microbiol. Infect. 2022, 28, 433–439. [Google Scholar] [CrossRef] [PubMed]
  22. Giannouchos, T.V.; Steletou, E.; Saridi, M.; Souliotis, K. Mandatory vaccination support and intentions to get vaccinated for COVID-19: Results from a nationally representative general population survey in October 2020 in Greece. J. Eval. Clin. Pract. 2021, 27, 996–1003. [Google Scholar] [CrossRef]
  23. Stead, M.; Ford, A.; Eadie, D.; Biggs, H.; Elliott, C.; Ussher, M.; Bedford, H.; Angus, K.; Hunt, K.; MacKintosh, A.M. A “step too far” or “perfect sense”? A qualitative study of British adults’ views on mandating COVID-19 vaccination and vaccine passports. Vaccine 2022, 40, 7389–7396. [Google Scholar] [CrossRef]
  24. Giannakou, K.; Kyprianidou, M.; Heraclides, A. Attitudes and determinants of mandatory vaccination against COVID-19 among the general population of Cyprus: A nationwide cross-sectional study. Vaccines 2022, 10, 438. [Google Scholar] [CrossRef]
  25. Tong, A.; Sainsbury, P.; Craig, J. Consolidated criteria for reporting qualitative research (COREQ): A 32-item checklist for interviews and focus groups. Int. J. Qual. Health Care 2007, 19, 349–357. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  26. Braun, V.; Clarke, V. Using thematic analysis in psychology. Qual. Res. Psychol. 2006, 3, 77–101. [Google Scholar] [CrossRef] [Green Version]
  27. Ritchie, J.; Lewis, J.; Nicholls, C.M.; Ormston, R. Qualitative Research Practice: A Guide for Social Science Students and Researchers; Sage: Newcastle upon Tyne, UK, 2013. [Google Scholar]
  28. Bradley, E.H.; Curry, L.A.; Devers, K.J. Qualitative data analysis for health services research: Developing taxonomy, themes, and theory. Health Serv. Res. 2007, 42, 1758–1772. [Google Scholar] [CrossRef] [Green Version]
  29. Hickey, G.; Kipping, C. A multi-stage approach to the coding of data from open-ended questions. Nurse Res. 1996, 4, 81–91. [Google Scholar] [CrossRef]
  30. Guba, E.G.; Lincoln, Y.S. Competing paradigms in qualitative research. Handb. Qual. Res. 1994, 2, 105. [Google Scholar]
  31. Jennings, W.; Stoker, G.; Bunting, H.; Valgarðsson, V.O.; Gaskell, J.; Devine, D.; McKay, L.; Mills, M.C. Lack of trust, conspiracy beliefs, and social media use predict COVID-19 vaccine hesitancy. Vaccines 2021, 9, 593. [Google Scholar] [CrossRef]
  32. Watermeyer, J.; Scott, M.; Kapueja, L.; Ware, L.J. To trust or not to trust: An exploratory qualitative study of personal and community perceptions of vaccines amongst a group of young community healthcare workers in Soweto, South Africa. Health Policy Plan. 2022, 37, 1167–1176. [Google Scholar] [CrossRef] [PubMed]
  33. Woodhead, C.; Onwumere, J.; Rhead, R.; Bora-White, M.; Chui, Z.; Clifford, N.; Connor, L.; Gunasinghe, C.; Harwood, H.; Meriez, P. Race, ethnicity and COVID-19 vaccination: A qualitative study of UK healthcare staff. Ethn. Health 2022, 27, 1555–1574. [Google Scholar] [CrossRef] [PubMed]
  34. Paul, E.; Steptoe, A.; Fancourt, D. Attitudes towards vaccines and intention to vaccinate against COVID-19: Implications for public health communications. Lancet Reg. Health-Eur. 2021, 1, 100012. [Google Scholar] [CrossRef] [PubMed]
  35. Neumann-Böhme, S.; Varghese, N.E.; Sabat, I.; Barros, P.P.; Brouwer, W.; van Exel, J.; Schreyögg, J.; Stargardt, T. Once We Have It, Will We Use It? A European Survey on Willingness to Be Vaccinated against COVID-19; Springer: Berlin/Heidelberg, Germany, 2020; Volume 21, pp. 977–982. [Google Scholar]
  36. Rhodes, A.; Hoq, M.; Measey, M.-A.; Danchin, M. Intention to vaccinate against COVID-19 in Australia. Lancet Infect. Dis. 2021, 21, e110. [Google Scholar] [CrossRef] [PubMed]
  37. Benham, J.L.; Lang, R.; Kovacs Burns, K.; MacKean, G.; Léveillé, T.; McCormack, B.; Sheikh, H.; Fullerton, M.M.; Tang, T.; Boucher, J.-C. Attitudes, current behaviours and barriers to public health measures that reduce COVID-19 transmission: A qualitative study to inform public health messaging. PLoS ONE 2021, 16, e0246941. [Google Scholar] [CrossRef]
  38. Dillard, J.P.; Tian, X.; Cruz, S.M.; Smith, R.A.; Shen, L. Persuasive messages, social norms, and reactance: A study of masking behavior during a COVID-19 campus health campaign. Health Commun. 2021, 38, 1338–1348. [Google Scholar] [CrossRef]
  39. Allington, D.; McAndrew, S.; Moxham-Hall, V.; Duffy, B. Coronavirus conspiracy suspicions, general vaccine attitudes, trust and coronavirus information source as predictors of vaccine hesitancy among UK residents during the COVID-19 pandemic. Psychol. Med. 2023, 53, 236–247. [Google Scholar] [CrossRef]
  40. Sallam, M.; Dababseh, D.; Eid, H.; Al-Mahzoum, K.; Al-Haidar, A.; Taim, D.; Yaseen, A.; Ababneh, N.A.; Bakri, F.G.; Mahafzah, A. High rates of COVID-19 vaccine hesitancy and its association with conspiracy beliefs: A study in Jordan and Kuwait among other Arab countries. Vaccines 2021, 9, 42. [Google Scholar] [CrossRef]
  41. Mousoulidou, M.; Christodoulou, A.; Siakalli, M.; Argyrides, M. The Role of Conspiracy Theories, Perceived Risk, and Trust in Science on COVID-19 Vaccination Decisiveness: Evidence from Cyprus. Int. J. Environ. Res. Public Health 2023, 20, 2898. [Google Scholar] [CrossRef]
Table 1. Sociodemographic characteristics of all participants.
Table 1. Sociodemographic characteristics of all participants.
Sociodemographic CharacteristicsOverall (N = 311)
Mean Age (SD)39.3 (10.5)
Gender (N a (%))
 Female 183 (59.4)
 Male 125 (40.6)
Geographical area (N b (%))
 Nicosia146 (48.3)
 Limassol 66 (21.9)
 Larnaca 48 (15.9)
 Paphos22 (7.3)
 Ammochostos 20 (6.6)
Occupation (N c (%))
 Private employee129 (42.2)
 State employee82 (26.8)
 Freelance 51 (16.7)
 Student 10 (3.3)
 Unemployed20 (6.4)
 Housewife 7 (2.3)
 Retired 7 (2.3)
Marital status (N d (%))
 Married/In cohabitation224 (73.0)
 Unmarried 60 (19.5)
 Divorced/separated/widowed 23 (7.5)
Underage children living in the household (N e (%))
 No 99 (36.7)
 Yes 170 (63.3)
Healthcare professional (N a (%))
 No 274 (89.0)
 Yes 34 (11.0)
Education level (N d (%))
 Up to secondary education46 (15.0)
 Undergraduate education152 (49.5)
 Postgraduate education109 (35.5)
Annual income (N f (%))
 Low (≤EUR 6500)38 (12.6)
 Moderate (EUR 6500–19,500)107 (35.6)
 High (>EUR 19,500)156 (51.8)
a N = 308; b N = 302; c N = 306; d N = 307; e N = 269; f N = 301.
Table 2. Information about participants’ COVID-19 vaccination status.
Table 2. Information about participants’ COVID-19 vaccination status.
COVID-19 Vaccination Overall (N = 311)
Vaccination status (N a (%))
 No198 (64.9)
 Yes107 (35.1)
Number of doses (N b (%))
 19 (8.3)
 267 (61.5)
 333 (30.2)
Type of COVID-19 vaccine (N b (%))
 Pfizer76 (69.7)
 Moderna 12 (11.0)
 Astra Zeneca 13 (11.9)
 Johnson & Johnson5 (4.7)
 Combination3 (2.7)
Intention to receive another dose if requested (N c (%))
 Not at all38 (28.6)
 Little 8 (6.0)
 Moderate 11 (8.2)
 A lot38 (28.6)
 Very much38 (28.6)
Believe that the vaccine helped to prevent COVID-19 disease? (N d (%))
 Not at all39 (25.8)
 Little 16 (10.6)
 Moderate 33 (21.9)
 A lot28 (18.5)
 Very much35 (23.2)
If you have not received the COVID-19 vaccine to date, do you plan to receive it? (N e (%))
 I do not know24 (12.0)
 No 173 (86.5)
 Yes 3 (1.5)
They belong to a vulnerable group (diabetic, immunosuppressed, etc.) for whom vaccination is recommended (N f (%))
 I do not know14 (4.6)
 No 257 (84.8)
 Yes 32 (10.6)
a N = 305; b N = 109; c N = 133; d N = 151; e N = 200; f N = 303.
Table 3. Themes and categories identified from qualitative data analysis.
Table 3. Themes and categories identified from qualitative data analysis.
Thematic AreaCategory
Vaccination Perceptions and ChallengesUniversal Mandatory Vaccination
Targeted Mandatory Vaccination
Insufficient Research
Inadequate Communication and Information Discrepancies
Protective Measures
Interdisciplinary Collaboration
Vaccine Side Effects and Lack of State Responsibility
Natural Immunity
COVID-19 Denial and Perceived Risk
Mandatory Vaccination Policies and Societal ImpactTwo-tiered Society
Human Rights
Reactive Response to Obligation
Political and Economical Expediencies
Social Impact Outweighed
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Giannakou, K.; Soulioti, E.; Kyprianidou, M. Mandatory Vaccination against COVID-19: Qualitative Analysis of Responses to Open-Ended Questions. Hygiene 2023, 3, 291-305. https://doi.org/10.3390/hygiene3030021

AMA Style

Giannakou K, Soulioti E, Kyprianidou M. Mandatory Vaccination against COVID-19: Qualitative Analysis of Responses to Open-Ended Questions. Hygiene. 2023; 3(3):291-305. https://doi.org/10.3390/hygiene3030021

Chicago/Turabian Style

Giannakou, Konstantinos, Eftychia Soulioti, and Maria Kyprianidou. 2023. "Mandatory Vaccination against COVID-19: Qualitative Analysis of Responses to Open-Ended Questions" Hygiene 3, no. 3: 291-305. https://doi.org/10.3390/hygiene3030021

Article Metrics

Back to TopTop