Next Article in Journal
Modelling SARS-CoV-2 Binding Antibody Waning 8 Months after BNT162b2 Vaccination
Next Article in Special Issue
Homologous and Heterologous Anti-COVID-19 Vaccination Does Not Induce New-Onset Formation of Autoantibodies Typically Accompanying Lupus Erythematodes, Rheumatoid Arthritis, Celiac Disease and Antiphospholipid Syndrome
Previous Article in Journal
Autochthonous Outbreak of SARS-CoV-2 Omicron Variant in Booster-Vaccinated (3 Doses) Healthcare Workers in Southern Italy: Just the Tip of the Iceberg?
Previous Article in Special Issue
Antibody Response of Combination of BNT162b2 and CoronaVac Platforms of COVID-19 Vaccines against Omicron Variant
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
Vaccines
Volume 10
Issue 2
10.3390/vaccines10020284
vaccines-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Propensity-Score-Matched Evaluation of Adverse Events Affecting Recovery after COVID-19 Vaccination: On Adenovirus and mRNA Vaccines

by
Chang-Sik Son
*,
Sang-Hyeon Jin
and
Won-Seok Kang
*
Division of Intelligent Robot, Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu 42988, Korea
*
Authors to whom correspondence should be addressed.
Vaccines 2022, 10(2), 284; https://doi.org/10.3390/vaccines10020284
Submission received: 5 January 2022 / Revised: 10 February 2022 / Accepted: 11 February 2022 / Published: 13 February 2022
(This article belongs to the Collection COVID-19 Vaccines and Vaccination)
Browse Figures
Versions Notes

Abstract

:
This study aimed to observe adverse events following immunisation (AEFIs) that affected recovery within two weeks after COVID-19 vaccination and investigate their risks in propensity-score-matched populations. Data were collected from 447,346 reports from the VAERS between 1 January 2021 and 31 July 2021. Propensity-score-matched populations were constructed by adjusting for demographic characteristics and 11 underlying diseases in eligible subjects who received 1 of 3 COVID-19 vaccines: 19,462 Ad26.COV2.S, 120,580 mRNA-1273, and 100,752 BNT162b2. We observed that 88 suspected AEFIs (22 in Ad26.COV2.S, 62 in mRNA-1273, and 54 in BNT162b2) were associated with an increased risk of delayed recovery within 2 weeks after COVID-19 vaccinations. Nervous system, musculoskeletal and connective tissue, gastrointestinal, skin, and subcutaneous tissue disorders were the most common AEFIs after COVID-19 vaccination. Interestingly, four local and systemic reactions affected recovery in different vaccine recipients during our study period: asthenic conditions and febrile disorders in Ad26.COV2.S and mRNA-1273; general signs and symptoms in mRNA-1273 and BNT162b2; injection site reactions in Ad26.COV2.S and BNT162b2. Although it is necessary to confirm a causal relationship with COVID-19 vaccinations, some symptoms, including paralysis, allergic disorders, breathing abnormalities, and visual impairment, may hinder the recovery of these recipients.

Graphical Abstract

1. Introduction

At the end of 2019, a novel coronavirus, which is now known as severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), was identified as the cause of a cluster of pneumonia cases in Wuhan, a city in the Hubei Province of China [1]. The rapidly expanding COVID-19 pandemic has impacted all areas of daily life. As of 3 June 2021, COVID-19 outbreaks were reported in 213 countries, with over 171 million confirmed cases and more than 3.68 million deaths [2].
Various vaccines to prevent SARS-CoV-2 infection are available in different countries. In the United States, COVID-19 mRNA vaccines—BNT162b2 (Pfizer-BioNTech COVID-19 vaccine), mRNA-1273 (Moderna COVID-19 vaccine), and COVID-19 adenovirus vector vaccine Ad26.COV2.S (Janssen COVID-19 vaccine)—received emergency use authorisation for use in individuals 18 years and older. BNT162b2 is also authorised for use in children and adolescents 12–17 years old [3,4,5]. In large placebo-controlled trials, these vaccines were shown to be highly effective in preventing laboratory-confirmed COVID-19, especially in patients with severe or critical disease [6,7,8].
After COVID-19 vaccination, local reactions, such as pain or redness at the injection site, were mild. Moderate systemic reactions, such as fatigue, myalgia, arthralgia, and headache, were observed in less than 50% of mRNA vaccine recipients, starting about 15 h after vaccination and resolving in most recipients by day 2 without sequelae [6,7]. However, very rare but severe vaccine-associated adverse events (AEs) have occurred, such as anaphylaxis [9] and myocarditis [10] with mRNA vaccines, and thrombocytopenia [11] and Guillain–Barre syndrome [12] with Ad26.COV2.S. Possible facial nerve paralysis [7,13,14], including Bell’s palsy, has been reported among mRNA vaccine recipients.
There are reports of unknown and unverified moderate-to-severe adverse reactions after COVID-19 vaccination, but a causal relationship has not been medically confirmed. Therefore, real-world monitoring of vaccine safety is crucial to help understand the suspected AEs following immunisation (AEFIs), assess their incidence, and determine risk factors to inform potential vaccine contraindications [15]. There are two monitoring systems in place to assess safety in post-licensure settings: the Vaccine Adverse Event Reporting System (VAERS), which is a passive system that relies on others reporting adverse events; and the Vaccine Safety Datalink, which is an active system that reviews databases or conducts studies to identify AEs [16]. The VAERS is a vaccine surveillance programme that is co-managed by the US Centre for Disease Control and Prevention and the US Food and Drug Administration, which is responsible for monitoring the safety of pharmaceutical drugs after being released to the market. Collecting useful information on AEs from various vaccines provides many advantages [17,18,19]: (1) detecting new, unusual, or rare AEs; (2) monitoring reporting trends that might reflect true increases in known AEs; (3) identifying potential risk factors for particular AEs; and (4) detecting persistent use safety problems and administration errors.
Recently, several studies investigated moderate-to-severe [15] or serious AEs (e.g., death and hospitalisation) [20] and underlying health conditions associated with deaths [15,21] among COVID-19-vaccinated cases. For example, the deaths reported by the VAERS were in vaccinated people aged 85 and older at long-term care facilities. These patients were frail older people with serious underlying diseases, including hypertension, diabetes, anaemia, heart failure, and chronic obstructive pulmonary disease [21]. Furthermore, other underlying diseases, such as obesity (i.e., body mass index > 30 kg/m2) [22,23,24,25], chronic kidney disease [25,26,27,28], cancer [25,29], and allergy [9,30,31], are known possible epidemiologic risk factors for severe COVID-19.
To examine various COVID-19-vaccine-induced AEs, we collected comprehensive data including 1122 AEFIs from three COVID-19 vaccines (i.e., Ad26.COV2.S, mRNA-1273, and BNT162b2), reported to the VAERS from 1 January 2021 to 31 July 2021. AEs were examined in different propensity-score-matched populations, adjusted for demographic characteristics (e.g., age and sex) and 11 underlying diseases (e.g., obesity, hypertension, diabetes, heart failure, pulmonary disease, stroke, allergy). Major AEs and their adjusted hazard ratios were estimated using a multivariate Cox regression model.

2. Materials and Methods

2.1. Data Source and Collection

We used a dataset of vaccinated cases from the USA containing various types of vaccine-related information. The raw dataset comprised three different files collected by the VAERS database [17,18,19] for subjects vaccinated from 1 January 2021 to 31 July 2021. The VAERS data included demographic and medical history, symptoms for vaccine-associated AEs, and type of vaccine used. In the raw dataset, we collected all cases classified with the COVID-19 vaccine type. There were some cases with the same identification number (ID), and the four vaccine manufacturers were Janssen, Moderna, Pfizer-BioNTech, and Unknown. We considered medical records including general information (i.e., VAERS ID, age, sex, current illness, and medical history at the time of vaccination), vaccination information (i.e., vaccine type, manufacturer, vaccination date, and AE onset date), and recovery from vaccine-induced AEs.
As illustrated in Figure 1, the three files used a common VAERS ID as the primary key. To extract all cases of COVID-19 vaccination, the VAERS IDs were initially screened and used to extract some medical information, such as demographics, prediagnosed illness or medical histories, and AEs for individuals who were vaccinated against COVID-19. In particular, AEs were defined using standard medical terms in The Medical Dictionary for Regulatory Activities (MedDRA) [32]. Each case had a maximum of five terms, with each term listed for the corresponding VAERS ID.

2.2. Standard Medical Terms for Vaccine Adverse Events

MedDRA is organised in a five-level hierarchy. The highest or broadest level is the system organ class (SOC), which is further divided into high-level group terms (HLGT), high-level terms (HLT), preferred terms (PT), and the most granular lowest-level terms (LLT); the hierarchy includes 27, 337, 1737, 24,820, and 83,291 terms in release version 24.0 (March 2021) [33], respectively.
In cases reported to the VAERS database, various AEFIs were provided at the PT level. However, PTs are highly fragmented into signs, symptoms, diagnosis, investigation, or medical procedures, which might lead to failures in identifying differences in the AE incidence [34,35]. Accordingly, we converted the standard terms associated with all AEFIs (expressed as PTs) into HLTs. We collected 1122 COVID-19-vaccine-induced AEs and analysed their relationships at the SOC level. Each AE encoded a binary value indicating whether the COVID-19 vaccine recipient had certain adverse symptoms, where 1 indicated YES and 0 indicated NO.

2.3. Underlying Diseases

We examined medical records related to individual medical histories (i.e., prediagnosed illnesses) at the time of COVID-19 vaccination. Among these records, we extracted 11 underlying disabilities associated with risk factors for severe COVID-19 by using the keyword matching approach that considers representative terms for each illness. The underlying diseases were obesity [22,23,24,25], hypertension [25,36,37], diabetes [25,37], atrial fibrillation [25,38], heart failure [25,39,40], kidney disease [25,26,27,28], pulmonary disease [25,37,41], stroke [42], asthma [25,43,44], cancer [25,29], and allergies [9,30,31]. These terms were selected based on a literature review of previous studies.

2.4. Study Design

The study population included 447,346 cases: 45,848 for Janssen vaccination, 197,006 for Moderna vaccination, 203,478 for Pfizer-BioNTech vaccination, and 1014 for unknown. After excluding unknown COVID-19 vaccinations, 431,341 individuals with a unique VAERS ID were screened. The primary endpoint was all adverse symptoms reported within two weeks after COVID-19 vaccination. Accordingly, we considered 256,994 individuals as eligible subjects according to the following exclusion criteria: 2 or more different COVID-19 vaccinations; missing age data or under 20 years of age; missing data on the number of days associated with vaccine-related adverse events; cases reported after two weeks; or unreported recovery from vaccine-induced AEs. Study subjects were categorised into 7 and 3 groups according to age and gender, respectively: (1) 20–29, 30–39, 40–49, 50–59, 60–69, 70–79, and ≥80 years; (2) unknown, male, and female. After propensity score matching, 19,462, 120,580, and 100,752 individuals were included in the analysis, as illustrated in Figure 2. This workflow shows the numbers of individuals (N) excluded at different stages and the number of cases before/after propensity score matching.

2.5. Statistical Analysis

2.5.1. Propensity Score Matching

Data pertaining to our study population collected from the VAERS database includes various age groups, genders, and various underlying diseases for each individual at the time of COVID-19 vaccination. The type of vaccine used might depend on individual characteristics, including age, sex, and comorbidities, and this choice could affect the vaccine-associated AEs and recovery from them. Therefore, we used propensity score matching (PSM) using the nearest neighbour method with a 1:1 matching ratio without replacement, to reduce the bias due to these confounding variables between two groups (i.e., recovery and no recovery from COVID-19-vaccine-induced AEs). The propensity score, specifically the conditional probability of not recovering from COVID-19-vaccine-induced AEs, given the observed covariates (i.e., in our study—age, sex, and 11 underlying diseases), was estimated through binomial logistic regression analysis. To rigorously control for confounding variables, the calliper width (i.e., considered only if the difference in the propensity score between paired subjects was within a prescribed range) was used during the PSM. The calliper threshold was examined between 0.25 and 0.1 of the standard deviation of the propensity score. A chi-square test was used as a criterion to determine the best calliper threshold, at which all covariates had no statistically significant difference (p > 0.05). The absolute standardised difference (ASD) [45] was set below 0.1 to confirm whether the two groups were well balanced.

2.5.2. Cox Proportional Hazards Regression Analysis

We considered 1122 COVID-19-vaccine-associated AEs (i.e., 753, 1019, and 1000 MedDRA HLTs for Ad26.COV2.S, mRNA-1273, and BNT 162b2, respectively) as independent variables, reported within 2 weeks from individuals being vaccinated with 1 type of the 3 vaccines. We examined the association between those AEs and recovery using a Cox proportional hazards regression analysis. The event of interest was defined as individuals who did not recover from the vaccine-induced AEs during our study period. We removed some AEs with a small sample size to resolve any monotonic likelihood issues that could occur in Cox proportional hazards regression analysis. These issues are observed when fitting a Cox model when at least one covariate estimate diverges to negative or positive infinity [46]. Accordingly, we selected AEs including the minimum number of cases with more than five adverse symptoms within the event (i.e., no recovery group from vaccine-induced AEs). Based on this criterion, we performed univariate Cox proportional hazards regression analysis to screen for possible AEs that delayed recovery after COVID-19 vaccination. Statistically significant AEs were included in the multivariate Cox regression model for adjustment. The results of the Cox regression model are presented as HRs with 95% confidence intervals. Statistical significance was set at p < 0.05. The concordance index or C-index was used to identify the discrimination power of the multivariate Cox regression model.

2.5.3. Experimental Environment and Implementation

All experiments were implemented and evaluated using the following hardware and software: AMD Ryzen 9 3900X 12-Core Processor @ 4.19GHz CPU; 128GB RAM; Windows 10; IntelliJ IDEA 2019.2.4 (Ultimate Edition) using Python (version 3.7); numpy (version 1.19.1), pandas (version 1.0.3), scikit-learn (version 0.22.2), scipy (version 1.4.1), statsmodels (version 0.12.2), and lifeline (version 0.26.0).

3. Results

3.1. Data Characteristics

Among the 256,994 individuals, the average age of each group was 50.01 ± 16.59 years: 44.69 ± 15.1 for Ad26.COV2.S; 51.45 ± 16.76 for mRNA-1273; and 49.4 ± 16.43 for BNT162b2. The total gender distribution was 718 unknown (0.28%), 66,280 men (25.79%), and 189,996 women (73.93%). The proportion of women was higher than that of men (approximately 66% to 76%). Three underlying disabilities, namely hypertension (8.31%), asthma (4.87%), and diabetes (4.66%), were the most common medical history terms or prediagnosed illnesses of individuals at the time of vaccination. In particular, the proportion was higher in the mRNA-1273 vaccine than the other two vaccines. Moreover, the AE onset interval for each COVID-19 vaccine was 2.07 ± 3.18 days: 1.67 ± 3.1 for Ad26.COV2.S; 2.51 ± 3.43 for mRNA-1273; and 1.64 ± 2.8 for BNT162b2. The AE onset interval of mRNA-1273 vaccine recipients was approximately 0.84–0.87 days longer than those of the other vaccine recipients. The results revealed that 128,821 (50.13%) recovered and the remaining 128,173 (49.87%) did not recover from AEFIs (Table 1).
Figure 3 shows the cumulative probability of AE onset for two weeks after vaccination with the three COVID-19 vaccines. Approximately 70% of the patients experienced local and systemic adverse reactions to the vaccine within one day, such as injection site pain, headache, muscle-related symptoms, fever, nausea, or vomiting. For Ad26.COV2.S, mRNA-1273, and BNT162b2, AE onset was observed in 11,918 (54.08%), 48,174 (37.98%), and 50,392 (46.61%) individuals on day 0 and 4936 (22.4%), 33,685 (26.56%), and 29,146 (26.96%) individuals on day 1, respectively. As noted in previous studies, there was a decreasing trend thereafter [6,7,8].

3.2. Major AEs and the Risk Levels Associated with the Three COVID-19 Vaccines

3.2.1. Janssen COVID-19 Vaccine

The distribution of demographic characteristics and the 11 underlying disabilities between the non-recovered and recovered groups, before and after PSM, are shown in Table S1. Before PSM, there were significant differences (p < 0.05) in age and sex. A total of 9 underlying diseases—namely hypertension (7.92%), asthma (5.63%), diabetes (4.55%), allergies (2.88%), obesity (1.54%), pulmonary disease (1.31%), cancers (1.08%), heart failure (0.8%), and kidney disease (0.45%)—were relatively high in the non-recovered group, with statistically significant differences compared with the recovered group (p < 0.05). After PSM, 2 groups were paired with a 1:1 matching ratio and the calliper width was adjusted to 0.25. This caused the differences between these groups to disappear (ASD < 0.1, p > 0.05).
In the propensity-score-matched population, 91 adverse symptoms were associated with the risk of developing an AE estimated with a univariate Cox proportional hazard regression model. The concordance index of a multivariate Cox regression model was then 0.6518. Overall, 36 AEs were associated with the Janssen COVID-19 vaccination (see Figure S1). Of these, 22 were major vaccine-induced AEs with extended recovery times. The adjusted HRs were distributed from 1.06 to 2.6 and grouped into 11 SOC terms (Table 2).
Among the SOCs, five were common AEFIs, such as nervous system disorders (38.73%), general disorders and administration site conditions (29.05%), musculoskeletal and connective tissue disorders (23.71%), gastrointestinal disorders (20.18%), and skin and subcutaneous tissue disorders (15.26%). In particular, some AEs—headaches (36.23%; HR, 1.16; 95% CI, 1.11–1.22), asthenic conditions (29.61%; HR, 1.06; 95% CI, 1.01–1.12), pain and discomfort (26.1%; HR, 1.17; 95% CI, 1.12–1.23), febrile disorders (25.94%; HR, 1.2; 95% CI, 1.14–1.27), musculoskeletal and connective tissue pain and discomfort (19.87%; HR, 1.12; 95% CI, 1.06–1.18), nausea and vomiting symptoms (19.09%; HR, 1.11; 95% CI, 1.05–1.18), and dermal and epidermal conditions (12.99%; HR, 1.08; 95% CI, 1.02–1.15)—generally exhibited an increased risk of non-recovery in individuals compared with recovered people who received the Ad26.COV2.S vaccine. The remaining six SOCs were also rare AEFIs. Some of these, such as allergic conditions (2.13%; HR, 1.27; 95% CI, 1.1–1.46), cognitive and attention disorders and disturbances (1.2%; HR, 1.21; 95% CI, 1.01–1.46), pharyngeal disorders (0.69%; HR, 1.34; 95% CI, 1.05–1.72), and cholecystitis and cholelithiasis (0.07%; HR, 2.6; 95% CI, 1.21–5.63), were identified as risk factors related to delayed recovery of vaccine recipients.

3.2.2. Moderna COVID-19 Vaccine

Table S2 shows the distribution of demographic characteristics and 11 underlying disabilities between individuals vaccinated with the Moderna vaccine in the non-recovered and recovered groups before and after PSM. Before PSM, there were significant differences (p < 0.05) in age and sex. Nine underlying diseases, namely, hypertension (10.16%), diabetes (5.79%), asthma (5.38%), allergies (2.81%), obesity (1.77%), cancers (1.47%), pulmonary disease (1.23%), heart failure (0.95%), and stroke (0.25%), were relatively high in the non-recovered group, with statistically significant differences compared with the recovered group (p < 0.05). After PSM, two groups were paired with a 1:1 matching ratio, the calliper width was adjusted to 0.15, and the differences between these groups disappeared (ASD < 0.1, p > 0.05).
In the propensity-score-matched population, we identified 193 adverse symptoms associated with the risk of developing an AE, estimated with a univariate Cox proportional hazard regression model. The concordance index of a multivariate Cox regression model was 0.6358. Overall, 94 AEs were associated with the Moderna COVID-19 vaccine (see Figure S2). Of these, 62 were major vaccine-induced AEs; their HRs ranged from 1.05 to 3.47 and were grouped into 21 SOC terms (Table 3).
A total of 6 AEFIs—general disorders and administration site conditions (27.29%), nervous system disorders (24.24%), musculoskeletal and connective tissue disorders (19.11%), gastrointestinal disorders (13.75%), skin and subcutaneous tissue disorders (12.09%), and cardiac disorders (10.98%)—were the most common (similar to the Janssen vaccine results). However, the detailed AEs and their distribution were different in the non-recovered group. Among these, some AEs exhibited increased risk of non-recovery cases: (1) asthenic conditions (20.65%; HR, 1.06; 95% CI, 1.03–1.08), pain and discomfort (19.28%; HR, 1.16; 95% CI, 1.02–1.35), febrile disorders (15.86%; HR, 1.14; 95% CI, 1.11–1.17), and general signs and symptoms (15.7%; HR, 1.1; 95% CI, 1.07–1.12); (2) headaches (19.41%; HR, 1.2; 95% CI, 1.17–1.23); (3) musculoskeletal and connective tissue pain and discomfort (16.09%; HR, 1.16; 95% CI, 1.13–1.18), joint-related signs and symptoms (8.65%; HR, 1.25; 95% CI, 1.21–1.29), and muscle pain (6.17%; HR, 1.05; 95% CI, 1.02–1.09); (4) nausea and vomiting symptoms (11.71%; HR, 1.07; 95% CI, 1.04–1.1); (5) dermal and epidermal conditions (12.09%; HR, 1.07; 95% CI, 1.05–1.1); (6) cardiac signs and symptoms (9.05%; HR, 1.07; 95% CI, 1.04–1.1). We also identified rare AEs with relatively increased risk in the non-recovered group when compared with the recovered group. In particular, decreased fluid intake (0.03%, HR, 1.83; 95% CI, 1.14–2.94), hypoglycaemic conditions (0.02%; HR, 2.13; 95% CI, 1.21–3.73), hepatobiliary signs and symptoms (0.02%; HR, 1.86; 95% CI, 1.11–3.11), clostridia infections (0.02%; HR, 3.47; 95% CI, 2.02–5.94), and chest and respiratory tract injuries (0.01%; HR, 2.42; 95% CI, 1.21–4.85) were observed.

3.2.3. Pfizer-BioNTech COVID-19 Vaccine

Table S3 shows the demographic characteristics and 11 underlying disabilities between persons treated with the Pfizer vaccine in the non-recovered and recovery groups before and after PSM. Before PSM, there were significant differences (p < 0.05) in age and sex. A total of 3 underlying diseases—diabetes (4.24%), pulmonary disease (0.86%), and heart failure (0.7%)—were relatively higher in the non-recovered group than in the recovered group (p < 0.05). However, asthma (4.26%) exhibited a relatively low distribution, with a statistically significant difference (p < 0.05) in the non-recovered compared with the recovered group. After PSM, 2 groups were paired using a 1:1 matching ratio and the calliper width was adjusted to 0.1. This caused the differences to disappear between these groups (ASD < 0.1, p > 0.05).
In the propensity-score-matched population, we observed 208 adverse symptoms associated with the risk of developing an AE estimated with a univariate Cox proportional hazard regression model. The concordance index of a multivariate Cox regression model was 0.6237. Overall, 92 AEs were associated with Pfizer-BioNTech COVID-19 vaccination (see Figure S3). Of these, 54 were the major vaccine-induced AEs; their hazard ratios ranged from 1.04 to 3.37 and were grouped into 16 SOC terms (Table 4).
A total of 5 adverse reactions were more common in the non-recovered group: general disorders and administration site conditions (31.8%), nervous system disorders (30.13%), musculoskeletal and connective tissue disorders (19.55%), gastrointestinal disorders (17.97%), and skin and subcutaneous tissue disorders (15.05%). Among these, 7 AEs—including headaches (22.9%, HR, 1.13; 95% CI, 1.11–1.16), pain and discomfort (19.47%; HR, 1.08; 95% CI, 1.05–1.11), musculoskeletal and connective tissue pain and discomfort (15.98%; HR, 1.19; 95% CI, 1.16–1.22), nausea and vomiting symptoms (13.76%; HR, 1.04; 95% CI, 1.01–1.07), general signs and symptoms (13.32%; HR, 1.07; 95% CI, 1.04–1.1), dermal and epidermal conditions (11.3%; HR, 1.2; 95% CI, 1.16–1.23), and joint-related signs and symptoms (10.22%; HR, 1.12; 95% CI, 1.09–1.16)—generally exhibited an increased risk of non-recovery compared with that in recovered people who received the BNT162b2 vaccine. Some other AEs—including lymphatic system disorders (7.41%; HR, 1.25; 95% CI, 1.21–1.29), breathing abnormalities (6.15%; HR, 1.12; 95% CI, 1.08–1.17), ischaemic coronary artery disorders (5.45%; HR, 1.05; 95% CI, 1.01–1.1), choroid and vitreous haemorrhages and vascular disorders (0.01%; HR, 3.37; 95% CI, 1.48–7.7), and cardiac infections (0.01%; HR, 2.72; 95% CI, 1.2–6.14)—were rare, but had elevated risk of non-recovery compared with that in recovered people who received the BNT162b2 vaccine.

4. Discussion

Our study focused on analysing major AEFIs and whether their risks affect the recovery after COVID-19 vaccination. We investigated all possible AEs of three COVID-19 vaccines based on data reported to the VAERS database from 1 January 2021 to 31 July 2021 and analysed their relationships for different SOCs.
Our findings showed that among the 21 SOCs, 5 terms (general disorders and administration site conditions, nervous system, musculoskeletal and connective tissue, gastrointestinal, and skin and subcutaneous tissue disorders) were associated with generally more common AEFIs in the propensity-score-matched populations. We confirmed that the most prevalent AEFIs were pain (Ad26.COV2.S, 26.1%; mRNA-1273, 19.28%; BNT162b2, 19.47%), headaches (Ad26.COV2.S, 36.23%; mRNA-1273, 19.41%; BNT162b2, 22.9%), joint-related signs and symptoms (Ad26.COV2.S, 10.5%; mRNA-1273, 8.65%; BNT162b2, 10.22%), muscle pain (Ad26.COV2.S, 10.34%; mRNA-1273, 6.17%; BNT162b2, 7.59%), musculoskeletal and connective tissue pain and discomfort (Ad26.COV2.S, 19.87%; mRNA-1273, 16.09%; BNT162b2, 15.98%), nausea and vomiting symptoms (Ad26.COV2.S, 19.09%; mRNA-1273, 11.71%; BNT162b2, 13.76%), and dermal and epidermal conditions (Ad26.COV2.S, 12.99%; mRNA-1273, 12.09%; BNT162b2, 11.3%). These conditions were observed in more than approximately 6% of the non-recovered cases who received each of the COVID-19 vaccines. A total of 4 adverse symptoms—namely, asthenic conditions (Ad26.COV2.S, 29.61%; mRNA-1273, 20.65%), general signs and symptoms (mRNA-1273, 15.7%; BNT162b2, 13.32%), febrile disorders (Ad26.COV2.S, 25.94%; mRNA-1273, 15.86%), and injection site reactions (Ad26.COV2.S, 11.2%; BNT162b2, 8.96%)—were regarded as the additional local and systemic reactions. Simultaneously, these were risk factors associated with recovery after COVID-19 vaccinations. These findings are consistent with the previous results demonstrating that local and systemic reactions were common [6,7,15]. Most were mild reactions that did not prevent daily activities and mainly occurred within the first two days after vaccination [47,48].
Rare neurological findings, especially facial paralysis, were noted in several previous studies [7,13,14,15]. A total of 7 cases suspected of facial paralysis were reported among 36,930 recipients during clinical phase 3 trials for mRNA vaccines [6,7]. A recent study demonstrated that neurological AEs after COVID-19 vaccination may be more than chance events, while reporting 18 cases of facial paralysis and other neurological events, such as syncope (37 cases) and seizure (12 cases), to the VAERS database in December 2020 [15]. Similar to the previous study, we identified that 200 recipients with suspected paralysis and paresis (excluding cranial nerve) did not recover after mRNA-1273 vaccination, even though they did not develop facial paralysis or Bell’s palsy. However, they showed a 1.17-fold increased risk compared with the recovered group. Seizure-related symptoms were observed in 169 and 222 non-recovered cases among recipients who received mRNA-1273 or BNT162b2, respectively; however, their risk degree was lower than that in the recovered group (See Figures S2 and S3).
Severe allergic reactions, including anaphylaxis, were very rarely reported as a crucial issue [30,31]. Anaphylaxis is diagnosed primarily based on clinical symptoms, signs, and a detailed description of the acute episode, including antecedent activities and events occurring within the preceding minutes–hours [49]. We did not find adverse cases and the risk of anaphylactic and anaphylactoid responses in the propensity-score-matched populations associated with mRNA vaccination. However, allergic conditions related to immune system disorders, including hypersensitivity reactions (type I, III, or IV), were adverse symptoms associated with adenovirus vector and mRNA vaccines. We identified that during our study period, 207, 751, and 716 cases were non-recovered after Ad26.COV2.S, mRNA-1273, and BNT162b2 vaccinations, respectively. Other adverse symptoms, such as angio-oedemas, yielded 1364 (2.26%) non-recovered cases after mRNA-1273 vaccination.
Another very rare but severe vaccine-associated AE involves unusual types of thrombotic events associated with thrombocytopenia. Thrombosis occurs at unusual sites, including cerebral venous sinuses and mesenteric vessels, and often at more than one site [11,50]. The pathogenetic mechanism underlying vaccine-induced immune thrombotic events was identified by Greinacher et al. [51]. The clinical features included increased D-dimer levels, decreased or normal fibrinogen levels, and the presence of antibodies against platelet factor 4 (PF4). High levels of antibodies against PF4 have been detected in the serum of almost all patients who received ChAdOx1 (AstraZeneca) vaccination [52]. Anti-PF4 antibodies, which mimic heparin-induced thrombocytopenia-like disease, cause platelet activation and may induce thrombosis in rare cases. Therefore, this pathophysiologic mechanism implies potential neurosurgical consequences that could be derived from at least the sequelae of cerebral sinus and vein thrombosis [51]. Most of these AEs occurred within 2 weeks after patients received the initial vaccine dose, especially in females under 60 years of age. In the US, the risk of this syndrome following Ad26.COV2.S vaccination was assessed as 3 cases per million overall, and 8.8 cases per million for women 30–49 years old [12]. Nevertheless, COVID-19 vaccine recipients should be made aware of the possible association and seek immediate care for signs and symptoms suggestive of thrombocytopenia or thrombotic complications, such as shortness of breath, chest pain, lower extremity oedema, persistent abdominal pain, unabatingly severe headache, blurred vision, and severe backache [53]. We observed some cases of peripheral embolism and thrombosis (143 in Ad26.COV2.S; 196 in mRNA-1273; 246 in BNT162b2) among recipients who received 1 of the 3 COVID-19 vaccines (See Figures S1–S3). Simultaneously, some AEFIs, such as pulmonary embolism and thrombosis (232 in mRNA-1273; 247 in BNT162b2) and thrombocytopaenia (35 in BNT162b2), were observed in non-recovered cases after mRNA vaccinations. However, their risk was lower in the recovery group after mRNA vaccination (See Figures S2 and S3). Furthermore, we confirmed that some AEFIs, such as breathing abnormalities (mRNA-1273, 4.3%; BNT162b2, 6.15%), visual impairment (mRNA-1273, 0.47%), and neurological visual problems (mRNA-1273, 1.2%; BNT162b2, 1.82%), might hinder the recovery of mRNA vaccine recipients. Some other symptoms were rare, but risk factors interfering with the recovery of vaccine recipients were: (1) ischaemic coronary artery disorders (Ad26.COV2.S, 6.72%; mRNA-1273, 3.34%; BNT162b2, 5.45%); (2) ocular disorders (Ad26.COV2.S, 1.98%; mRNA-1273, 1.08%; BNT162b2, 1.77%); (3) gastrointestinal and abdominal pain (Ad26.COV2.S, 5.26%; mRNA-1273, 1.87%; BNT162b2, 2.99%); (4) cognitive and attention disorders (Ad26.COV2.S, 1.2%; mRNA-1273, 0.69%; BNT162b2, 0.95%); and (5) disturbances in initiating and maintaining sleep (Ad26.COV2.S, 2.13%; mRNA-1273, 1.05%; BNT162b2, 1.57%).
Our study has several limitations. First, unlike clinical trials that report adverse reactions using standardised data collection procedures, the VAERS database includes various adverse cases [15] that may include reporting bias, over-reporting, or under-reporting of AEs [17], because clinicians are encouraged to report all AEFIs. Therefore, the actual mild–moderate or severe AEs may be rarer than our findings. Second, our study data may include individuals who received the SARS-CoV-2 test before our analytic period and exhibited some symptoms of SARS-CoV-2 infection. To reduce such potential bias, we attempted to demonstrate possible major AEFIs concerning each COVID-19 vaccine in the propensity-score-matched populations adjusted for age, sex, and 11 underlying diseases. However, unmeasured and residual confounding may have biased our estimates. For example, we did not know the individual occupation characteristics (e.g., virus exposure degree and protective equipment use) [54], race, and residence. Consequently, our study findings may involve unknown or unverified AEs, since the VAERS database does not provide medically confirmed or valid diagnostic evidence. Third, to investigate AEFIs related to COVID-19 vaccines, we used only limited covariates that satisfied a user-defined sample threshold in propensity-score-matched target outcomes. Even though it is possible to reduce the monotonic likelihood issue (i.e., the bias of maximum likelihood estimates) in the Cox regression model, further investigations will be necessary.

5. Conclusions

We found that an increased risk of local and systemic reactions (e.g., headache, joint-related symptoms, muscle pain, musculoskeletal and connective tissue pain, nausea or vomiting symptoms, dermal and epidermal conditions, and febrile disorders) was associated with delayed recovery in non-recovered cases of different propensity-score-matched populations when compared with the group that recovered after COVID-19 vaccination. Furthermore, we observed that some notable AEs, such as paralysis (excluding cranial nerve), allergic disorder, breathing abnormality, and visual impairment, might hinder the recovery of non-recovery cases; although, it is necessary to confirm a causal relationship with COVID-19 vaccination. In the future, we aim to perform an observational study on the effectiveness and side effects of heterologous prime–boost, particularly in preventing severe disease and infection with Delta (B.1.617.2) and Omicron (B.1.1.529) variants.

Supplementary Materials

The following supporting information can be downloaded at: https://www.mdpi.com/article/10.3390/vaccines10020284/s1. Table S1: Baseline characteristics of individuals vaccinated with the Janssen COVID-19 vaccine before and after PSM, Table S2: Baseline characteristics of individuals vaccinated with the Moderna COVID-19 vaccine before and after PSM, Table S3: Baseline characteristics of individuals vaccinated with the Pfizer-BioNTech COVID-19 vaccine before and after PSM, Figure S1: AE hazard ratios after Janssen COVID-19 vaccination estimated from a multivariate Cox model, Figure S2: AE hazard ratios after Moderna COVID-19 vaccination estimated from a multivariate Cox model, Figure S3: AE hazard ratios after Pfizer-BioNTech COVID-19 vaccination estimated from a multivariate Cox model.

Author Contributions

Conceptualisation, C.-S.S.; methodology, C.-S.S.; software, C.-S.S. and S.-H.J.; validation, C.-S.S. and S.-H.J.; investigation, all authors; data curation, all authors; writing—original draft preparation, C.-S.S.; writing—review and editing, all authors; project administration, C.-S.S. and W.-S.K. All authors have read and agreed to the published version of the manuscript.

Funding

This research was supported by the Strategy Technology Development Programs of the Ministry of Trade, Industry & Energy of Korea (No. 10077260), the Basic Science Research Program of the National Research Foundation of Korea (NRF) funded by the Ministry of Education (2020R1A6A1A03040516), and the DGIST R&D Program of the Ministry of Science and ICT of Korea (21–DPIC–15).

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The raw data used in the current study are available in the VAERS (https://vaers.hhs.gov/data/datasets.html, accessed on: 6 August 2021). All data generated in this study are available on request from the corresponding author.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Grasselli, G.; Zangrillo, A.; Zanella, A.; Antonelli, M.; Cabrini, L.; Castelli, A.; Cereda, D.; Coluccello, A.; Foti, G.; Fumagalli, R. Baseline characteristics and outcomes of 1591 patients infected with SARS-CoV-2 admitted to ICUs of the Lombardy Region, Italy. JAMA 2020, 323, 1574–1581. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. Guo, W.; Pan, B.; Sakkiah, S.; Ji, Z.; Yavas, G.; Lu, Y.; Komatsu, T.E.; Lal-Nag, M.; Tong, W.; Patterson, T.A.; et al. Informing selection of drugs for COVID-19 treatment through adverse events analysis. Sci. Rep. 2021, 11, 1–7. [Google Scholar] [CrossRef] [PubMed]
  3. FDA Emergency Use Authorization (EUA) of the Pfizer-BioNTech COVID-19 Vaccine to Prevent Coronavirus. Fact Sheet for Healthcare Providers Administering Vaccine. Available online: https://www.fda.gov/media/144413/download (accessed on 9 December 2021).
  4. FDA Emergency Use Authorization (EUA) of the Moderna COVID-19 Vaccine to Prevent Coronavirus Disease 2019 (COVID-19). Fact Sheet for Healthcare Providers Administering Vaccine. Available online: https://www.fda.gov/media/144637/download?utm_medium=email&utm_source=govdelivery (accessed on 26 October 2021).
  5. FDA Emergency Use Authorization (EUA) of the Janssen COVID-19 Vaccine to Prevent Coronavirus Disease 2019 (COVID-19). Available online: https://www.fda.gov/media/146304/download (accessed on 26 October 2021).
  6. Polack, F.P.; Thomas, S.J.; Kitchin, N.; Absalon, J.; Gurtman, A.; Lockhart, S.; Perez, J.L.; Marc, G.P.; Moreira, E.D.; Zerbini, C. Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine. N. Engl. J. Med. 2020, 383, 2603–2615. [Google Scholar] [CrossRef] [PubMed]
  7. Baden, L.R.; El Sahly, H.M.; Essink, B.; Kotloff, K.; Frey, S.; Novak, R.; Diemert, D.; Spector, S.A.; Rouphael, N.; Creech, C.B. Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. N. Engl. J. Med. 2021, 384, 403–416. [Google Scholar] [CrossRef] [PubMed]
  8. FDA Brifing Document Janssen Ad26.COV2.S Vaccine for the Prevention of COVID-19. Available online: https://www.fda.gov/media/146217/download (accessed on 26 October 2021).
  9. Shimabukuro, T.T.; Cole, M.; Su, J.R. Reports of anaphylaxis after receipt of mRNA COVID-19 vaccines in the US—December 14, 2020-January 18, 2021. JAMA 2021, 325, 1101–1102. [Google Scholar] [CrossRef]
  10. Gargano, J.W.; Wallace, M.; Hadler, S.C.; Langley, G.; Su, J.R.; Oster, M.E.; Broder, K.R.; Gee, J.; Weintraub, E.; Shimabukuro, T. Use of mRNA COVID-19 vaccine after reports of myocarditis among vaccine recipients: Update from the advisory committee on immunization practices—United States, June 2021. Morb. Mortal. Wkly. Rep. 2021, 70, 977–982. [Google Scholar] [CrossRef]
  11. See, I.; Su, J.R.; Lale, A.; Woo, E.J.; Guh, A.Y.; Shimabukuro, T.T.; Streiff, M.B.; Rao, A.K.; Wheeler, A.P.; Beavers, S.F. US case reports of cerebral venous sinus thrombosis with thrombocytopenia after Ad26.COV2.S vaccination, March 2 to April 21, 2021. JAMA 2021, 325, 2448–2456. [Google Scholar] [CrossRef]
  12. Rosenblum, H.G.; Hadler, S.C.; Moulia, D.; Shimabukuro, T.T.; Su, J.R.; Tepper, N.K.; Ess, K.C.; Woo, E.J.; Mba-Jonas, A.; Alimchandani, M. Use of COVID-19 vaccines after reports of adverse events among adult recipients of Janssen (Johnson & Johnson) and mRNA COVID-19 vaccines (Pfizer-BioNTech and Moderna): Update from the advisory committee on immunization practices—United States, July 2021. Morb. Mortal. Wkly. Rep. 2021, 70, 1094–1099. [Google Scholar] [CrossRef]
  13. Cirillo, N. Reported orofacial adverse effects of COVID-19 vaccines: The knowns and the unknowns. J. Oral Pathol. Med. 2021, 50, 424–427. [Google Scholar] [CrossRef]
  14. FDA Briefing Document Vaccines and Related Biological Products Advisory Committee Meeting. Available online: https://www.fda.gov/media/144434/download (accessed on 26 October 2021).
  15. Chen, G.; Li, X.; Sun, M.; Zhou, Y.; Yin, M.; Zhao, B.; Li, X. COVID-19 mRNA vaccines are generally safe in the short term: A vaccine vigilance real-world study says. Front. Immunol. 2021, 12, 1843. [Google Scholar] [CrossRef]
  16. Centers for Disease Control and Prevention Vaccine Safety Datalink. Available online: https://www.cdc.gov/vaccinesafety/ensuringsafety/monitoring/vsd/index.html (accessed on 27 October 2021).
  17. Shimabukuro, T.T.; Nguyen, M.; Martin, D.; DeStefano, F. Safety monitoring in the vaccine adverse event reporting system (VAERS). Vaccine 2015, 33, 4398–4405. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  18. Varricchio, F.; Iskander, J.; Destefano, F.; Ball, R.; Pless, R.; Braun, M.M.; Chen, R.T. Understanding vaccine safety information from the vaccine adverse event reporting system. Pediatr. Infect. Dis. J. 2004, 23, 287–294. [Google Scholar] [CrossRef] [PubMed]
  19. Chen, R.T.; Rastogi, S.C.; Mullen, J.R.; Hayes, S.W.; Cochi, S.L.; Donlon, J.A.; Wassilak, S.G. The vaccine adverse event reporting system (VAERS). Vaccine 1994, 12, 542–550. [Google Scholar] [CrossRef]
  20. Xiong, X.; Yuan, J.; Li, M.; Jiang, B.; Lu, Z.K. Age and gender disparities in adverse events following COVID-19 vaccination: Real-world evidence based on big data for risk management. Front. Med. 2021, 8, 700014. [Google Scholar] [CrossRef] [PubMed]
  21. Lv, G.; Yuan, J.; Xiong, X.; Li, M. Mortality rate and characteristics of deaths following COVID-19 vaccination. Front. Med. 2021, 8, 670370. [Google Scholar] [CrossRef] [PubMed]
  22. Kompaniyets, L.; Goodman, A.B.; Belay, B.; Freedman, D.S.; Sucosky, M.S.; Lange, S.J.; Gundlapalli, A.V.; Boehmer, T.K.; Blanck, H.M. Body mass index and risk for COVID-19–related hospitalization, intensive care unit admission, invasive mechanical ventilation, and death—united states, march–december 2020. Morb. Mortal. Wkly. Rep. 2021, 70, 355–361. [Google Scholar] [CrossRef]
  23. Lighter, J.; Phillips, M.; Hochman, S.; Sterling, S.; Johnson, D.; Francois, F.; Stachel, A. Obesity in patients younger than 60 years is a risk factor for Covid-19 hospital admission. Clin. Infect. Dis. 2020, 71, 896–897. [Google Scholar] [CrossRef] [Green Version]
  24. Tartof, S.Y.; Qian, L.; Hong, V.; Wei, R.; Nadjafi, R.F.; Fischer, H.; Li, Z.; Shaw, S.F.; Caparosa, S.L.; Nau, C.L. Obesity and mortality among patients diagnosed with COVID-19: Results from an integrated health care organization. Ann. Intern. Med. 2020, 173, 773–781. [Google Scholar] [CrossRef]
  25. CDC Science Brief: Evidence Used to Update the List of Underlying Medical Conditions That Increase a Person’s Risk of Severe Illness from COVID-19. Available online: http://www.ncbi.nlm.nih.gov/pubmed/34009770 (accessed on 8 September 2021).
  26. Bowe, B.; Cai, M.; Xie, Y.; Gibson, A.K.; Maddukuri, G.; Al-Aly, Z. Acute kidney injury in a national cohort of hospitalized US veterans with COVID-19. Clin. J. Am. Soc. Nephrol. 2021, 16, 14–25. [Google Scholar] [CrossRef]
  27. Robbins-Juarez, S.Y.; Qian, L.; King, K.L.; Stevens, J.S.; Husain, S.A.; Radhakrishnan, J.; Mohan, S. Outcomes for patients with COVID-19 and acute kidney injury: A systematic review and meta-analysis. Kidney Int. Rep. 2020, 5, 1149–1160. [Google Scholar] [CrossRef]
  28. Cheng, Y.; Luo, R.; Wang, K.; Zhang, M.; Wang, Z.; Dong, L.; Li, J.; Yao, Y.; Ge, S.; Xu, G. Kidney disease is associated with in-hospital death of patients with COVID-19. Kidney Int. 2020, 97, 829–838. [Google Scholar] [CrossRef] [PubMed]
  29. Dai, M.; Liu, D.; Liu, M.; Zhou, F.; Li, G.; Chen, Z.; Zhang, Z.; You, H.; Wu, M.; Zheng, Q. Patients with cancer appear more vulnerable to SARS-CoV-2: A multicenter study during the COVID-19 outbreak. Cancer Discov. 2020, 10, 783–791. [Google Scholar] [CrossRef] [PubMed]
  30. CDC COVID-19 Response Team. Allergic reactions including anaphylaxis after receipt of the first dose of Pfizer-BioNTech COVID-19 vaccine—United States, December 14–23, 2020. Morb. Mortal. Wkly. Rep. 2021, 70, 46–51. [Google Scholar] [CrossRef]
  31. CDC COVID-19 Response Team. Allergic reactions including anaphylaxis after receipt of the first dose of Moderna COVID-19 Vaccine—United States, December 21, 2020–January 10, 2021. Morb. Mortal. Wkly. Rep. 2021, 70, 125–129. [Google Scholar] [CrossRef]
  32. Brown, E.G.; Wood, L.; Wood, S. The medical dictionary for regulatory activities (MedDRA). Drug Saf. 1999, 20, 109–117. [Google Scholar] [CrossRef]
  33. MedDRA What’s New MedDRA Version 24.0. Available online: https://admin.new.meddra.org/sites/default/files/guidance/file/whatsnew_24_0_English.pdf (accessed on 2 September 2021).
  34. Brown, E.G. Using MedDRA. Drug Saf. 2004, 27, 591–602. [Google Scholar] [CrossRef]
  35. Fescharek, R.; Kübler, J.; Elsasser, U.; Frank, M.; Güthlein, P. Medical dictionary for regulatory activities (MedDRA). Int. J. Pharm. Med. 2004, 18, 259–269. [Google Scholar] [CrossRef]
  36. Reynolds, H.R.; Adhikari, S.; Pulgarin, C.; Troxel, A.B.; Iturrate, E.; Johnson, S.B.; Hausvater, A.; Newman, J.D.; Berger, J.S.; Bangalore, S. Renin–angiotensin–aldosterone system inhibitors and risk of COVID-19. N. Engl. J. Med. 2020, 382, 2441–2448. [Google Scholar] [CrossRef]
  37. Guan, W.; Liang, W.; Zhao, Y.; Liang, H.; Chen, Z.; Li, Y.; Liu, X.; Chen, R.; Tang, C.; Wang, T.; et al. Comorbidity and its impact on 1590 patients with COVID-19 in China: A nationwide analysis. Eur. Respir. J. 2020, 55, 2000547. [Google Scholar] [CrossRef] [Green Version]
  38. Baldi, E.; Sechi, G.M.; Mare, C.; Canevari, F.; Brancaglione, A.; Primi, R.; Klersy, C.; Palo, A.; Contri, E.; Ronchi, V. Out-of-hospital cardiac arrest during the Covid-19 outbreak in Italy. N. Engl. J. Med. 2020, 383, 496–498. [Google Scholar] [CrossRef]
  39. Alvarez-Garcia, J.; Lee, S.; Gupta, A.; Cagliostro, M.; Joshi, A.A.; Rivas-Lasarte, M.; Contreras, J.; Mitter, S.S.; LaRocca, G.; Tlachi, P. Prognostic impact of prior heart failure in patients hospitalized with COVID-19. J. Am. Coll. Cardiol. 2020, 76, 2334–2348. [Google Scholar] [CrossRef] [PubMed]
  40. DeFilippis, E.M.; Reza, N.; Donald, E.; Givertz, M.M.; Lindenfeld, J.; Jessup, M. Considerations for heart failure care during the COVID-19 pandemic. JACC Heart Fail. 2020, 8, 681–691. [Google Scholar] [CrossRef] [PubMed]
  41. Poissy, J.; Goutay, J.; Caplan, M.; Parmentier, E.; Duburcq, T.; Lassalle, F.; Jeanpierre, E.; Rauch, A.; Labreuche, J.; Susen, S.; et al. Pulmonary embolism in patients with COVID-19. Circulation 2020, 142, 184–186. [Google Scholar] [CrossRef]
  42. Kremer, S.; Lersy, F.; Anheim, M.; Merdji, H.; Schenck, M.; Oesterlé, H.; Bolognini, F.; Messie, J.; Khalil, A.; Gaudemer, A. Neurologic and neuroimaging findings in patients with COVID-19: A retrospective multicenter study. Neurology 2020, 95, e1868–e1882. [Google Scholar] [CrossRef]
  43. Rosenthal, J.A.; Awan, S.F.; Fintzi, J.; Keswani, A.; Ein, D. Asthma is associated with increased risk of intubation but not hospitalization or death in coronavirus disease 2019. Ann. Allergy Asthma Immunol. 2021, 126, 93–95. [Google Scholar] [CrossRef]
  44. Yang, J.M.; Koh, H.Y.; Moon, S.Y.; Yoo, I.K.; Ha, E.K.; You, S.; Kim, S.Y.; Yon, D.K.; Lee, S.W. Allergic disorders and susceptibility to and severity of COVID-19: A nationwide cohort study. J. Allergy Clin. Immunol. 2020, 146, 790–798. [Google Scholar] [CrossRef]
  45. Austin, P.C. Balance diagnostics for comparing the distribution of baseline covariates between treatment groups in propensity-score matched samples. Stat. Med. 2009, 28, 3083–3107. [Google Scholar] [CrossRef] [Green Version]
  46. Heinze, G.; Schemper, M. A solution to the problem of monotone likelihood in Cox regression. Biometrics 2001, 57, 114–119. [Google Scholar] [CrossRef]
  47. Chapin-Bardales, J.; Gee, J.; Myers, T. Reactogenicity following receipt of mRNA-based COVID-19 vaccines. JAMA 2021, 325, 2201–2202. [Google Scholar] [CrossRef]
  48. Gee, J.; Marquez, P.; Su, J.; Calvert, G.M.; Liu, R.; Myers, T.; Nair, N.; Martin, S.; Clark, T.; Markowitz, L. First month of COVID-19 vaccine safety monitoring—United States, December 14, 2020–January 13, 2021. Morb. Mortal. Wkly. Rep. 2021, 70, 283–288. [Google Scholar] [CrossRef]
  49. Sampson, H.A.; Muñoz-Furlong, A.; Campbell, R.L.; Adkinson , N.F., Jr.; Bock, S.A.; Branum, A.; Brown, S.G.A.; Camargo, C.A., Jr.; Cydulka, R.; Galli, S.J. Second symposium on the definition and management of anaphylaxis: Summary report—Second National Institute of Allergy and Infectious Disease/Food Allergy and Anaphylaxis Network symposium. J. Allergy Clin. Immunol. 2006, 117, 391–397. [Google Scholar] [CrossRef]
  50. Muir, K.-L.; Kallam, A.; Koepsell, S.A.; Gundabolu, K. Thrombotic thrombocytopenia after Ad26.COV2.S vaccination. N. Engl. J. Med. 2021, 384, 1964–1965. [Google Scholar] [CrossRef]
  51. Gessler, F.; Schmitz, A.K.; Dubinski, D.; Bernstock, J.D.; Lehmann, F.; Won, S.-Y.; Wittstock, M.; Güresir, E.; Hadjiathanasiou, A.; Zimmermann, J. Neurosurgical considerations regarding decompressive craniectomy for intracerebral hemorrhage after SARS-CoV-2-vaccination in vaccine induced thrombotic thrombocytopenia—VITT. J. Clin. Med. 2021, 10, 2777. [Google Scholar] [CrossRef]
  52. Greinacher, A.; Thiele, T.; Warkentin, T.E.; Weisser, K.; Kyrle, P.A.; Eichinger, S. Thrombotic thrombocytopenia after ChAdOx1 nCov-19 vaccination. N. Engl. J. Med. 2021, 384, 2092–2101. [Google Scholar] [CrossRef]
  53. European Medicines Agency AstraZeneca’s COVID-19 Vaccine: EMA Finds Possible Link to Very Rare Cases of Unusual Blood Clots with Low Blood Platelets. Available online: https://www.ema.europa.eu/en/news/astrazenecas-covid-19-vaccine-ema-finds-possible-link-very-rare-cases-unusual-blood-clots-low-blood (accessed on 18 October 2021).
  54. Thompson, M.G.; Stenehjem, E.; Grannis, S.; Ball, S.W.; Naleway, A.L.; Ong, T.C.; DeSilva, M.B.; Natarajan, K.; Bozio, C.H.; Lewis, N.; et al. Effectiveness of COVID-19 vaccines in ambulatory and inpatient care settings. N. Engl. J. Med. 2021, 385, 1355–1371. [Google Scholar] [CrossRef]
Vaccines 10 00284 g001 550
Figure 1. Data collection process.
Figure 1. Data collection process.
Vaccines 10 00284 g001
Vaccines 10 00284 g002 550
Figure 2. Study workflow.
Figure 2. Study workflow.
Vaccines 10 00284 g002
Vaccines 10 00284 g003 550
Figure 3. Cumulative probability of AE onset following COVID-19 vaccination.
Figure 3. Cumulative probability of AE onset following COVID-19 vaccination.
Vaccines 10 00284 g003
Table
Table 1. Demographic and clinical characteristics of all individuals vaccinated against COVID-19.
Table 1. Demographic and clinical characteristics of all individuals vaccinated against COVID-19.
VariableAd26.COV2.SmRNA-1273BNT162b2Total
Total, N (%)22,037 (8.57)126,838 (49.35)108,119 (42.07)256,994
Age, mean (SD)44.69 ± 15.151.45 ± 16.7649.4 ± 16.4350.01 ± 16.59
Gender
 Unknown, N (%)34 (0.15)225 (0.18)459 (0.42)718 (0.28)
 Male, N (%)7438 (33.75)30,161 (23.78)28,681 (26.53)66,280 (25.79)
 Female, N (%)14,565 (66.09)96,452 (76.04)78,979 (73.05)189,996 (73.93)
Underlying disability
 Obesity, N (%)280 (1.27)2027 (1.6)1378 (1.27)3685 (1.43)
 Hypertension, N (%)1437 (6.52)12,148 (9.58)7760 (7.18)21,345 (8.31)
 Diabetes, N (%)792 (3.59)6899 (5.44)4288 (3.97)11,979 (4.66)
 Atrial fibrillation, N (%)34 (0.15)364 (0.29)287 (0.27)685 (0.27)
 Heart failure, N (%)126 (0.57)1073 (0.85)676 (0.63)1875 (0.73)
 Kidney disease, N (%)78 (0.35)782 (0.62)524 (0.48)1384 (0.54)
 Pulmonary disease, N (%)187 (0.85)1396 (1.1)798 (0.74)2381 (0.93)
 Asthma, N (%)1061 (4.81)6580 (5.19)4880 (4.51)12,521 (4.87)
 Stroke, N (%)36 (0.16)285 (0.22)234 (0.22)555 (0.22)
 Cancers, N (%)200 (0.91)1768 (1.39)1221 (1.13)3189 (1.24)
 Allergies, N (%)559 (2.54)3256 (2.57)2557 (2.36)6372 (2.48)
Recovery
 Yes, N (%)11,763 (53.38)63,668 (50.2)53,390 (49.38)128,821 (50.13)
 No, N (%)10,274 (46.62)63,170 (49.8)54,729 (50.62)128,173 (49.87)
AE onset interval (days), mean (SD *)1.67 ± 3.12.51 ± 3.431.64 ± 2.82.07 ± 3.18
* Standard deviation. AE onset interval denotes days from the vaccination date to the adverse event onset date.
Table
Table 2. AEFIs related to delayed recovery after receiving the Janssen COVID-19 vaccine.
Table 2. AEFIs related to delayed recovery after receiving the Janssen COVID-19 vaccine.
SOCHLTNon-Recovered GroupHR (95% CI)
Total (%)N (%)
Cardiac disordersIschaemic coronary artery disorders654 (6.72)654 (6.72)1.18 (1.09–1.29)
Eye disordersOcular disorders NEC193 (1.98)193 (1.98)1.16 (1.01–1.35)
Gastrointestinal disordersGastrointestinal and abdominal pains (except oral and throat)1964 (20.18)512 (5.26)1.11 (1.02–1.22)
Nausea and vomiting symptoms1858 (19.09)1.11 (1.05–1.18)
General disorders and administration site conditionsAsthenic conditions2827 (29.05)2881 (29.61)1.06 (1.01–1.12)
Febrile disorders2524 (25.94)1.2 (1.14–1.27)
Injection site reactions1090 (11.2)1.29 (1.21–1.38)
Pain and discomfort NEC2540 (26.1)1.17 (1.12–1.23)
Hepatobiliary disordersCholecystitis and cholelithiasis7 (0.07)7 (0.07)2.6 (1.21–5.63)
Immune system disordersAllergic conditions NEC207 (2.13)207 (2.13)1.27 (1.1–1.46)
Musculoskeletal and connective tissue disordersJoint-related signs and symptoms2307 (23.71)1022 (10.5)1.12 (1.04–1.19)
Muscle pains1006 (10.34)1.17 (1.09–1.26)
Musculoskeletal and connective tissue pain and discomfort1934 (19.87)1.12 (1.06–1.18)
Nervous system disordersAuditory nerve disorders3769 (38.73)393 (4.04)1.41 (1.27–1.57)
Headaches NEC3526 (36.23)1.16 (1.11–1.22)
Sensory abnormalities NEC335 (3.44)1.22 (1.09–1.36)
Psychiatric disordersCognitive and attention disorders and disturbances NEC314 (3.23)117 (1.2)1.21 (1.01–1.46)
Disturbances in initiating and maintaining sleep207 (2.13)1.19 (1.03–1.37)
Respiratory, thoracic, and mediastinal disordersCoughing and associated symptoms367 (3.77)314 (3.23)1.17 (1.03–1.32)
Pharyngeal disorders (except infections and neoplasms)67 (0.69)1.34 (1.05–1.72)
Skin and subcutaneous tissue disordersDermal and epidermal conditions NEC1485 (15.26)1264 (12.99)1.08 (1.02–1.15)
Erythemas417 (4.29)1.12 (1.01–1.24)
Note: Non-Recovered Group—includes individuals who had not recovered from Janssen COVID-19-vaccine-induced AEs. For 2 weeks, no information on recovery was an exclusion criterion. NEC—not elsewhere classified—a standard abbreviation used to denote miscellaneous terms that do not readily fit into other hierarchical classifications for a particular SOC. The NEC designation is used only with HLTs and HLGTs for grouping purposes. Total—denotes the number of cases for one or more adverse symptoms belonging to each SOC. HR—hazard ratios.
Table
Table 3. AEFIs related to delayed recovery after receiving the Moderna COVID-19 vaccine.
Table 3. AEFIs related to delayed recovery after receiving the Moderna COVID-19 vaccine.
SOCHLTNon-Recovered GroupHR (95% CI)
Total (%)N (%)
Blood and lymphatic system disordersLymphatic system disorders NEC2943 (4.88)2943 (4.88)1.15 (1.11–1.2)
Cardiac disordersCardiac signs and symptoms NEC6622 (10.98)5455 (9.05)1.07 (1.04–1.1)
Ischaemic coronary artery disorders2016 (3.34)1.06 (1.01–1.11)
Left ventricular failures84 (0.14)1.39 (1.11–1.73)
Ventricular arrhythmias and cardiac arrest252 (0.42)1.15 (1–1.31)
Ear and labyrinth disordersEar disorders NEC743 (1.23)743 (1.23)1.1 (1.02–1.19)
Endocrine disordersHypoglycaemic conditions NEC12 (0.02)12 (0.02)2.13 (1.21–3.73)
Eye disordersConjunctival infections, irritations, and inflammations1093 (1.81)30 (0.05)1.46 (1.02–2.1)
Lacrimation disorders170 (0.28)1.22 (1.04–1.42)
Ocular disorders NEC654 (1.08)1.14 (1.05–1.24)
Ocular infections, inflammations, and associated manifestations347 (0.58)1.16 (1.04–1.29)
Visual impairment and blindness (except colour blindness)285 (0.47)1.13 (1–1.27)
Gastrointestinal disordersDiarrhoea (except infective)8291 (13.75)2049 (3.4)1.05 (1–1.1)
Gastrointestinal and abdominal pains (except oral and throat)1127 (1.87)1.07 (1–1.14)
Gastrointestinal disorders NEC66 (0.11)1.29 (1.01–1.65)
Nausea and vomiting symptoms7059 (11.71)1.07 (1.04–1.1)
Oral soft tissue disorders NEC76 (0.13)1.3 (1.03–1.63)
Oral soft tissue signs and symptoms2022 (3.35)1.15 (1.09–1.2)
Oral soft tissue swelling and oedema10 (0.02)2.28 (1.23–4.23)
Stomatitis and ulceration183 (0.3)1.24 (1.06–1.44)
Tongue signs and symptoms203 (0.34)1.21 (1.05–1.39)
General disorders and administration site conditionsAsthenic conditions16,454 (27.29)12,452 (20.65)1.06 (1.03–1.08)
Death and sudden death1179 (1.96)1.57 (1.47–1.67)
Febrile disorders9564 (15.86)1.14 (1.11–1.17)
General signs and symptoms NEC9468 (15.7)1.1 (1.07–1.12)
Pain and discomfort NEC11,624 (19.28)1.16 (1.14–1.19)
Hepatobiliary disordersHepatobiliary signs and symptoms15 (0.02)15 (0.02)1.86 (1.11–3.11)
Immune system disordersAllergic conditions NEC2035 (3.38)751 (1.25)1.25 (1.16–1.34)
Angio-oedemas1364 (2.26)1.1 (1.04–1.16)
Infections and infestationsClostridia infections13 (0.02)13 (0.02)3.47 (2.02–5.94)
Injury, poisoning and procedural complicationsChest and respiratory tract injuries NEC96 (0.16)8 (0.01)2.42 (1.21–4.85)
Muscle, tendon, and ligament injuries88 (0.15)1.32 (1.07–1.63)
InvestigationsGastrointestinal histopathology procedures264 (0.44)9 (0.01)1.98 (1.04–3.79)
Musculoskeletal and soft tissue imaging procedures255 (0.42)1.23 (1.08–1.4)
Metabolism and nutrition disordersFluid intake decreased18 (0.03)18 (0.03)1.83 (1.14–2.94)
Musculoskeletal and connective tissue disordersBursal disorders11,522 (19.11)49 (0.08)1.65 (1.25–2.2)
Joint-related disorders NEC121 (0.2)1.56 (1.31–1.87)
Joint-related signs and symptoms5216 (8.65)1.25 (1.21–1.29)
Muscle pains3721 (6.17)1.05 (1.02–1.09)
Musculoskeletal and connective tissue conditions NEC844 (1.40)1.08 (1.01–1.16)
Musculoskeletal and connective tissue pain and discomfort9701 (16.09)1.16 (1.13–1.18)
Skull and face fractures7 (0.01)2.45 (1.14–5.28)
Soft tissue disorders NEC171 (0.28)1.24 (1.07–1.44)
Nervous system disordersAuditory nerve disorders14,617 (24.24)2110 (3.5)1.49 (1.42–1.56)
Dementia (except Alzheimer’s type)23 (0.04)1.86 (1.22–2.83)
Headaches NEC11,704 (19.41)1.2 (1.17–1.23)
Neurologic visual problems NEC722 (1.2)1.15 (1.06–1.24)
Neuromuscular disorders NEC695 (1.15)1.17 (1.08–1.26)
Paraesthesia and dysesthesias908 (1.51)1.22 (1.15–1.31)
Paralysis and paresis (except cranial nerve)200 (0.33)1.17 (1.02–1.35)
Sensory abnormalities NEC1277 (2.12)1.2 (1.13–1.27)
Psychiatric disordersCognitive and attention disorders and disturbances NEC2483 (4.12)418 (0.69)1.29 (1.16–1.42)
Confusion and disorientation579 (0.96)1.15 (1.06–1.25)
Disturbances in initiating and maintaining sleep632 (1.05)1.24 (1.14–1.34)
Sleep disorders NEC1168 (1.94)1.19 (1.12–1.26)
Reproductive system and breast disordersErection and ejaculation conditions and disorders15 (0.02)15 (0.02)2.2 (1.32–3.67)
Respiratory, thoracic, and mediastinal disordersBreathing abnormalities3293 (5.46)2595 (4.3)1.1 (1.06–1.15)
Coughing and associated symptoms1344 (2.23)1.22 (1.15–1.29)
Skin and subcutaneous tissue disordersDermal and epidermal conditions NEC7289 (12.09)7289 (12.09)1.07 (1.05–1.1)
Social circumstancesDisability issues1335 (2.21)1335 (2.21)1.08 (1.02–1.14)
Vascular disordersBruising, ecchymosis, and purpura1724 (2.86)934 (1.55)1.17 (1.1–1.25)
Cerebrovascular and spinal vascular disorders NEC810 (1.34)1.14 (1.06–1.23)
Note: Non-Recovered Group—includes individuals who had not recovered from Moderna COVID-19-vaccine-induced AEs. For 2 weeks, no information on recovery was an exclusion criterion. NEC—not elsewhere classified—a standard abbreviation used to denote miscellaneous terms that do not readily fit into other hierarchical classifications for a particular SOC. The NEC designation is used only with HLTs and HLGTs for grouping purposes. Total—denotes the number of cases for one or more adverse symptoms belonging to each SOC. HR—hazard ratios.
Table
Table 4. AEFIs related to delayed recovery after receiving the Pfizer-BioNTech COVID-19 vaccine.
Table 4. AEFIs related to delayed recovery after receiving the Pfizer-BioNTech COVID-19 vaccine.
SOCHLTNon-Recovered GroupHR (95% CI)
Total (%)N (%)
Blood and lymphatic system disordersLymphatic system disorders NEC3735 (7.41)3735 (7.41)1.25 (1.21–1.29)
Cardiac disordersHeart failure signs and symptoms3108 (6.17)204 (0.4)1.2 (1.04–1.38)
Ischaemic coronary artery disorders2745 (5.45)1.05 (1.01–1.1)
Ventricular arrhythmias and cardiac arrest287 (0.57)1.34 (1.18–1.52)
Endocrine disordersDiabetic complications neurological14 (0.03)14 (0.03)1.73 (1.02–2.92)
Eye disordersChoroid and vitreous haemorrhages and vascular disorders897 (1.78)6 (0.01)3.37 (1.48–7.7)
Ocular disorders NEC891 (1.77)1.13 (1.05–1.21)
Gastrointestinal disordersFlatulence, bloating, and distension9051 (17.97)269 (0.53)1.13 (1–1.28)
Gastrointestinal and abdominal pains (except oral and throat)1505 (2.99)1.11 (1.05–1.17)
Gastrointestinal disorders NEC99 (0.2)1.3 (1.07–1.59)
Gastrointestinal dyskinetic disorders35 (0.07)1.54 (1.1–2.15)
Nausea and vomiting symptoms6933 (13.76)1.04 (1.01–1.07)
Oral soft tissue signs and symptoms2635 (5.23)1.1 (1.06–1.15)
Tongue signs and symptoms251 (0.5)1.14 (1–1.29)
General disorders and administration site conditionsDeath and sudden death16,022 (31.8)848 (1.68)1.39 (1.3–1.5)
Feelings and sensations NEC3245 (6.44)1.08 (1.04–1.12)
Gait disturbances930 (1.85)1.07 (1–1.15)
General signs and symptoms NEC6709 (13.32)1.07 (1.04–1.1)
Injection site reactions4516 (8.96)1.1 (1.06–1.13)
Pain and discomfort NEC9810 (19.47)1.08 (1.05–1.11)
Vaccination site reactions1628 (3.23)1.88 (1.79–1.98)
Immune system disordersAllergic conditions NEC716 (1.42)716 (1.42)1.3 (1.21–1.4)
Infections and infestationsCardiac infections6 (0.01)6 (0.01)2.72 (1.2–6.14)
Injury, poisoning and procedural complicationsMedication errors, product use errors and issues NEC491 (0.97)110 (0.22)1.51 (1.17–1.95)
Off label uses134 (0.27)2.16 (1.71–2.74)
Overdoses NEC14 (0.03)1.84 (1.08–3.11)
Product administration errors and issues507 (1.01)1.26 (1.15–1.38)
Radiation injuries15 (0.03)1.67 (1–2.76)
InvestigationsHeart rate and pulse investigations3858 (7.66)1458 (2.89)1.1 (1.04–1.17)
Physical examination procedures and organ system status2411 (4.79)1.13 (1.09–1.18)
Vascular tests NEC (include blood pressure)850 (1.69)1.1 (1.02–1.18)
Musculoskeletal and connective tissue disordersJoint-related disorders NEC9847 (19.55)102 (0.2)1.26 (1.03–1.53)
Joint-related signs and symptoms5150 (10.22)1.12 (1.09–1.16)
Muscle pains3822 (7.59)1.08 (1.04–1.12)
Musculoskeletal and connective tissue conditions NEC842 (1.67)1.09 (1.02–1.17)
Musculoskeletal and connective tissue pain and discomfort8048 (15.98)1.19 (1.16–1.22)
Nervous system disordersAuditory nerve disorders15,176 (30.13)2701 (5.36)1.34 (1.29–1.4)
Dementia (except Alzheimer’s type)13 (0.03)2.13 (1.23–3.67)
Headaches NEC11,537 (22.9)1.13 (1.11–1.16)
Neurologic visual problems NEC917 (1.82)1.08 (1.01–1.15)
Neuromuscular disorders NEC806 (1.6)1.08 (1–1.16)
Paraesthesia and dysesthesias981 (1.95)1.1 (1.03–1.17)
Peripheral neuropathies NEC205 (0.41)1.21 (1.05–1.39)
Sensory abnormalities NEC1671 (3.32)1.18 (1.12–1.24)
Psychiatric disordersCognitive and attention disorders and disturbances NEC1490 (2.96)478 (0.95)1.16 (1.05–1.27)
Communications disorders79 (0.16)1.27 (1.01–1.58)
Disturbances in initiating and maintaining sleep792 (1.57)1.17 (1.08–1.25)
Emotional and mood disturbances NEC283 (0.56)1.13 (1–1.28)
Parasomnias53 (0.11)1.34 (1.02–1.76)
Respiratory, thoracic, and mediastinal disordersBreathing abnormalities3346 (6.64)3097 (6.15)1.12 (1.08–1.17)
Pharyngeal disorders (except infections and neoplasms)419 (0.83)1.22 (1.1–1.34)
Skin and subcutaneous tissue disordersDermal and epidermal conditions NEC7584 (15.05)5694 (11.3)1.2 (1.16–1.23)
Pruritus NEC2898 (5.75)1.07 (1.03–1.12)
Vascular disordersCerebrovascular and spinal vascular disorders NEC917 (1.82)917 (1.82)1.1 (1.03–1.17)
Note: Non-Recovered Group—includes individuals who had not recovered from Pfizer-BioNTech COVID-19-vaccine-induced AEs. For 2 weeks, no information on recovery was an exclusion criterion. NEC—not elsewhere classified—a standard abbreviation used to denote miscellaneous terms that do not readily fit into other hierarchical classifications for a particular SOC. The NEC designation is used only with HLTs and HLGTs for grouping purposes. Total—denotes the number of cases for one or more adverse symptoms belonging to each SOC. HR—hazard ratios.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Son, C.-S.; Jin, S.-H.; Kang, W.-S. Propensity-Score-Matched Evaluation of Adverse Events Affecting Recovery after COVID-19 Vaccination: On Adenovirus and mRNA Vaccines. Vaccines 2022, 10, 284. https://doi.org/10.3390/vaccines10020284

AMA Style

Son C-S, Jin S-H, Kang W-S. Propensity-Score-Matched Evaluation of Adverse Events Affecting Recovery after COVID-19 Vaccination: On Adenovirus and mRNA Vaccines. Vaccines. 2022; 10(2):284. https://doi.org/10.3390/vaccines10020284

Chicago/Turabian Style

Son, Chang-Sik, Sang-Hyeon Jin, and Won-Seok Kang. 2022. "Propensity-Score-Matched Evaluation of Adverse Events Affecting Recovery after COVID-19 Vaccination: On Adenovirus and mRNA Vaccines" Vaccines 10, no. 2: 284. https://doi.org/10.3390/vaccines10020284

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop