Next Article in Journal
Indicators of HSV1 Infection, ECM–Receptor Interaction, and Chromatin Modulation in a Nuclear Family with Schizophrenia
Previous Article in Journal
Revolutionizing Cancer Research: The Impact of Artificial Intelligence in Digital Biobanking
Previous Article in Special Issue
Sleep Quality Aspects in Post-COVID-19 Patients
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Cardiac Autonomic Function and Functional Capacity in Post-COVID-19 Individuals with Systemic Arterial Hypertension

by
Edelvita Fernanda Duarte Cunha
1,2,3,†,
Matheus Sobral Silveira
1,2,4,†,
Juliana Cristina Milan-Mattos
2,5,*,†,
Heitor Fernandes Silveira Cavalini
1,2,3,
Ádrya Aryelle Ferreira
1,2,3,
Joice de Souza Batista
2,3,
Lara Cazé Uzumaki
2,3,
João Paulo Coelho Guimarães
2,3,
Pedro Igor Lustosa Roriz
1,2,3,
Fabianne Maisa de Novaes Assis Dantas
2,3,
Arto J. Hautala
6,
Raphael Martins de Abreu
7,8,
Aparecida Maria Catai
2,5,
Paulo Adriano Schwingel
1,4,* and
Victor Ribeiro Neves
1,2,3,*
1
Programa de Pós-Graduação em Reabilitação e Desempenho Funcional (PPGRDF), Universidade de Pernambuco (UPE), Petrolina 56328-900, PE, Brazil
2
Grupo de Estudos e Pesquisas em Fisioterapia Cardiorrespiratória (GEFIC), Universidade de Pernambuco (UPE), Petrolina 56328-900, PE, Brazil
3
Laboratório de Fisioterapia Cardiopulmonar (LAFIC), Universidade de Pernambuco (UPE), Petrolina 56328-900, PE, Brazil
4
Laboratório de Pesquisas em Desempenho Humano (LAPEDH), Universidade de Pernambuco (UPE), Petrolina 56328-900, PE, Brazil
5
Postgraduate Program in Physical Therapy (PPGFT), Federal University of São Carlos (UFSCar), São Carlos 13565-905, SP, Brazil
6
Faculty of Sport and Health Sciences, University of Jyväskylä, P. O. Box 35, FI-40014 Jyväskylä, Finland
7
Department of Physiotherapy, LUNEX University—International University of Health, Exercise & Sports SA, 4671 Differdange, Luxembourg
8
LUNEX ASBL Luxembourg Health & Sport Sciences Research Institute, 4671 Differdange, Luxembourg
*
Authors to whom correspondence should be addressed.
These authors contributed equally to this work.
J. Pers. Med. 2023, 13(9), 1391; https://doi.org/10.3390/jpm13091391
Submission received: 28 August 2023 / Revised: 7 September 2023 / Accepted: 13 September 2023 / Published: 18 September 2023

Abstract

:
Individuals diagnosed with systemic arterial hypertension (SAH) are considered risk groups for COVID-19 severity. This study assessed differences in cardiac autonomic function (CAF) and functional capacity (FC) in SAH individuals without COVID-19 infection compared to SAH individuals post-COVID-19. Participants comprised 40 SAH individuals aged 31 to 80 years old, grouped as SAH with COVID-19 (G1; n = 21) and SAH without COVID-19 (G2; n = 19). CAF was assessed via heart rate variability (HRV), measuring R–R intervals during a 10-min supine period. Four HRV indices were analyzed through symbolic analysis: 0V%, 1V%, 2LV%, and 2UV%. FC assessment was performed by a 6-min walk test (6MWT). G1 and G2 showed no significant differences in terms of age, anthropometric parameters, clinical presentation, and medication use. G2 exhibited superior 6MWT performance, covering more distance (522 ± 78 vs. 465 ± 59 m, p < 0.05). Specifically, G2 demonstrated a moderate positive correlation between 6MWT and the 2LV% index (r = 0.58; p < 0.05). Shorter walking distances were observed during 6MWT in SAH individuals post-COVID-19. However, the study did not find impaired cardiac autonomic function in SAH individuals post-COVID-19 compared to those without. This suggests that while COVID-19 impacted FC, CAF remained relatively stable in this population.

1. Introduction

Systemic arterial hypertension (SAH) is considered one of the most prevalent risk factors for the development of coronavirus disease 2019 (COVID-19). Hypertensive individuals are classified as high-risk groups due to their strong association with disease severity and mortality [1,2,3,4]. Studies report that this association is due to the high affinity of SARS-CoV-2 for the angiotensin-converting enzyme 2 (ACE2) receptor, an enzyme highly expressed in these individuals, rendering them more susceptible to the disease. This promotes endothelial cell dysfunction, elevated blood pressure, and other cardiovascular abnormalities [5,6].
The inflammatory process triggered during the disease disrupts the cholinergic anti-inflammatory pathway. This pathway controls cytokine release through afferent vagal stimulation, as cytokine release is controlled by the parasympathetic branch of the autonomic nervous system (ANS). However, hypertensive individuals often present with a pre-existing sympathovagal imbalance, which might be interpreted as an adaptive condition to maintain homeostasis [7,8,9]. Given this imbalance, an adequate anti-inflammatory response to SARS-CoV-2 in these individuals appears unlikely, as autonomic imbalance could be associated with an inflammatory cytokine storm during the disease progression [1,10,11].
In addition to the imbalance in cardiac autonomic function (CAF), patients after the acute phase of SARS-CoV-2 infection may experience persistent and debilitating symptoms such as chronic fatigue, dyspnea, myalgia, arrhythmias, chest pain, orthostatic intolerance, and exercise intolerance. These symptoms lead to reduced functional capacity and worsened quality of life [11,12,13].
As reported in the literature, individuals with cardiovascular diseases (CVDs) and associated risk factors such as SAH experience a decline in functional capacity (FC) due to exercise intolerance, leading to a significant impact on their quality of life. This results from their physical inability to perform daily and occupational activities. When combined with the functional limitations imposed by COVID-19, even in mild cases, these individuals might experience further deterioration [13,14]. In this study, the aim was to assess differences in CAF and FC in SAH individuals without COVID-19 infection compared to SAH individuals post-COVID-19.

2. Materials and Methods

2.1. Study Design

The observational study was conducted from July 2021 to September 2022 at the University of Pernambuco (UPE) in Petrolina, PE, Brazil. The research was approved by the Research Ethics Committee of the Centro Integrado de Saúde Amaury de Medeiros (protocol 4848824 on 29 June 2021), and all participants provided voluntary consent prior to participation. Eligible individuals were recruited in Petrolina, facilitated by the Municipal Health Department (SESAU), based on a list of those diagnosed with systemic arterial hypertension (SAH) and infected with SARS-CoV-2. The study was promoted through various media channels and the assessments were carried out at the UPE Cardiopulmonary Physiotherapy Laboratory, with assessors being blinded to participant groups.

2.2. Participants

Individuals of both genders, aged between 31 and 80 years, diagnosed with SAH for at least one year, and on continuous antihypertensive medication for at least three months were evaluated in this study. These individuals were either infected or not with COVID-19. Participants were divided into two groups as follows:
  • Group 1 (G1): SAH patients infected with SARS-CoV-2, with confirmed diagnosis through the reverse transcription polymerase chain reaction (RT-PCR), within six months of test confirmation, asymptomatic or experiencing mild symptoms of COVID-19 and fully recovered of infection.
  • Group 2 (G2): Individuals with SAH without a clinical diagnosis of COVID-19, and with a negative RT-PCR test confirming the absence of the virus.
Individuals were not included if they experienced moderate to severe COVID-19 symptoms, had pre-existing diagnoses or neurological sequelae of diseases, had chronic obstructive pulmonary disease (COPD), had blood pressure exceeding 180/100 mmHg, were pregnant, had physical or mental limitations at the time of assessment, or had significant clinical conditions that contraindicated the tests. Volunteers who were on medications affecting the studied variables’ responses and who were unable to complete the tests and/or assessment protocols were also excluded. Participants with poor-quality electrocardiogram (ECG) signals (interference, signal instability) were excluded from the research. Furthermore, participants classified as having good, excellent, or superior cardiorespiratory fitness (CRF) during the six-minute walk test (6MWT) were also excluded (Figure 1).

2.3. Assessments

2.3.1. Anthropometric and Hemodynamic Parameters Measurements

Participants’ total body mass was measured in kilograms (kg), and height was measured in centimeters (cm) using a properly calibrated anthropometric scale (PL-200, Filizola S.A., São Paulo, SP, Brazil), adhering to the standards set by NBR ISO/IEC 17025:2005. Body mass index (BMI) was calculated by dividing body mass (kg) by the square of height (m2). For the measurement of systolic blood pressure (SBP) and diastolic blood pressure (DBP), an automatic, noninvasive, calibrated, and validated arterial blood pressure monitor, HEM-7200 (Omron Healthcare Inc., Lake Forest, IL, USA), was employed. The measurement procedures followed the guidelines outlined in the Brazilian Guidelines of Hypertension [15].

2.3.2. Evaluation of Cardiac Autonomic Control System

The CAF was evaluated by heart rate variability (HRV) assessment and was conducted in the supine position for 10 min. The R–R interval (iRR) recordings from the ECG were acquired using a twelve-lead electrocardiograph Wincardio (MICROMED BIOTECNOLOGIA S.A., Brasília, DF, Brazil). A sequence of 256 iRR with the highest stability was chosen for each subject to calculate the mean, variance of iRR, and HRV indices in the frequency domain. All assessment methodologies of HRV were analyzed in specific routines, developed by Prof. Dr. Alberto Porta of the University of Milan, Italy [16].
Frequency domain HRV analysis was executed using an autoregressive model applied to the previously selected iRR sequence data [17]. Two spectral components were considered: low frequency (LF—0.04 to 0.15 Hz), representing mixed (sympathetic and parasympathetic) modulation, and high frequency (HF—0.15 to 0.50 Hz), representing parasympathetic cardiac modulation [18]. The spectral components were expressed in normalized units (LFnu and HFnu) and absolute units (LFabs and HFabs). Respiratory frequency (RF) was evaluated during the protocol to ensure the presence of respiratory activity within its corresponding bandwidth for spectral analysis. All the volunteers presented a respiratory rate in the HF range.
Symbolic analysis (SA) grouped all patterns into four families: (a) patterns without variation (0V); (b) patterns with one variation (1V: two consecutive symbols are the same, and one symbol is different); (c) patterns with two similar variations (2LV); (d) patterns with two different variations (2UV: three symbols forming a peak or valley). The occurrence rate of each pattern is denoted as 0V%, 1V%, 2LV%, and 2UV%. Here, 0V% and 2UV% are considered markers of sympathetic and vagal modulation, respectively [16,19].
The complexity analysis was executed following the methodology proposed by Porta et al. [20]. The analysis encompassed the evaluation of Shannon entropy (SE) and conditional entropy (CE), in addition to the assessment of the complexity index (CI) and normalized complexity index (NCI).

2.3.3. Evaluation of FC

The 6MWT was employed to assess FC by the American Thoracic Society (ATS) guidelines [21]. A 30-m-long corridor devoid of obstacles was utilized for the test, marked every 3 m with cones at the ends. The participant was familiarized with the path before the walk commenced. After six minutes, the distance covered by the individual in meters (6-min walk distance—6MWD) was recorded [21,22]. Heart rate and oxygen saturation (SpO2) were recorded with the finger pulse oximeter OXIOLCM (Beijing Choice Electronic Technology Co., Ltd., Beijing, China). At the beginning and end of the experiment, the blood pressure of all volunteers was measured by the auscultatory method. The rate of perceived exertion (RPE) at the end of the 6MWT was recorded with the modified 0–10 Borg Scale [23]. The reference equation used in this research for the prediction of the total distance to be walked during the 6MWT was developed by Iwama et al. [24]:
6MWD = 622.461 − (1.846 × Age years) + (61.503 × Gender males = 1; females = 0)
According to Dourado et al. [25], fitness levels were classified using CRF and the 6MWD covered in meters during the 6MWT, stratified by age groups. The levels were categorized as very low, low, fair, good, excellent, and superior.

2.4. Statistical Analysis

The data were processed and analyzed using the SigmaPlot software (Systat Software Inc., San Jose, CA, USA, Release 11.0, 2011). The normality of data distribution was assessed using the Shapiro–Wilk test. Categorical variables were compared by Pearson’s chi-squared test (Χ2) or Fisher’s exact test and expressed as percentages and frequency. For continuous variables, the independent samples t-test was applied to parametric variables, and the Mann–Whitney U test was used for non-parametric data. To explore the relationship between functional testing and HRV indices, Spearman’s rank correlation coefficient was utilized for non-parametric variables, while Pearson’s product–moment correlation coefficient was used for parametric variables. All statistical methods were two-tailed, with exact p values, and statistical significance was defined as p < 0.05 [26]. The strength of correlations was categorized as follows: 0.00–0.25, very low; 0.26–0.49, low; 0.50–0.69, moderate; 0.70–0.89, high; 0.90–1.00, very high [26]. Additionally, Cohen’s d-effect size (ES) was employed to estimate the magnitude of differences between groups [27]. The ES was categorized as follows: 0.2–0.3, small; 0.5–0.8, medium; >0.8, large [27].

3. Results

3.1. Sample Characteristics

Out of the initial pool of 113 volunteers who fulfilled the eligibility criteria, a refined group of 40 individuals was subjected to evaluation, following the application of exclusion criteria. It is important to highlight that all study participants were under continuous medication for controlling SAH. The comprehensive overview of the study’s participant profile is depicted in Table 1. Notably, no statistically significant differences emerged between the two groups in terms of crucial attributes, including age, gender, height, total body mass, and BMI.
Regarding the COVID-19 symptoms reported by volunteers in G1 during the disease, the majority experienced cough (57%), headache (48%), fever (48%), anosmia (43%), runny nose (33%), shortness of breath (19%), fatigue (14%), and myalgia (10%).

3.2. Cardiac Autonomic Function

The HRV indices of the studied groups are presented in Table 2. No statistical differences were observed in the linear and non-linear HRV indices between the groups with very low ES (d < 0.5). The groups exhibited similarity in RF, with G1 presenting an RF of 18 ± 4 breaths per minute (bpm) and G2 exhibiting an RF of 17 ± 4 bpm (p > 0.05). It is worth noting that in the spectral analysis data, both groups displayed a predominance of high frequency (HF), with no statistical difference between them. Regarding the symbolic analysis, both groups demonstrated a predominance of the 1V% pattern, which represents mixed (sympathetic and parasympathetic) modulation, with no statistical difference between the groups.

3.3. Functional Capacity

Of the forty study participants, eight (20%) did not complete the 6MWT protocol (three in the G1 and five from G2). Within G1, three (14%) participants were classified as having a level of cardiorespiratory fitness (CRF) equal to or higher than good and were subsequently excluded from all analyses. Consequently, for the variables obtained through the 6MWT, the groups were composed as follows: G1 with 18 post-COVID-19 individuals with SAH (10 males; 51 ± 11 years; 88.1 ± 23.5 kg; 1.64 ± 0.1 m; BMI: 31.03 ± 10.4 kg/m2); G2 with 14 individuals with SAH (9 males; 56 ± 10 years; 81.9 ± 14.5 kg; 1.70 ± 0.1 m; BMI: 29.3 ± 2.8 kg/m2). These variables did not exhibit statistically significant differences in the comparison conducted using the independent-samples t-test.
The distance covered during the 6MWT displayed a statistically significant difference between G2 and G1 (as shown in Table 3), with G2 covering a significantly greater distance (p = 0.0241). The mean difference indicated that G2 walked 58 m more than G1, illustrating a large ES of d > 0.8. Conversely, there were no statistically significant differences observed between the groups in relation to other variables under analysis, including heart rate frequency, systolic blood pressure, diastolic blood pressure, blood oxygen level, Borg rating of perceived exertion during the 6MWT, and the percentage difference between predicted and covered distances on the 6MWT. Importantly, both G1 and G2 fell short of achieving the full percentage of predicted distance, with G1 reaching 85 ± 12% and G2 achieving 93 ± 10% (p = 0.0489).
The fitness levels classified eight (44%) post-COVID-19 individuals with SAH (G1) as very low CRF, seven (39%) as low CRF, and three (17%) as fair CRF. In contrast, only one (7%) individual with SAH (G2) presented very low CRF, while seven (50%) had low CRF and six (43%) had fair CRF.
Figure 2 illustrates the correlation between the distance covered in the 6MWT and the 2UV% index in the studied groups. A positive and moderate correlation was observed between these variables solely in G2, indicating that greater parasympathetic modulation was associated with increased distance covered (r = 0.58; p < 0.05). No significant correlations were observed for other variables in both groups.

4. Discussion

The current study aimed to assess differences in cardiac autonomic modulation and functional capacity in individuals with systemic arterial hypertension without COVID-19 infection compared to post-COVID-19 individuals. The main finding of this study was that post-COVID-19 individuals with SAH exhibited a reduced distance covered on the 6MWT compared to SAH individuals without clinical diagnosis of COVID-19. Thus, SAH patients infected with SARS-CoV-2, even with mild symptoms, experienced a decline in FC. Additionally, the greater physical capacity observed in SAH individuals without COVID-19 showed a positive correlation with higher parasympathetic modulation at rest. Ultimately, there were no significant differences in linear and non-linear HRV indices between the groups.
The 6MWT has been widely utilized to evaluate the functional capacity of individuals with various conditions, including cardiovascular, respiratory, and metabolic diseases, as well as those who have recovered from COVID-19 [28,29]. Wong et al. [30], studying post-COVID-19 patients, found an association between the severity of the disease and the distance covered on 6MWT, where individuals with moderate to severe symptoms covered less distance compared to those with mild symptoms. The participants in that study (31 men and 32 women, 42 ± 11 years old) were hospitalized and did not report hypoxemia, with only a few risk factors such as hypertension (13%) and diabetes mellitus (5%). This group covered a mean distance of 491 (±72) m in the 6MWT, achieving a distance predicted of 83%. In our study, none of the post-COVID-19 participants were hospitalized; all had mild symptoms and covered a mean distance of 465 (±60) m, 27 m less in mean than the study mentioned above. This discrepancy may be attributed to the fact that the study performed by Wong et al. [30] was conducted on a younger population with fewer associated risk factors. Furthermore, all participants in the present study had a diagnosis of SAH, which is considered a significant risk factor associated with COVID-19 complications.
In a retrospective study, Huang et al. [31] investigated the influence of COVID-19 on physical capacity during the early convalescent phase and 30 days post-hospital discharge. They evaluated two groups based on clinical severity, severe (n = 17) and non-severe (n = 40), who underwent the 6MWT. According to their results, the non-severe group covered 574 (±38) m, corresponding to a distance predicted of 96%. Another study assessed the impact of COVID-19 on functional capacity in 87 patients with mild COVID-19 symptoms after 60 days of symptom onset and hospital discharge. The 6MWT distance for this group was 538 (±57) m [30]. In the present study, the covered distance on 6MWT was lower than in the studies mentioned above. The participants from previous studies presented mild COVID-19 symptoms but were hospitalized, which did not occur in the present study. Additionally, the discrepancy in 6MWT values could be attributed to the profile of the populations in the cited studies. This study evaluated only SAH individuals who had mild COVID-19 symptoms and were not hospitalized. Considering these results, even mild COVID-19 can lead to a decline in functional capacity, especially in SAH individuals. Although the underlying pathophysiological mechanisms remain unclear, COVID-19 may negatively influence physical capacity, even in individuals with mild clinical presentation.
Regarding risk factors, several studies have described SAH as one of the most common factors associated with disease severity. However, the pathophysiological mechanism of hypertension and its relationship with COVID-19, as well as its impact on physical capacity, remain unclear [2]. In the present study, participants with SAH without COVID-19 covered nearly 60 m more on the 6MWT than post-COVID-19 individuals with SAH. A systematic review demonstrated that an increase of 37 m in 6MWT distance is clinically significant [32]. Therefore, we infer that even mild COVID-19 in SAH individuals may contribute to a decline in FC assessed by the 6MWT. Further studies with larger sample sizes are needed to confirm these findings.
In addition to the observed higher physical capacity in G2, a positive correlation was found between the distance covered on the 6MWT and the 2LV% HRV index. According to Porta et al. [20], this index corresponds to vagal modulation in the heart. Regular physical activity has been shown to positively affect the vagal system [33]. Thus, non-infected SAH individuals showed better physical capacity as assessed by the 6MWT due to greater vagal modulation.
Asarcikli et al. [34] evaluated both healthy and recovered COVID-19 individuals, performing 24-h Holter monitoring for HRV analysis. They observed significantly higher rMSSD, PNN50, and HF indices in the post-COVID-19 group. These authors justified that even 12 weeks after infection, increased parasympathetic tone can persist. Furthermore, they suggested that prolonged parasympathetic activity might be associated with chronic symptoms such as dyspnea, fatigue, and arthralgia, which affect physical capacity and quality of life. Another study compared 63 patients with mild to moderate COVID-19 symptoms to 43 healthy individuals matched for age and sex [35]. Increased parasympathetic modulation was observed in COVID-19 patients, independent of important factors like age, sex, and comorbidities. Authors reported alterations in HRV indices such as rMSSD and SDNN, as well as frequency domain indices (LF and HF) [35]. Additionally, 33% of the patients in the studied group had hypertension.
On the other hand, Marques et al. [36] studied 155 patients with long COVID and 94 healthy volunteers. They observed an overall reduction in HRV, with an increase in sympathetic modulation and a decrease in parasympathetic modulation in the COVID-19 group. The authors concluded that COVID-19 led to an increase in sympathetic influence at rest and a decrease in vagal modulation compared to the non-infected group, and that reduced vagal activity might be related to mediation of the inflammatory process.
In the present study, no differences in cardiac autonomic control were observed between SAH patients with or without COVID-19. While hypertension and reduced physical capacity can influence reduced HRV with increased sympathetic and decreased vagal modulation [8,37], it is still uncertain how COVID-19 affects hypertensive individuals. We hypothesized that COVID-19 might further reduce HRV due to the inflammatory process, even in mild cases of the disease [34,35]. These factors, even when considering that the non-COVID-19 group had better physical capacity, may have contributed to the absence of differences in HRV results.
Regarding cardiac autonomic control during the post-COVID-19 period, the literature remains contradictory. This could be due to the heterogeneity of the samples studied to date, varying times of analysis post-infection, as well as different methods of HRV analysis. Although this study considered linear and non-linear HRV approaches to assess CAF, the impact of the disease on the autonomic nervous system may be limited when univariate HRV analysis is applied [38]. The nature of COVID-19 is not fully understood and its impact on the cardiac neural regulation may be driven by changes in other subsystems, such as the respiratory and vascular systems. A previous study reported that changes in CAF, when comparing patients with serious and mild COVID-19, were only observed when bivariate analysis of heart rate and systolic blood pressure variabilities were considered [38]. Additionally, it is known that HRV fluctuates spontaneously according to respiratory cycles, which may be determined by physiological, mechanical, and neural mechanisms, such as changes in intrathoracic pressure, the central influence of respiratory centers modulating vagal motoneuron activity, as well as changes in stroke volume during respiratory phases [39]. Therefore, it seems reasonable that changes in pulmonary diffusion as well as lung capacities caused by the disease may affect the cardiac regulation driven by respiration, called cardiorespiratory coupling [40,41]. Thus, multivariate approaches considering blood pressure oscillations as well as cardiorespiratory coupling analysis would be useful in the future to elucidate the impact of COVID-19 on the autonomic nervous system.
The main limitations of this study included difficulties in participant recruitment due to pandemic-related factors, which resulted in a reduced sample size and potential bias in HRV data. The variation in the time post-SARS-CoV-2 infection for the assessment was not consistent among all participants (up to 6 months post-infection), which is a notable limitation. Also, our sample primarily comprised asymptomatic patients or individuals with mild symptoms during their infection. Consequently, they did not report lasting symptoms typically associated with post-COVID syndrome, especially those who were either hospitalized or received outpatient treatment [42,43]. Furthermore, the lack of gender matching may have also had an influence. Finally, the absence of a healthy control group without COVID-19 and hypertension limits our ability to make comprehensive comparisons.
On the other hand, the results of the study can have clinical implications for various healthcare professionals in guiding rehabilitation programs for individuals and improving their physical capacity. Additionally, this study provides insight into hypertensive individuals, a population with a high prevalence of complications and mortality associated with COVID-19. The results contribute to the scientific knowledge in this field and emphasize the need for more research to better understand the impact of COVID-19 on individuals with pre-existing conditions.

5. Conclusions

This study demonstrated the negative impact of post-COVID-19 on the functional capacity of medicated hypertensive individuals who experienced a mild clinical presentation of the disease. However, the study did not find impaired cardiac autonomic function in systemic arterial hypertension individuals post-COVID-19 compared to those without COVID-19.

Author Contributions

Conceptualization, J.C.M.-M., A.J.H., R.M.d.A., P.A.S. and V.R.N.; methodology, M.S.S., F.M.d.N.A.D., A.M.C. and V.R.N.; software, H.F.S.C., Á.A.F., P.I.L.R. and P.A.S.; validation, E.F.D.C., J.d.S.B., L.C.U., J.P.C.G. and V.R.N.; formal analysis, M.S.S., J.C.M.-M., R.M.d.A., A.J.H., A.M.C., P.A.S. and V.R.N.; investigation, E.F.D.C., M.S.S., H.F.S.C., Á.A.F., J.d.S.B., L.C.U., J.P.C.G., P.I.L.R. and F.M.d.N.A.D.; resources, R.M.d.A., P.A.S. and V.R.N.; data curation, J.C.M.-M., A.J.H. and A.M.C.; writing—original draft preparation, E.F.D.C., M.S.S., H.F.S.C., Á.A.F., P.I.L.R. and F.M.d.N.A.D.; writing—review and editing, J.C.M.-M., A.J.H., R.M.d.A., A.M.C., P.A.S. and V.R.N.; visualization, R.M.d.A.; supervision, J.C.M.-M.; project administration, P.A.S. and V.R.N.; funding acquisition, P.A.S. and V.R.N. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by Fundação de Amparo à Ciência e Tecnologia do Estado de Pernambuco (FACEPE), grant number APQ-1413-4.08/2; Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq), grant number 426706/2016-1; and Coordenação de Aperfeiçoamento de Pessoal de Nível Superior—Brazil (CAPES), Finance Code 001. Matheus Sobral Silveira was granted a doctorate scholarship by FACEPE, grant number IBPG-0550-4.09/22. Ádrya Aryelle Ferreira and Heitor Fernandes Silveira Cavalini were granted master’s scholarships by the CAPES Social Demand Program. Joice de Souza Batista and João Paulo Coelho Guimarães received scientific research scholarships from CNPq.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and approved by the Research Ethics Committee of the Centro Integrado de Saúde Amaury de Medeiros—CISAM (protocol code 4848824, signed on 29 June 2021).

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

Data from the present study can be made available for sharing purposes from the principal investigator upon reasonable request.

Acknowledgments

We are grateful to all the participants of the study.

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript; or in the decision to publish the results.

References

  1. Del Rio, R.; Marcus, N.J.; Inestrosa, N.C. Potential Role of Autonomic Dysfunction in COVID-19 Morbidity and Mortality. Front. Physiol. 2020, 11, 561749. [Google Scholar] [CrossRef] [PubMed]
  2. Gallo, G.; Calvez, V.; Savoia, C. Hypertension and COVID-19: Current Evidence and Perspectives. High Blood Press. Cardiovasc. Prev. 2022, 29, 115–123. [Google Scholar] [CrossRef] [PubMed]
  3. Hessami, A.; Shamshirian, A.; Heydari, K.; Pourali, F.; Alizadeh-Navaei, R.; Moosazadeh, M.; Abrotan, S.; Shojaie, L.; Sedighi, S.; Shamshirian, D.; et al. Cardiovascular Diseases Burden in COVID-19: Systematic Review and Meta-Analysis. Am. J. Emerg. Med. 2021, 46, 382–391. [Google Scholar] [CrossRef] [PubMed]
  4. Ssentongo, P.; Ssentongo, A.E.; Heilbrunn, E.S.; Ba, D.M.; Chinchilli, V.M. Association of Cardiovascular Disease and 10 Other Pre-Existing Comorbidities with COVID-19 Mortality: A Systematic Review and Meta-Analysis. PLoS ONE 2020, 15, e0238215. [Google Scholar] [CrossRef]
  5. Dhakal, B.P.; Sweitzer, N.K.; Indik, J.H.; Acharya, D.; William, P. SARS-CoV-2 Infection and Cardiovascular Disease: COVID-19 Heart. Heart Lung Circ. 2020, 29, 973–987. [Google Scholar] [CrossRef]
  6. Silva, R.N.; Goulart, C.D.L.; Oliveira, M.R.; Tacao, G.Y.; Back, G.D.; Severin, R.; Faghy, M.A.; Arena, R.; Borghi-Silva, A. Cardiorespiratory and Skeletal Muscle Damage Due to COVID-19: Making the Urgent Case for Rehabilitation. Expert Rev. Respir. Med. 2021, 15, 1107–1120. [Google Scholar] [CrossRef]
  7. Mancia, G.; Grassi, G. The Autonomic Nervous System and Hypertension. Circ. Res. 2014, 114, 1804–1814. [Google Scholar] [CrossRef]
  8. Singh, J.P.; Larson, M.G.; Tsuji, H.; Evans, J.C.; O’Donnell, C.J.; Levy, D. Reduced Heart Rate Variability and New-Onset Hypertension. Hypertension 1998, 32, 293–297. [Google Scholar] [CrossRef]
  9. Valensi, P. Autonomic Nervous System Activity Changes in Patients with Hypertension and Overweight: Role and Therapeutic Implications. Cardiovasc. Diabetol. 2021, 20, 170. [Google Scholar] [CrossRef]
  10. Leitzke, M.; Stefanovic, D.; Meyer, J.-J.; Schimpf, S.; Schönknecht, P. Autonomic Balance Determines the Severity of COVID-19 Courses. Bioelectron. Med. 2020, 6, 22. [Google Scholar] [CrossRef]
  11. Dani, M.; Dirksen, A.; Taraborrelli, P.; Torocastro, M.; Panagopoulos, D.; Sutton, R.; Lim, P.B. Autonomic Dysfunction in ‘Long COVID’: Rationale, Physiology and Management Strategies. Clin. Med. 2021, 21, e63–e67. [Google Scholar] [CrossRef] [PubMed]
  12. Carfì, A.; Bernabei, R.; Landi, F. Persistent Symptoms in Patients after Acute COVID-19. JAMA 2020, 324, 603. [Google Scholar] [CrossRef] [PubMed]
  13. Machado, F.V.C.; Meys, R.; Delbressine, J.M.; Vaes, A.W.; Goërtz, Y.M.J.; van Herck, M.; Houben-Wilke, S.; Boon, G.J.A.M.; Barco, S.; Burtin, C.; et al. Construct Validity of the Post-COVID-19 Functional Status Scale in Adult Subjects with COVID-19. Health Qual. Life Outcomes 2021, 19, 40. [Google Scholar] [CrossRef] [PubMed]
  14. Pedrosa, R.; Holanda, G. Correlation between the Walk, 2-Minute Step and Tug Tests among Hypertensive Older Women. Braz. J. Phys. Ther. 2009, 13, 252–256. [Google Scholar] [CrossRef]
  15. Barroso, W.K.S.; Rodrigues, C.I.S.; Bortolotto, L.A.; Mota-Gomes, M.A.; Brandão, A.A.; Feitosa, A.D.d.M.; Machado, C.A.; Poli-de-Figueiredo, C.E.; Amodeo, C.; Mion Júnior, D.; et al. Brazilian Guidelines of Hypertension—2020. Arq. Bras. Cardiol. 2021, 116, 516–658. [Google Scholar] [CrossRef]
  16. Porta, A.; Tobaldini, E.; Guzzetti, S.; Furlan, R.; Montano, N.; Gnecchi-Ruscone, T. Assessment of Cardiac Autonomic Modulation during Graded Head-up Tilt by Symbolic Analysis of Heart Rate Variability. Am. J. Physiol.-Heart Circ. Physiol. 2007, 293, H702–H708. [Google Scholar] [CrossRef]
  17. Dantas, E.M.; Sant’Anna, M.L.; Andreão, R.V.; Gonçalves, C.P.; Morra, E.A.; Baldo, M.P.; Rodrigues, S.L.; Mill, J.G. Spectral Analysis of Heart Rate Variability with the Autoregressive Method: What Model Order to Choose? Comput. Biol. Med. 2012, 42, 164–170. [Google Scholar] [CrossRef]
  18. Heart Rate Variability: Standards of Measurement, Physiological Interpretation and Clinical Use. Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology. Circulation 1996, 93, 1043–1065. [CrossRef]
  19. Guzzetti, S.; Borroni, E.; Garbelli, P.E.; Ceriani, E.; Bella, P.D.; Montano, N.; Cogliati, C.; Somers, V.K.; Mallani, A.; Porta, A. Symbolic Dynamics of Heart Ratevariability. Circulation 2005, 112, 465–470. [Google Scholar] [CrossRef]
  20. Porta, A.; Gnecchi-Ruscone, T.; Tobaldini, E.; Guzzetti, S.; Furlan, R.; Montano, N. Progressive Decrease of Heart Period Variability Entropy-Based Complexity during Graded Head-up Tilt. J. Appl. Physiol. 2007, 103, 1143–1149. [Google Scholar] [CrossRef]
  21. American Thoracic Society. ATS Statement: Guidelines for the Six-Minute Walk Test. Am. J. Respir. Crit. Care Med. 2002, 166, 111–117. [Google Scholar] [CrossRef] [PubMed]
  22. Holland, A.E.; Spruit, M.A.; Troosters, T.; Puhan, M.A.; Pepin, V.; Saey, D.; McCormack, M.C.; Carlin, B.W.; Sciurba, F.C.; Pitta, F.; et al. An Official European Respiratory Society/American Thoracic Society Technical Standard: Field Walking Tests in Chronic Respiratory Disease. Eur. Respir. J. 2014, 44, 1428–1446. [Google Scholar] [CrossRef] [PubMed]
  23. Borg, G.A. Psychophysical Bases of Perceived Exertion. Med. Sci. Sports Exerc. 1982, 14, 377–381. [Google Scholar] [CrossRef] [PubMed]
  24. Iwama, A.M.; Andrade, G.N.; Shima, P.; Tanni, S.E.; Godoy, I.; Dourado, V.Z. The Six-Minute Walk Test and Body Weight-Walk Distance Product in Healthy Brazilian Subjects. Braz. J. Med. Biol. Res. 2009, 42, 1080–1085. [Google Scholar] [CrossRef]
  25. Dourado, V.Z.; Nishiaka, R.K.; Simões, M.S.M.P.; Lauria, V.T.; Tanni, S.E.; Godoy, I.; Gagliardi, A.R.T.; Romiti, M.; Arantes, R.L. Classification of Cardiorespiratory Fitness Using the Six-Minute Walk Test in Adults: Comparison with Cardiopulmonary Exercise Testing. Pulmonology 2021, 27, 500–508. [Google Scholar] [CrossRef]
  26. Armitage, P.; Berry, G.; Matthews, J.N.S. Statistical Methods in Medical Research, 4th ed.; Wiley-Blackwell: Hoboken, NJ, USA, 2013; ISBN 978-0-632-05257-8. [Google Scholar]
  27. Cohen, J. Statistical Power Analysis for the Behavioral Sciences, 2nd ed.; Routledge: New York, NY, USA, 2013; ISBN 9781134742707. [Google Scholar]
  28. Peroy-Badal, R.; Sevillano-Castaño, A.; Torres-Castro, R.; García-Fernández, P.; Maté-Muñoz, J.L.; Dumitrana, C.; Sánchez Rodriguez, E.; de Frutos Lobo, M.J.; Vilaró, J. Comparison of Different Field Tests to Assess the Physical Capacity of Post-COVID-19 Patients. Pulmonology, 2022; online ahead of print. [Google Scholar] [CrossRef]
  29. Spruit, M.A.; Holland, A.E.; Singh, S.J.; Tonia, T.; Wilson, K.C.; Troosters, T. COVID-19: Interim Guidance on Rehabilitation in the Hospital and Post-Hospital Phase from a European Respiratory Society- and American Thoracic Society-Coordinated International Task Force. Eur. Respir. J. 2020, 56, 2002197. [Google Scholar] [CrossRef]
  30. Wong, A.W.; López-Romero, S.; Figueroa-Hurtado, E.; Vazquez-Lopez, S.; Milne, K.M.; Ryerson, C.J.; Guenette, J.A.; Cortés-Telles, A. Predictors of Reduced 6-Minute Walk Distance after COVID-19: A Cohort Study in Mexico. Pulmonology 2021, 27, 563–565. [Google Scholar] [CrossRef]
  31. Huang, Y.; Tan, C.; Wu, J.; Chen, M.; Wang, Z.; Luo, L.; Zhou, X.; Liu, X.; Huang, X.; Yuan, S.; et al. Impact of Coronavirus Disease 2019 on Pulmonary Function in Early Convalescence Phase. Respir. Res. 2020, 21, 163. [Google Scholar] [CrossRef]
  32. Bohannon, R.W.; Crouch, R. Minimal Clinically Important Difference for Change in 6-minute Walk Test Distance of Adults with Pathology: A Systematic Review. J. Eval. Clin. Pract. 2017, 23, 377–381. [Google Scholar] [CrossRef]
  33. Melo, R.C.; Quiterio, R.J.; Takahashi, A.C.M.; Silva, E.; Martins, L.E.B.; Catai, A.M. High Eccentric Strength Training Reduces Heart Rate Variability in Healthy Older Men. Br. J. Sports Med. 2007, 42, 59–63. [Google Scholar] [CrossRef] [PubMed]
  34. Asarcikli, L.D.; Hayiroglu, M.İ.; Osken, A.; Keskin, K.; Kolak, Z.; Aksu, T. Heart Rate Variability and Cardiac Autonomic Functions in Post-COVID Period. J. Interv. Card. Electrophysiol. 2022, 63, 715–721. [Google Scholar] [CrossRef] [PubMed]
  35. Kaliyaperumal, D.; RK, K.; Alagesan, M.; Ramalingam, S. Characterization of Cardiac Autonomic Function in COVID-19 Using Heart Rate Variability: A Hospital Based Preliminary Observational Study. J. Basic Clin. Physiol. Pharmacol. 2021, 32, 247–253. [Google Scholar] [CrossRef] [PubMed]
  36. Marques, K.C.; Silva, C.C.; Trindade, S.d.S.; Santos, M.C.d.S.; Rocha, R.S.B.; Vasconcelos, P.F.d.C.; Quaresma, J.A.S.; Falcão, L.F.M. Reduction of Cardiac Autonomic Modulation and Increased Sympathetic Activity by Heart Rate Variability in Patients with Long COVID. Front. Cardiovasc. Med. 2022, 9, 862001. [Google Scholar] [CrossRef]
  37. Rodrigues, P.H.; Borges de Oliveira, M.; Cazelato, L.; Federighi Baisi Chagas, E.; Quitério, R.J. The Influence of Risk Factors for Cardiovascular Diseases on Cardiac Autonomic Modulation. Rev. Bras. Ciências Saúde USCS 2016, 14, 34–40. [Google Scholar] [CrossRef]
  38. Bajić, D.; Đajić, V.; Milovanović, B. Entropy Analysis of COVID-19 Cardiovascular Signals. Entropy 2021, 23, 87. [Google Scholar] [CrossRef]
  39. De Abreu, R.M.; Cairo, B.; Porta, A. On the Significance of Estimating Cardiorespiratory Coupling Strength in Sports Medicine. Front. Netw. Physiol. 2022, 2, 1114733. [Google Scholar] [CrossRef]
  40. Thomas, M.; Price, O.J.; Hull, J.H. Pulmonary Function and COVID-19. Curr. Opin. Physiol. 2021, 21, 29–35. [Google Scholar] [CrossRef]
  41. Dick, T.E.; Hsieh, Y.-H.; Dhingra, R.R.; Baekey, D.M.; Galán, R.F.; Wehrwein, E.; Morris, K.F. Cardiorespiratory Coupling: Common Rhythms in Cardiac, Sympathetic, and Respiratory Activities. Prog. Brain Res. 2014, 209, 191–205. [Google Scholar] [CrossRef]
  42. Scala, I.; Rizzo, P.A.; Bellavia, S.; Brunetti, V.; Colò, F.; Broccolini, A.; Della Marca, G.; Calabresi, P.; Luigetti, M.; Frisullo, G. Autonomic Dysfunction during Acute SARS-CoV-2 Infection: A Systematic Review. J. Clin. Med. 2022, 11, 3883. [Google Scholar] [CrossRef]
  43. Natarajan, A.; Shetty, A.; Delanerolle, G.; Zeng, Y.; Zhang, Y.; Raymont, V.; Rathod, S.; Halabi, S.; Elliot, K.; Shi, J.Q.; et al. A Systematic Review and Meta-Analysis of Long COVID Symptoms. Syst. Rev. 2023, 12, 88. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Flowchart of the study.
Figure 1. Flowchart of the study.
Jpm 13 01391 g001
Figure 2. Correlation between the distance covered in the 6 min walk test and the 2LV% index of heart rate variability acquired in the supine position from individuals with systemic arterial hypertension (n = 32). (a) Eighteen post-COVID-19 individuals—G1, 2LV% from post-COVID-19 individuals; (b) fourteen non-COVID-19-diagnosed—G2, 2LV% from non-COVID-19-diagnosed.
Figure 2. Correlation between the distance covered in the 6 min walk test and the 2LV% index of heart rate variability acquired in the supine position from individuals with systemic arterial hypertension (n = 32). (a) Eighteen post-COVID-19 individuals—G1, 2LV% from post-COVID-19 individuals; (b) fourteen non-COVID-19-diagnosed—G2, 2LV% from non-COVID-19-diagnosed.
Jpm 13 01391 g002
Table 1. Characteristics of the individuals with systemic arterial hypertension (SAH) divided into infected with SARS-CoV-2 (SAH + COVID-19) or without clinical diagnosis of COVID-19 (SAH).
Table 1. Characteristics of the individuals with systemic arterial hypertension (SAH) divided into infected with SARS-CoV-2 (SAH + COVID-19) or without clinical diagnosis of COVID-19 (SAH).
VariablesTotal
(N = 40)
SAH + COVID-19
G1 (n = 21)
SAH
G2 (n = 19)
p
Age 1, years53 ± 1253 ± 1353 ± 111.000
Male gender, n (%)22 (55%)9 (43%)13 (68%)0.105
Height 1, meters1.65 ± 0.111.63 ± 0.101.67 ± 0.120.258
Total body mass 1, kg85.8 ± 18.487.7 ± 22.483.9 ± 13.90.528
Body mass index 1, kg/m231.4 ± 5.432.6 ± 6.829.9 ± 3.40.126
Antihypertensives use, n (%)40 (100%)21 (100%)19 (100%)1.000
Obesity, n (%)21 (53%)12 (57%)9 (47%)0.382
Sedentary lifestyle, n (%)25 (63%)15 (71%)10 (53%)0.220
Smoking, n (%)2 (5%)0 (-)2 (11%)0.127
Hypothyroidism, n (%)2 (5%)2 (10%)0 (-)0.168
Dyslipidemia, n (%)11 (28%)6 (29%)5 (26%)0.873
Statins use, n (%)7 (18%)4 (19%)3 (16%)0.073
Diabetes mellitus, n (%)11 (28%)6 (29%)5 (26%)0.873
Hypoglycemics use, n (%)11 (28%)6 (29%)5 (26%)0.873
1 Data are shown as mean ± standard deviation.
Table 2. Comparison of linear and non-linear analysis of heart rate variability (HRV) indices in individuals with systemic arterial hypertension stratified into SARS-CoV-2-infected (G1) and non-COVID-19-diagnosed (G2) groups (n = 40).
Table 2. Comparison of linear and non-linear analysis of heart rate variability (HRV) indices in individuals with systemic arterial hypertension stratified into SARS-CoV-2-infected (G1) and non-COVID-19-diagnosed (G2) groups (n = 40).
HRV IndicesG1 (n = 21)
Mean ± SD
G2 (n = 19)
Mean ± SD
pd
Linear analysis
 Mean iRR, ms784.9 ± 109.3848.0 ± 169.90.2910.44
 iRR variance, ms2540.9 ± 484.5787.8 ± 761.10.3430.39
Spectral analysis
 LFun, un44.1 ± 21.045.7 ± 21.10.8110.07
 HF, ms165.6 ± 192.3298.3 ± 489.40.7250.36
Non-linear analysis
 0V%22.1 ± 14.720.2 ± 12.80.1680.14
 1V%46.4 ± 6.944.0 ± 4.80.8730.41
 2LV%10.7 ± 6.913.4 ± 7.80.0730.36
 2UV%20.8 ± 11.922.4 ± 13.50.8730.13
Entropy
 Shannon entropy3.51 ± 0.443.62 ± 0.400.4100.26
 Complexity index1.07 ± 0.181.10 ± 0.190.6600.16
 Normalized complexity index0.76 ± 0.090.74 ± 0.090.5420.22
SD: standard deviation; iRR: R–R interval; LFun: low frequency expressed in normalized units; HF: high frequency; 0V%: patterns without variations; 1V%: patterns with one variation; 2LV%: patterns with two equal variations; 2UV%: patterns with two different variations.
Table 3. Comparison of the functional capacity by individuals with systemic arterial hypertension with low cardiorespiratory fitness stratified into SARS-CoV-2-infected or post-COVID-19 individuals (G1) and non-COVID-19-diagnosed (G2) groups submitted to 6 min walk test (n = 32).
Table 3. Comparison of the functional capacity by individuals with systemic arterial hypertension with low cardiorespiratory fitness stratified into SARS-CoV-2-infected or post-COVID-19 individuals (G1) and non-COVID-19-diagnosed (G2) groups submitted to 6 min walk test (n = 32).
VariablesG1 (n = 18)G2 (n = 14)MDpd
BaselineAfterBaselineAfter
Mean ± SDMean ± SDMean ± SDMean ± SD
Heart rate frequency, bpm79 ± 11106 ± 1674 ± 16102 ± 2540.5800.19
Systolic blood pressure, mmHg135 ± 12137 ± 13129 ± 25132 ± 2750.6000.24
Diastolic blood pressure, mmHg90 ± 990 ± 1087 ± 1887 ± 1930.6100.20
Perceived exertion (Borg scale), n0 ± 13 ± 20 ± 13 ± 200.8500.00
Oxygen saturation (SpO2), %98 ± 197 ± 195 ± 195 ± 420.3330.68
6 min walk distance covered, m-464.7 ± 59.5-522.2 ± 77.6−580.0240.83
Δ predicted and covered, %-85 ± 12-93 ± 10−80.049-
SD: standard deviation; MD: mean difference after the test between the two groups.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Cunha, E.F.D.; Silveira, M.S.; Milan-Mattos, J.C.; Cavalini, H.F.S.; Ferreira, Á.A.; Batista, J.d.S.; Uzumaki, L.C.; Guimarães, J.P.C.; Roriz, P.I.L.; Dantas, F.M.d.N.A.; et al. Cardiac Autonomic Function and Functional Capacity in Post-COVID-19 Individuals with Systemic Arterial Hypertension. J. Pers. Med. 2023, 13, 1391. https://doi.org/10.3390/jpm13091391

AMA Style

Cunha EFD, Silveira MS, Milan-Mattos JC, Cavalini HFS, Ferreira ÁA, Batista JdS, Uzumaki LC, Guimarães JPC, Roriz PIL, Dantas FMdNA, et al. Cardiac Autonomic Function and Functional Capacity in Post-COVID-19 Individuals with Systemic Arterial Hypertension. Journal of Personalized Medicine. 2023; 13(9):1391. https://doi.org/10.3390/jpm13091391

Chicago/Turabian Style

Cunha, Edelvita Fernanda Duarte, Matheus Sobral Silveira, Juliana Cristina Milan-Mattos, Heitor Fernandes Silveira Cavalini, Ádrya Aryelle Ferreira, Joice de Souza Batista, Lara Cazé Uzumaki, João Paulo Coelho Guimarães, Pedro Igor Lustosa Roriz, Fabianne Maisa de Novaes Assis Dantas, and et al. 2023. "Cardiac Autonomic Function and Functional Capacity in Post-COVID-19 Individuals with Systemic Arterial Hypertension" Journal of Personalized Medicine 13, no. 9: 1391. https://doi.org/10.3390/jpm13091391

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop