Next Article in Journal
Prevalence Rates of Loneliness and Its Impact on Lifestyle in the Healthy Population of Madrid, Spain
Next Article in Special Issue
Addition of Probiotics to Anti-Obesity Therapy by Percutaneous Electrical Stimulation of Dermatome T6. A Pilot Study
Previous Article in Journal
The Feasibility of Connecting Conversations: A Narrative Method to Assess Experienced Quality of Care in Nursing Homes from the Resident’s Perspective
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Project Report

Evaluation of Changes in Gut Microbiota in Patients with Crohn’s Disease after Anti-Tnfα Treatment: Prospective Multicenter Observational Study

by
Laura Sanchis-Artero
1,
Juan Francisco Martínez-Blanch
2,*,
Sergio Manresa-Vera
2,
Ernesto Cortés-Castell
3,
Josefa Rodriguez-Morales
1 and
Xavier Cortés-Rizo
1
1
Inflammatory Bowel Disease Unit. Department of Digestive Diseases, Hospital of Sagunto, Av. Ramón y Cajal s/n, 46520 Sagunto, Valencia, Spain
2
Genomics Laboratory. ADM-Lifesequencing. Parque Científico Universidad de Valencia. Catedrático Agustín Escardino Benlloch, 9. Edificio 2, 46980 Paterna, Valencia, Spain
3
Department of Pharmacology, Pediatrics and Organic Chemistry Miguel Hernández University, Carretera de Valencia—Alicante S/N, 03550 San Juan de Alicante, Alicante, Spain
*
Author to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2020, 17(14), 5120; https://doi.org/10.3390/ijerph17145120
Submission received: 4 June 2020 / Revised: 10 July 2020 / Accepted: 10 July 2020 / Published: 15 July 2020
(This article belongs to the Special Issue New Perspectives in Health: Gut Microbiota)

Abstract

:
Background: Crohn’s disease is believed to result from the interaction between genetic susceptibility, environmental factors and gut microbiota, leading to an aberrant immune response. The objectives of this study are to evaluate the qualitative and quantitative changes in the microbiota of patients with Crohn’s disease after six months of anti-tumor-necrosis factor (anti-TNFα) (infliximab or adalimumab) treatment and to determine whether these changes lead to the recovery of normal microbiota when compared to a control group of healthy subjects. In addition, we will evaluate the potential role of the Faecalibacterium prausnitzii/Escherichia coli and Faecalibacterium prausnitzii/Clostridium coccoides ratios as indicators of therapeutic response to anti-TNFα drugs. Methods/Design: This prospective multicenter observational study will comprise a total of 88 subjects: 44 patients with Crohn’s disease scheduled to start anti-TNFα treatment as described in the drug specifications to control the disease and 44 healthy individuals who share the same lifestyle and eating habits. The presence of inflammatory activity will be determined by the Harvey-Bradshaw index, analytical parameters in blood, including C-reactive protein, and fecal calprotectin levels at commencement of the study, at three months and at six months, allowing the classification of patients into responders and non-responders. Microbiota composition and the quantitative relationship between Faecalibacterium prausnitzii and Escherichia coli and between Faecalibacterium prausnitzii and Clostridium coccoides group as indicators of dysbiosis will be studied at inclusion and six months after initiation of treatment using ultra sequencing with Illumina technology and comparative bioinformatics analysis for the former relationship, and digital droplet PCR using stool samples for the latter. Upon inclusion, patients will complete a survey of dietary intake for the three days prior to stool collection, which will be repeated six months later in a second collection to minimize dietary bias. Discussion: In this study, massive sequencing, a reliable new tool, will be applied to identify early biomarkers of response to anti-TNF treatment in patients with Crohn’s disease to improve clinical management of these patients, reduce morbidity rates and improve efficiency.

1. Background

Gut microbiota is defined as the total number of living microorganisms (bacteria, fungi, archaea, viruses and others) present in the intestine. The gut microbiome is a taxonomic characterization of microbial diversity including the set of genomes and, via genes, their physicochemical capabilities [1,2]. In healthy individuals, the gut bacterial microbiota is composed of more than 1018 different microorganisms. The vast majority of these are bacteria, with some 1100 more prevalent species; of these an estimated 160 species of bacteria are specific to each individual [3].
The microbiome, therefore, is a highly complex structure, involving thousands of microorganisms belonging to very different taxonomic classifications and, consequently, millions of associations between them, making its study a great challenge [4]. Through advances in bioinformatics, in 2003 the human genome was decoded, a milestone in science. Since then, much attention has been focused on deciphering this extensive network of microbes, the microbiota, also known as the second human genome, or as another organ. These microbes coexist with us and have a larger total number of genes than the human genome. In short, there is no simple description of these structures. However, due to their importance, considerable interest has been generated in the identification of patterns associated with human health and disease states which may even lead to the development of microbiota-based diagnostics and therapies as well as having implications for nutritional or pharmaceutical interventions [2]. To this end, reproducible patterns of gut microbiome variation have been observed in healthy adults, determining the existence of three major microbiota communities, based on the predominance or absence of species of the key genera [5]. Any other combination of key genera or genera not described by Arumugam et al. [5], together with a reduction in biodiversity, is considered dysbiosis [6,7]. Numerous technologies have been applied with the aim of examining the gut microbiota, which has resulted in the capacity and cost of microbiota research being significantly reduced in recent years, mainly due to advances in massive sequencing technologies such as next-generation sequencing. These techniques allow information of interest to be obtained quickly and efficiently by sequencing regions of the prokaryotic 16S ribosomal RNA gene [8].
In inflammatory bowel disease (IBD), the role of the microbiota in disease development and onset is very clear. While it is true that to date no specific pathogen has been conclusively identified as a trigger, we do know that, for the disease to develop, dysbiosis or a definitive change in the intestinal microbiota must occur and is likely to be the defining event in the development of Crohn’s disease (CD). In IBD, it has been documented that the gut microbiota bacterial composition transitions from saprophytic to predominantly pathogenic [9]. Indeed, there is evidence of a significant increase in Escherichia coli concentrations, including pathogenic variants, in CD patients with ileal involvement [10]. It unknown at present whether dysbiosis is a cause or a consequence of the development of CD. It appears that the combination of a genetic predisposition and an alteration in gut microbiota are the final triggers of a chronic IBD-type inflammatory process. Specifically, we know that the disease develops in genetically susceptible individuals through dysregulation of homeostasis between commensal microbiota and/or other environmental elements and an altered immune response in the patient. An error in the interpretation of the stimulus or in the regulation of the immune response leads to an imbalance between pro- and anti-inflammatory factors, perpetuating the inflammatory process [11].
The gut microbiota play a crucial role in the development of the immune system and maintenance of the intestinal epithelial barrier. Inflamed ileal mucosa in CD patients shows increased production of tumor necrosis factor-α (TNFα), compared to normal ileum, induced by a dysbiosis in the gut microbiota, with a significant increase in bacteria that stimulate TNF production. Numerous bacteria in the commensal microbiota inhibit the release of TNF and other pro-inflammatory cytokines, e.g., bacteria that produce short-chain fatty acids (SCFA), inducing a potent anti-inflammatory effect in the intestinal mucosa. In contrast, other types of bacteria, such as Escherichia coli ECOR-26, which have been linked to CD, induce increased TNF release and stimulation of IL-10 release [12]. Current hypotheses in favor of a higher release of TNFα induced by intestinal dysbiosis support the idea that restoration of a less pathogenic microbe in the intestinal mucosa (inducing a reduced release of TNF and other pro-inflammatory cytokines) could lead to better disease control [13].
Today, anti-TNF therapy is one of the therapeutic pillars in the management of CD, but these drugs only treat the consequences of the disease and not the possible cause. Approximately one quarter of CD patients will be primary non-responders to anti-TNF agents, and one third of responders will experience a loss of response over time [14]. To improve treatment effectiveness, it is essential to study why these patients do not have an optimal response.
Traditionally, the prognosis and monitoring of treatments in patients with CD have been limited to the control of clinical symptoms (e.g., through the Harvey-Bradshaw Index [HBI]: see Table 1) accompanied by imaging techniques (primarily endoscopy and magnetic resonance imaging). However, these tools for assessing disease activity have many drawbacks and limitations. Among them, clinical scoring systems are highly subjective and can be misleading in this disease, characterized by alternating periods of exacerbation and remission [15,16]. Ileo-colonoscopy with biopsies is the current gold standard for the diagnosis and evaluation of inflammatory activity, with the great disadvantage that it is an invasive procedure [17] and that it is not always possible to reach the diseased area. In an attempt to identify non-invasive markers, fecal calprotectin (FC), a protein originating from the migration of neutrophils to the intestinal mucosa, was introduced as an indirect trait of intestinal inflammation, allowing more objective monitoring than clinical indices, although with low specificity and low positive predictive value depending on the chosen cut-off point [18,19].
Currently, the role of bacterial gut microbiota is described as a key factor in the development of CD. Various authors defend the reduction in biodiversity and abundance of the phyla Bacteroidetes and Firmicutes such as Faecalibacterium prausnitzii (SCFA-producing bacteria), as well as an increase in the phylum Proteobacteria such as Escherichia coli, characteristic of patients with this disease compared to healthy individuals [6,7,20]; a decrease in abundance of this species has even been observed after anti-TNF treatment [20]. In this line, one study has identified certain specific microbial profiles that correlate with the recurrence of disease after achieving remission with infliximab treatment [14]. Several studies have also shown that the greater abundance of SCFA-producing bacteria predicted the effectiveness of infliximab [21,22] and another study associated the greater abundance of SCFA with a sustained response to this treatment [23].
A greater understanding of the composition of the bacterial gut microbiota in CD patients, such as the persistence of a significant proportion of certain pathogenic bacteria or low bacterial biodiversity, would make it possible to determine the role of gut microbiota in therapeutic responses and to establish biomarkers of response and relapse, as well as to determine whether it is necessary to restore intestinal normo-biosis in these patients. In addition, different gut microbiota profiles can be found, which enable us to predict the response to different therapeutic lines, thus being more efficient from the outset. Accordingly, this study has been designed to analyze biomarkers of response to anti-TNF treatment in CD through gut microbiota as an alternative non-invasive tool for predicting treatment response.

2. Methods

2.1. Ethics, Consent and Permission

The final protocol was approved by the Sagunto Hospital Clinical Research Ethics Committee, in accordance with applicable national and local laws and requirements. The study was classified by the Spanish Agency for Medicine and Health Products as a prospective follow-up post-authorization study.
The study adheres to the European guidelines for the protection of human research subjects, the Declaration of Helsinki and the recommendations of the European Network of Centres for Pharmacoepidemiology and Pharmacovigilance. Approval was obtained from all local ethics committees at all participating centers. Prior to inclusion, each patient will receive a detailed report on the nature, scope and possible consequences of the study from a physician and then will provide written informed consent. No action specifically required for the study will be taken without the valid consent of the patient.

2.2. Investigators

This multicenter study in the Valencian Community (Spain) includes eight academic medical centers. The patients will be recruited by the respective participating centers of the Public Health System of the Valencian Community, in which there are approximately 15,000 patients with IBD [24]. All the researchers will be gastroenterologists with experience in the follow-up and treatment of patients with CD. The centers participating in the study will have a total load of 3600 patients with CD, 10–15% of whom will be candidates for biological treatment according to estimated data from each center.

2.3. Study Objectives

The main objective of this study is to evaluate the modification of the gut microbiota in CD patients prior to and six months after anti-TNF therapy (infliximab or adalimumab). The secondary objectives are to evaluate the association between the changes in microbiota and the clinical, biological and endoscopic response of the patients; to correlate the normalization of the gut microbiota with the response to anti-TNF treatment; to determine the level of biodiversity of the fecal microbiota at the inclusion and completion of the study in each participant; to evaluate the potential role of the Faecalibacterium prausnitzii/Escherichia coli and Faecalibacterium prausnitzii/Clostridium coccoides ratios as an indicator of therapeutic response; and finally, to describe the clinical, biological and epidemiological characteristics of the patients included in the study prior to and six months after anti-TNF treatment.

2.4. Primary Study Variable

Normalization of gut microbiota after treatment: percentage of patients with dysbiosis of the gut microbiota before the introduction of anti-TNF therapy whose microbiota is normalized after six months of treatment. Dysbiosis is defined as a gut microbiota pattern different from the established patterns of normality according to Arumugam et al. [5]: enterotype 1 (ET B) predominantly contains Bacteroides, enterotype 2 (ET P) is characterized by the high abundance of Prevotella inversely correlated with Bacteroides, and enterotype 3 (ET F) can be distinguished by the presence of Firmicutes, highlighting the genus Ruminococcus.

2.5. Secondary Study Variables

Percentage of patients with dysbiosis at inclusion (after initiation of anti-TNF treatment) and at study completion, measured as dichotomous qualitative variables.
Levels of biodiversity of the fecal microbiota for the microbiome analyzed are defined as follows: low biodiversity is a total of five or fewer species, medium biodiversity is a total of between 6 and 10 species, and high biodiversity is a total of 11 or more species present.
Determine the increase in biodiversity pre- and post-anti-TNF treatment, defining an improvement as an increase in the number of species greater than or equal to five with respect to baseline.
Determine the relationship between Faecalibacterium prausnitzii/Escherichia coli and Faecalibacterium prausnitzii/Clostridium coccoides pre- and post-anti-TNF treatment.
Associate the presence and type of modifications in the gut microbiota after anti-TNF treatment with the clinical and biological response of the patient during the study.
Definitions:
Clinical remission: HBI ≤4 (Table 1)
Clinically active disease: HBI >4
Clinical response: when the HBI falls by three or more points.
Relapse: increased activity assessed by clinical, laboratory, radiology or endoscopic findings leading to a change in treatment to control the disease or an HBI > 4.
Biological remission: C-reactive protein (CRP) < 5 mg/L and (FC) < 250 μg/g.
Active biological disease: CRP ≥ 5 mg/L and FC ≥ 250 μg/g.
Overall response: the evaluation of clinical response will be established subjectively by the responsible physician according to clinical and analytical parameters, classifying patients as non-responders, responders without remission and responders in remission.
Clinical, demographic and complementary test variables included in the study protocol (Table 2):
Epidemiological characteristics: age, sex, smoking habits and body mass index.
Clinical characteristics: date of diagnosis and disease pattern, Montreal classification, activity index, presence of extra-intestinal manifestations, pharmacological history, concomitant medication, adverse effects, presence/absence of initial and final overall response, and clinical decisions derived from this response.
Anti-TNF treatment data: indication, type, start date, induction pattern, maintenance pattern, specify whether anti-TNF drugs were used prior to 24 weeks before inclusion and reason for discontinuation.
Complementary examinations for each patient:
-
Laboratory tests: complete blood count, erythrocyte sedimentation rate, CRP, fibrinogen, ferritin, transferrin saturation index, total proteins, albumin, urea, creatinine, GOT/AST (Aspartate Aminotransferase), GPT/ALT (Alanine Aminotransferase), GGT (Gamma-glutamyl transpeptidase), ALP (Alkaline Phosphatase), cholesterol and triglycerides
-
FC
-
Stool culture, parasites and Clostridium difficile toxin in feces
-
Stool collection for microbiota analysis
-
Radiological (ultrasound/computerized tomography/magnetic resonance imaging) and/or endoscopic testing if available prior to treatment (at least 12 weeks prior to inclusion) or 6 months after treatment.
-
72-h dietary record prior to stool sample collection for microbiota analysis. Patients will be provided with a daily survey in which they must record food and beverage intake, specifying characteristics, quantity and brands of packaged products.
-
Record of adherence to the Mediterranean diet. Together with the dietary record, the patients will also complete a validated survey of adherence to the Mediterranean diet [25] classifying this adherence as low (0–6 points), moderate (7–10 points) or high (11–14 points).

2.6. Study Design

This prospective, observational, multicenter study will include patients with CD who require anti-TNF therapy to control IBD. Anti-TNF (infliximab or adalimumab) treatment will be initiated under medical prescription according to the drug data sheet and the MAISE (drugs with a high health or economic impact) of the Valencian Health Agency. Administration of treatment will be performed in routine clinical practice and will not be promoted by this study. Fecal samples will be collected for gut microbiota analysis prior to exposure to the anti-TNF drug and six months after commencement of treatment. The patients will be monitored prospectively by their responsible physician during routine clinical practice during the first six months of anti-TNF treatment, recording the presence of inflammatory activity and whether there was no response, partial response or clinical remission at three months and at the end of the study follow-up.
Data to be recorded will include the presence of inflammatory activity (through calculation of the HBI), laboratory analytical data (complete blood count, general biochemistry, CRP) and the FC value prior to and three and six months after anti-TNF drug exposure, as well as treatments at the time of inclusion and any treatment changes throughout the duration of the study. The patient will be instructed to avoid taking probiotics and antibiotics during the study period. However, if antibiotics have to be taken, the patient must contact the study coordinator. In all cases, the second stool sample for mass sequencing will be performed after four weeks without treatment with probiotics/antibiotics, maintaining the same dietary conditions to avoid variability.
Whenever available, radiological and endoscopic activity data used in clinical practice to assess disease activity will be collected when performed, up to 12 weeks prior to patient inclusion and during the study.
Patients who undergo surgical resection within the study period will remain included but will be classified as non-responders to anti-TNF. Prior to surgery, clinical evaluation will be performed and blood and stool samples collected for fecal microbiota analysis, identical to the last determination at week 24 performed per protocol.
The patients will be provided with a 72-h dietary survey designed to record their dietary intake three days prior to stool collection at inclusion, repeated at six months to minimize bias concerning diet. Similarly, patients must complete a validated survey of adherence to the Mediterranean diet at inclusion and at six months to assess significant changes in their eating habits. An outline of the different data collection phases is shown in Figure 1.

2.7. Inclusion Criteria

All the patients must be adults (aged ≥ 18 years) diagnosed with CD with a clinical course of moderate-severe disease requiring anti-TNF treatment, in whom the indication for such treatment was inflammatory bowel activity.

2.8. Exclusion Criteria

Excluded from the study will be all patients who had taken antibiotics, probiotics, or proton pump inhibitors four weeks prior to the start of the study and stool collection; patients with chronic hepatitis C virus and chronic HIV infection; indication for anti-TNF treatment for reasons other than control of their luminal disease (e.g., enteropathic arthropathy, perianal disease, or prevention of recurrence); patients with previous ileum or colon surgery; and patients who will have received previous anti-TNF treatment in the 24 weeks prior to the start of the study.

2.9. Sample Size Considerations

Few studies have evaluated the variability in gut microbiota after anti-TNF therapy in adult patients with IBD [14,21,22,23,26,27,28,29,30]. However, from the results obtained it is not possible to determine the exact percentage of expected variability. Accepting an alpha risk of 0.05 and a beta risk of 0.2 in a bilateral contrast, 44 subjects will be required assuming that the initial proportion of events is 0.95 (degree of dysbiosis in the population at the start of the study) and 0.70 at the end. A loss to follow-up rate of 10% has been estimated.

2.10. Planning of the Sample Collection

The participating centers will be responsible for taking samples and determining conventional analytical and fecal data (complete blood count, stool samples, parasitology, FC, etc.) within routine clinical practice. The specific samples for the analysis of microbiota to be conducted externally will be collected by the patients themselves. The samples must be frozen after collection (by the patient in his or her freezer) until they are delivered to the laboratory of the hospital of origin, where they will be stored at −20 °C. The samples will subsequently be centralized.

2.11. Sequencing and Bioinformatics

The samples will be coded and the bacterial microbiota present will be analyzed using capture of the v3-v4 region of the 16S rRNA subunit [8], ultra-sequencing with Library Illumina 15044223 B protocol (ILLUMINA) comparative bioinformatics analysis. From 200 mg of stool, DNA will be obtained through a combination of mechanical and enzymatic lysis, and purified using the PowerFecal Pro DNA isolation (Qiagen) protocol with modifications. The DNA will be processed and prepared for sequencing. The sequences obtained will be filtered by parameters of quality (threshold value Q20) and length (sequences greater than 250 nucleotides). This strategy avoids erroneous ascriptions that generate an incorrect distribution of taxa. The minimum number of readings per sample will be 5000 and the mean length greater than 400 nt. A rarefaction analysis will be performed on the sample sequences to assess saturation for microbial biodiversity. Subsequently, the analysis of taxonomic identification at different taxonomic levels will be performed using the Microbiome bioinformatics with QIIME 2 2019.4 protocol [31] (Figure 2).

2.12. Identification and Evaluation of Potential Biomarkers

To identify potential non-invasive biomarkers from the characterization of the intestinal microbiome in various stages of CD, we will use LEfSe (Linear Discriminant Analysis Effect Size), an algorithm designed for the discovery of metagenomic biomarkers through class comparison, biological consistency testing and effect size estimation [32].
Due to the need to develop a rapid method of analysis such as the ratios of Faecalibacterium prausnitzii/Escherichia coli and Faecalibacterium prausnitzii/Clostridium coccoides group, a triplex digital droplet PCR will be implemented to allow the absolute quantification of the Faecalibacterium prausnitzii, Escherichia coli and Clostridium coccoides species, as well as the total number of bacteria present in a given sample through the 16S rRNA gene, all with a minimum volume and in a single reaction. Therefore, the appropriate primers and probes have been selected to perform this digital droplet PCR (Table 3), as well as to optimize the different concentrations and hybridization temperatures, based on numerous relevant studies [20,33,34,35,36].

2.13. Anti-TNF Dosage and Safety Evaluation

Anti-TNFs are monoclonal antibodies that neutralize pro-inflammatory cytokine tumor necrosis factor (TNF) involved in the inflammatory cascade of CD and other immune-mediated diseases. The anti-TNFs used are infliximab and adalimumab by indication for induction of remission in patients with moderate-severe CD. The administered doses of anti-TNFs are those listed in the data sheet, both at induction and maintenance doses. Infliximab dose regimen for adult patients with CD 5 mg/kg is given as an IV induction regimen at 0, 2, and 6 weeks followed by a maintenance regimen of 5 mg/kg IV every 8 weeks thereafter; treatment with 10 mg/kg IV may be considered for patients who respond and then lose their response. Adalimumab dose regimen for adult patients with CD is 160 mg initially on Day 1 (given in one day or split over two consecutive days), followed by 80 mg two weeks later (Day 15). Two weeks later (Day 29) they begin a maintenance dose of 40 mg every other week. During the prospective follow-up, according to usual clinical practice, the gastroenterologist responsible will modify or maintain a maintenance pattern according to the response to the drug by means of clinical (HBI) and analytical (fecal C-reactive protein and calprotectin) variables, which will be recorded in each patient’s data collection notebook.
To identify any adverse effects associated with the administration of the anti-TNFα drug, a combination of physical examination, recording of vital signs (BP, pulse, temperature), and questionnaires are evaluated after administration (30 min later) and at follow-up visits at weeks 12 and 24. These questionnaires inquire about possible adverse effects related to adverse reactions to infliximab and adalimumab drugs (nausea, headache, dizziness, fever, hives, reactions at the infusion site, nervous system disorders, cardiac arrhythmia or myocardial ischemia, hypertensive or hypotensive events, skin reactions, gastrointestinal disorders, infections, hypersensitivity or anaphylaxis). In addition, patients are asked open-ended questions about their general well-being to request notification of any other adverse effects not listed in the data sheet. In order to assess the causality of adverse effects related to infliximab and adalimumab, the ADR (Adverse Drug Reaction) Naranjo probability scale is applied [37].
Laboratory parameters, including white blood cell count, liver transaminase levels, phosphate levels, and kidney function are studied prior to anti-TNF treatment and at the 12-week and 24-week follow-up (post-anti-TNF).

2.14. Statistical Analysis

As this is an observational non-interventional study, there may be confounding factors between treatment allocation and outcomes in analyses of comparative response rates and samples may be included of patients that do not represent real-world clinical practices. Accordingly, all analyses in this study will be considered to be of an exploratory nature. Statistical analysis will be performed with SPSS v.22 software version 9.2 or later.
Population characteristics (including demographic characteristics, medical conditions, disease duration, types of treatment used at the start of the study and other variables collected in the data collection logbook) and all primary and secondary endpoints will be summarized by indicating the mean, standard deviation, minimum value, maximum value, median, 25th and 75th percentile and 95% confidence interval (CI) of the mean for continuous variables, and the absolute and relative frequencies, with a 95% CI of the proportion for categorical data. In the bivariate analysis, the chi-square test will be used to determine differences in proportions and Student’s t-test for paired data before and after the administration of anti-TNF treatment.
The statistical analyses conducted throughout this study on bioinformatics data management to determine significant differences between established groups, their corresponding graphs and the study of ROC curves (AUC values) will be performed using GraphPad Prism software version 8.2.0. STAMP (Statistical Analysis of Metagenomic Profiles), another bioinformatics tool based on Python, will also be used. STAMP was specifically designed for statistical processing and creation of plots from large amounts of biological data [38].

3. Discussion

The gut microbiome has a highly complex structure, involving thousands of microorganisms belonging to very different taxonomic classifications with millions of relationships between them, making its study a great challenge. In IBD, no specific pathogen has been definitively identified, but the gut microbiota as a whole has been shown to be pathogenic, contributing to the development of a deregulated inflammatory response in susceptible hosts. Several authors defend the suggestion that there is a reduction in biodiversity and abundance of Bacteroidetes and Firmicutes, as well as an increase in Proteobacteria, characteristic of patients compared to healthy individuals [6,7].
Treatment for CD is selected according to the severity of the disease and the response to previous therapies. Among the drugs used are biological treatments based on TNFα monoclonal antibodies such as infliximab and adalimumab, developed for the induction and maintenance of remission, allowing the control of symptoms and an improvement in the quality of life of responders, as well as changes in the natural course of the disease [38,39,40]. Nonetheless, although approximately one third of patients do not respond to these inhibitory therapies, we currently do not have a non-invasive biomarker that serves as a tool to predict this response, and invasive methods such as colonoscopy continue to be the gold standard for assessing therapeutic response.
In this prospective observational study, a new tool, massive sequencing of bacterial DNA, will be applied to the study of a clinical problem that affects an important number of patients. This will enable both the identification of early biomarkers of response in patients with CD after anti-TNF treatment as well as the prediction of therapeutic response from the start to thus improve the clinical management of these patients, reduce morbidity rates and increase efficiency. Since this is a longitudinal study, the patients will be analyzed before and after exposure to anti-TNF treatment, and the data will be paired, thereby diminishing the effect of the high variability of gut microbiota.
We know that the composition of the microbiota varies due to multiple external factors, particularly diet, and that a dietary intervention of just three days can cause a change in enterotype [41,42]. Nevertheless, after ten days, the enterotypes stabilized in one study suggesting a tendency to return to the original state [41]. Even so, we added to the protocol a 3-day dietary record prior to stool collection for microbiota analysis and will repeat this dietary record prior to the second assessment after six months of anti-TNF treatment to minimize significant diet-induced changes. An additional measure to objectively determine whether significant changes in the eating behavior of patients occur during the six months of the study will be undertaken through completion of a validated survey on adherence to the Mediterranean diet, both at inclusion and completion of the study. Given that long-term disturbances have a more profound effect, with a one-year diet modification having a strong impact on the Bacteroidetes/Firmicutes ratio [41,43], this may lead to changes in enterotype.
The proposed design takes diet into consideration and is therefore novel with respect to similar studies published to date, which did not evaluate this factor known to modify the composition of the microbiota. This study may provide additional evidence regarding potential non-invasive tools such as biomarkers of the response before and after anti-TNF therapy in CD as a starting point for future clinical trials. These trials could determine the most effective treatment among not only these therapies but all therapies used in the management of CD based on patient microbiota and provide more appropriate, inexpensive and non-invasive tools for predicting clinical response to treatment.

4. Conclusions

Currently, the role of bacterial gut microbiota is described as a key factor in the development of Crohn’s disease, we do know that, for the disease to develop, dysbiosis or a definitive change in the intestinal microbiota must occur. This study may provide additional evidence regarding potential non-invasive tools such as microbiota-based biomarkers of the response before and after anti-TNF therapy in Crohn’s disease as a starting point for future clinical trials. These trials could determine the most effective treatment among not only these therapies but all therapies used in the management of Crohn’s disease based on patient microbiota and provide more appropriate, inexpensive and non-invasive tools for predicting clinical response to treatment, reduce morbidity rates and improve efficiency.

Author Contributions

L.S.-A. participated in the design of the study and drafted the manuscript, revising for important intellectual content, and fully approved the final version for submission. J.F.M.-B. participated in the design of the study, contributed to the statistical analysis plan and was also involved in the drafting of the manuscript. S.M.-V. participated in design of the study related to gut microbiota analysis and contributed to the statistical analysis plan. E.C.-C. participated in the design of the study, contributed to the statistical analysis plan and was also involved in the drafting of the manuscript, revising for important intellectual content, and fully approved the final version for submission. J.R.-M. participated in the design of the study and its coordination. X.C.-R. conceived of the study and was involved in the overall design and helped to draft the manuscript revising for important intellectual content, and fully approved the final version for submission. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Acknowledgments

The authors thank Maria Repice and Ian Johnstone for their help with the English version of the text.

Conflicts of Interest

The authors declare that there are no conflicts of interest to disclose.

Abbreviations

CDCrohn’s disease
CIconfidence interval
CRPC-reactive protein
ETenterotype
FCfecal calprotectin
HBIHarvey-Bradshaw Index
IBDinflammatory bowel disease
RNAribonucleic acid
SCFAshort chain fatty acids
STAMPStatistical Analysis of Metagenomic Profiles
TNFTumor necrosis factor

References

  1. Amon, P.; Sanderson, I. What is the microbiome? Arch. Dis. Child. Educ. Pract. Ed. 2017, 102, 258–261. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  2. Young, V.B. The role of the microbiome in human health and disease: An introduction for clinicians. BMJ 2017, 356, j831. [Google Scholar] [CrossRef] [PubMed]
  3. Qin, J.; Li, R.; Raes, J.; Arumugam, M.; Burgdorf, K.S.; Manichanh, C.; Nielsen, T.; Pons, N.; Levenez, F.; Yamada, T.; et al. A human gut microbial gene catalogue established by metagenomic sequencing. Nature 2010, 464, 59–65. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Methé, B.A.; Nelson, K.E.; Pop, M.; Creasy, H.H.; Giglio, M.G.; Huttenhower, C.; Gevers, D.; Petrosino, J.F.; Abubucker, S.; Badger, J.H.; et al. A framework for human microbiome research. Nature 2012, 486, 215–221. [Google Scholar] [CrossRef] [Green Version]
  5. Arumugam, M.; Raes, J.; Pelletier, E.; Le Paslier, D.; Yamada, T.; Mende, D.R.; Fernandes, G.R.; Tap, J.; Bruls, T.; Batto, J.M.; et al. Enterotypes of the human gut microbiome. Nature 2011, 473, 174–180. [Google Scholar] [CrossRef]
  6. Zuo, T.; Ng, S.C. The Gut Microbiota in the Pathogenesis and Therapeutics of Inflammatory Bowel Disease. Front. Microbiol. 2018, 9, 2247. [Google Scholar] [CrossRef]
  7. Collins, S.M. A role for the gut microbiota in IBS. Nat. Rev. Gastroenterol. Hepatol. 2014, 11, 497–505. [Google Scholar] [CrossRef]
  8. Klindworth, A.; Pruesse, E.; Schweer, T.; Peplies, J.; Quast, C.; Horn, M.; Glöckner, F.O. Evaluation of general 16S ribosomal RNA gene PCR primers for classical and next-generation sequencing-based diversity studies. Nucleic Acids Res. 2013, 41, 1–11. [Google Scholar] [CrossRef]
  9. Kaur, N.; Chen, C.C.; Luther, J.; Kao, J.Y. Intestinal dysbiosis in inflammatory bowel disease. Gut Microbes 2011, 2, 211–216. [Google Scholar] [CrossRef] [Green Version]
  10. Erickson, A.R.; Cantarel, B.L.; Lamendella, R.; Darzi, Y. Integrated metagenomics/metaproteomics reveals human host-microbiota signatures of Crohn’s disease. PLoS ONE 2012, 7, e49138. [Google Scholar] [CrossRef] [Green Version]
  11. Manichanch, C.; Borruel, N.; Casellas, F.; Guarner, F. The gut microbiota in IBD. Nat. Rev. Gastroenterol. Hepatol. 2012, 9, 599–608. [Google Scholar] [CrossRef] [PubMed]
  12. Da Silva, A.C.; Gomes, F.; Sassaki, Y.; Rodrigues, J. Escherichia coli from Crohn’s disease patient displays virulence features of enteroinvasive (EIEC), enterohemorragic (EHEC), and enteroaggregative (EAEC) pathotypes. Gut Pathog. 2015, 7, 2. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Borruel, N.; Carol, M.; Casellas, F.; Antolín, M.; De Lara, F.; Espín, E.; Naval, J.; Guarner, F.; Malagelada, J.R. Increased mucosal tumour necrosis factor alpha production in Crohn’s disease can be downregulated ex vivo by probiotic bacteria. Gut 2002, 51, 659–664. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Rajca, S.; Grondin, V.; Louis, E.; Vernier-Massouille, G.; Grimaud, J.-C.; Bouhnik, Y.; Laharie, D.; Dupas, J.-L.; Pillant, H.; Picon, L.; et al. Alterations in the intestinal microbiome (dysbiosis) as a predictor of relapse after infliximab withdrawal in Crohn’s disease. Inflamm. Bowel Dis. 2014, 20, 978–986. [Google Scholar] [CrossRef] [PubMed]
  15. Peyrin-Biroulet, L.; Reinisch, W.; Colombel, J.F.; Mantzaris, G.J.; Kornbluth, A.; Diamond, R.; Rutgeerts, P.; Tang, L.K.; Cornillie, F.J.; Sandborn, W.J. Clinical disease activity, C-reactive protein normalisation and mucosal healing in Crohn’s disease in the SONIC trial. Gut 2014, 63, 88–95. [Google Scholar] [CrossRef] [PubMed]
  16. Harvey, R.; Bradshaw, J. A simple index of Crohn’s disease activity. Lancet 1980, 1, 514. [Google Scholar] [CrossRef]
  17. Rogler, G.; Vavricka, S.; Schoepfer, A.; Lakatos, P.L. Mucosal healing and deep remission: What does it mean? World J. Gastroenterol. 2013, 19, 7552–7560. [Google Scholar] [CrossRef]
  18. Roseth, A.G.; Aadland, E.; Grzyb, K. Normalization of faecal calprotectin: A predictor of mucosal healing in patients with inflammatory bowel disease. Scand. J. Gastroenterol. 2004, 39, 1017–1020. [Google Scholar] [CrossRef]
  19. Sipponen, T.; Savilahti, E.; Kolho, K.L.; Nuutinen, H.; Turunen, U.; Färkkilä, M. Crohn’s disease activity assessed by fecal calprotectin and lactoferrin: Correlation with Crohn’s disease activity index and endoscopic findings. Inflamm. Bowel Dis. 2008, 14, 40–46. [Google Scholar] [CrossRef]
  20. Lopez-Siles, M.; Martinez-Medina, M.; Busquets, D.; Sabat-Mir, M.; Duncan, S.H.; Flint, H.J.; Aldeguer, X.; Garcia-Gil, L.J. Mucosa-associated Faecalibacterium prausnitzii and Escherichia coli co-abundance can distinguish Irritable Bowel Syndrome and Inflammatory Bowel Disease phenotypes. Appl. Environ. Microbiol. 2015, 81, 7582–7592. [Google Scholar] [CrossRef] [Green Version]
  21. Zhou, Y.; Xu, Z.Z.; He, Y.; Yang, Y.; Liu, L.; Lin, Q.; Nie, Y.; Li, M.; Zhi, F.; Liu, S.; et al. Gut Microbiota Offers Universal Biomarkers across Ethnicity in Inflammatory Bowel Disease Diagnosis and Infliximab Response Prediction. mSystems 2018, 3, 1–14. [Google Scholar] [CrossRef] [Green Version]
  22. Magnusson, M.K.; Strid, H.; Sapnara, M.; Lasson, A.; Bajor, A.; Ung, K.-A.; Öhman, L. Anti-TNF therapy response in patients with ulcerative colitis is associated with colonic antimicrobial peptide expression and microbiota composition. J. Crohns Colitis 2016, 10, 943–952. [Google Scholar] [CrossRef]
  23. Wang, Y.; Gao, X.; Ghozlane, A.; Hu, H.; Li, X.; Xiao, Y.; Li, D.; Yu, G.; Zhang, T. Characteristics of faecal microbiota in paediatric Crohn’s disease and their dynamic changes during infliximab therapy. J. Crohns Colitis 2017, 1, 1–10. [Google Scholar] [CrossRef] [PubMed]
  24. Puig, L.; Ruiz de Morales, J.G.; Dauden, E.; Andreu, J.L.; Cervera, R.; Adán, A.; Marsal, S.; Escobar, C.; Hinojosa, J.; Palau, J.; et al. La prevalencia de diez enfermedades inflamatorias inmunomediadas (IMID) en España. Rev. Esp. Salud Pública 2019, 93, 25. [Google Scholar]
  25. Martínez-González, M.A.; Salas-Salvadó, J.; Estruch, R.; Corella, D.; Fitó, M.; Ros, E. Predimed Investigators. Benefits of the Mediterranean Diet: Insights from the PREDIMED Study. Prog. Cardiovasc. Dis. 2015, 58, 50–60. [Google Scholar] [CrossRef] [Green Version]
  26. Lewis, J.D.; Chen, E.Z.; Baldassano, R.N.; Otley, A.R.; Griffiths, A.M.; Lee, D.; Bittinger, K.; Bailey, A.; Friedman, E.S.; Hoffmann, C.; et al. Inflammation, Antibiotics, and Diet as Environmental Stressors of the Gut Microbiome in Pediatric Crohn’s Disease. Cell Host Microbe 2015, 18, 489–500. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  27. Scaldaferri, F.; Gerardi, V.; Pecere, S.; Petito, V.; Lopetuso, L.R.; Zambrano, D.; Schiavoni, E.; D’Ambrosio, D.; di Agostini, A.; Laterza, L.; et al. Anti-TNF-α induction regimen modulates gut microbiota molecular composition while inducing clinical response in Crohn’s Disease patients: Toward a personalized Medicine. Gastroenterology 2015, 148, S-852. [Google Scholar] [CrossRef]
  28. Kolho, K.L.; Korpela, K.; Jaakkola, T.; Pichai, M.V.; Zoetendal, E.G.; Salonen, A.; De Vos, W.M. Fecal microbiota in pediatric inflammatory bowel disease and its relation to inflammation. Am. J. Gastroenterol. 2015, 110, 921–930. [Google Scholar] [CrossRef]
  29. Busquets, D.; Mas-de-Xaxars, T.; Lpez-Siles, M.; Martínez-Medina, M.; Bahí, A.; Sàbat, M.; Louvriex, R.; Miquel-Cusachs, J.O.; Garcia-Gil, J.L.; Aldeguer, X. Anti-tumour necrosis factor treatment with adalimumab induces changes in the microbiota of Crohn’s Disease. J. Crohns Colitis 2015, 9, 899–906. [Google Scholar] [CrossRef] [Green Version]
  30. Ribaldone, D.G.; Caviglia, G.P.; Abdulle, A.; Pellicano, R.; Ditto, M.C.; Morino, M.; Fusaro, E.; Saracco, G.M.; Bugianesi, E.; Astegiano, M. Adalimumab Therapy Improves Intestinal Dysbiosis in Crohn’s Disease. J. Clin. Med. 2019, 8, 1646. [Google Scholar] [CrossRef] [Green Version]
  31. Bolyen, E.; Rideout, J.R.; Dillon, M.R.; Bokulich, N.A.; Abnet, C.C.; Al-Ghalith, G.A.; Alexander, H.; Alm, E.J.; Arumugam, M.; Asnicar, F.; et al. Reproducible, interactive, scalable and extensible microbiome data science using QIIME 2. Nat. Biotechnol. 2019, 37, 852–857. [Google Scholar] [CrossRef] [PubMed]
  32. Segata, N.; Izard, J.; Waldron, L.; Gevers, D.; Miropolsky, L.; Garrett, W.S.; Huttenhower, C. Metagenomic biomarker discovery and explanation. Genome Biol. 2011, 12, R60. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  33. Dobnik, D.; Štebih, D.; Blejec, A.; Morisset, D.; Žel, J. Multiplex quantification of four DNA targets in one reaction with Bio-Rad droplet digital PCR system for GMO detection. Sci. Rep. 2016, 6, 1–9. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  34. Postel, M.; Roosen, A.; Laurent-Puig, P.; Taly, V.; Wang-Renault, S.F. Droplet-based digital PCR and next generation sequencing for monitoring circulating tumor DNA: A cancer diagnostic perspective. Expert Rev. Mol. Diagn. 2018, 18, 7–17. [Google Scholar] [CrossRef] [PubMed]
  35. Huijsdens, X.W.; Linskens, R.K.; Mak, M.; Meuwissen, S.G.; Vandenbroucke-Grauls, C.M.; Savelkoul, P.H. Quantification of Bacteria Adherent to Gastrointestinal Mucosa by Real-Time PCR. J. Clin. Microbiol. 2002, 40, 4423–4427. [Google Scholar] [CrossRef] [Green Version]
  36. Furet, J.P.; Firmesse, O.; Gourmelon, M.; Bridonneau, C.; Tap, J.; Mondot, S.; Dore, J.; Corthier, G. Comparative assessment of human and farm animal faecal microbiota using real-time quantitative PCR. FEMS Microbiol. Ecol. 2009, 68, 351–362. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  37. Naranjo, C.A.; Busto, U.; Sellers, E.M.; Sandor, P.; Ruiz, I.; Roberts, E.A.; Janecek, E.; Domecq, C.; Greenblatt, D.J. A method for estimating the probability of adverse drug reactions. Clin. Pharmacol. Ther. 1981, 30, 239–245. [Google Scholar] [CrossRef]
  38. Parks, D.H.; Tyson, G.W.; Hugenholtz, P.; Beiko, R.G. STAMP: Statistical analysis of taxonomic and functional profiles. Bioinformatics 2014, 30, 3123–3124. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  39. Monif, G.R. Understanding Therapeutic Concepts in Crohn’s Disease. Clin. Med. Insights Gastroenterol. 2018, 11, 1–3. [Google Scholar] [CrossRef] [Green Version]
  40. Na, S.Y.; Moon, W. Perspectives on Current and Novel Treatments for Inflammatory Bowel Disease. Gut Liver 2019, 13, 604–616. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  41. Wu, G.D.; Chen, J.; Hoffmann, C.; Bittinger, K.; Chen, Y.-Y.; Keilbaugh, S.A.; Bewtra, M.; Knights, D.; Walters, W.A.; Knight, R.; et al. Linking long-term dietary patterns with gut microbial enterotypes. Science 2011, 334, 105–108. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  42. Kovatcheva-Datchary, P.; Nilsson, A.; Akrami, R.; Lee, Y.S.; de Vadder, F.; Arora, T.; Hallen, A.; Martens, E.; Björck, I.; Bäckhed, F. Dietary Fiber-Induced Improvement in Glucose Metabolism Is Associated with Increased Abundance of Prevotella. Cell Metab. 2015, 22, 971–982. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  43. Ley, R.E.; Turnbaugh, P.J.; Klein, S.; Gordon, J.I. Human gut microbes associated with obesity. Nature 2006, 444, 1022–1023. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Overall study design.
Figure 1. Overall study design.
Ijerph 17 05120 g001
Figure 2. Sequencing and bioinformatics scheme.
Figure 2. Sequencing and bioinformatics scheme.
Ijerph 17 05120 g002
Table 1. Harvey-Bradshaw Index (HBI).
Table 1. Harvey-Bradshaw Index (HBI).
General Well-Being (Previous Day)0 (Very Well)1 (Slightly Below Par)2 (Poor)3 (Very Poor)4 (Terrible)
Abdominal pain0 (none)1 (mild)2 (moderate)3 (severe)
Number of liquid or soft stools per day (previous day)
Complications
(score 1 per item)
Arthritis/
arthralgia
Iritis/
uveitis
Erythema
nodosum/
aphthous ulcers/Pyoderma gangrenosum
Anal fissure new fistula/
abscess
Abdominal mass0 (none) 1 (dubious) 2 (definite) 3 (definite + tender)
Remission ≤ 4 points; mild disease 5–6 points; moderate disease 6–12 points; severe disease > 12 points.
Table 2. Summary of variables to be collected during the study.
Table 2. Summary of variables to be collected during the study.
Study Variables
Primary:
Normalization of gut microbiota (after anti-TNF treatment) (yes/no)
Secondary variables:
A. Gut microbiota
-
Initial dysbiosis (yes/no)
-
Final dysbiosis (yes/no)
-
Initial biodiversity level (low/medium/high)
-
Final biodiversity level (low/medium/high)
-
Increase in biodiversity (yes/no)
-
Initial Faecalibacterium prausnitzii/Escherichia coli
-
Final Faecalibacterium prausnitzii/Escherichia coli
-
Initial Faecalibacterium prausnitzii/Clostridium coccoides group
-
Final Faecalibacterium prausnitzii/Clostridium coccoides group
B. Clinical-biological parameters:
-
Initial Harvey-Bradshaw Index
-
Final Harvey-Bradshaw Index
-
Initial C-reactive protein (mg/l)
-
Final C-reactive protein (mg/l)
-
Initial fecal calprotectin (µg/g)
-
Final fecal calprotectin (µg/g)
C. Epidemiological data
-
Age (years)
-
Sex (male/female)
-
Time from diagnosis to anti-TNF treatment (months)
-
Body Mass Index (kg/m2)
-
Smoking habit (yes/no)
-
72-h dietary record
-
Adherence to Mediterranean diet
Table 3. Primers and probes used in digital droplet PCR [20,35,36].
Table 3. Primers and probes used in digital droplet PCR [20,35,36].
TargetPrimers and ProbeSequences 5′-3′Reference
F. prausnitziiFpra_428_FTGTAAACTCCTGTTGTTGAGGAAGATAA[20]
Fpra_583_RGCGCTCCCTTTACACCCA
Fpra_493_PRFAM/CAAGGAAGTGACGGCTAACTACGTGCCAG/IABkFQ
E. coliEcoli_395_FCATGCCGCGTGTATGAAGAA[35]
Ecoli_490_RCGGGTAACGTCAATGAGCAAA
Ecoli_437_PRFAM/TATTAACTTTACTCCCTTCCTCCCCGCTGAA/IABkFQ
C. coccoidesF_Ccoc_07GACGCCGCGTGAAGGA[36]
R_Ccoc_14AGCCCCAGCCTTTCACAT
P_Erec_482VIC/CGGTACCTGACTAAGAAG/IABkFQ
BacteriaF_Bact_1369CGGTGAATACGTTCCCGG[36]
R_Prok_1492TACGGCTACCTTGTTACGACTT
P_TM_1389FFAM/CTTGTACACACCGCCCGTC/IABkFQ

Share and Cite

MDPI and ACS Style

Sanchis-Artero, L.; Martínez-Blanch, J.F.; Manresa-Vera, S.; Cortés-Castell, E.; Rodriguez-Morales, J.; Cortés-Rizo, X. Evaluation of Changes in Gut Microbiota in Patients with Crohn’s Disease after Anti-Tnfα Treatment: Prospective Multicenter Observational Study. Int. J. Environ. Res. Public Health 2020, 17, 5120. https://doi.org/10.3390/ijerph17145120

AMA Style

Sanchis-Artero L, Martínez-Blanch JF, Manresa-Vera S, Cortés-Castell E, Rodriguez-Morales J, Cortés-Rizo X. Evaluation of Changes in Gut Microbiota in Patients with Crohn’s Disease after Anti-Tnfα Treatment: Prospective Multicenter Observational Study. International Journal of Environmental Research and Public Health. 2020; 17(14):5120. https://doi.org/10.3390/ijerph17145120

Chicago/Turabian Style

Sanchis-Artero, Laura, Juan Francisco Martínez-Blanch, Sergio Manresa-Vera, Ernesto Cortés-Castell, Josefa Rodriguez-Morales, and Xavier Cortés-Rizo. 2020. "Evaluation of Changes in Gut Microbiota in Patients with Crohn’s Disease after Anti-Tnfα Treatment: Prospective Multicenter Observational Study" International Journal of Environmental Research and Public Health 17, no. 14: 5120. https://doi.org/10.3390/ijerph17145120

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop