Next Article in Journal
The Gut–Brain–Microbiome Axis in Bumble Bees
Previous Article in Journal
The Inability of Spotted Lanternfly (Lycorma delicatula) to Vector a Plant Pathogen between its Preferred Host, Ailanthus altissima, in a Laboratory Setting
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Identification of Immune Regulatory Genes in Apis mellifera through Caffeine Treatment

1
Department of Entomology, National Taiwan University, Taipei 106, Taiwan
2
Department of Plant Physiology, Swammerdam Institute for Life Sciences, University of Amsterdam, 1098 XH Amsterdam, The Netherlands
3
Biology Centre of the Czech Academy of Science, Institute of Entomology, 37005 Ceske Budejovice, Czech Republic
4
Faculty of Science, University of South Bohemia, 37005 Ceske Budejovice, Czech Republic
*
Author to whom correspondence should be addressed.
Insects 2020, 11(8), 516; https://doi.org/10.3390/insects11080516
Submission received: 13 July 2020 / Revised: 5 August 2020 / Accepted: 5 August 2020 / Published: 10 August 2020

Abstract

:

Simple Summary

The preference of honeybees to consume nectar with caffeine has been recorded. To investigate the effects of caffeine to this important pollinator, we first focus on the influences on immunity which is seldom explored in insects regarding caffeine. In our results, we discovered the suppressive effects on viral pathogens and the boosting effects on immunity after caffeine treatment. At least six different latent-infecting viruses in Taiwan were suppressed by caffeine. Nevertheless, the enhancement on immunity may not be effective if the bees have not been exposed to the environment or potential natural secondary metabolites like caffeine. These findings provide a basic but valuable insight into how caffeine can aid honeybees in fighting against viral invasion.

Abstract

Plants and pollinators are mutually beneficial: plants provide nectar as a food source and in return their pollen is disseminated by pollinators such as honeybees. Some plants secrete chemicals to deter herbivores as a protective measure, among which is caffeine, a naturally occurring, bitter tasting, and pharmacologically active secondary compound. It can be found in low concentrations in the nectars of some plants and as such, when pollinators consume nectar, they also take in small amounts of caffeine. Whilst caffeine has been indicated as an antioxidant in both mammals and insects, the effect on insect immunity is unclear. In the present study, honeybees were treated with caffeine and the expression profiles of genes involved in immune responses were measured to evaluate the influence of caffeine on immunity. In addition, honeybees were infected with deformed wing virus (DWV) to study how caffeine affects their response against pathogens. Our results showed that caffeine can increase the expression of genes involved in immunity and reduce virus copy numbers, indicating that it has the potential to help honeybees fight against viral infection. The present study provides a valuable insight into the mechanism by which honeybees react to biotic stress and how caffeine can serve as a positive contributor, thus having a potential application in beekeeping.

1. Introduction

The western honeybee, Apis mellifera, is one of the most important pollinators in the world [1,2]. They contribute to the production of many different fruits, vegetables, crops, and aromatic seeds. However, honeybees are weak at resisting their stressors, which can cause direct/indirect injuries and affect their development, thus influencing the structure and functionality of plant-pollination networks. In recent years, the incidence rate of colony collapse disorder (CCD) [3,4], characterized by hives missing their worker bees, has increased significantly [5]; this has contributed to significant losses in the produce economy. Several factors have been suggested as contributing to this phenomenon, including poor nutrition, pesticides, pathogens, parasites, and interactions between these two factors resulting in a stressful environment for the colony [6,7,8].
Colonies suffering from CCD often exhibit signs of viral infections caused by Israel acute paralysis virus (IAPV), acute bee paralysis virus (ABPV), and deformed wing virus (DWV) [9,10,11]. DWV has been isolated in over 90% of colonies with CCD, making it the most prevalent bee virus. This RNA virus infection, spread by Varroa destructor mites, can cause the abnormal development of wings and memory loss since the highest titer of the virus is found in the head after infection [12,13,14]. Furthermore, DWV has a negative impact on the immune system of the honeybee [15,16]: the virus tends to negatively regulate the transcription of some genes such as dorsal 1A, a transcription factor in the family NF-κB [17,18], and the Toll pathway, which are known to target viruses [19,20,21,22]. As a result of this transcription dysregulation, transcription of antimicrobial peptides, clotting and melanization are reduced in infected honeybees [23].
Plants and pollinators benefit each other since plants provide nectar as a source of food and, in return, their pollen grains are disseminated by the visiting insect. In nectars of plants in the genera Citrus and Coffea, low concentrations of caffeine (1,3,7-trimethylxanthine) are found, which acts as an antiherbivory compound [24,25]. Previous studies have indicated that the ingestion of caffeine as nectar ingredient could increase the foraging efficiency of honeybees and thus promote pollination [26,27,28]. Furthermore, it is believed that caffeine could enhance both the learning behavior and long-term memory of forager bees [29]. In other insects, it has been found that adequate caffeine consumption influences locomotion and enhances memory; this includes studies involving hornets [Vespa orientalis [30]], honeybees, the green scale insect [Coccus viridis [31]], and flour beetles [Tribolium castaneum and Tribolium confusum [32,33]]. In addition, one study of European honeybees indicated that caffeine can enhance the tolerance to infection with the fungus parasite Nosema ceranae as well as extend their lifespan [34]. However, the beneficial effects of caffeine on the immune response against other pathogens, including viruses, are unknown.
Based on the above information, we hypothesized that caffeine could help the honeybee resist viral infection. We compared bees supplied with/without caffeine and assessed their immune responses against DWV infection. Furthermore, we analyzed the expression of a set of immune-related genes to determine how they are influenced by caffeine. Our results showed that the expression of immune-related genes was up-regulated after the infected honeybees were fed with caffeine, and that the replication of DWV was inhibited. These results provide evidence that caffeine is beneficial to the honeybee in fighting against pathogens in the environment. Furthermore, it provides us with a greater understanding of why bees prefer to harvest nectar from caffeine-synthesizing plants.

2. Materials and Methods

2.1. Bee Rearing

Western honeybees (Apis mellifera) were purchased in Hsinchu county, Taiwan, and kept at the National Taiwan University. About 50~100 bees for the experiment were collected directly from one hive as a mixed group. Then, the bees were caged in different Bugdorms (18 × 18 × 18 cm, MegaView Science, Taichung, Taiwan) at 37 °C. Prior to the experiments, bees were fed with 0.7 M sucrose water with/without caffeine (0.1 mM, Sigma-Aldrich, St. Louis, MO, USA) for one week [29,35]. One frame of the hive was taken back to the lab and incubated in 30 °C, waiting for newly emerged bees to come out. The newly emerged bees were first fed with 0.7 M sucrose water and pollen as protein source for one week to stabilize their physiological condition, then caffeine (0.1 mM) was added to the diet for another week. All food for the bees were changed every three days, and the dead bees were removed. After this feeding stage, all bees were collected for gene analysis or bioassays.

2.2. Deformed Wing Virus (DWV) Purification

About 80~100 bees were directly collected from their hive, and then put into cages, incubated in 30 °C. DWVs were mixed with sugar solution (0.7 M) as a regular treatment for one week. After treatment, bees were frozen in −80 °C, and then homogenized with 10 mL phosphate-buffered saline (PBS). The liquid was collected through a nylon filter to keep out residue, then we centrifuged it (16,000× g). Supernatant was moved to another tube and centrifuge again, then use Minisart® Syringe Filters (0.45 µm) to filtrate the supernatant.

2.3. DWV Infection and Caffeine Treatment

Four treatment groups were prepared: control (0.7 M sucrose water only), caffeine only (0.1 mM) [29], DWV only (106 virus copies), and both (caffeine/DWV). DWV was diluted with phosphate-buffered saline (PBS) to prepare a working virus solution with 5 × 106 copies/µL. The sucrose water was replaced with water only one day prior to experiment and the following day the bees were forced to take 4 µL of solution mixed with 2 µL of virus and 2 µL of sucrose. After 48 h, the bees were frozen at −80 °C for total RNA extraction and gene expression analysis. After one week stabilizing the physiological condition, the bees were separated into two groups, and caffeine was then added to the diet of one of the groups for another week as the caffeine-only treatment group. Then, bees were randomly chosen from each group and forced to take 4 µL of solution mixed with 2 µL of virus and 2 µL of sucrose as virus only and both treatment group. After this feeding stage, all bees were collected for gene analysis or bioassays.

2.4. Total RNA Extraction

After the treatments, total RNA was extracted using a TRIzol™ reagent kit (Thermo Fisher Scientific). Whole bodies of two bees were pooled together for homogenization. RNA quantification was determined using a NanoDrop 2000 (Thermo Fisher Scientific).

2.5. cDNA Synthesis

cDNA synthesis of each treatment group was performed using a High-Capacity cDNA Reverse Transcription Kit (Applied Biosystems). For each sample, 2 µg of total RNA was used. The reaction was incubated in a FlexCycler 2 PCR thermal cycler (Analytic Jena AG) for 10 min at 25 °C, 120 min at 37 °C, 5 min at 85 °C, and then stopped at 4 °C. The final products were used for quantitative real-time polymerase chain reaction (RT-PCR) analysis or stored at −20 °C for later analysis.

2.6. Real-Time Polymerase Chain Reaction (PCR) and Data Analysis

Real-time PCR was performed using a StepOnePlus™ Real-Time PCR System (Applied Biosystems) using SensiFAST™ SYBR Hi-ROX Kit (BIOLINE, London, UK). The result (fold change) was calculated following the relative quantification theory [36]. For quantitative PCR, honeybee-specific primers for including immune, viral and carbohydrate metabolism genes (Table 1) were used as described in previous studies [35,37,38]. All samples were amplified simultaneously, and three independent experiments were performed. PCR-array images were analyzed with the software R (Version XX). Fold changes were calculated using the relative quantification method (2−△△Ct) [39]. Each group of tested genes was normalized to a reference gene (18s rRNA), and fold changes in the control group were used as a calibrator.

2.7. DWV Titer Calculation

A plasmid containing a DNA fragment of DWV was transformed to Escherichia coli (DH5 α ) and then grown in LB broth (ARROWTEC) containing ampicillin as a selection marker at 37 °C for 16 h. Plasmid extraction was conducted using a Presto™ Mini Plasmid Kit (Geneaid) and the concentration was determined using a NanoDrop 2000 (Thermo Fisher Scientific). A DNA Copy Number and Dilution Calculator (Thermo Fisher scientific website) were used to determine the amount of DNA sample equivalent to 1010 plasmid copies. Serial dilution was performed to prepare 1010 to 101 plasmid copies. Real-time PCR was conducted using serially diluted plasmid samples. A standard curve was plotted using data obtained from the real-time PCR results on serially-diluted plasmid samples; regression analysis was used to calculate the virus copy numbers in the virus-treated bees.

2.8. Other Latent Infecting Viruses in Taiwan

Based on a previous study in Taiwan [35], we choose the also latent-infecting virus species to see if caffeine can aid the bees to resist them. The virus we use including Vorroa destructor virus-1 (VDV), Kashmir bee virus (KBV), Kakugo virus (KV), Israeli acute bee paralysis virus (IAPV), Sacbrood Virus (SBV), black queen cell virus (BQCV), and Chronic bee paralysis virus (CBPV) to see whether caffeine can also suppress their replication or not. After treated with caffeine, honeybees were freeze-killed and the mRNA extraction, complimentary DNA (cDNA) reverse transcription, and RT-PCR was performed to compare the amount of viral infection of honeybees treated with/without caffeine.

2.9. Statistical Analysis

Gene expression level results were analyzed using one-way analysis of variance (ANOVA) followed by Tukey’s post hoc test for significance using Statistica software (Version 8). The virus number was analyzed using one-way Student’s t-test. A p < 0.05 indicated a statistically significant result.

3. Results

3.1. Effects of Caffeine on Immunity Gene Signaling Factors and Anti-Microbial Peptides

The expression of immune response-related genes after DWV infection (Table 2), especially those involved in the Toll and Imd pathways, as well as anti-microbial peptide (AMP), were analyzed 2 days after infection. It was found that both caffeine alone (caffeine group) and DWV infection alone (DWV group) could stimulate gene expression compared with the control group, with DWV being a stronger stimulant than caffeine (Figure 1A–C). However, the highest stimulation was observed in DWV-infected bees receiving the caffeine diet prior to infection (caffeine/DWV group), which exhibited up to 5-fold stimulation in gene expression (Figure 1A–C and Table 2). Genes that exhibited more than a 1-fold increase in the DWV and caffeine/DWV groups include parseph and Myd88 in the Toll signaling pathway, PGRPLS, Tak1, Kenny, and Tab in the Imd pathway, and prophenoloxidase-activating enzyme (PPOact), Lysozyme-1 (Lys-1), and AMP defensin-2 (Figure 1D). Stimulation of these factors may provide a strong immune response against pathogen infection.

3.2. Caffeine Inhibits DWV Replication by Enhancing Carbohydrate Metabolism

To further investigate how caffeine stimulates the expression of immune response-related genes, the replication of DWV in infected bees with or without the caffeine diet was analyzed. It was found that the DWV genome copy number was significantly reduced in bees receiving the caffeine diet (Figure 2A), suggesting that caffeine induced the expression of immune response related genes, which in turn suppressed DWV genome replication. It is known that an optimal immune response requires a significant amount of energy [40]; therefore, we hypothesized that caffeine stimulates the expression of immune response-related genes by imposing positive effects on pathways relating to carbohydrate metabolism. To test this hypothesis, we analyzed the expression of genes involved in glycolysis and the tricarboxylic acid cycle (TCA) cycle in bees with or without the caffeine diet by real-time PCR (Table 3). Although the result had no significant difference, but still we can see the result showed the expression of genes involved in glycolysis (Figure 2B) and the TCA cycle (Figure 2C) has an up-regulated trend.

3.3. Caffeine also Inhibits the Infection of Other Prevalent Viruses in Taiwan

More than 8 persistent infectious viruses are prevalent among bees in Taiwan. These infected honeybees were given caffeine in their diet to evaluate its effect on viral activity in the host. Quantification of viral gene expression by real-time PCR showed a significant decrease in 6 of the viral genes in the caffeine-treated bees except for Black queen cell virus (BQCV) and Varroa destructor virus (VDV) (Figure 3). Thus, BQCV/VDV infection might not induce a noticeable immune response in honeybees.

3.4. Caffeine Does not Affect the Expression of Immunity Related Genes in 16-Days Old Honeybee

The results presented thus far were obtained from a mixture of bees from different tasks and ages. However, we also explored whether caffeine has a similar effect on 16-days old bees. For this experiment, newly-emerged bees received caffeine in their diet for 7 days and were subsequently fed with DWV. The expression of genes involved in the Toll and Imd pathways and AMPs were analyzed 2 days post-infection. It was found that the expression of genes involved in the Toll and Imd pathways were down-regulated in the DWV, caffeine, and caffeine/DWV groups (Figure 4A,B). The expression of many AMPs was also down-regulated in the DWV infection and caffeine only groups (Figure 4C). However, for several genes whose expressions were suppressed by DWV infection (including pgrps2 and cactus1 in the Toll signaling pathway, kenny in the Imd pathway, and the AMPs abaecin and defensin-2, and prophenoloxidase-activating enzyme (PPOact), and Lysozyme-1 (Lys-1)), the inhibitory effect could still be prevented significantly by a caffeine diet prior to infection (Figure 4D). These results indicate that for 16-day-old bees, caffeine (to which they have had no previous exposure) may have negative regulatory effects on immune response-related gene expression. This, in turn, affects the overall physiological condition of the insects in a similar manner to if the insects were exposed to an adverse environment.

4. Discussion

In this study, the influence of caffeine on the immune system of honeybees was evaluated. Gene expression levels were measured to demonstrate the interaction between caffeine and DWV. Although the effect of caffeine on mammals is well known [41], particularly its effects on anti-oxidation and neural activation, there are only a limited number of similar studies on insects. The results of the present study provide a basic yet valuable insight into the effect of caffeine on gene regulation in insects.
The effect of caffeine on the immune system of honeybees is almost unknown. In humans, a high concentration of caffeine can mitigate the damage caused by inflammation [42]. This is due to the inhibition on phosphodiesterase (PDE) activity, leading to an increased level of intracellular cAMP and the activation of the PKA pathway [43]. This may explain the changes in gene expression found in the present study, since the PKA pathway is also involved in regulating the immune system [44]. Analysis of the expression level of immune genes showed that they were significantly up-regulated after caffeine treatment in DWV-infected bees (Figure 1A–C), namely for parseph and myd88 in the Toll pathway, PGRPLC, tak1, kenny and tab in the Imd pathway, and lysozyme-1 (Lys-1), prophenoloxidase-activating enzyme (PPOact), and the AMP gene amPPO (Figure 1D). Both the Toll and Imd pathways have been previously demonstrated to be involved in fighting against viral invasion in insects [45,46]. Our results also showed that caffeine treatment prior to DWV infection could sufficiently inhibit DWV infection (Figure 2). This indicates that caffeine can boost the immune system by up-regulating the expression of genes involved in pathways known to influence immune responses (such as Toll and Imd) and protect honeybees from the external stress caused by viral infections. In the results, after DWV infection, the gene expression level of the immune system is up-regulated. Although it is indicated in previous study that DWV can suppress the host’s immune system, this is the consequence of long-term and latent infection of Vorroa destructor and DWV [16]. In our experiment, the virus was fed to the healthy bees, so this situation is different from the previous study. This acute infection induced the spike in the immune response of honeybees against the viral invasion [47].
Until now, the effect of caffeine on gene expression in 16-day-old bees has remained unexplored. Caffeine is a natural compound present in the nectar of certain plants and, therefore, honeybees can easily consume caffeine from the environment [25,29]. Bees that are 16 days old, however, were not exposed to caffeine (and other compounds) as the foragers because they were kept in lab. The effect of a stressor and caffeine treatment was, therefore, worth investigating for these honeybees with “clean” background. Interestingly, the results are contrary to those obtained in the mixed group: caffeine does not stimulate the immune system of 16-day-old bees as it does to the mixed group in the presence of DWV infection and the expression of most genes involved in the Toll and Imd pathways and AMPs are down-regulated (Figure 4A–C).
One previous study indicated that older honeybees, such as foragers, have a high basal gene expression level related to detoxification and immune pathways for dealing with more environmental stressors [48]. This may explain different outcomes under the same experimental conditions from mixed nursing bees/forager bees and 16-day-old bees. Nevertheless, the expression of PPOact, Lys-1 and Kenny were also significantly enhanced in DWV-infected bees, similar to in the caffeine/DWV group (Figure 4), suggesting that caffeine has a marginal boosting effect on the immune system of 16-day-old bees. The differential influences of caffeine on the immune system during pathogen infection in 16-day-old bees and older nursing bees/forager bees thus require further study.

5. Conclusions

In summary, caffeine has the potential to improve the resistance of honeybees to viral infection. For 16-day-old bees, caffeine has a similar effect but it is not as extensive and positive as within the mixed group. This study provides a basic knowledge of the influence of caffeine on the expression profiles of immune-related genes of honeybees. Even though the mechanism by which caffeine regulates the immune system in honeybees is not yet clear, future experiments on its effects on responses to other pathogens such as bacteria and fungi should be undertaken. This would help define a more definitive role of caffeine in promoting the immune responses of honeybees against pathogens.

Author Contributions

Guarantor of the integrity of the entire study, study concepts, and manuscript preparation: Y.-H.L. (Yun-Heng Lu), and Y.-L.W.; study design, data acquisition/analysis, literature research, and manuscript preparation: Y.-H.L. (Yun-Heng Lu), C.-P.W., C.-K.T., Y.-C.C., and Y.-L.W.; data acquisition/analysis, manuscript editing, and revision: Y.-H.L. (Yun-Heng Lu), C.-P.W., C.-K.T., and Y.-L.W. All authors reviewed the manuscript. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by the grant MOST107-2311-B-002-024-MY3 to Y.-L.W. from the Ministry of Science and Technology, Taiwan.

Acknowledgments

We thank Alexander Barton for kindly revising the manuscript.

Conflicts of Interest

The authors declare that they have no competing financial interests.

References

  1. Moritz, R.; Härtel, S.; Neumann, P. Global invasions of the western honeybee (Apis mellifera) and the consequences for biodiversity. Ecoscience 2005, 12, 289–301. [Google Scholar] [CrossRef]
  2. Kremen, C.; Williams, N.M.; Thorp, R.W. Crop pollination from native bees at risk from agricultural intensification. Proc. Natl. Acad. Sci. USA 2002, 99, 16812–16816. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  3. Francis, R.M.; Nielsen, S.L.; Kryger, P. Varroa-virus interaction in collapsing honey bee colonies. PLoS ONE 2013, 8, e57540. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  4. Vanengelsdorp, D.; Traynor, K.; Andree, M.; Lichtenberg, E.M.; Chen, Y.; Saegerman, C.; Cox-Foster, D.L. Colony Collapse Disorder (CCD) and bee age impact honey bee pathophysiology. PLoS ONE 2017, 12, e0179535. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  5. Nazzi, F.; Pennacchio, F. Disentangling multiple interactions in the hive ecosystem. Trends Parasitol. 2014, 30, 556–561. [Google Scholar] [CrossRef] [PubMed]
  6. Cornman, R.S.; Tarpy, D.R.; Chen, Y.; Jeffreys, L.; Lopez, D.; Pettis, J.S.; Vanengelsdorp, D.; Evans, J.D. Pathogen Webs in Collapsing Honey Bee Colonies. PLoS ONE 2012, 7, e43562. [Google Scholar] [CrossRef] [Green Version]
  7. Cox-Foster, D.L.; Conlan, S.; Holmes, E.C.; Palacios, G.F.; Evans, J.D.; Moran, N.A.; Quan, P.-L.; Briese, T.; Hornig, M.; Geiser, D.M.; et al. A Metagenomic Survey of Microbes in Honey Bee Colony Collapse Disorder. Science 2007, 318, 283–287. [Google Scholar] [CrossRef] [Green Version]
  8. Ellis, J.D.; Evans, J.D.; Pettis, J.S. Colony losses, managed colony population decline, and Colony Collapse Disorder in the United States. J. Apic. Res. 2010, 49, 134–136. [Google Scholar] [CrossRef]
  9. Dolezal, A.G.; Hendrix, S.D.; Scavo, N.A.; Carrillo-Tripp, J.; Harris, M.A.; Wheelock, M.J.; O’Neal, M.E.; Toth, A.L. Honey Bee Viruses in Wild Bees: Viral Prevalence, Loads, and Experimental Inoculation. PLoS ONE 2016, 11, e0166190. [Google Scholar] [CrossRef]
  10. Grozinger, C.M.; Flenniken, M.L. Bee Viruses: Ecology, Pathogenicity, and Impacts. Annu. Rev. Èntomol. 2019, 64, 205–226. [Google Scholar] [CrossRef]
  11. McMenamin, A.J.; Genersch, E. Honey bee colony losses and associated viruses. Current Opinion in Insect Science 2015, 8, 121–129. [Google Scholar] [CrossRef]
  12. Wilfert, L.; Long, G.; Leggett, H.C.; Schmid-Hempel, P.; Butlin, R.K.; Martin, S.J.M.; Boots, M. Deformed wing virus is a recent global epidemic in honeybees driven by Varroa mites. Science 2016, 351, 594–597. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  13. Nazzi, F.; Le Conte, Y. Ecology ofVarroa destructor, the Major Ectoparasite of the Western Honey Bee, Apis mellifera. Annu. Rev. Èntomol. 2016, 61, 417–432. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Berényi, O.; Bakonyi, T.; Derakhshifar, I.; Köglberger, H.; Topolska, G.; Ritter, W.; Pechhacker, H.; Nowotny, N. Phylogenetic Analysis of Deformed Wing Virus Genotypes from Diverse Geographic Origins Indicates Recent Global Distribution of the Virus. Appl. Environ. Microbiol. 2007, 73, 3605–3611. [Google Scholar] [CrossRef] [Green Version]
  15. Yang, X.; Cox-Foster, D.L. Impact of an ectoparasite on the immunity and pathology of an invertebrate: Evidence for host immunosuppression and viral amplification. Proc. Natl. Acad. Sci. USA 2005, 102, 7470–7475. [Google Scholar] [CrossRef] [Green Version]
  16. Navajas, M.; Migeon, A.; Alaux, C.; Martin-Magniette, M.; Robinson, G.E.; Evans, J.D.; Cros-Arteil, S.; Crauser, D.; Le Conte, Y. Differential gene expression of the honey bee Apis mellifera associated with Varroa destructor infection. BMC Genom. 2008, 9, 301. [Google Scholar] [CrossRef] [Green Version]
  17. Li, Q.; Verma, I.M. NF-κB regulation in the immune system. Nat. Rev. Immunol. 2002, 2, 725–734. [Google Scholar] [CrossRef]
  18. Gordon, M.D.; Dionne, M.S.; Schneider, D.S.; Nusse, R. WntD is a feedback inhibitor of Dorsal/NF-κB in Drosophila development and immunity. Nature 2005, 437, 746–749. [Google Scholar] [CrossRef]
  19. Xi, Z.; Ramirez, J.L.; Dimopoulos, G. The Aedes aegypti Toll Pathway Controls Dengue Virus Infection. PLoS Pathog. 2008, 4, e1000098. [Google Scholar] [CrossRef]
  20. Vanwalscappel, B.; Tada, T.; Landau, N.R. Toll-like receptor agonist R848 blocks Zika virus replication by inducing the antiviral protein viperin. Virology 2018, 522, 199–208. [Google Scholar] [CrossRef]
  21. Evans, J.D.; Aronstein, K.; Chen, Y.P.; Hetru, C.; Imler, J.; Jiang, H.; Kanost, M.; Thompson, G.J.; Zou, Z.; Hultmark, D. Immune pathways and defence mechanisms in honey bees Apis mellifera. Insect Mol. Biol. 2006, 15, 645–656. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  22. Brutscher, L.M.; Daughenbaugh, K.F.; Flenniken, M.L. Antiviral defense mechanisms in honey bees. Curr. Opin. Insect Sci. 2015, 10, 71–82. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  23. Nazzi, F.; Pennacchio, F. Honey Bee Antiviral Immune Barriers as Affected by Multiple Stress Factors: A Novel Paradigm to Interpret Colony Health Decline and Collapse. Viruses 2018, 10, 159. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  24. Kennedy, D.O.; Wightman, E.L. Herbal Extracts and Phytochemicals: Plant Secondary Metabolites and the Enhancement of Human Brain Function. Adv. Nutr. 2011, 2, 32–50. [Google Scholar] [CrossRef] [PubMed]
  25. Kretschmar, J.A.; Baumann, T.W. Caffeine in Citrus flowers. Phytochemistry 1999, 52, 19–23. [Google Scholar] [CrossRef]
  26. Couvillon, M.J.; Al Toufailia, H.; Butterfield, T.M.; Schrell, F.; Ratnieks, F.L.W.; Schurch, R. Caffeinated forage tricks honeybees into increasing foraging and recruitment behaviors. Curr. Biol. 2015, 25, 2815–2818. [Google Scholar] [CrossRef] [Green Version]
  27. Mustard, J.; Dews, L.; Brugato, A.; Dey, K.; Wright, G.A. Consumption of an acute dose of caffeine reduces acquisition but not memory in the honey bee. Behav. Brain Res. 2012, 232, 217–224. [Google Scholar] [CrossRef]
  28. Si, A.; Zhang, S.-W.; Maleszka, R. Effects of caffeine on olfactory and visual learning in the honey bee (Apis mellifera). Pharmacol. Biochem. Behav. 2005, 82, 664–672. [Google Scholar] [CrossRef]
  29. Wright, G.A.; Baker, D.D.; Palmer, M.J.; Stabler, D.; Mustard, J.A.; Power, E.F.; Borland, A.M.; Stevenson, P.C. Caffeine in floral nectar enhances a pollinator’s memory of reward. Science 2013, 339, 1202–1204. [Google Scholar] [CrossRef] [Green Version]
  30. Ishay, J.S.; Paniry, V.A. Effects of Caffeine and Various Xanthines on Hornets and Bees. Psychopharmacology 1979, 65, 299–309. [Google Scholar] [CrossRef]
  31. Fernandes, F.L.; Picanço, M.C.; Fernandes, M.; Queiroz, R.B.; Xavier, V.; Martinez, H. The Effects of Nutrients and Secondary Compounds of Coffea arabica on the Behavior and Development of Coccus viridis. Environ. Èntomol. 2012, 41, 333–341. [Google Scholar] [CrossRef] [PubMed]
  32. Nakayama, S.; Sasaki, K.; Matsumura, K.; Lewis, Z.; Miyatake, T. Dopaminergic system as the mechanism underlying personality in a beetle. J. Insect Physiol. 2012, 58, 750–755. [Google Scholar] [CrossRef] [PubMed]
  33. Nishi, Y.; Sasaki, K.; Miyatake, T. Biogenic amines, caffeine and tonic immobility in Tribolium castaneum. J. Insect Physiol. 2010, 56, 622–628. [Google Scholar] [CrossRef] [PubMed]
  34. Bernklau, E.; Bjostad, L.; Hogeboom, A.; Carlisle, A.; Arathi, H.S. Dietary Phytochemicals, Honey Bee Longevity and Pathogen Tolerance. Insects 2019, 10, 14. [Google Scholar] [CrossRef] [Green Version]
  35. Hu, Y.-T.; Wu, T.-C.; Yang, E.-C.; Wu, P.-C.; Lin, P.-T.; Wu, Y.-L. Regulation of genes related to immune signaling and detoxification in Apis mellifera by an inhibitor of histone deacetylation. Sci. Rep. 2017, 7, 41255. [Google Scholar] [CrossRef] [Green Version]
  36. Schmittgen, T.D.; Livak, K.J. Analyzing real-time PCR data by the comparative CT method. Nat. Protoc. 2008, 3, 1101–1108. [Google Scholar] [CrossRef]
  37. Gregorc, A.; Evans, J.D.; Scharf, M.; Ellis, J.D. Gene expression in honey bee (Apis mellifera) larvae exposed to pesticides and Varroa mites (Varroa destructor). J. Insect Physiol. 2012, 58, 1042–1049. [Google Scholar] [CrossRef]
  38. Biergans, S.D.; Galizia, C.G.; Reinhard, J.; Claudianos, C. Dnmts and Tet target memory-associated genes after appetitive olfactory training in honey bees. Sci. Rep. 2015, 5, 16223. [Google Scholar] [CrossRef] [Green Version]
  39. Livak, K.J.; Schmittgen, T.D. Analysis of relative gene expression data using real-time quantitative PCR and the 2(T)(-Delta Delta C) method. Methods 2001, 25, 402–408. [Google Scholar] [CrossRef]
  40. Chang, Y.; Tang, C.K.; Lin, Y.H.; Tsai, C.H.; Lu, Y.H.; Wu, Y.L. Snellenius manilae bracovirus suppresses the host immune system by regulating extracellular adenosine levels in Spodoptera litura. Sci. Rep. 2020, 10, 2096. [Google Scholar] [CrossRef]
  41. Mandal, A.; Poddar, M.K. Long-term caffeine consumption reverses tumor-induced suppression of the innate immune response in adult mice. Planta Med. 2008, 74, 1779–1784. [Google Scholar] [CrossRef]
  42. Barcelos, R.P.; Lima, F.D.; Carvalho, N.R.; Bresciani, G.; Royes, L.F. Caffeine effects on systemic metabolism, oxidative-inflammatory pathways, and exercise performance. Nutr. Res. 2020, 80, 1–17. [Google Scholar] [CrossRef] [PubMed]
  43. Horrigan, L.A.; Kelly, J.P.; Connor, T.J. Caffeine suppresses TNF-alpha production via activation of the cyclic AMP/protein kinase A pathway. Int. Immunopharmacol. 2004, 4, 1409–1417. [Google Scholar] [CrossRef] [PubMed]
  44. Torgersen, K.M.; Vang, T.; Abrahamsen, H.; Yaqub, S.; Taskén, K. Molecular mechanisms for protein kinase A-mediated modulation of immune function. Cell. Signal. 2002, 14, 1–9. [Google Scholar] [CrossRef]
  45. Gil Ferreira, Á.; Naylor, H.; Esteves, S.S.; Pais, I.; E Martins, N.; Teixeira, L. The Toll-Dorsal Pathway Is Required for Resistance to Viral Oral Infection in Drosophila. PLoS Pathog. 2014, 10, e1004507. [Google Scholar] [CrossRef] [Green Version]
  46. Avadhanula, V.; Weasner, B.P.; Hardy, G.G.; Kumar, J.P.; Hardy, R.W. A Novel System for the Launch of Alphavirus RNA Synthesis Reveals a Role for the Imd Pathway in Arthropod Antiviral Response. PLoS Pathog. 2009, 5, e1000582. [Google Scholar] [CrossRef] [Green Version]
  47. Galbraith, D.A.; Yang, X.Y.; Nino, E.L.; Yi, S.; Grozinger, C. Parallel Epigenomic and Transcriptomic Responses to Viral Infection in Honey Bees (Apis mellifera). PLoS Pathog. 2015, 11, e1004713. [Google Scholar] [CrossRef] [Green Version]
  48. Vannette, R.L.; Mohamed, A.; Johnson, B.R. Forager bees (Apis mellifera) highly express immune and detoxification genes in tissues associated with nectar processing. Sci. Rep. 2015, 5, 16224. [Google Scholar] [CrossRef] [Green Version]
Figure 1. Caffeine up-regulates the expression of most genes involved in immune responses upon deformed wing virus (DWV) infection compared to control group. The relative expression rate of immune-related genes involved in (A) Toll pathway, (B) Imd pathway, and (C) anti-microbial peptides (AMPs). C: caffeine; DWV: deformed wing virus treatment group; C/D: caffeine treatment followed by DWV infection. The results were from real-time polymerase chain reaction (RT-PCR). Clustering analysis was based on the Euclidean distance. (D) Relative expression of immune-related genes with significant changes (p < 0.05). The values from the control groups (without any treatment or infection) were set to 1 and the values of individual genes from treatment groups were normalized accordingly. The results are from data collected from three independent experiments and presented as mean ± standard deviation (SD), n = 3. Statistical analysis was performed using one-way analysis of variance (ANOVA) test followed by Tukey’s post hoc test. * p < 0.05 as compared to the DWV infection only group.
Figure 1. Caffeine up-regulates the expression of most genes involved in immune responses upon deformed wing virus (DWV) infection compared to control group. The relative expression rate of immune-related genes involved in (A) Toll pathway, (B) Imd pathway, and (C) anti-microbial peptides (AMPs). C: caffeine; DWV: deformed wing virus treatment group; C/D: caffeine treatment followed by DWV infection. The results were from real-time polymerase chain reaction (RT-PCR). Clustering analysis was based on the Euclidean distance. (D) Relative expression of immune-related genes with significant changes (p < 0.05). The values from the control groups (without any treatment or infection) were set to 1 and the values of individual genes from treatment groups were normalized accordingly. The results are from data collected from three independent experiments and presented as mean ± standard deviation (SD), n = 3. Statistical analysis was performed using one-way analysis of variance (ANOVA) test followed by Tukey’s post hoc test. * p < 0.05 as compared to the DWV infection only group.
Insects 11 00516 g001
Figure 2. Caffeine inhibits DWV infection by promoting carbohydrate metabolism. (A) Relative changes in virus copy number in honeybees infected with DWV and in honeybees receiving caffeine diet prior to DWV infection. Mean and standard deviation (SD) values are shown, n = 6, and p-values were calculated using one-way Student’s t-test (* p < 0.05). WV: DWV virus infection; C/D: caffeine diet followed by DWV infection. Real-time PCR was used to quantitate the gene expression levels of metabolic enzyme genes involved in (B) glycolysis and (C) the TCA cycle after honeybees were fed with artificial feed containing caffeine. The relative expression rate of metabolic genes involved in the carbohydrate metabolism of honeybees is presented as a heat map. Clustering analysis was based on the Euclidean distance. For the caffeine treatment group in comparison to the control group, green represents a relative gene expression change <1.0 and red represents a relative gene expression change between 1.0 and 2.0.
Figure 2. Caffeine inhibits DWV infection by promoting carbohydrate metabolism. (A) Relative changes in virus copy number in honeybees infected with DWV and in honeybees receiving caffeine diet prior to DWV infection. Mean and standard deviation (SD) values are shown, n = 6, and p-values were calculated using one-way Student’s t-test (* p < 0.05). WV: DWV virus infection; C/D: caffeine diet followed by DWV infection. Real-time PCR was used to quantitate the gene expression levels of metabolic enzyme genes involved in (B) glycolysis and (C) the TCA cycle after honeybees were fed with artificial feed containing caffeine. The relative expression rate of metabolic genes involved in the carbohydrate metabolism of honeybees is presented as a heat map. Clustering analysis was based on the Euclidean distance. For the caffeine treatment group in comparison to the control group, green represents a relative gene expression change <1.0 and red represents a relative gene expression change between 1.0 and 2.0.
Insects 11 00516 g002
Figure 3. Caffeine suppresses pathogen amplification in infected honeybees of mixed group. Replicative analysis of viruses in honeybees. Relative viral DNA replication was analyzed by real-time PCR. VDV: Varroa destructor virus; DWV: deformed wing virus; KBV: Kashmir bee virus; KV: Kakugo virus; IAPV: Israel acute paralysis virus; SBV: sacbrood virus; BQCV: black queen cell virus; CBPV: chronic bee paralysis virus. The data are presented as mean ± SD, n = 6. Values from the untreated groups were set to 1 and the values from caffeine-treated group were adjusted accordingly. Statistical analysis was performed using the Mann–Whitney U-test, * p < 0.05 relative to data collected from the control group.
Figure 3. Caffeine suppresses pathogen amplification in infected honeybees of mixed group. Replicative analysis of viruses in honeybees. Relative viral DNA replication was analyzed by real-time PCR. VDV: Varroa destructor virus; DWV: deformed wing virus; KBV: Kashmir bee virus; KV: Kakugo virus; IAPV: Israel acute paralysis virus; SBV: sacbrood virus; BQCV: black queen cell virus; CBPV: chronic bee paralysis virus. The data are presented as mean ± SD, n = 6. Values from the untreated groups were set to 1 and the values from caffeine-treated group were adjusted accordingly. Statistical analysis was performed using the Mann–Whitney U-test, * p < 0.05 relative to data collected from the control group.
Insects 11 00516 g003
Figure 4. Relative expression (rER) of immune-related genes in 16-day-old adult bees. (A) Toll pathway, (B) Imd pathway, (C) anti-microbial peptide (AMP) (D) relative expression of immune-related genes with significant changes (p < 0.05). The scale is the logarithm of the relative fold change (control group = 0), n = 5. Red indicates enhanced; green indicates inhibited. C: caffeine; DWV, deformed wing virus; C/D, caffeine with DWV treatment. Clustering analysis was based on the Euclidean distance. Fold changes were compared to those in the control groups. All results were analyzed based on data collected from three independent experiments and assessed by two-way ANOVA followed by Tukey’s post hoc test. * p < 0.05 as compared to the DWV infection only group.
Figure 4. Relative expression (rER) of immune-related genes in 16-day-old adult bees. (A) Toll pathway, (B) Imd pathway, (C) anti-microbial peptide (AMP) (D) relative expression of immune-related genes with significant changes (p < 0.05). The scale is the logarithm of the relative fold change (control group = 0), n = 5. Red indicates enhanced; green indicates inhibited. C: caffeine; DWV, deformed wing virus; C/D, caffeine with DWV treatment. Clustering analysis was based on the Euclidean distance. Fold changes were compared to those in the control groups. All results were analyzed based on data collected from three independent experiments and assessed by two-way ANOVA followed by Tukey’s post hoc test. * p < 0.05 as compared to the DWV infection only group.
Insects 11 00516 g004
Table 1. List of real-time polymerase chain reaction (RT-PCR) primers and target genes.
Table 1. List of real-time polymerase chain reaction (RT-PCR) primers and target genes.
Gene NameForward SequenceReverse Sequence
PersephoneCCGGTGAACTTGGAAAAGATATCGCAATTTGTCCCAAAAC
TollTAGAGTGGCGCATTGTCAAGATCGCAATTTGTCCCAAAAC
SpaetzleTGCACAAATTGTTTTTCCTGAGTCGTCCATGAAATCGATCC
PGRPS1TTTGAAAATTTCCTATGAAAGCATTTTTAATTGGTGGAGATGGAAA
PGRPS2TAATTCATCATTCGGCGACATGTTTGTCCCATCCTCTTCC
PGRPS3GAGGCTGGTACGACATTGGTTTATAACCAGGTGCGTGTGC
PGRPLCTCCGTCAGCCGTAGTTTTTCCGTTTGTGCAAATCGAACAT
Myd88TCACATCCAGATCCAACTGCCAGCTGACGTTTGAGATTTTTG
AbaecinCAGCATTCGCATACGTACCAGACCAGGAAACGTTGGAAAC
Defensin-1TGCGCTGCTAACTGTCTCAGAATGGCACTTAACCGAAACG
Defensin-2GCAACTACCGCCTTTACGTCGGGTAACGTGCGACGTTTTA
Cactus-1CACAAGATCTGGAGCAACGAGCATTCTTGAAGGAGGAACG
Cactus-2TTAGCAGGACAAACGGCTCTCAGAAAGTGGTTCCGGTGTT
Dorsal-1AAATGGTTCGCTCGTAGCACTCCATGATATGAGTGATGGAAA
PPOactGTTTGGTCGACGGAAGAAAACCGTCGACTCGAAATCGTAT
AmPPOAGATGGCATGCATTTGTTGACCACGCTCGTCTTCTTTAGG
HymenoptCTCTTCTGTGCCGTTGCATACGTCTCCTGTCATTCCATT
ApidaecTAGTCGCGGTATTTGGGAATTTTCACGTGCTTCATATTCTTCA
ApisiminTGAGCAAAATCGTTGCTGTCAACGACATCCACGTTCGATT
Lys-1GAACACACGGTTGGTCACTGATTTCCAACCATCGTTTTCG
Lys-2CCAAATTAACAGCGCCAAGTGCAATTCTTCACCCAACCAT
Lys-3ATCTGTTTGCGGACCATTTCTCGATGAATGCGAAGAAAATC
ImdTGTTAACGACCGATGCAAAACATCGCTCTTTTCGGATGTT
Tak-1ATGGATATGCTGCCAATGGTTCGGATCGCATTCAACATAA
DreddGCGTCATAAAGAAAAAGGATCATTTCGGGTAATTGAGCAACG
KennyGCTGAACCAGAAAGCCACTTTGCAAGTGATGATTGTTGGA
TabGCTATCATGCAGCTGTTCCAACACTGGGTCAGCCAATTTC
G6PGGTTGGAGGACGTTATTCCCATAAAGTGTGCTCCAC
PFKATCAGGGTATGGTAGATGGTGGTTTTGCGACCAGCGTGAT
TPIGAGGTTGTTGTTGGTGTACCCCAAAAGCATAGCAGGAC
PGMGGTACGTCATGGAGAAAGTGTTGCGTCTCTGATTGCTT
LDHGATGGAGGATAAATTAAAGGGAGAGATTGATTTTCGCGTTCCTCAAG
ENOACCTACAGGTGCTTCTAGGTAGCATCAAGACCAAAC
TrehalaseATGGAGCGGCACGAACAGGGTCGAACGTGTCGTTGA
SDHAGGAGGAGCGGGAAGATGTTCAAGACCATCTCCTGTGCATGT
ISDCATTGGCAGACATGCTCATGTGCAATGCCAGGTCCTTT
PCSGCAAATATTGGGAGTACGTCTATGAAGAATCGACAGATACGGTGTTACCA
Table 2. Relative expression of immune-related genes in honeybees of mixed group. The results were done by real-time PCR. The values in the table represent how much percent the relative gene expression level up-/down-regulated than control group in each treatment group. Statistical analysis was done by one-way ANOVA followed by Tukey’s post hoc test. “*” indicate p-values < 0.05. (Caffeine/DWV compared to DWV).
Table 2. Relative expression of immune-related genes in honeybees of mixed group. The results were done by real-time PCR. The values in the table represent how much percent the relative gene expression level up-/down-regulated than control group in each treatment group. Statistical analysis was done by one-way ANOVA followed by Tukey’s post hoc test. “*” indicate p-values < 0.05. (Caffeine/DWV compared to DWV).
Gene NameReferencesCaffeineDWVCaffeine/DWV
Toll PathwayUpDnUpDnUpDn
PersephHu et al., 201760%-94%-478% *-
TollHu et al., 201712%-73%-258%-
SpaetzleHu et al., 201732%-8%-101%-
PGRPS1Hu et al., 2017-11%-40%55% *-
PGRPS2Hu et al., 2017-6%304%-167%-
PGRPS3Hu et al., 2017-26%-51%28%-
Myd88Hu et al., 201718%-38%-220% *-
Cactus1Hu et al., 2017-21%64%-203%-
Cactus2Hu et al., 2017-9%31%-176%-
Dorsal1Hu et al., 2017182%-163%-165%-
Gene NameReferencesCaffeineDWVCaffeine/DWV
Imd PathwayUpDnUpDnUpDn
PGRPLCHu et al., 201747%-92%-215% *-
ImdHu et al., 2017107%-9%-144%-
Tak1Hu et al., 201723%-39%-160% *-
DreddHu et al., 201715%-29%-115%-
KennyHu et al., 201743%-68%-187% *-
TabHu et al., 20178%-99%-413% *-
Gene NameReferencesCaffeineDWVCaffeine/DWV
Amp (Anti-Microbial Peptide)UpDnUpDnUpDn
AbaecinHu et al., 201728%-15%--1%
Defensin-2Hu et al., 2017-4%198%-2712%-
Defensin-1Hu et al., 2017-55%-31%67%-
PPOactHu et al., 2017-14%--71% *-
HymenoptHu et al., 2017-3%97%-35%-
AmPPOHu et al., 2017-5%87%-315% *-
ApidaecHu et al., 2017-4%-33%-75%
ApisminHu et al., 2017-42%1%-40%-
Lys-1Hu et al., 201711%-21%-151% *-
Lys-2Hu et al., 2017-1%-35%50%-
Lys-3Hu et al., 201742%--2%119%-
Table 3. Relative expression of carbohydrate metabolism in honeybees of mixed group. The results were done by real-time PCR. The values in the table represent how much percent the relative gene expression level up-/down-regulated than control group in caffeine only treatment group. Statistical analysis was undertaken by one-way ANOVA followed by Tukey’s post-hoc test. Asterisks indicate p-values < 0.05.
Table 3. Relative expression of carbohydrate metabolism in honeybees of mixed group. The results were done by real-time PCR. The values in the table represent how much percent the relative gene expression level up-/down-regulated than control group in caffeine only treatment group. Statistical analysis was undertaken by one-way ANOVA followed by Tukey’s post-hoc test. Asterisks indicate p-values < 0.05.
Gene NameReferencesCaffeine
GlycolysisUpDn
G6P (Glucose-6-phosphate isomerase)Froman et al., 198940%-
PFK (6-phosphofructokinase)Vora et al., 198526%-
ENO (Enolase)Marcaida et al., 20064%-
PGM (Phosphoglycerate mutase)Lu and Kreckner, 1994-1.4%
TPI (Triose-phosphate isomerase)Daar et al., 198617%-
TrehalaseMori et al., 20096%-
LDH (L-lactate dehydrogenase)Chung et al., 1985-5%
Gene NameReferencesCaffeine
TCA CycleUpDn
SDH (Succinate dehydrogenase)Renkema et al., 201551%-
ISD (isocitrate dehydrogenase)Ceccarelli et al., 200212%-
PCS (citrate synthase)Goldenthal et al., 199831%-

Share and Cite

MDPI and ACS Style

Lu, Y.-H.; Wu, C.-P.; Tang, C.-K.; Lin, Y.-H.; Maaroufi, H.O.; Chuang, Y.-C.; Wu, Y.-L. Identification of Immune Regulatory Genes in Apis mellifera through Caffeine Treatment. Insects 2020, 11, 516. https://doi.org/10.3390/insects11080516

AMA Style

Lu Y-H, Wu C-P, Tang C-K, Lin Y-H, Maaroufi HO, Chuang Y-C, Wu Y-L. Identification of Immune Regulatory Genes in Apis mellifera through Caffeine Treatment. Insects. 2020; 11(8):516. https://doi.org/10.3390/insects11080516

Chicago/Turabian Style

Lu, Yun-Heng, Carol-P Wu, Cheng-Kang Tang, Yu-Hsien Lin, Houda Ouns Maaroufi, Yi-Chi Chuang, and Yueh-Lung Wu. 2020. "Identification of Immune Regulatory Genes in Apis mellifera through Caffeine Treatment" Insects 11, no. 8: 516. https://doi.org/10.3390/insects11080516

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop