Next Article in Journal
Covalent Inhibitors from Saudi Medicinal Plants Target RNA-Dependent RNA Polymerase (RdRp) of SARS-CoV-2
Next Article in Special Issue
AAV Vectors Pseudotyped with Capsids from Porcine and Bovine Species Mediate In Vitro and In Vivo Gene Delivery
Previous Article in Journal
Virus-like Particle Vaccines of Infectious Bursal Disease Virus Expressed in Escherichia coli Are Highly Immunogenic and Protect against Virulent Strain
Previous Article in Special Issue
Insights from the Construction of Adenovirus-Based Vaccine Candidates against SARS-CoV-2: Expecting the Unexpected
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Review

Suitable Disinfectants with Proven Efficacy for Genetically Modified Viruses and Viral Vectors

1
Laboratory Prof. Dr. G. Enders MVZ GbR, Rosenbergstr. 85, 70193 Stuttgart, Germany
2
Expert Committee on Virus Disinfection of the German Association for the Control of Viral Diseases (DVV) e.V. and the Society for Virology (GfV) e.V., 69126 Heidelberg, Germany
3
Paul-Ehrlich-Institute, Department of Virology, Paul-Ehrlich-Straße 51-56, 63225 Langen, Germany
4
Federal Institute for Drugs and Medical Devices, Kurt-Georg-Kiesinger-Allee 3, 53175 Bonn, Germany
5
Institute for Infection Medicine, Christian-Albrechts-University Kiel, University Clinic Schleswig-Holstein, Bruinswiker Straße 4, 24105 Kiel, Germany
6
VAH c/o Institute for Hygiene and Public Health, Venusberg-Campus 1, 53127 Bonn, Germany
7
Institute of Hygiene and Environmental Medicine, University Medicine Greifswald, W. Rathenaustr. 49, 17475 Greifswald, Germany
8
Institute of Virology, Hans-Meerwein Straße 2, 35043 Marburg, Germany
9
Institute for Medical Virology, University Hospital, Goethe University Frankfurt am Main, 60596 Frankfurt, Germany
10
Boehringer Ingelheim Therapeutics GmbH, Beim Braunland 1, 88416 Ochsenhausen, Germany
11
Friedrich-Loeffler-Institute, Federal Research Institute for Animal Health, Department of Experimental Animal Facilities and Biorisk Management, Suedufer 10, 17493 Greifswald-Insel Riems, Germany
12
Department for Molecular & Medical Virology, Ruhr University Bochum, 44801 Bochum, Germany
13
Dr. Bill + Partner GmbH Institute for Hygiene and Microbiology, Norderoog 2, 28259 Bremen, Germany
14
Institute of Hygiene and Applied Immunology, Medical University of Vienna, Kinderspitalgasse 15, 1090 Vienna, Austria
15
Austrian Society for Hygiene, Microbiology and Preventive Medicine (ÖGHMP) c/o MAW, Freyung 6/3, 1010 Vienna, Austria
*
Author to whom correspondence should be addressed.
Viruses 2023, 15(11), 2179; https://doi.org/10.3390/v15112179
Submission received: 18 September 2023 / Revised: 20 October 2023 / Accepted: 23 October 2023 / Published: 30 October 2023
(This article belongs to the Special Issue Novel Viral Vectors for Gene Therapy 2023)

Abstract

:
Viral disinfection is important for medical facilities, the food industry, and the veterinary field, especially in terms of controlling virus outbreaks. Therefore, standardized methods and activity levels are available for these areas. Usually, disinfectants used in these areas are characterized by their activity against test organisms (i.e., viruses, bacteria, and/or yeasts). This activity is usually determined using a suspension test in which the test organism is incubated with the respective disinfectant in solution to assess its bactericidal, yeasticidal, or virucidal activity. In addition, carrier methods that more closely reflect real-world applications have been developed, in which microorganisms are applied to the surface of a carrier (e.g., stainless steel frosted glass, or polyvinyl chloride (PVC)) and then dried. However, to date, no standardized methods have become available for addressing genetically modified vectors or disinfection-resistant oncolytic viruses such as the H1-parvovirus. Particularly, such non-enveloped viruses, which are highly resistant to disinfectants, are not taken into account in European standards. This article proposes a new activity claim known as “virucidal activity PLUS”, summarizes the available methods for evaluating the virucidal activity of chemical disinfectants against genetically modified organisms (GMOs) using current European standards, including the activity against highly resistant parvoviridae such as the adeno-associated virus (AAV), and provides guidance on the selection of disinfectants for pharmaceutical manufacturers, laboratories, and clinical users.

1. Introduction

In Europe, the spectrum of the efficacy of disinfection processes against viruses currently includes the following three activity levels [1,2], which are “active against enveloped viruses”, “limited spectrum of virucidal activity”, and “virucidal activity” (which includes both activity against enveloped viruses and a limited spectrum of virucidal activity). Depending on the risk assessment and area of application of the disinfectant (hands, surfaces, instruments, and laundry), different activity levels are required and can be declared on the basis of existing standardized test methods. For hygienic hand disinfection and surface disinfection, the required activity levels are “active against enveloped viruses”, “limited spectrum of virucidal activity”, and “virucidal activity”. For instrument disinfection they are “active against enveloped viruses” and “virucidal activity”, and for laundry disinfection it is “virucidal activity”.
The above-mentioned activity levels were originally developed for the medical area. For other areas, additional terms with different content are under discussion in the technical committee (TC) 216 of the European Committee for Standardisation (CEN, Comité Européen de Normalisation in French).
In addition to the efficacy of disinfection processes against viruses, this overview draws attention to genetically modified organisms (GMOs) of viral origin. They are used in numerous laboratories for very different purposes, e.g., for basic research (viral transduction), for research on and the production of vaccines (e.g., recombinant vaccine viruses), for gene transfer (viral vectors), or for direct tumour therapy (e.g., native oncolytic viruses (OV) such as H1-Parvovirus or recombinant oncolytic viruses such as the approved 2015 Talimogene Laherparepvec based on HSV-1) [3,4]. OVs include numerous virus families that exhibit natural tropisms for specific extracellular oncogenic receptors or have been genetically engineered to display targeted tropisms for tumour cells while being restricted in their effects on normal cells [5]. Furthermore, engineered OVs often additionally express immunotherapeutic factors. In contrast to gene therapy vectors, which are used to transfer therapeutic genetic material to target tissues and cannot actively infect the host cells, OVs are less attenuated and can replicate in infected tissues [6].
The continuous progress of research in this field and the constantly evolving technical possibilities have also led to the development of new perspectives and applications for cell- and gene-based medicines. For example, viral vectors are successfully used in the production of functionally modified T cells with chimeric antigen receptors (CAR-T cells) [7] or offer promising tools for the treatment of genetic diseases. Other gene therapy treatments have been designed to treat haemophilia A, haemophilia B, or patients with inherited rare diseases [8,9]. However, in the wake of the COVID-19 pandemic, there is a public perception and a growing trend of scepticism towards research institutions regarding poor lab safety standards or even lab leaks due to inadequate biosafety levels. Viral vectors or genetically modified viruses are assigned to different genetic safety levels. Initially, this occurs independently of the type and function of the nucleic acid insertions. In Germany, the classification of viral vectors or GMOs is carried out by the Central Commission for Biological Safety (ZKBS) on the basis of a risk assessment of the GMOs or viral vectors to determine their safety for humans, animals and the environment. Viral vectors are classified as GMOs. As [10] the ZKBS points out, organisms are not only biological entities that are capable of reproducing but also transmitting genetic material. In addition to the typical GMOs of retroviral, adenoviral, or parvoviral origin, oncolytic viruses that are not genetically modified, such as H-1 parvovirus, are also the subject of this overview.
In the production and use of GMOs or native oncolytic viruses, the question arises as to which disinfectants should be used when handling them (e.g., in research, development, production, or clinical application). The selection of a suitable disinfectant cannot be derived from genetic safety classification, as this indicates potential risks to other organisms and the environment but does not consider the tolerance or sensitivity of the GMOs or native oncolytic viruses to disinfectants.
The aim of this overview is to make the various aspects of efficacy testing and the spectrum of antiviral activity in the selection of disinfectants transparent for supervisory and monitoring authorities, biosafety officers, and operators of genetic engineering facilities. In this context, we primarily consider activity against viruses and address the anticipated changes caused by the European biocide legislation.

2. Efficacy of Disinfectants Depends on Viral Structure

The infectivity of virus particles is essentially determined by the nature of the viral envelope or the viral capsid (in the case of non-enveloped viruses). Their basic biological structure can be derived from their viral families or from the respective “donor and recipient organisms” that determine their biochemical and physical properties. Since disinfectants initially attack the viral envelope and/or the capsid of virus particles, the selection of the disinfectant for recombinant viruses or viral vectors (such as chimeric viruses) depends on the origin of the viral envelope or the viral capsid. Enveloped recombinant viruses such as viral vectors (GMOs) or “infectious” nucleic acids are unique cases since GMOs often contain modified viral proteins or other additionally expressed proteins in their viral membrane. Nevertheless, these genetically modified proteins are not expected to significantly alter the stability of the lipid envelope when exposed to disinfectants.
Comparative studies with a large number of enveloped viruses have shown that vaccinia virus—the model virus for enveloped viruses—still has the highest stability and is thus a good benchmark [11,12,13,14] and a well-suited model virus for enveloped viruses. Its stability might be higher than that of others, as the cell-associated enveloped virus (CEV) and the extracellular enveloped virus (EEV) are both surrounded by two membrane layers, while the intracellular enveloped virus (IEV) has three envelopes [15]. Aside from the structural proteins and lipids, viruses contain nucleic acid that can still be considered as infectious. However, not all disinfectants destroy or inactivate the nucleic acids (DNA or RNA) contained in the virus particles. Certain virus groups (e.g., vaccinia viruses, herpes viruses, adenoviruses, simian virus 40 (SV40), picornaviruses, or parvoviruses) possess so-called “infectious” virus genomes. In principle, progeny viruses can develop from these genomes if they enter or are introduced into suitable host cells. However, eukaryotic cells do not have natural mechanisms for the uptake of free (non-encapsulated) viral genomes. Therefore, extremely high concentrations and special technical procedures are required to introduce free nucleic acids into cells in sufficient quantities for productive replication. Therefore, the infectivity of isolated viral genomes plays only a minor role compared to the infectivity of complete viral particles in the context of the potential risk of infection they pose.
According to Klein and DeForest [16], the activity level for the declaration of disinfectants is derived from the virus structure [1,17]. Disinfectants that are “active against enveloped viruses” are sufficient for use against all enveloped viruses. Within the group of non-enveloped viruses, adenoviruses, noroviruses, and rotaviruses form the group of lipophilic non-enveloped viruses, for which products with the activity level “limited spectrum of virucidal activity” are used. For the more hydrophilic non-enveloped virus agents, the activity level “virucidal activity” is required. Correspondingly, the same applies mutatis mutandis to GMOs: GMOs or native oncolytic viruses based on enveloped viruses can be inactivated using disinfectants that are “active against enveloped viruses”. In the case of more lipophilic non-enveloped viruses—rotaviruses, noroviruses, adenoviruses or GMOs derived from them—disinfectants are required with at least the “limited spectrum of virucidal activity” activity level [2]. However, some GMOs or native oncolytic viruses based on hydrophilic non-enveloped viruses such as parvoviruses are not inactivated by products with a “virucidal activity” level. Thus, we require a disinfectant with activity against parvoviruses [18,19]. Therefore, the German Committee of Virus Disinfection (DVV/GfK) and the Disinfectant Commission of the VAH (Association of Applied Hygiene) [20] have implemented a new virucidal activity level “virucidal activity PLUS”, requiring a practical test against parvoviridae such as those described in the German guideline of 2012 for the quantitative evaluation of the virucidal activity of chemical disinfectants on non-porous surfaces [21].

3. Test Methods for the Declaration of Activity against Viruses

For the evaluation of the virucidal efficacy of disinfectants, a two-step procedure is provided at a national [22,23,24,25] and a European level [26,27,28,29,30,31]. In this model, the efficiency must be confirmed in quantitative suspension tests (phase 2, step 1) followed by tests under practical conditions (phase 2, step 2). In suspension tests, virus particles in solution are mixed with disinfectant plus interfering substances to simulate organic solutions and the remaining titres determined via endpoint titration. This generally determines whether a substance reduces viral infectivity and is thus active against viruses. Suspension tests are well suited for use in preliminary investigations but are carried out with a considerable excess of disinfectants, thus simulating conditions that are usually not present in practice. Therefore, the use of suspension tests often results in activities at lower concentrations and/or shorter exposure times compared to the tests performed under practical conditions (phase 2, step 2) and thus might overestimate efficacy.
Therefore, the tests performed under practical conditions must correspond to the situation in which the disinfectants are to be used, e.g., for hand hygiene, spraying surfaces, and immersing instruments. However, to ensure the efficacy of disinfectants, the test methods must not only simulate the type of application but should also consider possible organic contamination. At present, coordinated phase 2, step 2 tests are only used at the European level to assess antiviral efficacy for surface disinfection via the spray method (application without mechanical action, EN 16777) [28] and for chemical and chemo-thermal instrument disinfection via the immersion method (EN 17111) [29]. However, this is not the case for surface disinfection via mechanical action or hand disinfection. For this reason, viral test methods are currently being developed for application to the various areas of application, which also include the different areas of effect, by specifying the respective test viruses.
Specifically, hand disinfectants must be tested on the hands of volunteers and the achieved reduction must be compared to that achieved using a reference product tested in parallel on the same study subjects [32]. For surface disinfectants, the tests must consider whether the surfaces are wiped (i.e., disinfection is carried out with a mechanical component) or whether the product is sprayed on (without mechanical action). Instrument disinfectants are normally analyzed for their efficacy with immersion procedures.
The desired efficacy of products can only be guaranteed if they are also used as specified in the respective application-specific test. To date, test methods have been developed for the medical and veterinary fields and for the food sector. Special test methods have not yet been developed for the laboratory sector. If surfaces or the hands of employees are to be disinfected in laboratories, however, products with proven efficacy can be used in accordance with the test methods appropriate to the medical or veterinary field. But if the products are to be used in a way that deviates from the established applications, additional tests should be performed that largely take into account the desired use.
If there are no official national lists for disinfectants with proven virucidal activity, or if the existing lists do not include virus-active products (i.e., [33,34,35]), the user must rely on the label from the manufacturer.
In conclusion, the claim or labelling of products as being “active against enveloped viruses”, or as displaying “limited spectrum of virucidal activity” and “virucidal activity”, does not mean that products have undergone uniform tests and have actually met the requirements for the confirmation of efficacy, especially in the application setting.

4. Efficacy Level of Disinfectants and Surrogate Viruses for Disinfectant Testing in Germany and Europe

In Germany, virucidal claims such as “active against enveloped viruses”, “limited spectrum of virucidal activity”, or “virucidal activity” are in principle harmonized with those in the rest of Europe. However, the proof of disinfectant efficacy in Germany is based on more extensive tests than those used in Europe as additional surrogate viruses (see tables) and reproducibility (i.e., more test runs) are required. Table 1a–c list the requirements for virus-active products in the medical field.
As shown in Table 1a (active against enveloped viruses) and Table 1b (limited spectrum of virucidal activity), there is little to no difference in the test viruses. While the European method [27] uses vaccinia virus as the test virus in the quantitative suspension test (phase 2/step 1), the German methods [22,23] additionally use bovine viral diarrhoea virus (BVDV) when testing oxidative products with the activity level “active against enveloped viruses”.
In terms of “virucidal activity” (Table 1c), enteroviruses (EV) such as EV71 and EVD68 and poliovirus are covered by both the German and the European methods. However, this claim does not cover the highly resistant outbreak viruses hepatitis A (HAV) and hepatitis E (HEV). Also not included are parvoviridae, including adeno-associated viruses (AAV) used in gene therapy. Recently, clusters of AAV type 2 were globally reported to occur in children with acute severe hepatitis in 2022 [38,39]. An additional test virus simian virus (SV) 40 (as surrogate for human papillomavirus (HPV)) in the DVV/RKI test method makes another relevant clinical difference as this oncogenic virus is not included in the European standards.
Products with activity against enveloped viruses remain the correct choice for all indications where viruses are enveloped. Additionally, the most common non-enveloped viruses need to be adequately targeted in the area of the application of surface disinfection, including the disinfection of medical devices.
Since parvoviruses are particularly resistant to disinfectants, products that are to be used against parvoviridae, including AAV, native oncolytic parvoviruses, HAV and HEV, must also have proof of activity against parvoviruses. This requires that additional tests be performed with parvoviruses. However, the current European standards for testing under practical conditions [28,29] and drafts of new standards published in recent years do not include murine parvovirus (MVM) as a test organism mandated for testing under the practical conditions of surface and instrument disinfectants (with the exception of chemo-thermal instrument disinfection).
The routine use of murine parvovirus as a test organism is problematic because the active ingredient concentration or exposure time must be significantly increased to demonstrate sufficient efficacy against parvoviruses and those active ingredients are limited.
Therefore, for routine disinfection, the previous virucidal claim remains valid, but a new claim needs to be implemented in the EU to define virucidal activity against highly resistant viruses. As in the case of Germany [17], this could be the implementation of either “virucidal activity PLUS” or “high level virucidal active” for highly resistant viruses.
Table 2 presents a proposal for a new virucidal activity claim (“virucidal activity PLUS”) as already introduced in Germany.

5. Disinfectant for Oncolytic Viruses or Viral Vectors in Therapy

For users of disinfectants, it is not easy to select suitable conditions or products based on the information available regarding the various test methods. For example, parvoviruses are intrinsically resistant to alcoholic formulations [18,37,40]. This high tolerance to alcohols was also found for HEV [37] and HAV [41,42]. Therefore, hand disinfectants are not available against parvovirus-based GMOs or native oncolytic parvoviruses, nor are they produced for HAV. For hand hygiene, wearing protective gloves and hand washing is recommended such as in the case of bacterial spores [43,44]. Note that hand washing can only mechanically reduce the viral load from the hands as soaps are not active against these resilient viruses. In contrast, for products based on peracetic acid, aldehydes, oxygen scavengers, and chloramine-T, activity against murine parvovirus was confirmed in practical tests in various testing laboratories, leading some to claim “virucidal activity PLUS” for surface disinfection without mechanical action [19,45].
For users of disinfectants, it is not easy to select suitable conditions or products based on the information available regarding the various test methods. For example, parvoviruses are intrinsically resistant to alcoholic formulations [18,37,40]. This high tolerance to alcohols was also found for HEV [37] and HAV [41,42]. Therefore, hand disinfectants are not available against parvovirus-based GMOs or native oncolytic parvoviruses, nor are they produced for HAV. For hand hygiene, wearing protective gloves and hand washing is recommended such as in the case of bacterial spores [43,44]. Note that hand washing can only mechanically reduce the viral load from the hands as soaps are not active against these resilient viruses. In contrast, for products based on peracetic acid, aldehydes, oxygen scavengers, and chloramine-T, activity against murine parvovirus was confirmed in practical tests in various testing laboratories, leading some to claim “virucidal activity PLUS” for surface disinfection without mechanical action [19,45].
Based on the different virucidal activity levels as described in Table 1a–c and Table 2, Table 3 and Table 4 assign the corresponding disinfectant levels of activity to the most commonly used viral GMOs.
It is evident that an oncolytic herpesvirus should not differ from a field herpesvirus regarding its sensitivity to disinfectants as the most important point in this is the envelope, i.e., a lipid bilayer. Just as with pseudo- or hybrid viruses, where, e.g., VSV-G protein is incorporated into a lentivirus, characteristics such as tropism might be changed, but the new virus retains a lipid envelope that can easily be inactivated by disinfectants. Therefore, a lentiviral vector in gene therapy should be as susceptible to disinfectants as HIV. The use of nonenveloped viruses with a completely new capsid structure due to modifications of the capsid proteins can eventually lead to the production of viral-based vectors, a variety of viral cage with a completely different symmetry and, therefore, with different gaps generated between capsomers. These differences may increment or reduce the possibility of disinfectants acting and always must be characterized virologically first (tropism, etc.) and with regard to their susceptibility to disinfectants. However, so far, the authors are not aware of any virus in which the biophysical properties have been altered to the point that disinfectants are no longer active. Note, the possible assignment of reoviruses is still under discussion. As long as no extensive data on the tenacity of reoviruses are available, disinfectants should be used with the virucidal activity claim of action according to European standards. Despite this limitation, the information provided in this review is relevant for users and authorities as genetically modified viruses are used in research and medicine. Additionally, the knowledge of how to inactivate them must be made available.
Table 4 also takes into account helper viruses. These are used in the production of some viral vectors, and some of their structural components are required for the replication of the viral vector.

6. Future Aspects Due to European Biocide Regulation

The implementation of the European Biocide Regulation No. 528/2012 [46] is unsettling the monitoring authorities and genetic engineering facilities in Europe, as many well-established products may or will no longer be available on the market. Nowadays, all biocidal products require authorization before they can be placed on the market, and the active substances contained in that biocidal product must be previously approved. The Biocidal Products Regulation (BPR) classifies the biocidal products into 22 biocidal product-types, with surface disinfectants belonging to product type (PT) 2 “Disinfectants and algaecides not intended for direct use on humans or animals”. This product type contains, in addition to disinfectants used in the medical field or in laboratories, products for completely different applications such as the treatment of bath water, air conditioning systems, or soils (i.e., sand, soil). However, such a collective group makes it difficult to clearly delineate the areas of application and thus to select suitable products, e.g., for laboratories.
At present, the majority of disinfectants are marketable on the basis of transitional regulations, but they do not yet have authorization as biocidal products. The authorization of biocidal products includes not only efficacy testing but also the assessment of risks to humans and the environment and is carried out in a two-step process. First, the active substances contained in the biocidal products are evaluated using a European procedure. Active substances that have been positively evaluated (“approved”) are included in the so-called Union list. This list results in the deadlines during which the authorization of the product must be applied for. If no application is submitted, product marketability ends after a fixed period. Against this background, impacts on the availability of disinfectants and potentially on their operational conditions are already present or expected as the majority of disinfectants are subject to this regulation. As an example, Greek authorities have proposed to classify ethanol, which has been reliable as a disinfectant in everyday use, as a “CMR” (carcinogenic, mutagenic, and toxic for reproduction) [47], which would severely affect its use and thus eliminate a safe, available, and inexpensive ingredient.

7. Conclusions

As has been shown, a variety of test methods exist at both the national and European level that can be used for the evaluation of disinfectants with declared virucidal activity. In addition to the methods and claims described here for use in the human medical area, there are also existing national and European methods that can be applied to the veterinary field or food, industrial, domestic, and institutional issues. However, there are still no methods or specifications for laboratories working with viral GMOs or oncolytic viruses available. Additionally, highly resistant parvoviridae, HAV and HEV, are not yet considered for surface disinfection in European standards. Also, practical test methods for virucidal efficacy are currently not available for all applications such as surface, instrument, textile, or hand disinfection, although much has been implemented in recent years [48]. However, as specified in EN 14885 [26], further tests under additional conditions such as test organisms, temperature, contact time, and interfering substances should be carried out according to the claimed use of a disinfectant. Therefore, EN 14885 enables tests with a highly resistant test virus such as parvovirus, using a modified standard to provide an opportunity to test disinfectants against resistant GMO or oncolytic parvoviruses. This could be the basis for the new claim “virucidal PLUS”.
In Europe, the labelling of the virucidal efficacy of products by the manufacturer is not uniformly regulated for medicinal products and biocides [49]. In both cases, however, only the area of application (e.g., food, medical, institutional) and the instructions for use must be indicated on the product label, but not the test methods (if available) on which the virucidal activity level is based. Unfortunately, as long as no practical tests for surface disinfection with mechanical action (four field test) or hand disinfection are approved at the European level, a manufacturer can state and advertise a “virucidal activity” of such products even though it has only undergone a quantitative suspension test.
Taken together, the recommendation for and selection of suitable disinfectants with proven efficacy for GMO and viral vectors is complex. Since the evaluation of laboratory test reports for the selection or comparison of different possible disinfectants with proven virucidal efficacy requires special knowledge and extensive experience, access to accordingly reviewed lists of disinfectants with proven efficacy for the different levels may be helpful for both the user and for supervisory or monitoring authorities.

Author Contributions

M.E. and M.S. conceptualize, wrote and edited the manuscript, I.S., J.B., F.B., J.G., N.H., J.A.M., H.F.R., I.R., S.R., E.S., J.S., H.F. and P.Z. conceptualize and critically revised the final version of the manuscript. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The author I.R. is an employee of the company Boehringer Ingelheim Therapeutics GmbH and declares that no conflicts of interest exist regarding the data presented here. All other authors declare no conflict of interest.

References

  1. Tarka, P.; Nitsch-Osuch, A. Evaluating the Virucidal Activity of Disinfectants According to European Union Standards. Viruses 2021, 13, 534. [Google Scholar] [CrossRef]
  2. Eggers, M.; Schwebke, I.; Suchomel, M.; Fotheringham, V.; Gebel, J.; Meyer, B.; Morace, G.; Roedger, H.J.; Roques, C.; Visa, P.; et al. The European tiered approach for virucidal efficacy testing—Rationale for rapidly selecting disinfectants against emerging and re-emerging viral diseases. Eurosurveillance 2021, 26, 2000708. [Google Scholar] [CrossRef]
  3. Hajda, J.; Leuchs, B.; Angelova, A.L.; Frehtman, V.; Rommelaere, J.; Mertens, M.; Pilz, M.; Kieser, M.; Krebs, O.; Dahm, M.; et al. Phase 2 Trial of Oncolytic H-1 Parvovirus Therapy Shows Safety and Signs of Immune System Activation in Patients With Metastatic Pancreatic Ductal Adenocarcinoma. Clin. Cancer Res. 2021, 27, 5546–5556. [Google Scholar] [CrossRef]
  4. Chulpanova, D.S.; Solovyeva, V.V.; Kitaeva, K.V.; Dunham, S.P.; Khaiboullina, S.F.; Rizvanov, A.A. Recombinant Viruses for Cancer Therapy. Biomedicines 2018, 6, 94. [Google Scholar] [CrossRef]
  5. Jhawar, S.R.; Thandoni, A.; Bommareddy, P.K.; Hassan, S.; Kohlhapp, F.J.; Goyal, S.; Schenkel, J.M.; Silk, A.W.; Zloza, A. Oncolytic Viruses—Natural and Genetically Engineered Cancer Immunotherapies. Front. Oncol. 2017, 7, 202. [Google Scholar] [CrossRef]
  6. Bezeljak, U. Cancer gene therapy goes viral: Viral vector platforms come of age. Radiol. Oncol. 2022, 56, 1–13. [Google Scholar] [CrossRef]
  7. Suerth, J.D.; Schambach, A.; Baum, C. Genetic modification of lymphocytes by retrovirus-based vectors. Curr. Opin. Immunol. 2012, 24, 598–608. [Google Scholar] [CrossRef]
  8. Long, B.R.; Veron, P.; Kuranda, K.; Hardet, R.; Mitchell, N.; Hayes, G.M.; Wong, W.Y.; Lau, K.; Li, M.; Hock, M.B.; et al. Early Phase Clinical Immunogenicity of Valoctocogene Roxaparvovec, an AAV5-Mediated Gene Therapy for Hemophilia A. Mol. Ther. 2021, 29, 597–610. [Google Scholar] [CrossRef]
  9. Keeler, A.M.; Flotte, T.R. Recombinant Adeno-Associated Virus Gene Therapy in Light of Luxturna (and Zolgensma and Glybera): Where Are We, and How Did We Get Here? Annu. Rev. Virol. 2019, 6, 601–621. [Google Scholar] [CrossRef]
  10. Available online: https://zag.bvl.bund.de/organismen/index.jsf?dswid=8616&dsrid=287 (accessed on 20 June 2023).
  11. Sauerbrei, A.; Schacke, M.; Glück, B.; Bust, U.; Rabenau, H.F.; Wutzler, P. Does limited virucidal activity of biocides include duck hepatitis B virucidal action? BMC Infect. Dis. 2012, 12, 276. [Google Scholar] [CrossRef]
  12. Eggers, M.; Eickmann, M.; Kowalski, K.; Zorn, J.; Reimer, K. Povidone-iodine hand wash and hand rub products demonstrated excellent in vitro virucidal efficacy against Ebola virus and modified vaccinia virus Ankara, the new European test virus for enveloped viruses. BMC Infect. Dis. 2015, 15, 375. [Google Scholar] [CrossRef] [PubMed]
  13. Eggers, M.; Eickmann, M.; Zorn, J. Rapid and effective virucidal activity of povidone-iodine products against Middle East Respiratory Syndrome Coronavirus (MERS-CoV) and Modified Vaccinia Virus Ankara (MVA). Infect. Dis. Ther. 2015, 4, 491–501. [Google Scholar] [CrossRef] [PubMed]
  14. Siddharta, A.; Pfaender, S.; Vielle, N.J.; Dijkman, R.; Friesland, M.; Becker, B.; Yang, J.; Engelmann, M.; Todt, D.; Windisch, M.P.; et al. Virucidal activity of World Health Organization-recommended formulations against enveloped viruses, including Zika, Ebola, and emerging coronaviruses. J. Infect. Dis. 2017, 215, 902–906. [Google Scholar] [CrossRef] [PubMed]
  15. Smith, G.L.; Vanderplasschen, A.; Law, M. The formation and function of extracellular enveloped vaccinia virus. J. Gen. Virol. 2002, 83 Pt 12, 2915–2931. [Google Scholar] [CrossRef] [PubMed]
  16. Klein, M.; De Forest, A. Principles of viral inactivation. In Disinfection, Sterilisation and Preservation, 3rd ed.; Block, S.S., Ed.; Lea & Febiger: Philadelphia, PA, USA, 1983; pp. 422–434. [Google Scholar]
  17. Schwebke, I.; Eggers, M.; Gebel, J.; Geisel, B.; Glebe, D.; Rapp, I.; Steinmann, J.; Rabenau, H.F. Prüfung und Deklaration der Wirksamkeit von Desinfektionsmitteln gegen Viren zur Anwendung im human-medizinischen Bereich Stellungnahme des Arbeitskreises Viruzidie beim Robert Koch-Institut. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz 2017, 60, 353–363. [Google Scholar] [CrossRef] [PubMed]
  18. Eterpi, M.; McDonnell, G.; Thomas, V. Disinfection efficacy against parvoviruses compared with reference viruses. J. Hosp. Infect. 2009, 73, 64–70. [Google Scholar] [CrossRef]
  19. Rabenau, H.F.; Steinmann, J.; Rapp, I.; Schwebke, I.; Eggers, M. Evaluation of a Virucidal Quantitative Carrier Test for Surface Disinfectants. PLoS ONE 2014, 9, e86128. [Google Scholar] [CrossRef]
  20. Gemeinsame Mitteilung der Desinfektionsmittel-Kommission im VAH und der Kommission Virusdesinfektion von DVV und GfV. Harmonisierung der Anforderungen an die viruzide Wirksamkeit von chemischen Flächendesinfektionsverfahren im praxisnahen Test. Hyg. Med. 2023, 48, 69–72.
  21. Rabenau, H.F.; Schwebke, I.; Steinmann, J.; Eggers, M.; Rapp, I.; Neumann-Haefelin, D. Quantitative test for the evaluation of virucidal activity of chemical disinfectants on non-porous surfaces. Hyg. Med. 2012, 37, 459–466. [Google Scholar]
  22. Rabenau, H.F.; Schwebke, I.; Blümel, J.; Eggers, M.; Glebe, D.; Rapp, I.; Sauerbrei, A.; Steinmann, E.; Steinmann, J.; Willkommen, H.; et al. Guideline of the German Association for Combating Viral Diseases (DVV) e.V. and the Robert Koch Institute (RKI) on testing chemical disinfectants for efficacy against viruses in human medicine, version of 1 December 2014. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz 2015, 58, 493–504. [Google Scholar] [CrossRef]
  23. 2nd Communication of the Expert Committee on Virus Disinfection of the DVV/GfV and the RKI on the examination temperature when testing chemical or chemothermal instrument disinfection procedures in accordance with the DVV/RKI Guideline in the version of 01.12.2014. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz 2015, 58, 888.
  24. DVG Test Methods. Available online: http://www.desinfektion-dvg.de/index.php?id=1810 (accessed on 20 October 2023).
  25. Disinfectants Commission in the Association for Applied Hygiene (VAH). Requirements and methods for VAH certification of chemical disinfection procedures. Loose-Leaf Collection, 1 September 2022. [Google Scholar]
  26. EN 14885:2023; Chemical Disinfectants and Antiseptics. Application of European Standards for Chemical Disinfectants and Antiseptics. CEN—Comité Européen de Normalisation: Brussels, Belgium, 2023.
  27. EN 14476:2013 + A2:2019; Chemical Disinfectants and Antiseptics—Quantitative Suspension Test for the Determination of Virucidal Activity in Human Medicine—Test Method and Requirements (Phase 2, Step 1). CEN—Comité Européen de Normalisation: Brussels, Belgium, 2019.
  28. EN 16777:2019; Chemical Disinfectants and Antiseptics—Quantitative Test on Non-Porous Surfaces without Mechanical Action to Determine Virucidal Activity in Human Medicine—Test Method and Requirements (Phase 2, Step 2). CEN—Comité Européen de Normalisation: Brussels, Belgium, 2019.
  29. EN 17111:2018; Chemical Disinfectants and Antiseptics—Quantitative Carrier Test for Virucidal Activity for Instruments Used in the Field of Human Medicine—Test Method and Requirements (Phase 2, Step 2). CEN—Comité Européen de Normalisation: Brussels, Belgium, 2018.
  30. EN 14675: 2015; Chemical Disinfectants and Antiseptics—Quantitative Suspension Test for the Determination of Virucidal Activity of Chemical Disinfectants and Antiseptics for Veterinary Use—Test Method and Requirements (Phase 2, Step 1). CEN—Comité Européen de Normalisation: Brussels, Belgium, 2015.
  31. EN 17122: 2020; Chemical Disinfectants and Antiseptics—Quantitative Surface Test for the Determination of Virucidal Activityof Chemical Disinfectants and Antiseptics for Veterinary Use on Non-Porous Surfaces—Test Method and Requirements (Phase 2, Step 2). CEN—Comité Européen de Normalisation: Brussels, Belgium, 2020.
  32. Eggers, M.; Suchomel, M. In-vivo efficacy of alcohol-based hand rubs against noroviruses: A novel standardized European test method simulating practical conditions. J. Hosp. Infect. 2023, 135, 186–192. [Google Scholar] [CrossRef] [PubMed]
  33. Disinfectant List of the VAH. Available online: https://vah-online.de/en/desinfektionsmittel-liste (accessed on 20 October 2023).
  34. Expertisenverzeichnis der Österreichischen Gesellschaft für Hygiene, Mikrobiologie und Präventivmedizin (ÖGHMP). Available online: https://expertisen.oeghmp.at/ (accessed on 20 October 2023).
  35. Liste der vom BAG zugelassenen Desinfektionsmittel zur Bekämpfung von Influenza und Coronaviren(BAG), Schweiz. Available online: https://echa.europa.eu/documents/10162/29202019/switzerland_list_auth_disinfectant_products_en.pdf/f96e52fc-6dc6-9a2b-c770-3833b5000043 (accessed on 20 October 2023).
  36. Miyazawa, D. Potential mechanisms by which adeno-associated virus type 2 causes unexplained hepatitis in children. J. Med Virol. 2022, 94, 5623–5624. [Google Scholar] [CrossRef] [PubMed]
  37. Servellita, V.; Gonzalez, A.S.; Lamson, D.M.; Foresythe, A.; Huh, H.J.; Bazinet, A.L.; Bergman, N.H.; Bull, R.L.; Garcia, K.Y.; Goodrich, J.S.; et al. Adeno-associated virus type 2 in US children with acute severe hepatitis. Nature 2023, 617, 574–580. [Google Scholar] [CrossRef]
  38. Hufbauer, M.; Wieland, U.; Gebel, J.; Steinmann, J.; Akgül, B.; Eggers, M. Inactivation of Polyomavirus SV40 as Surrogate for Human Papillomaviruses by Chemical Disinfectants. Viruses 2021, 13, 2207. [Google Scholar] [CrossRef]
  39. Steinmann, J.; Eggers, M.; Rapp, I.; Todt, D.; Steinmann, E.; Brill, F.H.H.; Schwebke, I. Evaluation of the substitution of poliomyelitis virus for testing virucidal activities of instrument and surface disinfection. J. Hosp. Infect. 2022, 122, 60–63. [Google Scholar] [CrossRef]
  40. Behrendt, P.; Friesland, M.; Wißmann, J.-E.; Kinast, V.; Stahl, Y.; Praditya, D.; Hueffner, L.; Nörenberg, P.M.; Bremer, B.; Maasoumy, B.; et al. Hepatitis E virus is highly resistant to alcohol-based disinfectants. J. Hepatol. 2022, 76, 1062–1069. [Google Scholar] [CrossRef]
  41. WoIff, M.; Schmitt, J.; Rahaus, M.; König, A. Hepatitis A virus: A test method for virucidal activity. J. Hosp. Infect. 2001, 48 (Suppl. SA), S18–S22. [Google Scholar] [CrossRef]
  42. Kampf, G. Efficacy of ethanol against viruses in hand disinfection. J. Hosp. Infect. 2018, 98, 331–338. [Google Scholar] [CrossRef]
  43. Recommendation of the Commission for Hospital Hygiene and Infection Prevention (KRINKO) at the Robert Koch Institute (RKI): Hygienemaßnahmen bei Clostridioides difficile-Infektion (CDI). Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz 2019, 62, 906–923. [CrossRef]
  44. Recommendation of the Commission for Hospital Hygiene and Infection Prevention (KRINKO) at the Robert Koch Institute (RKI): Händehygiene in Einrichtungen des Gesundheitswesens. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz 2016, 59, 1189–1220. [CrossRef] [PubMed]
  45. Robert Koch Institute. List of disinfectants and disinfection methods tested and approved by the Robert Koch Institute. Bundesgesundheitsblatt 2017, 60, 1274–1297. [Google Scholar]
  46. Regulation (EU) No 528/2012 of the European Parliament and of the Council of 22 May 2012 Concerning the Making Available on the Market and Use of Biocidal Products. Official Journal of the European Union L; 167, pp. 1–123. Available online: https://eur-lex.europa.eu/LexUriServ/LexUriServ.do?uri=OJ:L:2012:167:0001:0123:en:PDF (accessed on 20 October 2023).
  47. Kramer, A.; Arvand, M.; Christiansen, B.; Dancer, S.; Eggers, M.; Exner, M.; Müller, D.; Mutters, N.T.; Schwebke, I.; Pittet, D. Ethanol is indispensable for virucidal hand antisepsis: Memorandum from the alcohol-based hand rub (ABHR) Task Force, WHO Collaborating Centre on Patient Safety, and the Commission for Hospital Hygiene and Infection Prevention (KRINKO), Robert Koch Institute, Berlin, Germany. Antimicrob. Resist. Infect. Control. 2022, 11, 93. [Google Scholar] [CrossRef]
  48. prEN 17430:2022; Chemical Disinfectants and Antiseptics—Hygienic Handrub Virucidal—Test Method and Requirements (Phase 2/Step 2). CEN—Comité Européen de Normalisation: Brussels, Belgium, 2022.
  49. Lilienthal, N.; Baumann, A.; Respondek, V.; Hübner, N.-O.; Schwebke, I. Quo vadis Händedesinfektionsmittel: Arzneimittel oder Biozidprodukt? Epidemiol. Bull. 2021, 17, 5–13. [Google Scholar] [CrossRef]
Table 1. (a) Overview of disinfectant activity level “active against enveloped viruses” and associated test methods for the human medical area. (b) Overview of disinfectant activity level “limited spectrum of virucidal activity” and associated test methods for the human medical area. (c) Overview of disinfectant activity level “virucidal activity” and associated test methods for the human medical area.
Table 1. (a) Overview of disinfectant activity level “active against enveloped viruses” and associated test methods for the human medical area. (b) Overview of disinfectant activity level “limited spectrum of virucidal activity” and associated test methods for the human medical area. (c) Overview of disinfectant activity level “virucidal activity” and associated test methods for the human medical area.
(a)
Active againstEnveloped viruses
(e.g., HBV, HCV, HIV, influenza viruses, herpes viruses)
Basis of the declarationTest methods and associated test viruses
DVV/VAHEuropean standards
Quantitative suspension test
(Phase 2/Step 1)
DVV/RKI Guideline [22,23]
EN 14476 [27]
EN 14476 [27]
Vaccinia virus *
BVDV (oxidative products)
Vaccinia virus *
Practical test:
Surface disinfection
(Phase 2/Step 2)
DVV Guideline [21]
EN 16777 [28]
EN 16777 [28]
Vaccinia virus *Vaccinia virus *
Practical test: Instrument disinfectio
(Phase 2/Step 2)
EN 17111 [29] (products for pre-cleaning with a combined cleaner/disinfectant)
Vaccinia virus *Vaccinia virus *
(b)
Active againstEnveloped viruses and lipophilic non-enveloped viruses (adenoviruses, noroviruses and rotaviruses)
Basis of the declarationTest methods and associated test viruses
DVV/VAHEuropean standards
Quantitative suspension test
(Phase 2/Step 1)
DVV/RKI Guideline [22,23]
EN 14476 [27]
EN 14476 [27]
Adenovirus
MNV
Adenovirus
MNV
Practical test:
Surface disinfection
(Phase 2/Step 2)
DVV Guideline [21]
EN 16777 [28]
EN 16777 [28]
Adenovirus
MNV
Adenovirus
MNV
Practical test: Instrument disinfection
(Phase 2/Step 2)
Claim cannot be issued
(c)
Active againstEnveloped and non-enveloped viruses (except parvoviridae such as AAV, HAV, HEV)Enveloped and non-enveloped viruses (except parvoviridae such as AAV, HAV, HEV, papillomaviruses)
Basis of the declarationTest methods and associated test viruses
DVV/VAHEuropean standards
Quantitative suspension test
(Phase 2/Step 1)
DVV/RKI Guideline [22,23]
EN 14476 [27]
EN 14476 [27]
Adenovirus
MNV
Poliovirus
SV40 [36]
MVM (instrument disinfection ≥ 40 °C)
Adenovirus
MNV
Poliovirus
MVM (instrument disinfection ≥ 40 °C)
Practical test:
Surface disinfection
(Phase 2/Step 2)
DVV Guideline [21]
EN 16777 [28]
EN 16777 [28]
Adenovirus
MNV
Adenovirus
MNV
Practical test:
Instrument disinfection
(Phase 2/Step 2)
EN 17111 [29]
Adenovirus (<40 °C)
MNV (<40 °C)
SV40 (<40 °C)
MVM (≥40 °C)
Adenovirus (<40 °C)
MNV (<40 °C)

MVM (≥40 °C)
* Modified vaccinia virus Ankara (MVA) or vaccinia virus strain Elstree. HBV, hepatitis B virus; HCV, hepatitis C virus; HIV, human immunodeficiency virus; BVDV, bovine viral diarrhoea virus. MNV, murine norovirus. AAV, adeno-associated virus; HAV, hepatitis A virus; HEV, hepatitis E virus; SV40, simian virus 40; MVM, murine parvovirus (minute virus of mice; rodent protoparvovirus 1). Although poliovirus is usually the most resistant test virus in the suspension test, it cannot be used for practical tests due to its loss in titre after drying [19,37].
Table 2. Proposal for a disinfectant activity level “virucidal activity PLUS” and associated test methods for the human medical area.
Table 2. Proposal for a disinfectant activity level “virucidal activity PLUS” and associated test methods for the human medical area.
Active againstenveloped and non-enveloped viruses (incl. parvoviridae such as AAV, HAV, HEV, papillomaviruses)Claim cannot be issued yet
Basis of the declarationTest methods and associated test viruses
DVV/VAHEuropean standards
Quantitative suspension test
(phase 2/step 1)
DVV/RKI Guideline [22,23] EN 14476 [27]n.a.
Adenovirus
MNV
Poliovirus
SV40
n.a.
Practical test:
Surface disinfection
(phase 2/step 2)
DVV Guideline [21]n.a.
Adenovirus
MNV
MVM
n.a.
AAV, adeno-associated virus; HAV, hepatitis A virus; HEV, hepatitis E virus; MNV, murine norovirus; SV40, simian virus 40; MVM, murine parvovirus (minute virus of mice; rodent protoparvovirus 1). n.a. not applicable.
Table 3. Selected oncolytic viruses and their required disinfectant activity levels.
Table 3. Selected oncolytic viruses and their required disinfectant activity levels.
Oncolytic VirusVirus Family/
Group
Replication-CompetentDisinfectant Activity Level to be Used
Recombinant adenovirus (GMO)Adenoviridaeyeslimited spectrum of virucidal activity
Herpes simplex virus type 1 (GMO or no GMO)Herpesviridaeyesactive against enveloped viruses
Newcastle disease virusParamyxoviridaeyesactive against enveloped viruses
Recombinant measles virus (GMO)Paramyxoviridaeyesactive against enveloped viruses
VSVRhabdoviridaeyesactive against enveloped viruses
ZikavirusFlaviviridaeyesactive against enveloped viruses
PoliovirusPicornaviridaeyesvirucidal activity
Parvovirus H-1 Parvoviridaeyesvirucidal activity PLUS
recombinant vaccinia virus (GMO)Poxviridaeyesactive against enveloped viruses
recombinant fowlpox virus (GMO)Poxviridaeyesactive against enveloped viruses
Reovirus 3Reoviridaeyesvirucidal activity *
Senecavirus APicornaviridaeyesvirucidal activity *
Coxsackievirus A21Picornaviridaeyesvirucidal activity *
Poliovirus type 1 Sabin (GMO)Picornaviridaeyesvirucidal activity *
* virucidal activity according to European standards is sufficient.
Table 4. Recombinant viral vectors for gene therapy and vaccines as GMOs and the activity level of disinfectants.
Table 4. Recombinant viral vectors for gene therapy and vaccines as GMOs and the activity level of disinfectants.
Recombinant Vectors and Virus DerivativesVirus Family/GroupReplication-Competent Helper VirusDisinfectant Activity Level to be Used
AdenovirusesAdenoviridaeyes limited spectrum of virucidal activity
noAdenoviruseslimited spectrum of virucidal activity
Herpes virusesHerpesviridaeyes active against enveloped viruses
Measles virusParamyxoviridaeyes active against enveloped viruses
AAV vectors (all AAV types)Parvoviridaeno Baculovirusesvirucidal activity PLUS
Adenovirusesvirucidal activity PLUS
Herpes virusesvirucidal activity PLUS
Helper virus-free system for the production of AAV vectorsvirucidal activity PLUS
Vaccinia virusPoxviridaeyes active against enveloped viruses
Lentiviral vectors (HIV)Retroviridaeyes active against enveloped viruses
no active against enveloped viruses
MuLV vectors and other retroviral vectorsRetroviridaeyes active against enveloped viruses
no active against enveloped viruses
VSV Rhabdoviridaeyes active against enveloped viruses
AAV, adeno-associated virus; MuLV, murine leukaemia virus; VSV, vesicular stomatitis virus.
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Eggers, M.; Schwebke, I.; Blümel, J.; Brandt, F.; Fickenscher, H.; Gebel, J.; Hübner, N.; Müller, J.A.; Rabenau, H.F.; Rapp, I.; et al. Suitable Disinfectants with Proven Efficacy for Genetically Modified Viruses and Viral Vectors. Viruses 2023, 15, 2179. https://doi.org/10.3390/v15112179

AMA Style

Eggers M, Schwebke I, Blümel J, Brandt F, Fickenscher H, Gebel J, Hübner N, Müller JA, Rabenau HF, Rapp I, et al. Suitable Disinfectants with Proven Efficacy for Genetically Modified Viruses and Viral Vectors. Viruses. 2023; 15(11):2179. https://doi.org/10.3390/v15112179

Chicago/Turabian Style

Eggers, Maren, Ingeborg Schwebke, Johannes Blümel, Franziska Brandt, Helmut Fickenscher, Jürgen Gebel, Nils Hübner, Janis A. Müller, Holger F. Rabenau, Ingrid Rapp, and et al. 2023. "Suitable Disinfectants with Proven Efficacy for Genetically Modified Viruses and Viral Vectors" Viruses 15, no. 11: 2179. https://doi.org/10.3390/v15112179

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop