Next Article in Journal
Super-Hydrophobicity of Polyester Fabrics Driven by Functional Sustainable Fluorine-Free Silane-Based Coatings
Next Article in Special Issue
Feasibility of Imaging Modalities Combined with a Silicone Gel-Filled Breast Implant in Korean Women
Previous Article in Journal
Niosomes for Topical Application of Antioxidant Molecules: Design and In Vitro Behavior
Previous Article in Special Issue
Recent Advances in Hydrogels via Diels–Alder Crosslinking: Design and Applications
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Antimicrobial Efficacy Assessment and Rheological Investigation of Two Different Hand Sanitizers Compared with the Standard Reference WHO Formulation 1

1
UB-CARE S.r.l. Spin-Off, University of Pavia, 27010 Prado, Italy
2
Head and Neck Unit, Clinica Cobellis, 84078 Vallo della Lucania, Italy
3
Centro Medico Polispecialistico, 27100 Pavia, Italy
4
Centro Avanzado de Dermatologia y Laser, San Pedro Sula 21101, Honduras
5
Lem Compounding Research S.r.l., 20823 Lentate sul Seveso, Italy
6
Department of Medicine and Surgery, University of Insubria, 21100 Varese, Italy
*
Authors to whom correspondence should be addressed.
These authors contributed equally to this work.
Gels 2023, 9(2), 108; https://doi.org/10.3390/gels9020108
Submission received: 13 December 2022 / Revised: 19 January 2023 / Accepted: 24 January 2023 / Published: 27 January 2023
(This article belongs to the Special Issue Women’s Special Issue Series: Gels)

Abstract

:
(1) Background: recently, the use of alcohol-based hand sanitizers (ABHSs) has become very frequent, and an evaluation of the stability and effectiveness of their formulations is a critical topic which should be carefully considered. (2) Methods: starting from the characterization of the hand sanitizers object of the study, our interest was focused on their rheological behavior in order to confirm their intrinsic features, but also the stability of each formulation in different conditions of shear and temperature; the second aspect concerns the antimicrobial assessment through a panel of in vitro and in vivo experimental trials. (3) Results: rheological investigation confirmed good stability for the two hand sanitizers in gel formula with respect to the reference in liquid formula; the antimicrobial activity evaluation showed good efficacy of each formulation both in vitro and in vivo. (4) Conclusions: altogether, our overview presents a valid quality control assessment to ensure the stability and efficacy of an alcohol-based hand sanitizer.

1. Introduction

Hospital-acquired infections (HAIs) and community-acquired infections are serious public diseases that are currently the result of prolonged hospitalization or the consequence of common transmission in social environments such as schools, offices or meeting places. The World Health Organization (WHO) and the Centers for Disease Control and Prevention (CDC) propose hand hygiene as the simplest and most preferential control measure in the prevention of transversal microbial or viral transmission [1].
Satisfactory hand cleaning can be achieved by frequent hands washing with water and soap or by alcohol-based hand sanitizers (ABHSs). The WHO definition of hand sanitizers is “An alcohol-containing preparation (liquid, gel, or foam) designed for application to the hands to inactivate microorganisms and/or temporarily suppress their growth” [2]. For a long time, ABHSs have been a useful means to improve sanitary conditions in developing countries.
Recently, during the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic, the habitual consumption of hand sanitizers has exponentially increased globally as a means to reduce and contain horizontal virus transmission. During this emergency state, the WHO published two different hand sanitizer formulations and the relative guidelines for their preparation [3,4]. At the same time, many brands and companies have shifted their core business to hand sanitizer production.
According to the WHO and CDC guidelines, preparations may contain one or more types of alcohol (ethanol, isopropanol, or propanol), other active ingredients with excipients, and humectants. To guarantee the best sanitization, hand sanitizers should contain a recommended content of alcohol (from 60 to 95%) with the function of antimicrobial and antiseptic agents [5]. WHO formulations present some limitations related to their liquid appearance, such as difficulty in spreading or in the delivery of insufficient quantities of the product. Gel hand sanitizer formulations have become popular due to their viscosity, which influences the efficiency, performance, and ease of dispensing, and the use of on-the-go sanitizer to avoid the risk of leakage. Another advantage of the gel formulation is its fast absorption, pleasant smell, and hand feel.
Gel preparations reduce the rate of alcoholic evaporation, so they allow deeper penetration into contaminating organisms and better spread ability [6]. To guarantee these properties, some viscosity enhancer excipients are incorporated in the formulation, such as carbomers, hydroxyethylcellulose (HEC) and xanthan gum (XG) [7]. Carbomers, also called carboxypolymethylene or carbopol (CBP), are cross-linked polyacrylic acid polymers that are suspending agents, stabilizers, and thickeners; in particular, their three-dimensional network is formed by polymeric chain entanglements that, in aqueous solution and upon hydration, start to uncoil and expand [8]. Carbomers can be subdivided into categories characterized by different degrees of cross-linking density. The maximum viscosity degree is reached at a pH of approximately 6.5–7.5, while it decreases at a pH ≥ 9. If a carbomer is added in a formulation that contains alcohol, such as hand sanitizers, the pH must be corrected using neutralizers that do not cause polymer precipitation [6]. HEC is an ether of cellulose that is easily dissolved in water but is insoluble in organic solvents. It is used as a rheological modifier, stabilizer, thickener, suspending agent, and colloidal protective agent. HEC viscosity decreases with increasing alcohol concentration in hand sanitizer [9]. XG is a nontoxic, biocompatible, biodegradable, and inexpensive viscosity enhancer; therefore, in recent years, the field of application of this excipient has increased.
On the other hand, viscous formulations promote dehydration of the skin after prolonged use, obtaining a skin dryness effect [7]. These side effects can be avoided by adding excipients in formulations that balance the side effects with efficacy, safety, and compatibility with different skin types. According to the WHO, fragrances must be limited or completely eliminated within the formulation, as they have been identified as the major cause of contact allergies [2].
Currently, during the SARS-CoV-2 pandemic, ABHSs have been distributed in many countries without an assessment of their effectiveness. In this study, a panel of rheological and microbiological tools were exploited to investigate the efficacy and the chemical-physical stability of two different formulations produced during the emergency state of 2020. In particular, the microbiological trials intend to demonstrate the antimicrobial efficacy of the formulations against the main pathogenic skin strains, demonstrating their sanitizing power. Rheology, on the other hand, is used to study the behavior of viscoelastic parameters when the formulations are subjected to different stress conditions.
The integration of the data obtained could represent a baseline point for a preliminary screening of hand sanitizers efficacy and stability, representing a rapid way to guarantee the safety and the satisfaction of the final consumer. At the same time, it could represent a valid starting point for further investigations that could complete the full characterization of similar formulas.

2. Results and Discussion

2.1. Organoleptic Characterization

For a better characterization of the formulations object of the study, a chemical and physical evaluation of these properties that are able to influence customer satisfaction, formulation stability, and efficacy was performed. In order to show the different physical state that determines the appearance of the three formulations, a representative image is shown in Figure 1. The results obtained for appearance, smell, and color (Table 1) are qualitative evaluations of the experimenters involved in the study, while for the pH assessment, data obtained from the mean with the relative standard deviation (SD) of three sample measurements are shown.

2.2. Amplitude Sweep Test

Data obtained with the amplitude sweep test were processed with rSpace for Kinexus software (Malvern Panalytical, Worcestershire, UK). The linear viscoelastic region (LVER) was determined with the sequence (temperature of 25 °C, shear strain 0.1–1000%, and frequency 1 Hz), and in this spectrum, G’ (elastic modulus), G’’ (viscous modulus), and tan δ (tangent of the phase angle) should be constant and run in parallel with increasing shear strain. Data collected are summarized in Table 2 and graphically represented in Figure 2. For the formulation with CBP, G’ was several times larger than G’’, defining a solid-like viscoelastic network of the gel, while for the formulation with HEC, G’ was greater than G’’, confirming a solid-like behavior, but the gap between the two moduli was minor. The crossover point (where G’ = G’’) of the HEC formulation took place at 100.06%, while for the CBP hand sanitizer, it was at approximately 63.12%.

2.3. Frequency Sweep Test

To carry out the frequency sweep test, 1% was selected as the fixed shear strain value from the preliminary amplitude sweep test. For hand sanitizers, Figure 3 shows G’ behavior, while Table 3 also includes G’’ and tan δ of both the formulation with hydroxyethylcellulose (HEC) and the formulation with carbopol (CBP), so the prevalence of G’ on G’’ was underlined for both hand sanitizers. For each product, G’ and G’’ increase with increasing frequency. Analyzing the same gel, the G’ modulus was always greater than G’’ within the investigated frequency range, and a solid-like behavior was confirmed because the tan δ value was lower than unity, so the elastic nature of hand sanitizers prevailed in the viscous nature. The formulation with carbopol presented a G’ value greater than the formulation with hydroxyethylcellulose (behaviors shown in Figure 3).

2.4. Shear Rate Ramp Test

For each hand sanitizer, the results show that shear viscosity (η) decreases as the shear rate increases, demonstrating pseudoplastic behavior for all formulations. As shown in Figure 4, at a low shear rate (0.10 s−1), when the formulation was at rest, the products had a high viscosity, which was promoted by the stable network structure formed by the enhancer viscosity excipients. Increasing the shear rate decreased the viscosity because the internal structure became destroyed and deformed, and hand sanitizers became less viscous to allow the product to be spread and improve consumer compliance. Table 4 shows the shear rate to simulate different conditions of the sanitizing gels (rest, extrusion, and application) and the viscosity value obtained for each product [10].

2.5. Evaluation of Antimicrobial Activity

Different in vitro tests were performed to better characterize the antimicrobial activity of the three formulations of the study.

2.5.1. Minimum Inhibitory Concentration (MIC) Determination

The data obtained from the evaluation of the MIC by the microtiter plate method are reported in Table 5 and shown for each formulation tested, and the first concentration was able to induce a bacteriostatic effect against the selected microbial strains. The results obtained show a comparable efficacy of the three formulations against the selected strains, except for CBP, which was more effective against Pseudomonas aeruginosa than other formulations, and HEC, which was effective at a concentration higher than the others.

2.5.2. Minimum Bactericidal Concentration (MBC) Determination

Starting from the MIC evaluation, the MBC was performed for each formulation against the reference strains to find the first concentration able to reduce the 99.9% of the viable microbial cells. The results obtained are summarized in Table 6 and confirm the MIC efficacy trend except for HEC, which shows a bactericidal concentration equal to the MIC.

2.5.3. Agar Diffusion Test for Antimicrobial Activity Evaluation

The antimicrobial activity of the hand sanitizers was also evaluated using agar disc inhibitory activity against the selected microbial strains Staphylococcus aureus (ATCC 6538), Pseudomonas aeruginosa (ATCC 9027), Staphylococcus epidermidis (ATCC 12228), and Escherichia coli (ATCC 87394). The obtained data confirm the antimicrobial activity of the three formulations tested against all strains, and for the interpretation of the results, the inhibitory zones of streptomycin were used as references [11]. The zones of inhibition of the hand sanitizers are shown in Table 7, and the efficacy of each of the three hand sanitizers was evaluated as susceptibility of the microbial strains with respect to each formulation.

2.5.4. In Vivo Efficacy on the Hands of a Cohort of Ten Volunteers

To better investigate the antimicrobial activity of the three formulations in the study, a test was conducted on ten volunteers. Table 8 shows the percentage of CFU reduction after treatment with each formulation (formulation 1—WHO, formulation with carbopol—CBP and formulation with hydroxyethylcellulose—HEC) compared to the baseline microflora of each subject involved in the study. Figure 5 shows a graphical representation of the CFU reduction in a representative subject after treatment with the three formulations with respect to the right- and left-hand microflora.

3. Conclusions

Alcohol-based hand sanitizers (ABHSs) are formulations commonly used to clean and sanitize hands in the absence of water and soap. Their application is necessary to cope with precarious sanitary conditions in areas lacking adequate water supply and to avoid the transmission of pathogens and infectious agents during the recent severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic [3]. The need for large-scale production due to their extensive use led to the development of new formulations by various companies, with the aim of benefitting and sustaining the health-care community. However, most of these products were placed on the market without information about the laboratory-based efficacy evaluation risks when the requirements of stability and efficacy are not satisfied, as they are necessary for the safety and satisfaction of the final consumer.
The disinfectant property of the formulation depends on the concentration and type of alcohol used. Although ethanol may appear to have low bactericidal activity compared to other types of alcohol, it has higher virucidal activity and better skin tolerance. The United States Food and Drug Administration (US FDA), World Health Organization (WHO), and the Centers for Disease Control and Prevention (CDC) have defined ethanol as safe and effective for disinfecting ethanol at concentrations from 60% to 95% (v/v) [5]. In this study, the experimental design proposed for the quality and efficacy assessment of two different hand sanitizers is presented. The starting point of our overview was the organoleptic definitions of the two formulations containing carbopol (CBP) and hydroxyethylcellulose (HEC) as cross-linker and stabilizer agents, compared with the standard WHO formulation 1. CBP and HEC hand sanitizers have a gel appearance with respect to the WHO-liquid formula thanks to the presence of gelling agents. All formulations present a characteristic ethanol smell and a pH between 6–7; pH plays a key role during gel polymerization, and scientific articles declare that the maximum viscosity degree is reached at a pH of approximately 6.5–7.5 [6]. As previously described, the gel matrix has numerous advantages, such as the reduction of alcohol evaporation, major deeper penetration, and better spread ability.
Rheological characterization was preliminarily carried out for each of the three formulations, but the results obtained for the WHO formulation were not compliant with each other due to the liquid consistency of the formulation and due to the high evaporation of alcohol, the main component of the formulation, caused by the lack of excipients that allowed the creation of a network able to reduce evaporation.
The amplitude sweep test should be conducted in the linear viscoelastic region (LVER), where the viscoelastic shear moduli G’ (elastic modulus) and G” (viscous modulus) of the material are independent of strain and are only related to its molecular structure. In this region, the material structure is maintained intact and the response is independent of the magnitude of the deformation. The point at which the viscoelastic modulus deviates by more than 10% from a constant (plateau) value indicates the critical strain that lead to departure from LVER [12]. Data obtained from this dynamic oscillation test show, for both formulations (CBP and HEC), a G’ greater than G’’, confirming a solid-like viscoelastic network. A difference between the two rheological behaviors is represented by the crossover point at approximately 63% for CBP and 100% for HEC, but also by the gap between G’ and G’’ that is higher for the CBP formulation than the HEC formulation. After the preliminary amplitude sweep test, a frequency sweep test was performed to confirm solid-like behavior for both formulations due to the tan δ value being lower than unity, so the elastic nature of hand sanitizers prevailed in a viscous manner.
The rheological stability was also assessed through an internal shear rate ramp test protocol to simulate different stress conditions to which the formulations can be subjected. The shear rate ramp test determines shear viscosity (η) increasing shear rates, and, based on the behavior, allows a classification of the sample. Fluids can be divided into Newtonian (increasing shear rate viscosity of the sample remains constant) and non-Newtonian fluids subdivided into pseudoplastic fluids (when shear rate increases, the viscosity decreases) and dilatant fluids (with shear rate increasing viscosity increases). Each gel was influenced by a high shear rate by increasing temperature: their viscosity decreased where the temperature increased. At a low shear rate, the temperature did not influence the viscosity of the hand sanitizers. Pseudoplastic behavior is important for a product to spread on the skin to guarantee complete and homogeneous absorption on the skin [13].
For hand sanitizers in gel formula, at low shear rate, simulating the rest in the packaging, a major viscosity is preferred in order to avoid phase separation effect, while at medium-high shear rate values, low viscosity allows for achieving a good level of application and spreading of the hand sanitizer on the human skin in order to guarantee the complete skin covering [14]. The correct viscosity is necessary to allow the spill from the dispenser at a suitable dose. The rheological stability of these formulations is closely linked to their antimicrobial activity, and the lack of bactericidal activity is often due to prolonged storage, which can lead to increased temperature, causing alcohol evaporation and a decrease in effectiveness [15].
After verifying the stability of the formulations object of the study, the evaluation of their antimicrobial efficacy through a panel of microbiological tests was performed. The results show that the growth of selected microbial strains (Staphylococcus aureus ATCC 6538, Pseudomonas aeruginosa ATCC 9027, Staphylococcus epidermidis ATCC 12228, and Escherichia coli ATCC 87394) is sensitive to all formulations and decreases with increasing concentrations of each hand sanitizer. This bacteriostatic/bactericidal activity is due to the presence of alcohol as a major component which is able to induce microbial protein denaturation, dissipation of membrane lipids, and consequently disaggregation of membranes [16]. The minimum inhibitory concentration (MIC) investigation demonstrates that Pseudomonas aeruginosa is the strain more sensitive to all formulations and, in particular, to the formulation with carbopol (CBP), which shows efficacy at the lower concentration (3.12%). Together with bacteriostatic activity, a bactericidal effect was also found for all hand sanitizers during the in vitro trials, and the trend of the results confirms that of the MICs, with a better minimum bactericidal concentration (MBC) found for the formulation with carbopol (CBP) against Pseudomonas aeruginosa (6.25%). Several studies highlight that the effectiveness of alcohol-based hand sanitizers (ABHSs) depends on alcohol concentration, and the final formula with an alcohol content between 60 and 85% is reported to reduce microorganism viability to 99.99% [17]. The sensitivity of the selected strains to the hand sanitizers was confirmed with the agar diffusion test performed according to the Kirby–Bauer method; all strains demonstrated intermediate susceptibility (I, 11 ≥ Ø ≤ 14), except for Pseudomonas aeruginosa and Staphylococcus epidermidis, which were found to be resistant to the WHO formulation (R, Ø ≤ 10), and Pseudomonas aeruginosa, which was sensitive (S, Ø ≥ 15) to the CBP formulation, confirming the previous data and the better efficacy of this formulation compared to the other two.
To better investigate the antimicrobial assessment of hand sanitizers, an in vivo study on ten volunteers was performed, and the results showed comparable efficacy with a microbial reduction of approximately 80% for each of the three formulations. These last data demonstrate an excellent ability to reduce the bacterial load but did not reach a reduction of 99.99%, and none of the three products can be defined as bactericidal in vivo.
In recent years, several studies have been published on the efficacy on the rheological characterization of alcohol-based hand sanitizers, with the final aim of optimizing the formulas by providing greater handleability and hand feel [18,19]. Similarly, this type of formula has been the subject of various efficacy studies to prove the antimicrobial action against the most common pathogenic microflora. The published data on various commercial formulas were comparable with the data obtained in this study for formulas containing an equal ethanol content [20,21]. However, rheological and microbiological tools have not been integrated in order to investigate the stability and efficacy of ethanol-based hand sanitizers, with the ultimate aim of performing a control quality assessment of these formulations.
In conclusion, our overview provides an innovative panel of in vitro and in vivo tests that allow quality control of hand sanitizers in a short time through a good balance between stability, effectiveness, and consequently safety and satisfaction of the final consumer.

4. Materials and Methods

4.1. Sample Collection

Two different alcohol-based hand sanitizers (ABHSs) were provided by LEM Compounding Research Srl (Lentante sul Seveso, MB, Italy), while the World Health Organization (WHO) formulation 1 was prepared following good manufacturing practices (GMPs) by mixing different ingredients according to the reference standard indications [4,22]. Ingredients for each of the final formulations are described in Table 9.

4.2. Organoleptic Properties Evaluation

For a better characterization of the formulations object of the study, a starter assessment of their physical and chemical properties was performed. Appearance, smell, color, and pH are the features investigated as influencing factors for daily use, for its degree of effectiveness, and for the satisfaction of the final customer. Appearance, smell, and odor are physical features defined visually by experimenters involved in the study. Measurement of the pH was performed with a pH meter (1100 L-PH-001, VWR, Milan, Italy) after calibration at 25 °C with two buffer solutions (4.01 and 7).

4.3. Amplitude Sweep Test

Hand sanitizer rheological characterization was carried out by Rheometer Kinexus Plus (Malvern Panalytical, Worcestershire, UK), and data processing was performed by rSpace for Kinexus software (Malvern Panalytical, Worcestershire, UK). Measurements were made in triplicate, and fresh samples were loaded. Amplitude sweep tests are usually performed to define the linear viscoelastic region (LVER), where the sample is subjected to a different deformation intensity without breaking its microscopic structure. The LVER is characterized by the linearity of G’ (elastic modulus), G’’ (viscous modulus), and tan δ (tangent phase angle) increasing the shear strain. The rheometer was equipped with cone-plate geometry (CP4/40 SR5476 SS), and analyses were performed under isothermal conditions (25 °C) with a working gap of 0.15 mm. The amplitude sweep sequence was set up at shear strain between 0.1 and 1000% and frequency 1 Hz. An amplitude sweep test was performed as a preliminary test to identify the constant strain value at which to set the frequency sweep test.

4.4. Frequency Sweep Test

The frequency was 0.15 mm of the working gap using a cone-plate geometry (CP4/40 SR5476 SS) within the range of 0.1–10 Hz at a shear strain of 1% to avoid changing the microscopic structure of the hand sanitizers remaining in the LVER. By the frequency sweep test, G’, G’’, and tan δ were determined. Tan δ is the result obtained by the ratio G’’/G’, so if the result is more than one, the elastic component is higher than the viscous component; if it is greater than one, the viscous component prevails; in case the ratio is one, the two parameters are equal [23]. The measurements were made in triplicate using fresh samples each time, and data were processed by rSpace for Kinexus software. Data were obtained at 25 °C to study the stability of the formulations at rest or subjected to external frequency during storage.

4.5. Shear Rate Ramp Test

The shear rate ramp test was performed with a Kinexus Plus rotational rheometer and processing by rSpace for Kinexus software to investigate the steady shear flow behavior of hand sanitizers at different temperatures (25, 37, and 45 °C) to simulate storage at room temperature and body temperature during application and storage at extreme temperatures. For each sample, 5 min were requested to heat the cartridge and the hand sanitizer. The broad range of shear rates values (0.1–1000 s−1 logarithmic scale) was evaluated in order to simulate rest, extrusion, and application of the product [10].

4.6. In Vitro Assessment of the Antimicrobial Activity

The antimicrobial activity of the alcohol-based hand sanitizers was evaluated against the most reliable strains of the skin microflora: Staphylococcus aureus (ATCC 6538), Pseudomonas aeruginosa (ATCC 9027), Staphylococcus epidermidis (ATCC 12228), and Escherichia coli (ATCC 87394). All strains were purchased by Mecconti S.A.R.L. Sp.z.o.o. (Warszawa, Poland). Bacterial strains were incubated for 24 at 35 ± 2 °C on Tryptic Soy Agar (TSA, Condalab, Madrid, Spain), and after the following subcultures, the third was used to prepare an inoculum in Tryptic Soy Broth (TSB, Condalab, Madrid, Spain) in order to obtain a calibrated culture for the efficacy tests assessment.

4.6.1. Determination of Minimum Inhibitory Concentration (MIC)

MIC is the lowest concentration of a specific antimicrobial needed to prevent the growth of a given antimicrobial substance in vitro (bacteriostatic action). MIC evaluation was performed by using a 96-well microtiter plate method [24]. Serial dilutions of the hand sanitizers in a broth medium suitable for the microbial growth of the selected strains (TSB) were set up according to the following ratio: diluted product 1:2 and subsequent dilutions (range tested from 50% to 0.78%) with respect to the control (broth medium in absence of hand sanitizer). For each dilution, 180 µL was dispensed in duplicate, and 20 µL of the calibrated inoculum (106 CFU/mL final concentration) was added to each well. At the same time, the selected concentrations of the sanitizer diluted in broth without bacteria were dispensed in duplicate and used as background for subsequent optical density (OD) correction readings. The plates were then incubated for 24 h at 37 °C under optimal growth conditions for the selected microbial strains. The same protocol was followed for all selected microbial strains. After 24 h of incubation, spectrophotometric reading was performed at a wavelength of 595 nm using a microplate reader (MultiskanTM Go, Thermo Scientific, Waltham, MA, USA). The value of the corresponding dilution of the product in the absence of bacteria (background) was subtracted from each reading for each concentration tested. Thus, the delta of absorbance was calculated, which measures the OD represented only by the bacterial load present in the well, which is directly proportional to the cells’ viable number.

4.6.2. Determination of Minimum Bactericidal Concentration (MBC)

MBC is represented by the lowest concentration of the product able to reduce 99.9% of cells of a given bacterial strain [17]. Its determination is subsequent to the MIC performance and is based on the preparation of subcultures in agar medium starting from the concentration of the product, which showed no visible growth (MIC). The absence of colonies in subcultures indicates a bactericidal effect. Once the MIC was determined in a 96-well plate, 10 µL of the control and the three increasing concentrations (starting from the MIC) were plated in duplicate on TSA plates incubated for 24 h at 37 °C, the optimal growth conditions for the microbial selected strains. After 24 h of incubation, the qualitative evaluation of presence/absence growth on the plate was performed to register for the hand sanitizer concentration for each strain that was able to induce absence of growth, and thus a bactericidal effect (MBC). The same protocol was followed for all selected microbial strains.

4.6.3. Determination of the Inhibition Zone by Agar Diffusion Test (Kirby–Bauer Method)

An agar diffusion test was performed according to the Kirby–Bauer method to evaluate the test organism susceptibility to the sand sanitizer object of the study. A stock culture in broth medium (TSB) was suitably prepared and incubated for 24 h at 37 °C under optimal growth conditions for each of the selected strains [25]. At the end of the incubation period, a calibrated suspension of 107 CFU/mL was prepared. To obtain a confluent culture, 100 µL of the calibrated inoculum of each strain was inoculated in duplicate on TSA plates. Each plate was divided into three sectors, and a disk of sterile absorbent paper was applied to each of them, containing 100 µL of each condition to be tested [22]. To avoid evaporation and ensure proper diffusion of the product, the plates were left for 3 h at room temperature (RT) and then incubated for 24 h at 37 °C. After 24 h of incubation, the diameters of the inhibition zones of each tested condition were measured and recorded. The diameter of the inhibition zones in millimeters is related to the sensitivity of the bacterium, which is considered sensitive (S), intermediate (I), or resistant (R) according to the criteria established by the guidelines of the Clinical and Laboratory Standards Institute (CLSI) [26]. The results obtained were compared with the standard inhibition parameters of streptomycin, a broad-spectrum antibiotic effective against both gram-positive and gram-negative bacteria [11].

4.6.4. In Vivo Efficacy of Hand Sanitizers

The antimicrobial activity of alcohol-based hand sanitizers was evaluated in vivo in a cohort of ten volunteers aged between 20 and 40 years. The experiments were performed in accordance with relevant named guidelines and regulations and each volunteer was previously informed on the nature and aim of the study. He/she gave a written consent statement and declared not to follow any medical treatment at the time of the study. At the beginning of the trial, the volunteers put their right and left unwashed hands on two TSA plates, pressing and rolling fingers, respectively. TSA agar plates were serially numbered and incubated at 37 °C for 24 h. After incubation, the number of viable colonies enumerated was registered for each plate as baseline microflora for each subject. After the unwashed finger impression, 1 mL of one of the three hand sanitizers was applied to the hands of the subjects. After a homogeneous distribution for at least 20–30 s and complete drying, the finger impression was repeated on two TSA plates (for right and left hand respectively) that were subsequently incubated at 37 °C for 24 h. The following day, the number of viable colonies was enumerated and registered to calculate, for each subject, the efficacy of the hand sanitizer as the ability to reduce the baseline microflora for the right and left hands of each volunteer. The same protocol was followed for each hand sanitizer at a distance of least three hours to obtain a good starter baseline microflora for each trial [25].

Author Contributions

Conceptualization, S.S., C.M. and R.M.; methodology, S.S. and M.C.C.; formal analysis, M.C.C.; investigation, M.C.C.; resources, M.C.C.; writing—original draft preparation, M.C.C.; writing—review and editing, S.S. and R.M.; visualization, R.R., E.D., M.C. and M.H.; supervision, N.Z. and R.M.; project administration, N.Z. and R.M. All authors have read and agreed to the published version of the manuscript.

Funding

This research was funded by Lem Compounding Research S.r.l. (MB, Italy).

Institutional Review Board Statement

Ethical review and approval were waived for this study, since the studies were conducted to evaluate the hedonistic aspects, the compliance and the acceptability of the cosmetic products by volunteers in comparison with a product suggested by WHO. No invasive techniques were applied.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study. Written informed consent has been obtained from the patient(s) to publish this paper.

Data Availability Statement

Data are included in the text; raw data are available from the corresponding author upon request.

Conflicts of Interest

M.C. is the CEO of Lem Compounding Research S.r.l. The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.

References

  1. Ishma, T.; Sayeed Uddin, H.M.; Paul, A.; Feroz, F.; Acharjee, M. Inhibitory effects of different hand sanitizers against the resident microflora of skin. Int. J. Sci. Rep. 2019, 5, 355–360. [Google Scholar] [CrossRef]
  2. First Global Patient Safety Challenge Clean Care is Safer Care; World Health Organization: Geneva, Switzerland, 2009.
  3. Manaye, G.; Muleta, D.; Henok, A.; Asres, A.; Mamo, Y.; Feyissa, D.; Ejeta, F.; Niguse, W. Evaluation of the Efficacy of Alcohol-Based Hand Sanitizers Sold in Southwest Ethiopia. Infect. Drug Resist. 2021, 14, 547–554. [Google Scholar] [CrossRef] [PubMed]
  4. Guide to Local Production: WHO-Recommended Handrub Formulations. Available online: https://apps.who.int/iris/bitstream/handle/10665/332005/WHO-IER-PSP-2010.5-eng.pdf?sequence=1&isAllowed=y (accessed on 25 June 2021).
  5. Villa, C.; Russo, E. Hydrogels in Hand Sanitizers. Materials 2021, 14, 1577. [Google Scholar] [CrossRef] [PubMed]
  6. Berardi, A.; Perinelli, D.R.; Merchant, H.A.; Bisharat, L.; Basheti, I.A.; Bonacucina, G.; Cespi, M.; Palmieri, G.F. Hand sanitizers amid COVID-19: A critical review of alcohol-based products on the market and formulation approaches to respond to increasing demand. Int. J. Pharm. 2020, 584, 119431. [Google Scholar] [CrossRef] [PubMed]
  7. Fallica, F.; Leonardi, C.; Toscano, V.; Santonocito, D.; Leonardi, P.; Puglia, C. Assessment of Alcohol-Based Hand Sanitizers for Long-Term Use, Formulated with Addition of Natural Ingredients in Comparison to WHO Formulation 1. Pharmaceutics 2021, 13, 571. [Google Scholar] [CrossRef] [PubMed]
  8. Hamed, R.; Al Baraghthi, T.; Alkilani, A.Z. Correlation Between Rheological Properties and In Vitro Drug Release from Penetration Enhancer-Loaded Carbopol® Gels. J. Pharm. Innov. 2016, 11, 339–351. [Google Scholar] [CrossRef]
  9. Brown, A.F.; Jones, D.S.; Woolfson, A.D. The effect of alcoholic solvents on the rheological properties of gels composed of cellulose derivatives. J. Pharm. Pharmacol. 1998, 50, 157. [Google Scholar] [CrossRef]
  10. Laba, D. The flow of Cosmetics and Toiletries. In Rheological Properties of Cosmetics and Toiletries, 2nd ed.; Marcel Dekker: New York, NY, USA, 1993; pp. 1–7. [Google Scholar]
  11. Hayat, A.; Munnawar, F. Antibacterial effectiveness of commercially available hand sanitizers. Int. J. Biol. Biotech. 2016, 13, 427–431. [Google Scholar]
  12. Hamed, R.; Basil, M.; AlBaraghthi, T.; Sunoqrot, S.; Tarawneh, O. Nanoemulsion-based gel formulation of diclofenac diethylamine: Design, optimization, rheological behavior and in vitro diffusion studies. Pharm. Dev. Technol. 2016, 21, 980–989. [Google Scholar] [CrossRef] [PubMed]
  13. Kwak, M.; Ahn, H.; Song, K. Rheological investigation of body cream and body lotion in actual application conditions. Korea-Aust. Rheol. J. 2015, 27, 241–251. [Google Scholar] [CrossRef]
  14. Censi, R.; Peregrina, D.V.; Lacava, G.; Agas, D.; Lupidi, G.; Sabbieti, M.G.; Di Martino, P. Cosmetic formulation based on an Açai extract. Cosmetics 2018, 5, 48. [Google Scholar] [CrossRef]
  15. Liu, P.; Yuen, Y.; Hsiao, H.M.; Jaykus, L.A.; Moe, C. Effectiveness of liquid soap and hand sanitizer against Norwalk virus on contaminated hands. Appl. Environm. Microbial. 2010, 76, 394–399. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  16. Kar, A. Pharmaceutical Microbiology; New Age International (P) Ltd. Publishers: Delhi, India, 2008; pp. 216–217. [Google Scholar]
  17. Reynolds, S.A.; Levy, F.; Walker, E.S. Hand Sanitizer Alert. Emerg. Infect. Dis. 2006, 12, 527–529. [Google Scholar]
  18. Silva, A.F.; Wood, T.A.; Hodgson, D.J.M. Rheological design of thickened alcohol-based hand rubs. Rheol Acta 2022, 61, 571–581. [Google Scholar] [CrossRef] [PubMed]
  19. Berardi, A.; Perinelli, D.R.; Bisharat, L.; Sabbatini, B.; Bonacucina, G.; Tiboni, M.; Palmieri, G.F.; Cespi, M. Factors affecting the rheological behaviour of carbomer dispersions in hydroalcoholic medium: Towards the optimization of hand sanitizer gel formulations. Int. J. Pharm. 2022, 616, 121503. [Google Scholar] [CrossRef] [PubMed]
  20. Jain, V.M.; Karibasappa, G.N.; Dodamani, A.S.; Prashanth, V.K.; Mali, G.V. Comparative assessment of antimicrobial efficacy of different hand sanitizers: An in vitro study. Dent. Res. J. 2016, 13, 424–431. [Google Scholar] [CrossRef] [PubMed]
  21. Chojnacki, M.; Dobrotka, C.; Osborn, R.; Johnson, W.; Young, M.; Meyer, B.; Laskey, E.; Wozniak, R.A.F.; Dewhurst, S.; Dunman, P.M. Evaluating the Antimicrobial Properties of Commercial Hand Sanitizers. mSphere 2021, 6, e00062-21. [Google Scholar] [CrossRef] [PubMed]
  22. Guide to Local Production: WHO-recommended Handrub Formulations; World Health Organization: Geneva, Switzerland, 2010; pp. 1–9.
  23. Moravkova, T.; Stern, P. Rheological and textural properties of cosmetic emulsions. Appl. Rheol 2011, 21, 35200. [Google Scholar]
  24. Otokunefor, K.; Princewill, I. Evaluation of antibacterial activity of hand sanitizers—An in vitro study. J. Appl. Sci. Environ. Manag. 2017, 21, 1276–1280. [Google Scholar] [CrossRef]
  25. Oke, M.A.; Bello, A.B.; Odebisi, M.B.; Ahmed El-Imam, A.M.; Kazeem, M.O. Evaluation of antibacterial efficacy of some alcohol-based hand sanitizers sold in Ilorin (North-Central Nigeria). Ife J. Sci. 2013, 15, 111–117. [Google Scholar]
  26. Clinical and Laboratory Standard Institute (CLSI). Methods for Dilution Antimicrobial Susceptibility Testing for Bacteria that Growth Aerobically, 9th ed.; Clinical and Laboratory Standard Institute: Wayne, PA, USA, 2012. [Google Scholar]
Figure 1. Graphical representation of the appearance of the three formulations object of the study: (A) formulation 1 (WHO); (B) formulation with carbopol (CBP); (C) formulation with hydroxyethylcellulose (HEC) provided by LEM Compounding Research S.r.l.
Figure 1. Graphical representation of the appearance of the three formulations object of the study: (A) formulation 1 (WHO); (B) formulation with carbopol (CBP); (C) formulation with hydroxyethylcellulose (HEC) provided by LEM Compounding Research S.r.l.
Gels 09 00108 g001
Figure 2. Graphical representation of the results obtained from the amplitude sweep test of formulation with hydroxyethylcellulose (HEC) and formulation with carbopol (CBP) at a temperature of 25 °C and at a frequency value of 1 Hz.
Figure 2. Graphical representation of the results obtained from the amplitude sweep test of formulation with hydroxyethylcellulose (HEC) and formulation with carbopol (CBP) at a temperature of 25 °C and at a frequency value of 1 Hz.
Gels 09 00108 g002
Figure 3. Graphical representation of frequency sweep G’ obtained with the formulation with hydroxyethylcellulose (HEC) and formulation with carbopol (CBP) at a temperature of 25 °C and at a shear strain value of 1%.
Figure 3. Graphical representation of frequency sweep G’ obtained with the formulation with hydroxyethylcellulose (HEC) and formulation with carbopol (CBP) at a temperature of 25 °C and at a shear strain value of 1%.
Gels 09 00108 g003
Figure 4. Graphical representation of the viscosity trend of the formulation with hydroxyethylcellulose (HEC) and formulation with carbopol (CBP) at temperatures of 25, 37, and 45 °C with increasing shear rate (s−1).
Figure 4. Graphical representation of the viscosity trend of the formulation with hydroxyethylcellulose (HEC) and formulation with carbopol (CBP) at temperatures of 25, 37, and 45 °C with increasing shear rate (s−1).
Gels 09 00108 g004
Figure 5. Graphical representation of TSA plates after finger impression in unwashed hands condition (Baseline Microflora) and after clean with the three hand sanitizers object of the study (Microflora post-sanitization): (A) formulation with hydroxyethylcellulose (HEC); (B) formulation with carbopol (CBP); (C) formulation 1—WHO. The picture shows the results obtained for each hand sanitizer on the right and left hand of a subject selected randomly.
Figure 5. Graphical representation of TSA plates after finger impression in unwashed hands condition (Baseline Microflora) and after clean with the three hand sanitizers object of the study (Microflora post-sanitization): (A) formulation with hydroxyethylcellulose (HEC); (B) formulation with carbopol (CBP); (C) formulation 1—WHO. The picture shows the results obtained for each hand sanitizer on the right and left hand of a subject selected randomly.
Gels 09 00108 g005
Table 1. Organoleptic properties and pH of the three formulations object of the study (formulation 1 (WHO); formulation with carbopol (CBP) and formulation with hydroxyethylcellulose (HEC). Values of pH are the results of three independent measurements with relative SD.
Table 1. Organoleptic properties and pH of the three formulations object of the study (formulation 1 (WHO); formulation with carbopol (CBP) and formulation with hydroxyethylcellulose (HEC). Values of pH are the results of three independent measurements with relative SD.
ProductsAppearanceSmellColorpH
WHOLiquidCharacteristic of ethanolTransparent6.77 ± 0.16
CBPgelCharacteristic of ethanolTransparent6.85 ± 0.01
HECgelCharacteristic of ethanolTransparent7.82 ± 0.19
Table 2. Amplitude sweep test data of formulation with hydroxyethylcellulose (HEC) and formulation with carbopol (CBP) at a temperature of 25 °C and at a shear strain value of 1%. The results are presented as averages and relative standard deviations in percentage (RSD %).
Table 2. Amplitude sweep test data of formulation with hydroxyethylcellulose (HEC) and formulation with carbopol (CBP) at a temperature of 25 °C and at a shear strain value of 1%. The results are presented as averages and relative standard deviations in percentage (RSD %).
ProductG’ (Pa)G’’ (Pa)tan δ
HEC17.42 ± 1.8811.86 ± 3.790.68 ± 3.47
CBP193.20 ± 0.7727.82 ± 2.250.14 ± 2.54
Table 3. Frequency sweep test data obtained for the formulation with hydroxyethylcellulose (HEC) and for the formulation with carbopol (CBP) at a temperature of 25 °C and at a frequency value of 1 Hz. The results are presented as averages and relative standard deviations in percentage (RSD %).
Table 3. Frequency sweep test data obtained for the formulation with hydroxyethylcellulose (HEC) and for the formulation with carbopol (CBP) at a temperature of 25 °C and at a frequency value of 1 Hz. The results are presented as averages and relative standard deviations in percentage (RSD %).
ProductG’ (Pa)G’’ (Pa)tan δ
HEC21.44 ± 6.6113.90 ± 3.160.65 ± 3.57
CBP188.47 ± 1.1427.93 ± 1.272.3 ± 1.46
Table 4. Shear rate ramp test’s results (Pa × s) obtained for the formulation with hydroxyethylcellulose (HEC) and for the formulation with carbopol (CBP) at temperatures of 25, 37, and 45 °C and at different shear rates to simulate different conditions. The results are presented as averages and relative standard deviation percentages (RSD%).
Table 4. Shear rate ramp test’s results (Pa × s) obtained for the formulation with hydroxyethylcellulose (HEC) and for the formulation with carbopol (CBP) at temperatures of 25, 37, and 45 °C and at different shear rates to simulate different conditions. The results are presented as averages and relative standard deviation percentages (RSD%).
Condition of the ProductShear Rate (s−1)HEC η 5 °CCBP η 5 °CHEC η 7 °CCBP η 7 °CHEC η 5 °CCBP η 45 °C
Rest10−280.67 ± 1.25332.70 ± 5.02 63.75 ± 2.47231.43 ± 3.4461.34 ± 5.47302.67 ± 6.20
Extrusion1030.14 ± 3.020.09 ± 1.910.11 ± 1.850.29 ± 3.460.08 ± 5.640.09 ± 7.15
Application 1020.55 ± 2.181.39 ± 3.090.41 ± 1.991.18 ± 3.510.30 ± 5.641.03 ± 6.11
Table 5. MIC obtained for the formulation with hydroxyethylcellulose (HEC), for the formulation with carbopol (CBP) and for the formulation 1—WHO against the selected strains Staphylococcus aureus (ATCC 6538), Pseudomonas aeruginosa (ATCC 9027), Staphylococcus epidermidis (ATCC 12228), and Escherichia coli (ATCC 87394) (n = 2; r = 2).
Table 5. MIC obtained for the formulation with hydroxyethylcellulose (HEC), for the formulation with carbopol (CBP) and for the formulation 1—WHO against the selected strains Staphylococcus aureus (ATCC 6538), Pseudomonas aeruginosa (ATCC 9027), Staphylococcus epidermidis (ATCC 12228), and Escherichia coli (ATCC 87394) (n = 2; r = 2).
ProductS. aureusP. aeruginosaS. epidermidisE. coli
WHO12.5%6.25%12.5%12.5%
CBP12.5%3.12%12.5%12.5%
HEC12.5%6.25%25%12.5%
Table 6. MBC obtained for the formulation with hydroxyethylcellulose (HEC), for the formulation with Carbopol (CBP), and for formulation 1—WHO against the selected strains Staphylococcus aureus (ATCC 6538), Pseudomonas aeruginosa (ATCC 9027), Staphylococcus epidermidis (ATCC 12228), and Escherichia coli (ATCC 87394) (n = 2; r = 2).
Table 6. MBC obtained for the formulation with hydroxyethylcellulose (HEC), for the formulation with Carbopol (CBP), and for formulation 1—WHO against the selected strains Staphylococcus aureus (ATCC 6538), Pseudomonas aeruginosa (ATCC 9027), Staphylococcus epidermidis (ATCC 12228), and Escherichia coli (ATCC 87394) (n = 2; r = 2).
ProductS. aureusP. aeruginosaS. epidermidisE. coli
WHO25%12.5%25%25%
CBP25%6.25%25%25%
HEC25%12.5%25%25%
Table 7. Agar diffusion test for the antimicrobial evaluation of the formulation with hydroxyethylcellulose (HEC), the formulation with carbopol (CBP), and for formulation 1—WHO against the selected strains Staphylococcus aureus (ATCC 6538), Pseudomonas aeruginosa (ATCC 9027), Staphylococcus epidermidis (ATCC 12228) and Escherichia coli (ATCC 87394). Data are presented as average and standard deviation (SD) of the diameters (Ø) measured (n = 2; r = 2). Ø ≤ 10 Resistant [R]; 11 ≥ Ø ≤ 14 Intermediate [I]; Ø ≥ 15 Sensitive [S].
Table 7. Agar diffusion test for the antimicrobial evaluation of the formulation with hydroxyethylcellulose (HEC), the formulation with carbopol (CBP), and for formulation 1—WHO against the selected strains Staphylococcus aureus (ATCC 6538), Pseudomonas aeruginosa (ATCC 9027), Staphylococcus epidermidis (ATCC 12228) and Escherichia coli (ATCC 87394). Data are presented as average and standard deviation (SD) of the diameters (Ø) measured (n = 2; r = 2). Ø ≤ 10 Resistant [R]; 11 ≥ Ø ≤ 14 Intermediate [I]; Ø ≥ 15 Sensitive [S].
ProductS. aureus
Ø (mm)
P. aeruginosa
Ø (mm)
S. epidermidis
Ø (mm)
E. coli
Ø (mm)
WHO11 ± 1.4 [I]10.5 ± 3.5 [R]10 ± 0.0 [R]12.5 ± 3.5 [I]
CBP14 ± 0.0 [I]15 ± 0.0 [S]11 ± 0.0 [I]11.5 ± 0.7 [I]
HEC13 ± 1.4 [I]12.5 ± 2.1 [I]12.5 ± 0.7 [I]12.5 ± 2.1 [I]
Table 8. CFU reduction in percentage (%) post-sanitization with the three formulations with respect to the baseline microflora of ten volunteers. For each subject, the CFU reduction (%) after treatment with the different hand sanitizers and the mean ± SD obtained from the data collected of the right and left hand of each volunteer for each formulation are reported.
Table 8. CFU reduction in percentage (%) post-sanitization with the three formulations with respect to the baseline microflora of ten volunteers. For each subject, the CFU reduction (%) after treatment with the different hand sanitizers and the mean ± SD obtained from the data collected of the right and left hand of each volunteer for each formulation are reported.
VolunteersHEC
CFU Reduction (%)
CBP
CFU Reduction (%)
WHO
CFU Reduction (%)
RXLXRXLXRXLX
V191.7681.6986.4066.6786.9698.04
V255.8489.7459.3297.1050.0651.85
V350.0085.5068.0966.6792.6794.75
V493.6797.8393.7595.3899.3276.62
V578.0089.3375.7035.8298.0093.00
V698.4498.0297.4999.5999.0099.50
V796.4399.0098.6518.4694.5595.83
V875.2075.3393.5358.4389.8339.39
V997.0090.0099.6198.7050.0059.17
V1087.4383.0599.0097.5499.6795.00
mean ± SD85.66 ± 13.5780.29 ± 23.5483.16 ± 20.54
Table 9. Formulations collected for the study design with relative composition. Formulation 1—WHO; Formulation with Carbopol—CBP; Formulation with Hydroxyethylcellulole—HEC.
Table 9. Formulations collected for the study design with relative composition. Formulation 1—WHO; Formulation with Carbopol—CBP; Formulation with Hydroxyethylcellulole—HEC.
FormulationIngredients
WHOEthanol, Hydrogen Peroxide, Glycerol, Aqua
CBPAqua, Alcohol Denat, Butylene Glycol, Acrylates/c10-30 Alkyl Acrylate Crosspolymer, Xanthan Gum, Aminomethyl Propanol
HECAqua, Alcohol Denat, Hydroxyethylcellulose
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Sommatis, S.; Capillo, M.C.; Maccario, C.; Rauso, R.; D’Este, E.; Herrera, M.; Castiglioni, M.; Mocchi, R.; Zerbinati, N. Antimicrobial Efficacy Assessment and Rheological Investigation of Two Different Hand Sanitizers Compared with the Standard Reference WHO Formulation 1. Gels 2023, 9, 108. https://doi.org/10.3390/gels9020108

AMA Style

Sommatis S, Capillo MC, Maccario C, Rauso R, D’Este E, Herrera M, Castiglioni M, Mocchi R, Zerbinati N. Antimicrobial Efficacy Assessment and Rheological Investigation of Two Different Hand Sanitizers Compared with the Standard Reference WHO Formulation 1. Gels. 2023; 9(2):108. https://doi.org/10.3390/gels9020108

Chicago/Turabian Style

Sommatis, Sabrina, Maria Chiara Capillo, Cristina Maccario, Raffaele Rauso, Edoardo D’Este, Martha Herrera, Mauro Castiglioni, Roberto Mocchi, and Nicola Zerbinati. 2023. "Antimicrobial Efficacy Assessment and Rheological Investigation of Two Different Hand Sanitizers Compared with the Standard Reference WHO Formulation 1" Gels 9, no. 2: 108. https://doi.org/10.3390/gels9020108

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop