Next Article in Journal
Targeting KRAS in Pancreatic Cancer
Previous Article in Journal
The Development of an Infrastructure to Facilitate the Use of Whole Genome Sequencing for Population Health
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
JPM
Volume 12
Issue 11
10.3390/jpm12111869
jpm-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Cytochromes P450 and P-Glycoprotein Phenotypic Assessment to Optimize Psychotropic Pharmacotherapy: A Retrospective Analysis of Four Years of Practice in Psychiatry

by
Clément Delage
1,2,3,*,
Léa Darnaud
4,
Bruno Etain
2,3,5,
Marina Vignes
1,
Tu-Ky Ly
1,
Alexia Frapsauce
1,
Marc Veyrier
1,
Marine Delavest
5,
Emeline Marlinge
5,
Vincent Hennion
3,5,
Manon Meyrel
3,5,
Aude Jacob
1,2,3,
Margot Chouchana
1,
Julie Smati
1,
Guillaume Pataud
5,
Nihel Khoudour
4,
Jean-Eudes Fontan
1,
Laurence Labat
2,3,6,
Frank Bellivier
2,3,5,
Célia Lloret-Linares
7,
Xavier Declèves
2,3,4 and
Vanessa Bloch
1,2,3add Show full author list remove Hide full author list
1
Service Pharmacie, AP-HP, Hôpital Lariboisière Fernand Widal, F-75010 Paris, France
2
Université Paris Cité, Inserm UMRS-1144, Optimisation Thérapeutique en Neuropsychopharmacologie, F-75006 Paris, France
3
Université Paris Cité, F-75006 Paris, France
4
Service Laboratoire de Biologie du Médicament et Toxicologie, AP-HP, Hôpital Cochin, F-75014 Paris, France
5
Département de Psychiatrie et Médecine Addictologique, AP-HP, Hôpital Lariboisière Fernand Widal, F-75010 Paris, France
6
Service Laboratoire de Toxicologie, AP-HP, Hôpital Lariboisière Fernand Widal, F-75010 Paris, France
7
Ramsay Santé-Hôpital Privé Pays de Savoie, F-74100 Annemasse, France
*
Author to whom correspondence should be addressed.
J. Pers. Med. 2022, 12(11), 1869; https://doi.org/10.3390/jpm12111869
Submission received: 27 September 2022 / Revised: 24 October 2022 / Accepted: 3 November 2022 / Published: 8 November 2022
(This article belongs to the Section Personalized Therapy and Drug Delivery)
Browse Figures
Versions Notes

Abstract

:
Altered cytochromes P450 enzymes (CYP) and P-glycoprotein transporter (P-gp) activity may explain variabilities in drug response. In this study, we analyzed four years of phenotypic assessments of CYP/P-gp activities to optimize pharmacotherapy in psychiatry. A low-dose probe cocktail was administered to evaluate CYP1A2, 2B6, 2D6, 2C9, 2C19, 3A4, and P-gp activities using the probe/metabolite concentration ratio in blood or the AUC. A therapeutic adjustment was suggested depending on the phenotyping results. From January 2017 to June 2021, we performed 32 phenotypings, 10 for adverse drug reaction, 6 for non-response, and 16 for both reasons. Depending on the CYP/P-gp evaluated, only 23% to 56% of patients had normal activity. Activity was decreased in up to 57% and increased in up to 60% of cases, depending on the CYP/P-gp evaluated. In 11/32 cases (34%), the therapeutic problem was attributable to the patient’s metabolic profile. In 10/32 cases (31%), phenotyping excluded the metabolic profile as the cause of the therapeutic problem. For all ten individuals for which we had follow-up information, phenotyping allowed us to clearly state or clearly exclude the metabolic profile as a possible cause of therapeutic failure. Among them, seven showed a clinical improvement after dosage adaptation, or drug or pharmacological class switching. Our study confirmed the interest of CYP and P-gp phenotyping for therapeutic optimization in psychiatry.

1. Introduction

Despite the numerous drugs available to treat mood disorders, therapeutic failures are common in psychiatry. It is estimated that only 25 to 50% of individuals respond to the first line of pharmacotherapy [1,2]. There are many possible causes of treatment-resistant depression, such as inadequate pharmacological targets, inadequate treatment dosage and/or duration, neuroprotective agent intake, inadequate diagnosis, lack of compliance, etc. [3] Among them, interindividual variability in drug plasma concentration is a possible reason for drug failure. At the same oral dose, plasma concentration can vary by a factor of 20 due to different pharmacokinetic parameters: primarily drug metabolism and elimination [4]. Thus, therapeutic drug monitoring (TDM) is recommended for most psychotropic drugs due to their narrow therapeutic margins [4]. However, when facing abnormal plasma levels or recurrent drug failures, such as non-response or adverse drug reactions, TDM has limits in identifying the origin of the therapeutic failure.
Cytochromes P450 (CYP) are the main drug-metabolizing enzymes [5]. They include many different isoforms, but only a few account for most drug metabolisms, such as CYP1A2, CYP2B6, CYP2C9, CYP2C19, CYP2D6, and CYP3A4, which are responsible for the metabolization of 90% of all drugs [6]. In addition to CYP, P-glycoprotein (P-gp) is an efflux membrane transporter located in enterocytes of the small intestine, the biliary pole of hepatocytes, the apical membrane of renal tubule epithelial cells, and in brain microvessel endothelial cells [7]. P-gp is responsible for the extrusion of molecules from the central nervous system to the blood and from the blood to the gastrointestinal lumen, bile, and urine. Thus, an increase in CYP or P-gp activity decreases the blood concentration of their substrates, which may lead to a lack of therapeutic effect. Conversely, a decrease in a CYP or P-gp activity increases the blood concentration of their substrates, which may lead to an accumulation of the drug and the potential for adverse drug reactions. This phenomenon is reversed in the case of a prodrug, which is active after metabolization. It is well known that CYP and P-gp genotypes vary substantially both between ethnic groups and within the same group [8]. Thus, these genetic polymorphisms represent a major factor in interindividual variability in drug response.
Evaluating CYP and P-gp activity by genotyping their encoding genes has proved its efficacy in predicting drug metabolization [9] and increasing the effectiveness of antidepressant therapies [10]. However, genotype does not always correlate exactly with plasma concentration [11,12], and phenoconversions (when the measured phenotype is different from the genotype) occur in between 30 and 60% of cases [10,13,14,15,16,17]. Phenoconversions may be due to different factors, among which are physiological situations [18] and environmental factors such as smoking, diet and drug intake [8]. Then, the phenotypic assessment of CYP and P-gp activity appears a more effective tool than genotyping for evaluating the metabolic profile of individuals [19]. One method of CYP and P-gp phenotypic assessment consists of a low-dose probe drug “cocktail” administration, together with the quantitative blood analysis of probes and their metabolites [20,21,22]. Some authors have described a potential clinical application of phenotyping for therapeutic optimization to anti-HIV and anti-tuberculosis drugs [23], anticoagulants [24], analgesics [17], anticancer agents [25], and psychotropic drugs [26], but the therapeutic and clinical impact of phenotyping remains largely unexplored, especially in psychiatry.
In our university-based hospital, we implemented a CYP/P-gp phenotypic assessment in the psychiatric department as a tool to optimize pharmacotherapy when physicians faced therapeutic problems, such as adverse drug reactions or non-response to psychotropic drugs. In this article, we retrospectively describe the data and outcomes of four years of phenotyping in individuals with mood disorders.

2. Materials and Methods

2.1. Patients and Phenotyping Decision

Phenotyping took place in Fernand Widal Hospital (Assistance Publique Hôpitaux de Paris (APHP), Paris, France). Physicians at in- and out-patient units facing adverse drug reactions or drug non-response for a given individual contacted the pharmacy unit to investigate the implication of the metabolic profile for this therapeutic problem. Based on a comprehensive medical and therapeutic history (past and current treatments) and previous drug-level blood tests, the pharmacist reported possible drug–drug interactions and CYP/P-gp alternative profiles that might explain this problem. This report was accompanied by recommendations with regard to therapeutic drug monitoring and/or CYP/P-gp phenotyping. A phenotyping decision was made after a multidisciplinary discussion between physicians and pharmacists, and the CYP/P-gp phenotyping to be explored was determined based on the pharmacist’s report. Individuals with documented intolerance or contraindications to one of the probe drugs were excluded from phenotyping.

2.2. Ethics

Ethical approval was obtained from the French Ethics Committee “Comité de Protection des Personnes Sud Méditerranée” and numbered as ID-RCB-2017-A00685-48. All patients signed a consent form before starting the phenotyping protocol.

2.3. Cytochrome and P-Glycoprotein Phenotyping

Phenotyping procedures took place in a psychiatric hospitalization unit or day hospital. CYP/P-gp activities were assessed using a cocktail approach previously described by Bosilkovska et al. [27,28] and called the “Geneva Cocktail”. The cocktail approach consists of administering several probe drugs, each one being metabolized by a specific CYP or substrate of the P-gp. Measuring the plasma level of both probe drugs and metabolites at a specific time allows us to determine the activity of the CYP. As a transporter, P-gp activity is assessed with probe drug plasma level only. The “Geneva Cocktail” has the advantage of containing substrates completely specific for each CYP of interest and the P-gp, and with low doses does not produce side effects. It is composed of caffeine 50 mg for CYP1A2, bupropion 150 mg for CYP2B6, omeprazole 10 mg for CYP2C19, flurbiprofen 50 mg for CYP2C9, dextromethorphan 10 mg for CYP2D6, midazolam 1 mg for CYP3A4 and fexofenadine 120 mg for P-gp. Full details of the cocktail (probe drugs, metabolites, CYP/P-gp-targeted, galenic form, dosage, and route of administration) can be found in Supplemental Data Table S1. The cocktail was adapted depending on the CYP/P-gp explored.
Individuals had to fast overnight before cocktail intake. They had to keep fasting until the first blood test and to delay their eventual morning drug intake until immediately afterwards. As CYP1A2 activity was assessed with caffeine, patients were asked to avoid caffeine intake (coffee, cola beverages, tea, etc.) for 24 h before the cocktail intake when this CYP was assessed to avoid biasing plasma measurements (Figure 1). The cocktail administration was supervised by a nurse. Blood samples were collected into a lithium heparin tube by a nurse and stored at room temperature (<25 °C) before shipment to the laboratory (<24 h). If not processed immediately, samples were stored between −15 °C and −25 °C awaiting analysis (<3 weeks).
Administered drugs and their metabolites were quantified in plasma by the toxicology laboratory in Cochin Hospital (AP-HP, Paris, France). The quantification was performed using a previously described HPLC-MS/MS method [27,29]. CYP activities were assessed by specific metabolite/probe concentration ratios (metabolic ratios (MRs)) determined 2 h after cocktail administration based on previous studies [27]. P-gp activity was assessed through the estimation of the area under the curve of the probe concentration using dosing at 3 and 6 h after the cocktail administration (AUC0–6h), as previously described [27,29]. P-gp and CYP activities, as measured by AUC0–6h and MRs, respectively, were classified as decreased, normal, or increased. CYP2D6 activity was classified as ultrarapid metabolizer (UM), extensive metabolizer (EM), intermediate metabolizer (IM), or poor metabolizer (PM). These classifications were based on ranges previously determined in a healthy population by Bosilkovksa et al. [27,29]. To ease the readability of the results, CYP2D6 UM and EM were in some cases considered as an increased phenotype, IM as a normal phenotype, and PM as a decreased phenotype.

2.4. Clinical Application

Based on the CYP/P-gp phenotype and considering current and past treatment, the pharmacist provided a therapeutic opinion on the involvement of the metabolic profile in the therapeutic problem. The report was accompanied by therapeutic advice such as drug switching or dosage adaptation. The report was delivered by the pharmacist to the physician during a dedicated appointment to discuss the different therapeutic options for the patients.

2.5. Retrospectively Analyzed Data

To assess our phenotyping activity, we retrospectively analyzed the following data:
  • Patient characteristics: age, gender, alcohol, tobacco and grapefruit consumption, and kidney and hepatic function.
  • Clinical context and therapeutic problem: adverse drug reaction with one or multiple drugs at usual doses (ADR), no therapeutic response to one or multiple drugs (NR), or adverse drug reaction for some drugs and non-response for others (ADR+NR).
  • Involved drugs: past and current drugs for which the physician questioned the implication of the metabolic profile in the therapeutic problem encountered, and the CYP/P-gp of which they were major or minor substrates [30].
  • Current treatment: drugs prescribed to the patient at the moment of phenotyping, their dosage, their possible inhibiting or inducing properties on CYP/P-gp, and their potency of action (weak or potent) [30].
  • CYP/P-gp activity: increased/UM, EM, normal/IM or decreased/PM.
  • Answer to the therapeutic problem: the metabolic profile was (Y) or was not (N) the cause of the therapeutic problem; could explain partially the therapeutic problem for some drugs only (P); or was not interpretable (NI) due to CYP/P-gp inducers or inhibitors or because too many metabolic pathways were involved in the metabolism of the drugs implicated.
  • Follow-up: therapeutic adaptations made by the physician and their clinical consequences.

3. Results

3.1. Population

Between January 2017 and June 2021, 37 phenotyping demands were received. Multidisciplinary discussions resulted in judging phenotyping unnecessary in five cases: four because of the involvement of drugs not metabolized by the CYP/P-gp, and one because of the recent and temporary introduction of a potent inducer. Thus, 32 phenotypings were performed on 31 different patients: 9 men and 22 women (one patient had two phenotypic assessments) (Table 1). Patients were aged from 25 to 75 years, with a mean age of 47.1 ± 14.5 years and median age of 49 years [35–58.5].
Nine patients were smokers and four drank an average of two or more glasses of alcohol per day. None consumed grapefruit or Hypericum.
Thirteen phenotypings (41%) were performed during hospitalization, others (n = 19; 59%) were performed in a day hospital. Phenotypings were performed in the context of depressive disorder in 15 cases (47%), bipolar disorder in 12 cases (38%), anxiety–depressive syndrome in 3 cases (9%), schizoaffective disorder in 1 case (3%), and neuropathic pain in 1 case (3%). The first of the two phenotypings performed for the same patients were in the context of depressive disorders. The second was in the context of neuropathic pain. Among the causes of phenotyping, 10 (31%) were due to ADR, 6 (19%) were due to NR, and 16 (50%) were due to ADR+NR (ADR for some drugs and NR for others) (Table 1).
Full details of individual patient characteristics can be found in the supplemental data (Table S2).

3.2. Drugs Involved

For each phenotypic assessment, the implication of the metabolic profile was questioned for an average of 7.0 ± 4.8 different drugs taken in the past or currently prescribed. Antidepressants were involved in 25 phenotypings (20 different drugs), antipsychotics in 24 (14 different drugs), anxiolytics in 12 phenotypings (7 different drugs) and antiepileptics in 12 phenotypings (9 different drugs). The molecules most involved in the phenotypings are listed in Table 2 (the full list can be found in the supplemental data—Table S3). These drugs correspond to the drugs the most prescribed in our hospital, as they are the most recommended in international guidelines.
CYP2D6 was implicated in almost all phenotypings, unlike 2B6, which was involved in only a few cases. CYP2D6 and CYP3A4 were implicated in the metabolization of the majority of the drugs questioned through phenotyping with 56.7% and 51.3% of the 224 cumulated drugs, respectively (cumulated drugs = sum of all drugs involved in each of the 32 phenotypings; a drug involved in more than one phenotyping was counted cumulatively for as many times as the phenotypings), that were substrates of at least these CYPs (Figure 2 and Supplementary Table S3).

3.3. Response to the Therapeutic Problems

Phenotyping indicated a metabolic profile compatible with under- or overdosing that might have caused the therapeutic problem encountered (Y) in 11/32 cases (34%). In 10/32 cases (31%), the phenotyping precluded the metabolic profile as a possible cause of the therapeutic problem (N). Thus, phenotyping gave a clear answer to the physician’s question (Y + N) in 21/32 cases (66%). Conversely, in 7/32 cases (22%), the metabolic profile might have been partially (P) responsible for the therapeutic problem, for some drugs only. The phenotyping was not interpretable (NI) in 4/32 cases (13%) because of the presence of CYP/P-gp inducers or inhibitors, or because too many metabolic pathways were involved in the elimination of the drugs implicated (Table 3).
When phenotyping allowed the involvement of the metabolic profile in the therapeutic problem to be identified (Y), an average of 4.5 ± 2.9 drugs were implicated in the therapeutic problem. Conversely, when the phenotyping did not bring a clear answer to the problem encountered (P + NI), an average of 8.0 ± 5.8 drugs were implicated (Table 3).

3.4. Individual CYP/P-gp Activities

Full details of individual CYP activities can be found in the Supplementary Data (Table S2). Among the 32 phenotypings, CYP/P-gp activities were normal in 23% to 56% of the cases, depending on the CYP/P-gp. CYP-P-gp activities were decreased in 0% to 57% of cases and increased in 9% to 60% of cases. (Table 4).
In the presence of an inhibitor (weak or potent), the activity of the corresponding CYP/P-gp was decreased in 13% to 89% of cases. This phenomenon seemed less important with weak inhibitors (13% to 83%) than with potent inhibitors (42% to 100% of cases) (Table 4). A detailed list of all the different CYP/P-gp inhibitors involved and the measured activity of the corresponding CYP/P-gp can be found in Supplementary Table S4. In the presence of a CYP/P-gp inducer (weak or potent), the activity of the CYP/P-gp involved was increased in 6/13 cases (46%), and in only 1/7 cases (14%) in the case of a weak inducer. The detailed list of all the CYP/P-gp inducers and the measured activity of the corresponding CYP/P-gp can be found in Supplementary Table S5.
In 6/7 patients (86%) who were smoking and whose CYP1A2 activity was measured, the activity was increased (none had inhibitor or inducer). In the four individuals that were consuming two or more glasses of alcohol per day, CYP1A2 and CYP3A4 activities were assessed in three and four cases, respectively. CYP1A2 was increased in one patient only, and the two others had normal activity (but one of the patients with normal activity had an inhibitor at the moment of phenotyping). CYP3A4 activity was decreased in 1/4 cases and was normal in 3/4 cases (but two of them had an inhibitor at the moment of phenotyping).

3.5. Follow-Up

Of the 31 patients who had undergone phenotyping, 12 were followed up by a physician at Fernand Widal Hospital after the phenotyping; among them, we obtained follow-up data for ten patients (Figure 3).
The metabolic profile was compatible with an underdosing of clozapine revealed by a plasma test in 2/10 patients. Daily dosages were increased, resulting in clinical improvement in 1/2 patients. When the decreased metabolic status was compatible with the side effects (4/10 patients), a drug switch was performed, resulting in clinical improvement in all patients (4/4) (Figure 3). When the metabolic status was not implicated in the therapeutic problem (4/10 patients), a switch of pharmacological drug class was performed in 3/4 patients, two of whom showed clinical improvement (Figure 3).

4. Discussions

The use of CYP/P-gp activity assessment can address two situations: as a complementary tool to TDM to select the medication and adjust the dosage through genotype-based recommendations [31,32], or to investigate a drug failure or an abnormal TDM [16,17]. Genotyping is increasingly being suggested for the former situation through genotype-based recommendations [31,33]. Pharmacogenetics has expanded since its introduction in some countries, and testing panels are marketed to physicians or directly to customers [33]. However, as phenoconversion happens in 30 to 60% of cases [13,14], genotyping is still imprecise unless several allelic variants are analyzed, which is not the case in the marketed tests [34]. Thus, phenotypic assessment appears more effective for investigating therapeutic drug failure [19], but studies assessing its benefits, especially in terms of clinical outcomes, are scarce.
In our hospital, we have been using phenotyping for the past four years to investigate drug failures. In cases of non-response or adverse drug reaction, we proceeded to a phenotypic assessment of one or several different drugs when TDM was not available (due to the difficulty of measuring blood levels of certain drugs or because the history of blood tests was missing) or showed unexplained plasma concentrations out of therapeutic range (i.e., to exclude other causes, for example, a lack of compliance, which concerns 10 to 60 % of individuals with mood disorders [35]).
In 34% of cases, the therapeutic failure was attributable to the metabolic profile. These findings are identical to those of Lloret-Linares et al. (34.7%) in a similar psychiatric context [16] and those of Rollason et al. (35.4%) with analgesics [36]. Both of these studies found a trend of a higher implication of metabolic status in ADR than in non-response, whereas we tended to find the opposite. However, our sample size was too small to proceed to statistical analysis.
In 31% of cases, phenotyping excluded the metabolic profile as the cause of the therapeutic problem and was more suggestive of a pharmacodynamic cause. It thus gave the physicians an argument to switch drugs or therapeutic class. In addition, in certain cases, it allowed the physicians to give factual arguments to patients which explained the role of their metabolism in the recurrent drug-related issues encountered.
Thus, phenotyping gave a clear answer to the therapeutic problem in two-thirds of cases. Among these cases, seven of the ten patients for which we had follow-up information showed a clinical improvement. Phenotyping has been of particular assistance in the therapeutic management of two situations: first, when it revealed that low plasma concentration was due to metabolic status rather than poor compliance as suspected by the physicians; second, in a case where phenotyping showed that the recurrence of adverse drug reactions was due to decreased metabolism and not to a systematic psychological apprehension to drugs.
In the remaining third of cases, phenotyping was able to determine the involvement of the metabolic profile in the therapeutic problem with only certain drugs, or was not interpretable. These situations seemed to occur more frequently when both non-response and ADR motivated the phenotyping, indicating multiple successive lines of pharmacotherapy. The multiplication of pharmacotherapy increases the involvement of multiple opposing metabolic pathways and makes it more difficult for phenotyping to give a clear answer, whatever the result. As Lloret-Linares et al. [16] and Rollason et al. [36] did not describe the therapeutic context of non-response or ADR in their cohorts, they might have identified these complex cases in which phenotyping would not be helpful, prior to phenotyping. Thus, we might have to be more careful in the selection of patients before phenotyping.
Only 23% to 56% of the explored CYP/P-gp had normal activity. This value is lower than the normal genotype values found in the literature for Caucasian populations [30,37,38,39]. This reflects phenoconversion due to environmental factors, as reported by previous genotype/phenotype studies [13,16]. In addition, we highlighted that the presence of enzyme inhibitors did not systematically decrease metabolic activity, which is consistent with the literature [40]. This phenomenon was particularly prevalent for “weak” inhibitors. However, these results should be interpreted with caution. First, some molecules might be more potent than others, and our sample size was too small to proceed to a multivariate analysis or a molecule-by-molecule evaluation to identify drugs with higher inhibiting potential. Second, we based our analysis of the qualification of inhibitors and inducers as weak or potent on the tools developed by the Division of Clinical Pharmacology and Toxicology, Geneva University Hospitals [30]. However, these qualifications seem to vary according to the sources, which do not necessarily consider the same metabolic pathways and the same degree of inhibition [4].
Phenotyping is easily implementable in a hospitalization unit. The main limitation is the burden for the patient due to the cocktail intake and the one-to-three blood tests when measuring P-gp. However, based on our patients’ feedback, and as previously described [40], the tolerance for the procedure was generally good. We had to face the objection of only one patient, who was unwilling to take the drug cocktail. Moreover, the phenotyping was generally favorably received by patients as an opportunity to explain the multiple drug failure and to effectively treat their pathology. This had been quantified by a previous study on genotyping, with the majority of patients hopeful that the test would identify the right psychiatric medication for them [41]. Furthermore, for patients referred by community psychiatrists after multiple therapeutic failure, the multidisciplinary involvement and the personalization of treatment open new avenues for recovery. Regarding the burden for the patient, recent findings suggest that endogenous markers might be used instead of drug probes to assess CYP activities [42,43]. If these findings are confirmed, it would open new possibilities for routine phenotyping. In addition, new assay techniques, particularly dried blood spot sampling—which allows the collection of a small volume of blood from the fingertip [5,27]—will ease the burden of the blood tests.
This study has some limitations. First, due to the retrospective approach, we partly or totally lacked some valuable data: the qualitative and quantitative evaluation of adverse drug reactions, drug dosages, medical and medication history, duration since treatment initiation, follow-up data, and patients’ and physicians’ satisfaction feedback. In particular, many data concerning drug-level blood tests were missing, as they were not performed routinely and results were not systematically registered in the medical file. Moreover, patient adherence to treatment was not quantified. Second, we had very small samples which precluded any statistical test. However, this preliminary study was only intended as a descriptive analysis to prepare a future prospective study of our phenotyping.

5. Conclusions

CYP and P-gp phenotyping is easily implementable in specialized centers and can provide solutions for complex psychiatric cases. Despite interest in the community, and in contrast to genotyping, only a few studies have been concerned with the clinical outcomes of the phenotypic assessment of metabolic pathways. Our study seems to show a clinical benefit of the phenotypic assessment of the metabolic pathway in clinical practice. However, as this study was only descriptive, these conclusions have to be confirmed by further larger-scale multicentered studies. Notwithstanding small samples and missing data, our study is, to our knowledge, the first to evaluate the consequences of CYP/P-gp phenotyping in terms of therapeutic adaptation and clinical outcomes in psychiatric patients.

Supplementary Materials

The following are available online at https://www.mdpi.com/article/10.3390/jpm12111869/s1. Table S1: Drug cocktail details. Table S2: Patient individual characteristics and key assessments. Table S3: Drugs implicated in the therapeutic problem that led to the 32 phenotypings and the CYP/P-gp for which they are a substrate. Table S4: Effect of CYP/P-gp inhibitor intake on the activity of the CYP/P-gp they inhibit. Table S5: Effect of CYP/P-gp inducer intake on the activity of the CYP/P-gp they induce.

Author Contributions

Conceptualization, C.D., B.E., C.L.-L., V.B. and X.D.; data investigation: C.D., L.D., M.V. (Marina Vignes), T.-K.L., M.C. and N.K.; methodology, C.D., L.D., X.D. and V.B.; validation, J.-E.F., L.L., F.B., X.D. and V.B.; formal analysis, C.D. and L.D.; clinical investigation, B.E., A.F., M.V. (Marc Veyrier), M.D., E.M., V.H., M.M., A.J., J.S., G.P. and F.B.; data curation, L.D., C.D., V.B. and B.E.; writing—original draft preparation, C.D.; writing—review and editing, B.E., C.L.-L., X.D. and V.B. All authors have read and agreed to the published version of the manuscript.

Funding

This research received no external funding.

Institutional Review Board Statement

The study was conducted in accordance with the Declaration of Helsinki and approved by the Institutional Review Board (or Ethics Committee) of the French Ethics Committee “Comité de Protection des Personnes Sud Méditerranée”, numbered as ID-RCB-2017-A00685-48.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Gartlehner, G.; Hansen, R.; Morgan, L.; Thaler, K.; Lux, L. Second-Generation Antidepressants in the Pharmacologic Treatment of Adult Depression: An Update of the 2007 Comparative Effectiveness Review No. 46; US Department of Health and Human Services: Washington, DC, USA; US Agency for Healthcare Research and Quality: Rockville, MD, USA, 2011.
  2. Rush, A.J.; Trivedi, M.H.; Wisniewski, S.R.; Nierenberg, A.A.; Stewart, J.W.; Warden, D.; Niederehe, G.; Thase, M.E.; Lavori, P.W.; Lebowitz, B.D.; et al. Acute and longer-term outcomes in depressed outpatients requiring one or several treatment steps: A STAR*D report. Am. J. Psychiatry 2006, 163, 1905–1917. [Google Scholar] [CrossRef] [PubMed]
  3. Berlim, M.T.; Fleck, M.P.; Turecki, G. Current trends in the assessment and somatic treatment of resistant/refractory major depression: An overview. Ann. Med. 2009, 40, 149–159. [Google Scholar]
  4. Hiemke, C.; Bergemann, N.; Clement, H.W.; Conca, A.; Deckert, J.; Domschke, K.; Eckermann, G.; Egberts, K.; Gerlach, M.; Greiner, C.; et al. Consensus Guidelines for Therapeutic Drug Monitoring in Neuropsychopharmacology: Update 2017. Pharmacopsychiatry 2018, 51, 9–62. [Google Scholar] [PubMed] [Green Version]
  5. De Kesel, P.M.M.; Lambert, W.E.; Stove, C.P. Alternative Sampling Strategies for Cytochrome P450 Phenotyping. Clin. Pharm. 2016, 55, 169–184. [Google Scholar]
  6. Guengerich, F.P. Common and uncommon cytochrome P450 reactions related to metabolism and chemical toxicity. Chem. Res. Toxicol. 2001, 14, 611–650. [Google Scholar]
  7. Amin, M.L. P-glycoprotein inhibition for optimal drug delivery. Drug Target Insights 2013, 7, 27–34. [Google Scholar] [CrossRef]
  8. Ou-Yang, D.S.; Huang, S.L.; Wang, W.; Xie, H.G.; Xu, Z.H.; Shu, Y.; Zhou, H.H. Phenotypic polymorphism and gender-related differences of CYP1A2 activity in a Chinese population. Br. J. Clin. Pharmacol. 2000, 49, 145–151. [Google Scholar]
  9. Rebsamen, M.C.; Desmeules, J.; Daali, Y.; Chiappe, A.; Diemand, A.; Rey, C.; Chabert, J.; Dayer, P.; Hochstrasser, D.; Rossier, M.F.; et al. The AmpliChip CYP450 test: Cytochrome P450 2D6 genotype assessment and phenotype prediction. Pharm. J. 2009, 9, 34–41. [Google Scholar] [CrossRef]
  10. Molden, E.; Jukić, M.M. CYP2D6 Reduced Function Variants and Genotype/Phenotype Translations of CYP2D6 Intermediate Metabolizers: Implications for Personalized Drug Dosing in Psychiatry. Front. Pharmacol. 2021, 12, 650750. [Google Scholar] [CrossRef]
  11. Maike Scherf-Clavel, A.; Weber, H.; Wurst, C.; Stonawski, S.; Hommers, L.; Unterecker, S.; Wolf, C.; Domschke, K.; Rost, N.; Brückl, T.; et al. Effects of Pharmacokinetic Gene Variation on Therapeutic Drug Levels and Antidepressant Treatment Response. Pharmacopsychiatry 2022, 55, 246–254. [Google Scholar] [CrossRef]
  12. Frederiksen, T.; Smith, R.L.; Wollmann, B.M.; Areberg, J.; Molden, E. Validation of a Population Pharmacokinetic Model of Vortioxetine Using Therapeutic Drug Monitoring Data. Clin. Pharm. 2021, 60, 1475–1486. [Google Scholar] [CrossRef] [PubMed]
  13. Gloor, Y.; Lloret-Linares, C.; Bosilkovska, M.; Perroud, N.; Richard-Lepouriel, H.; Aubry, J.M.; Daali, Y.; Desmeules, J.A.; Besson, M. Drug metabolic enzyme genotype-phenotype discrepancy: High phenoconversion rate in patients treated with antidepressants. Biomed. Pharm. 2022, 152, 113202. [Google Scholar] [CrossRef]
  14. Ing Lorenzini, K.; Desmeules, J.; Rollason, V.; Bertin, S.; Besson, M.; Daali, Y.; Samer, C.F. CYP450 Genotype—Phenotype Concordance Using the Geneva Micrococktail in a Clinical Setting. Front. Pharm. 2021, 12, 730637. [Google Scholar] [CrossRef] [PubMed]
  15. De Andrés, F.; Terán, S.; Hernández, F.; Terán, E.; Llerena, A. To Genotype or Phenotype for Personalized Medicine? CYP450 Drug Metabolizing Enzyme Genotype-Phenotype Concordance and Discordance in the Ecuadorian Population. Omi. J. Integr. Biol. 2016, 20, 699–710. [Google Scholar]
  16. Lloret-Linares, C.; Rollason, V.; Lorenzini, K.I.; Samer, C.; Daali, Y.; Gex-Fabry, M.; Aubry, J.-M.; Desmeules, J.; Besson, M. Screening for genotypic and phenotypic variations in CYP450 activity in patients with therapeutic problems in a psychiatric setting, a retrospective study. Pharm. Res. 2017, 118, 104–110. [Google Scholar] [CrossRef] [PubMed]
  17. Rollason, V.; Lloret-Linares, C.; Lorenzini, K.I.; Daali, Y.; Gex-Fabry, M.; Piguet, V.; Besson, M.; Samer, C.; Desmeules, J. Evaluation of phenotypic and genotypic variations of drug metabolising enzymes and transporters in chronic pain patients facing adverse drug reactions or non-response to analgesics: A retrospective study. J. Pers. Med. 2020, 10, 198. [Google Scholar] [CrossRef] [PubMed]
  18. Lenoir, C.; Rollason, V.; Desmeules, J.A.; Samer, C.F. Influence of Inflammation on Cytochromes P450 Activity in Adults: A Systematic Review of the Literature. Front. Pharmacol. 2021, 12, 733935. [Google Scholar] [CrossRef]
  19. Eap, C.B.; Gründer, G.; Baumann, P.; Ansermot, N.; Conca, A.; Corruble, E.; Crettol, S.; Dahl, M.L.; de Leon, J.; Greiner, C.; et al. Tools for optimising pharmacotherapy in psychiatry (therapeutic drug monitoring, molecular brain imaging and pharmacogenetic tests): Focus on antidepressants. World J. Biol. Psychiatry 2021, 22, 561–628. [Google Scholar]
  20. Andrés, F.; LLerena, A. Simultaneous Determination of Cytochrome P450 Oxidation Capacity in Humans: A Review on the Phenotyping Cocktail Approach. Curr. Pharm. Biotechnol. 2016, 17, 1159–1180. [Google Scholar] [CrossRef]
  21. Tracy, T.S.; Chaudhry, A.S.; Prasad, B.; Thummel, K.E.; Schuetz, E.G.; Zhong, X.B.; Tien, Y.C.; Jeong, H.; Pan, X.; Shireman, L.M.; et al. Interindividual Variability in Cytochrome P450-Mediated Drug Metabolism. Drug Metab. Dispos. 2016, 44, 343–351. [Google Scholar] [CrossRef] [Green Version]
  22. Magliocco, G.; Thomas, A.; Desmeules, J.; Daali, Y. Phenotyping of Human CYP450 Enzymes by Endobiotics: Current Knowledge and Methodological Approaches. Clin. Pharmacokinet. 2019, 58, 1373–1391. [Google Scholar] [PubMed]
  23. Varshney, E.; Tandon, M.; Saha, N.; Ali, S. In Vivo phenotyping of cytochrome 450 isoforms involved in the metabolism of anti-HIV and anti-tubercular drugs in human using cocktail approach: An LC–MS/MS analysis. J. Pharm. Biomed. Anal. 2019, 164, 698–705. [Google Scholar] [CrossRef] [PubMed]
  24. Lenoir, C.; Terrier, J.; Gloor, Y.; Gosselin, P.; Daali, Y.; Combescure, C.; Desmeules, J.A.; Samer, C.S.; Reny, J.-L.; Rollason, V. Impact of the Genotype and Phenotype of CYP3A and P-gp on the Apixaban and Rivaroxaban Exposure in a Real-World Setting. J. Pers. Med. 2022, 12, 526. [Google Scholar] [CrossRef]
  25. Mathijssen, R.H.J.; Van Schaik, R.H.N. Genotyping and phenotyping cytochrome P450: Perspectives for cancer treatment. Eur. J. Cancer 2006, 42, 141–148. [Google Scholar] [CrossRef] [PubMed]
  26. Hiemke, C.; Shams, M. Phenotyping and Genotyping of Drug Metabolism to Guide Pharmacotherapy in Psychiatry. Curr. Drug Deliv. 2013, 10, 46–53. [Google Scholar] [CrossRef]
  27. Bosilkovska, M.; Samer, C.F.; Déglon, J.; Rebsamen, M.; Staub, C.; Dayer, P.; Walder, B.; Desmeules, J.A.; Daali, Y. Geneva Cocktail for Cytochrome P450 and P-Glycoprotein Activity Assessment Using Dried Blood Spots. Clin. Pharm. Ther. 2014, 96, 349–359. [Google Scholar]
  28. Bosilkovska, M.; Samer, C.; Déglon, J.; Thomas, A.; Walder, B.; Desmeules, J.; Daali, Y. Evaluation of Mutual Drug-Drug Interaction within Geneva Cocktail for Cytochrome P450 Phenotyping using Innovative Dried Blood Sampling Method. Basic Clin. Pharm. Toxicol. 2016, 119, 284–290. [Google Scholar] [CrossRef]
  29. Bosilkovska, M.; Déglon, J.; Samer, C.; Walder, B.; Desmeules, J.; Staub, C.; Daali, Y. Simultaneous LC-MS/MS quantification of P-glycoprotein and cytochrome P450 probe substrates and their metabolites in DBS and plasma. Bioanalysis 2014, 6, 151–164. [Google Scholar]
  30. Samer, C.F.; Lorenzini, K.I.; Rollason, V.; Daali, Y.; Desmeules, J.A. Applications of CYP450 testing in the clinical setting. Mol. Diagn. Ther. 2013, 17, 165–184. [Google Scholar] [CrossRef] [Green Version]
  31. Hicks, J.K.; Sangkuhl, K.; Swen, J.J.; Ellingrod, V.L.; Müller, D.J.; Shimoda, K.; Bishop, J.R.; Kharasch, E.D.; Skaar, T.C.; Gaedigk, A.; et al. Clinical pharmacogenetics implementation consortium guideline (CPIC) for CYP2D6 and CYP2C19 genotypes and dosing of tricyclic antidepressants: 2016 update. Clin. Pharmacol. Ther. 2017, 102, 37–44. [Google Scholar]
  32. International Society for Psychiatric Genetics (ISPG). Genetic Testing Statement: A Statement from the International Society of Psychiatric Genetics; International Society for Psychiatric Genetics (ISPG): Brentwood, TN, USA.
  33. Drozda, K.; Müller, D.J.; Bishop, J.R. Pharmacogenomic Testing for Neuropsychiatric Drugs: Current Status of Drug Labeling, Guidelines for Using Genetic Information, and Test Options. Pharm. J. Hum. Pharm. Drug Ther. 2014, 34, 166–184. [Google Scholar]
  34. Van Schaik, R.H.N.; Müller, D.J.; Serretti, A.; Ingelman-Sundberg, M. Pharmacogenetics in Psychiatry: An Update on Clinical Usability. Front. Pharm. 2020, 11, 1443. [Google Scholar]
  35. Lingam, R.; Scott, J. Treatment non-adherence in affective disorders. Acta Psychiatr. Scand. 2002, 105, 164–172. [Google Scholar]
  36. Rollason, V.; Mouterde, M.; Daali, Y.; Čížková, M.; Priehodová, E.; Kulichová, I.; Posová, H.; Petanová, J.; Mulugeta, A.; Makonnen, E.; et al. Safety of the Geneva Cocktail, a Cytochrome P450 and P-Glycoprotein Phenotyping Cocktail, in Healthy Volunteers from Three Different Geographic Origins. Drug Saf. 2020, 43, 1181–1189. [Google Scholar] [CrossRef] [PubMed]
  37. Ingelman-Sundberg, M.; Sim, S.C.; Gomez, A.; Rodriguez-Antona, C. Influence of cytochrome P450 polymorphisms on drug therapies: Pharmacogenetic, pharmacoepigenetic and clinical aspects. Pharmacol. Ther. 2007, 116, 496–526. [Google Scholar] [CrossRef] [PubMed]
  38. Zhou, S.F.; Liu, J.P.; Chowbay, B. Polymorphism of human cytochrome P450 enzymes and its clinical impact. Drug. Metab. Rev. 2009, 41, 89–295. [Google Scholar]
  39. McGraw, J.; Waller, D. Cytochrome P450 variations in different ethnic populations. Expert Opin. Drug Metab. Toxicol. 2012, 8, 371–382. [Google Scholar]
  40. Lloret-Linares, C.; Bosilkovska, M.; Daali, Y.; Gex-Fabry, M.; Heron, K.; Bancila, V.; Michalopoulos, G.; Perroud, N.; Richard-Lepouriel, H.; Aubry, J.-M.; et al. Phenotypic Assessment of Drug Metabolic Pathways and P-Glycoprotein in Patients Treated With Antidepressants in an Ambulatory Setting. J. Clin. Psychiatry 2018, 79, 14196. [Google Scholar]
  41. Liko, I.; Lai, E.; Griffin, R.J.; Aquilante, C.L.; Lee, Y.M. Patients’ Perspectives on Psychiatric Pharmacogenetic Testing. Pharmacopsychiatry 2020, 53, 256–261. [Google Scholar] [CrossRef]
  42. Magliocco, G.; Desmeules, J.; Bosilkovska, M.; Thomas, A.; Daali, Y. The 1β-Hydroxy-Deoxycholic Acid to Deoxycholic Acid Urinary Metabolic Ratio: Toward a Phenotyping of CYP3A Using an Endogenous Marker? J. Pers. Med. 2021, 11, 150. [Google Scholar] [CrossRef]
  43. Magliocco, G.; Desmeules, J.; Matthey, A.; Quirós-Guerrero, L.M.; Bararpour, N.; Joye, T.; Marcourt, L.; Queiroz, E.F.; Wolfender, J.; Gloor, Y.; et al. Metabolomics reveals biomarkers in human urine and plasma to predict cytochrome P450 2D6 (CYP2D6) activity. Br. J. Pharm. 2021, 178, 4708–4725. [Google Scholar]
Jpm 12 01869 g001 550
Figure 1. Phenotyping protocol.
Figure 1. Phenotyping protocol.
Jpm 12 01869 g001
Jpm 12 01869 g002 550
Figure 2. (A) Implications of the different CYP/P-gp profiles with the 32 phenotypings and (B) proportion of drugs that are minor (light blue) or major (dark blue) substrates of the CYP/P-gp profiles among the 224 cumulated drugs implicated in the therapeutic problem that led to the 32 phenotypic assessments (n = 224 cumulated drugs = sum of all drugs involved in each of the 32 phenotypings; a drug involved in more than one phenotyping was counted cumulatively for as many times as the phenotypings).
Figure 2. (A) Implications of the different CYP/P-gp profiles with the 32 phenotypings and (B) proportion of drugs that are minor (light blue) or major (dark blue) substrates of the CYP/P-gp profiles among the 224 cumulated drugs implicated in the therapeutic problem that led to the 32 phenotypic assessments (n = 224 cumulated drugs = sum of all drugs involved in each of the 32 phenotypings; a drug involved in more than one phenotyping was counted cumulatively for as many times as the phenotypings).
Jpm 12 01869 g002
Jpm 12 01869 g003 550
Figure 3. Therapeutic decision and clinical consequences following phenotyping.
Figure 3. Therapeutic decision and clinical consequences following phenotyping.
Jpm 12 01869 g003
Table
Table 1. Population and phenotyping characteristics.
Table 1. Population and phenotyping characteristics.
Age (year) (mean ± SD)47.1 ± 14.5
Number of female patients (n, % of the 31 patients)22 (71%)
Tobacco consumption (n, % of the 31 patients)9 (29%)
Alcohol consumption >2 glass/day (n, % of the 31 patients)4 (13%)
Grapefruit consumption (n, % of the 31 patients)0 (0%)
Hypericum consumption (n, % of the 31 patients)0 (0%)
Type of patient care at the assessment (n, % of the 32 phenotypings)
Inpatients13 (41%)
Outpatients19 (59%)
Clinical context (n, % of the 32 phenotypings)
Depressive disorder15 (47%)
Bipolar disorder12 (38%)
Anxiety–depressive syndrome 3 (9%)
Schizoaffective disorder1 (3%)
Neuropathic pain1 (3%)
Number of drugs implicated (mean ± SD)7 ± 4.8
Therapeutic problem (n, % of the 32 phenotypings)
ADR10 (31%)
NR6 (19%)
ADR+NR16 (50%)
ADR: adverse drug reaction; NR: non-response.
Table
Table 2. Drugs most implicated in the therapeutic problems that led to a phenotypic assessment.
Table 2. Drugs most implicated in the therapeutic problems that led to a phenotypic assessment.
DrugNumber of PhenotypingsImplication of the Drug in Therapeutic Problems
Non-ResponseAdverse Drug Reaction
Venlafaxine17611
Fluoxetine1569
Paroxetine1028
Mirtazapine1064
Clomipramine1046
Quetiapine1019
Lithium1037
Lamotrigine945
Aripiprazole936
Olanzapine743
Sertraline606
Escitalopram624
Risperidone624
Duloxetine514
Amitriptyline532
Vortioxetine431
Chlorpromazine431
Table
Table 3. Interpretation of the phenotypic assessment, in terms of the involvement of the metabolic profile in the therapeutic problem, depending on the context of the demand and the number of drugs implicated.
Table 3. Interpretation of the phenotypic assessment, in terms of the involvement of the metabolic profile in the therapeutic problem, depending on the context of the demand and the number of drugs implicated.
Number of Phenotypings (n)Involvement of Metabolic Profile in the Therapeutic Problem
(n–% of Responses in the Corresponding Therapeutic Problem)
YNY+NPNIP+NI
Therapeutic problemADR104 (40%)4 (40%)8 (80%)2 (20%)0 (0%)2 (20%)
NR64 (67%)1 (17%)5 (83%)0 (0%)1 (17%)1 (17%)
ADR+NR163 (19%)5 (31%)8 (50%)5 (31%)3 (19%)8 (50%)
Total3211 (34%)10 (31%)21 (66%)7 (22%)4 (13%)11 (34%)
Number of drugs involved
(mean ± SD)		7.0 ± 4.84.5 ± 2.98.7 ± 4.56.5 ± 4.28.0 ± 7.18.0 ± 3.48.0 ± 5.8
ADR: adverse drug reaction; N: the phenotyping precluded the metabolic profile as a possible cause of the therapeutic problem; NI: phenotyping results were not interpretable; NR: non-response; P: the metabolic profile might have been partially responsible for the therapeutic problem; Y: the metabolic profile is compatible with under- or overdosing that might have caused the therapeutic problem.
Table
Table 4. Cytochrome (CYP) and P-glycoprotein (P-gp) measured activity in the presence and absence of an inhibitor.
Table 4. Cytochrome (CYP) and P-glycoprotein (P-gp) measured activity in the presence and absence of an inhibitor.
Number of Phenotypic Assessments (n)CYP/P-gp Measured Activity
(% of the Phenotypic Assessments)
Decreased/PMNormal/IMIncreased/EMUM
CYP1A23023%43%33%
In the presence of an inhibitor838%50%13%
Weak838%50%13%
Potent
CYP2B6300%40%60%
In the presence of an inhibitor1 100%
Weak1 100%
Potent
CYP2C93033%23%43%
In the presence of an inhibitor8 25%75%
Weak4 25%
Potent4 25%
CYP2C193238%53%9%
In the presence of an inhibitor989%11%
Weak683%17%
Potent3100%
CYP2D63222%56%16%6%
In the presence of an inhibitor2921%62%17%
Weak176%65%29%
Potent1242%58%
CYP3A43033%43%23%
In the presence of an inhibitor813%75%13%
Weak813%75%13%
Potent
P-gp3057%40%3%
In the presence of an inhibitor 1173%27%
Weak475%25%
Potent771%29%
EM: extensive metabolizer; IM: intermediate metabolizer; PM: poor metabolizer; UM: ultrarapid metabolizer.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Delage, C.; Darnaud, L.; Etain, B.; Vignes, M.; Ly, T.-K.; Frapsauce, A.; Veyrier, M.; Delavest, M.; Marlinge, E.; Hennion, V.; et al. Cytochromes P450 and P-Glycoprotein Phenotypic Assessment to Optimize Psychotropic Pharmacotherapy: A Retrospective Analysis of Four Years of Practice in Psychiatry. J. Pers. Med. 2022, 12, 1869. https://doi.org/10.3390/jpm12111869

AMA Style

Delage C, Darnaud L, Etain B, Vignes M, Ly T-K, Frapsauce A, Veyrier M, Delavest M, Marlinge E, Hennion V, et al. Cytochromes P450 and P-Glycoprotein Phenotypic Assessment to Optimize Psychotropic Pharmacotherapy: A Retrospective Analysis of Four Years of Practice in Psychiatry. Journal of Personalized Medicine. 2022; 12(11):1869. https://doi.org/10.3390/jpm12111869

Chicago/Turabian Style

Delage, Clément, Léa Darnaud, Bruno Etain, Marina Vignes, Tu-Ky Ly, Alexia Frapsauce, Marc Veyrier, Marine Delavest, Emeline Marlinge, Vincent Hennion, and et al. 2022. "Cytochromes P450 and P-Glycoprotein Phenotypic Assessment to Optimize Psychotropic Pharmacotherapy: A Retrospective Analysis of Four Years of Practice in Psychiatry" Journal of Personalized Medicine 12, no. 11: 1869. https://doi.org/10.3390/jpm12111869

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop