Next Article in Journal
The Impact of COVID-19 Vaccination on Anxiety Levels of Turkish Dental Professionals and Their Attitude in Clinical Care: A Cross-Sectional Study
Previous Article in Journal
Teamworking in Healthcare during the COVID-19 Pandemic: A Mixed-Method Study
 
 
Search for Articles:
Title / Keyword
Author / Affiliation / Email
Journal
Article Type
 
 
Advanced Search
 
Section
Special Issue
Volume
Issue
Number
Page
 
Journals
IJERPH
Volume 18
Issue 19
10.3390/ijerph181910372
ijerph-logo
Submit to this Journal Review for this Journal Propose a Special Issue
► Article Menu

Article Menu

share Share announcement Help format_quote Cite question_answer Discuss in SciProfiles thumb_up
...
Endorse textsms
...
Comment
first_page
settings
Order Article Reprints
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Development and Validation of the Psychotherapeutic Effectiveness Attribution Questionnaire (PEAQ-12) in a Spanish Population

by
Antonio Romero-Moreno
1,2,
Alberto Paramio
1,2,*,
Serafín J. Cruces-Montes
1,2,
Antonio Zayas
1,2,*,
Diego Gómez-Carmona
2,3 and
Ana Merchán-Clavellino
1,2
1
Department of Psychology, Faculty of Education Sciences, University of Cádiz, 11519 Puerto Real, Spain
2
University Institute of Research in Social Sustainable Development, University of Cadiz, 11405 Jerez de la Frontera, Spain
3
Department of Marketing and Communication, Faculty of Social Sciences, University of Cadiz, 11405 Jerez de la Frontera, Spain
*
Authors to whom correspondence should be addressed.
Int. J. Environ. Res. Public Health 2021, 18(19), 10372; https://doi.org/10.3390/ijerph181910372
Submission received: 28 June 2021 / Revised: 24 September 2021 / Accepted: 27 September 2021 / Published: 1 October 2021
(This article belongs to the Section Mental Health)
Browse Figures
Versions Notes

Abstract

:
In recent decades, the study of psychotherapy effectiveness has been one of the pillars of clinical research because of its implication for therapeutic cure. However, although many studies have focused their interest on the patient’s perception, there are no instruments oriented to the study of psychotherapists’ attributions of effectiveness: to what factors psychotherapists attribute responsibility for the cure of the therapies they provide. The present study aimed to develop and validate an instrument for assessing the attribution of the effectiveness of psychotherapy in a population of 69 psychotherapists of different theoretical orientations. After an initial process of inter-judge content validation, 12 items were selected for validation in the targeted population, adequately fulfilling the quality requirements in the validity–reliability tests, and grouped into four factors after principal component analysis. These factors were as follows: (1) therapeutic alliance enhancers; (2) psychotherapist emotional characteristics; (3) therapy-specific variables; and (4) facilitators of patient engagement with therapy. This four-factor structure also showed a good fit for the fit indices checked in confirmatory factor analysis. In summary, we can conclude that the Psychotherapeutic Effectiveness Attribution Questionnaire (PEAQ-12) developed in our research can be helpful if tested on a larger number of individuals. The results can be replicated in other populations of psychotherapists.

1. Introduction

The study of the variables involved in the effectiveness of psychotherapy has occupied ample space in psychotherapeutic research in recent decades. The possible elements that may affect psychotherapy outcomes have usually been classified into two broad categories: psychotherapy-specific variables and common factors (or non-specific variables) [1,2,3,4]. The specific variables refer to genuine and idiosyncratic behaviours in each therapy modality, making them recognisable and different from each other. We refer to the particular techniques and procedures used and the therapeutic approach employed based on a specific theoretical orientation. Common variables point to active ingredients present in all psychotherapies and are instrumental in therapeutic change [5]. Such variables common to different psychotherapies would include those related to the patient, with the effect of the therapist on the therapy and the therapeutic interaction [6,7].
Traditionally, specific variables were attributed full responsibility for the healing process [8]. However, Lambert et al. [9] set the contribution of specific therapeutic techniques at only 15%, a figure with which other authors concur [10,11,12]. Wampold’s [13] meta-analysis assigned therapist-performed techniques 13% of the change variance, which Duncan [14] set at only 1%. Cuijpers et al. [15] established that specific factors were only responsible for 17% of patient improvement in their meta-analysis.
In recent research on therapeutic outcomes and processes, there seems to be a broad consensus that common factors are primarily responsible for therapeutic change [11,16,17,18,19,20,21,22]. Thus, research on common factors indicates that 85% of change is due to such elements present in all psychotherapy modalities [23,24]. However, it is important to note that the common factors are not just a set of therapeutic elements present in all or most psychological treatments, but rather a theoretical model of the mechanisms of psychotherapeutic change [25].
Of all the common variables, those related to the therapeutic relationship or interaction (and, specifically, the therapeutic alliance that occurs within it) are among those that contribute most to the effectiveness of psychological treatments [26,27,28,29].
In this sense, studies such as Gaston et al. [30] or Luborsky et al. [31] found that the therapeutic alliance explained between 36% and 57% of the variance in the therapy outcome. Safran and Segal [32] concluded that 45% of therapeutic change was due to factors related to the therapeutic relationship. A review of 132 research studies by Orlinsky et al. [29] concluded a strong relationship between the quality of the therapeutic alliance and psychotherapy outcome. Thus, a large body of research emerging worldwide establishes that various aspects of the therapeutic alliance correlate positively with treatment outcomes [33,34]. Thus, developing a close patient–therapist bond is considered a crucial component of successful psychotherapy [35,36,37,38,39,40,41].
Patient-related variables are also considered extremely important [42], as there are studies that estimate their contribution to therapeutic success at up to 40% of the variance [9]. However, demographic factors (age, socioeconomic status, educational level, race or gender) or those related to the locus of control do not play a significant role in psychotherapy outcomes, or contradictory data are obtained in different studies [43,44,45]. Crucial elements in the therapeutic healing process are the patient’s expectation of cure, the patient’s involvement in the course of therapy and the faith and credibility the patient assigns to the therapist.
The study of patient expectations, even from the first days of therapy, has been one of the best and longest studied variables, being considered a crucial element in the healing process [11,46,47,48]. On the other hand, treatment involvement is a powerful indicator of the therapeutic alliance in the treatment process [49]. As the patient is an active agent of change [50], treatment adherence is an essential element in explaining the success of an intervention [51,52]. The patient’s expectation of cure and involvement in treatment could be assimilated to what other studies have called motivation to change or motivation to follow treatment [53]. Finally, the faith and credibility the patient gives to the therapist are shown to be fundamental factors in the expected results [54], both for maintaining a positive belief towards the treatment and the techniques used and for the trust established with the psychotherapist.
Regarding common therapist variables, that is, variables related to the effect of the therapist, a large number of them are relevant. Regarding the therapist’s emotional well-being, the therapist’s level of emotional adjustment is related to therapeutic success [55], further noting that a disturbed therapist may impede their patients’ growth and induce negative changes in them [28,56,57]. The empathy shown has also been related to better adherence to the patient’s treatment [58,59,60,61]. The absence of empathy and the therapist’s lack of understanding of the problem were conducive to the patient’s therapeutic desertion [62,63]. Regarding the degree of acceptance, interest, understanding and encouragement shown by the therapist to the patient, it has been established that the psychotherapist must have an attitude that favours a therapeutic climate that facilitates change based on listening to, understanding and accepting the patient [1,51,64]. However, it is just as important to show a willingness to listen and understand the patient as it is to make patients feel listened to and understood [65].
Another component to consider is the therapist’s directivity/support, understood as the degree to which instructions, information and specific help are provided and tasks are structured and delimited [66]. To obtain favourable results, a good therapist must modulate their directivity and support depending on the phase of treatment, the type of problem addressed in the consultation and the patient’s personality characteristics [66,67,68,69].
Similarly, the therapist’s perception of patient involvement is mentioned as one of the main desirable characteristics in therapists [70,71,72,73]. According to the study by Lafferty et al. [60], more effective therapists felt that their patients were less involved in their treatment and made less progress, whereas less effective therapists perceived greater involvement and improvement.
The variable of the therapist’s ability to influence the patient, which is also present in other health professionals, seems to influence the outcome and effectiveness of treatment [74,75,76], where the therapist’s ability to persuade is established as a crucial element [77].
The last therapist variable to consider is the therapist’s experience. In general, the therapist’s expertise is related to positive changes [57]. Thus, older and more experienced therapists tend to act with more empathy and tolerance than beginners towards patients’ expressions of negative emotions related to the development and improvement of the therapeutic alliance [75,78].
Thus, the scientific evidence shows that it is the common elements present in all psychotherapy, regardless of the theoretical model on which it is based, that explains its effectiveness to a greater extent, meaning both of the variable types should be taken into consideration in terms of theory and research and practice [25]. Based on this evidence, it is interesting to question to what extent therapists have assumed the fundamental role that common factors play in the healing process, or whether, on the contrary, as Botella and Feixas [79] pointed out, it is to be expected that they will continue to point to specific variables as the main precursors of therapeutic change.
This separation between research and praxis (already pointed out by Beitman, 1987 [80]) can have repercussions on therapeutic success in such a way that the therapist targets strategies that are not as effective as those recognised by scientific evidence. For example, it is interesting that the therapist takes responsibility for their mental health and self-care, making it easier to respond therapeutically to patients’ problems [26]. Furthermore, flexibility is considered a fundamental quality of the therapist [42] and their ability to persuade. Although psychological treatments are usually highly protocolised, their effectiveness depends on a high degree of these skills [77]. The therapist’s willingness to listen to and understand themselves is also crucial [81], as the reactions and associations to the material brought by the patient are crucial information for understanding the dynamics of the client [82].
For all these reasons, and with the intention to develop programmes for the training of psychotherapists and for helping them to establish strategies that improve the success of their treatments, it is necessary to draw up a questionnaire to detect the variables to which they attribute the most significant responsibility in the process of therapeutic change. The aim is for therapists to be aware of any incorrect notions or ingrained attitudes they may have in order to correct them, and to favour the establishment of the focus of their intervention on the axis of the variables of the patient, the therapist and the therapeutic interaction, considered by psychotherapeutic research to be the main providers of effectiveness. Ultimately, the construction of this assessment tool would help therapists to check whether the therapeutic elements to which they attribute the greatest effectiveness are those that, in fact, bring the greatest efficacy to psychotherapy, which would ultimately help them to assign an appropriate weight to the different active components present in the treatments they carry out.

2. Methods

2.1. Sample

The study population chosen was all psychotherapists included in the directory of the Official College of Psychologists of Western Andalusia (Spain). A total of 69 psychotherapists (50.7% male and 49.3% female; mean age = 41.5, SD = 6.41) participated in the validation of the instrument. The theoretical orientation of these psychotherapists was divided into 4 orientations: 44.9% cognitive-behavioural, 26.1% psychodynamic, 15.9% eclectic, 10.1% humanistic-systemic, and those who chose not to declare their theorical orientation 3%.

2.2. Instrument

A questionnaire was sent to a selected group of 12 experts in psychotherapies and psychological treatments, who were asked to collaborate to assess the degree of congruence in the assignment of the different items to the objectives proposed. The composition of the questionnaire comprised a wide variety of items, coded and closed-ended (Annex 1). As a preliminary step in developing the questionnaire, a thorough literature review on outcome research and therapeutic processes was carried out to include the relevant variables in the form of items. Once those variables that have been analysed with the greatest emphasis as possibly being responsible for cure had been selected, the items corresponding to them were drafted, and their content was validated using the procedure described by Osterlind [83] by means of expert judgement.
The items of the selection before expert judgement corresponded to each of the psychotherapeutic variables relevant to the healing process and to which the psychotherapists had to assign a rating from 1 to 5 (where 1 = “does not influence at all on the patient’s improvement” and 5 = “has a great influence”). These variables were as follows: (1) therapeutic approach used; (2) techniques or procedures used; (3) patient’s expectation of cure; (4) patient’s involvement in the therapy; (5) patient’s faith and credibility assigned to the therapist; (6) therapist’s emotional well-being; (7) empathy shown by the therapist; (8) directivity and support shown by the therapist; (9) therapist’s perception of the patient’s involvement; (10) therapist’s ability to influence the patient; (11) degree of acceptance, interest, understanding and encouragement shown by the therapist to the patient; (12) experience of the therapist; (13) establishment of a therapeutic alliance between the therapist and patient. In addition to the scale, items related to the following variables were incorporated in order to describe the study population:
  • Demographic characteristics: sex and age, level of studies and place of practice of psychotherapy;
  • Therapist clinical characteristics: experience as a psychotherapist, theoretical orientation and access to publications on psychotherapy research.

2.3. Data Collection

The sample selected for the study was sent a letter of introduction, informed consent and the final version of the scale (Appendix A). It was facilitated by email because the participants resided in distant locations. Respondents were informed that feedback would be provided to them after presenting the study in response to their collaboration.

2.4. Ethical Considerations

This study was conducted in compliance with the Declaration of Helsinki of 1975. An information letter was sent to all participants together with the questionnaire. The participants were guaranteed confidentiality and anonymity. All participants signed the informed consent.

2.5. Data Analysis

The normality of the quantitative variables was verified using the Kolmogorov–Smirnov–Lilliefors test [84]. Descriptive statistics were used to summarise demographic data. Means and standard deviations were examined to determine items and overall score distributions.
For content validity, the congruence index of each item with the objective it intends to measure was calculated [85,86], selecting those that achieved a higher score to form part of the instrument (Iik > 0.5).
Construct validity was assessed using exploratory factor analysis (EFA) and the known group technique [87]. The EFA was undertaken [88] using principal component analysis (PCA). Varimax rotation was selected as this minimises the number of factors needed to explain each variable to obtain a clearer factorial structure. The rotated factor matrix was examined to identify the items that load on factors. Items with factor loadings > 0.40 have great practical relevance and define the factors properly [89] and were thus extracted. Previously, factorability was assessed through the Kaiser–Meyer–Olkin (KMO) test and Bartlett’s test of sphericity [90]. Based on the known groups technique, the responses were compared between the different psychotherapeutic orientations. Construct validity was also assessed with confirmatory factor analysis (CFA) to verify the factor structure of our sets of observed variables. Based on the recommendations by Bentler and Chou [91], who suggest at least five subjects for each free parameter, 69 subjects were considered adequate to test the single- and four-factor models of the scale with CFA. A maximum likelihood estimator of model parameters and a full maximum likelihood approach (FIML) [92] were used to manage missing data. We used several fit indices to assess the model fit (Table 1), as suggested by Kline [93].
Contrasting group validity was tested by comparing self-reported wine relationship items among participants with one-way ANOVA.
Internal consistency was assessed by determining the Cronbach’s alpha coefficient, corrected item-to-total score correlations using Pearson correlation coefficients and alpha estimation when an item was removed from the scale [98]. The criterion used for acceptable overall internal consistency was a value between 0.70 and 0.90 of Cronbach’s alpha, which is considered adequate for instruments used in research, whereas values above 0.90 may suggest redundancies in the scale [99]. The proportion of respondents with the lowest (1 point) or highest possible score (5 points) was calculated to examine the presence of floor and ceiling effects. Values higher than 20% were considered a moderate effect and higher than 50% as major [100].
Internal consistency reliability of the scale was estimated considering the following indices:
  • Greatest Lower Bound (glb) [101].
  • McDonald’s omega (ωt) [102];
  • Standardised Cronbach’s alpha (α) [103].
The results were considered statistically significant if the p-values were <0.05. Statistical analyses were performed using SPSS Statistics version 25, except for the CFA, which was performed with AMOS version 25.

3. Results

3.1. Content Validity

The three items considered by the judges as the most congruent with the proposed objectives were as follows: “therapist experience” with the complete agreement of the judges (Iik = 1), “patient’s expectation of cure” and “patient involvement” (both with Iik = 0.916). The variable “emotional well-being of the therapist” was eliminated as it did not reach the minimum level required for acceptance in the questionnaire (Iik = 0.16).
The remaining variables reached a level above the minimum level (Iik > 0.5), meaning that a total of 12 items were finally selected for inclusion in the questionnaire (Table 2).

3.2. The Overall Scale and Item Score Distribution

The mean overall score of the scale in the sample was 3.96 (SD = 0.46). All items displayed a short range of median values (3.0–5.0). The mean for each item is presented in Table 3, together with the floor and ceiling effects. Eight items presented a major ceiling effect, but their presence was justified due to the variation in the internal reliability if the items were removed.

3.3. Internal Consistency

The Cronbach’s alpha coefficient was 0.727 for the whole scale. This value is accepted due to the number of items [104,105]. Table 1 shows Cronbach’s alpha when an item was deleted.
The factors finally presented the following reliability indices: 0.669 on the first factor; 0.621 on the second factor; 0.757 on the third factor; and 0.348 on the fourth factor. No items were eliminated in the four subscales as no significant gains in reliability were obtained, and even the elimination of items with marginal loadings decreased the reliability. Thus, the estimated reliability for the subscales corresponding to the four factors was, taking into account the number of items, optimal for three of them, obtaining fairly acceptable alpha values for the first three factors. For the fourth factor, an alpha value of 0.348 was obtained, which is considered low. However, due to the small number of items, Cronbach’s alpha values of this magnitude are acceptable [104].
This can be proved if we equate the four subscales to a larger number of items (e.g., to 12, as in the full scale). Following the Spearman–Brown formula, in this case, we would obtain the following reliability indices for the four subscales: 0.83 for the first factor; 0.86 for the second; 0.95 for the third; and 0.76 for the fourth.

3.4. Inter-Rater Reliability

The ICC for the overall scores reached a value of 0.727 (95% confidence interval = 0.616–0.817; p = 0.000) for no interaction effect, and for each item, it ranged from 0.118 to 0.271. The results obtained indicate that the responses were very stable for both the full scale and all items.

3.5. Known Groups Technique

The difference between orientations was highly statistically significant (F = 2.44; p = 0.035). In addition, the cognitive-conduct orientation reached higher mean scores in every single item (except for item 11: therapist’s experience), and this difference was significant for all items (p < 0.05), except for items 3, 6, 8 and 10 (techniques or procedures used, therapist’s empathy, therapist’s perception of patient involvement and therapist’s degree of acceptance, interest, understanding and encouragement, respectively).

3.6. Exploratory Factor Analysis

The significance of Bartlett’s test of sphericity (X2 = 181.345; df = 66; p = 0.000) and the size of the KMO measure of sampling adequacy (KMO = 0.72) revealed a common variance of the items of the scale suitable for factor analysis [106]. The PCA and varimax rotation revealed four factors with eigenvalues exceeding 1 (Figure 1). The first component had an eigenvalue of 3.282, which explained 27.351% of the variance, the second one had an eigenvalue of 1.685, explaining 14.039% of the variance, the third one had an eigenvalue of 1.34, explaining an additional 11.166% of the variance, and the fourth one had an eigenvalue of 1.216, explaining 10.136% of the variance. The rotated factor matrix was examined to identify items that loaded on these factors (Table 4).

3.7. Confirmatory Factor Analysis

The four-factor model (Figure 2) showed the following fit indices: χ2(84) = 48.47, p = 0.000; CFI = 0.995; TLI = 0.992; RMSEA = 0.012 (90% CI = 0.000–0.080); SRMR = 0.043. The correlations among the four factors also indicated a high percentage of variance shared by the factors. The loadings of the selected model are presented in Table 3.

4. Discussion

The present study developed and validated a questionnaire that allows for detecting the most responsible variables for the process of therapeutic change. The idea is to try to help therapists to establish strategies to improve the success of their treatments. To this end, the aim is to provide an instrument for therapists, through its administration, to find out their attributional biases in terms of the importance they attach to the specific factors of the therapies they practice (techniques and procedures used) and to the common factors of the therapeutic process (variables involving the patient, the therapist and the interaction between them). Once the attributed errors have been detected, the therapist can focus on enhancing the common skills and attitudes that, according to psychotherapy research, are responsible for greater effectiveness in a psychotherapeutic intervention [11,19,20,22].
Firstly, it can be confirmed that there is an adequate level of content validity of the core variables that make up the questionnaire, corresponding to 12 items (Iik > 0.5), according to [83] the congruence index. The 12 judges, experts in psychotherapeutic processes and clinical or health psychology, assessed the congruence between each indicator and the domain it measures. After a thorough review of the literature, 13 items were initially selected. All items obtained adequate levels of congruence, except for “emotional well-being of the therapist”, which was eliminated. It did not reach the minimum level required for acceptance in the final questionnaire (Iik = 0.16).
Similarly, the questionnaire presents adequate levels of reliability, showing a Cronbach alpha of 0.727 for the total scale, meeting the criterion of being between the range of 0.7 and 0.9 [99]. Moreover, it should be taken into account that it is a scale with a small number of items, meaning that such an internal consistency is considered more than satisfactory for that number of items [105] and would correspond, following the Spearman–Brown attenuation formula, to an alpha value of ≈ 0.9, for an instrument with a number of items equal to 36.
However, in the analysis of the discriminatory capacity of each item, it was observed that several do not meet the criteria established between the total score and the item by means of the Pearson correlation coefficients, identifying ceiling percentages of over 20% in a total of 10 items, with 3 items of these with scores of over 50% and thus showing a more significant effect. If we eliminate some of the items, the reliability of the total scale is lost.
Concerning Cronbach’s alpha and the criticism that it is the best parameter to determine internal consistency [107], two more scores were calculated: the greatest lower bound (GLB) coefficient [101] and the coefficient ω [102]. In general, they scored very close to 1, which is an adequate level of reliability, with a difference between these coefficients of only 0.10. Despite this, and according to the estimates established on when it is appropriate to use one or the other coefficient [108], we assume values of ω = 0.79 and α = 0.71, as these two coefficients are more appropriate when the total test scores are normally distributed, with ω even being the first option, followed by α, as they avoid the problems of overestimation presented by the GLB coefficient. The last coefficient is recommended when the proportion of asymmetric items is high, whatever the sample size.
As for construct validity, first, all tests of suitability for factor analysis, the KMO (0.72), Bartlett’s test of sphericity (p < 0.05) and the determinant matrix (0.042), were met. The exploratory analysis confirmed four factors that explain 62.7% of the variance with adequate levels of reliability. Therefore, our research shows that the questionnaire, elaborated to elucidate the attributions of effectiveness in the psychotherapeutic process, can be divided into four dimensions, related to the specific variables of psychotherapy and the common factors (or non-specific variables) [1,4].
As determined by the previous literature [42], the variables specific to psychotherapy refer to the idiosyncratic characteristics of each therapy modality, consisting mainly of the therapeutic approach and the techniques and procedures used. These items explain 11.17% of the variance.
The importance of these common factors in terms of their contribution to the effectiveness of therapy should be underlined [109]. Research even attributes 85% of the change in all psychotherapy modalities to these factors [24]. In fact, these three types of factors explain 51.53% of the variance in our scale.
Among all the common variables, it has been shown in the literature that interaction and therapeutic alliance are among those that contribute most to the effectiveness of psychological treatments [28,38]. Therefore, this dimension was initially included with only one item, “establishment of therapeutic alliance”, but the internal structure of the questionnaire through exploratory and confirmatory analysis unifies this item in a first factor that also includes “patient’s expectation of cure”, “patient’s faith and credibility assigned to the therapist”, “therapist’s perception of the patient’s involvement” and “therapist’s ability to influence the patient”. Therefore, a factor is found which is composed of two items referring to common therapist variables, two items referring to common patient variables and an item referring to the therapist–patient relationship. Considering the theoretical framework in which the variables are inserted, this factor would group the enhancers of the therapeutic alliance, that is, common variables of a perceptual order that favour such an alliance, and which are mainly related to the expectations or perceptions that both the therapist has of the patient and vice versa, and the interaction between the two.
On the other hand, it is confirmed that the common variables specific to the therapist are formed as one factor, including the “empathy shown by the therapist”, the “directivity and support shown by the therapist” and the “degree of acceptance, interest, understanding and encouragement shown by the therapist to the patient”, with strong emotional content. Similarly, the two items referring to the variables specific to psychotherapy, “therapeutic approach used” and “techniques or procedures used”, are included in a single factor.
Finally, one factor includes “therapist experience” and “patient’s involvement in the therapy”, with the first item corresponding to a common therapist variable and the second to a common patient variable. Thus, this factor would include two important common variables facilitating patient adherence to therapy and pointing to significant agents of change in the therapeutic process. Although the therapist’s experience has been a variable that has generated controversy in the literature, in the sense of whether it is an agent of change that acts directly or through the improvement it produces in other factors [78], the patient’s involvement in the treatments has been shown to be a direct agent of success in the intervention [50,52].
Exploratory and confirmatory analyses identified therapy-specific variables and common patient, therapist and interaction variables but divided into four different factors as follows: (1) therapeutic alliance enhancers; (2) psychotherapist emotional characteristics; (3) therapy-specific variables, and (4) facilitators of patient engagement with therapy. Consequently, the resulting 12-item scale, the Psychotherapeutic Effectiveness Attribution Questionnaire (PEAQ-12), is considered to be an enriching contribution to the study of psychotherapists for research and the development of training programmes.
In summary, it would be interesting to further investigate the validity of this questionnaire, its internal structure and its dimensions, using larger samples and in populations with varied psychotherapeutic orientations. It should be emphasised that the results should be interpreted with caution as the statistical power of confirmatory analysis with such a small sample may be limited. However, the results of this analysis have been included because they may be of value considering the pilot nature of the study. In addition, it would be a great contribution to associate, through this questionnaire, the attributions that therapists establish and the percentage of effect of the treatment on their patients. In fact, in this study, the interpretation of statistical results has been limited to what the therapists consider effective elements of psychotherapy and not to the factors that make the treatment effective, as the previous literature has already very extensively addressed the real effect of the different psychotherapeutic variables.

Author Contributions

Conceptualization, A.R.-M.; methodology, validation and formal analysis, A.P. and D.G.-C.; investigation, writing—original draft preparation, A.M.-C., A.Z. and S.J.C.-M.; writing—review and editing, A.R.-M.; supervision, A.R.-M. and A.Z.; funding acquisition, S.J.C.-M. and D.G.-C. All authors have read and agreed to the published version of the manuscript.

Funding

This publication and research has been funded by the Program for the Promotion and Promotion of Research and Transfer activity of the University of Cádiz and by the Institute of Research and Development Social and Sustainability (INDESS).

Institutional Review Board Statement

The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Institutional Review Board (or Ethics Committee) of the UNIVERSITY OF CÁDIZ (protocol code 189). The Bioethics Committee of the University of Cadiz extends its competence to the field of research that uses data of a personal nature for the purposes foreseen in sub-section d), paragraph 1, of article 1, subject to reserved treatment due to affecting the rights and freedoms of individuals, the interests linked to the defense and protection of the environment or other constitutionally and legally protected assets.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The data presented in this study are available on request from the corresponding author. The data are not publicly available due to privacy issues.

Conflicts of Interest

The authors declare no conflict of interest.

Appendix A

You will find some statements below. Read each statement carefully and think how those statements influence the improvement of a patient undergoing psychotherapy.
Please, check each item according to the following: 1 is “no influence at all” and 5 is “great influence”).
Table
Table A1. Psychotherapeutic Effectiveness Attribution Questionnaire (PEAQ-12).
Table A1. Psychotherapeutic Effectiveness Attribution Questionnaire (PEAQ-12).
1 Therapeutic approach used (the theoretical orientation on which the psychotherapy is based)12345
2 Techniques or procedures used12345
3 Patient’s expectations of cure12345
4 Patient’s involvement in the therapy12345
5 Patient’s faith and credibility in the therapist12345
6 Empathy shown by the therapist12345
7 Directivity and support shown by the therapist12345
8 Therapist’s perception of the patient’s involvement12345
9 Therapist’s ability to influence the patient12345
10 Degree of acceptance, interest, understanding and encouragement shown by the therapist to the patient12345
11 Experience of the therapist12345
12 Establishment of a therapeutic alliance between the therapist and patient12345

References

  1. Castillo, J.A.; Poch, J. La Efectividad de la Psicoterapia: Método y Resultados de la Investigación; Hogar del Libro: Madrid, Spain, 1991. [Google Scholar]
  2. Garske, J.P.; Jay Linn, S. Hacia un esquema general de la psicoterapia: Eficacia, factores comunes e integración. In Psicoterapias Contemporáneas; Jay Linn, S., Garske, J.P., Eds.; Desclée de Brouwer: Bilbao, Spain, 1988; pp. 623–647. [Google Scholar]
  3. Opazo, R. Psicoterapia Integrativa. Delimitación Clínica; Ediciones ICPSI: Providencia Santiago, Chile, 2001. [Google Scholar]
  4. Uribe, M. Factores comunes e integración de las psicoterapias. Rev. Colomb. Psiquiatr. 2008, 37, 14–28. [Google Scholar]
  5. Frank, J.D. Therapeutic components shared by all psychothe–rapies. In Psychotherapy Research and Behavior Change; Harvey, J.H., Parks, M.M., Eds.; American Psychological Association: Washington, DC, USA, 1982; pp. 9–37. [Google Scholar]
  6. Frances, A.; Sweeney, J.; Clarkin, J. Do Psychotherapies have Specific Effects? Am. J. Psychother. 1985, 39, 159–174. [Google Scholar] [CrossRef]
  7. Garfield, S.L. Psychotherapy: A 40-year appraisal. Am. Psychol. 1981, 35, 174–183. [Google Scholar] [CrossRef]
  8. Critelli, J.W.; Neumann, K.F. The placebo: Conceptual analysis of a construct in transition. Am. Psychol. 1984, 39, 32–39. [Google Scholar] [CrossRef] [PubMed]
  9. Lambert, M.J.; Shapiro, D.A.; Bergin, A.E. The effectiveness of psychotherapy. In Handbook of Psychotherapy and Behavior Change, 3rd ed.; Garfield, S., Bergin, A., Eds.; Wiley: Hoboken, NJ, USA, 1986; pp. 157–212. [Google Scholar]
  10. Germer, C.K.; Siegel, R.D.; Fulton, P.R. Mindfulness and Psychotherapy; Guilford Press: New York, NY, USA, 2005. [Google Scholar]
  11. Lambert, M.J.; Ogles, B.M. The efficacy and effectiveness of psychotherapy. In Bergin and Garfield’s Handbook of Psychotherapy and Behavior Change, 5th ed.; Lambert, M.J., Ed.; Wiley: Hoboken, NJ, USA, 2004; pp. 139–193. [Google Scholar]
  12. Norcross, J.C. Psychotherapy Relationships that Work: Therapist Contributions and Responsiveness to Patients; Oxford University Press: Oxford, UK, 2002. [Google Scholar]
  13. Wampold, B.E. The Great Psychotherapy Debate: Models, Methods, and Findings; Lawrence Erlbaum Associates: Mahwah, NJ, USA, 2001. [Google Scholar]
  14. Duncan, B.L. On Becoming a Better Therapist; American Psychological Association: Washington, DC, USA, 2010. [Google Scholar]
  15. Cuijpers, P.; Driessen, E.; Hollon, S.D.; van Oppen, P.; Barth, J.; Andersson, G. The efficacy of non-directive supportive therapy for adult depression: A meta-analysis. Clin. Psychol. Rev. 2012, 32, 280–291. [Google Scholar] [CrossRef] [PubMed]
  16. Caro, I. Psychotherapy integration: A main obstacle and challenge. Ter. Psicológica 2018, 36, 177–180. [Google Scholar] [CrossRef] [Green Version]
  17. Karson, M.; Fox, J. Common skills that underlie the common factors of successful psychotherapy. Am. J. Psychother. 2010, 64, 269–281. [Google Scholar] [CrossRef]
  18. Nathan, P.; Stuart, S.; Dolan, S. Research on psychotherapy, efficacy and effectiveness: Between Scylla and Charybdis? Psychol. Bull. 2000, 126, 964–981. [Google Scholar] [CrossRef]
  19. Norcross, J.C. A primer on psychotherapy integration. In Handbook of Psychotherapy Integration, 2nd ed.; Norcross, J.C., Goldfried, M.R., Eds.; Oxford University Press: Oxford, UK, 2005; pp. 3–23. [Google Scholar]
  20. Safran, J.D.; Muran, J.C. Negotiating the Therapeutic Alliance: A Relational Treatment Guide; Guilford Press: New York, NY, USA, 2000. [Google Scholar]
  21. Samstag, L.; Muran, J.C.; Safran, J.D. Defining and identifying ruptures in psychotherapy. In Core Concepts in Brief Dynamic Therapy: Training for Effectiveness; Charman, D., Ed.; Lawrence Erlbaum Associates: Mahwah, NJ, USA, 2004. [Google Scholar]
  22. Wampold, B.E. What should we practice? A contextual model for how psychotherapy works. In The Cycle of Excellence: Using Deliberate Practice to Improve Supervision and Training; Rousmaniere, T.G., Goodyear, R.K., Miller, S.D., Wampold, B.E., Eds.; Wiley Publishers: Hoboken, NJ, USA, 2017; pp. 49–65. [Google Scholar]
  23. Asay, T.P.; Lambert, M.J. The Empirical Case for the Common Factors in Therapy: Quantitative Findings; American Psychological Association: Washington, DC, USA, 1999. [Google Scholar]
  24. Lambert, M.J.; Bergin, A.E. Achievements and Limitations of Psychotherapy Research; American Psychological Association: Washington, DC, USA, 1992. [Google Scholar]
  25. Wampold, B.E. How important are the common factors in psychotherapy? An update. World Psychiatry. 2015, 14, 270–277. [Google Scholar] [CrossRef] [Green Version]
  26. Araya-Véliz, C.; Porter, B. Habilidades del terapeuta y mindfulness. Rev. Argent. Clínica Psicológica 2017, 26, 232–240. [Google Scholar]
  27. Eubanks, C.F.; Goldfried, M.R. A principle-based approach to psychotherapy integration. In Handbook of Psychotherapy Integration, 3rd ed.; Norcross, J.C., Goldfried, M.R., Eds.; Oxford University Press: Oxford, UK, 2019; pp. 88–104. [Google Scholar]
  28. Jiménez, J.P. El vínculo, las intervenciones técnicas y el cambio terapéutico en terapia psicoanalítica. Apert. Psicoanalíticas 2005, 2, 1–19. [Google Scholar]
  29. Orlinsky, D.; Grawe, K.; Parks, K. Process and outcome in psychotherapy. In Handbook of Psychotherapy and Behavior Change; Bergin, L., Garfield, L., Eds.; John Wiley and Sons: Hoboken, NJ, USA, 1994; pp. 270–376. [Google Scholar]
  30. Gaston, L.; Marmar, C.R.; Gallager, D.; Thompson, L.W. Alliance prediction of outcome beyond in treatment symptomatic change as psychotherapy progress. Psychother. Res. 1991, 1, 104–112. [Google Scholar] [CrossRef]
  31. Luborsky, L.; Barber, J.; Crits-Christoph, P. Testing psychoanalytic propositions about personality change in psychotherapy. In Interface of Psychoanalysis and Psychology; Barron, J.W., Eagle, M.N., Wolitzky, D.L., Eds.; American Psychological Association: Washington, DC, USA, 1992; pp. 573–585. [Google Scholar] [CrossRef]
  32. Safran, J.D.; Segal, Z.V. El proceso Interpersonal en la Terapia Cognitiva; Ediciones Paidós: Barcelona, Spain, 1994. [Google Scholar]
  33. Agnew-Davies, R.; Stiles, W.B.; Hardy, G.E.; Barkham, M.; Shapiro, D.A. Alliance structure assessed by the Agnew Relationship Measure (ARM). Br. J. Clin. Psychol. 1998, 37, 155–172. [Google Scholar] [CrossRef]
  34. Romero-Moreno, A.F. Psicoterapia: Investigación del Resultado y del Proceso de la Psicoterapia y Atribuciones Sobre su Efectividad; Ediciones Absalon: Barcelona, Spain, 2010. [Google Scholar]
  35. Horvarth, A.O.; Del Re, A.C.; Flückinger, C.; Symonds, D. Alliance in individual Psychotherapy. In Psychotherapy Relationships that Work: Evidence-Based Responsiveness, 2nd ed.; Norcross, J.C., Ed.; Oxford University Press: Oxford, UK, 2011; pp. 25–69. [Google Scholar]
  36. Norcross, J.C.; Lambert, M.J. Psychotherapy relationships that work III. Psychotherapy 2018, 55, 303–315. [Google Scholar] [CrossRef]
  37. Norcross, J.C.; Lambert, M.J. Psychotherapy Relationships that Work: Volume 1: Evidence-Based Therapist Contribution; Oxford University Press: Oxford, UK, 2019. [Google Scholar]
  38. Norcross, J.C.; Wampold, B.E. Evidence-based therapy relationships: Research conclusions and clinical practices. In Psychotherapy Relationships that Work: Evidence-Based Responsiveness, 2nd ed.; Norcross, J.C., Ed.; Oxford University Press: Oxford, UK, 2011; pp. 423–430. [Google Scholar]
  39. Norcross, J.C.; Wampold, B.E. A new therapy for each patient: Evidence-based relationships and responsiveness. J. Clin. Psychol. 2018, 74, 1889–1906. [Google Scholar] [CrossRef] [PubMed]
  40. Norcross, J.C.; Wampold, B.E. Relationships and responsiveness in the psychological treatment of trauma: The tragedy of the APA Clinical Practice Guideline. Psychotherapy 2019, 56, 391–399. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  41. Wampold, B.E.; Imel, Z.E. The Great Psychotherapy Debate: The Evidence for what Makes Psychotherapy Work; Routledge: Abingdon, UK, 2015. [Google Scholar]
  42. Rodríguez-Morejón, A. Manual de Psicoterapias. Teoría y Técnicas; Herder Editorial: Barcelona, Spain, 2019. [Google Scholar]
  43. Craig, A.; Andrews, G. The prediction and prevention of relapse in stuttering: The value of self-control techniques and Locus of Control measures. Behav. Modif. 1985, 9, 427–442. [Google Scholar] [CrossRef] [PubMed]
  44. Garfield, S.L. Research on client variables in psychotherapy. In Handbook of Psychotherapy and Behavior Change, 4th ed.; Garfield, S.L., Bergin, A.E., Eds.; Wiley: Hoboken, NJ, USA, 1994. [Google Scholar]
  45. Trull, T.; Jerry, P. Psicología Clínica: Conceptos, Métodos y Aspectos Prácticos de la Profesión; International Thomson Editores, S.A.: Ciudad de México, Mexico, 2003. [Google Scholar]
  46. Constantino, M.J.; Arnkoff, D.B.; Glass, C.R.; Ametrano, R.M.; Smith, J.Z. Expectations. J. Clin. Psychol. 2011, 67, 184–192. [Google Scholar] [CrossRef] [PubMed]
  47. Lambert, M.J.; Anderson, E.M. Assessment for the time-limited psychotherapies. Am. Psychiatr. Press Rev. Psychiatry 1996, 15, 23–42. [Google Scholar]
  48. Leventhal, H.; Zimmerman, R.; Gutmann, M. Compliance: A self-regulation perspective. In Handbook of Behavioral Medicine; Gentry, W.D., Ed.; Guilford: New York, NY, USA, 1984; pp. 369–436. [Google Scholar]
  49. Greenberg, L.S.; Pinsof, W.M. Research strategies: Current trends and future perspectives. In The Psychotherapeutic. Process: A Research Handbook; Greenberg, L.S., Pinsof, W.M., Eds.; Guilford Press: New York, NY, USA, 1986; pp. 3–20. [Google Scholar]
  50. Bohart, A.C.; Tallman, K. Clients: The neglected common factor in psychotherapy. In The Heart and Soul of Change: Delivering what Works in Therapy, 2nd ed.; Duncan, B., Miller, S., Wampold, B., Hubble, M., Eds.; American Psychological Association: Washington, DC, USA, 2010; pp. 83–111. [Google Scholar]
  51. Orlinsky, D.E.; Ronnestad, M.H.; Willutzki, U. Fifty years of psychotherapy process-outcome research: Continuity and change. In Bergin and Garfield’s Handbook of Psychotherapy and Behavior Change, 5th ed.; Lambert, M., Ed.; Wiley: Hoboken, NJ, USA, 2004; pp. 307–390. [Google Scholar]
  52. Rodríguez-Morejón, A. El cliente en psicoterapia: Contribución al resultado terapéutico. An. Psicol. Ann. Psychol. 2016, 32, 1–8. [Google Scholar] [CrossRef] [Green Version]
  53. Krebs, P.; Norcross, J.C.; Nicholson, J.M.; Prochaska, J.O. Stages of change and psychotherapy outcomes: A review and meta-analysis. J. Clin. Psychology. 2018, 74, 1964–1979. [Google Scholar] [CrossRef]
  54. Hofman, S.G.; Weinberger, J. The Art and Science of Psychotherapy; Taylor & Francis Group: Abingdon, UK, 2007. [Google Scholar]
  55. Castonguay, L.G.; Hill, C.E. How and why are Some Therapists Better than Others?: Understanding Therapist Effects; American Psychological Association: Washington, DC, USA, 2017; p. XV-356. [Google Scholar]
  56. Kolden, G.G.; Klein, M.H.; Wang, C.C.; Austin, S.B. Congruence/genuineness. Psychotherapy 2011, 48, 65. [Google Scholar] [CrossRef]
  57. Santibáñez, P.M.; Román, M.F.; Lucero, C.; Espinoza, A.E.; Irribarra, D.E.; Müller, P. Variables inespecíficas en psicoterapia. Ter. Psicológica 2008, 26, 89–98. [Google Scholar] [CrossRef]
  58. Bohart, A.C.; Elliott, R.; Greenberg, L.S.; Watson, J.C. Empathy. Psychotherapy 2011, 48, 43. [Google Scholar]
  59. Elliott, R.; Bohart, A.C.; Watson, J.C.; Murphy, D. Therapist empathy and client outcome: An updated meta-analysis. Psychotherapy 2018, 55, 399–410. [Google Scholar] [CrossRef]
  60. Lafferty, P.; Beutler, L.E.; Crago, M. Differences between more and less effective psychotheapists: A study of select therapist variables. J. Consult. Clin. Psychol. 1989, 57, 76–80. [Google Scholar] [CrossRef]
  61. Watson, J. Role of the therapeutic relationship in emotion-focused therapy. In Clinical Handbook of Emotion-Focused Therapy; Greenberg, L.S., Goldman, R.N., Eds.; American Psychological Association: Washington, DC, USA, 2019; pp. 111–128. [Google Scholar] [CrossRef]
  62. Corning, A.F.; Malofeeva, E.V.; Bucchianeri, M.M. Predicting termination type from client-therapist agreement on the severity of the presenting problem. Psychother. Theory Res. Pract. Train. 2007, 44, 193–204. [Google Scholar] [CrossRef]
  63. Rondón, A.P.; Otálora, I.L.; Salamanca, Y. Factores que influyen en la deserción terapéutica de los consultantes de un centro universitario de atención psicológica. Int. J. Psychol. Res. 2009, 2, 137–147. [Google Scholar] [CrossRef] [Green Version]
  64. Farber, B.A.; Doolin, E.M. Positive regard. Psychotherapy 2011, 48, 58–64. [Google Scholar] [CrossRef] [PubMed]
  65. Krause, M. Psicoterapia y Cambio: Una Mirada Desde la Subjetividad; Dirección de Investigación y Postgrado de la Pontificia Universidad Católica de Chile: Santiago, Chile, 2005. [Google Scholar]
  66. Bados López, A.; García Grau, E. Habilidades terapéuticas; Universitat de Barcelona: Barcelona, Spain, 2011. [Google Scholar]
  67. Karmo, M.; Beutler, L.; Harwood, T. Interactions between psychotherapy procedures and patient attributes that predict alcohol treatment effectiveness: A preliminary report. Addict. Behav. 2002, 27, 779–797. [Google Scholar] [CrossRef]
  68. Keijsers, G.; Schaap, C.; Hoodgduin, C.; Lammers, M. Patient-therapist interaction in the behavioral treatment of panic disorder with agoraphobia. Behav. Modication 1995, 19, 491–517. [Google Scholar] [CrossRef] [PubMed]
  69. Urzúa, A.; Arbach, K.; Lourido, M.; Mas, N. La directividad en la relación terapéutica: Análisis de la evidencia encontrada en investigación. Ter. Psicológica 2010, 28, 95–107. [Google Scholar] [CrossRef] [Green Version]
  70. Fernández-González, O.M.; Herrera-Salinas, P.; Escobar-Martínez, M.J. Adolescentes en psicoterapia: Su representación de la relación terapéutica. Rev. Latinoam. De Cienc. Soc. Niñez Juv. 2016, 14, 559–575. [Google Scholar]
  71. Green, J. Annotation: The therapeutic alliance–a significant but neglected variable in child mental health treatment studies. J. Child Psychol. Psychiatry 2006, 47, 425–435. [Google Scholar] [CrossRef]
  72. Karver, M.S.; Handelsman, J.B.; Fields, S.; Bickman, L. Meta-analysis of therapeutic relationship variables in youth and family therapy: The evidence for different relationship variables in the child and adolescent treatment outcome literature. Clin. Psychol. Rev. 2006, 26, 50–65. [Google Scholar] [CrossRef]
  73. Páramo, M. Psicoterapia para adolescentes y perfil del psicólogo: Análisis de contenido de discursos grupales. Rev. Argent. Clínica Psicológica 2011, 20, 133–142. [Google Scholar]
  74. Beutler, L.E.; Crago, M.; Arizmendi, T.G. Therapist variables in psychotherapy process and outcome. In Handbook of Psychotherapy and Behavior Change; Wiley: Hoboken, NJ, USA, 1986; Volume 3, pp. 257–310. [Google Scholar]
  75. Feixas, G.; Miró, M.T. Aproximaciones a la Psicoterapia. Una Introducción a los Tratamientos Psicológicos; Paidós: Barcelona, Spain, 1993. [Google Scholar]
  76. LaCrosse, M.B. Perceived counselor social influence and counseling outcomes: Validity of the Counselor Rating Form. J. Couns. Psychol. 1980, 27, 320–327. [Google Scholar] [CrossRef]
  77. Valdés-Miyar, M. La Arquitectura de la Psiquiatría; Plataforma: Barcelona, Spain, 2016. [Google Scholar]
  78. Shelef, K.; Diamond, G.M.; Diamond, G.S.; Liddle, H.A. Adolescent and Parent Alliance and Treatment Outcome in Multidimensional Family Therapy. J. Consult. Clin. Psychol. 2005, 73, 689–698. [Google Scholar] [CrossRef] [Green Version]
  79. Botella, L.; Feixas, G. Eficacia de la psicoterapia: Investigaciones de resultados. In Psicoterapia: Modelos Contemporáneos y Aplicaciones; Garrido, M., Ed.; Promolibro: Valencia, Spain, 1994; pp. 91–104. [Google Scholar]
  80. Beitman, B.D. The Structure of Individual Psychotherapy; Guilford Press: New York, NY, USA, 1987. [Google Scholar]
  81. Neff, K. The Science of self-compassion. In Compassion and Wisdom in Psychotherapy; Siegel, C.G.R., Ed.; Guilford Press: New York, NY, USA, 2012. [Google Scholar]
  82. Araya-Véliz, C.; Moncada, L. Auto-compasión: Origen, concepto y evidencias preliminares. Rev. Argent. Clínica Psicológica 2016, 25, 67–78. [Google Scholar]
  83. Osterlind, S.J. Constructing Test Items; Kluwer Academic Publisher: Dordrecht, The Netherlands, 1989. [Google Scholar]
  84. Lilliefors, H.W. On the Kolmogorov-Smirnov test for normality with mean and variance unknown. J. Am. Stat. Assoc. 1967, 62, 399–402. [Google Scholar] [CrossRef]
  85. Hambleton, R.K. Test score validity and standard-setting methods. Criterion-Ref. Meas. State Art 1980, 8, 123. [Google Scholar]
  86. Rovinelli, R.J.; Hambleton, R.K. The use of content specialists in the assessment of criterion-referenced test item validity. Dutch J. Educ. Res. 1977, 2, 49–60. [Google Scholar]
  87. de Winter, J.C.; Dodou, D.I.M.I.T.R.A.; Wieringa, P.A. Exploratory factor analysis with small sample sizes. Multivar. Behav. Res. 2009, 44, 147–181. [Google Scholar] [CrossRef] [PubMed]
  88. Abdi, H.; Williams, L.J. Principal component analysis. Wiley Interdiscip. Rev. Comput. Stat. 2010, 2, 433–459. [Google Scholar] [CrossRef]
  89. Hair, J.; Anderson, R.E.; Tatham, R.L.; Black, W.C. Multivariate Data Analysis, 4th ed.; Prentice-Hall: Hoboken, NJ, USA, 1995. [Google Scholar]
  90. Williams, B.; Onsman, A.; Brown, T. Exploratory factor analysis: A five-step guide for novices. Australas. J. Paramed. 2010, 8, 1–13. [Google Scholar] [CrossRef] [Green Version]
  91. Bentler, P.M.; Chou, C.P. Practical issues in structural modeling. Sociol. Methods Res. 1987, 16, 78–117. [Google Scholar] [CrossRef]
  92. Arbuckle, J.L.; Marcoulides, G.A.; Schumacker, R.E. Full information estimation in the presence of incomplete data. Adv. Struct. Equ. Modeling Issues Tech. 1996, 243, 277. [Google Scholar]
  93. Kline, R.B. Principles and Practice of Structural Equation Modeling; Guilford Publications: New York, NY, USA, 2015. [Google Scholar]
  94. Bentler, P.M. Comparative fit indexes in structural models. Psychol. Bull. 1990, 107, 238. [Google Scholar] [CrossRef]
  95. Tucker, L.R.; Lewis, C. A reliability coefficient for maximum likelihood factor analysis. Psychometrika 1973, 38, 1–10. [Google Scholar] [CrossRef]
  96. Hu, L.T.; Bentler, P.M. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct. Equ. Modeling Multidiscip. J. 1999, 6, 1–55. [Google Scholar] [CrossRef]
  97. Jöreskog, K.G.; Sörbom, D. LISREL 8: User’s Reference Guide; Scientific Software International: Skokie, IL, USA, 1996. [Google Scholar]
  98. Netemeyer, R.G.; Bearden, W.O.; Sharma, S. Scaling Procedures: Issues and Applications; Sage: Thousand Oaks, CA, USA, 2003. [Google Scholar]
  99. DeVellis, R.F. Scale Development: Theory and Applications; Sage: Thousand Oaks, CA, USA, 2003. [Google Scholar]
  100. Parent, E.C.; Dang, R.; Hill, D.; Mahood, J.; Moreau, M.; Raso, J.; Lou, E. Score distribution of the scoliosis research society-22 questionnaire in subgroups of patients of all ages with idiopathic scoliosis. Spine 2010, 35, 568–577. [Google Scholar] [CrossRef] [PubMed]
  101. Woodhouse, B.; Jackson, P.H. Lower bounds for the reliability of the total score on a test composed of non-homogeneous items: II: A search procedure to locate the greatest lower bound. Psychometrika 1977, 42, 579–591. [Google Scholar] [CrossRef]
  102. McDonald, R.P.; Marsh, H.W. Choosing a multivariate model: Noncentrality and goodness of fit. Psychol. Bull. 1990, 107, 247. [Google Scholar] [CrossRef]
  103. Cronbach, L.J. Coefficient alpha and the internal structure of tests. Psychometrika 1951, 16, 297–334. [Google Scholar] [CrossRef] [Green Version]
  104. Pedhazur, E.J.; Schmelkin, L.P. Measurement, Design, and Analysis: An Integrated Approach; Lawrence Erlbaum Associates: Mahwah, NJ, USA, 1991. [Google Scholar]
  105. Field, A. Discovering Statistics Using IBM SPSS Statistics; Sage: Thousand Oaks, CA, USA, 2013. [Google Scholar]
  106. Tabachnick, B.G.; Fidell, L.S. Using Multivariate Statistics, 4th ed.; Allyn & Bacon: Boston, MA, USA, 2001. [Google Scholar]
  107. Cho, E.; Kim, S. Cronbach’s coefficient alpha: Well known but poorly understood. Organ. Res. Methods 2015, 18, 207–230. [Google Scholar] [CrossRef]
  108. Trizano-Hermosilla, I.; Alvarado, J.M. Best alternatives to Cronbach’s alpha reliability in realistic conditions: Congeneric and asymmetrical measurements. Front. Psychol. 2016, 7, 769. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  109. Wachtel, P.L.; Siegel, J.P.; Baer, J.C. The Scope of Psychotherapy Integration: Introduction to a Special Issue. Clin. Soc. Work J. 2020, 48, 231–235. [Google Scholar] [CrossRef]
Ijerph 18 10372 g001 550
Figure 1. Sedimentation graph of factor components.
Figure 1. Sedimentation graph of factor components.
Ijerph 18 10372 g001
Ijerph 18 10372 g002 550
Figure 2. The four-factor model diagram with standardised estimates.
Figure 2. The four-factor model diagram with standardised estimates.
Ijerph 18 10372 g002
Table
Table 1. Fit indices suggested for the CFA.
Table 1. Fit indices suggested for the CFA.
Chi-Square SignificanceComparative Fit Index (CFI)Tucker–Lewis Index (TLI or NNFI)Root Mean Square Error of Approximation (RMSEA)Standardised Root Mean Square Residual (SRMR)
If chi-square was not significant, the model had reached a perfect fit with the observed data.Values ≥ 0.90 indicated a good fit [94].Values ≥ 0.90 indicated a good fit [95].Values ≤ 0.05 or
0.08 indicated a good fit, such as the upper bound of 90% confidence interval [96].		Values ≤ 0.05 indicated a good fit [97].
Table
Table 2. Expert judgement. Congruence indices based on the objective congruence method (12 experts).
Table 2. Expert judgement. Congruence indices based on the objective congruence method (12 experts).
Ijk
The therapeutic approach used0.75
The techniques or procedures used0.83
The patient’s expectations of cure0.916
The patient’s involvement in the therapy0.916
The patient’s faith and credibility assigned to the therapist0.66
Emotional well-being of the therapist0.16 (removed)
The empathy shown by the therapist0.66
The directivity and support shown by the therapist0.75
The therapist’s perception of the patient’s involvement0.58
The therapist’s ability to influence the patient0.83
The degree of acceptance, interest, understanding and encouragement shown by the therapist to the patient0.83
The experience of the therapist1
The establishment of a therapeutic alliance between the therapist and patient0.66
Table
Table 3. Position of items with score means and standard deviations and reliability results.
Table 3. Position of items with score means and standard deviations and reliability results.
MeanSDCronbach’ Alpha if Item DeletedFloor Effect “1 Point” (%)Ceiling Effect “5 Points” (%)
1 Therapeutic approach used3.791.0030.7232.927.5
2 Techniques or procedures used4.110.9350.7291.536.8
3 Patient’s expectations of cure3.980.9070.6901.636.8
4 Patient’s involvement in the therapy4.590.6630.7080.066.7
5 Patient’s faith and credibility assigned to the therapist3.710.9060.6980.021.7
6 Empathy shown by the therapist4.160.8070.6961.940.6
7 Directivity and support shown by the therapist3.591.0260.7013.019.4
8 Therapist’s perception of the patient’s involvement3.411.0260.6764.416.2
9 Therapist’s ability to influence the patient4.160.9020.6881.541.8
10 Degree of acceptance, interest, understanding and encouragement shown by the therapist to the patient3.980.9420.7201.436.2
11 Experience of the therapist4.250.8610.7400.050
12 Establishment of a therapeutic alliance between the therapist and patient4.350.8060.731156.5
Table
Table 4. Rotated component matrix of PCA.
Table 4. Rotated component matrix of PCA.
Component
1234
1 The patient’s faith and credibility assigned to the therapist0.755
2 The therapist’s ability to influence the patient0.736
3 The patient’s expectations of cure0.638
4 The therapist’s perception of the patient’s involvement 0.614
5 The establishment of a therapeutic alliance between the therapist and patient0.529
6 The empathy shown by the therapist 0.777
7 The directivity and support shown by the therapist 0.762
8 The degree of acceptance, interest, understanding and encouragement shown by the therapist to the patient 0.745
9 The techniques or procedures used 0.911
10 The therapeutic approach used 0.868
11 The experience of the therapist 0.792
12 The patient’s involvement in the therapy 0.728
Extraction method: principal component analysis. Rotation method: varimax with Kaiser normalisation. a. Rotation converged in 5 iterations.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Romero-Moreno, A.; Paramio, A.; Cruces-Montes, S.J.; Zayas, A.; Gómez-Carmona, D.; Merchán-Clavellino, A. Development and Validation of the Psychotherapeutic Effectiveness Attribution Questionnaire (PEAQ-12) in a Spanish Population. Int. J. Environ. Res. Public Health 2021, 18, 10372. https://doi.org/10.3390/ijerph181910372

AMA Style

Romero-Moreno A, Paramio A, Cruces-Montes SJ, Zayas A, Gómez-Carmona D, Merchán-Clavellino A. Development and Validation of the Psychotherapeutic Effectiveness Attribution Questionnaire (PEAQ-12) in a Spanish Population. International Journal of Environmental Research and Public Health. 2021; 18(19):10372. https://doi.org/10.3390/ijerph181910372

Chicago/Turabian Style

Romero-Moreno, Antonio, Alberto Paramio, Serafín J. Cruces-Montes, Antonio Zayas, Diego Gómez-Carmona, and Ana Merchán-Clavellino. 2021. "Development and Validation of the Psychotherapeutic Effectiveness Attribution Questionnaire (PEAQ-12) in a Spanish Population" International Journal of Environmental Research and Public Health 18, no. 19: 10372. https://doi.org/10.3390/ijerph181910372

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop