The Effects of Learning Contexts on the Development of Reflective Thinking in University Education: Design and Validation of a Questionnaire

Abstract

Reflective thinking is a key skill for constructing meaning in university. Its development requires appropriate learning contexts which can function as spaces for reflection, along with approaches, conditions and methods which can boost students’ training in thinking, framed within their process of knowledge construction and their development of competencies and professional skills. This paper reports the development and testing of a questionnaire on the value of the learning contexts designed to foster reflective thinking. To ensure validity, the constructs measured were derived from the extensive literature on conditions for planning learning and teaching activities for reflective thinking based on narrative-based methods. The instrument was validated with a sample of students (n = 375) from five universities. The results obtained from the estimation of a 10–factor model offer appropriate goodness of fit and parsimony with acceptable and consistent indices of reliability. The results contribute to the evidence supporting the reliability and validity of the questionnaire and confirm the value of the model’s components for devising higher education teaching activities to promote a reflective thinking process.

1. Introduction

Higher education, which has always been influenced by the reflective tradition [1], now lays increasing stress on the development of reflective skills—“twenty–first century skills” [2] for students’ acquisition of professional skills [3,4,5,6,7]. The social and symbolic context of contemporary societies [8,9,10] situates the meaning of information at the core of educational quality when transforming it into knowledge. This requires a type of learning which is necessarily reflective, based on a holistic, comprehensive and functional interpretation of the competencies [11,12,13] and closely linked to the concept of practical knowledge [14], if students are to be equipped to understand and intervene in personal, social and professional life.

Reflection requires training, effective habits, appropriate learning spaces [15], learning environments [16,17] and even a culture of thinking in the classroom [18]. Aimed at addressing this concern, the present article is the result of a research project in university teaching (REDICE16–1660) which centred on studying the value of activities based on the narrative approach [19,20] for fostering reflective thinking around what was learnt, students’ own learning processes and their development of competencies and professional skills [21].

In order to assess these learning contexts, a questionnaire was designed and tested, grounded in: (a) definitions of reflective thinking as a key general competency; (b) different models with stages and hierarchical levels of reflection [22,23,24,25,26]; and (c) conditions for planning learning and teaching activities [1,27,28] favourable to the construction of contexts and spaces for reflection in university education. This article presents the process and results of the validation of this instrument, which measures students’ opinions on the value of narrative methods and the learning contexts designed to develop reflective thinking.

1.1. Reflective Thinking in the Learning Process

John Dewey [29] defined reflective thinking as the in-depth examination of our beliefs in the light of what underlies them and their products. This is a cognitive activity through which a careful examination of the principles underlying reflective practice is made [30]. Reflective thinking is seen as a key general competency for university students’ overall development [31]. Comparing the various theoretical definitions of teaching in this context [22,27,30,32] thinking reflectively allows us to become aware of our ‘style of thinking’ and of our ‘in-progress and completed learning,’ its interpretation and assessment. All of these are key factors in the exercise of sustainable thinking, in the same way as competency-based learning [28].

In this process different stages of reflective thinking have been identified [22,23,25,26,33,34,35]. Scholars concur that raising awareness is the essential basis of transformational reflective thinking [36], and that the end result is the (re)incorporation of new qualities into the knowledge and skills of those exercising it [37]. Analysts such as Ryan [25] have outlined four hierarchical levels of reflective thinking:

• Reporting/Responding. As the most basic level of reflection, students are taught to narrate, to become aware and to discuss aspects of practice. It stresses the identification of the focus of reflection, identifying “what is important to you.” Knowing how to identify a theoretical or practical situation on which to focus one’s reflection is an essential factor in developing the reflective process.
• Relating. This involves connecting with prior skills and/or knowledge (of a related problem or in a similar environment), along with values and priorities, and with how these are related to the values and priorities of others.
• Reasoning. Here students take the step from evidence to reasoning and thus can make more rigorous reflections: arguing a case in detail, consulting theory and literature to support an argument and considering different perspectives. Explanation and justification appear as the different possibilities are examined and at times even the ethical implications are taken into consideration.
• Reconstructing. On this final level, the reflective process reaches its highest expression. Students should demonstrate new ideas and ways of thinking about a problem, and these should be transferable to their future practice.

In the theoretical model underpinning this study, the development of thinking is seen to require appropriate learning spaces [15] which bring reflection into play and influence students’ own self-regulation of this process [38]. These are spaces of comparison, interaction and reflection “enabling us to carry out premeditated thought acts, such as taking decisions, arguing a case and other analytical, creative or critical actions” [39] (p.15) and which awaken students’ curiosity and stimulate autonomy in their own learning process [40].

1.2. University Learning Contexts as Spaces for Reflection

Situated social–constructivist theory ascribes a social origin to the development of reflective thinking [41]. Teaching and learning activities are conceived as social and emotional experiences through which students can become aware of what they say and do. The task of teaching requires teachers to create the right conditions: contexts which bring reflection into play and influence the student’s own self-regulation of learning [42]. In accordance with González–Moreno [22], in this project learning environments were structured as spaces for reflection with the following characteristics: permanent feedback in a social climate of trust (the emotional dimension); the leading role of the students in their own learning process, boosting self–regulation and autonomy; provision of clear instructions on the task to be carried out, the reason behind it and the conditions required; and the provision of various types of support depending on the requirements of learning (in the areas of introspection, group reflection and persistence).

It is interesting to consider the idea of a culture of thinking which has been consolidated in recent years, understood as an atmosphere in the classroom in which various forces (language, values, expectations and habits) work together to express and strengthen good thinking [18,43].

In our project we decided to structure learning environments on the basis of narrative-based methods [19,20] in order to promote reflective thinking about what was learned and about students’ own learning processes. Stories and writing about experience through narrative strategies, in addition to the responsibility and independence required by approaches centred on student autonomy [44], are potentially rich sources for producing a type of knowledge which can re-evoke experiences and transform them into the contents of what is to come. Also, such narratives have been shown to be enormously useful in developing reflective learning among students [22,24,27,40,45] and fostering professional and skills development in university education [20,21,32].

Writing and narration help to construct thought [46], forging relationships between it and action, experience and affective and motivational factors (self–efficacy, goal–setting, interest in tasks). In this project, narrative and (auto) biographical methods were understood as resonant personal, experiential stories which had created relationships between the academic curriculum and the construction not only of identity but also of meaning, practical knowledge, time and the key aspects of daily life. In this way, the construction of embodied knowledge was made possible by creating a space for intimate reflection by means of stories. This experience [47] showed that promoting a narrative in the learning process has a three–fold value. On the level of both the individual and the group, it (a) offers students an opportunity to open up their thinking, linking back to what has happened, but going beyond it in order to learn, (re)interpreting and (re)signifying knowledge through writing and rewriting. Also, it (b) creates social learning situations, favouring teacher-student and student-student relationships in order to understand themselves based on what they know and what they have. All this endows the learning process with an emotional and contextual dimension which strongly influences the development of reflective thinking. Lastly, it (c) shapes the teacher’s role as ‘guide and facilitator’ who supports and nourishes students on the path of listening, thinking and reflecting. Both steering the learning process using the rubric as a self–evaluation tool and giving feedback and feed-forward in order to widen, reflect on and develop learning have been confirmed as highly valuable strategies for improving the self–regulation of learning [48].

1.3. Instruments for Measuring Reflective Thinking

A general literature review yielded five questionnaires linked to the assessment of reflective thinking [35,49,50,51,52] that present limitations involving the factors of validity (cultural and linguistic) and central tendency. Further, the difficulty of measuring a complex construct such as reflective practice limits the possibilities of these instruments, which should be combined with others in order to triangulate relevant information on how students reflect on their own personal and professional development in relation to the teaching activities in their universities. This results in a serious difficulty in establishing sequential levels of reflectivity, since the processes of reflection, awareness-raising and action are more complex and interwoven than described in the theory. Issues with reliability can be found, in the sense that teaching apparatuses designed and applied with students, and which involve quantitative output on them, have not always spurred the development of reflective thinking.

In addition to this, we observed that studies on reflective thinking in higher education have paid plentiful attention to identifying the sequential levels of reflection [23,24,25,26,30] and, to a lesser extent, to how students reflect on their own personal and professional development in relation to the educational activities in their universities [27,53]. However, an important theme is overlooked: identifying the key factors of the most suitable learning environments for the development of this overall competency in university education. For this reason, the instrument presented here takes a wide range of dimensions to be measured into consideration.

2. Materials and Methods

The overall objective of this study was to design and verify an instrument for measuring students’ opinions on the value of narrative approaches and the learning contexts created for developing reflective thinking in higher education. A survey was carried out in the form of a self-administered questionnaire on the value of narrative approaches and the contexts created for promoting reflective thinking in higher education based on the conceptualisation of the levels of reflective thinking (see annex I). The questionnaire investigates students’ perceptions through a total of 65 items grouped into 8 scales coming from the extensive literature review on learning contexts as spaces for reflection: students’ feelings when doing the tasks set them (7 items); the reflective skills developed (7 items); their levels of reflective thinking (16 items); their awareness of the process of learning (autonomy and reflective self-knowledge) (7 items); their awareness of learning (8 items); the teacher’s role (6 items); the potential of reflective thinking in the learning process (6 items); and their assessment of narrative approaches in fostering reflection in their process of learning and skills development (3 items). Lickert–type scales with numerical values from 1 to 5 were used, with 1 representing the lowest value (never) and 5 the highest (always).

2.1. Participants

The total sample invited to participate comprised 520 students from 5 Spanish universities, with a final participation of 372. All the students invited were part of courses and degrees where narrative apparatuses were devised to trace the impact of narrative methodologies in reflective thinking for learning process, and personal and professional development. The details of each course as well as the strategies carried out to promote reflective thinking are described in [32], as part of a research and innovation project in university teaching (REDICE16-1650). In all of these courses, the reflection of students was increased through strategies such as thinking aloud when solving a problem, the analysis and exchange of materials of personal elaboration and the formulation of questions about the process and mechanisms that they used before, during and after the tasks were performed. The reflection was not only about the accomplishment of their tasks, but of the whole process carried out for each activity. Among the principles that guided the inclusion of courses and degrees where the questionnaire was applied, the following were crucial since they placed the academic curriculum in relation to the construction of students’ identity, meanings, and practical knowledge: (a) The theoretical introduction of what an autobiographical perspective implies and advances in class; (b) the design and implementation of activities demanding that students put themselves in a new situation where they need to think and face what they have experienced, as well as give meaning to what happened; (c) the provision of significant learning environments (mainly in group) to encourage the thinking, thus to “exchange”, to “think with” and “in presence of”.

Cases in which students did not respond to at least 70% of the questions were eliminated, since the information provided was insufficient. Also, outliers were excluded through a Mahalanobis distance test. The mean age was 20.76 (SD = 2.88), with an age range from 18 to 41. A total of 80.9% were women and 18.55 were men. The universities represented from highest to lowest participation were: the University of Burgos (UBU) with 29.6%, the University of Valencia (UVEG) with 22.8%, the Autonomous University of Barcelona (UAB) with 22.5%, the University of Barcelona (UB) with 17.2% and the University of Castilla la Mancha (UCLM) with 7.9%. A total of 33.3% of the sample were studying degrees in Primary Education, 29.3% were studying Infants Education, 23.1% were studying Pedagogy and 14.2% were studying Social Education. Of these, 62.4% stated that they had some previous experience in the educational field. Also, 49.3% did not combine their studies with working; of the rest, 22.5% worked between 10 and 20 hours a week, 20.1% worked less than 10 hours and 8.1% worked more than 20 hours. The sample mainly comprised students who had gained access to their degree courses through the university access test, PAAU (62.4%), or through a higher-level vocational course (28.8%). The rest had gained access via other routes, such as the access test for adults over 40.

2.2. Procedure

The design and validation procedure adopted the method of expert judgement and pilot application to a wide sample, in line with the recommendations in the literature [54]. It comprised 2 phases with 4 actions in each:

• Phase 1. Design and validation of the instrument’s first draft:

  o

  Action I. Literature review and documentary analysis;

  o

  Action II. Expert judgment. The questionnaire’s grammar, language and semantics were assessed, and its legibility was analysed.

• Phase 2. Piloting the instrument. Analysis of metric properties:

  o

  Action III. Administering the questionnaire. It was administered online to the students at this address: goo.gl/qWw8CT. The researchers at each university informed students of the objective of the study and gave instructions on how to administer it, in order to minimise negative impacts on reliability. Voluntary participation was sought, and confidentiality was ensured through anonymous coding in the virtual register;

  o

  Action IV. Analysis and creation of the validated version of the questionnaire. An analysis appropriate to assessing the psychometric properties of the questionnaire was used based on the data obtained from the fieldwork in phase III.

2.3. Data Analysis

The data analysis was performed using SPSS 23.0 y AMOS 22.0 with licences form the University of Barcelona. Univariate (means, deviations, asymmetry, kurtosis, etc.), bivariate (Pearson correlations) and multivariate (exploratory factor analysis and confirmatory factor analysis) analyses were performed. A reliability analysis was also carried out for the internal consistency of the scales using Chronbach’s alpha. The discriminatory power of the items was estimated prior to the exploratory factor analysis with the aim of identifying the latent variables enabling the organization of the items making up the scale, and a confirmatory factor analysis to reduce the number of items while conserving maximum variance. Finally, the consistency of the instrument subscales thus identified was tested with an analysis of their internal consistency.

After determining the psychometric properties of the instrument, the construct validity was analysed through an exploration of the factor structure of the total set of items, with the objective of identifying the underlying variables. The method of principal component extraction with varimax rotation was adopted. The instrument’s dimensionality was tested by calculating the Kaiser–Meyer–Olkin (KMO) measure of sample adequacy and carrying out a Bartlett sphericity test in order to determine the adequacy of the exploratory factor analysis (EFA) previously undertaken. Once the factors had been established, a confirmatory factor analysis (CFA) was performed. To ascertain the goodness of fit of the CFA models, the comparative fit index (CFI) and the Tucker–Lewis index (TLI) were used. Also, the Root Mean Square Residual (RMR) and the Root Mean Square Error of Approximation (RMSEA) were estimated. Lastly, for the RMSEA, we took values of less than 0.05 as indicating a good fit and for the RMR values close to 0 and less than 0.08 suggested a good fit. For the fit of the models we took TLI and CFI higher than 0.9 as acceptable values. Also, with the objective of comparing the relative fit of the resulting models, the Akaike information criterion (AIC) was used.

3. Results

3.1. Descriptive Statistics and Reliability

The latent variables of the questionnaire present response values of between 1 and 5. The mean ranges from a maximum of 3.95 for the variable awareness of learning and a minimum of 2.91 for teacher role, and the distribution of responses for the variables does not exceed a standard deviation except for the variable levels of reflective thinking, with an ST = 1.02.

Table 1. Descriptive statistics and reliability.

Dimensions	Min.	Max.	Mean	SD	α
Emotions (EMO)	1	5	2.99	0.57	0.787
Reflective skills (RS)	1	5	3.42	0.63	0.893
Levels of reflective thinking (LRT)	1	5	3.71	1.02	0.816
Awareness of learning process: autonomy and reflective self-awareness (ARSA)	1	5	3.38	0.66	0.905
Awareness of learning (AL)	1	5	3.95	0.68	0.881
Teacher role (TR)	1	5	2.91	0.64	0.885
Reflective thinking in the learning process (RTL)	1	5	2.96	0.59	0.923
Assessment of narrative approaches (AN)	1	5	3.70	0.22	0.626

shows the descriptive statistics and the Cronbach’s alpha coefficients for each of the 8 scales making up the questionnaire. The alpha of the questionnaire is 0.966. The reliability of the 8 latent variables shows response values ranging from 0.923 for reflective thinking in the learning process to 0.626 for assessment of narrative approaches.

Verification of fulfilment of the applicability conditions of the 8 scales yielded positive values in relation to the discriminatory power of their items. Normality and linearity showed values within the range (−1.1), and an analysis of multicollinearity among the items [55] enabled us to rule out redundant variables; i.e., inter–item correlations greater than 0.95. Atypical cases were not found. Also, the corrected item–total correlations revealed values higher than 0.20, thus eliminating items 1.1, 1.3 and 1.4 from the emotions scale and items 8.1 and 8.3 from the assessment of narrative approaches scale.

3.2. Exploratory Factor Analysis

After determining the psychometric properties of the instrument, the construct validity was analysed by exploring the factor structure of the total set of items, in order to identify the underlying variables. The extraction of main components with varimax rotation (

Table 2. Grid of rotated component ^a.

		Component
		1	2	3	4	5	6	7	8	9	10	11
RS	2.6. Enrich my ideas…	0.695
RS	2.2. Argue a case and question myself…	0.671
RS	2.3. Awareness of how to transfer…	0.670
RS	2.1. Reflecting…	0.625
RS	2.4. Connecting knowledge…	0.623
RS	2.7. Taking up a stance towards my reading…	0.621
RS	2.5. Learning for myself…	0.571
RTL	7.2. Learning connecting to theory…	0.551
RTL	7.3. Clarifying ideas…	0.448	0.417
RTL	7.1. Knowing how to do things…	0.441		0.417
LRT	Kember_3.7
AL	5.3. Reflecting…		0.706
AL	5.1. Being aware…		0.697
AL	5.4. Becoming aware of…		0.692
AL	5.2. Understanding more…		0.654
AL	5.5. Knowing my strengths…		0.628
AL	5.8. Being aware of how I learn…		0.531
AL	5.7. Recognizing teamwork…		0.524
AL	5.6. Giving value to the story…		0.500
LRT	Kember_3.15		0.406
TR	6.4. Creating a climate of trust…			0.806
TR	6.3. Helping me to think…			0.734
TR	6.5. Providing spaces…			0.733
TR	6.1. Providing clear instructions…			0.704
TR	6.2. Making constructive feedback…			0.624
TR	6.6. Enabling independent action…			0.523
RTL	7.5. Learning the contents…	0.441		0.460
RTL	7.6. Being aware…			0.431
RTL	7.4. Being motivated…			0.428
EMO	1.4. I felt unsure_R				0.797
EMO	1.3. I felt anxiety_R				0.730
EMO	1.10. I felt confused_R				0.711
EMO	1.1. I had doubts				−0.643
EMO	1.9. I felt comfortable				0.466
EMO	1.7. I felt confident				0.449						0.448
EMO	1.8. I was indifferent_R
ARSA	4.5. Making more effort to learn…					0.578
ARSA	4.7. Applying solutions…					0.561
ARSA	4.3. Being more persevering…					0.522
ARSA	4.6. Involving myself more…		0.437			0.510
ARSA	4.1. Knowing how to plan my work…					0.481
ARSA	4.4. Being more creative…					0.474
ARSA	4.2. Regulating my learning…
LRT	Kember_3.11
LRT	Kember_3.13						0.788
LRT	Kember_3.9						0.724
LRT	Kember_3.5						0.696
LRT	Kember_3.1						0.650
LRT	Kember_3.8							0.716
LRT	Kember_3.12							0.609
LRT	Kember_3.4							0.560
LRT	Kember_3.16		0.441					0.458
LRT	Kember_3.6								0.806
LRT	Kember_3.10								0.736
LRT	Kember_3.2								0.599
LRT	Kember_3.14								0.491
AN	8.4. Importance…									0.696
AN	8.3. Demand…									0.608
AN	8.1. Difficulty…									0.529
AN	8.2. Novelty…									0.496
AN	8.5. Appropriateness…									0.477
EMO	1.6. I was surprised										0.623
EMO	1.5. I was enthusiastic										0.577
EMO	1.2. I was curious										0.425
LRT	Kember_3.3											0.783

Extraction method: principal components analysis. Rotation method: varimax with Kaiser normalization. a. Rotation converged in 12 iterations.

) was chosen because the hypothesis was that there were no relationships among the projected underlying factors or structures, and thus that they were independent of each other.

The indicator of construct validity yielded information on the suitability of the sample fit (KMO = 0.954). Bartlett’s sphericity index (χ²₍₂₀₈₀₎ =14510.632) showed a p = 0.000; therefore, we can affirm that there were significant inter-variable interrelations which provided information on the fit of the data to a factor analysis model.

The first factor (reflective skills) contained 11 items, some with double load and low weight. We eliminated the last three items: 7.3, 7.1 and item 3.7 from the Kember et al. scale [35], so that this first factor was left with 8 variables. All the other items except for one-assigned to the potential of reflective thinking in the learning process (7.2)—belonged to the ‘reflective skills’ dimension, hence the scale name was retained. The second factor (n = 9) contained 8 items from the autonomy and reflective self-knowledge scale in the ‘awareness and learning’ dimension and one from the levels of reflective thinking scale. Since this last item (3.15) contributed little to the explanation of the factor, we decided to eliminate it. As in the previous case, the name of the initial scale was retained. The following factor was made up of 6 variables after the elimination of 3 items (7.5, 7.6 and 7.4), which as before showed uneven behaviours. The teacher role scale was retained, since all the variables were from this scale. The fourth factor was made up of 5 variables after the elimination of two redundant items (1.7 and 1.8). While these variables initially belonged to the emotions scale, we decided to nuance the factor and name it emotions with unease (unsureness, anxiety, confusion, doubts), since these are consistent with the instrumental nature of reflective thinking (i.e., without predisposition there is no action). Following the same line of argument, we noted that the last of the factors to emerge from the analysis was activating emotions, which have to do with the creation of spaces allowing interesting issues to be addressed. After the emotions, the following factor (n = 8) clearly corresponded to the autonomy and reflective self-knowledge in learning scale from the ‘awareness of one’s own learning’ dimension. In this case we eliminated items 4.2 and 3.11. The three following factors fell under the heading ‘levels of reflective thinking’, conforming to the initial scale from Kember et al. [35]: the first (n = 4) referred to the habitual action scale; the second (n = 3) to critical reflection, although an item was lost, as it was highly shared; and the third (n = 4) to the understanding scale. Regarding these analysts’ initial scale, we noted that in our analysis that the factor of ‘reflection’ disappeared, since we chose to eliminate the last item (3.3) from the table as it was a factor with a single item only. Finally, the analysis yielded the factor relating to the assessment of the ‘integration of narrative approaches’ (n = 5) into the learning process.

It is important to note that there were certain correlations between the factors or structures assumed to be underlying, since some items with a double load were identified. In the discussion below, we put forward some possible explanations for this.

3.3. Confirmatory Factor Analysis

Based on the foregoing exploration, a confirmatory factor analysis was performed to test whether the exploratory factor structure showed correct fit indices and, if this was not so, to modify the model in order to obtain the best possible model. Prior to the estimation analysis of the models, Mardia’s coefficient was calculated to ensure multivariate normality in the data obtained. In this case this condition was not met, since the resulting coefficient (1664.7), in accordance with Bollen [56], was lower than p (p + 2), p being the number of variables observed. Due to the absence of normal multivariate distribution, the method of weighted least squares (WLS) was used. The statistic of the RMR index of goodness of fit (residue analysis) was taken as a reference, which represented an indicator of the fit of the items to the model proposed.

The hypothetical theoretical model resulting from the exploratory factor analysis, composed of 10 factors and 52 variables, obtained acceptable results (

Table 3. Indices of goodness of the confirmatory factor analysis.

	Model Fits (with the Method of Maximum Likelihood, ML)
	Chi-Square *	df	CFI	AIC	TLI	RMR	RMSEA	90% CI
								Lo	Hi
Initial model	1700,017	1035	0.909	1916,089	0.901	0.046	0.046	[0.042	0.050]
Final model	1607,781	1031	0.922	1822,053	0.914	0.043	0.043	[0.039	0.047]

χ² = Chi square p< 0.01; df = degrees of freedom; CFI = Comparative Fit Index; TLI = Tucker-Lewis Index; RMR = Weighted Root Mean Square Residual; RMSEA = Root Mean Square Error of Approximation; CI = Confidence interval of RMSEA; AIC = Akaike’s Information Criterion.

). The incremental fit indices showed excellent values (CFI = 0.909; TLI = 0.901); while the indices for absolute fit showed good values (RMR = 0.046; RMSEA = 0.046). Finally, the χ2/gl quotient and AIC also presented extremely good values. Therefore, it would be correct to affirm that the model fits the proposed structure.

The previous model was modified with the aim of achieving one with better goodness of fit to the criteria established. To this end various criteria were used: (1) the significance of the factor weights; (2) the information provided by the residual grid; and (3) the modification indices put forward by the program. The result was the elimination of the items 1.9, 3.2, 8.1 and 8.3. Thus, the final model comprised 48 items grouped into 10 factors (Figure 1). All the indices of fit in this new model were also correct, with increases of fit (Table 3).

The reliability of the final instrument was measured with the Cronbach’s alpha coefficient, giving a value of 0.943, indicating that the psychometric characteristics of the questionnaire were very good. The values by subscales (

Table 4. Reliability analysis of the subscales of the final model.

Dimensions	Factors	N elements	Cronbach α
Reflective skills (RS)		8	0.904
Teacher role (TR)		6	0.891
Awareness of learning	Awareness of learning (AL)	8	0.882
	Awareness of the learning process: autonomy and reflective self-awareness (ARSA)	6	0.902
Emotions	Emotions of unease (EMOUN)	4	0.754
	Emotions of activation (EMOACT)	3	0.699
Levels of reflective thinking	Critical reflection (LRT_CR)	3	0.539
	Habitual action (LRT_HA)	4	0.742
	Comprehension (LRT_C)	3	0.705
Assessment of narrative approaches (AN)		3	0.648
		48	0.943

) were also acceptable and in most cases either good or very good. Their reliability was within what authors such as Nunnally and Bernstein [57] suggest for initial or exploratory studies: reliability values of up to 0.6 can be valid. Likewise, Loewenthal [58] writes that, in scales with few items, i.e., with a maximum of 10, a reliability of 0.4 can be accepted. This shows that in forthcoming applications of the questionnaire, the stability or variation of this indicator should be studied.

4. Discussion

Promoting in-depth and higher-order thinking skills (reflective thinking, critical thinking) in university education contributes to the development of professional competences associated with good thinkers [36]. This premise is essential in implementing constructive and self-regulating experiences of in–depth learning and, therefore, in the achievement of successful learning through innovative and relevant teaching practices which can shape students’ subsequent teaching practice [59]. Thus, there is a need to determine which elements help to establish best practices for reflective teaching and learning; and students’ perceptions are key in obtaining information on this issue [27]. Using instruments such this one enables us to measure the effects of higher education learning contexts on the development of reflective thinking.

The results obtained by exploring the factor structure of the final model present appropriate goodness of fit and parsimony [60], thus verifying the suitability of the indicators chosen for measuring the questionnaire’s constructs of interest. Therefore, the instrument may be applied for gathering data from students’ points of view in university contexts in which strategies and methods to aid learning how to think critically are designed and implemented.

The most outstanding contribution of this questionnaire has to do with its dimensions of analysis. Consistently with the theoretical model, its construct validity and underlying relational structure confirmed 10 scales. This enables us to corroborate the key elements influencing the development of reflective thinking in the university environment through methods and learning contexts designed as spaces for reflection; a desirable triad of guidance, autonomy in a climate of trust and reflection [47]. While the design of the instrument started by dividing it into 8 dimensions, the results yielded 10 factors which clearly corresponded with the theoretical approaches incorporated into the initial instrument. The two large–scale changes which altered the instrument’s factor structure had to do with, firstly, the emotional dimension, and secondly, the levels of reflective thinking defined by Kember et al. [35].

Regarding the emotions, our findings evidenced a difference between what we have termed ‘emotions of unease’ and ‘emotions of activation.’ This may be explained since the root of reflective practice [30] certain emotions (i.e., unsureness, anxiety, confusion, doubt) are seen to be necessary, as they drive the activities of seeking, researching, problem-solving and decision-making [61], consistently with the instrumental nature of reflective thought. Similarly, reflective practice also has to do with the creation of spaces enabling us to address interesting issues, involving surprise, enthusiasm and curiosity, which we see as emotions of activation. Another important aspect is motivation and the climate offered in the classroom for carrying out activities. In the innovations implemented here, the tasks set were conceived as ways of communicative exchange, of cooperative relationships between students and teachers, intended to induce experiences (feelings, estimations, judgements about oneself and the learning tasks) which would enable the expression of a way of being and feeling propitious to learning.

The teacher role as guide and facilitator of the learning process is also key, since the path of listening, thinking and reflecting cannot be undertaken individually. It is not possible to deliberate, make judgements, take decisions or progress in the cognitive self-regulation of learning activities without creating developmentally appropriate learning environments [62]. Both dimensions, the emotional and the contextual, represent influential factors in the self-regulation of learning [38], favouring enhanced awareness of one’s own learning, from the standpoint of reflective self-knowledge (knowing what to improve/develop in the professional and/or academic areas and in one’s own learning process) and the exercise of greater autonomy in this process.

As for the factors corresponding to the levels of reflective thinking in the initial scale from Kember et al. [35], the indicators these analysts attributed to the ‘reflection’ factor disappeared in the outcomes of our analysis. Kember et al. defined this factor as “the process of internally examining and exploring an issue of concern, triggered by an experience, which creates and clarifies meaning in terms of self, and which results in a changed conceptual perspective”, following [63] (p.100). This may well be explained by the fact that our prototype questionnaire included two scales on awareness of learning, which also enquired into reflective self-awareness and autonomy. As we noted above, awareness-raising represents the fundamental dimension for transformative reflective action [36,64], and in this case the final instrument widens the dimensions of one of the most widely known models of reflective thinking [35].

The development of narrative approaches in the classroom involves a shift in orientation towards new spaces, where support and feedback integrated into educational assessment are key to the optimization of the aimed-for skills and professional development. We hope that university teachers are aware that the higher-order thinking skills should form part of university instruction, because they shape the structuring of activities and techniques which cultivate and nourish reflective and critical thought. There is a pressing need for educational systems and instructors to equip learners with the regulatory skills and abilities required for the demands of lifelong learning [65]. Also, thanks to the verification of the instrument presented here, we now have a valid means of establishing measures for assessing reflective thinking widely and holistically when it is fostered with narrative approaches.

Overall, this instrument might help teachers to face the challenge of creating spaces for reflection with a culture of thought [18] in higher education, where group thinking–both individual and collective–is valued, made visible and actively promoted as part of everyday experience of all members. On this basis, it might be useful to evaluate reflective learning environments [1], considering the experiential and emotional dimensions (experiences, intuition, feelings, values, creativity and uniqueness of each person) that promote deep learning, the quality of which requires of specific strategies and contexts. The instrument itself allows to gather data about the following three dimensions of these reflective learning situations [32]:

a)

the existing opportunities for learning from deep and effective thinking in classrooms [66,67]. This questionnaire is useful to obtain evidence of how students learn, from the personal awareness of their previous experience, thinking of themselves and connecting with their social context of reference.

b)

the value of self-regulation and the role of students in the learning process, herein based on the use of narrative methodologies. This questionnaire highlights the value of the teaching role as a guide and facilitator to sustain students on the path of listening, thinking and reflecting.

c)

the creation of a social climate of trust in the classroom. This instrument helps to evaluate the teaching environment needed for the development of thinking activities. The emotional and contextual dimensions are key in the self-regulation of learning [68] and have also been shown to be favourable for the development of reflective thinking in previous works [32].

Regarding the limitations of this study, this is a transversal study, which does not allow us to draw conclusions on the evolution of the dimensions analysed. We should take into account the effects of the social desirability intrinsic to self-responding instruments and the need to constantly test and verify the degree of fit and the metric behaviour of the questionnaire in subsequent studies, as is habitual in validation studies. Turning to future lines of research, the relationships between narrative approaches and improvements in students’ learning, and their effects in developing teaching should be analysed. Also, it is also important to examine what influence the promotion of reflective learning has on the degree to which students are capable of thinking more widely and more effectively. Finally, future research should focus on both assignments and in-class practices that challenge students to engage in higher-order thinking [69,70] using multi-institutional samples, as in the present case.