Validated Questionnaires in Flow Theory: A Systematic Review

Abstract

Psychological flow has been measured in several areas to analyse to what extent users are engaged in particular tasks, and is relevant in the design of products like software, videogames, and eLearning courses. Although there is an unknown number of questionnaires for evaluating different aspects of psychological flow, the research problem faced in this paper is the analysis of the validity of these questionnaires, since it has only been probed for some of them. Thus, our goal is to synthesize the current evidence regarding validated questionnaires in the English language for psychological flow measurement by conducting a systematic review according to the PRISMA framework. As a result, we found a total number of 34 validated questionnaires to assess flow. The number of their items ranged from 3 to 66, while 63 different dimensions of optimal experiences were taken into consideration. Moreover, the contexts of use differed, including methods to assess flow intensity, prevalence, variations, proneness, metacognitions, in crowds, observed, as dimensions of autotelic personalities, or to differentiate flow from clutch states. As a consequence, this paper facilitates the selection of the questionnaires for research or applied aims, far beyond the classic dichotomy of prevalence–proneness. Moreover, we present a reinterpretation of the nine-dimensional scheme of flow in stages, and recommend future research for engineering and computer science.

1. Introduction

1.1. The Flow Theory

Flow Theory [1] is framed within Positive Psychology because it is related to the improvement and knowledge of human capacities [2,3]. It should also be noted that Csikszentmihalyi [4] frames it within the theories of self, since it focuses on what people feel about themselves [5,6,7,8]. Likewise, interest in human enjoyment distances Flow Theory from paradigms such as Behaviourism or Psychoanalytic Theory, which explain participation in experiences as being in order to obtain rewards or meet libido needs [1,9]. Conversely, Flow Theory explains the causes that lead people to participate in activities that do not bring them immediate or tangible benefit [1]. Csikszentmihalyi [1] and Privette [10], among others, justified such behaviour because individuals live peak experiences [11] with a perceived optimum between fun and the perfection achieved. In this sense, Flow Theory explains persistence in non-trivial activities [12]. Moreover, they must be carried out smoothly, without apparent effort [13]. That is why they are called optimal experiences, and that particular psychological state is known as flow [14,15]. In summary, participating in activities perceived as optimal experiences reinforces motivation [16]. Moreover, since people seek to participate in activities they enjoy, Flow Theory has a hedonistic nature [17].

1.2. Justification and General Description of Psychological Flow

The term flow is defined as “the holistic sensation that people feel when they act with total engagement” [1] (p. 36) while they participate in any activity. Additionally, there are other definitions that stress flow preconditions, the core flow experience, or the consequences of such a psychological state [14]. However, the most prominent definition is the one proposed by Csikszentmihalyi [1], which considers that the flow state is a subjective situation, since it depends on how the person perceives his or her participation in the task. In order to achieve flow, the subjective balance between perceived challenges and skills related to the task (experience) is an important element [18], since the flow only starts when such perceptions are over a certain limit and in balance [15]. In addition, immediate feedback and clear goals are the other necessary preconditions to achieve flow [1,19].

Other definitions associate flow with fun [10,20], which summarizes the core of the experience. According to [21], when people are in flow, they report feelings of happiness, motivation, and cognitive efficiency. However, for Ghani [22], the two key features of the core experience are concentration and enjoyment during the activity. Even simple approaches such as these provide one of the main justifications for the existence of Flow Theory: it studies what makes experiences fun for people to take part in, even when they do not receive immediate benefits [1]. Moreover, Kotler et al. [23] consider flow to be an altered state of consciousness, because people feel a distortion of the sense of time, and also a lack of self-consciousness, similar to addictions, as demonstrated with neurophysiological experiments. Furthermore, participants have described the sensation of being in flow as an effortless effort, in which actions move fluidly to the next [14]. In addition, some consequences are derived from such pleasurable state [14] because individuals want to participate in experiences again and increase their performance [1,19,24,25]. However, there are other voices who consider flow to be a process [26], rather than a state [25]. Thus, any static or linear definition may not be precise.

As we can see, several definitions have been posed over the years regarding flow [27]. In summary, we could say that flow can be considered either as a process or a personality trait, linked to a specific task, where a subjective psychological state associated with enjoyment and personal growth may emerge, and which has its own phenomenology.

On the other hand, Novak and Hoffman [28] established general methods in flow research, in which questionnaires and scales are paramount. Thus, experimental works may start with the participation in or the recall of a particular experience, before completing some questionnaires or interviews at the end of the session [28]. Moreover, any experimental work on Flow Theory involving questionnaires will also include physiological measures in its methodology [29,30].

However, several authors [31,32,33] have reported that there is no agreement on the number of available questionnaires, as well as the dimensions of flow that the instruments can assess [21,22,34,35,36]. Since there is also no agreement on the relative importance of such dimensions [31], it is not clear whether definitions miss information, or the phenomenology of the flow state differs with the activity [23,37]. In fact, other authors have delved into the inconsistencies in the definition of the flow concept and report methodological issues in the development of the flow-related scales [38]. Another remarkable feature of these questionnaires is that, even when flow state phenomenology is similar among cultures [17], most of them only include positive aspects of the experience, forgetting to assess negative aspects and sweetening the recall of experiences [39,40]. Taking all these elements into consideration, several authors [26] claim that flow research is approaching a “crisis point” or a paradigm shift, due to accumulating methodological and conceptual inconsistencies.

Since methods to determine the flow are frequently based on questionnaires (not all of them validated) [1,29,31,32], an updated comprehensive study is necessary where validated questionnaires are compiled. This paper collects and studies these elements with a systematic approach and without being limited to any specific context in the search queries. In particular, we address the following research questions:

RQ1:

What flow study methods based on validated non-ESM questionnaires in English are known?

RQ2:

What dimensions of flow experiences can be assessed with such questionnaires?

RQ3:

What limitations are present in such flow questionnaires, considering their development and validation process?

The rest of the paper is organized as follows: in the Materials and Methods section, we present the methodology, divided into eight phases, according to the PRISMA framework [41]. In the Results section, we answer the research questions, describing the validated questionnaires, and assessing the quality of the works that included them. Finally, we discuss the results in the Discussion section, and extract the main conclusions in the last section.

2. Materials and Methods

Building the research questions was particularly difficult, since the most meaningful keywords in databases related to Flow Theory are very polysemic (i.e., flow, optimal, and experience). Because of this, many restrictions are usually included in search queries to yield clean and viable results for further analysis. However, every restriction implies a latent risk of bias, limiting the research field to a very particular scenario, or excluding works not using those keywords. Thus, it is very difficult to establish how many validated flow questionnaires in English are available, even approximately. Because of this, we used the PRISMA framework [41] as a standard for systematic reviews (Figure 1), including under the epigraph “other sources of information”, articles which we obtained by means of grounded theory [42] from preselected works and references.

Grounded theory [42] is a constant comparative and iterative method that allows the discovery of meaning units (categories) to elaborate a theory that emerges from the text [43]. Following [43], these categories are established by coding units of meaning from the texts. At the beginning, this coding is open, and then relates terms to other categories until saturation. Saturation occurs when no new categories appear or when new categories do not provide additional information. As a result of applying grounded theory, a category book was developed and summary records for the texts [44] were generated. Specifically, when a fragment of the text indicates a form, questionnaire or flow scale, we followed a recursive reference screening and search process, until the full questionnaire and validation methodology may be contrasted. The initial categories were defined as follows, bearing in mind that every time that the term flow is cited, it tacitly refers to the definition of Mihaly Csikszentmihalyi [1]:

A.

Definitions: the authors present a definition of flow state;

B.

Models: the text indicates the existence or shows a model of flow;

C.

Study methods: the authors indicate or show a measurement method of flow, on the basis of forms, questionnaires, or scales;

D.

Factors: the authors extract and show factors from validated questionnaires, forms, or scales of flow;

E.

Limitations: the work indicates limitations affecting questionnaires, forms, or scales about Flow Theory;

F.

Contexts: the text specifies contexts in which the study methods were developed or applied.

Additionally, since our research is focused on questionnaires, we also coded four subcategories for each questionnaire, indicating whether the method has been cited (m), as well as its factors (f), limitations (l), and contexts (c). In cases where a method had a name, we coded it with its initials. If not, we coded it with the initials of the first surnames of the authors. Then, we added “_m, _f, _l, _c”, respectively, depending on the selected subcategory. As the codebook is monotonous, and the methods are summarized in tables in this paper, we present the codes and descriptions for a particular method, affected by research questions, as an example (

Table 1. Codebook example for a specific flow questionnaire. Source: self-made.

Subcategory for	Code	Definition
C. Method	FPS_m	The Flow Process Scale method is cited (FPS)
D. Factor	FPS_f	FPS factor is cited
E. Limitation	FPS_l	FPS limitation is cited
F. Context	FPS_c	FPS context is cited

).

To perform the grounded theory analysis, we used Mendeley (Mendeley Ltd, London, UK, v.1.19.8) [45] as a reference management tool, Atlas-Ti (Scientific Software Development GmbH, Berlin, Germany, v. 8.4.3) [46] for management and analysis, and CmapTools (IHMC, Pensacola, FL, USA, v. 6.4) [47] to design conceptual maps. In this work, we present our results as summary records of the methods, in coherence with the PRISMA framework. Figure 1 summarizes the process, as described in [43].

Below, we summarize the general methodology, articulated in 8 phases (Figure 2).

First of all, we conducted three systematic searches of papers, books, and systematic reviews, named phases 1, 2, and 3, respectively. For each phase, low restrictive eligibility criteria were described. Then, in phase 4, we extracted precandidate methods by means of grounded theory and reference checking. After that, in phase 5, we systematically extracted flow dimensions and contexts of use from selected articles using grounded theory, starting from papers found in phase 1, and followed the saturation principle. In phase 6, we performed quality assessment using the Mixed Methods Appraisal Tool (MMAT) [48]. Next, we presented the results as tables (phase 7), clearly indicating the sources. Finally, in phase 8, we checked the reproducibility of results in the tables.

Once the previous process was finished, we completed the PRISMA 2009 checklist [41] (see supporting information), and the corresponding workflow scheme, indicating the already described research phases when applied. Eligibility criteria were systematically defined using the Population–Intervention–Comparison–Outcome–Study Design (PICOS) [41] approach. As shared criteria for PICOS in phases 1 (PICOS 1), 2 (PICOS 2), 3 (PICOS 3), and 4 (PICOS 4), we agree with the concerns raised in [34] about Experience Sampling Questionnaires, especially when they contain open-ended questions: such questionnaires are continuously administered to the participants, using the Experience Sampling Method (ESM) [49]. Thus, they perturb flow measures, and the answers given are conditioned by previous answers [34]. Nevertheless, we cite them in the Results section within the “Flow questionnaires not matching PICOS 4” group.

2.1. Phase 1: Systematic Search of Non-Systematic Review Papers

We searched with the query “flow state questionnaire scale validation” without restrictions in Web of Science (WoS) (last revision: 17 September 2022), obtaining a set of 41 records. In each phase, we used #n to designate any of the n documents obtained in the search. The search results, in CSV format, may be consulted in Supplementary Materials in the file phase1.csv.

In phase 1, the eligibility criteria (PICOS 1) included peer-reviewed studies, or preprints with a DOI, related to Flow Theory described in Csikszentmihalyi [1], with no restrictions on publication date. Exclusion criteria were: (a) papers in languages other than English; (b) systematic or narrative reviews on the flow state; (c) books or book chapters; (d) flow questionnaires translated or validated in other languages; (e) papers validating ESM questionnaires.

Out of the 41 results, 34 were discarded considering PICOS 1 for the following reasons:

• There was no relation to the Flow Theory (#17, 21, 22, 23, 26, 27, 28, 29, 30, 31, 32, 33, 34, 36, 37, 38, 39, 40).
• Works were written in languages other than English (#3,5,9).
• The studies did not contain references to flow questionnaires (#6, 12, 13, 15, 20, 25, 35, 41).
• Studies were translations and validations into other languages of primary questionnaires in English included in this search (#4, 7, 8, 16, 24).
• Systematic or narrative reviews (#18).

In this regard, 6 works were preselected for in-depth study with grounded theory at this stage (#1, 2, 10, 11, 14, 19). Thus, references [39,50,51,52,53,54] were selected for phase 4.

2.2. Phase 2: Systematic Search of Systematic Reviews

To find systematic reviews in English on Flow Theory, the search query “flow optimal experience systematic review” was performed without restrictions in WoS, yielding 57 outcomes. The search results, in CSV format, may be consulted in Supplementary Materials as phase2.csv.

In phase 2, eligibility criteria included systematic or narrative reviews in English related to Flow Theory as described in Csikszentmihalyi [2], with no restrictions on publication date. Exclusion criteria were (PICOS 2): (a) books or book chapters on flow research; (b) non-systematic or narrative review papers on the flow state. Accordingly, by means of abstract reading, we preselected registers #1, 4, 7, 12, 16, 17, 20, 21, 28, 33, 48. Thus, references [37,55,56,57,58,59,60,61,62,63] were selected for phase 4. Item #9 was not included as it was not in English, and the rest had no relation to the topic.

Following the PRISMA framework [41], we still included new references if they arose in any of the documents under study (as “additional records identified through other sources”). In those cases, we added it to the pool. However, if the reference to be included was already in the pool, we did not include it again to avoid duplication. According to this, we included [34,35,36] for further analysis, since they are relevant systematic reviews.

2.3. Phase 3: Systematic Search of Monographs

In this phase, the query “flow optimal experience” was searched in WoS, with two restrictions: “Books” as source type and 1975 as starting publication date (because Csikszentmihalyi presented the first monograph about Flow Theory in that year [1]) (last revision: 18 September 2022). The search obtained a total of 52 documents. The search results, in CSV format, may be consulted in Supplementary Materials as phase3.csv.

In this phase, eligibility criteria included books or book chapters related to Flow Theory as described in Csikszentmihalyi [1]. Exclusion criteria were (PICOS 3): (a) works in languages other than English; (b) the works are not books or book chapters about the flow state.

By reading the of abstracts, only registers #33, 35, 37, and 39, out of 52 results, offered information about 2 books [31,32] matching preselected criteria. As previously explained in phase 2, the PRISMA framework [29] allows us to include “additional records identified through other sources”. In this case, we included the references [31,32,33] because they represent well-considered seminal reviews on Flow Theory written by different authors than [1].

2.4. Phase 4: Systematic Extraction of Methods and References with Grounded Theory and Recursive Reference Screening

In this phase, we worked with the documents retrieved in phases 1, 2, and 3 to identify methods of flow measurement based on questionnaires. As in previous phases, we applied the PICOS 4 criteria, bearing in mind that we considered peer-reviewed indexed articles and methods fully described in peer-reviewed book chapters. However, in this case we present the criteria in

Table 2. PICOS 4 summary. Source: self-made.

	Inclusion/Exclusion Criteria
Population	Any
Intervention	Researchers use questionnaires, forms, or scales of flow
Comparison	Any
Outcome	A full non-ESM questionnaire, with validation procedures and extracted dimensions (when applies)
Study Design	Experimental research, with introduction, participants, objectives, methods, results, conclusions, and references

, since they need further explanation than that required in previous phases.

We completed the PRISMA 2009 flow diagram [41], summarizing the described phases until now. The PRISMA checklist can be found in Supplementary Materials under the heading “Prisma 2009 checklist”.

2.5. Phase 5: Systematic Extraction of Flow State Dimensions with Grounded Theory

In this phase, we extracted the number of items and dimensions of validated flow questionnaires using grounded theory [42]. We started with papers showing full questionnaires, validation methods, and dimensions. Although, after reviewing the papers, finding new dimensions or items was unlikely, we analysed systematic reviews and books in order to avoid missing any of them.

2.6. Phase 6: Quality Assessment (MMAT)

In phase 6, we considered the Mixed Methods Appraisal Tool (MMAT) [48] to derive comparative parameters of quality assessment in empirical studies. MMAT [48] provides a set of categories to classify the research under study: qualitative research, randomized controlled trials, non-randomized studies, quantitative descriptive studies, and mixed-methods studies [48]. Once the category in which each research is classified had been chosen, we assessed 7 quality items which can be rated with “yes”, “no” or “can’t tell”. In Annex B1, we summarize the MMAT [48], including each study supported by experimental work (thus, we omitted the first two screening questions, as reiterative) into the category of quantitative descriptive studies. The other questions were:

4.1.

Is the sampling strategy relevant to address the research question?

4.2.

Is the sample representative of the target population?

4.3.

Are the measurements appropriate?

4.4.

Is the risk of nonresponse bias low?

4.5.

Is the statistical analysis appropriate to answer the research question?

2.7. Phase 7: Presenting Summaries

Grounded theory [42] indicates that the final outcome of the process could be (1) a conceptual map, (2) a summary record, or (3) a categories book. In this paper, we produced a summary record in the form of a set of tables in annexes: contexts and dimensions (

Table A2. Summary of contexts and dimensions for each of the flow measurement method dimensions. Source: self-made.

Methods	Main Author	Contexts of Use	Dimensions
(Items)
FSS-1 (36)	Jackson [19]	Physical activity, art, free time, music, education, general environment	Challenge competence (4 and 6, respectively)
			Action awareness (4 and 6, respectively)
			Clear goals (4 and 6, respectively)
			Unambiguous feedback (4 and 6, respectively)
FSS-1 (53)			Concentration (4 and 6, respectively)
FSS-2 (36)	Jackson [53]		Sense of control (4 and 6, respectively)
FSS-2 (9)			Loss of self-awareness (4 and 6, respectively)
			Transformation of temporal sense (4 and 6, respectively)
			Autotelic experience (4 and 6, respectively)
DFS-1 (36)	Jackson [64]	Flow proneness in the same contexts as FSS-1 and FSS-2	The same as FSS-1 and FSS-2
DFS-2 (36)	Jackson [64]
DFS-2 (9)
CFS (10)	Martin [65]	Flow prevalence and proneness	Assess the central experience of flow phenomenologically, rather than dimensionally
SFPQ (22)	Ullén [66]	Flow proneness in work, routine, and leisure	Concentration (1 item × 3 domains)
			Balance between skills and challenge (1 × 3 domains)
			Explicit goals (1 item × 3 domains)
			Clear feedback (1 item × 3 domains)
			Sense of control (1 item × 3 domains)
			Lack of a sense of boredom (1 item × 3 domains)
			Enjoyment (1 item × 3 domains)
			One additional question out of the dimensions
FKS (13)	Rheinberg [67]	Statistics course Computer games	Progressivity of a course that makes it appear self-regulated (6)
			Absorption (4)
			Concern (importance given to the task performed and to achievement motivation) (3)
FKS (10)	Engeser [68]	Statistics course Computer games French Course	Fluency of performance (6)
			Absorption (4)
WOLF (13)	Bakker [69]	Work	Absorption (4)
			Enjoyment of work (4)
			Intrinsic motivation for work (5)
RFSS (8)	Thissen [70]	Reading fictional texts	Absorption (3)
			Fluency of reading process (5)
FHCI (11)	Webster [71]	Human–computer interaction	Control (2)
			Attention focus (3)
			Curiosity/Intrinsic interest (6)
FS (66)	Novak [72]	Internet browsing	Use of the web (3)
			Arousal (4)
			Challenge (6)
			Control (4)
			Exploratory behaviour (8)
			Flow (3)
			Focused attention (4)
			Speed of interaction (3)
			Importance or addictiveness (5)
			Playfulness (7)
			Positive affect (4)
			Skill (6)
			Telepresence (7)
			Time distortion (2)
FPS (63)	Saxena [40]	Internet and electronic commerce users	Antecedents of flow	Skills (4)
				Challenge (4)
				Interactive speed (5)
				Enduring involvement (5)
				Control (5)
				Arousal (7)
				Telepresence (5)
				Focused Attention (4)
				OSL (3)
				Playfulness (9)
			Consequences of flow	Consequences of perfect flow (5)
				Consequences of imperfect flow (4)
				Use of external stimuli (1)
				Incomplete tasks characteristics (2)
EduFlow1 (12)	Heutte [12]	Education, MOOCS	Cognitive absorption (3)
			Altered perception of time (3)
			Loss of self-concern (3)
			Autotelic experience or well-being (3)
Eduflow2 (12)	Heutte [73]	Education, MOOCS, STEAM	Cognitive control (3)
			Immersion (3)
			Loss of self-concern (3)
			Autotelic experience or well-being (3)
GSR (14)	Ghani [22]	Face-to-face and computer-mediated problem solving	Engagement (4)
			Enjoyment (4)
			Control (4)
			Skills (1 added to the questionnaire)
			Challenges (1 added to the questionnaire)
VFM (21)	Shin [74]	Education/online courses	Enjoyment (5)
			Time distortion (3)
			Telepresence (4)
			Focused attention (5)
			Engagement (4)
FMI (30)	Guo [75]	E-commerce/HCI	Balance (2)
			Clear goals (4)
			Feedback (4)
			Concentration (4)
			Control (4)
			Mergence of action and awareness (2)
			Transformation of time (3)
			Transcendence of the self (3)
			Autotelic experience (4)
AK (55)	Agarwal [76]	Web-based contexts	Cognitive absorption (20)
			Perceived ease of use (4)
			Perceived utility (4)
			Personal tendency to innovation (7)
			Gameplay (7)
			Intention of use (3)
			Self-efficacy (10)
FSG (22)	Kiili [77]	Educational gamification	Precedents	Challenge (2)
				Clear goals (2)
				Feedback (2)
				Control (2)
				Playability (2)
			Flow experience	Concentration (4)
				Time distortion (2)
				Autotelic experience (4)
				Loos of self-consciousness (2)
EGame Flow (56)	Fu [79]	Educational gamification	Concentration (8)
			Goal clarity (5)
			Feedback (6)
			Challenge (10)
			Autonomy (9)
			Immersion (7)
			Social interaction (6)
			Knowledge improvement (5)
GameFlow (42)	Kiili [78]	Educational gamification	Flow antecedents	Goal (3)
				Feedback (3)
				Playability (3)
				Challenge (3)
				Gamefulness (3)
				Frame story (3)
			Flow state	Concentration (3)
				Time distortion (3)
				Autotelic experience (3)
				Sense of control (3)
				Loss of self-consciousness (3)
			Flow consequences	Exploratory behaviour (3)
				Learning (6)
FSSOT (14)	Yoshida [80]	Occupational therapy	Sense of control of a task (6)
			Experience of positive emotion (4)
			Experience of absorption by concentrating on a task (4)
FOG (13)	Tordet [39]	Videogames	Concentration (5 observational options)
			Enjoyment (4 observational options)
			Frustration (4 observational options)
FCS (22)	Swann [50]	Physical activity	Flow states (9)
			Clutch states (5)
			Effortlessness of flow (3)
			Overlapping characteristics (5)
WOLF-S (13)	Bakker [52]	Study settings	Absorption (4)
			Enjoyment of work (4)
			Intrinsic motivation for work (5)
SFWS (3)	Moneta [81]	Work settings	No dimensions extracted
FMQ (12)	Wilson [51]	Work settings	Beliefs that Flow Fosters Achievement (6)
			Confidence in Ability to Self-Regulate Flow (6)
APQ (26)	Tse [54]	Amazon Mechanical Turk workers	Curiosity (4)
			Persistence (4)
			Low self-centeredness (4)
			Intrinsic motivation (4)
			Transformation of challenges into enjoyment (3)
			Transformation of boredom into enjoyment (4)
			Attentional control (3)
SFS (27)	Zumeta [82]	Collective sports and massive folk drummers’ concentrations	Balance between challenge and skill (3)
			Action–awareness merging (3)
			Clear proximal goals (3)
			Unambiguous and direct feedback (3)
			Focused concentration on the activity (3)
			Sense of control over one’s actions (3)
			Loss of self-consciousness (3)
			Loss of time awareness/time acceleration (3)
			Autotelic experience (3)
FIMA (18)	Custodero [84]	Musical activities in school	Affective indicators (9)
			Behavioural indicators (9)

), and limitations (

Table A3. Summary of declared limitations for each method, extracted with grounded theory, references and information sources. Source: self-made.

Method (Items)	Main Author	Source	Limitations Declared
FSS-1 (36)	Jackson [19]	Phase 3	- General limitations of the surveying methods
FSS-1 (53)		Dellefave [31]
FSS-2 (36)	Jackson [53]	Phase 1	- The construct is a partial representation of the flow experience - General limitations of the surveying methods
FSS-2 (9)
DFS-1 (36)	Jackson [64]	Phase 3	- Age selection (older athletes) - Not controlled when participants completed questionnaires
		Dellefave [31]
DFS-2 (36)	Jackson [64]	Phase 1	- The construct is a partial representation of the flow experience - General limitations of the surveying methods
DFS-2 (9)
CFS (10)	Martin [65]	Phase 2	- Only preliminary findings - Less detailed than long scales (FSS, DFS) - Grouped averaged representation of the experience - General limitations of the surveying methods
		Ottiger [59]
SFPQ (21)	Ullén [66]	Phase 2	- No limitations declared
		Peifer [56]
FKS (13)	Rheinberg [67]	Engeser [32]	- Not proven that in one experimental group there were more individuals with more probability of success - Not proven that in a second experimental group there were more participants with fear to fail
FKS (10)	Engeser [68]	Phase 3	- No limitations declared
		Engeser [32]
WOLF (13)	Bakker [69]	Phase 1	- No participants with high flow scores - No other ratings on work conditions than self-reports - General limitations of the surveying methods
RFSS (8)	Thissen [70]	Suggested by authors	- Educational background and gender - Only selected people that experience flow when reading - Uncontrolled research conditions (not laboratory)
FHCI (11)	Webster [71]	Phase 2	- Data are cross-sectional, perceptual, and subject to common method variance - Outcome data not very objective - Small sample size - Method in an early stage of development
		Ottiger [59]
FS (66)	Novak [72]	Phase 4	- The measurement of tele-presence and interactivity dimensions may be improved
		Saxena [40]
FPS (63)	Saxena [40]	Suggested by authors	- Long-term advanced Web users prevail
EduFlow-1 (12)	Heutte [12]	Phase 2	- No limitations declared
		Peifer [56]
EduFlow-2 (12)	Heutte [73]	Phase 2	- General limitations of the surveying methods
		Peifer [56]
GSR (12)	Ghani [22]	Phase 4	- Skill and challenge dimensions assessed by only one item - One group was anonymous and the other not - Groups participated in sequenced activities
		Agarwal [76]
VFM (21)	Shin [74]	Suggested by authors	- The telepresence dimension Cronbach’s alpha is low (0.63)
FMI (30)	Guo [75]	Phase 2	- It is difficult to define and measure website complexity - Responded by college students only - Preselected real web sites
		Peifer [56]
AK (52)	Agarwal [76]	Phase 4	- All respondents were young students in workplace - Questionnaire not generalizable to other technologies than the Web - Shared variance between questionnaires applied at the same time
		Guo [75]
FSG (25)	Kiili [77]	Phase 2	- Method in an early stage of development - Selected games were simple
		Perttula [22]
GameFlow (45)	Kiil [67]	Phase 2	- Method in an early stage of development - Small sample size - The clarity of some items may be improved (challenge, goals, gamefulness, and exploratory behaviour) - Challenge dimension items should be redefined to consider the activities of the selected game - The game-controlling activities should belong to the playability dimension, and they were treated as independent
		Perttula [34]
EGameFlow (66)	Fu [79]	Phase 2	- Risk of respondent burden (length of the questionnaire) - Not varied selection of games - Kind of users - Selected games were simple
		Perttula [34]
FSSOT (14)	Yoshida [80]	Phase 2	- The scale evaluates comparative change in flow, not absolute flow - The scale is intended for leisure tasks with low physical intensity - The sensitivity is not evaluated - Respondents were Japanese speakers with no disabilities
		Ottiger [59]
FOG (13)	Tordet [39]	Phase 1	- Respondents were students - Short sessions - Flow intensity may change during the tasks
FCS (22)	Swann [50]	Phase 1	- Unbalanced datasets
WOLF-S (13)	Bakker [52]	Phase 1	- Sample selection and contexts (workers)
SFWS (3)	Moneta [81]	Phase 1	- Sample selection and contexts (highly educated workers) - Small sample size - The opportunity for creativity in the job need further information - This questionnaire only measures the prevalence of flow, not the intensity
		Wilson [51]
FMQ (12)	Wilson [51]	Phase 1	- Sample selection (adult Australian English speakers) - Only some kind of sports/activities
APQ (26)	Tse [54]	Phase 1	- Not all subject matter experts consulted considered the attributes included in the method essential - Method in an early stage of development - Respondents’ jobs (MTurk) - Type of activities performed
SFS (27)	Zumeta [82]	Phase 2	- The period between longitudinal studies should be longer - Loss of participants because the study was longitudinal - Women and students prevail
		Pels [62]
FIMA (18)	Custodero [85]	Phase 1	- Participants were small children only
		Tordet [39]

). In these tables, we included the main author of the method and the source from which the information arose, if it differs (in which case it was included in the “additional records identified through other sources”). Because of this, the reproducibility of the outcomes is easy to track, without losing the information needed to answer research questions.

2.8. Phase 8: Reproducibility Assessment

Researchers checked the reproducibility of table summaries. In simple words, they probed every reference one by one, correcting the dimensions and contexts of the methods, if necessary.

3. Results

As a result of the process described in the previous sections, we show the Prisma 2009 flow chart [41] in Figure 3.

The 34 papers finally included in our literature review cover the period from 1975 (when Csikszentmihalyi published his first monograph on Flow Theory) to 2022 (see Supplementary Materials). Nevertheless, we could identify no works containing any non-ESM fully validated questionnaire published in English until 1991. Furthermore, we observed an irregular publication rate until 2022, with a maximum of four works in 2009, an approximated mean of one work per year (0.94), with a standard deviation of 1.01 (Figure 4). We must clarify that some papers may contain more than one questionnaire.

In the next sub-sections, we answer our research questions.

3.1. Flow Study Methods Based on Validated Non-ESM Questionnaires in English (RQ1)

We found 34 validated non-ESM questionnaires in English. Some of the questionnaires have longer and shorter versions, and they appear with different codes. The final codification process is summarized in

Table 3. Summary of codes and names of validated questionnaires. Source: self-made.

Code	Name	Main Author
FSS-1	Flow Short Scale 1	Jackson [19]
DFS1	Dispositional Flow Scale 1	Jackson [64]
CFS	Core Flow Scale	Martin [65]
FSS-2	Flow Short Scale 2	Jackson [53]
DFS-2	Dispositional Flow Scale 2	Jackson [64]
SFPQ	Swedish Flow Proneness Questionnaire	Ullén [66]
FKS	Flow–Kurzskala	Rheinberg [67]
		Engeser [68]
WOLF	Work-Related Flow Inventory	Bakker [69]
RFSS	Reading Flow Short Scale	Thissen [70]
FHCI	Flow in Human–Computer Interactions	Webster [71]
FS	Flow survey	Novak [72]
FPS	Flow Process Scale	Saxena [40]
EduFlow-1	EduFlow 1	Heutte [12]
EduFlow-2	EduFlow 2	Heutte [73]
GSR	Ghani, Suppnik and Rooney method	Ghani [22]
VFM	Virtual-Course Flow Measurement	Shin [74]
FMI	Flow Measurement Inventory	Guo [75]
AK	Agarwal and Karahanna scale	Agarwal [76]
FSG	Flow Scale for Games	Kiili [77]
GameFlow	GameFlow	Kiil [78]
EGameFlow	EGameFlow	Fu [79]
FSSOT	Flow State Scale for Occupational Tasks	Yoshida [80]
FOG	Flow Observational Grid	Tordet [39]
FCS	Flow–Clutch Scale	Swann [50]
WOLF-S	Study-Related Flow inventory	Bakker [52]
FMQ	Flow Metacognitions Questionnaire	Wilson [51]
SFWS	Short Flow in Work Scale	Moneta [81]
APQ	Autotelic Personality Questionnaire	Tse [54]
SFS	Shared Flow Scale	Zumeta [82,83]
FIMA	Flow Indicators of Musical Activities Form	Custodero [84,85]

.

We also identified some interesting flow questionnaires which were not included in the final report because they did not match the PICOS 4 selection criteria. We summarize them in

Table 4. Questionnaires not matching PICOS 4, with reasoning. Source: self-made.

Questionnaire	Exclusion Criteria
Flow Questionnaire [1]	Not fully validated [81]
Flow Scale [86]	Unpublished thesis
Experience Sampling Form [49]	ESM questionnaire [34]
Pearce, Anley, and Howard questionnaire [26]	ESM; Contains open-ended questions
Flow in Environment Scale [87]	Unpublished thesis

, with reasoning.

3.2. Contexts and Dimensions of Flow Experiences Assessed with Validated Questionnaires (RQ2)

Due to its extension, we have summarized the contexts and dimensions for each of the methods in Appendix B as a table.

We identified the same nine flow dimensions in FSS-1, FSS-2, DFS-1, and DFS-2 questionnaires [19,62,64]. They assess flow prevalence or proneness in physical activity, art, free time, music, education, or for general purposes. They show the following dimensions:

• Challenge competence;
• Action awareness;
• Clear goals;
• Unambiguous feedback;
• Concentration;
• Sense of control;
• Loss of self-awareness;
• Transformation of temporal sense;
• Autotelic experience.

As an alternative to FSS scales in physical activities, FCS [50] distinguishes dimensions related to flow states, clutch states, effortlessness of flow, and overlapping characteristics. However, CFS [65] has no dimensionality, and it is suitable for assessing flow prevalence or proneness phenomenologically in the same contexts as the FSS and DFS scales.

SFPQ [66] is a scale that evaluates flow proneness by asking the same seven questions three times, considering work, leisure, and routine settings (called domains). It contains several dimensions:

• Concentration;
• Balance between skills and challenge;
• Explicit goals;
• Clear feedback;
• Sense of control;
• Lack of a sense of boredom;
• Enjoyment.

We will now discuss the FKS scales. We found two scales with a different number of items: FKS (13) and FKS (10) [67,68]. Both of them asses absorption and fluency in e-Learning, face-to-face courses, and computer games, but only FKS (13) evaluates the importance given to the task. We note that the first 10 items are the same for FKS (10) and FKS (13). Similarly, the RFSS scale evaluates absorption and fluency dimensions, but it does it in the context of reading [71].

On the other hand, some scales assess flow in work or study contexts. This is the case with the WOLF and WOLF-S questionnaires, which look at absorption, and also include enjoyment and intrinsic motivation [52,69]. Alternatively, the SFWS may be used [81], with no dimensions extracted.

Regarding flow in human–computer interaction, Internet browsing, or e-commerce, the FHCI [71], FS [72], FPS [40], AK [67], and FMI [75] questionnaires were valid. We compare and describe their dimensionality in the following paragraphs.

FHCI [71] is a short questionnaire used to measure flow during human–computer interaction. It shows a construct with control, attention focus, and curiosity dimensions. Conversely, FS [72] is very detailed, but is limited to assessing flow prevalence for Internet browsing. It contains the following dimensions:

• Use of the web;
• Arousal;
• Challenge;
• Control;
• Exploratory behaviour;
• Flow;
• Focused attention;
• Speed of interaction;
• Importance or addictiveness;
• Playfulness;
• Positive affect;
• Skill;
• Telepresence;
• Time distortion.

FPS [40] is a very detailed method for determining flow in Internet use and e-commerce, like FS [72]; however, it also distinguishes flow antecedents and consequences, not all of them positive (perfect or imperfect flow). For antecedents, it contains the following:

• Skills;
• Challenge;
• Interactive speed;
• Enduring involvement;
• Control;
• Arousal;
• Telepresence;
• Focused attention;
• Playfulness.

Moreover, AK [67] is also an appropriate and detailed method for measuring flow in web environments, with a different dimensionality than FPS [40] and FS [72]:

• Cognitive absorption;
• Perceived ease of use;
• Perceived utility;
• Personal tendency to innovation;
• Gameplay;
• Intention of use;
• Self-efficacy.

To construct questionnaires to evaluate flow prevalence in human–computer interactions and e-commerce, FMI questionnaires [75] include the following dimensions:

• Balance;
• Clear goals;
• Feedback;
• Concentration;
• Control;
• Mergence of action and awareness;
• Transformation of time;
• Transcendence of the self;
• Autotelic experience.

Additionally, we identified other scales validated in learning and educational contexts. The EduFlow-1 [12] and EduFlow-2 [73] scales determine flow prevalence in face-to-face and on-line education with 12 items grouped in four dimensions. They share loss of self-concern and well-being. However, the absorption dimension is slightly different, and immersion substitutes time distortion in the second version of the scale. As an alternative, GSR [22] contains engagement, enjoyment, and control dimensions. VFM [74] offers a similar option.

We can also distinguish other questionnaires linked to gamification, like FSG, EGameFlow, and GameFlow [77,78,79]. We will describe them as a group. GameFlow questionnaires are longer than the FSG. They share dimensions, but GameFlow [78] also evaluates gamefulness, frame story, exploratory behaviour, and learning. Moreover, as a difference, GameFlow dimensions contain flow consequences as well. EGameFlow makes no mention of flow antecedents or consequences, discerning:

• Concentration;
• Goal clarity;
• Feedback;
• Challenge;
• Autonomy;
• Immersion;
• Social interaction;
• Knowledge improvement.

Conversely, FSG [77] contains flow antecedents and flow experience dimensions to evaluate gamification as follows:

• Challenge;
• Clear goal;
• Feedback;
• Control;
• Playability;
• Concentration;
• Time distortion;
• Autotelic experience;
• Loos of self-consciousness.

Additionally, we can see a different group of questionaries, like FOG [39] and FIMA [85], able to determine flow by means of direct observation in video games and musical activities, respectively. FOG evaluates concentration, enjoyment, and frustration, whereas FIMA contains affective and behavioural indicators.

Moreover, FMQ [51] is the only scale to assess flow metacognitions, with two dimensions describing beliefs and confidence in ability to self-regulate flow. We also distinguish the FSSOT [80] scale, validated in occupational therapy to evaluate control and positive emotion.

As we found, APQ [54] is the only scale that determines the factors of autotelic personalities, as follows:

• Curiosity;
• Persistence;
• Low self-centeredness;
• Intrinsic motivation;
• Transformation of challenges into enjoyment;
• Transformation of boredom into enjoyment;
• Attentional control.

We also can observe that SFS [83] is unique, because it determines flow in groups, with several dimensions:

• Balance between challenge and skill;
• Action–awareness merging;
• Clear proximal goals;
• Unambiguous and direct feedback;
• Focused concentration on the activity;
• Sense of control over one’s actions;
• Loss of self-consciousness;
• Loss of time awareness/time acceleration;
• Autotelic experience.

3.3. Limitations of Use of Flow Questionnaires Considering Their Development and Validation Process

Several limitations were manifested by the authors regarding their own methods. Although a wide analysis can be found in Appendix C, in a table format, we include a summary of those limitations here.

Although some of the scales were described with no limitations (FKS (10) [68], SFPQ [66], EduFlow-1 [12]), most of them presents a common set of limitations, as we explain below.

The most common limitations are those related to general limitations of the surveying methods (DFS-1 [64], WOLF [69], RFSS [70], FPS [40], FMI [75], AK [76], EGameFlow [79], FSSOT [80], FOG [39], WOLF-S [52], SFWS [81], FMQ [51], APQ [54], SFS [82], FIMA [85], FKS (13) [67]).

Another highly cited limitation is related to the biased selection of participants, affecting the balance of datasets (FSS-1 [19], EduFlow-2 [73], FSS-2 [53], DFS-2 [73], CFS [65], WOLF [69], FOG [39], FCS [50]), or due to a small sample size (WOLF-S [52], FHCI [71], GameFlow [45], SFWS [81]).

We can also find a high occurrence of limitations related to preliminary findings (CFS [65], FHCI [71], FSG [77], GameFlow [45], APQ [54]) and to experimental conditions that were not fully controlled (DFS-1 [65], WOLF [70], RFSS [71], FHCI [72], GSR [22], FOG [39], SFS [84]).

Other authors suggest that their questionnaires were developed to assess flow in particular tasks (FSG [79], FMI [77], EGameFlow [79], FSSOT [80], FMQ [51], APQ [54], AK [76], FOG [39]).

Regarding items included in the scales, some authors indicated some limitations for particular items (VFM [77], GameFlow [45], SFWS [81], FS [72], GSR [22]) or lack of consensus about the need for some of them (APQ [54]). Additionally, the risk of respondent burden due to the length of the questionnaire is indicated in EGameFlow [79].

With the same occurrence, several authors found the risk of shared variance (FHCI [71], GSR [22], AK [76], SFS [82]) and the aim of their questionnaires (CFS [65], FSSOT [80], SFWS [81]) to be restrictive.

4. Discussion

In response to RQ1, we have identified 39 methods to determine flow using questionnaires (see Table 3 and Table 4). Nevertheless, only 34 methods matched PICOS 4, being validated non-ESM scales in English described in empirical peer-reviewed papers or book chapters. Moreover, although differences exist in quality evaluation with MMAT [48] among the selected works (see

Table A1. Mixed-Methods Appraisal Tool summary [48]. Source: self-made.

Flow Questionnaire			4. Quantitative Descriptive Studies					Comments
RefID	First Author	Year	4.1	4.2.	4.3	4.4	4.5
FSS-1 (36)	Jackson	1996	Yes	No	Yes	Can’t tell	Yes	Omitted nonresponse rates
FSS-1 (53)	Jackson	1996	Yes	No	Yes	Can’t tell	Yes	Omitted nonresponse rates
FSS-2 (36)	Jackson	2008	Yes	No	Yes	Can’t tell	Yes	Omitted nonresponse rates
FSS-2 (9)	Jackson	2008	Yes	No	Yes	Can’t tell	Yes	Omitted nonresponse rates
DFS-1(36)	Jackson	1998	Yes	No	Yes	Can’t tell	Yes	Omitted nonresponse rates
DFS-2(36)	Jackson	2008	Yes	No	Yes	Can’t tell	Yes	Omitted nonresponse rates
DFS-2 (9)	Jackson	2008	Yes	No	Yes	Can’t tell	Yes	Omitted nonresponse rates
CFS (10)	Martin	2008	Yes	No	Yes	Yes	Yes
SFPQ (21)	Ullén	2012	Yes	No	Yes	Can’t tell	Yes	Omitted nonresponse rates
FKS (13)	Rheinberg	2003	Yes	No	Yes	Can’t tell	Yes	Omitted nonresponse rates
FKS (10)	Engeser	2008	Yes	No	Yes	Yes	Yes
WOLF (13)	Bakker	2008	Yes	No	Yes	No	Yes
RFSS (8)	Thissen	2018	Yes	No	Yes	Yes	Yes
FHCI (11)	Webster	1993	Yes	Can’t tell	Yes	Can’t tell	Yes	Omitted nonresponse rates and students’ number
FS (65)	Novak	2000	Yes	No	Yes	Yes	Yes
FPS (63)	Saxena	2004	Yes	No	Yes	No	Yes
EduFlow1 (12)	Heutte	2016	Yes	Yes	Yes	No	Yes
Eduflow2 (12)	Heutte	2021	Yes	Yes	Yes	No	Yes
GSR (12)	Ghani	1991	Yes	No	Yes	Can’t tell	Yes	Omitted nonresponse rates
VFM (21)	Shin	2005	Yes	Yes	Yes	No	Yes
FMI (30)	Guo	2009	Yes	No	Yes	Can’t tell	Yes	Omitted nonresponse rates
AK (52)	Agarwal	2009	Yes	No	Yes	Yes	Yes
FSG (25)	Kiili	2006	Yes	No	Yes	Can’t tell	Yes	Omitted nonresponse rates
GameFlow (45)	Kiili	2008	Yes	No	Yes	Can’t tell	Yes	Omitted nonresponse rates
EGame Flow (66)	Fu	2009	Yes	Yes	Yes	No	Yes
FSSOT (14)	Yoshida	2013	Yes	No	Yes	Can’t tell	Yes	Omitted nonresponse rates
FOG (13)	Tordet	2021	Yes	No	Yes	Yes	Yes
FCS (22)	Swann	2022	Yes	No	Yes	Can’t tell	Yes	Omitted nonresponse rates
WOLF-S (13)	Bakker	2017	Yes	No	Yes	Can’t tell	Yes	Omitted nonresponse rates
SFWS (3)	Moneta	2012	Yes	No	Yes	No	Yes
FMQ (12)	Wilson	2016	Yes	No	Yes	No	Yes
APQ (26)	Tse	2020	Yes	Yes	Yes	Yes	Yes
SFS (27)	Zumeta	2015	Yes	No	Yes	Can’t tell	Yes	Omitted nonresponse rates
FIMA (18)	Custodero	2021	Yes	No	Yes	Yes	Yes

), none were discarded, as suggested by [48]. In particular, the methods were coded as FSS-1, DFS-1, CFS, FSS-2, DFS-2, SFPQ, FKS, WOLF, RFSS, FHCI, FS, FPS, EduFlow-1, EduFlow-2, GSR, VFM, FMI, AK, FSG, EGameFlow, GameFlow, FSSOT, FOG, FCS, WOLF-S, FMQ, SFWS, APQ, SFS, and FIMA. Of those surveys, 18 were short (between 3 and 13 questions), 6 had between 18 and 27 items, and 10 were quite detailed (between 30 and 66 items). In consequence, they are suitable for different research methods. In this sense, short questionnaires are preferred to avoid respondent burden [86]. However, they have the disadvantage that they describe fewer comprehensive models of flow and, if they only have a single item for each dimension (i.e., CFS or short FSS/DFS scales), they may not be acceptable psychometrically, but only be usable for phenomenological research [32,65]. Additionally, such questionnaires are intended to determine flow intensity (FSS-1), prevalence (most of them), proneness (DFS, CFS, SFPQ), variation (SFWS), metacognitions (FMQ), in crowds (SFS), observed (FOG, FIMA), and as dimensions of autotelic personalities (APQ). Moreover, only one questionnaire distinguished dimensionally between flow–clutch states (FCS), which differ mainly in the level of effort exerted. Nevertheless, in some questionnaires (GameFlow, FSG, FPS), such dimensions assess flow precursors, flow consequences, and/or the central experience of flow, without agreement about what dimensions must be included. Other questionnaires shed light on the flow attributable to means used [88] (i.e., computers, software, materials, etc.), such as FMI; few evaluated the negative aspects of the experience (FPS, FOG).

Furthermore, we describe the dimensionality of selected questionnaires in response to RQ2. In particular, we identified questionnaires with a minimum of 2 and a maximum of 14 dimensions. We also counted 63 flow dimensions from all of the questionnaires, conditioned by the context and aim.

Csikszentmihalyi [15] justified nine dimensions of optimal experiences, grouped into flow antecedents, core flow experience, and flow consequences [40,77,78] among others. With our proposal of dimensionality of the flow concept, presented in

Table 5. Reinterpretation of the nine-dimensional scheme of flow in three stages cited in Csikszentmihalyi and Csikszentmihalyi [14]. Occurrence of cited dimensions in validated questionnaires (in brackets). Source: self-made.

Dimensions of Flow Antecedents Related to Tasks

- Balanced challenge competence (13) - Clear, unambiguous, and direct feedback (8) - Clear, explicit, and proximal goals (7) - Playfulness/Gameplay/Gamefulness (6) - Progressivity of a task that makes it appear self-regulated (2) - Addictiveness (1) - Perceived ease of use (1) - Perceived utility (1)

Dimensions of flow antecedents related to personality traits

- Intrinsic motivation (3) - Curiosity (2) - Concern/Importance of the task (2) - Self-efficacy (1) - Lack of a sense of boredom (1) - Confidence in the ability to self-regulate flow (1) - Personal tendency to innovation (1) - Autonomy (1) - Intention of use (1)

Dimensions of core flow experience

- Concentration, immersion, or focused attention on a task (17) - Sense of cognitive control over one’s actions in the task (8) - Loss of self-concern/self-awareness/self-centeredness (8) - Cognitive absorption (8) - Altered perception of time (8) - Fluently/effortlessness of performance (7) - Action–awareness merging (4) - Engagement/persistence/enduring involvement (4) - Telepresence (3) - Arousal (2) - Exploratory behaviour (2) - Interactive speed (1) - Social interaction (1) - Speed of interaction (1) - Use of external stimuli (1)

Consequences of perfect flow

- Autotelic experience/well-being/enjoyment/positive affect (19) - Engagement/persistence/enduring involvement (4) - Intrinsic motivation (3) - Knowledge improvement (2) - Transcendence of the self (1) - Intention of use (1)

Consequences of imperfect flow

- Frustration (1)

, we group eight facets as flow antecedents, related to the tasks involved, and another nine to personality traits. Moreover, 15 dimensions may be associated with the core flow experience. Finally, flow consequences may be divided into consequences of perfect flow (six dimensions) and imperfect flow (one dimension). We would like to highlight that these two main subdivisions in the flow antecedents and consequences constitute a significant difference with the scheme of Csikszentmihalyi [15], which is justified because flow may also be seen as a personality trait [53,54,62,64,65,66], and not all flow consequences are positive [39,40]. We also summarize the dimensions, with the number of times that they are included in any validated questionnaire in brackets, in Table 5. Nevertheless, some dimensions may be interpreted either as antecedents, core, or as a consequence of flow (i.e., intention of use, intrinsic motivation, or persistence), due to the fact that psychological flow is not a linear process [14,26,34] among others.

Table 5 shows a significant difference between the dimensionality of all the questionnaires analysed, because it summarizes them and is not constricted by the contexts, giving a more detailed description of the flow concept. Moreover, the number of times that any dimension is cited may not be interpreted as a criterion to unveil which are the most important, because consensus may be a fallacy and all the questionnaires selected have been validated properly. Nevertheless, concentration (17), well-being (19), and personal skills and challenges perceived in the tasks (13), are the most assessed dimensions of optimal experiences. Additionally, bearing in mind such dimensions, we were unable to decide whether there are different flow states or if this diversity is explained because researchers focused on them in particular, which is enough to assess flow in some contexts (see [37]). In addition, Table 5 breaks the dichotomy among flow prevalence/proneness questionnaires with respect to FSS-1, FSS-2, DFS-1, DFS-2, and the items of the CFS scales. This also affects SFPQ surveys, as far as we identified some flow antecedents that may rely on personality traits, and not only in the task performed.

Table 5 inherits most flow dimensions from instruments used to evaluate optimal experiences in technology-mediated contexts, such as EduFlow-1, EduFlow-2, FOG, FKS, FHCI, GSR, VFM, AK, FSG, GameFlow, or EGameFlow. This does not mean that other questionnaires, with a previous validation process, may not be of interest for technology research, and highlights the importance reached in this field. For example, consulting Table A2, a developer interested in video games may search for the closest context to choose a short over a detailed survey, or maybe select the items to assess some aspects of interest only (like engagement, playfulness, addictiveness, arousal, fun, fluently of performance, etc.). Otherwise, regarding e-learning courses, questionnaires containing concentration or cognitive absorption dimensions may be preferred, like the FKS or EduFlow scales. Finally, only a recent questionnaire is intended to evaluate flow in groups, and this aspect of research should be considered.

We found general and particular limitations for each method, according to RQ3 (see Table A3). To not be reiterative, this time we focus on metanalytic limitations. Surveying has received some general criticisms in the scientific literature. In particular, participants may not be honest in their answers [89], respond randomly [90], or results may be affected by extraneous variables, like weather [91], mood [92], or time of the day [93]. Furthermore, self-reporting may be biased by the emotional intelligence of the participant, and the way in which the questions are formulated [62]. Moreover, in surveys, unconscious emotions cannot be captured [83] and the outcomes may be influenced by social desirability or culture, due to semantic differences [85]. Additionally, they are unable to survey participants constantly, because their activities may be interrupted, or affect their natural behaviour [73], and the retrospective recall is distorted by the memory processes [72,73,74,94,95]. All the questionnaires analysed in this review are highly contextualized to a particular activity or situation, which they evaluate (i.e., the FHCI [71] or FSG [77] questionnaires refer to a specific software). This fact means they cannot be used in other contexts without modification and a new validation process. Furthermore, we know that the items of the questionnaires were selected using previous definitions, questionnaires, and flow models, and the number of questions was conditioned by the final use of the instrument (i.e., long for in-depth models and short for ESM works).

We can distinguish some questionnaires according to their main purpose: to determine flow prevalence (most of them), intensity (i.e., FSS-1 [5]), shared flow (SFS [82]), observed flow (FOG [39], FIMA [84]), metacognition (FMQ [51]), traits (APQ [54]), or proneness (DFS-1 [65], DFS-2 [53], CFS [65], SFPQ [66]). Thus, propensity to the flow state may have an influence on the flow experience, to the point that with non-autotelic personalities, other flow questionnaires may not be useful [15].

Moreover, many flow prevalence scales have been used to predict performance in different settings, but the predictability depends closely to the contexts. For example, in a correlational study, FKS predicted performance in two courses, but not in playing a classic video game, like Pac-man [68]. Consequently, the selection of a particular questionnaire may impose a bias on the results of any research.

Some questionnaires may include factors or items that are not entirely clear, like those related to the concept of balance between skills and challenges [96], such as FKS [67,68], FPS [40], FS [72], FSG [76], GameFlow [78], EGameFlow [79], and FMI [75]. In addition, except for FPS [40] and FS [72], the questionnaires do not include questions about negative experiences which may be part of the processes that are taking place, thus they may be overly positive about the situations studied [39].

We can observe ambiguities in the instructions of the questionnaires and non-dimensionality, offering a combination that makes interpretation difficult. For example, SFPQ [66] may result in ambiguities in the specific tasks referring to work, routine, and leisure contexts illustrated in the instructions of the questionnaire. The scale is calculated as the sum of Likert points of the answered questionnaire, resulting in a loss of information about such contexts. Particularly, one such ambiguous instructions may be “When you are doing household work or other routine chores (e.g., cooking, cleaning, shopping) how often does it happen that…” [66] (p. 169), because one can be amused while shopping but bored when cleaning.

Scales show limitations due to the fact that flow states in sport can overlap with clutch states, which differ in the apparently low and high levels of exertion experienced by participants, respectively (i.e., FSS-1 [19], FSS-2 [53], and CFS [65]). Therefore, Swann et al. [38] suggest a general problem derived from poorly defined parametrizations of the flow state.

As a disadvantage of some agile questionnaires with only one item for each dimension, Engeser et al. [32] report that the results may not be acceptable psychometrically, as they are an aggregate. This may be something to consider when using GSR [22] or the FKS [67,68] questionnaires, because the skills and challenge variables are measured with a single item.

Occasionally, there may be bias derived from unbalanced datasets. In this regard, concerning a constraint of the WOLF inventory, Bakker [69] advises that he only managed to get the participants in his study to reach levels of low-intensity flow, which can bias it (the jobs were monotonous for them and there are few examples of attractive jobs). Additionally, Engeser et al. [32] consider that some methods, like the WOLF inventory, may be very dependent on the autotelic nature of flow experiences, so they may be more suitable for assessing amusing occupations.

This work also presents some limitations, because the language used is English, excluding questionnaires not reported in that language. In addition, the most important concern with works on Flow Theory is the difficulty of finding clean and comprehensive searches in scientific indexers, because the terminology of flow research is polysemic with many disciplines. Other works may be occluded because their titles and abstracts do not include any term or indication suggesting that a new questionnaire of flow determination is developed. Furthermore, the systematic and narrative reviews identified imposed scope limitations to particular contexts of Flow Theory. Still, in this work, we also used grounded theory to address the content analysis of selected texts and to identify potential works. Nevertheless, methodological limitations are imposed by the saturation principle, time, and budget constrictions [43].

5. Conclusions

In this systematic review, we found 34 articles containing validated questionnaires on Flow Theory, presented in English language papers, without context or scope restrictions. This was possible, despite the difficulties imposed by the polysemic term “flow” in scientific databases, by using a methodology that combines the PRISMA framework with grounded theory, in an eight-phase process. The literature review explained in this paper shows a significative increase in the number of questionnaires reported in previous systematic or narrative reviews [34,35,36,37,55,56,57,58,59,60,61,62,63], increasing the quality of the questionnaire validation process and limitations. This is coherent with our previous expectations, because the nature of the methodology used is cumulative.

The aim of the questionnaires identified in this paper goes far beyond the dichotomy of flow proneness–prevalence, as reported in the monographs analysed [31,32,33]. We also highlight that such a dichotomy may be artificial, because flow antecedents related to personality traits can be identified as preconditions of flow experiences.

Our results suggest that questionnaires are appropriate for identifying flow states, but not detailed enough to describe the complexity of optimal experiences. In this sense, the dimensionality of the surveys sheds light on the flow concept itself. Although not all dimensions are assessed with the same frequency, we cannot conclude here that there are different flow states depending on the activity performed, because every questionnaire was developed with particular aims.

This paper also facilitates the selection of flow questionnaires among a variety of possibilities for researchers and technology developers, due to the fact that most flow surveys were validated in technology-mediated contexts or may be extended to such fields. Additionally, new questionnaires may be improved or derived, considering the flow dimensions manifested here, to assess engagement in software development, e-Commerce, video games, gamification, augmented reality, e-Learning, sports, or work settings, among others.

Our results suggest a redefinition of psychological flow, as manifested by [38] (p. 249) when they conclude that “flow research is approaching a crisis point” due to inconsistencies in the definition of the flow concept and issues in the development of the flow measurement scales. Because of this, we also propose a shift to a more practical theory. Nevertheless, we found several questionnaires with a rigorous validation process published after 1991, where the limitations are mainly conditioned by the surveying method itself, and budget or contextual constraints.

Moreover, the flow concept is being improved by physiological research [29,30,34,97]. As a consequence, future research may include new measurement and computational methods to contrast flow dimensions and processes, in combination with questionnaires, which are very valuable, despite their limitations. In particular, we identified future work in which computer science may have a remarkable role, due to the fact that flow in groups is not very well understood, and only one questionnaire is devoted to it. In this regard, we are validating other questionnaires, like FKS or EduFlow, by using non-invasive wearable physiology devices in groups of students in natural contexts to shed light on the ongoing processes of optimal experiences.