Associations among Variables in Technology-Enhanced Phonological Awareness Programmes Based on a Meta-Analysis

Abstract

This article presents a quantitative approach of a systematic literature review, namely a meta-analysis, using 19 technology-based phonological awareness interventions carried out over the last decade, aiming at technology-mediated reading acquisition and focusing specifically on the capabilities of phonological awareness. The study showed consistent positive effects when compared with non-technological intervention programmes in preschoolers. The selected interventions fall into quasi-experimental designs with post-intervention measurements, and experimental and control groups. Aspects related to the participants, intervention or programme, methodology, and external factors to the research itself were coded and controlled. Associations found for the moderating variables were the type of technology used, the age of the participant, and the duration of the programme. We conclude by recognising the usefulness of a meta-analysis as an appropriate method that is capable of distinguishing among the various studies on the presence of effective factors in the development of phonological awareness instruction.

1. Introduction

Nowadays, more and more children are becoming familiar with the use of technology and digital resources at a very early age, constituting daily resources in their homes, which is why it is said that most preschool children already have experience with digital resources. Digital resources drive their interest and provide them with playful and interactive benefits [1]. At the same time, technology has been widely used to support and enhance literacy learning particularly literacy learning in preschoolers, putting into practice skills such as syllabic, lexical, intrasyllabic, and phonological awareness [2]. Technology can be a key factor in improving intervention programmes for phonological awareness [3]. This function is regarded as part of language development and is of high relevance at preschool age. Phonological awareness also becomes important to recognise and manipulate phonological units at the sentence, word, syllable, and phoneme level [4] and is key to intervention in the development of reading using capabilities such as isolating, identifying, segmenting, blending, deleting, adding, or substituting sounds of smaller units of language such as the word and syllable [2]. Research has abundantly shown that “phonological awareness is related to later literacy outcomes and that later literacy problems are associated with persistent problems in phonological awareness. The predictive power of phonological awareness for later literacy has prompted curriculum specialists to develop phonological awareness interventions in preschool and kindergarten” ([5] p. 2).

From a practical perspective, Balikci [3] considered five types of interventions: (1) technology-based interventions, which observe intervention programmes as mediated by educational electronic book activities and computer-aided phonological awareness programmes, where technology is used in training schemes and for preparing books in electronic forms; (2) shared paper book reading interventions, which consider the participation of families in interactive reading activities; (3) language-based interventions and (4) early literacy-based interventions, which implement phonetic awareness training programmes with teachers in classrooms; and (5) art-based interventions, which use effective methods for developing children’s phonological awareness skills using songs and other routines. This work is part of the first approach: technology-based intervention programmes.

Studies on meta-analyses were also found. The literature review showed a meta-analysis of 67 studies in primary education that enhanced technology in word reading interventions producing, on average, moderate positive effects on accuracy and speed of word learning across all types of programmes and languages [6]. Another study showed the results of 20 randomised and quasi-experimental interventions run on 7000 primary school students [7], reinforcing the idea of using technology in schoolers with greater attention to phonological awareness. One of the studies distinguished two types of use of technology in early childhood education: the first one supports basic skills and attitudes, and the other focuses on content and individual learning needs [8]. These authors concluded that the first use of technology frequently occurs in teachers’ professional development as a crucial factor in stimulating its use. The second use of technology supports content and individual learning needs to favour the expected learning processes. Another aspect to consider is that teachers must become aware of their role in using digital resources, since integrating technologies in early childhood is different from integrating them at primary or secondary education levels.

Another meta-analysis with 46 experimental and quasi-experimental interventions brought evidence that suggests students can receive phonological awareness instruction from a teacher, computer program, or parent to help them develop phonological awareness skills, although parents and teachers need access to quality resources and programs. “Overall, teachers can utilize both computer and parent instruction to supplement phonological awareness instruction occurring in the classroom, including students who are struggling or at-risk of reading difficulties” (p. 1286) [9].

A recent systematic literature review reported on learning with digital resources as a component that improves phonological awareness at preschool age using articles indexed in Scopus from 2010 to 2020 [1]. Their research showed that technology-based or technology-assisted reading interventions for elementary grades, when experimental interventions were applied to phonological awareness, had a positive effect on individual results. Finally, a conclusion given in this study pointed to two broad conceptual categories: a cognitive approach called “computer-assisted learning systems”, and a formative approach called “computer-assisted instructional systems”. Both categories had a limited number of interventions when technological resources in the development of phonological awareness were in use, but declared they may strengthen early literacy, language, reading, and writing. Few studies have examined the effectiveness of using technology to improve phonological awareness interventions in children [10]. Recently, [1] reported that “there is no compilation study for experimental interventions applied to phonological awareness in early childhood (p. 286)”.

Why a Meta-Analysis to Study Phonological Awareness Technology-Based Interventions?

The meta-analysis is understood as the quantitative approach of a systematic review. In a meta-analysis, the analysed information combines the effects from pooling independent and multiple studies, and is used (1) to provide a combination of the estimate of the size of the effects, and (2) to analyse the flaws the effects may present due to the model’s fitting or the analysis of the moderator variables [11]. Moreover, a meta-analysis increases the precision, objectivity, and replicability of estimating the effects. Precision refers to the measure of the likelihood of random errors in the results for the group of articles identified in the search strategy. Likewise, objectivity means the explicit operationalisation of the concepts involved, and replicability refers to sufficient transparency in the meta-analysis process so that an independent repetition can be carried out with the same decision criteria, leading to the same results [11]. To observe the size of the effects, a graphical representation is provided, among which is the forest plot, which is well known for associating the points and intervals of the studies pooled in a meta-analysis.

A meta-analysis study is an appropriate method when there is heterogeneity in the studies collected. The present research required us to predict the acquisition of reading skills, a heterogeneous feature. It allows factors to be improved before any formal reading instruction to ensure the positive effects in multiple studies.

This research is a continuation of the study published by [1], and the objective of our work is to show the usefulness of meta-analyses for the study of phonological awareness based on technology-based interventions. Our study concluded that a meta-analysis shows positive effects when using technology-based programs, such as learning apps and electronic toolkits such as multimedia content, in an integrated system of a learning environment designed with a phonological awareness purpose. Our contribution focused only on the technology-based interventions identified in [1].

2. Methodology

To identify the positive associations between the moderating variables in the studies synthesised in [1], we consistently followed the standard Cochrane methodology (2012).

2.1. Research Questions

Consistent with the standard Cochrane methodology, we formulated the general research question based on the systematic review standard for a research question: population, intervention, comparison, and outcomes (the PICO elements), allowing the following research question’s replicability:

To what extent does a meta-analysis (O for outcomes) in the phonological awareness technology-based intervention programmes (I for intervention) confirm there is an improvement in preschoolers’ reading skills (P for population), in comparison with non-technological intervention programmes (C for comparisons)?

In addition, to formulate a general research question, the following specific research questions were complementary:

(a)

What associations can be found between moderator variables such as the age of the preschoolers, the intervention time, diversification of educational needs, languages in the intervention programmes, and the technology use?

(b)

To what extent can the intentional allocation of the subject to the analysis of the groups’ influence be randomised or measured by an instrument?

2.2. Research Design

This methodology included the five steps proposed in [11]: (a) a formulation of the problem, (b) the search for studies, (c) selection of the moderating variables, (d) statistical compilation, and (e) the dissemination and replicability of the study.

(a)

Formulation of the problem

The problem is grounded on the idea that phonological awareness interventions in preschoolers may improve whenever they are mediated by technology. The proposition seeks to prove the effectiveness of the comparison among the interventions.

(b)

Search for studies

In a complete research body of 24 selected papers shown in Appendix A [1], only 12 papers met the following eligibility criteria (these are the studies marked with * in references [12,13,14,15,16,17,18,19,20,21,22,23]):

• The study included a quasi-experimental design during the pretesting, intervention, and post-testing processes.
• The study included experimental and control groups.
• The study included one or more practical perspectives on technology-based phonological awareness interventions [1].
• The study included published a standardised norm-referenced achievement test or at least a researcher-designed criterion-referenced instrument [24].
• The study included a complete description of the phonological awareness programme which specified its objectives, contents, resources, and implementation settings (place and time).
• The study included available data, descriptive statistics of the experimental and control groups, and the number of subjects in each group.

(c)

Selection of moderating variables

The codification of moderating variables was for primary studies. They included the following categories:

(c.1) Moderating variables of participants

• Age coded the participants’ age in months. The age range may vary between 51 and 75 months (4.25 and 6.25 years), although one study included an age of 168 months (14 years) for a participant with a special condition.
• Diversification of educational needs coded studies on various educational needs such as cognitive, socioeconomic (low socioeconomic status (LSES) and middle socioeconomic status (MSES)), and cultural needs (ethnic minorities).
  (c.2) Moderating variables of the type of intervention programme.
• The nature of the technology coded the type of digital resources used in the intervention programme, as well as the teaching strategy and the evaluation of these resources. The intervention must promote technology with other conventional materials such as books and electronic books. Other interventions may include gamification (based on play) and simulation (based on imitation) resources.
  (c.3) Moderating variables of methodological features:
• The duration of the programme coded the time spent in the instructional processes and the total duration of the programme. There were two options: (1) technology-based phonological awareness intervention programmes with less than 500 min overall, and (2) technology-based phonological awareness intervention programmes with more than 500 min overall. This variable was coded as binary because of its outcome.
• The assignment of participants coded random or non-random assignment of the participants in both the experimental and control groups.
• The measured instrument variable coded the standardised measure and ad hoc instrument used for each specific intervention programme.
  (c.4) Moderating variables of external factors to the intervention programme, such as orthography and code attributes such as the degree of opaqueness, depth, and transparency of the language.

(d)

Statistical compilation

Statistics for the selected studies were compiled, leaving out non-relevant qualitative measures. These were uploaded in the free Metafor meta-analysis package for R (2019) [25], the user interface Jamovi (2020), and the online free software MAVIS v.1.1.3 [26].

We opted for the effect size measure and the selection of studies with pretest interventions and post-test designs (with experimental and control groups). The effect size was calculated as the standardised mean difference among the studies and the estimator was Hedges’ g.

Considering the population differences for the phonological awareness intervention programmes, that is, the high heterogeneity expected, we used statistical models with random effects. However, for the detection and control of publication bias, we used graphical methods such as the funnel plot and Rosenthal’s fail-safe N, Begg’s test, and Mazumdar’s rank correlation indexes.

(e)

Dissemination and replicability of the study

We claim the replicability of our meta-analysis based on three out of six elements for any generic meta-analysis methodology according to [27], namely:

• The standard PICO elements were reported.
• We determined how sensitive the meta-analysis was by giving the studies’ inclusion and exclusion criteria and by providing reproducible scripts with both the data and the reported analyses in the open-source software R. The current study can easily be analysed in any statistical program.
• The cumulative statistics for future-proofing meta-analyses are reported, such as the effect sizes, the sample sizes for each selected study, the test statistics, the degrees of freedom, the means, the standard deviations, and the correlations between dependent variables for each data point or study selected.
  Additionally, the effect sizes, the sample sizes for each study, the test statistics, the means, the standard deviations, and the correlations between moderating variables were disclosed.
  Table 1. Compilation of the selected studies in the meta-analysis.

  Intervention Programme	Year	Nc	M_c	DS_c	N_e	M_e	DS_e	Intervention (Minutes)	Diversity	Language	ICT	Age (Months)	Measuring Instrument	Randomisation
  Korat_Blau-PK	2010	21	0.14	1.33	21	2.08	3.67	125	Yes: LSES	Hebrew	Simulation	54	V	Yes
  Korat_Blau-K	2010	20	0.65	2.08	20	1.8	2.9	125	Yes: LSES	Hebrew	Simulation	66	V	Yes
  Korat_Blau-PK	2010	21	1.75	2.69	21	1.65	3.11	125	Yes: MSES	Hebrew	Simulation	54	V	Yes
  Korat_Blau-K	2010	20	0.65	2.08	20	0.15	3.66	125	Yes: MSES	Hebrew	Simulation	66	V	Yes
  Karemaker_PAT_graphemes	2010a	9	26.94	7.3	8	28.29	9.1	500	No: 0	English	Gamification	73	E	No
  Karemaker_PAT_graphemes	2010b	9	26.94	7.3	8	28.29	9.1	500	No: 0	English	Gamification	73	E	No
  Hein et al.	2010	10	0.1	2.13	10	7.5	2.8	1280	Yes: intellectual disability	Portuguese	Gamification	168	E	Yes
  Wolgemuth et al._GradeK_Non_Indigenous	2011	19	0.81	0.23	51	1.46	0.14	1200	No: 0	English	Gamification	70	E	No
  Wolgemuth et al._GradeK_Indigenous	2011	29	−0.10	0.18	67	0.83	0.12	1200	Yes: Australian Aborigines	English	Gamification	70	E	No
  Willoughby et al._TOPAK_storybook	2015	30	4.60	2.87	29	4.66	2.60	320	No: 0	English	Simulation	51	E	Yes
  Willoughby et al._TOPAK_eBook	2015	30	4.60	2.87	33	4.76	2.49	320	No: 0	English	Simulation	51	E	Yes
  Korat et al._PA_5Act_PK	2016	36	0.38	1.71	36	1.66	3.3	125	No: 0	Hebrew	Simulation	61	V	Yes
  Korat et al._PA_5Act_K	2016	36	0	2.96	35	1.27	2.43	125	No: 0	Hebrew	Simulation	70	V	Yes
  Kartal and Terziyan	2016	10	35.5	14.56	10	42.4	13.4	167	No: 0	Turkish	Gamification	60	E	No
  Kartal et al._K	2016	20	13.57	10.04	16	17.55	11.75	204	No: 0	Turkish	Gamification	61	E	No
  Kartal et al._G1	2016	22	40.79	15.17	22	40.99	12.28	204	No: 0	Turkish	Gamification	75	E	No
  Rogowsky et al._TOPEL	2017	22	98.68	14.42	24	105.46	13.5	550	No: 0	English	Gamification	54	E	Yes
  Elimelech and Aram_word_spelling	2020	33	0.57	0.87	33	1.98	0.79	160	No: 0	Hebrew	Gamification	69	E	Yes
  Elimelech and Aram_word_decoding	2020	33	0.82	1.26	33	2.08	1.53	160	No: 0	Hebrew	Gamification	69	E	Yes
  Amorim et al.	2020	392	11.5	6.2	306	12.4	6.6	900	No: 0	Portuguese	Gamification	56	E	No

  Note: author_PK indicates that the study was conducted in prekindergarten children, author_K indicates that the study was conducted in kindergarten children, author_Grade 1 indicates that the study was conducted in Grade 1 children, author_PA indicates that the study was conducted with a focus on phonological awareness, N_c is the number of subjects in the control group, M_c is the mean score of the control group, DS_c is the standard deviation of the control group, N_e is the number of subjects in the experimental group, M_e is the mean score of the experimental group, DS_e is the standard deviation of the experimental group, V indicates validated instrumental measurement, and E indicates ad-hoc instrumental measurement.

  reports the data collection that best fits the technology-based intervention programmes for the 12 studies included in the meta-analysis. The methodology allowed for the preparation, the quality of the information, and an evaluation of the moderating variables.

3. Results

In answer to what extent the meta-analysis in the technology-based phonological awareness interventions showed positive effects in comparison with the non-technological intervention programmes in preschoolers, Figure 1 displays a forest plot graphical representation of the individual results included in the meta-analysis together with the pooled result of the meta-analysis. The result was 0.92 (p = 0.01, 95% CI [0.22, 1.62]). The results in the forest plot indicated that technology-based phonological awareness interventions programmes (the experimental group), on the right of the vertical line, were favourably measured (a higher impact factor) in comparison with non-technological intervention programmes for preschoolers (control group), on the left of the vertical line. It can also be seen in the forest plot that the studies showed the direction of the effect, and the confidence interval of the combined effect did not include zero within it. Thus, it would not be reasonable to conclude that there was no effect in the analysis [28].

In

Table 2. Random effects and heterogeneity statistics in the meta-analysis model of technology-based phonological awareness interventions.

Random-Effects Model (k = 19)

Estimate

se

Z

p

CI Lower Bound

CI Upper Bound

Intercept

0.989

0.387

2.55

0.011

0.230

1.748

Heterogeneity Statistcs

Tau

Tau2

I2

H2

R2

df

Q

p

1.651

2.7269 (SE = 0.9514)

97.59%

41.431

18.000

266.773

<0.001

Note. Tau² Estimator: Hedges.

, we used a random effects model [29] which showed the heterogeneity among the selected studies. The random effects model was chosen with the I² estimator hedges, i.e., the variability in the effect due to heterogeneity, where I² = 97.59% was observed, meaning that there was high heterogeneity among the studies. The assessment of publication bias consisting of Rosenthal’s fail-safe N was high (value = 852.0, p < 0.001) and the p-value of Begg’s and Mazumdar’s rank correlation tests was 0.186, suggesting the absence of publication bias [30]. However, Egger’s test yielded low results that indicated the presence of bias (value = 2.86, p = 0.004). This situation must be considered when assessing and interpreting the assessment of publication bias. In contrast, regarding the existence of publication bias, a graphical method, namely a funnel plot, detected heterogeneity by sensing the data points within a triangular shape (Figure 2) that had symmetrical behaviour. Figure 2 shows the absence of bias; the dotted line allows us to perceive a symmetrical graph, which indicates that all relevant articles have been included in the analysis.

Answering what associations were found between the moderator variables, namely (a) the age of the preschoolers, (b) the diversification of educational needs, and (c) the nature of the technology implemented, the existing heterogeneity among the different studies was analysed by calculating the random effects model for each subgroup using the moderator variables’ categories (attributes). To observe the pooled effect in the moderating variables, the effect size, variance estimates, and regression analysis were used. The regression analysis showed an explanatory model of the selected studies [31].

To determine the influence of the moderating variables on the model’s variability and the intensity of the effects for each selected intervention programme, the following statistics were considered: (1) the average of the effect (variance estimates) for each moderator variable’s category, (2) the statistical significance of the model, (3) the inter-category (Qw) and intra-category (Qb) homogeneity, and (4) the variability in the effect due to heterogeneity (I²).

3.1. Associations among the Moderating Variables of Participants

Two key moderating variables of the participants were analysed.

(a)

Age: The analysis of the results was based on the participants’ age.

Table 3. Associations between the moderating variables of participants.

Associations found moderator variable configured on participants: AGE
mod	k	estimate	var	se	ci.l	ci.u	z	p	Q
>5 years	3	0.8891	0.4063	0.6374	−0.3603	21.384	13.948	0.1631	16.730
4 years	5	0.1326	0.2104	0.4587	−0.7664	10.316	0.2891	0.7725	1.890
5years	11	13.476	0.1006	0.3171	0.7260	19.692	42.492	0.0000	195.730
Overall	19	0.9452	0.0583	0.2414	0.4720	14.184	39.152	0.0001	2088.870
df	p.h	I2
1	0.0002	89%
2	0.7548	0%
3	0.0000	95%
4	0.0000	93%
Heterogeneity
Q	Qw	Qw.df	Qw.p (inter)	Qb	Qb.df	Qb.p (intra)
266.7727	214.3623	16	0	4.7562	2	0.0927
Associations found moderator variable configured on participants: DIVERSIFICATION OF EDUCATIONAL NEEDS
mod	k	estimate	var	se	ci.l	ci.u	z	p	Q
no	13	0.6773	0.0823	0.2868	0.1151	12.395	23.612	0.0182	1065.485
yes	6	15.608	0.1901	0.4360	0.7063	24.153	35.802	0.0003	1494.238
Overall	19	0.9442	0.0574	0.2396	0.4746	14.139	39.404	0.0001	2667.727
df	p.h	I2
12	0.0000	89%
15	0.0000	97%
18	0.0000	93%
Heterogeneity
Q	Qw	Qw.df	Qw.p (inter)	Qb	Qb.df	Qb.p (intra)
266.7727	255.9723	17	0	2.8663	1	0.0905

shows the summary of this model under three categories: the participants’ age could be 4, 5, or above 5 years old. The average effect was high in the group of those aged 5 years and above. In the category of those aged 4our years, there was no effect. Only the model relative to those aged 5 years was statistically significant (p = 0.00). The I² indicates the percentage of heterogeneity in these groups. I² was low for the 4-year-old subgroup but high in the other two subgroups. We considered the 4-year-old subgroup to assess the validation of the effects in the meta-analysis. There were no significant inter-category differences in the groups (Qw) (p = 0.00) but significant differences within a category (Qb) (p = 0.09).

(b)

Diversification of educational needs: The analysis of the results based on this moderating variable was ample. The concept of the diversification of educational needs can be in a cognitive, socioeconomic, or cultural context. This was observed in 6 out of 19 instances in Table 1. The average effect became high when the concept of diversification appeared in the studies. The categories informed us whether the diversification of educational needs was considered in the study. In Table 3, a high effect is observed for each subgroup. The model was statistically significant for both subgroups (p = 0.01; p = 0.00). I² is high for each subgroup. There were no significant inter-group differences (Qw) (p = 0.00) but significant differences within a category (Qb) (p = 0.009).

3.2. Associations among Moderating Variables of the Type of Intervention Programme

The moderating variable of the nature of the technology implemented was based on the type of technology used in the intervention programme.

Table 4. Associations among the moderating variables of the nature of the ICT resources.

Associations found moderator variable configured on the type of intervention program: NATURE OF THE ICT IMPLEMENTED RESOURCES
Subgroup	k	estimate	var	se	ci.l	ci.u	z	p	Q
Gamification	11	14.954	0.1145	0.3384	0.8321	21.587	44.187	0.0000	2548.282
Simulation	8	0.2458	0.1488	0.3858	−0.5103	10.019	0.6372	0.5240	79.157
Overall	19	0.9519	0.0647	0.2544	0.4533	14.506	37.419	0.0002	2667.727
df	p.h	I2
10	0.0000	96%
7	0.3401	12%
18	0.0000	93%
Heterogeneity
Q	Qw	Qw.df	Qw.p	Qb	Qb.df	Qb.p
266.7727	262.7439	17	0	5.9297	1	0.0148

shows an average effect of high magnitude when gamification-based tools were used and the low magnitude of the effect when the technology was used for simulation purposes. The average effect of gamification was statistically significant (p = 0.00) but this was not found for simulation (p = 0.52). I² for simulation interventions was low (12%) while that for gamification was highly variable (96%). The inter-category homogeneity statistic (Qw) showed significant differences in the groups (p = 0.00), and the same was seen for the intra-category homogeneity statistic (Qb) (p = 0.01), indicating the insufficiency of this variable to correctly specify the model.

3.3. Associations among the Moderating Variables of the Methodological Features

The methodological features studied were the duration of the programme, the assignment of participants, and the measurement instruments.

(a)

Duration of the programme. This moderating variable considered the duration (time) in minutes.

Table 5. Associations among moderating variables of the methodological features.

Associations found moderator variable configured on methodological features: INTERVENTION TIME WITH PARTICIPANTS
mod	k	estimate	var	se	ci.l	ci.u	z	p	Q
<=500 min.	13	0.4138	0.1062	0.3259	−0.225	10.527	12.698	0.2042	352.187
>500 min.	6	22.240	0.2459	0.4959	1.252	31.960	44.845	0.0000	2315.522
Overall	19	0.9598	0.0742	0.2724	0.426	14.937	35.241	0.0004	2667.727
df	p.h	I2
12	4 × 10−4	66%
5	0.00E	98%
18	0.00E	93%
Heterogeneity
Q	Qw	Qw.df	Qw.p	Qb	Qb.df	Qb.p
266.7727	266.7709	17	0	9.3043	1	0.023
Associations found moderator variable configured on methodological features: ASSIGNMENT OF PARTICIPANTS TO GROUPS
mod	k	estimate	var	se	ci.l	ci.u	z	p	Q
Non-random	7	15.610	0.1983	0.4454	0.6881	24.339	35.051	0.0005	2180.472
random	12	0.6185	0.1115	0.3340	−0.0361	12.730	18.520	0.0640	471.084
Overall	19	0.9577	0.0714	0.2672	0.4340	14.814	35.845	0.0003	2667.727
df	p.h	I2
6	0	97%
11	0	77%
18	0	93%
Heterogeneity
Q	Qw	Qw.df	Qw.p	Qb	Qb.df	Qb.p
266.7727	265.155	17	0	2.867	1	0.0904
Associations found moderator variable configured on methodological features: MEASUREMENT INSTRUMENT
mod	k	estimate	var	se	ci.l	ci.u	z	p	Q
Standardized	13	12.553	0.0926	0.3043	0.6588	18.518	41.249	0.0000	260.270
Validated	6	0.3155	0.1923	0.4385	−0.5439	11.749	0.7195	0.4718	59.401
Overall	19	0.9498	0.0625	0.2500	0.4597	14.398	37.989	0.0001	266.772
df	p.h	I2
12	0.0000	95%
5	0.3121	16%
18	0.0000	93%
Heterogeneity
Q	Qw	Qw.df	Qw.p	Qb	Qb.df	Qb.p
266.7727	266.2103	17	0	3.1007	1	0.0783

shows two categories: less or equal to 500 min on average, and more than 500 min on average for the intervention programme. The duration of the programme was a relatively homogeneous category (I² = 66%). However, interventions lasting more than 500 min on average showed a high effect and statistical significance (p = 0.00). The average effect in other cases was moderate and not statistically significant (p = 0.20). The inter-category homogeneity statistic (Qw) showed significant differences in the groups (p = 0.00) and the same was seen for the intra-category homogeneity statistic (Qb) (p = 0.02).

(b)

Assignment of participants to groups. This moderating variable considered the assignment of the students to the groups. The model used two categories: probabilistic and non-probabilistic assignments. Table 5 shows a heterogeneous model (I² = 97%, 77%); however, it was less heterogeneous when the assignment was randomised. The model for non-randomised assignment was statistically significant (p = 0) but this was not the case for randomised assignment (p = 0.06). The average effect was of moderate magnitude for the randomised assignment; otherwise, it was of high magnitude. The inter-category homogeneity statistic (Qw) showed differences between the categories (p = 0.00) but non-significant differences within the same category (p = 0.09).

(c)

Measurement instruments. This moderating variable refers to the technical characteristics of the instrument used for measuring phonological awareness during the research process. Table 5 shows two categories: standardised or validated tests to measure the intervention programme. The category of validated tests was a relatively homogeneous category (I² = 16%) and was not statistically significant (p = 0.47), while the standardised test category showed heterogeneity (I² = 95%) and statistical significance (p = 0.00). The average effect was high for standardised tests, and the opposite was true for the other category. The inter-category homogeneity statistic (Qw) showed no differences between the categories (p = 0.47), but significant differences within the same category (Qb) (p = 0.00). This result is not surprising, since several ad hoc tests included in the technology-based phonological awareness interventions were translated from other languages; presumably, this has also to do with the transparency or opaqueness nature of the spelling of the language included in the body of literature in our work.

3.4. Associations among the Moderating Variables of the External Factors to the Intervention Programme

The moderating variable for the orthography of the language considered the transparency or opaqueness of the spelling of the language.

Table 6. Associations among moderating variables of external factors to the intervention programme.

Associations found moderator variable configured on external factors to the intervention program: ORTHOGRAPHY OF THE LANGUAGE
mod	k	estimate	var	se	ci.l	ci.u	z	p	Q
Opaque	14	10.503	0.0908	0.3014	0.4595	16.410	34.846	0.0005	223.329
Transparent	5	0.6774	0.2628	0.5126	−0.3273	16.821	13.214	0.1864	189.324
Overall	19	0.9545	0.0675	0.2598	0.4452	14.637	36.736	0.0002	266.772
df	p.h	I2
13	0.0000	94%
4	0.0000	79%
18	0.0000	93%
Heterogeneity
Q	Qw	Qw.df	Qw.p	Qb	Qb.df	Qb.p
266.7727	242.2621	17	0	0.3933	1	0.5306

shows a high average effect in both categories; however, only the opaqueness was statistically significant (p = 0.00). This was a heterogeneous model (I² = 94%). It was less heterogeneous for a transparent language. The inter-category homogeneity statistic (Qw) showed differences between categories (p = 0.00) but non-significant differences within a category (Qb) (p = 0.53).

4. Discussion

Regarding the Meta-Analysis

In answering the main question, the meta-analysis led to detailed conclusions on the effectiveness of the technology-based phonological awareness intervention programmes for preschoolers in comparison with programmes that did not drive the need for technology. The authors of [16,17,22] have shown that technology is more effective than traditional printed texts in training phonological skills and developing phonological awareness, and in the acquisition of reading and writing in preschoolers, especially in those who have difficulties learning to read; likewise, they reported that learning with this type of resource has a playful component that can improve phonological skills in preschoolers.

The calculation of the pooled effect size of the 19 technology-based phonological awareness intervention programmes included in the 12 selected studies produced a Hedges’ estimator (g) result of 0.92 (p = 0.01, 95% CI [0.22, 1.62], which leads us to question the significance of the results obtained. If we used Cohen’s criterion [32], we would be looking at a relatively large effect; however, this decision seems to be excessively arbitrary and the act of assimilating this index to a p-value may be misleading. Assessing the significance of a result solely based on the statistical results could lead us to overlook the substantive significance of the result. Thus, the first conclusion was established by estimating the pooled effect (i.e., the pooled result of the meta-analysis) as significant and positive. This means that technology-based phonological awareness intervention programmes improve with technological support.

For instance, the average effect of technology-based phonological awareness intervention programmes focusing on gamification improved, given specific conditions such as when the age of the participant was around 5 years old and the duration of the programme was longer than 500 min. However, the moderating variable of the age of the participants in the selected body of research was highly significant and also true because in most of the intervention programmes, the participants were around 5 years old (53%). In another case, the moderating variable of the duration of the programme showed a significant average effect when dichotomised and set higher than 500 min. Additionally, the percentage of studies using technology based on simulation tools was similar to those using technology based on gamification; however, the latter showed a significant average effect over the pooled effect, as observed by [2,3].

On the other hand, probabilistic and non-probabilistic assignment of the participants to groups notably influenced the meta-analysis. There were better results observed when the assignment was non-probabilistic, since technology-based phonological awareness intervention programmes present an intervention that is highly dependent on the responsible professional. It seems that the fact of a non-random assignment to experimental and control groups is associated with higher values of the average effect. Otherwise, if the assignment was random, the values of the effect were lower. A similar discussion can be found in [33,34]. Moreover, [5] showed the importance of the methodological rigor of the study designs used, as the effect sizes were larger in cases of no randomisation and comparisons with regular classroom teaching instead of an active control group.

Additionally, when the measurement instruments were standardised, they provided measures with better quality for comparative purposes, although some were translated into another language, other than ad hoc instruments specifically created in the research work, where the results showed higher values for the average effect and statistical significance.

The associations among moderating variables of the methodological features sufficiently explained the meta-analysis model, leading to the conclusions that the duration of the programme, the assignment of participants to groups, and the measurement instrument, can be treated as statistical explanatory variables instead of moderating variables. However, the moderating variables considered here displayed non-significant variability within a category (Qb), with the opposite happening when the inter-category homogeneity statistics were observed (Qw). This may suggest that the variables by themselves did not fully explain the variability of the meta-analysis model.

The moderating variable of the diversification of educational needs was presented in a broader sense and was identified in a small percentage of the studies (21%), categorised as ethnic minorities at risk of exclusion and cognitive diversity. The average effect of this group was significantly high, validating this variable in the meta-analysis according to its importance for future work in this domain.

Similarly, the moderating variable of the orthography of the language showed that 26% of the works had transparent languages (Portuguese and Turkish) and 74% of the works had opaque languages (English and Hebrew), with a higher effect for opaque languages due to the weight of these interventions.

It is interesting to notice that the meta-analysis in [9] included the moderating variables of the type of instructor, the intervention groups’ size, phonological awareness skills, and literacy use. We offer a meta-analysis that includes a major group of variables either for the intervention programme or the participants’ knowledge. Both meta-analyses mentioned the importance of technology in the improvement of phonological awareness programmes, also due to the regular use of computers and tablets in homes and schools.

5. Conclusions

• The main contribution of the meta-analysis of technology-enhanced phonological awareness is given by demonstrating that it is a proper method capable of distinguishing among multiple studies and selecting effective variables for developing phonological awareness instruction. It is recommended to use a systematic and explicit approach to structure any formal intervention and to ensure positive effects in practicing experimental studies.
• Introducing phonological awareness intervention programmes with educational and gamification functions gives a novel curricular approach to linguistic competence in preschoolers.
• A standard protocol is provided to be shared with professionals, practitioners, and the scientific community, allowing the reproducibility of the phases in phonological awareness intervention programmes.
• This work may be a guide for experts and practitioners who are aware of the implications of phonological awareness interventions and technological programmes in children’s development.

5.1. Research Implications and Opportunities

Assignment of the participants to groups is a methodological challenge that opens research questions such as (1) what ways of standardising the data can be obtained during the intervention programme, and (2) how it may affect the dissemination of results for both experimental and control groups in the pretest and post-test phases. The assignment task can also be assisted by technology, simplifying the intervention that is highly dependent on the responsible professional.

The research gap of technology-based phonological awareness intervention programmes is aimed at considering various educational needs such as cognitive, socioeconomic, and cultural needs, as well as interventions that aim at languages with a certain degree of opaqueness, depth, and transparency. More specifically, the greater or lesser depth of their orthographies in reading mediated by new technologies is suggested as a research opportunity.

5.2. Practical Implications

An improvement in phonological awareness was observed with technological support. A recommendation is to introduce phonological awareness intervention programmes with educational and gamification functions to support novel curricular subjects of linguistic competence for preschoolers. Such curricular content should be considered also at a first-grade level, since studies aimed at children are cognitively paired with preschoolers.

For teachers, educators, and curriculum decision-makers, it is recommended to promote the use of technological resources in interventions of phonological awareness. Moreover, since the effect of gamification technologies was more significant, it is recommended that this technology be prioritised over others. Given young children’s preference for games, a game-related intervention is sound.

On the other hand, given the intrinsic characteristics at the epistemological and practical level of the educational field, we were likely to find little homogeneity in the analysed body of research. The variability was as high as I² = 97% because of the variability observed in each study. A possible pathway to adopt standardised methodologies such as systematic literature reviews and their embedded protocols (i.e., PICO questions, meta-analyses, and so on) may deal with the intrinsic heterogeneity fundamentally present in this research.

Finally, this research work is meant to be a standard protocol to be shared with professionals, practitioners, and the scientific community, allowing the reproducibility of the phases for other intervention programmes, and serving as a guide for experts and practitioners who are aware of the implications presented for children’s development.

5.3. Considerations of the Limitations of the Meta-Analysis

As the research body of 12 selected papers met the initial inclusion criteria to develop the meta-analysis, it would be convenient to increase the sample size with estimations coming from future-proofing meta-analyses, as [27] affirmed.

Given the existence of studies in which data relating to the pretest were not included, the premeasures were ignored in the meta-analysis. Whether or not the selected studies obtained positive results within their programmes, these were included in the meta-analysis, as shown in the funnel plot, and the statistical tests carried out concluded that there was no publication bias in the present meta-analysis.

This meta-analysis focused on preschoolers’ phonological awareness interventions because this approach is most effective in pre-elementary education and can be extended to children who start elementary grades; however, the educational level was not specified as a moderating variable, since it was an inclusion criterion for the meta-analysis.

6. Future Directions

This meta-analysis has shown the high effect size for intervention groups that use technology in phonological awareness. Research challenges can be tackled with studies on student groups with language difficulties (dyslexia, dyslalia…) or bilingual participants. Studies can be carried out in social, familiar, and academic contexts to make the most possible use of technology, particularly when the students’ language deficits are present. Likewise, the moderating variable of the orthography of the language indicated the current difficulties in the acquisition of phonological awareness if the studies are run in opaque languages (such as English and Hebrew), more research can be carried out on bilingualism and its effects on phonological awareness.

Finally, an interesting gap was found in advanced technologies such as virtual reality and the use of robotics, which may be combined with phonological awareness interventions. This may be through the study of other moderating variables that focus on the type of technology and its implications in early childhood education. There is already evidence that points to children’s learning with robotics principles in educational contexts in many places around the world.