J. Intell., Volume 12, Issue 1 (January 2024) – 11 articles

Previous studies show that students’ goal orientation and conscientiousness are related to academic performance. Few studies, however, allow conclusions to be drawn about the factor structure of goal orientation and its distinctions from conscientiousness. In a study with N = 145 secondary school students (M = 13.9, SD = 0.85; 41% male), we investigated if the residuals of latent factors of goal orientation are still meaningfully correlated with academic performance and reasoning. Based on structural equation models, we have replicated the theoretically derived four-factor structure and showed that conscientiousness explains 29% of the variance in learning goals and 40% of the variance in work avoidance. Furthermore, we show that the residuals of goal orientation are mainly not significantly related to reasoning or educational standards (only work avoidance correlated with reasoning, and performance goals correlated with educational standards). Educational standards were highly correlated with reasoning. Implications for school practice and possible interventions are discussed. Full article

Generational IQ test scores in the general population were observed to increase over time (i.e., the Flynn effect) across most of the 1900s. However, according to more recent reports, Flynn effect patterns have seemingly become less consistent. So far, most available evidence on this phenomenon has been categorized by drawing on the classic fluid vs. crystallized intelligence taxonomy. However, recent evidence suggests that subdomain-specific trajectories of IQ change may well be more complex. Here, we present evidence for cross-temporal changes in measurement-invariant figural reasoning tasks in three large-scale, population-representative samples of German secondary school students (total N = 19,474). Analyses revealed a consistent pattern of significant and meaningful declines in performance from 2012 to 2022. Results indicate a decrease in figural reasoning of 4.68 to 5.17 IQ points per decade (corresponding to small-to-medium effects, Cohen ds from 0.34 to 0.38). These findings may be interpreted as tentative evidence for a decreasing strength of the positive manifold of intelligence as a potential cause of the increasing number of recent reports about inconsistent IQ change trajectories. Full article

Computer-based assessments provide the opportunity to collect a new source of behavioral data related to the problem-solving process, known as log file data. To understand the behavioral patterns that can be uncovered from these process data, many studies have employed clustering methods. In contrast to one-mode clustering algorithms, this study utilized biclustering methods, enabling simultaneous classification of test takers and features extracted from log files. By applying the biclustering algorithms to the “Ticket” task in the PISA 2012 CPS assessment, we evaluated the potential of biclustering algorithms in identifying and interpreting homogeneous biclusters from the process data. Compared with one-mode clustering algorithms, the biclustering methods could uncover clusters of individuals who are homogeneous on a subset of feature variables, holding promise for gaining fine-grained insights into students’ problem-solving behavior patterns. Empirical results revealed that specific subsets of features played a crucial role in identifying biclusters. Additionally, the study explored the utilization of biclustering on both the action sequence data and timing data, and the inclusion of time-based features enhanced the understanding of students’ action sequences and scores in the context of the analysis. Full article

Spatial thinking skills are associated with performance, persistence, and achievement in science, technology, engineering, and mathematics (STEM) school subjects. Because STEM knowledge and skills are integral to developing a well-trained workforce within and beyond STEM, spatial skills have become a major focus of cognitive, developmental, and educational research. However, these efforts are greatly hampered by the current lack of access to reliable, valid, and well-normed spatial tests. Although there are hundreds of spatial tests, they are often hard to access and use, and information about their psychometric properties is frequently lacking. Additional problems include (1) substantial disagreement about what different spatial tests measure—even two tests with similar names may measure very different constructs; (2) the inability to measure some STEM-relevant spatial skills by any existing tests; and (3) many tests only being available for specific age groups. The first part of this report delineates these problems, as documented in a series of structured and open-ended interviews and surveys with colleagues. The second part outlines a roadmap for addressing the problems. We present possibilities for developing shared testing systems that would allow researchers to test many participants through the internet. We discuss technological innovations, such as virtual reality, which could facilitate the testing of navigation and other spatial skills. Developing a bank of testing resources will empower researchers and educators to explore and support spatial thinking in their disciplines, as well as drive the development of a comprehensive and coherent theoretical understanding of spatial thinking. Full article

Measurement models traditionally make the assumption that item responses are independent from one another, conditional upon the common factor. They typically explore for violations of this assumption using various methods, but rarely do they account for the possibility that an item predicts the next. Extending the development of auto-regressive models in the context of personality and judgment tests, we propose to extend binary item response models—using, as an example, the 2-parameter logistic (2PL) model—to include auto-regressive sequential dependencies. We motivate such models and illustrate them in the context of a publicly available progressive matrices dataset. We find an auto-regressive lag-1 2PL model to outperform a traditional 2PL model in fit as well as to provide more conservative discrimination parameters and standard errors. We conclude that sequential effects are likely overlooked in the context of cognitive ability testing in general and progressive matrices tests in particular. We discuss extensions, notably models with multiple lag effects and variable lag effects. Full article

Measurement invariance of the Wechsler Intelligence Scale for Children, Fifth Edition (WISC-V) 10-primary subtest battery was analyzed across a group of children and adolescents with ADHD (n = 91) and a control group (n = 91) matched by sex, age, migration background, and parental education or type of school. First, confirmatory factor analyses (CFAs) were performed to establish the model fit for the WISC-V second-order five-factor model in each group. A sufficiently good fit of the model was found for the data in both groups. Subsequently, multigroup confirmatory factor analyses (MGCFAs) were conducted to test for measurement invariance across the ADHD and control group. Results of these analyses indicated configural and metric invariance but did not support full scalar invariance. However, after relaxing equality constraints on the Vocabulary (VC), Digit Span (DS), Coding (CD), Symbol Search (SS), and Picture Span (PS) subtest intercepts as well as on the intercepts of the first-order factors Working Memory (WM) and Processing Speed (PS), partial scalar invariance could be obtained. Furthermore, model-based reliability coefficients indicated that the WISC-V provides a more precise measurement of general intelligence (e.g., represented by the Full-Scale IQ, FSIQ) than it does for cognitive subdomains (e.g., represented by the WISC-V indexes). Group comparisons revealed that the ADHD group scored significantly lower than the control group on four primary subtests, thus achieving significantly lower scores on the corresponding primary indexes and the FSIQ. Given that measurement invariance across the ADHD and the control group could not be fully confirmed for the German WISC-V, clinical interpretations based on the WISC-V primary indexes are limited and should only be made with great caution regarding the cognitive profiles of children and adolescents with ADHD. Full article

The classroom environment significantly affects the development of creativity. This study examined the impact of the classroom environment on students’ creativity and the moderating role of thinking styles in this relationship. For this study, we recruited 451 students from six secondary schools. Data were collected using the Chinese Language Creativity Test, Classroom Environment Inventory, and Thinking Styles Inventory. Hierarchical regression analysis examined the moderating effect of thinking styles on the relationship between the classroom environment and creativity. The results showed that peer relationships in the classroom environment negatively influence students’ fluency and originality in creativity. At the same time, teachers’ evaluation and teaching methods positively affect the fluency of creativity. Thinking styles moderated the impact of the classroom environment on language creativity. This study identified four different moderating effects: the thinking styles matching the classroom environment can enhance language creativity, whereas the mismatched ones hinder it. However, matching would limit language creativity for individuals with creative thinking styles (e.g., legislative and anarchic thinking styles), while a mismatch can boost creative performance. The findings help educators understand students’ creativity with different thinking styles in various classroom environments and provide individualized and effective strategies for optimizing educational environments and enhancing language creativity. Full article

Judgments of learning (JOLs) reactivity refers to the finding that the mere solicitation of JOLs modifies subsequent memory performance. One theoretical explanation is the item-specific processing hypothesis, which posits that item-level JOLs redound to the benefit of later memory performance because they enhance item-specific processing. The current study was designed to test this account. We factorially manipulated the organization (blocked vs. randomized) of categorized lists and JOL condition (item-JOLs, list-JOLs, no-JOLs) between participants, and fit the dual-retrieval model to free recall data to pinpoint the underlying memory processes that were affected by JOL solicitation. Our results showed that item-level JOLs produced positive reactivity for randomized but not for blocked categorized lists. Moreover, we found that the positive JOL reactivity for randomized categorized lists was tied to a familiarity judgment process that is associated with gist processing, rather than to item-specific recollective processes. Thus, our results pose a challenge to the item-specific processing explanation of JOL reactivity. We argue that JOL reactivity is not restricted to item-specific processing; instead, whether JOLs predominantly engage participants with item-specific or relational processing depends on the interaction between learning stimuli and JOLs. Full article

A fundamental concept in psychological and intelligence testing involves the assumption of comparability in which performance on a test is compared to a normative standard derived from prior testing on individuals who are comparable to the examinee. When evaluating cognitive abilities, the primary variable used for establishing comparability and, in turn, validity is age, given that intellectual abilities develop largely as a function of general physical growth and neuromaturation. When an individual has been raised only in the language of the test, language development is effectively controlled by age. For example, when measuring vocabulary, a 12-year-old will be compared only to other 12-year-olds, all of whom have been learning the language of the test for approximately 12 years—hence, they remain comparable. The same cannot be said when measuring the same or other abilities in a 12-year-old who has been raised only in a different language or raised partly with a different language and partly with the language of the test. In such cases, a 12-year-old may have been learning the language of the test at some point shortly after birth, or they might have just begun learning the language a week ago. Their respective development in the language of the test thus varies considerably, and it can no longer be assumed that they are comparable in this respect to others simply because they are of the same age. Psychologists noted early on that language differences could affect test performance, but it was viewed mostly as an issue regarding basic comprehension. Early efforts were made to address this issue, which typically involved simplification of the instructions or reliance on mostly nonverbal methods of administration and measurement. Other procedures that followed included working around language via test modifications or alterations (e.g., use of an interpreter), testing in the dominant language, or use of tests translated into other languages. None of these approaches, however, have succeeded in establishing validity and fairness in the testing of multilinguals, primarily because they fail to recognize that language difference is not the same as language development, much like cultural difference is not the same as acquisition of acculturative knowledge. Current research demonstrates that the test performance of multilinguals is moderated primarily by the amount of exposure to and development in the language of the test. Moreover, language development, specifically receptive vocabulary, accounts for more variance in test performance than age or any other variable. There is further evidence that when the influence of differential language development is examined and controlled, historical attributions to race-based performance disappear. Advances in fairness in the testing of multilinguals rest on true peer comparisons that control for differences in language development within and among multilinguals. The BESA and the Ortiz PVAT are the only two examples where norms have been created that control for both age and degree of development in the language(s) of the test. Together, they provide a blueprint for future tests and test construction wherein the creation of true peer norms is possible and, when done correctly, exhibits significant influence in equalizing test performance across diverse groups, irrespective of racial/ethnic background or language development. Current research demonstrates convincingly that with deliberate and careful attention to differences that exist, not only between monolinguals and multilinguals of the same age but also among multilinguals themselves, tests can be developed to support claims of validity and fairness for use with individuals who were in fact not raised exclusively in the language or the culture of the test. Full article

Feeling with our conspecifics and understanding their sentiments and intentions is a crucial part of our lives. What is the basis for these forms of social understanding? If individuals ground their understanding of others’ thoughts and feelings in their own perceptual and factual experiences, it could present a challenge to empathize and mentalize with those whose reality of life is significantly different. This preregistered study compared two groups of participants who differed in a central perceptual feature, their visual abilities (visually impaired vs. unimpaired; total N = 56), concerning their social understanding of others who were themselves either visually impaired or unimpaired. Employing an adjusted version of the EmpaToM task, participants heard short, autobiographic narrations by visually impaired or unimpaired individuals, and we assessed their empathic responding and mentalizing performance. Our findings did not reveal heightened empathy and mentalizing proclivities when the narrator’s visual abilities aligned with those of the participant. However, in some circumstances, cognitive understanding of others’ narrations benefitted from familiarity with the situation. Overall, our findings suggest that social understanding does not mainly rely on perceptual familiarity with concrete situations but is likely grounded in sharing emotions and experiences on a more fundamental level. Full article

This study assesses cognitive abilities through video games for entertainment (Blek, Edge, and Unpossible) that were programmed from scratch to record players’ behavior and the levels achieved in a sample without emotional problems and in one with emotional problems. The non-emotional-problem sample was recruited from three universities and two bachelor’s degree programs. The emotional-problem sample was recruited from two outpatient centers. The participants in the emotional-problem sample completed reduced versions of the ability tests and video games, as required by their emotional problems. Three subtests of the Differential Aptitude Test that assessed abstract reasoning, visuospatial reasoning, and perceptual speed were selected as ability tests. All participants were required to complete a mental health questionnaire (PROMIS) and a brief questionnaire on their gaming habits and previous experience with the video games used. The results that were obtained showed good convergent validity of the video games as measures of cognitive abilities, and they showed that the behavior of players in the sample without emotional problems while playing predicted the level achieved in the Blek and Unpossible game fragments, but this was only true for Unpossible in the emotional-problem sample; finally, shorter versions of the Blek and Edge game fragments can be used because they maintain their good psychometric properties. Full article