The Pernicious Predictability of State-Mandated Tests of Academic Achievement in the United States

Abstract

The purpose of this study was to determine the predictiveness of community and family demographic variables related to the development of student academic background knowledge on the percentage of students who pass a state-mandated, commercially prepared, standardized Algebra 1 test in the state of New Jersey, USA. This explanatory, cross-sectional study utilized quantitative methods through hierarchical regression analysis. The results suggest that family demographic variables found in the United States Census data related to the development of student academic background knowledge predicted 75 percent of schools in which students achieved a passing score on a state standardized high school assessment of Algebra 1. We can conclude that construct-irrelevant variance, influenced in part by student background knowledge, can be used to predict standardized test results. The results call into question the use of standardized tests as tools for policy makers and educational leaders to accurately judge student learning or school quality.

1. Introduction

It has long been known that the accurate interpretation and use of standardized test results to make decisions about student academic achievement in public elementary and secondary schools in the United States can be affected by psychometric design elements that result in irrelevant fluctuations in scores among different groups of test takers [1]. Factors that can lead to misinterpretation of, or patterns in, test results include mismatches between the selected content of the test and the subject area construct, unclear instructions, complexities in test items unrelated to the construct being measured, inappropriate testing conditions, mismatches between the test takers’ native language and the language used on the test, cognitive or physical disabilities of the test takers, adverse childhood experiences such as trauma, and biased or incorrect expectations or assumptions about the test takers’ anticipated responses [2]. Students’ existing background knowledge of topics included on standardized tests is another factor that produces variance and can predict standardized test results [3]. Ecological systems theory [4,5] and the construction-integration model [6] provide lenses from which to view where students gain background knowledge outside of school and how background knowledge influences standardized test performance.

Like policy makers in over 40 other states in America, New Jersey education bureaucrats use the results from state-mandated high school mathematics tests to make determinations about various aspects of education in the state including: (a) the effectiveness of the high school principal, (b) the effectiveness of the high school assistant principal(s), (c) the effectiveness of the high school mathematics teachers, (d) whether a student is prepared for college, (e) whether a student is prepared for a career, (f) a student’s strengths and weaknesses in mathematics as explained on the standardized test student report, (g) whether the student can graduate high school, (h) whether a school is “failing” in the area of mathematics instruction, and (i) whether an entire school district is “failing” in the area of mathematics [7]. The “failing” status of a school district can carry with it the consequence of the school being monitored by the state.

Some school leaders in New Jersey and other states voluntarily choose to use standardized test results to (a) make judgments about the quality of the school district’s mathematics curricula and overall program, (b) decide future course placements for students, and (c) determine whether students must attend mandatory remedial mathematics courses. The results from the state-mandated standardized tests used in New Jersey have not been independently validated for all of the ways the results are used, yet some education leaders rely on them for decision-making purposes.

Existing studies about the effect of student background knowledge on the results of standardized tests of reading are well documented [8]. Less is known about the influence of student background knowledge on state-mandated standardized tests of mathematics, particularly at the high school level. The state of New Jersey provides fertile ground for such an inquiry because it has a racially and ethnically diverse student population and vast diversity in the socio-economic background of its communities and students. In some towns in New Jersey, up to 95 percent of the student population are considered economically disadvantaged and qualify for free or reduced-price school lunch due to their families’ level of poverty, whereas in other communities, less than 5 percent of the students qualify for free or reduced-price lunch because of their high family wealth. The income inequality evident among New Jersey’s almost 600 communities is one of largest in the United States [9].

Our purpose for this study was to explain the predictiveness of family and community demographic variables that influence the development of student background knowledge on the percentage of students who received a passing score, at the school level, on a New Jersey state-mandated standardized test of Algebra in their first year of high school. We believe the results will help to inform educational leaders and policy makers of the potential issues associated with using results from commercially prepared standardized assessments as the deciding factor about student learning and the quality of education provided at the school and district levels.

2. Construct-Irrelevant Variance

In K-12 public education in the United States, the accurate interpretation of standardized test results can be hindered by various factors that result in irrelevant variations in scores or predictable patterns of scores for groups of test takers. The irrelevant variations can lead to inaccurate interpretations of academic achievement by policy makers, particularly for students from poverty and those with limited life experiences. Construct-irrelevant variance is a particular concern for public school students whose background knowledge and life experiences do not align with the assumed background knowledge and life experiences represented by the items of the test and the contexts and formats in which those items are presented [10].

One potential source of construct-irrelevant variance that affects test scores arises from inappropriate test content. Inappropriate test content is content that confounds the measurement of the target construct and may disproportionately favor certain subgroups of students [11]. For example, a critical reading test should avoid content that can be influenced by out-of-school factors, such as words, phrases, or topics closely tied to specific occupations, disciplines, cultural backgrounds, socioeconomic statuses, racial/ethnic groups, or geographical locations. The focus of any critical reading test should be to measure the construct itself, the ability to read critically, and minimize any potential bias caused by prior knowledge and experiences that could advantage or disadvantage students from specific subgroups.

Construct-irrelevant variance can be a symptom of a lack of opportunity for students to learn the content included on the test. It can also arise by way of a mismatch between the life experiences assumed by the test developer and the actual experiences of the test taker. In short, construct-irrelevant variance impacts the fair and precise interpretation of test scores for their intended purposes. Moreover, a failure to account for an individual’s existing background knowledge of the tested domain, known as domain knowledge [12], or the opportunity to learn the domain knowledge of the test, can lead to misdiagnoses of academic achievement, inappropriate academic placement, and/or inappropriate assignment of services for students, which could have profound current and future consequences for the academic achievement and life pathways of individual students.

3. Construction-Integration Model and Student Background Knowledge

The construction-integration model identifies a critical role that factors such as student background knowledge in a specific subject area play in achievement on standardized tests [6,12]. Knowledge can be classified according to its specificity; background knowledge comprises the world knowledge that a student brings to a specific task. This can include episodic (events), declarative (facts), and procedural (how-to) knowledge, as well as related vocabulary [6]. A subset of background knowledge, domain knowledge, refers to knowledge of a specific and defined field [13].

The construction-integration model proposed by Knitch [6] serves as a critical framework for understanding the influence of background knowledge in reading comprehension within an academic context. The model emphasizes that reading comprehension extends beyond decoding words and understanding individual meanings, and instead involves the active construction of mental representations of text content. Key to this process is the integration of new information with the student’s existing background knowledge to construct meaning [14,15].

The results from a large-scale study of high school students demonstrated the strong influence of background knowledge on student achievement on standardized tests of academic achievement [16]. The study sample involved 3534 high school students, and it confirmed a robust positive association and advantage between students’ background knowledge of specific topics and their overall academic performance [16].

The high school study corroborated and extended the work of previous studies on the influence of student background knowledge on academic achievement, like the well-known Baseball Experiment [17]. The experiment involved 64 middle school students who were instructed to read a fictional story depicting a segment of a baseball game. Students were assessed on various comprehension tasks after they completed the reading. First, they were required to reenact and articulate their understanding of the story using a miniature baseball field model. Then they engaged in traditional reading comprehension and writing tasks based on question prompts. They had to provide a summary of the events they read, make predictions about future events in the game and justify their reasoning, as well as categorize groups of sentences from the passage based on their significance for comprehending the story. The findings from the experiment revealed that regardless of their reading ability or level, students with more background knowledge about baseball achieved higher scores on the assessments [17].

4. Student Background Knowledge and Mathematics Achievement

Because federal legislation requires all 50 United States to administer standardized tests of reading and mathematics, the implications of the construction-integration model loom large on the test results. But the influence of the construction-integration model is not only seen on standardized reading assessments. Correlations between results on the reading and mathematics sections of large-scale state standardized tests in the United States are strong, ranging from 0.700–0.900 for most state tests as well as for the international Programme for International Assessment (PISA) suite of tests [18,19,20]. In many cases, standardized state tests of mathematics are nothing more than reading tests that include numbers within the text.

Students utilize their background knowledge to establish connections, infer meanings, and aid their overall comprehension of the text. Students with rich and relevant background knowledge are better equipped to comprehend and retain information from texts as they can effectively integrate new information with their preexisting knowledge [12]. Students with expanded background knowledge across topics and domains generally perform better on state standardized tests of reading and mathematics [16,17,18].

One problem with text-heavy standardized tests of mathematics can be found in the text-based contexts and situations included in the questions. In many cases, standardized tests of mathematics include question contexts or situations that are not universally common to the test takers, especially students from poverty, students whose first language is not English, and students with limited time in the United States. State tests of academic achievement and other standardized assessments have included questions that use contexts or situations related to visiting a state park, visiting a farm, ecology and environmental topics, space exploration, travel and vacations, yachting, contemporary suburban life, roller coasters, ski gondolas, a day at the beach, and other contexts and topics that require students to have varied life experiences and background knowledge in order to successfully navigate and understand the passages to answer the questions [2].

5. Student Background Knowledge and Poverty

Results from studies of the influence of poverty on student learning suggest that students from impoverished backgrounds often encounter barriers that limit their access to life experiences that build background knowledge often found on standardized tests, such as travel, leisure activities, and extracurricular pursuits, when compared to their more affluent peers [21]. Varied life experiences are instrumental in the development of academic background knowledge and vocabulary, and they complement and extend explicit domain knowledge learned through formal schooling.

Furthermore, out-of-school factors like family income and the educational attainment of a child’s mother have been identified as key factors influencing background knowledge and vocabulary acquisition, subsequently impacting performance on standardized assessments [22,23]. Children whose mothers had a college degree were exposed to an estimated 3000 additional words per day, resulting in a significant four-million-word disparity between the highest and lowest socioeconomic status (SES) groups by the age of four [22] (pp. 260–261). Enhanced sight and vocabulary at younger ages and general reading vocabulary skills are closely associated with the development of strong reading comprehension abilities.

According to a report from the Federal Reserve Bank of New York [24], households with at least one parent who has received a college education exhibit an average annual income that is approximately 70% higher than households without a college-educated parent. The higher income enables families to afford a wider range of life experiences that build academically related background knowledge.

6. Materials and Methods

This non-experimental, correlational, explanatory cross-sectional study [25] was guided by two questions:

(1) What is the relationship of the 2016 New Jersey test scores in Algebra 1 at the school level and family and community variables that influence student background knowledge?

Ha1. 

There is no statistically predictive relationship between community characteristics or socioeconomic variables and the 2016 New Jersey test scores in Algebra 1.

(2) How accurately can family and community variables related to the development of student background knowledge predict a student’s meeting expectations or exceeding expectations on the 2016 Algebra 1 test at the school level?

We chose the 2016 test results because the U.S. Census was updated in 2015, and the 2016 version of the New Jersey Algebra 1 exam had remained stable for two years. Later versions of the test underwent content and format changes. Furthermore, some of 2020 U.S. Census questions were changed, and the reliability of the New Jersey Algebra 1 test results became unstable after the onset of the COVID-19 pandemic, resulting in emergency remote teaching and ongoing school closures.

Hierarchical regression modeling was used to determine the predictive accuracy of out-of-school factors related to family and community capital that can influence student acquisition of background knowledge and impact standardized test results in Algebra 1 at the high school level.

6.1. Variables

The percentages of students who scored proficient or above at the school level for the state-mandated New Jersey test of Algebra 1 constituted the dependent variable. We located (12) independent variables in the U.S. Census American FactFinder database that were consistently found in the extant literature as being related to the development of family human capital and community social capital that contribute to students gaining background knowledge (See

Table 1. Names and labels of independent variables.

Variable	Label
Percentage of Population Employed	Employ Status
Percentage of Households Under $25,000	% House under 25 K
Percentage of Households Under $35,000	% House under 35 K
Percentage of Households Over $200,000	% House over 200 K
Percentage of Families Under $25,000	% Family under 25 K
Percentage of Families Under $35,000	% Family under 35 K
Percentage of Families Over $200,000	% Family under 200 K
Percentage of Population with less than 9th grade Education	Less than 9th
Percentage of Population with No High School	No High School Diploma
Percentage of Population with Some College	Some College
Percentage of Population with Bachelor’s Degree	BA Degree
Percentage of Population with Advanced Degree	Advanced Degree

). We identified seven (7) community social capital variables and five (5) family human capital variables that demonstrated relationships to student background knowledge in the extant literature.

Community Social Capital Variables.

Household income, which is defined as:

• Employment status
• Percentage of annual household income under $25,000
• Percentage of annual household income under $35,000
• Percentage of annual household income above $200,000
• Percentage of family income under $25,000
• Percentage of family income under $35,000
• Percentage of family income above $200,000

Family Human Capital Variables.

Parent level of education, which is defined as:

• Parents with less than a 9th grade education
• Percentage with no high school diploma
• Percentage that are high school graduates with some college education
• Percentage with a bachelor’s degree
• Percentage with an advanced degree

6.2. Sample

The final sample included 159 public high schools out of 390 total high schools. The state of New Jersey consists of approximately 590 operating public school districts, but not every school district has a high school. In order to ensure consistent matches between family and community demographic data at the local level and the characteristics of the students that attended the schools, the schools in the sample met the following criteria:

(1)

Schools that only served the traditional American high school grades of 9, 10, 11, and 12.

(2)

Schools that serviced students within their hometown district only. Regional schools or other schools that accepted students from outside of the town in which the high school was located were excluded.

(3)

Schools that were the only high school in the school district.

(4)

Schools that had at least 25 valid student scores for the administration of the Algebra 1 test.

(5)

Schools whose town had complete U.S. Census data for the 12 home and community variables included in this study.

We conducted a priori sample size calculations to ensure the sample size was large enough to accommodate working with up to 12 variables in our models. However, none of our final predictive models included all 12 variables. Field [26] recommended the formula 50 + 8(k) for simultaneous multiple regression, with k equaling the total number of predictor variables in the model, to determine an appropriate sample size to detect an effect size of at least 0.50 at the 95% confidence level and a p value of at least 0.05. This study initially included 12 potential predictor variables to represent k. Using Field’s formula, we calculated 50 + 8(12) = 146. Therefore, in order to reach an appropriate effect size and p value equal to or less than 0.05, we needed to include at least 146 schools. Our sample exceeded the required minimum number.

Then we calculated the required sample sizes for hierarchical multiple regression. Green [27], as cited in Field [26], recommended 104 + k, where k represents the number of predictor variables to be entered into the model for hierarchical multiple regression. Our sample size of 159 exceeded the minimum requirement of 116 cases to conduct hierarchical linear regression.

6.3. Data Collection

We obtained the data for the dependent variables from the New Jersey Department of Education’s website. The percentages of students rated as meeting expectations and exceeding expectations were combined into one total percentage. Next, we matched the percentages to corresponding town demographic data from the U.S. Census for the communities served by each school that met our sampling requirements. We retrieved data for the family and community level variables from the American Community Survey section of the U.S. Census and localized the data with American Factfinder section from the Census. Finally, we matched town-level demographic data from the U.S. Census in the American Factfinder to school assessment data for each local high school.

6.4. Data Analysis

We used two forms of regression as the primary methods to analyze results: simultaneous multiple regression (SMR) to narrow down variables and check for multicollinearity, and then hierarchical linear regression (HLR) to identify the most efficient statistically significant predictor variables and models of best fit for our predictive algorithms. Considering the goal was to predict the aggregate performance on the Algebra 1 state-mandated standardized test of mathematics at the school level, hierarchical regression analysis was an appropriate strategy. Predictions require the identification of models of best fit and hierarchical regression is an accepted method to determine such models [28].

Prior to running the regressions, we inspected the skewness of the dependent variable to determine whether the data were normally distributed within the 1.00 to −1.00 ranges. The dependent variable met the assumption of normality.

We created correlation matrices and scatterplots to help develop more refined SMR and HLR models. We reviewed the correlation matrices for relationships between independent and dependent variables and among dependent variables to anticipate possible multicollinearity. We loaded the independent variables into an initial simultaneous multiple regression model. Then, we began to remove variables from the model that were statistically insignificant above 0.10 or that exhibited initially high levels of multicollinearity above 7.000, then above 4.000, and finally above 3.000.

The process of removing variables was important because in order to obtain accurate measures of R-squared, one must correctly identify predictor variables that correlate strongly with the dependent variable [29]. The extant literature guided in the process of variable removal. We attempted to isolate predictor variables so that the variance in the criterion was accounted for only once to ensure that the predictor variables accounted for different proportions of the variance in the criterion variable. Therefore, we sought to refine each model so the predictor variables exhibited low correlations among themselves with variance inflation factors (VIF) below accepted levels of 3.000 [29]. The process of factor elimination and substitution continued until we arrived at the fewest number of predictor variables that maximized R-squared.

We placed the statistically significant (p ≤ 0.05) predictor variables in highest to lowest rank order based upon beta values to run the hierarchical regression models. The hierarchical regression models allowed us to identify how much influence each specific variable had on the dependent variable and to identify the model of best fit. The formal representation of our final regression equation for each model of best fit was y1 = b0 + (b1Xi) + (b2Xii) + (b3Xiii) + e with b representing the unstandardized beta for the predictor variable, X representing the percentage of the variable in the community, and e representing the constant for each model [28].

7. Results

The mean percentage of students, at the school level, meeting or exceeding expectations on the 2016 New Jersey high school Algebra 1 exam ranged from a low of 5.40 to a high of 79.70. The highest possible percentage is 100. The test had a possible scale score range of 500 to 850. Standard deviations of mean scores at the district level ranged from 11.42 to 16.40.

We sought to identify a model that accounted for the largest amount of variance while having the lowest amount of error. The hierarchical regression calculations resulted in a model of best fit. The model of best fit had a standard error of the estimate of 11.47. The test itself has a reported error range for the reported scores of 6–12 points, and so we believe that some of the error in our model of best fit can be attributed to the reported error in the test scores themselves. The model of best fit accurately predicted the aggregate percentage of students who met or exceeded expectations (passed the exam) for 119 of the 159 schools, or 75 percent of the sample, within the standard error of the estimate of 11.47 points. The R-squared value of the model of best fit was 0.627, indicating that almost 63% of the variance in the scores was accounted for by the variables in the model (See

Table 2. Hierarchical linear regression model of best fit for Algebra 1 test results.

Model	R	R Square	Adjusted R Square	Std. Error of the Estimate
1	0.792 a	0.627	0.622	11.47301

^a. Predictors: (Constant), % Family under 200 K, % Family under 35 K.

).

The model of best fit included two variables: (a) percentage of families in a community with income over $200,000 a year and (b) percentage of families in a community with income under the United States poverty level for a family of four ($35,000).

The U.S. Census defines a family as two or more people related by birth, marriage, or adoption living in the same house. In this case, the variable percentage of families in a community with income over $200,000 a year provides an indication of the income wealth of families in a particular community. We surmise one reason that this variable was statistically significant while a similar variable, percentage of households in a community with income over $200,000 a year, was not is because the family aspect connects to the students who attend the school in that community, whereas the definition of households does not include children. The variable percentage of families in a community with income over $200,000 a year relates directly to the family capital of students.

Our second significant variable in the model of best fit, percentage of families in a community with income less than $35,000, relates to family human capital and, specifically, represents the percentage of families in the community living near or below the poverty line, which in the United States is an income of approximately $30,000 per year for a family of four.

The standardized beta for the percentage of families with household income under $35,000 a year was −0.280 at p = 0.000. The standardized beta for the percentage of families with household income over $200,000 a year was 0.599 at p = 0.000 (See

Table 3. Betas and statical significance of variables in the model of best fit.

Model 1	Unstandardized Beta	Standard Error	Standardized Beta	T Score	Significance	Collinearity VIF
Constant	36.757	2.667		13.78	0.000
% Fam < 35 K	−0.520	0.110	−0.280	−4.727	0.000	1.469
% Fam > 200 K	0.964	0.095	0.599	10.103	0.000	1.469

).

The model of best fit accounted for almost two-thirds of the variance in the Algebra 1 test results for students in their first year of high school. Overall, the results suggest that U.S. Census data related to family and community capital factors that contribute to the development of student background knowledge can predict three-quarters of the schools in which students will score at or above the expected scale score to pass the exam.

8. Discussion

The results align with previous studies cited throughout the paper on the subject. They add another brick to the wall of research that demonstrates family and community factors have historically been shown to influence standardized test results. Our results extend the extant research by focusing on the ways that student background knowledge influence results on standardized tests of mathematics, an understudied area of research.

Bronfenbrenner’s ecological systems theory [4] established the foundation for understanding how the various layers of a child’s environment shape that child’s development. He postulated that children are embedded within an intricate ecological system consisting of multiple interconnected layers that include family, school, peer groups, and community. The layers exert both direct and indirect influences on children’s behaviors, academic achievement, and later life outcomes. Moreover, it is essential to recognize that the layers do not operate in isolation; instead, they interact with one another, creating a dynamic and complex web of influences and background knowledge gained through life experiences [5].

Ecological systems theory aligns with contemporary research perspectives on the profound impacts of poverty on learning and future life outcomes. The extant literature emphasizes the importance of affording children access to formal and informal educational experiences and varied life experiences that build background knowledge and sight vocabulary. Additionally, the theory underscores the need for establishing robust social systems that support children’s effective utilization of available resources within formal learning environments. The effective use of available resources for education, referred to as the capabilities perspective, posits that students possess varying capacities to harness and transform educational resources into tangible learning gains [30]. The ecological system in which a student is nurtured impacts his or her ability to fully actualize the educational resources provided [5,30].

Incorporating the capabilities perspective as another layer within the ecological system is imperative when interpreting standardized test results. It adds a crucial dimension to understanding a child’s abilities and academic performance and how a dearth of background knowledge can act as an inhibiting factor to students’ capabilities to fully make use of education resources. By considering the capabilities perspective within the broader ecological framework, we gain a more accurate and comprehensive understanding of how various factors, both internal and external to the student, interact to produce background knowledge and influence a child’s learning outcomes.

Bronfenbrenner’s [4,5] ecological systems theory underscores the multifaceted nature of a child’s developmental environment, where family, school, peer groups, community, and larger public policy initiatives are connected and influential in students having varied life experiences, acquiring background knowledge, and making full use of that knowledge within formal school settings. Ecological systems theory provides insights as to how background knowledge develops and how the construction-integration model functions.

Family human capital and community social capital are key aspects of the ecological systems theory and they are contributing factors to the development of student background knowledge [5,10]. Family human capital, such as income levels and family structure, as well as the community social capital characteristics in which a student lives, such as the percentage of community members who are unemployed or have a high school diploma, have been used in previous studies to predict academic achievement on standardized tests, e.g., [31,32,33]. Family capital factors that influence the development of background knowledge and predict levels of achievement include families headed by two parents and family income levels [34,35,36].

The term “human capital” is used by researchers to encompass people’s skills, experiences, and abilities that contribute to their potential economic success and skill development [37]. In our study, we focus on the student’s family as the closest level of human capital, directly influencing the development of student background knowledge. It has been found that students with more family human capital tend to have access to more academically oriented life experiences, such as travel and co-curricular activities outside of school, collateral learning experiences that extend beyond school, and additional support to connect with and apply academic content in their daily lives [30]. Consequently, these students often perform better on standardized tests measuring traditional academic achievement [7]. Family demographic factors such as income, household structure, and the percentage of female-headed households in poverty provide insights into the level of human capital within a family and can serve as proxies for the student’s experienced human capital [5], and that is how we used them in this study.

Moreover, students with higher family human capital also tend to perform better on standardized tests that are text based or require reading comprehension because students with more life experiences tend to have larger sight and academically oriented vocabularies [38]. Family demographic factors such as median income, household structure, and the percentage of female-headed families living in poverty can be used as proxies to describe the level of human capital within a family and the human capital experienced by the student [3,37,38].

The community in which a student resides also plays a role in the development of background knowledge and influences academic achievement as measured by standardized tests through community social capital. Coleman [36] (p. 98) defined social capital as a variety of different entities that consist of social structures that facilitate certain actions of individuals within the structure. Social capital, like other forms of capital, is productive and enables the achievement of ends that would not be possible otherwise. Formal and informal interactions, familial networks, and relationships within a community contribute to the creation of social capital at the community level [5,36]. Factors such as the presence of professionals, community groups for adults, structured recreation programs for children, religious organizations, libraries, services for senior citizens and disabled residents, arts commissions, local social advocacy groups, quality and affordable daycare and preschool opportunities, and other similar resources intersect to enhance the overall social capital of a community [37,38].

Alternatives

Districts can use teacher-designed, curriculum-based assessments to help reduce some of the construct-irrelevant variance inherent in standardized tests. Some examples include assessing student reading levels and comprehension through techniques like running records, readers workshop formats, and standards-based writing prompts. These methods require the teacher to interact with the student. The interactions provide the teacher with more information about student understanding and misconceptions than do the results from standardized tests.

Because the teacher assessments are curriculum-based, there is a higher probability that the test questions reflect the content that was taught during classroom instruction and the context of that teaching, including the scenarios and examples provided by the teacher, thus increasing the probability that the test results provide better information about student learning and understanding of the content tested. Teachers tend to customize their instruction to the students seated in front of them. The instruction will include examples and contexts that are meaningful and understood by most students. They will attempt to connect the new content to prior student experiences. Teachers will often scaffold instruction from less complex to more complex.

Overall, the contexts and scenarios included in teacher-made curriculum-based tests will more likely reflect what was taught in class and how it was taught, including the level of difficulty, format of questions, and levels of complexity. The strategies noted above lead to better understanding on the part of students and better demonstration of their content knowledge on subsequent assessments.

Some detractors of the use of teacher assessments claim factors like low reliability and grade inflation make them inappropriate indicators of student achievement. We disagree for the reasons stated above and point to ongoing research in the United States that demonstrates high school students’ grade point average is a stronger indicator of student success in college than the two large-scale standardized tests currently used in college admissions decisions, the ACT and SAT [39,40]. For the most part, grade point average is a function of student achievement on teacher made assessments and has consistently been a better indicator of student knowledge than large-scale standardize tests.

Quality models of assessment systems that use curriculum-based assessments already exist in the United States. One of the most mature systems, having operated for more than two decades, is the New York Performance Standards Consortium. The consortium is a group of thirty-eight public schools in New York City that serve over 12,000 students. The Consortium developed a suite of problem-based assessments in areas such as higher-order thinking, writing, mathematical problem solving, technology use, science research, appreciation and performance in the arts, service learning, and career skills. The schools contract external experts from universities and the community along with the teachers to review student work and provide feedback to students [20].

9. Conclusions

Bronfenbrenner’s [4,5] ecological systems theory helps explain the various factors that work to influence student achievement on standardized tests of academic achievement. Ecological systems theory together with the construction-integration model helps to explain how student background knowledge plays a role in construct-irrelevant variance. It also helps to explain why construct-irrelevant variance can decrease the validity of standardized test results. Similar to other standardized tests, the New Jersey Algebra 1 standardized test appears to be a blunt measure of mathematics achievement. The final results of this study were influenced more by construct-irrelevant variance caused by student background knowledge than in-school factors such as the quality of instruction.

Our results suggest that the Algebra 1 test was not a precise measurement of student mathematics achievement. The student results on the test explained more about the background knowledge the students possessed about the topics on the test than what they learned in mathematics class. Our results, like the results from numerous other studies on the topic, suggest that increased family capital provides opportunities for students to gain academic background knowledge from collateral learning opportunities that result from varied life experiences.

Because many commercially prepared standardized tests of mathematics require large amounts of reading, student background knowledge casts a large shadow over the results because of its influence on reading comprehension skills derived in part from human and social capital. Background knowledge influences students’ ability to comprehend test questions and use their existing knowledge to successfully answer questions or generate answers.

The results, in the context of Ecological Systems Theory, led us to suggest that the school-level outcomes from the New Jersey Algebra 1 test are reflections from the societal mirror of the state of New Jersey. They present a stronger picture of the family social capital of the students in a given society than the quality of the education provided by educational personnel in schools. Almost 63% of the variance in test performance was explained by social capital family income variables that influence the development of background knowledge. Background knowledge is a known predictor of standardized test results. Family income variables are immutable by schools. Only public policies, outside the control of school personnel, can influence family income.

The United States has one of the highest levels of childhood poverty among Organisation of Economic Co-operation and Development (OECD) countries. It is well known that the social safety net in the United States is not as strong as some nations in Europe and other parts of the world [20]. Neoliberal policies have greatly reduced government support for families in the United States. Important social policy frameworks that reduce poverty, such as monetary, labor, fiscal, and health policies, have been weakened over the last 40 years, causing increases in childhood poverty in the United States compared to other democratic countries [41].

Although some education policy makers in the United States claim that standardized test results are an important component of a comprehensive system of educational quality control, the results from decades of research on the topic suggest otherwise [42]. The influence of family social capital variables manifests itself in standardized test results. Policy makers and education leaders should rethink the current reliance on standardized test results as the deciding factor to make decisions about student achievement, teacher quality, school effectiveness, and school leader quality. In effect, policies that use standardized test results to evaluate, reward, and sanction students and school personnel are doing nothing more than rewarding schools that serve advantaged students and punishing schools that serve disadvantaged students.

Additional research is necessary to further elucidate relationships among student background knowledge, human and social capital, and standardized test results. Our study was aimed at practitioners and policy makers, and we reflected that with our choice of methods. Researchers also need more information about these relationships. To that end, we recommend future analyses that use structural equation modeling and multilevel modeling techniques to provide another layer of examination. The relationship among student background knowledge, human and social capital, and standardized test results is multifaceted and requires examination from multiple vantage points.