Where can we find hope for our schools?

Bringing It Back Homea report issued by the Economic Policy Institute at the end of October, provides a distinct service in reminding Americans that they can learn more about how to improve their schools by looking at successful American states than they can by heading overseas to pry lessons out of foreigners.

The authors, Stanford’s Martin Carnoy, EPI’s Emma Garcia, and Tatiana Khavenson at the National Research University Higher School of Economics in Moscow, have produced an impressive piece of scholarship. Their work makes a genuine new contribution to the discussion about how to improve American schools.

In considering this study, several points need to be born in mind.

First, the United States has very real problems in its schools. We cannot be Pollyannas about where we are. Average student performance is not where we would like it to be, and the average conceals terrible gaps between students doing well and those bringing up in the rear.

Historically, we have done a reasonably good job with the traditional students our schools were designed to serve. But now we face a new challenge: a student population in which the majority of students are, for the first time in our history, both low-income and children of color.

This is an unprecedented educational challenge. It is hard to think of another advanced post-industrial nation that faces the same conundrum to the same degree. We need to get on with that work, a task that is not helped by hyperventilating about questionable international comparisons of student achievement.

The second issue is discussed fully in Bringing It Back Home. The subtitle lays the foundation: Why state comparisons are more useful than international comparisons for improving U.S. education policy. American education is not a single system. It is 51 different systems, made up of the 50 states and the District of Columbia.

People who work closely with states and localities consider this to be so obvious that it barely needs saying, but Bringing It Back Home is one of the first national reports to rest on that self-evident reality.

In that light, there is an argument to be made that the United States should follow Canada’s approach on the Trends in International Mathematics and Science Survey (TIMSS). Canada as a whole does not report TIMSS scores. Results are reported for individual provinces.

Nearly a dozen states qua states have participated in either TIMSS or Program on International Student Assessment (PISA) over the years. There is precedent in the United States for moving in the state-based direction.

With the billions of dollars invested in new assessments by the U.S. Department of Education and the various states, it should be possible to find the funds to move U.S. participation in international assessments in the direction of state-based participation and reporting of results.

It is encouraging also that Carnoy, Garcia, and Khavenson focus on a limited number of nations in examining international assessments. In the main, they are interested in comparing the United States with similar post-industrial countries or with countries with very high achievement as reported on TIMSS or PISA.

Earlier this year, the National Superintendents Roundtable along with the Horace Mann League released School Performance in Context, aka The Iceberg Effect. This report adopted a similar approach. It examined the United States and eight other nations that in combination with the United States account for more than 50 percent of global GDP.

There is good reason to focus more sharply on a more limited number of nations. What do Americans, with nearly 56 million students, expect to learn from a tiny, wealthy principality like Lichtenstein, with 5,600 students? Or from religious monarchies such as Qatar, in which monarchs hire and fire the legislature, the cabinet, and members of the judiciary? Or from dictatorships such as those in Kazakhstan or Shanghai, with their top-down approaches? How practical is it to think that American citizens who seem unwilling to accept a curriculum based on the Common Core will sit still for five-year plans of the sort announced regularly by the Central Committee of the Communist Party of China?

Therefore, there is very good reason to focus on nations that are more similar to us politically and economically. And it is encouraging that more and more American educators are beginning to insist on comparing apples with apples.

The authors are also to be applauded for re-introducing students’ socio-economic status into the discussion about American schools. Their construction of a Family Academic Resources (FAR) index creates a proxy for poverty and disadvantagement. Just as The Iceberg Effect argued, students’ academic performance cannot be properly understood outside the context of students’ life experiences.

Early in October, the National Superintendents Roundtable heard from an elementary school principal in Las Vegas whose school enrollment was almost entirely made up of homeless children. Fully 85 percent of the students in her school were homeless, most living in homeless motels, some with relatives, and a few in cars with their families.

In this regard, the EPI report includes a very significant new finding. The authors say, “The effect of poverty on education performance is a three-level effect: In addition to the well-documented impact of individual and school-level poverty, state-level poverty puts students at all socioeconomic levels at additional educational disadvantage.” The tripartite nature of the assault poverty makes on children has not been identified before.

Surely, factors such as these should be taken into account in understanding and responding to school outcomes, not dismissed with focus-group-tested bromides that “poverty is just an excuse”?

Finally, it is to be hoped that national and state policymakers will pay special attention to a very clever analysis within Bringing it Back Home. This is an unusual pairing of National Assessment of Educational Progress (NAEP) results in similar or neighboring states. This paper makes for very complicated reading. It is a complex analysis involving different tests, different years, different subjects, and different nations, frequently crosscut by complex tables involving disadvantagement and socio-economic status as revealed by the FAR analysis.

It is reassuring to see that some American states are competitive with the highest-achieving nations in some subjects and grades, particularly when socio-economic status is taken into account. We do not hear that often enough.

Yet there is a brilliant simplicity to the final portion of the paper, which examines NAEP data over the years and pairs both low- and high-achieving American states with their neighbors, to see who makes the most progress over time.

These pairings—of Massachusetts with Connecticut, of New York with New Jersey, of California with Texas, of Minnesota with Iowa, and of North Carolina with Kentucky and Tennessee—are enlightening in the extreme. Many of these states started out at the same point—sometimes low achieving, sometimes high achieving—but one of them took off and the other did not.

Figure A

Connecticut and Massachusetts NAEP 8th grade mathematics scores, 1992–2013

Connecticut adjusted Massachusetts adjusted  Connecticut observed Massachusetts observed
1992 265.72 262.07 273.739345 272.783201
1996 271.76 270.46 279.591245 277.565642
2000 274.26 274.74 280.785142 278.949013
2003 280.52 279.57 283.729811 286.520694
2005 278.42 286.16 281.068953 291.513541
2007 277.74 291.23 282.47279 297.923344
2009 282.70 295.20 288.608513 298.854347
2011 282.83 295.57 286.999945 298.512431
2013 281.45 298.37 285.00000 301.00000
ChartData Download data

The data below can be saved or copied directly into Excel.

Note: Scores are adjusted within each test year for student characteristics, student family socioeconomic (SES) characteristics, and racial and SES composition of school. Test scores are adjusted across years for changing social class composition of sample by using 1996 student characteristics, family academic resources, and school racial and SES composition to weight all years.

Source: EPI analysis of NCES NAEP microdata

Copy the code below to embed this chart on your website.

This is where we can find hope. It is in these paired comparisons that we can begin to explore why some states (whether they started out low or high achieving) made progress while others languished. The explanation is likely to be found in state policy, not in questionable interpretations of what is going on abroad.

It is worth quoting directly from Bringing it Back Home: “The lessons embedded in how these states increased student achievement in the past two decades are much more relevant to improving student outcomes in other U.S. states than looking to high-scoring countries with social, political, and educational histories that differ markedly from the U.S. experience.”

That is extremely well said. It sums up the report in a nutshell.