Smell Testing Arizona's Terra Nova

The Goldwater Institute ^ | October 16, 2007 | Dr. Matthew Ladner

[GoldwaterInstitute]

In June, the Goldwater Institute released A Test of Credibility. The essence of the argument: Arizona's Terra Nova exam produces unrealistically high scores (above the national average in every subject and grade level), when national tests show Arizona consistently below the national average. Both results can not be true.

ADE recently issued a response filled with misconceptions. The Goldwater Institute will soon post a response to the ADE's critique on its website.

If you'd like to get to the bottom of this issue in a hurry, however, simply look at Figure 1 below. Figure 1 compares Arizona's performance on the 4th grade Terra Nova in 2005 to that of the highest scoring state on the 4th grade NAEP math exam in that same year, Massachusetts. NAEP, also known as the Nation's Report Card, is by far the nation's most respected source of K-12 testing data.

Massachusetts scored 4.2% higher than the national average. Incredibly, however, Arizona's Terra Nova test put Arizona's 4th graders 18% above the national average, more than 4 times higher above the national average than Massachusetts.

These results, which are not limited to 4th grade math, fly in the face of everything we know about Arizona's student population. Arizona has far higher percentages of low-income and non-native English speakers than the national average, and far greater still than Massachusetts. Arizona's version of the Terra Nova lacks even the most basic level of surface credibility.

There are two broad possibilities to explain this situation. First, the Dual Purpose Assessment (the combined AIMS and Terra Nova exam given to Arizona students) may contain serious flaws, producing unreliable data.

Second, it is possible that the Arizona Department of Education has excluded broad categories of students when reporting statewide results. If, for example, the ADE collected the testing data but then chose to exclude special education and/or English language learners, it would greatly inflate the scores.

One or both of these things could explain ADE's laughable claim that our students are soaring far above the national average. I'll do my best to get to the bottom of it. In any case, Arizona suffers from an unreliable testing system. The State Board of Education should investigate these matters with their own experts and put a reliable test in place.

Dr. Matthew Ladner is vice president of research at the Goldwater Institute.