Fuzzy Math Fails Vouchers

Accuracy in Academia ^ | September 15, 2006 | Matthew Hickman

[JSedreporter]

According to a recent study published by the National Center for Education Statistics (NCES), public school students are outperforming their private school counterparts in fourth grade mathematics and have equaled “private school students in fourth grade reading and eighth grade math.” However, as Shanea Watkins, Policy Analyst in Empirical Studies in the Center for Data Analysis at The Heritage Foundation, explains, these results require greater scrutiny.

Some commentators that reference the NCES report believe the study points to a causal relationship—that attending a public school will cause higher academic achievement in math. However, the study focuses on data provided by the 2003 National Assessment of Educational Progress (NAEP), which detailed how students were performing in math and reading at a specific time. In fact, because the data is limited, the NCES added two sections in their report, appropriately titled “Cautions in Interpretation.”

Indeed, the Department of Education suggests that in order to establish causality the research study must “include either random assignment or a pretest measure for the research results to be considered effective.” The NAEP assessments, from which the NCES based their report, did not meet either of these standards. That said, a number of organizations are using this report to discredit private school voucher programs, but the evidence supporting these programs is overwhelming.

In fact, there have been several studies sponsored to evaluate the effects that voucher programs have on reading and math ability. These studies have been carried out in a number of states, including Wisconsin, North Carolina, and also the District of Columbia. The studies compared the accomplishments of students that were randomly awarded positions in private schools and those that remained in public schools. The studies revealed that “Students who received vouchers experienced greater math and/or reading achievement gains than did the students who remained in the public school system.” Unlike the NCES report, these studies consistently meet causality standards set down by the Department of Education.

Watkins rejects the policymakers, journalists, and commentators that would attempt to use the NCES report for the advancement of public schools. She is open to reports and studies that support public schools, but only if they are measurements of students over different semesters. Researchers have generally opposed reports, such as the NCES study, that use test scores from one specific time. These results could simply point to external pressure on students leading to a bad test that day. Watkins contends that if commentators want to accurately reflect educational studies, then they must use those that abide by the Department of Education’s conventions; otherwise they’re simply misleading the public.

Matthew Hickman is an intern at Accuracy in Media

[azemt]

ANy time there's a report talking about statistics, I keep this old adage in mind:

Figures dont lie, but liars figure.

[USFRIENDINVICTORIA]

Without random assignment -- or at least tracking the progress of individual students who change schools -- there are all sorts of possibilities that the researchers can't rule out. One obvious one: students, who are having the most trouble in public school, are more likely to switch to a private school than those who are doing well in public schools. These students could be doing much better in private schools, than they would have in a public school -- but the research won't settle that, because of faulty methodology.

[Captain Rhino]

Well here's a press release that isn't going to make it into the evening news.