US Department of Education Claims That Vouchers Improve Graduation Rates: Junk Science at Its Best

I recently began reading Dan Agin’s book Junk Science: An Overdue Indictment of Government, Industry, and Faith Groups That Twist Science for Their Own Gain. In that book, Agin goes to great lengths to point out how special interests in government, commerce, and the “faith” industry are using “junk” science to change or sway public opinion toward their own biased positions. Yesterday, the US Department of Education released their report entitled “Evaluation of the DC Opportunity Scholarship Program.” The results of that report include the following:

• There is no conclusive evidence that the Opportunity Scholarship Program affected student achievement. “On average, after at least four years students who were offered (or used) scholarships had reading and math scores that were statistically similar to those who were not offered scholarships.”
• The program significantly improved students’ chances of graduating from high school. “The offer of an Opportunity Scholarship Program Scholarship raised students’ probability of completing high school by 12 percentage points overall. The graduation rate based on parent-provided information was 82 percent for the treatment group compared to 70 percent for the control group.
• The Opportunity Scholarship Program raised parents’, but not students’, ratings of school safety and satisfaction. “Parents were more satisfied and felt school was safer if their child was offered or used an Opportunity Scholarship Program Scholarship.The program had no effect on students’ reports on school conditions.”

Immediately, the right-wingers pounced on the findings as if they were earth-shattering. Never mind that the first point clearly contradicts one their oft-repeated claims that vouchers increase student achievement. The headings on all kinds blogs biased toward vouchers made the statement loudly and clearly that “DC Vouchers Improve Graduation Rates.” For example the right-wing Heritage Foundation declares unashamedly that “the evidence is clear school vouchers work for Washington DC students.” (See Here.) Jay P. Greene’s Pro-voucher blog continued the same refrain. But if you look at the study, tucked away in it’s language about the graduation rate claim is the following: “There are some limitations to this analysis, however, it is based on parent reports rather than school administrative records, and it represents relatively a small share of the study sample.”

 In other words, the authors of the study did not use actual graduation statistics for their claims, they used PARENT-REPORTED claims, and that the sample size may be problematic. Why not use the actual graduation rates? Using parent reported graduations is not convincing because of possible reporting errors. The sample size being to small also limits generalization.

The study also includes a “Conflict of Interest” statement that clearly points on that Patrick Wolf and his research team from the University of Arkansas Department of Education Reform were a part of the research team. This is the same Patrick Wolf who is the “Professor and 21st Century Chair in School Choice.” (See here.) Keep in mind this whole department was established by the Walton Family Foundation (Wal-mart) with the goal of pushing school choice and vouchers. This whole department at the University of Arkansas acts just like a think tank, providing biased research to support the narrow-minded interests of the faculty and those of the Walton Family Foundation. Clearly the University of Arkansas Department of Education Reform offers the best research results money can buy.

Any credibility to this study released by the United States Department of Education is swept away by serious flaws in its design, and the unmistakable biases that individuals like Patrick Wolf bring to it.

Dan Agin states that “Often the most important characteristic of junk science is that much of it is based on no science at all, but on wishful thinking, fantastical thinking, and so on. In general, any political group or movement that twists science with bias in order to support a political agenda is producing junk science in a possible dangerous mode. Junk science is often a deliberate tool used by those who seek to exploit the public for one end or another.” This study is an illustration of Agin’s version of junk science. There are clear biases by those who worked in the research. The wishful thinking of Patrick Wolf taints any conclusions that he draws from it. And, the whole purpose of the study supports his political agenda of promoting school vouchers. What amazes me is that our own United States Department of Education would allow itself to be used by those who have narrow political agendas.

The fact that this study was released by our own United States Department of Education does not make it credible. In fact, it should remind us that the very department that is supposed to be helping us carry out our endeavors to educate the young of this nation is still a political animal and its research should be examined with an eye toward skepticism.

As an educator in the 21st century, we have no choice but to examine critically all of the educational research being touted by our own government, think tanks, non-profit organizations, and even our own professional organizations. As I attended two educational conferences in the past two weeks, every time a presenter used the words “according to research” I could not help but slide toward skepticism. As Dan Agin would say, research with an agenda is junk science. Sometimes that agenda is clear and sometimes it is not, but in the 21st century, as an advocate of public education, I remain skeptical every time I hear those words.