Is EVAAS a 'Clear Path to Global Ed Excellence' or Product of Grandiose Marketing?

According to a recent post by Audrey Amrein-Beardsley on her blog VAMBoozled! "VAMs (Value-added measures) have been used in Tennessee for more than 20 years" and that they are the brainchild of William Sanders, who was an agricultural statistician/adjunct professor at the University of Knoxville when introduced. Sanders simply thought, according to Amrein-Beardsley, "that educators struggling with student achievement in the state could simply use more advanced statistics, similar to those used when modeling genetic reproductive trends among cattle, to measure growth, hold teachers accountable for that growth, and solve educational measurement woes facing the state at that time."

Sanders went on to develop the TVAAS (Tennessee Value-Added Assessment System) that later became EVAAS (Education Value-Added Assessment System) which is now owned and marketed by SAS Institute in North Carolina. Today, SAS EVAAS is the "most widely adopted and used, and likely the most controversial VAM in the country" according to Amrein-Beardsley. According to her post "What's Happening in Tennessee?" these are some of the lesser known and controversial aspects of SAS's EVAAS:

• "It is a proprietary model (costly and used/marketed under the exclusive legal rights of the inventors/operators.)" EVAAS is the property of a private company whose responsibility is to profits, not necessarily to what's good for kids or teachers. Four states, Tennessee, North Carolina, Ohio, and Pennsylvania, pay millions for the ability to use this Value-added model.
• EVAAS is "akin to a 'black box' model. It is protected by SAS with a great deal of secrecy and total lack of transparency. This model has not been independently validated, and Sanders has never allowed access for others to independently validate the model.
• "The SAS EVAAS web site developers continue to make grandiose marketing claims without much caution or any research evidence to support these claims. 
• "VAMs have been pushed  on American public schools by the Obama Administration and Race to the Top."
• SAS makes this marketing claim on their web site: "Effectively implemented, SAS EVAAS for K-12 allows educators to recognize progress and growth over time, and provides a clear path to achieve the US goal to lead the world in college completion by the year 2020."

There's no doubt that EVAAS or some other VAM product has been foisted on states and school districts by direct mandate by the Obama administration. It is also true that one could argue that EVAAS is a "black-box" model. It hasn't been independently studied and the inferences our state is making using this model have not been independently validated. SAS keeps the model hidden behind claims of proprietary ownership.

Finally, are the marketing claims grandiose as Amrein-Beardsley indicates? I would have to agree that the claim that "EVAAS is a clear path to achieve the US goal to lead the world in college completion by the year 2020" is pretty out there. On what research do they make that claim? What studies have they used to validate that claim? No research studies are provided. The SAS web site does employ a number of statements that do not offer any supporting research. But, then again, its about "marketing" a product, not about making a case for its validity. But the problem, is, SAS does not make those research-based claims anywhere else either.

But I set aside the concerns about the technical aspects of the model. For me, the whole problem behind EVAAS is that it elevates test scores to a level they do not deserve. North Carolina's state testing system is haphazardly assembled, and is far from being trustworthy enough to base any kind of high stakes decisions upon. I also fundamentally find something a bit inequitable in using EVAAS to determine any kind of rating for educators. I do think educators deserve to understand how those ratings are derived, down to the decimal points and computations. If the formula can't be explained so that educators can understand all aspects of it, it has no place in evaluations. 

But it seems there are issues surfacing in the birthplace of EVAAS. Interestingly, Amrein-Beardsley points out that Tennessee is having some trouble with its use. School boards across the state are increasingly opposing the use of TVAAS in high stakes decisions. Some of the reasons? According to Amrein-Beardsley:

• TVAAS is too complex to understand.
• Teachers' scores are highly and unacceptably inconsistent from one year to the next which makes them invalid.
• Teachers are being held accountable for things that are out of their control, such as what happens to students outside the school building.

North Carolina has jumped on the VAM bandwagon and is holding on for dear life. To make the whole system work, our state has implemented the largest number of state tests in history. Let's just hope all this emphasis on test scores doesn't destroy our schools. I certainly hope we don't have to live with this for 20 years!