Accountability and Testing: Systems of Educator Mistrust

Accountability systems whether in education, business, or government are based on mistrust; a mistrust that those who are its subjects are unable to or unwilling to carry out the jobs they have been assigned. Theodore Porter, author of the book, Trust in Numbers: The Pursuit of Objectivity in Science and Public Life, writes:

"Perhaps most crucially, reliance on numbers and quantitative manipulation minimizes the need for intimate knowledge and personal trust."

It stands to reason that when education systems, (or any governmental or business system) come to rely on accountability systems, trust is shifted from the “intimate knowledge" of individuals and their capabilities to systems of numbers and statistics that are declared by the hierarchy in the system as being both more trustworthy and representative of truth. In order words, when there’s an intense distrust that those who occupy production positions in the system, systems of accountability and audit are established in order to force the system to do what it’s designed to do. In education systems, “learning” is the object of production, so naturally accountability systems are designed to force the system, made up of administrators, teachers and students’ to “produce learning.” However, “learning” is an object of contention in the first place, with few people agreeing on what it is.

Even with the disagreement on what learning is and what learning is worthwhile, there is more contention with how to measure “learning" it in a way that accurately captures it. Accountability systems look to tests for this task. Tests are developed, one after another in a fruitless effort to measure this idea of “learning” which is actually an exercise in trying to grab water. Just when educational measurement thinks it has “grasped the learning” that it thinks is significant; it escapes through their fingers. That’s while since the dawn of the accountability era, there have been wave after wave of “new standards and new tests,” all in an effort to try to capture the elusive quarry, “learning.

But I have a novel idea, at least novel in the face of accountability and testing; if teachers are professionals, then what if we were to transform teaching back into a profession where practitioners exercise “professional judgment” to determine whether learning takes place and that the system “trust their judgment"? Teachers could once again be educated to teach and use their judgment to decide whether learning has happened, and be trusted, rather than subjected them and to the mistrust of an accountability and auditing system that fails to capture the nature of “learning” in the first place. This endless pursuit of new standards and new tests that have costs millions and billions of educational funding could be shifted to fostering more effective professional teachers and a teacher professionalization system, that avoids trying to mimic and ill-suited medico-professionalization system, to create its own, never-before-realized profession.

At its heart, we are deforming our education system with both accountability and auditing methods that inadequately define “learning” and by default, are incapable of capturing “effective teaching." The mistrust of the professional educator and “trust in numbers and quantitative manipulation” doesn’t fit the task of teaching. It’s perhaps time to stop trying to make education into the image of either business or medicine, and invent a whole new profession that remembers that teaching and learning are much too complex to reduce to numbers anyway.

Do PISA Scores Really Mean Anything? Not Much!

Do PISA (Program for International Student Assessment) scores really matter? One would think so, because every time the latest round of scores are revealed, there’s a barage of “Sputnik-like” declarations of educational doom and gloom within the United States, all reminscent of when the Soviets launched the first satelite into orbit. Just like it did then, each time these scores come out, the declarations of“educational third world country” status is declared anew. The United States’ future economically is seen a bleak, and tales of woe begin. But there are some really good questions to ask about PISA.
If the United States suddenly vaults into first place in the PISA rankings will our country suddenly experience full employment and economic prosperity for all?

Will American businesses suddenly find all those “mythical-but-can’t-find-qualified workers?

Do PISA rankings by country really mean anything?

I would say “No” to all these questions. The United States’ “weak” rankings in PISA scores tell us absolutely nothing helpful nor does it indicate all the gloom and doom that policymakers public policy wonks like Arne Duncan have claimed in recent years.

Having a number one ranking in PISA scores is not a ticket to the economic promise land. Nor does it mean that we should take up the short-sighted view that our education system’s purpose—from kindergarter to higher education— is to feed the “human capital” machine for corporations. Sure, we want out students to be employable, but making sure they score high on an international test won’t do that. That’s short-sighted thinking. Education should never be about preparing students for the factory sitting down the street; it should be about preparing students so that they can learn and adapt for jobs their entire lives. It is simply short-sighted to see education as worker-training program. We need to train our students to be able to learn for a lifetime, be creative, be critical thinkers, and adapt to whatever comes their way. The jobs down the street will probably move to another country in a few years any way.

Using PISA data, or any standardized test data such as SAT as a scare-tactic and propaganda tool is a moral problem. Those who do that, are like the barkers selling a special elixir guaranteed to fix what ails. They should have the same credibility as snake oil salesmen. It is simply policymakers pushing their own brand of reform with the cry of “wolf.” Sure, our schools can always improve. We can teach our students more effectively. But, this clarion call of doom and gloom every time a round of PISAscores or any standardized test scores needs to met with skepticism if not just ignored.

“After School:” Mobile SM App for High Schoolers Offers Potential Headaches for Educators

Yesterday, I stumbled across this post on the Washington Post web site: “Millions of Teens Are Using a New App to Post Anonymous Thoughts, and Most Parents Have No Idea.” It turns that our students now have access to a high school version of the app Yik Yak called “After School.” Yik Yak has been quite popular among college age users for some time. For those not familiar with Yik Yak, or the high schooler app After School, these apps allows users to post messages anonymously that only other users in the general vicinity of the message poster can see. Apps like Yik Yak have been implicated in posting anonymous threats and cyberbullying online. (For example, see “What Is Yik Yak, the App That Fielded Racist Threats at the University of Missouri.”) What’s more, this “After School” app is also at the heart of possible postings of cyberbullying and postings where users threaten to bring guns to school. (See “Why Parents and Administrators Are Freaking Outv About an App Called After School.”)

It is easy to sound the alarm when these kinds of social media platforms arise, but it turns out that this particular app has been around at least a year. It appeared in the App Store and the Google Play Store last fall according to web site Fusion (See here), but it was removed twice because it was used for cyberbulling and posting gun threats. It returned this past April, and its developers claim to have a whole bunch of new safety features, including algorithms that block certain words and it is also said to use human reviewers who examine each post.

Well, for those of us familiar with web filters in schools, we know all too well that the best of these kinds of safety measures are far from foolproof. Many students are masters at trying to get things through the filters, and we’ve given them years of practice. But my beef with After School actually goes beyond that and is captured best by a question: Why in the world would we give high school students an anonymous platform like this when we know from experience that many are not ready to handle the freedom of any posting much less “anonymous posting?” It seems to be the equivalent of giving students the keys to a sports car with a cooler full of beer in the front seat and simply telling them not to drink and drive. After School is simply a platform that has the potential to facilitate irresponsiblity and possibly dangerous cyber-behaviors because students often thrive on web anonymity.

What’s more, I take exception to After School’s FAQ page which seems to suggest by its language there that schools have somehow endorsed this product. (See the After School FAQ page here.) On that page it states at the very top, “Anonymous and Private Message Board for Your School.” Those words seem, at first glance, to make a link between the message board in After School and the school the student attends, but that is far from the case. The school and its administration have no connection, control, or power over this product. That’s why many school administrators are quite upset about this product.
I am not sure an all out panic about this product is warranted, but I do think we need to educate both our students, and especially our parents about the potentially dangerous uses of this app.

We also need to make clear that this app basically allows students to set up their own cyber community, totally disconnected from our schools, that is supposedly monitored by this company. We as parents and educators, by allowing our students and children to use this app, are placing a large amount of trust in algorithms and faceless individuals out there somewhere to monitor the safety. The truth is, we at the school and district level, with all our web filters and cyber-safety gadgets have an extremely difficult time with cyberbullying and web threats. What on earth would make us think that out there in cyberspace a company can do that better? While After School has been around and is not really new, it operates on assumptions that ultimately will probably cause its own demise. That is the idea that anonymous posting for high schoolers can be effectively monitored. Just looking at their disclaimers in the app store should be concern enough. Check it out:

That’s quite a laundry list of items of things parents would object to having their students exposed to.

I don’t usually rate apps with thumbs up or thumbs down, but this one gets double thumbs down because it is a potential cyberdisaster at every school.

Blogo: An Easy-to-Use Blogging App for the Mac

I have waited quite some time for a blogging app for my Mac. I’ve used several, but my personal favorite is Blogo. Blogo provides Mac users with a simple interface. (See below.) It also provides all the features one would expect from a blogging platform as well. It is extremely easy to use, Some of Blogo’s most useful features include:

• Easy photo editing
• Post previews while writing
• Manage blog comments
• Syncs with Evernote
• Use with multiple blogs
• Write blog posts while offline
• Compatible with both Blogger, WordPress, and Medium

Blogo Interface

For more information about Blogo, the Blogging App for Mac, check out their web site here. By the way, I haven’t received any compensation for this post. This is truly my favorite Mac Blogging app.

Chief State School Officers Continue to Believe We Can Test Our Way to Equity

Somehow I am not surprised at all that the Council of Chief State School Officers still believe that we can “test our way to school equity.” They still believe if we “just get the tests right” and hold those teachers’ noses to the grindstones, then “Poof” all our students will receive an equal education. They met and pledged their allegiance to the “Accountability Doctrine” recently in a Council of Chief State School Officers policy forum in Charlotte, North Carolina. (See Ed Weeks Article “State Chiefs Pledge to Continue Focus on Accountability.”) What they don’t seem to get that we had at least 15 years of “Accountability and Testing” and schools are no more equitable, in fact, there’s evidence that they are more unequal than ever.

“Our members want to be held accountable,” said Chris Minnich, the executive director of the CCSSO. That’s nice for him to say, coming from a man who has never spent a single day as a teacher in a public school classroom. If you check out this glowing biography from CCSSO, you can see he knows his policy though. Too bad he has no clue about the nuts and bolts of classroom teaching.

That’s the real problem here. It’s not that the accountability is always a bad thing; its that we have people pushing these damaging policy initiatives who are policy wonks, but know nothing about what No Child Left Behind and current Accountability fetishes have done to schools and ultimately to students. We are not going to make our schools equitable by the having the right tests and accountability systems. We are going to address equity by advocating for a society that does not favor those who have over those who do not have.

Accountability and Testing: Distorting Teaching, Learning, and Public Education

To continue my critique of the “accountability and testing regime,” I have been thinking about what the ultimate goals of those whose faith and belief in the promise of standardized testing, statistical technologies, and classroom surveillance are. They have captured the discourse in education and conveniently made unacceptable anything critical anyone else has to say about testing and its high stakes deployment. An educator who questions it is not taken seriously and is deemed out of bounds. Testing and accountability seeks, in a nutshell, to make teaching and learning “measurable, calculable, in order to be controllable."

What does it mean to make teaching and learning “measurable?” It means reducing teaching and learning to “indicators” or “standards” that can simply be determined to be not present or present. It means making teaching and learning into something that can be captured using the available technologies at our disposal, such as teacher observations and standardized testing. Teaching, then, is made measurable by teacher evaluations, and, more recently, using statistical measures such as value-added models, which both result in what is hopefully “objective” and widely accepted as being “true” measures of acceptable teaching and learning, because they happen to be numerical.

As an administrator, I have heard many of my colleagues make the statement, “If it isn’t measurable; it didn’t happen.” That statement captures beautifully the complete faith in testing and measurement that currently exists in education. But it is also a statement of ignorance. Even the best psychometricians will say that “NOT EVERYTHING IN TEACHING AND LEARNING THAT IS WORTHWHILE IS MEASURABLE.” But this faith in “educational measurement” is at the heart of current educational reform, and it is still believed by many educators, politicians, and policymakers to hold the “silver bullet” that will finally make all public education effective. “We just don’t measure enough and measure effectively” is the belief that keeps driving round after round of testing-and-accountability-based reforms” in education. Tests are cheap in comparison to really dealing with the equity issues of healthcare and poverty. With tests and statistical tools, the belief that one can erase these social justice problems, but sadly that is not the case.

For those of us in the schools, those of us in tune with the teachers and students there, we see the results of this: an education system that continues to be distorted and twisted, that ultimately meets the needs of a few, mainly those who can use these “measurable results” to determine their own effectiveness and the effectiveness of their own ideas. An education where test results are still valued over individuals, and any old methodology that results in higher test scores is acceptable. Testing takes precedence over everything else schools do: just look at a state’s testing regulations if you want to see this. In other words, no matter the rhetoric coming from testing and accountability addicts, testing is driving everything in schools, and that’s they way they want it. That keeps them in power and needed.

Making teaching and learning “calculable” is very much akin to making it “measurable.” Making what we do in schools “calculable” is seeking to reduce what we are supposed to be doing to numbers. Somehow, our current system views “numbers” as somehow more objective, therefore superior to other things like judgment or intuition. This desire to make everything “calculable” leads to bizarre decision-making, where outcomes are ridiculously reduced to numerical values, even if those values distort the process and result. Standardized tests do this very well. They can’t measure an “effective essay” for example. Determining whether an essay , or musical composition, or painting is “effective” is by nature a “judgment.” And, whether it is effective in all instances and in all ways is relative. It might be effective at one thing or in one instance, but not another. Rarely are major literary pieces simply “effective for all time” or “in all ways.” The same applies to music, art, and so many other human endeavors. So, in the name of “objectivity,” current testing manics send essays, compositions, and even paintings to “outside” observers to evaluate all in the quest for “objectivity.” But such actions might create a facade of objectivity based on faith, but it completely results in an unfair evaluation of student work. For, who knows better whether a student has progressed than that teacher who has been in the trenches with that student, day after day and seen their incremental growth first-hand. So, the pursuit of making teaching and learning “calculable” is to simply turn it into numerical values or make it have the facade of “objectivity” because the belief is that “numbers don’t lie.” Testing and accountability becomes more about distrust of teachers and their judgments, than really trying to provide an effective education for students. "We can't trust teachers' judgments about students, so must use tests and other outside evaluators," is the rationale.

It is this desire to make teaching and learning both measurable and calculable that leads me to the final goal of accountability and testing as I see it: to make teaching and learning controllable. Policymakers, education reformers, and even politicians all believe they hold the “ultimate vision” of what effective teaching and learning is. They believe, armed with their many contradictory studies on the subject, that they hold the answers. Answers in hand, they seek to control teaching and learning in order to mold it into their image of effectiveness. Through tactics of measurement and calculability based in standardized testing and measurement, they use high stakes decision-making to weed out the “deviant” practices that don’t meet “best practices standards.” The problem lies though with the truth that both teaching and learning is so complex that to reduce it to universal rules of effectiveness ends up distorting it and neutralizing it to simply a “technical knowledge” that anyone can understand, including administrators and policymakers and education reformers who have never spent a day engaging in teaching in classrooms and making decisions about student learning. Teachers, as a result, find themselves engaging in a strangely distorted form of teaching that must jump through the hoops of “best practices” in order to get the “results” desired by this twisted system of education. Teaching the test and test prep are two examples of this distortion. They have become assembly-line workers who “add” knowledge to students as they roll down the assembly-line, and testing with this value-added component is the “quality control mechanism” that drives teachers in the entire system to produce even more “globally competent graduates" that can produce ‘number one test scores’ on international tests such as PISA. Under the testing and accountability regime, teachers are reduced to technicians whose judgement does not count and means nothing. Test results and other “quantitative” measures are hierarchically superior.

In the end, if you wanted to design an education system that turns education into a factory-like system that produces standard results, you couldn’t have done better with that created by our current accountability and testing regime. If you wanted to create a system that transforms and de-professionalizes teaching as a profession, you can’t do much better. In the end, our public education system might ultimately match up to the vision of those who adhere avidly to accountability and testing practices, but I can’t help but wonder whether those teachers in this system find the same level of satisfaction and dedication to students when test results are valued so highly. I also have to wonder what kinds of students such a system of this really produces. Perhaps, that’s what’s desired by accountability and testing advocates: they want students who don’t question; who don’t criticize; who don’t engage in learning deemed irrelevant such as the arts, and learning seen as deviant. They want both students and teachers who “just do their jobs” and not engage in dreams of how things might be different or better.

Pursuit of Better Classroom Data: Is It Improvement of Teaching or of Classroom Surveillance?

Perhaps there's a truth here about accountability and testing that we have ignored: the use of standardized testing, value-added models, and growth models in teacher evaluations is all about subjecting the classroom to techniques of surveillance. To put it bluntly, they are spy tactics. Their purpose is to peer into the classroom to see if teachers are teaching the "prescribed" curriculum, and to see if teachers are adhering to the "rules of best practice" as they teach. These surveillance techniques are based on a fundamental mistrust of teachers' professional judgment regarding how they should be teaching and how their students perform.

The current efforts to perfect the evaluation of teachers aren't really just about improving teacher effectiveness: they are about sharpening the gaze into the classroom. They are about making the classroom more visible to those higher up the administrative chain. They are about finding the means to "objectively" determine whether teachers are teaching in the manner prescribed by "best practices" and whether they are teaching only the content that can be subjected to testing.

What testing and accountability experts have discovered though, is that tests are imperfect. The image they project of the teachers' performance is at best blurred and opaque. Even with new-fangled "value-added models," seeing teachers' effectiveness is foggy and unreliable. Now, they are seeking other ways to increase classroom surveillance. They are seeking other "spies" which might provide them with a clearer gaze of what's happening in the classroom. What are those new techniques for gazing into classrooms? They are called "student surveys."

Interestingly, student surveys turn students into 25 or 30 pairs of eyes that can report back to administration regarding whether classrooms are conducted in the manner dictated by the laws of "best practices." The student survey becomes another instrument with which the administration and government can sharpen its gaze into the classroom to make sure teachers' conduct adheres to "best practices." Underlying the use of student surveys is the assumption as well that if teachers know that 25 or 30 pairs of eyes are watching that might potentially report deviance back to the administration, those teachers will engage in the expected teaching behaviors. Thus, the control of the classroom becomes more complete.

We need to perhaps realize that the push to ever better classroom data is maybe more about control and transformation of the teaching and the teaching profession into a non-profession where teachers are simply "technicians of learning" whose professional judgment means nothing. The next generation of teachers will not need to exercise professional judgment: they will only need to conduct their classes in the prescribed manner by the "sciences of teaching." The question then becomes, "Are we really producing the kinds of students that our world needs?" Students newly manufactured and stamped with the US Department of Education's approval as being "Globally competent."




- Posted using BlogPress from my iPad