Just What Do These International PISA Scores Mean? Nothing!

"There is no association between test scores and national success, and, contrary to one of the major beliefs driving US education policy for nearly a half a century, international test scores are nothing to be concerned about." Keith Baker, "Are International Tests Worth Anything?" Phi Delta Kappan October 2007

Do the PISA international test scores really mean anything? No, according to Keith Baker, who once worked as a researcher for the US Department of Education. Diane Ravitch quoted Baker in her post at the Washington Post, "Four Lessons on the New PISA Scores." Ravitch, who certainly knows more about the history of international test scores than Secretary Duncan, points out, if anything, these PISA scores show that "the billions invested in testing, test prep, and accountability" have not done anything to raise our international test score standings. Duncan's mythical connection between international test scores and economic success is a tired one, and as Baker points out in his article, "Are International Tests Worth Anything?" it is wrong.

Besides, it appears that this data isn't all it is cracked up to be. David Stout at Time magazine points out in "China Is Cheating the World Student Rankings,"  that the Chinese, unlike the United States and other countries did not release all their scores. They only released the scores for ShangHai. As Stout points out, Hong Kong does not count for the Chinese, because they send in their own data because of their level of independence from China. Naturally, policymakers and media pundits interpret the scores as representing all of China when they only represent a portion. Makes one wonder what other games other countries are playing to manipulate the scores.

This illustrates a BIG problem with this international score carnival. There's no way to be sure that we're comparing apples to apples as they say, so these comparisons are meaningless. These rankings are worthless because they tell us absolutely nothing about the state and conditions of our schools. They only serve as a political talking point for a Department of Education still pushing a failed standards-testing-accountability policy that's failing because it does not address the real issues in our schools. And, if you disagree with them, then you're defending the status quo. I hate to break the news to them, but methinks they're actually perpetuating the status quo. It's really sad that Arne Duncan and his Department of Education would use these scores for propaganda purposes to continue to push his failing education agenda, but he's living up to his true calling: he's a politician, not an educator.

Obsession with International Test Scores and Arne Duncan 'Crying Wolf''

Yesterday, the media continued the tradition of sounding the alarm: "Our schools are doomed according to the latest PISA, or Program for International Assessment, scores." NPR chimes in with this one, "PISA Test Results for US Students Are Sobering," and Huffington Post has this headline, "US Test Scores Remain Stagnant While Other Countries See Rapid Rise." NBC news echoed Huffington Post with this one, "US Teens Lag in Global Education Rankings as Asian Countries Rise to the Top,"  One has to question when this incessant obsession with international test scores is going to stop. Why all this fuss about being first in test scores? Do they really think that somehow, magically, our nation will be transformed and educated when we suddenly move up the rankings?

Then there's Education Secretary Arne Duncan who is "Crying-Wolf" once more, when he says, "We're seeing a Picture of Educational Stagnation" as he pointed out at Townhall.com. About the only thing stagnant is his incessant droning about these test scores every time they come out.  Duncan hasn't learned the old wisdom that says "If you cry wolf too many times, people stop listening to you." Perhaps its time we do just that. He, no doubt, will use these scores as an opportunity to push his educational agenda of National Standards, National Testing, and tying teacher evaluations to test scores. His playbook of propaganda has become all too transparent over the past several years.

The truth is out there though. As Diane Ravitch pointed out in her new book Reign of Error, and as she points out in this Washington Post op-ed, "The myth persists that once our nation led the world on international tests, but we have fallen from that exalted position in recent years. Wrong, wrong, wrong." Ravitch points out that "THE UNITED STATES HAS NEVER BEEN FIRST IN THE WORLD, NOR EVEN NEAR THE TOP, ON INTERNATIONAL TESTS."

So why this continued obsession with being first? We have never been first since international assessments were first given in the 1960s or 1970s. Does that mean we haven't ever been economically competitive since? I think the history of business and industry shows that the economy did well during various times even when our international test scores were in the tank. The fact is, OUR ECONOMIC VIABILITY IS NOT TIED TO TEST SCORES and I would add, being first on international assessments isn't going to change our economic fortunes.

Perhaps it's time we, as educators, stopped accepting this mythology perpetuated by Arne Duncan and his Department of Education. It's time for us to demand that the media quit participating in this absurd obsession with test scores and comparing our students' performance with other countries, when we know that other countries game the system and test only more selective students.

Arne Duncan has not yet learned that apples do not compare to oranges, except perhaps in the narrow world he lives in. Educators at all levels need to start countering and questioning this Duncanesque perversion of the truth, and quit buying-in to the false mythologies his department of education is perpetuating. Sure, our schools sometimes struggle. We who are in the schools fight to reach students every single day. We teach our hearts out, and we have Duncan's Doom and Gloom constantly bellowing from Washington.

As far as I am concerned, he has "Cried Wolf" for the last time. He has nothing else left to say worthwhile. So I am no longer listening to him. I can't remove him from the Department of Education, but I can choose to stop listening to his blather. The sooner the Obama administration moves on and Duncan moves out, we can hopefully stop chasing myths and get down to the real business of improving education.

The truth is we are not going to test our way to economic prosperity, so it's time to realize that.

Chalk Talk: Low-Tech Solution to Engage Students in Substantive Discussions

How can we engage students in high, quality substantive discussions on profound topics? One potential way to do this might be to adopt a tradition like our school has practiced since its founding: it's our "the Chalk Talk" tradition.  It is called that because our original "Chalk Talks" were done on a true blackboard and chalk located in our main hallway. Now, we use a large white board that extends several feet along the wall in our main hallway.

The philosophy behind our "Chalk Talk" is simple: We want to to give our students opportunities to engage in profound questions and ideas. Using our Chalk Talk board, our students take the lead by writing the question on the board, and as students walk through the building during the day or after school, they respond to the posted question and to each other's responses. After several days, the end result is a graffiti of student responses, many of them demonstrating much deeper and more profound thinking than what they show in other settings.

One recent question really got students to thinking on the nature of religion in society and in their own lives. That question was, "Does society need religion?" While educators often avoid these kinds of questions like the plague due to the controversial nature of possible answers, if we're going to really get students to thinking in-depth and critically, then we can't wall off the topics and questions just because they make us uncomfortable. In fact, I submit that much of the polarization in our society exists because we don't know any longer how to engage in civil discourse on the controversial and simply be tolerant. This "Chalk Talk" gives us an opportunity to teach students that they can disagree civilly and respect other's beliefs. We can agree to disagree.

Perhaps the "Chalk Talk Principle of Substance and Civility" is best illustrated by an incident where a student recently asked me, "Does it ever make you nervous about the questions we post here?" I thought for a moment, and then answered, "Not really. I might find myself having to answer some parent or community concern questions, but as long as I can defend the civility and substantiveness of the activity, I have no problem with it." We teach students how to respect differences and diversity by not enforcing conformity, but by our willingness to allow students to engage the what is often uncomfortable and that might cause an administrator to be nervous.

Here's the process for our Chalk Talk:

1. A student writes a substantial question on the Chalk-talk board. There are no guidelines or rules posted on what the topic should be about. The communicated expectation is that it must be substantive and require thought and debate as well as response.

2. Once the question is posted, any of our students, and staff (they often like to engage in the discussions too, not control them) post responses. Students simply use the white board markers left at the board for this purpose to write their responses to the question or to other responses. The only rule we have is that we must be respectful, considerate, and tolerant in our postings. Being civil and tolerant in word and deed is the rule.

3. A question may stay up for a week or two, or just a few days. Nonetheless, students are in charge of the entire process for the most part. Occasionally, a staff member will post a question too just to join into the discussion. Students are free to write responses as they pass by the board throughout the day.

There are some who would ask, "Why don't you post some kind of discussion board online instead? It would be much easier for students to access and post." My answer to that is simple. Having a physical board in the hallway means more. Often, it's common for student responses on this board to spill into what we call extended hallway chalk talk discussions. I for one found myself engaged in one the other day with staff members and students. The discussion was extremely profound, and I suggest that I learned as much as the students. The physical board serves as a kind physical place, not unlike a city square or like a Roman forum, where the exchange is with ideas not goods or services. An electronic space could not provide this experience. Besides, high-tech does not always mean better.

If we as school leaders want students really engaged in substantive talk and discussions, we have to realize we can't control the conversation. We can provide the environment. We can encourage respect and tolerance and model civility, but we can't wall off everything that makes us uncomfortable and tell students they can't talk about that. Chalk Talks are a means to foster a true sense of learning community.

Chalk Talk Board in Action

Posted with Blogsy

Merit Pay Once Again Proven to Be A Wash Out According to New Research Study

For those still holding out hope for that merit pay will be the salvation of public education, here's yet another study that points out that such practices are a waste of time. Roland Fryer from Harvard University and the National Bureau of Economic Research, has a study entitled "Teacher Incentives and Student Achievement: Evidence from New York City Public Schools" that is to be published in The Journal of Labor Economics.This study once again affirms that many of us who have spent our lives in education know full well:

Merit pay schemes are a waste of effort and time.

In this study, Fryer points to these findings about merit pay:

• No evidence that teacher incentives had a positive effect on student achievement. In fact, in this study, STUDENT ACHIEVEMENT MAY HAVE DECLINED.
•  Incentives did not change student nor teacher behavior.

One can't but help how many of these studies will have to be done before our politicians and state policymakers will finally understand what Daniel Pink has been saying all along:

"Rewards can perform a weird sort of behavioral alchemy: they can transform an interesting task into a drudge. They can turn play into work. And by diminishing intrinsic motivation , they can send performance, creativity, and even upstanding behavior toppling like dominoes." Daniel Pink, Drive: The Surprising Truth About What Motivates Us

I would send a copy of Drive to our state legislators and even our governor, but I'm not sure they read books.

Misplaced Faith in Value-Added Measures for Teacher Evaluations

Due to Race to the Top and the No Child Left Behind waivers, 41 states have now elected to use Value-Added Measures or VAMs as a part of teacher evaluations. This is done, without regard to the limitations these statistical models have and without any supporting research that says doing so will increase student achievement. What are those limitations? In a recent post, the authors of Vamboozled, provided this post entitled  "Top Ten Bits of VAMmuniton" that educators can use to defend themselves with research-based data against this massive non-research-based shift toward a model of teacher evaluation that will most likely do more to damage education than No Child Left Behind or any other education "reforms" of modern times.

I recently uncovered a journal article entitled "Sentinels Guarding the Grail: Value-Added Measurement and the Quest for Education Reform." which describes a rhetorical study by Rachel Gabriel and Jessica Nina Lester which examined the discourse during a meeting of the Tennessee Teacher Evaluation Advisory or TEAC from March 2010 through April 2011. TEAC was a 15 member panel appointed by the governor of Tennessee to develop a new teacher evaluation policy. The authors of this study examined the language used by those on this panel as they deliberated through the various components of a teacher evaluation policy.

What is interesting about this study is that the language employed by those in this meeting betray some important assumptions and beliefs about teaching, learning, testing, and value-added measures that aren't entirely supported by research or common sense.

According to Gabriel and Lester, Value Added Measurement became a sort of "Sentinel of Trust" and sort of a "Holy Grail" in measuring teacher effectiveness during these meetings in spite of all the research and literature that points to its limitations. According to the author's of this study, here's some of the assumptions those in this TEAC meeting demonstrated through the language they used:

1) Value-added measures alone defines effectiveness.
2) Value-added measures are the only "objective" option.
3) Concerns about Value added measures are minimal and not worthy of consideration.

As far as I can see, there is enormous danger when those making education policy buy into these three mistaken assumptions about value added measures.

First of all, VAMs do not alone define effectiveness. They are based on imperfect tests and often a single score collected at one point in time. Tests can't possibly carry out the role of defining teacher effectiveness because no test is even capable of capturing all that students learn. Of course, if you believe by faith that test scores alone equal student achievement, then sure, VAMs are the "objective salvation" you've been waiting for. However, those of us who have spent a great deal of time in schools and classrooms know tests hardly deserve such an exalted position.

Secondly, even value added measures are not as objective as those who push them would like to be. For example, the selection of which value added model to use is riddled with subjective judgements. Which factors to include and exclude from the model is a subjective judgment too. Choices of how to rate teachers using these requires subjective judgment as well, not to mention that VAMs are not entirely based on "objective tests" either. All the decisions surrounding their development, implementation and use require subjective judgment based on values and beliefs. There is nothing totally objective about VAMs. About the only objective number that results from value-added measures is the amount of money states pay consulting and data firms to generate them.

Finally, those who support value added measures often just dismiss concerns about the measures as not a real problem. They use the argument that VAMs are the "best measures" we've got currently as flawed as they are. Now that's some kind of argument! Suppose I was your surgeon, and used "tapping on your head" to decide whether to operate for a brain tumor because "tapping" was the best tool I've got? The whole 'its-the-best-we-have' argument does not negate the many flaws and issues and the potential harm using value-added measures have. Instead of dismissing the issues and concerns about VAMs, those who advocate for their use in teacher evaluations need to address every concern. They need to be willing to acknowledge the limitations, not simply discard them.

I offer one major, final caution to my fellow teachers and school leaders: it is time to begin really asking the tough difficult questions about the use of VAMs in evaluations. I strongly suggest that we learn all we can about the methodology. If anyone uses the phrase, "Well, it's too difficult to explain" we need to demand that they explain anyway. Just because something looks complicated does not mean its effective. Sometimes we as educators are too easily dazzled by the "complicated" anyway. The burden is on those who support these measures to adequately explain them and to support their use with peer-reviewed research, not company white-papers and studies by those who developed the measures in the first place.

Mendeley: Free or Inexpensive Research Reference Manager for Use Across Devices

One of the grandest lessons I've learned in one semester in my EdD, doctoral program, is that being able to organize all my journal articles and documents is a must. I tried several reference manager options, but Mendeley seems to help me manage all the articles I am collecting as a part of my research in a manner most compatible to my own organizational needs. While certainly not as fully-featured as other options like Endnote, Mendeley gives me exactly the features I want to help me with the collection and reading of the many journal articles and documents I have. My favorite features of the software include:

• Cloud Access: I can upload PDF copies of my articles and have web access wherever I need it. The articles are stored in the cloud, also making it possible to download them to other devices as needed.
• Desktop App: Mendeley offers an easy-to-use desktop app as well. The app interface is much simpler and easier to use than some of the other reference management software I've tried.
• iPad app: Mendeley offers a fully featured iPad app that immediately gives me access to my research on my tablets. 
• Free with 2 GB of Storage Online but inexpensive to upgrade: While a user only gets 2 GB of cloud space to begin, there are inexpensive options for increasing that storage up to having an unlimited account.
• PDF reader that allows sticky notes, highlighting and note-taking: Once users have imported journal articles or documents into Mendeley, they can then read them with a PDF reader, highlight passages, add sticky notes or take notes within the reader. These can then be exported for use elsewhere.
• Groups Feature Offers Ability to Collaborate with Others: Users of Mendeley can use the groups feature to collaborate with others on research projects. I haven't fully explored this yet.

Mendeley Desktop Interface

Mendeley iPad

Mendeley is a free and relatively inexpensive reference manager for those, like me, in the middle of a degree program. For more information regarding Mendeley, check out their website. Mendeley Website

LiveMinutes: Add Real-Time Collaboration to Your Evernote Account

With a new Evernote extension called LiveMinutes, users of the popular notetaking app can now collaborate with others in real-time on notes. Of course, you've always been able to share notes with others, but this process can end up with conflicting versions of the notes. With LiveMinutes, users can work on notes simultaneously. They can also participate in conference calls, create notes, and exchange and annotate documents all within the Evernote environment. With LiveMinutes, users can turn their Evernote app into a real-time, collaborative environment for just about any project. For more information, here's a video from the makers of LiveMinutes.

LiveMinutes currently offers a free plan and a premium plan for $5 a month. Check out their web site for more information. LiveMinutes Web Site