Wednesday, June 19, 2013

Data-Driven Decision Making Usually Means “Test-Score Driven Decision-Making"

I received another one of those “Data-Driven Decision-Making Workshop” email advertisements the other day. I usually receive several of those during the course of a week. It’s as if someone out there has drawn the conclusion that I do not make decisions based on data, so I am in need of more training in this area.

So what is it that most of these workshops offer? They should be entitled, “Test-Score Driven Decision-Making” because that’s what most of them are really about. They are about using test scores to drive all school improvement activities. They're most often about not moving a muscle until you've consulted the numbers. Those who push making decisions based on test scores alone, elevate them to "sacred" status because it is somehow seen as "objective" and above reproach. Their true "objectivity" is certainly always debatable, but test scores are clearly not all the data we have at our disposal as school leaders. I really can’t help but resent the fact that these individuals actually think the decisions I make as an administrator aren't data-driven. I submit that every decision I make as principal is data-driven, otherwise I would be leading a school by the old proverbial “seat of my pants.” But perhaps the data I use isn't "sacred" enough to be considered data by the data-purists.

So what do I mean when I say my decision-making is always data-driven? It means that every single time I make a decision, I use data. When I am staring at each and every decision-point, I always begin the process by gathering data. For example, if I notice a student is failing multiple subjects, I have used data (grades) to determine that a student is in trouble. I might then call that student in my office and ask questions to gather additional data. "Why are you struggling in Algebra I?" I might ask. Or, "Is there something that’s keeping you from being successful in biology?" The answers to these questions are data. I would add that this kind of data allows me to make much better inferences regarding a student’s classroom learning and what to do about it than the test scores alone would do.

The problem with the term “data-driven decision-making" is that, most of the time, people most often always mean “just test scores.” They wall off a whole world of data as irrelevant because it isn't “sufficiently objective.” It’s almost as if speaking to a teacher to get their impression of a student’s performance is somehow tainted because it isn't numerical or something. Don’t get me wrong. I do see test scores as useful. They can tell us a bit of what we need to know about a student’s learning. What they do not tell us is the whole picture. For example, a student who repeatedly gets poor test scores in reading could allow us to draw any number of inferences about the reasons for their performance. Those might be:
  • They have had bad teachers in former years.
  • The school they attend is more or less a war zone rather than a school.
  • They didn't have breakfast. In fact, perhaps they are homeless.
  • They didn't get any sleep the night before the test.
  • They hate the subject being tested.
Any number of inferences based on those scores may be valid, but I can't immediately jump to the conclusion that his score or the class's test scores are poor because the teacher is a bad teacher. It’s only when I investigate further that I can make any of these inferences valid. When we check with the parents and teachers, we might discover that the real problem is that the child can’t see, and is in need of glasses.

Making decisions entirely based on test scores alone is educational malpractice; there's simply no other way to state it. Data-Driven decision making can consider test scores, but it should also include that qualitative stuff that drives data purists nuts. But then, those people don't live in a real world anyway.

The point of today’s post isn't to create excuses for bad test scores. The point is to make clear that “data-driven decision-making” should never, ever be just about test scores. Those pushing “promotion-based-on-test-score” schemes for students” and “teacher-evaluations-based-on-test score” schemes would do well to remember that our best decisions about the welfare of the kids, and teachers, in our schools are not based on the latest end of grade tests or latest SAT scores. The best decisions we make might glance at these scores, but are based on data that is much closer to what the student needs are. Our best decisions are made when we take considerable time to investigate and gather much more than numbers on a spreadsheet. 

So, in order to keep from having to read any more these "data-driven workshop emails, I have set up a Gmail filter to dump them into my trash folder immediately. The whole idea that any one in education could make decisions without data is just plain ludicrous.

Tuesday, June 18, 2013

Establishing a Culture of Creativity by Paying Attention to Failure

“An organization’s language in relation to ‘failure’ is crucially important to creativity.” Andrew Grant & Gaia Grant, Who Killed Creativity?…And How Can We Get It Back
Those with great accomplishments started out at a stage of zero recognition. They began with little, but they reached the pinnacle of accomplishment only after facing rejection and failure. Those who we think have accomplished much, such as Steve Jobs and Bill Gates did so, in part, because of resiliency and creativity. It is this same resiliency and creativity that many of us would like to instill within our students. But while our schools often talk boldly about fostering a sense of resiliency and creativity in our students, our will often does not match our language nor our actions.

As Grant & Grant point out, an “organization’s language in relation to ‘failure’ is crucially important to creativity." I would that our actions toward failure matter as well. Our schools have become (perhaps they've always been) places that dislike creativity because they value standardization and conformity more. Those who fail to conform are shamed for their failure and prodded to get back in line. Those who fail to live up to “standards” are labeled as “failures.” In this, our schools engage in “language” and "actions" that reveal where the heart truly is in relation to creativity. We use words like “standards,” “grades,” “assessments,” and “tests” as tools to identify and label failure. Then we often leave students to their own devices instead of fostering the idea that failure is a learning opportunity.

When it comes to our actions, our schools celebrate the victories, and not the struggles. We make the most noise about the wins, and often ignore or minimize the losses. State championships in athletics, scholarships awarded, and spelling bee wins are celebrated loudly and continuously. In doing this though, are we not teaching our students that only “winning” matters because that is what we make the most fuss about? Then, we wonder why our students aren't more creative, and why they fail to demonstrate resiliency in the face of failure.

Let me be clear. I am certainly not advocating the celebration of mediocrity and non-accomplishment, or even celebrating failure. We need to celebrate the wins, the accomplishments and the successes. But we need to pay closer attention to how we treat "failure," with our words and our actions, if we want a culture of creativity and schools where students are willing to take risks. 

As Grant and Grant point out:
“Creative individuals have to be resilient in the face of rejection, self-sustaining, and self-reinforcing.”
Perhaps we can begin to establish a culture of creativity when we begin to pay closer attention to the language we use in relation to failure. But more than that, we can begin to pay attention to how we view and react to both succeeding and failing, winning and losing. It’s how we speak of and react to these that teach our students resiliency, and ultimately foster a culture of creativity we seek.

Monday, June 17, 2013

Evernote Journal App for iPad: Excellent Way to Take Meeting Notes and Keep Logs

I NO LONGER RECOMMEND EVERNOTE TO PERSONAL USERS. THE PRODUCT IS NO LONGER VIABLE IN MY OPINION FOR PERSONAL USERS.

There are quite a few Evernote apps that extend the note taking program’s functionality, and I have used quite a few of them. Recently, I  found  Evernote's Journal iPad app. This app makes it possible to keep a journal right in my Evernote application. Being able to do this makes keeping a log or meeting notes even simpler because this app allows you to create those notes with automatic dates right within an Evernote note.

Journal for Evernote has the following features of capabilities:
  • Creates a stack in your Evernote account called “My Journal” and stores your notes by date within yearly notebooks.
  • Notes can be saved and uploaded automatically.
  • Date and time is automatically added to the top of your note.
  • Each entry in the Journal app is recorded as a separate note.
  • Easy to use interface makes entering notes quite simple.
Evernote users looking for a simple way to take meeting notes or keep logs may want to add Journal for Evernote to their iOS device immediately. 


Journal's iPad App Interface

Call for Skepticism and Caution When Using Test Scores in Teacher Evaluations

“We need to be careful that the tests we use in a properly designed teacher-appraisal system do, in fact, contribute to a valid (that is accurate) inference about a teacher’s quality.” W. James Popham, Evaluating America’s Teachers: Mission Possible?
North Carolina took the plunge this year and started using test scores as part of teacher and principal evaluations. The state has even invented a "new" kind of test, called a "Measure of Student Learning" in order to make sure there is plenty of test data to go around. What is particularly telling is how "carefully" the state crafted the term "Measures of Student Learning." It's as if somehow, not calling it a test, makes it not a test. State level educational logic never ceases to amaze me. Of course, the state then started calling these "Measures of Student Learning" something else. They started calling them "Common Exams." Notice again, the careful use of the word "exam" rather than "test." It's almost as if you don't call it a test, it isn't a test, but apparently state level policymakers haven't heard the old saw about a rose still being a rose even if it has another name.

Besides North Carolina's struggle with what to call their newly implemented tests, there's still the question of what the unintended consequences of having thousands of teachers "teaching to the test" is going to do for students in our state. Ultimately, being able brag that your students "Have the best scores in the world" is most likely what politicians and state level education officials are after. That's why they see salvation through test scores as the means to the "Educational Promised Land." Ultimately, there's a flawed logic driving this whole accountability and testing movement: it's the whole idea that learning can be entirely reduced to bubble sheet answer sheets and taken in a single sitting. And, that teachers can't be trusted to tell when a student has demonstrated that they have learned or not.

In my years as an educator, I have been amazed how trusting and accepting educators in North Carolina are when it comes to the latest policy flowing down from on high. It's as if they accept that those at the state level know more than they do, or somehow have access to magical information they do not have. So, when they implement something like the use of test scores in evaluations, many educators accept that the powers that be at the state level know what they are doing, so they trust them. Given the history of reform ideas and educational policy that travels down from on high, this "trust" is highly misplaced. I like to think that state level education officials mean well, but what often has happened during my career, these ideas when implemented locally have sometimes been a disaster and have been sometimes downright bad for kids. Instead of being so trusting, I submit that all educators in the schools and districts need to become skeptics and ask tough questions of our state-level, and federal level policymakers. We should never accept the "trust me, this will work" answer.

It is in this spirit of skepticism, I turn to Popham's book, Evaluating America's Teachers: Mission Possible? and our state's venture into making high stakes testing even more high stakes. In spite of what our state-level policymakers say,  I am not fully satisfied that North Carolina's tests are adequate measures of educator effectiveness, and  a healthy skepticism is still in order. This whole push to add test scores to teacher and principal evaluations has been a rush from the start. Depending on when you asked questions, how the tests were to be implemented has changed multiple times throughout the last two years. Never mind the fact that not a single teacher in North Carolina even saw the test before they were implemented. In their rush to have "test data" it's as if our state level policymakers think "any old data will do." They have failed to take the time to establish whether any of these tests really tell us anything about teaching quality.

In light of our state's push into "higher stakes testing", I think Popham reminds us of some important key issues and ideas about tests and teacher evaluations that state politicians and policymakers seem to forget.
  • “Tests are not valid or invalid. Instead, it is a test-based inference whose validity is at issue.” In other words, it isn't the test that’s valid or invalid, it is the inferences drawn from those tests that have these qualities. It boils down to whether you can actually make an inference based on the test or not. The question is whether North Carolina's tests, which have been implemented haphazardly and a thrown-together-manner, actually tell us anything at all about the quality of teaching in our classrooms. Can I honestly say Teacher A is a good teacher because she added "this much" value to her students' Measures of Student Learning? Seems to me that it puts a great deal of faith in a single test.
  • “Tests allow us to make inferences about a test taker. This inference, depending on the appropriateness of the test as a support for the inference being made, may be valid or invalid.” As Popham points out, the inference we make about the learner may be valid or invalid depending on the “appropriateness of the test” in its role to support the inference being made. As we know, the word validity is the extent to which that inference, or conclusion, is well-founded or corresponds to the real world. This boils down to whether the inference we draw about a student is valid or not. For example, should we infer, based on a student’s test scores that he is not proficient in the subject, we must be satisfied that the test we are using is the “appropriate measure,” and we must also make sure the conclusion we draw considers all real world facts. Ignoring a student’s socio-economic status, or even whether he experienced  a death in the family, can make our inference about the student’s proficiency invalid. Then there's the whole issue about making an inference about a teacher or principal's effectiveness using this same test. Has North Carolina sufficiently established the appropriateness of their Measures of Student Learning, End of Grade Tests, End of Course Tests, as instruments that allow for making inferences about teacher and principal quality? I'm not sure they have. Another question, do these Measures of Student learning allow us to make valid inferences about teacher quality? I'm not convinced they do.
As North Carolina moves forward with a teacher and principal appraisal instrument that uses test scores to determine effectiveness, all educators need to educate themselves and scrupulously ask questions of policymakers at all levels.

As Popham suggests, “If heavy importance is being given to students’ performances on state tests for which there is no evidence supporting such an evaluative usage, then teachers (I would add principals too) might wish to engage in further study of this issue so that, armed with pertinent arguments, they can attempt to persuade educational decision makers that more appropriate evidence should be sought.” In other words, all educators, administrators, and teachers need to study how North Carolina or any state is using test scores to determine educator effectiveness.

Administrators owe it to their teachers, and themselves, to understand that some of these tests were never designed to determine educator effectiveness, so that data needs to be viewed with skepticism. And, I would add that the manner in which these Measures of Student Learning were developed and are administered may not allow them to draw valid inferences about teacher quality. Test scores in North Carolina currently are only 1/6th of the teacher evaluation, and effective administrators are going to keep this in mind and not let the allure of numbers numb them to the other 5 standards.

Tuesday, June 11, 2013

Taking Time to Reflect as a School Leader: Courageously and Respectfully

“The most fundamental aggression to ourselves, the most fundamental harm we can do to ourselves, is to remain ignorant by not having the courage and respect to look at ourselves honestly and gently.” Pema Chodron
One, sometimes ignored, fundamental practice of leadership is “self-reflection” or “introspection.” In our busyness of the day, we forget, or fail, to take time to reflect and look within, which is where the core of who we are as leaders comes from. As Pema Chodron, author and Buddhist teacher, suggests, this failure to look at ourselves is a both a “fundamental aggression” and “fundamental harm” to ourselves. I would add that failing to have to courage  and respect, for ourselves and others, to examine ourselves a little bit each day causes so many of our own leadership problems. Perhaps what we should call this is “Leadership for the Ignorant 101.”

We remain ignorant of ourselves as leaders when we do not take time throughout the day to reflect with courage about ourselves. In this self-examination, we demonstrate a much-needed respect for ourselves as leaders. Today, take some time to reflect on yourself, your leadership.

Wednesday, June 5, 2013

5 Words Educators Need to Forget

You can tell a great deal about education by the words and phrases educational policymakers and educators are using currently. Not too long ago, I remember educators, administrators and policymakers throwing around the words “total quality management,” “outcome-based education,.” and "Site-Based Management." You don't hear those words as often for a variety of reasons. The ideas are no longer popular, or someone decided to repackage it and call it something else, but the words and language in vogue say a great deal about the values of those determining education policy and reform. It is with this thought in mind, with a bit of sincerity and lighthearted fun, I give you my personal list of “Most Currently Mis-Used Words in Education.” 

1. Value-Added: This word, obviously commandeered from business and industry, is my personal number one Worst Word in Education for a reason. As it's used in education, it seems to imply that our students, our kids, are things to which we make more valuable in some way by the processes and “education” we subject them to. By its nature, it implies that the object to which the value is being added, has no say or part in creating that value. At the worst, it is demeaning because it reduces the kids in our classrooms to objects or raw materials. This term has no place in education, unless of course, you are educating widgets.

2. Technology Integration: The definition of integrate is “to form, coordinate, or blend into a functioning or unified whole.” Educators have been talking for years about “technology integration” as if that’s somehow going to change things and students will suddenly learn more. The problem is, if we are trying to blend technology into a classroom that is already dysfunctional, or we’re trying to unite technology into an education system that already fails too many students, we get  a classroom where students use technology but still don't learn. We also get an education system that uses technology to perhaps simply streamline the process of failing too many students. The word “integration” when used with technology, implies that we can successfully blend all these wonderful devices into the classrooms we have and "Presto," we have successful teaching and learning. This naive view of technology and education has outlived its usefulness. This term should no longer have a place in our discussions about technology's role in education.

3. Technology Infusion: I’m not sure this word is any better than the word "integration." Infuse means “to cause to be permeated with something (as a principle of quality) that alters usually for the better.” Some in education have talked about “infusing technology in the classroom” but the problem with this word, like “integration” is that it is simply taking what exists and adding technology to it. What if the existing pedagogy or educational practice is bad? Will giving it an “infusion” of technology somehow make it better? Perhaps, but only if that “tech infusion” addresses the underlying problems to begin with. Tech infusion and tech integration are words that educators need to jettison. Both impart “salvation” abilities to technology that it simply doesn't have by itself.

4. Achievement: No, I do not advocate doing away with student achievement, after all, we're in the student achievement business. What I do advocate is that we drop the use of the word “achievement” when everybody knows what we’re talking about are test scores. Why not just say “test scores?" We know that that’s what is meant when policymakers and politicians start talking about achievement. Let’s keep in mind though, achievement and test scores are not entirely interchangeable terms, because whether or a single test score represents what a student has achieved is open for discussion and debate.

5. School Executive: What's wrong with "principal" or "administrator?" Does calling oneself an "executive" fundamentally change what we do and who we are? This trend to start calling school administrators an "executive" betrays thinking that executives somehow have more power or prestige. The truth is, you can change the name all you want, but unless some changes about the role or job, it is still what it's always been.

There's a great lesson in the current verbiage used in education. If you really want to assess the current feeling of educational reform and policymaking, pay attention to the language. It always betrays what the people who are making the rules are really thinking and what their real agendas are.

Monday, June 3, 2013

Empathetic Leadership: Sign of Being a Mature Leader

Are you facing the prospect of making a tough decision today or in the near future? Perhaps, you are having to deliver bad news to a staff member, or a parent, or student? There are certainly what we perceive to be easier ways to deliver that news. We could write a letter, send an email, or maybe call, but each of these seems to be perhaps a bit to heartless and cowardly in some ways.  Inside, we think, "I would not want to hear this news in that manner." If we choose the wrong means of delivering that news, the price we often pay is diminished trust and feelings of goodwill, which are no doubt important to our leadership and our organizations. 

Making tough decisions the wrong way can breed harsh feelings and a total lack of good will in an organization. What is needed in these kinds of situations is leadership with a heavy dose of empathy. When we deliver bad news to people, whether it’s that one of our employees failed to get a promotion or that their job is being cut, the news is going to be quite painful.  We can simply deliver the news in our “Tough SOB” persona, or we can do so with empathy. Even in the toughest of circumstances, it is possible to create trust and goodwill, if we exercise “leadership with empathy.”

As you make tough decisions, you might use “Just Like Me” thoughts suggested by Chade-Meng Tan in his book Search Inside Yourself: The Unexpected Path to Achieving Success, Happiness (And World Peace). As you face the prospect of delivering bad news, you might reflect on the following from Chade-Meng Tan's book:

Just Like Me
This person has a body and a mind, just
like me.
 
This person has feelings, emotions, and
thoughts, just like me.
 
This person has, at some point in his or
her life, been sad, disappointed, angry,
hurt, or confused, just like me.
 
This person has, in his or her life,
experienced physical and emotional
pain and suffering, just like me.
 
This person wishes to be healthy and 
loved, and to have fulfilling
relationships, just like me.
 
This person wishes to be happy, just like
me.

 
By reflecting on the reality that the person you are talking to is "Just Like Me" it becomes easier to be empathetic and understanding. Exercising empathy in the face to tough decision-making is definitely a sign of mature leadership. We can be tough, and not be an SOB.