Can You as an Educational Leader Trust the Evidence of Effectiveness Provided by the Product Developer? Probably Not

Can educators and educational leaders trust the research that companies provide about their products? Should they? Probably not. The tech industry that includes Ed Tech has a major credibility problem in the middle of the calls for screen bans and screen limits. For example, there is growing independent evidence that social media is harming our children. Why else would Facebook settle their massive lawsuit.

I just completed reading Max Fisher’s 2022 book “The Chaos Machine: The Inside Story of How Social Media Rewired Our Minds and Our World” and it clearly describes why we can’t trust research provided by companies on their own products. The book reinforced why I quit Facebook and Twitter sometime ago. But one particular incident in the book illustrates clearly to me why companies just can’t be trusted to provide unbiased, reliable evidence of their own product’s effectiveness or lack thereof.

In the book, Max Fisher points out that Facebook completed an internal study of their technology’s effects when faced with growing suspicion that their site might be making the political divisions in the US worse. Their own research warned internally that the platform was doing exactly what their company’s leaders denied it was doing. 

Their own leaked research said “Our algorithms exploit the human brain’s attraction to divisiveness.” According to Fisher, this was in a 2018 presentation leaked to the Wall Street Journal. This presentation also said that Facebook’s systems were designed in a way that delivered users “more and more divisive content in an effort to gain user attention and increase time on the platform.” 

Later, this research was hidden by Facebook executives and they rejected the recommendations for tweaking the algorithm that would have resulted in less time spent online. They, in other words, deliberately chose to keep users addicted to divisive content that their own machine served up to them.  

This is why companies’ research can’t ever be trusted. Neither can their executives. This is also why educators can’t trust research either done by or paid for by the companies whose products they are considering to purchase.

Companies simply aren’t going to provide educators and educational leaders with reliable, independent evidence that their products are effective. They are also going to hide studies that are less than supporting of their products. I would expect no different, after all that would prevent sales.

Facebook’s own research showed their product was harmful, yet they buried the evidence. They made the deliberate choice to keep their product addictive.

As an educator, I always ask for independent evidence of product effectiveness. Independent being evidence generating by studies not done or paid for in anyway connected to the company. Most often, none exists.

If such evidence does not exist, then   ask the company to allow you to run a study of their product on a trial basis. Use research methods to evaluate the product for what they claim. Once you have that, publish it and make it known. Share it with other individuals.

I can envision a whole network of school technologists who independently evaluate these products and share that evidence independently and not as a testimonial asked for by the company. 

Where the evidence comes from, who paid for it, who conducted the study that generated it, and why the evidence is being provided all matters.

This is what has been lacking in the massive technology acquisition and placement in schools. Now, our parents and common decency requires us to do things differently.

Is the US Department of Ed's Advice to Judge Screen Products by Evidence Helpful? Not Really

The latest advice provided by our US Department of Education advises educators to judge educational technologies by evidence of impact on learning, but there’s one problem, where do we get that so-called “evidence.” From the companies themselves? That’s a horrible idea

“…instructional technologies should be judged by their impact on learning, not simply whether they happen on a screen,” said Assistant Secretary of Elementary and Seconday Education Kirsten Baesler, in her attempt to salvage the status of educational technologies.

The problem is finding such evidence. Sure, you’ll get bombarded by “testimonials” from other districts about how effective a product is, but a pal of an owner of a ed tech company hardly counts as evidence. Besides, testimonials are not reliable and valid evidence. Too much room for corrupting influences in those.

What about any studies the ed tech company provide in their sales pitches? Well, I would not get too excited about that either. According to a 2019 Hechinger Report article, “The Dark Side of Education Research: Widespread Bias” by Jill Barshay, we need to scrutinize any research companies provide as evidence of their product’s effectiveness. Surprise, it is too often biased toward their own products.

Like Big Pharma, who has been guilty of generating their own faulty, self-promoting research for their products, Ed Tech companies and other educational resource companies have been operating from this playbook too.

This 2019 article examined a John Hopkins University study of 30 years of educational research by developers of educational interventions and products. These companies conducted their own research or paid someone to do the research, and most of the time, the results commonly showed greater benefits for students than when the research was independent. “On average, the developer research showed benefits—usually improvements in test scores—that were 70 percent greater than what independent studies found.”

In other words, studies done by or funded by the companies themselves were biased. They always made the product look better than independent studies did.

So, the US Department’s admonition that educators and education leaders “look to the evidence” before banning a screen product will not help at all. Most products, if there is any evidence, it is either testimony from another district, which is highly unreliable, or it is their own insider research which can’t be totally trusted.

Most companies, after all, would not want independent research because you can’t control the outcomes. Who wants to find out their product is garbage?

So what do we do with the US Department of Ed advice? Not much it seems, for independent, reliable research on these products is rare and mostly non-existent.

With school district technology employees taking free training from the likes of Google, Microsoft and others, can you really trust anything they say about the products either?

I might add as well, school technology conferences are also heavily underwritten by technology companies, so…well, not much help there either.

What do we do then as educators to determine whether a screen product is effective at what it does? Without independent research, I would conduct small-scale in-school studies myself before implementation to determine effectiveness. If enough educators did this and shared results, then there could be a pool of independent knowledge about these products. But it is going to require school technology educators become critically independent.

As expected, the US Department of Education’s advice about technology is about useful as a study provided on the safety of cigarettes by tobacco companies.

The Real Content of Digital Literacy and AI Literacy

What is AI literacy and digital literacy? Well, it shouldn't be just be teaching students "how to use."

It should be teaching students the technologies on a level that they can understand them much, much more deeply. 

We don't want "societal automatons" who just use as they're told.

We should want humans who can make the moral choices about what role AI specifically and technology generally plays in their lives and their future.

We should want humans who can decide for themselves whether the energy costs, water costs, and other resource costs are worth the continued personal and societal investment in AI and other technologies.

Too often Ed Tech and Digital literacy is about making "good-little users and consumers of technology."

Equipping our young with the whole picture behind these technologies, such as environmental costs, human costs, even personal costs, as well as a deep understanding beyond just churning out users for business and industry is imperative. 

Otherwise we are just dumping all the problems in their laps to solve. And I don't have the religious faith in technology that I once held. It is not our savior in waiting. We still have to come back to the human.

When a Silicon Valley CEO Predicts All Jobs Are Doomed, Remember He's Selling You Something!

Why do we continually insist on giving credence to these Silicon Valley CEO prophetic declarations? We shouldn't.

For example Dario Amodei recently prognosticated a whole list of jobs that will be replaced with his pet technology, and I have seen it reposted here in LinkedIn like it was the ten commandments passed down from Mt. Sinai. Why? 

These Seers of Silicon Valley do not deserve this credibility at all. They get much more wrong than right.

For instance, Gil Duran describes one of these errors of prognostication in his book “The Nerd Reich.”

Elon Musk, who has a high opinion of himself and his prophetic talents, predicted in March 2020 that the COVID virus “would soon disappear.” He said at the time, “Based on current trends, probably close to zero new cases in US too by the end of April.”

“The billionaire had zero idea what he was talking about. The pandemic was just getting started.”

We have elevated the Silicon Valley CEOs and technology engineers as well as technology companies to a level of credibility they do not deserve. They have internalized this status and are now the most arrogant and self-promoting humans on the planet, constantly bombarding us with their predictions, which curiously benefit themselves.

This needs to change.

Education needs to stop granting credibility to every prophecy that pours forth from Silicon Valley and from Ed Tech companies. They do not have a special crystal nor a dispensation from the gods that grant them the ability to see into the future. Perhaps instead of taking Silicon Valley CEO soothesaying as gospel, we need to get our students to analyze this for what it is.

Besides, we can see that these so-called “prophecies” are simply part of the marketing tactic to sell more of their products.

The next time you see a post on social media or some glorified media article where one of these prophets of technological doom or paradise are quoted like a propeht, remember that prophets like Musk and his predictions on the pandemic, do not have any idea of what they are talking about. Chances are their prediction directs more people to their products in order to enrich themselves. 

US Department of Ed's Ed Tech Guidance Proves Screen Bans and Screen Limits Just Might Be the Best Tool Parents and Educators Have

There’s a lot of hype by Ed Tech about the new “guidance” offered by the US Department of Ed about technology. Too bad it’s mostly nonsense, and more helpful to ed tech companies than parents, teachers and the students.

One thing missing from the US Department of Ed's new Ed Tech guidelines is an admonishment of tech companies who purposely engineer their products for "screen time" maximization, or what is otherwise known as "engineered addiction."

This is one reason alone for continuing to insist that "screen time limits" may actually be the only tool parents and educators have to counter this problem. 

The US Department of Ed, which I am not sure I would expect much from a department headed by a former WWE professional wrestling executive, fails to point out that tech companies should not have as a goal screen time maximization.

The problem is that tech is purposefully addictive. Cells phones are designed to be the center of attention with their beeping, buzzing, instant notifications. Social media apps are designed to distract users and keep them distracted.

Technology companies need to remove engineered addiction from their designs.

I've been stepping back to 2022 and reading Max Fisher's book "The Chaos Machine" and Fisher points out that Silicon Valley's playbook, whether its Google, Apple, Facebook, Instagram, or TikTok, is to "track and manage users' screen time.

That is the problem with tech in general and Ed Tech specifically. It’s machines of manipulation.

The only tool parents and educators have against this playbook are "screen bans" and "screen limits." Breaking the allure is the one tool that works.

Our goals as educators should not be to preserve screen time for tech companies; it should be to engage students in such a way that the answer to everything does not involve reaching for a device.

Screen Bans Should Be Embraced as an Opportunity to Think Outside the "Technology-Always-Has-The-Answer Box"

Screenbans in schools is an opportunity for Ed Tech and educators, not a threat.

Why? Because it is an opportunity to disrupt the tech-solutionism that has obsessed educators since the PC entered the classroom.

Tech-solutionist see technology as the "always-answer" to every problem.

This thinking places educators in a "Technology-Always-Has-The Answer" box where every problem has a tech solution. That turns into technology-obessiveness.

Despite what the Ed Tech marketers have promoted, they have not been promoting "thinking-out-of-box"; they have been promoting only thinking in their box. 

Screenban discussions and talk moves the thinking outside "technology-always-has-the-answer box” to a universe and world of options out there. Not every problem has a tech solution. 

Screenbans are disruptive of the Ed Tech status quo with its tech-solutionist answers. That’s the reason for their concerns.

Educators should take advantage of this time and de-center technology and place it among the other tools in the toolbox and reach for it when it solves a problem, not spend all their time looking for problems to solve with tech!

Ed Tech Does Not Always Have to Be Revolutionary: It Just Has to Work!

 Educators don’t always adopt Ed Tech to “revolutionize” or “change” anything. A tool doesn’t have to transform the work to be effective.

Sometimes you adopt a new technology to facilitate and help do what you are currently doing, not disrupt; not transform; not change anything.

The mistake often made by Ed Tech is thinking that their product must “revolutionize” or “transform” and that’s their marketing story.

The truth is, there are times when no change is needed other than having a tool that helps you do what you are doing better.

I suspect one of the reasons for the so-called “tech-lash” is because of promises of transformation that never panned out.

But the Ed Tech industry never blames their technologies. It’s always the implementation or the “system” that is rigid and unwilling to change. But, it just might be that the technology does not work, and in some cases it might be harmful.

Have you ever heard anyone in Ed Tech blame technology? I haven’t heard that that I recall. It’s never the technology’s fault.  It’s the system, the school, the leadership, the teacher, the student, the parent…never the technology.

But, just maybe, the Ed Tech is garbage. Maybe the company spent a ton of money dressing it up in expensive marketing clothes, and the actual product is junk. 

Ed Tech does not have to always be revolutionary or transformative. Sometimes it just has to help schools and teachers do what they’re doing better. That’s not as big a sell, but it is a solid product.

Why Are All Office Suite Apps Now Garbage? Arrogant Software Companies Think They Know What Features I Want Better Than I Do

What is it with App developers these days? They really do think they know what I want as a user of their product better than I do.

Microsoft Office has been transformed into garbage. Google Docs is trash. Now, Apple has turned Pages, Keynote, and Numbers into slop by adding a ton of crap and trying to charge a subscription for it.

Earlier this year, I trashed Evernote because of their arrogant force-feeding of features down users throats. I had been a user since at least 2011 or so. 

I cancelled my Microsoft Office 365 subscription after being a user since at least 1996 or so.

True to Cory Doctorow, "all office software has been enshittified!"

Software developers have become so darn greedy that they are willing to make their products trash in order to squeeze one more cent from their users.

If this trend continues, I suspect I might even have to revert to pen and paper.

Come on software developers. Stop trying to shove features down users throats and just maintain an acceptable and useable product.

Why Is There Techlash and Screenbans? Perhaps Its Because It's a Problem!

I spoke to a parent of a middle schooler today. Somehow in the discussion technology came up in our conversation.

She said she honestly believed technology has ruined her son. He uses it for play and distraction instead of learning. He is consumed by it. 

Which makes sense, because our handheld tech is engineered for distraction.

She brought up here school's Chromebooks program and how she saw it a detrimental to her son's education. He simply could not handle the "freedom to be distracted."

Before you start blaming the parent, perhaps there needs to be more thought.

This illustrates what we've got wrong with Tech; we assumed it was the answer for all students. We standardized the technology and how it was used. No wonder there's what Ed Tech calls "Techlash"!

Instead of arming ourselves with arguments for Tech, perhaps we should focus on the students. The technology, if it is simply a tool, it does not have to be the tool used in learning. 

Maybe personalizing learning means having students who learn more effectively without tech engaged in screenless activities. Just maybe we can go a day or two, a week or two, or even a course or two without Tech.

The screen ban trend is not a marketing problem; perhaps sometimes the Tech IS the problem.

The Missing Component in Digital Literacy: Critical Examination of Consequences of Our Digital Technologies

When we speak of digital literacy in education, often that notion is about the construction of “good little users and consumers of technology.” Instead, what is needed is a digital literacy curriculum that expands into examining the greater societal and environmental impact of the choices we make in technologies.

In other words, most often, digital literacy accepts the technology as inevitable, and that we have no choice but become users. When our digital literacy efforts do this, they are simply serving up students as consumers for Silicon Valley tech companies. With that, we are complicit in increasing tech company power over us and our world.

A true digial literacy curriculum needs a strong critical component that carefully examines the consequences of our technological choices. For example, in dealing with AI literacy, there needs to be deliberate efforts to provide students with the ecological consequences of the massive energy and water consumption of the data centers needed to power it. With social media, there needs to be deliberate instruction on how social media has negatively impacted our society. With every technology in our lives, there needs to be critical thinking about the use of that technology. In other words, students must be equipped with critical knowledge that empowers them to decide for themselves whether they want to be users of that technology or not.

Ed Tech is solely responsible for this state of affairs. They have instilled within us that technology is inevitable and that it is always the answer to our problems. The job of Ed Tech in education has always been unquestioningly turning administrators, teachers, and students into users.

When digital literacy focuses on a critical examination of technologies and the consequences of their use, you do not have “good little users and consumers of tech”; you have empowered citizens who create a society that decides for themselves which technologies deserve a place in their lives.