Can You as an Educational Leader Trust the Evidence of Effectiveness Provided by the Product Developer? Probably Not

Can educators and educational leaders trust the research that companies provide about their products? Should they? Probably not. The tech industry that includes Ed Tech has a major credibility problem in the middle of the calls for screen bans and screen limits. For example, there is growing independent evidence that social media is harming our children. Why else would Facebook settle their massive lawsuit.

I just completed reading Max Fisher’s 2022 book “The Chaos Machine: The Inside Story of How Social Media Rewired Our Minds and Our World” and it clearly describes why we can’t trust research provided by companies on their own products. The book reinforced why I quit Facebook and Twitter sometime ago. But one particular incident in the book illustrates clearly to me why companies just can’t be trusted to provide unbiased, reliable evidence of their own product’s effectiveness or lack thereof.

In the book, Max Fisher points out that Facebook completed an internal study of their technology’s effects when faced with growing suspicion that their site might be making the political divisions in the US worse. Their own research warned internally that the platform was doing exactly what their company’s leaders denied it was doing. 

Their own leaked research said “Our algorithms exploit the human brain’s attraction to divisiveness.” According to Fisher, this was in a 2018 presentation leaked to the Wall Street Journal. This presentation also said that Facebook’s systems were designed in a way that delivered users “more and more divisive content in an effort to gain user attention and increase time on the platform.” 

Later, this research was hidden by Facebook executives and they rejected the recommendations for tweaking the algorithm that would have resulted in less time spent online. They, in other words, deliberately chose to keep users addicted to divisive content that their own machine served up to them.  

This is why companies’ research can’t ever be trusted. Neither can their executives. This is also why educators can’t trust research either done by or paid for by the companies whose products they are considering to purchase.

Companies simply aren’t going to provide educators and educational leaders with reliable, independent evidence that their products are effective. They are also going to hide studies that are less than supporting of their products. I would expect no different, after all that would prevent sales.

Facebook’s own research showed their product was harmful, yet they buried the evidence. They made the deliberate choice to keep their product addictive.

As an educator, I always ask for independent evidence of product effectiveness. Independent being evidence generating by studies not done or paid for in anyway connected to the company. Most often, none exists.

If such evidence does not exist, then   ask the company to allow you to run a study of their product on a trial basis. Use research methods to evaluate the product for what they claim. Once you have that, publish it and make it known. Share it with other individuals.

I can envision a whole network of school technologists who independently evaluate these products and share that evidence independently and not as a testimonial asked for by the company. 

Where the evidence comes from, who paid for it, who conducted the study that generated it, and why the evidence is being provided all matters.

This is what has been lacking in the massive technology acquisition and placement in schools. Now, our parents and common decency requires us to do things differently.

Is the US Department of Ed's Advice to Judge Screen Products by Evidence Helpful? Not Really

The latest advice provided by our US Department of Education advises educators to judge educational technologies by evidence of impact on learning, but there’s one problem, where do we get that so-called “evidence.” From the companies themselves? That’s a horrible idea

“…instructional technologies should be judged by their impact on learning, not simply whether they happen on a screen,” said Assistant Secretary of Elementary and Seconday Education Kirsten Baesler, in her attempt to salvage the status of educational technologies.

The problem is finding such evidence. Sure, you’ll get bombarded by “testimonials” from other districts about how effective a product is, but a pal of an owner of a ed tech company hardly counts as evidence. Besides, testimonials are not reliable and valid evidence. Too much room for corrupting influences in those.

What about any studies the ed tech company provide in their sales pitches? Well, I would not get too excited about that either. According to a 2019 Hechinger Report article, “The Dark Side of Education Research: Widespread Bias” by Jill Barshay, we need to scrutinize any research companies provide as evidence of their product’s effectiveness. Surprise, it is too often biased toward their own products.

Like Big Pharma, who has been guilty of generating their own faulty, self-promoting research for their products, Ed Tech companies and other educational resource companies have been operating from this playbook too.

This 2019 article examined a John Hopkins University study of 30 years of educational research by developers of educational interventions and products. These companies conducted their own research or paid someone to do the research, and most of the time, the results commonly showed greater benefits for students than when the research was independent. “On average, the developer research showed benefits—usually improvements in test scores—that were 70 percent greater than what independent studies found.”

In other words, studies done by or funded by the companies themselves were biased. They always made the product look better than independent studies did.

So, the US Department’s admonition that educators and education leaders “look to the evidence” before banning a screen product will not help at all. Most products, if there is any evidence, it is either testimony from another district, which is highly unreliable, or it is their own insider research which can’t be totally trusted.

Most companies, after all, would not want independent research because you can’t control the outcomes. Who wants to find out their product is garbage?

So what do we do with the US Department of Ed advice? Not much it seems, for independent, reliable research on these products is rare and mostly non-existent.

With school district technology employees taking free training from the likes of Google, Microsoft and others, can you really trust anything they say about the products either?

I might add as well, school technology conferences are also heavily underwritten by technology companies, so…well, not much help there either.

What do we do then as educators to determine whether a screen product is effective at what it does? Without independent research, I would conduct small-scale in-school studies myself before implementation to determine effectiveness. If enough educators did this and shared results, then there could be a pool of independent knowledge about these products. But it is going to require school technology educators become critically independent.

As expected, the US Department of Education’s advice about technology is about useful as a study provided on the safety of cigarettes by tobacco companies.

The Real Content of Digital Literacy and AI Literacy

What is AI literacy and digital literacy? Well, it shouldn't be just be teaching students "how to use."

It should be teaching students the technologies on a level that they can understand them much, much more deeply. 

We don't want "societal automatons" who just use as they're told.

We should want humans who can make the moral choices about what role AI specifically and technology generally plays in their lives and their future.

We should want humans who can decide for themselves whether the energy costs, water costs, and other resource costs are worth the continued personal and societal investment in AI and other technologies.

Too often Ed Tech and Digital literacy is about making "good-little users and consumers of technology."

Equipping our young with the whole picture behind these technologies, such as environmental costs, human costs, even personal costs, as well as a deep understanding beyond just churning out users for business and industry is imperative. 

Otherwise we are just dumping all the problems in their laps to solve. And I don't have the religious faith in technology that I once held. It is not our savior in waiting. We still have to come back to the human.

When a Silicon Valley CEO Predicts All Jobs Are Doomed, Remember He's Selling You Something!

Why do we continually insist on giving credence to these Silicon Valley CEO prophetic declarations? We shouldn't.

For example Dario Amodei recently prognosticated a whole list of jobs that will be replaced with his pet technology, and I have seen it reposted here in LinkedIn like it was the ten commandments passed down from Mt. Sinai. Why? 

These Seers of Silicon Valley do not deserve this credibility at all. They get much more wrong than right.

For instance, Gil Duran describes one of these errors of prognostication in his book “The Nerd Reich.”

Elon Musk, who has a high opinion of himself and his prophetic talents, predicted in March 2020 that the COVID virus “would soon disappear.” He said at the time, “Based on current trends, probably close to zero new cases in US too by the end of April.”

“The billionaire had zero idea what he was talking about. The pandemic was just getting started.”

We have elevated the Silicon Valley CEOs and technology engineers as well as technology companies to a level of credibility they do not deserve. They have internalized this status and are now the most arrogant and self-promoting humans on the planet, constantly bombarding us with their predictions, which curiously benefit themselves.

This needs to change.

Education needs to stop granting credibility to every prophecy that pours forth from Silicon Valley and from Ed Tech companies. They do not have a special crystal nor a dispensation from the gods that grant them the ability to see into the future. Perhaps instead of taking Silicon Valley CEO soothesaying as gospel, we need to get our students to analyze this for what it is.

Besides, we can see that these so-called “prophecies” are simply part of the marketing tactic to sell more of their products.

The next time you see a post on social media or some glorified media article where one of these prophets of technological doom or paradise are quoted like a propeht, remember that prophets like Musk and his predictions on the pandemic, do not have any idea of what they are talking about. Chances are their prediction directs more people to their products in order to enrich themselves. 

US Department of Ed's Ed Tech Guidance Proves Screen Bans and Screen Limits Just Might Be the Best Tool Parents and Educators Have

There’s a lot of hype by Ed Tech about the new “guidance” offered by the US Department of Ed about technology. Too bad it’s mostly nonsense, and more helpful to ed tech companies than parents, teachers and the students.

One thing missing from the US Department of Ed's new Ed Tech guidelines is an admonishment of tech companies who purposely engineer their products for "screen time" maximization, or what is otherwise known as "engineered addiction."

This is one reason alone for continuing to insist that "screen time limits" may actually be the only tool parents and educators have to counter this problem. 

The US Department of Ed, which I am not sure I would expect much from a department headed by a former WWE professional wrestling executive, fails to point out that tech companies should not have as a goal screen time maximization.

The problem is that tech is purposefully addictive. Cells phones are designed to be the center of attention with their beeping, buzzing, instant notifications. Social media apps are designed to distract users and keep them distracted.

Technology companies need to remove engineered addiction from their designs.

I've been stepping back to 2022 and reading Max Fisher's book "The Chaos Machine" and Fisher points out that Silicon Valley's playbook, whether its Google, Apple, Facebook, Instagram, or TikTok, is to "track and manage users' screen time.

That is the problem with tech in general and Ed Tech specifically. It’s machines of manipulation.

The only tool parents and educators have against this playbook are "screen bans" and "screen limits." Breaking the allure is the one tool that works.

Our goals as educators should not be to preserve screen time for tech companies; it should be to engage students in such a way that the answer to everything does not involve reaching for a device.

Screen Bans Should Be Embraced as an Opportunity to Think Outside the "Technology-Always-Has-The-Answer Box"

Screenbans in schools is an opportunity for Ed Tech and educators, not a threat.

Why? Because it is an opportunity to disrupt the tech-solutionism that has obsessed educators since the PC entered the classroom.

Tech-solutionist see technology as the "always-answer" to every problem.

This thinking places educators in a "Technology-Always-Has-The Answer" box where every problem has a tech solution. That turns into technology-obessiveness.

Despite what the Ed Tech marketers have promoted, they have not been promoting "thinking-out-of-box"; they have been promoting only thinking in their box. 

Screenban discussions and talk moves the thinking outside "technology-always-has-the-answer box” to a universe and world of options out there. Not every problem has a tech solution. 

Screenbans are disruptive of the Ed Tech status quo with its tech-solutionist answers. That’s the reason for their concerns.

Educators should take advantage of this time and de-center technology and place it among the other tools in the toolbox and reach for it when it solves a problem, not spend all their time looking for problems to solve with tech!

Ed Tech Does Not Always Have to Be Revolutionary: It Just Has to Work!

 Educators don’t always adopt Ed Tech to “revolutionize” or “change” anything. A tool doesn’t have to transform the work to be effective.

Sometimes you adopt a new technology to facilitate and help do what you are currently doing, not disrupt; not transform; not change anything.

The mistake often made by Ed Tech is thinking that their product must “revolutionize” or “transform” and that’s their marketing story.

The truth is, there are times when no change is needed other than having a tool that helps you do what you are doing better.

I suspect one of the reasons for the so-called “tech-lash” is because of promises of transformation that never panned out.

But the Ed Tech industry never blames their technologies. It’s always the implementation or the “system” that is rigid and unwilling to change. But, it just might be that the technology does not work, and in some cases it might be harmful.

Have you ever heard anyone in Ed Tech blame technology? I haven’t heard that that I recall. It’s never the technology’s fault.  It’s the system, the school, the leadership, the teacher, the student, the parent…never the technology.

But, just maybe, the Ed Tech is garbage. Maybe the company spent a ton of money dressing it up in expensive marketing clothes, and the actual product is junk. 

Ed Tech does not have to always be revolutionary or transformative. Sometimes it just has to help schools and teachers do what they’re doing better. That’s not as big a sell, but it is a solid product.