How spyware turned this Kansas high school into a ‘red zone’ of dystopian surveillance

I hated high school.

So I ditched as much class as I could and spent my time racing muscle cars on Route 66 outside of my hometown of Baxter Springs and pursuing other misadventures. I was always reading, though, and in between repairing blown head gaskets and thrown timing chains I had my nose in books, trash and treasure alike, from “The Monkey Wrench Gang” to Hemingway and Harper Lee. My high school guidance counselor told me I should give up my dreams of being a writer and join the Navy instead. I managed to graduate from high school by the intervention of a school superintendent who reckoned I was smarter than I looked and allowed me to test out of some required classes. I still have my graduation photo around someplace, me at 17 in a cap and gown and leaning against the hood of an old GTO.

Later I went to college and washed out after a semester or two and then gave it another try after a couple of years and did better. I eventually graduated from a four-year public university in Kansas and then got an advanced degree and spent some years as an investigative reporter at daily newspapers and published a couple of dozen books with New York houses. I sometimes thought about calling my old high school guidance counselor to gauge her reaction but always thought better of it because, after all, it wasn’t bad advice, considering.

But high school today? It makes my blood run cold.

I wouldn’t last 10 minutes.

Sartre was wrong. Hell used to be other people. Now it’s high school.

I’m convinced of this because I’ve been following the news coverage of Lawrence High School. Just imagine you’re a student at Lawrence High (go Chesty the Lion!) and every homework assignment, email, photo, and chat on your school-supplied device is being monitored by artificial intelligence for indicators of drug and alcohol use, anti-social behavior, and suicidal inclinations.

That’s been the reality since last November, when the district began a $162,000 contract with Gaggle, a Dallas-based student safety technology company to provide around-the-clock surveillance. If a word or an image triggers an alert in the AI software, the result could range from the student being sent to an administrator to being referred to online counseling to getting a visit from local police.

The district says Gaggle is a tool to increase the safety and welfare of its students and staff. That’s an admirable goal, because suicide is the second leading cause of death for youths 15-19, according to the National Institute for Mental Health. In Kansas, the suicide rate among young people has outpaced the national average, according to the Kansas Health Institute.

“With Gaggle, our district is better equipped to proactively identify students who are at risk for potential unsafe behaviors, provide support where needed, and foster a safer school environment,” according to the USD 497 website.

Gaggle claims that it has saved an “estimated” 5,790 student lives between 2018 and 2023. It did this, according to its website, by analyzing 28 billion student items and flagging 162 million of those for review.

AI surveillance flags “concerning content” on school-issued devices and software accounts for review and blocks potentially harmful content, according to its website. Expert human review, it says, helps district officials to take action before students harm themselves or others, and in severe situations it alerts “district-appointed” contacts, even after hours or on weekends. If no district representative is available, the police might be summoned.

What Gaggle is selling is an antidote for fear — for administrators, for parents, for students — in exchange for civil liberties. It’s difficult to argue with 5,790 lives saved, if you take it at face value, but I have my doubts about that number.

At what point is the safety you think you’re buying for students actually doing harm in unintended ways? Won’t teachers avoid assignments that challenge students to consider real-world problems like violence, depression and suicide? Won’t students learn just to keep their emotions to themselves, instead of confiding in a teacher or another trusted adult? What about the chilling effect on student creativity and expression?

Gaggle is the thought police for K-12 campuses.

Lawrence school board president Kelly Jones confers with principal Quentin Rials during an April 19, 2024, meeting with student journalists about their concerns with the district’s use of spyware. (Sherman Smith/Kansas Reflector)

It would be easy to describe what’s happening to students at Lawrence High School as Orwellian, but that would be an easy and not exactly fitting metaphor. It does resemble in general the dystopian novels we used to be assigned to read in high school — “1984” and “Brave New World” — but a more accurate comparison is to a science fiction novella you may have never heard of.

In Philip K. Dick’s “The Minority Report,” published in 1956 and made into a movie by Steven Spielberg in 2002, a predictive policing system is used to arrest people before they have the chance to commit the crime they are expected to. Dick’s story — like the use of Gaggle — pits authoritarianism and conformity against creativity and individual liberty.

Just consider what happened to photography students at Lawrence High School shortly after Gaggle was introduced. Nearly an entire class, reported the student newspaper, was called in to explain to administrators the contents of their art portfolios.

“To have administrators reach out to a student,” editor-in-chief Maya Smith wrote in February, “a file in their Google account must be in what Gaggle calls ‘red zone,’ whether it be a photo, document or video. For photography students, photos for various projects were flagged for what was deemed ‘nudity.’ ”

But, Smith reported, none of the students said there was nudity in their work. It was difficult to discuss the images with administrators because the files had been removed from the student accounts, so even the creators couldn’t see them.

Much of what we know about Gaggle at Lawrence High has come from the student newspaper, the Budget. Its enterprising staff of student journalists questioned whether the district’s use of Gaggle was proper under the First Amendment, the Kansas Shield Law and the Kansas Student Publications Act. Newsgathering is a constitutionally protected activity and those in authority shouldn’t have access to a journalist’s notes, photos and other unpublished work.

“People in authority can violate your rights while believing they are protecting you,” the Budget staff wrote in explaining their coverage. “It’s up to you to protect your work process and product. Adults didn’t tell us to fight the good fight. We did it ourselves.”

The staff also reported that when Gaggle flags an email, it stops delivery, which could delay students seeking help. This concern was echoed by their journalism adviser, Barbara Tholen, when earlier this month the district voted to renew Gaggle’s contract, at $53,000. Tholen said a student had shared her thoughts in an email to a trusted teacher, only to have the email rerouted to an administrator.

“Imagine the person that student trusted most with news of their struggles never knowing to reach back out to them with words of comfort,” Tholen told the Lawrence school board, as reported by the Lawrence Times. “We need students to share concerns openly with us — that saves lives.”

The district reached an agreement in April with Budget staff to remove Gaggle from the devices of student journalists. But with the renewal of the contract, the rest of the student body has no such reprieve. A school board member made it clear that students and staff had no expectation of privacy when using district-issued devices. As Reflector intern Grace Hills reported, “Gaggled” has become a new verb in the Lawrence school district.

The question of student surveillance is made more difficult by a lack of clear data on whether it works and if so, whether the collateral damage to privacy is justified. School officials across the country defend the use of such surveillance by arguing that if it saves just one life, it’s worth it. But is it worth it if it turns schools into virtual prisons?

“Through a careful review of the existing evidence, and through interviews with dozens of school staff, parents and others,” wrote a group of Rand researchers in February, “we found that AI based monitoring, far from being a solution to the persistent and growing problem of youth suicide, might well give rise to more problems than it seeks to solve.”

Surveillance software, which became prevalent during the pandemic, may disproportionately target minority students, according to a 2021 piece in the Conversation. According to reported Fast Company magazine, Gaggle previously flagged the words “gay” and “lesbian” in assignments and chat messages.

And about those estimated 5,790 student lives that Gaggle claims to have saved? I asked the company to share its methodology with me. It seemed a suspiciously exact number for an estimate. It also seemed out of proportion, considering there are fewer than 10,000 suicide deaths of those ages 10-24 each year, as of 2018, according to statistics from the Centers for Disease Control and Prevention.

“Each of our Possible Student Situations (PSSs), which are the highest priority alerts our safety team receives, are reviewed by the head of our Safety Team against a number of different criteria that include nature of the alert, context around the alert, and if the alert resulting [sic] in school or district leaders providing intervention or support that prevented a student death,” Shelby McIntosh Goldman, Gaggle’s vice president of marketing and research, told me. “If the incident meets all criteria, it is counted as a life saved (and yes, counts are exact and tracked annually.) The classification also goes through a secondary audit to create as much inter-rater reliability as possible.”

That’s a lot of “Minority Report” verbiage that says it’s a guess.

And what about Gaggle’s failures? Does it count lives lost?

“By ‘live lost’ I’m assuming you’re asking about situations where the Gaggle Safety Team received an alert but intervention or support did not prevent a student death,” McIntosh Goldman said. “We haven’t received any feedback indicating that occurred in the five plus years that we’ve been tracking lives saved, but due to student privacy issues there is no way for us to confirm that.”

That last sentence represents a positively Orwellian decline in the use of the English language. Tortured prose aside, wouldn’t keeping track of tragedies on their watch be just as important to Gaggle, and more accurate, than guessing at number of lives saved? But then, I’m not trying to sell districts on around-the-clock student surveillance.

The erosion of student privacy by Gaggle and other educational spyware firms, such as Bark and Gnosis IQ, should be of concern to anybody who takes their Bill of Rights seriously. Spyware represents a grave threat to the student press and free speech on campus. It also conditions students to expect government surveillance and may create an assumption that nothing a student does at school — or at home — should be beyond the authority of administrators.

When I was in high school, challenging authority meant cutting class and doing burnouts in front of school. About the only surveillance I was subject to was the speed gun of the local cops.

There were a few teachers I knew I could trust, and that was enough.

But who do you trust in a red zone?

Max McCoy is an award-winning author and journalist. Through its opinion section, the Kansas Reflector works to amplify the voices of people who are affected by public policies or excluded from public debate. Find information, including how to submit your own commentary, here.

Kansas Reflector is part of States Newsroom, a nonprofit news network supported by grants and a coalition of donors as a 501c(3) public charity. Kansas Reflector maintains editorial independence. Contact Editor Sherman Smith for questions: info@kansasreflector.com. Follow Kansas Reflector on Facebook and X.