What types of local news stories should be automated? The Toronto Star is figuring it out

Automated news stories often get dunked on when a publication first starts testing them out. Auto-generated crime stories, then, are likely to invite even more criticism.

“Let’s be honest: Crime coverage is terrible,” Tauhid Chappell and Mike Rispoli wrote in their Nieman Lab prediction for 2021. “It’s racist, classist, fear-based clickbait masking as journalism.”

In April, the Toronto Star began using data from the Toronto Police Service to publish stories about break-and-enters across the city, using data from the Toronto Police Service. Since then, it’s published around six stories a week that round up the break-and-enter reports from each of Toronto’s six districts.

The stories all follow pretty much the same format. They don’t include personal information, like names or specific addresses. The headlines report the number of break-and-enters in a specific district from the last week. The first few paragraphs of the total number of break-and-enters in the entire city for that week, followed by the total number since the start of the year. It then notes whether break-and-enters have increased or decreased and by what percentage in the same period from the prior year. It then lists each break-and-enter report, categorized by district neighborhood in alphabetical order.

The final paragraph of each story notes that it was was automatically generated from open data that’s collected and maintained by the Toronto Police Service. It notes that “recent crime data is preliminary and subject to change upon further police investigation.”

The practice of automatically generating some news coverage isn’t new. The Los Angeles Times’ Quakebot, launched in 2014, generates stories about earthquakes. Reuters experimented with a tool that helps reporters find interesting anomalies in data. The Toronto Star itself has published other automated series on local highway closures, election results, home prices, and restaurant inspections.

But its automated reporting on break-and-enters attracted some extra attention. Over the summer, Jordan Heath-Rawlings, the host of the Big Story Podcast in Canada, posed questions about the Star’s automated reports in a Twitter thread. “I’m not saying THIS particular series of articles (which seems to be writing a piece based on reported break-ins for each Toronto neighbourhood) is bad,” he wrote. “But I think streamlining the process that goes from police through the media even further is … a choice.”

Cody Gault, the product manager for content at the Toronto Star, told me that strategically using automation on some stories has helped free up reporters to cover deeper stories.

“The best example of automated content out-and-out ‘replacing’ newsroom-produced content is our automated DineSafe series, where we report the results of health inspections for local restaurants, bars, cafes, bakeries and grocery stores,” Gault said. “Reporters were producing stories like this by hand because readers really want this information. But I don’t think any reporter or editor misses writing or editing this series before we automated it — and the readership has only increased since we did.”

Another example: The Star had already been covering the rise in auto thefts in the city, so having automated weekly reports of the number of auto thefts complemented those investigations. Break-and-enter stories, however, weren’t previously being covered manually by Star reporters.

Jean-Hugues Roy, a journalism professor at the University of Quebec in Montreal, likened The Star’s practice to when newspapers used to send a reporter to a police precinct to report for the police blotter. Back then, he said, you had to trust that the reporter and the police chief didn’t leave any information out. Now, you have to trust that all the data is there.

“I am wondering what needs it answers,” Roy said. “The local newspaper’s job is to talk about relevant things. Whereas here, everything’s published. So what’s relevant? We are missing the human to make sense of this deluge of data and articles.”

In the case of the break-and-enter stories, “everybody recognized that a poor execution of the idea would be a problem,” Gault told me in an email. The series was reviewed and approved by the paper’s editor-in-chief, managing editor of news, managing editor of digital, and public editor, as well as by editorial leaders in other parts of the company, before publishing began, he told me.

Within the paper, initial concerns fell into a few different buckets: What was the source of the data and how reliable was it? Would the Star be violating the privacy of victims or suspects? And would the series stigmatize some communities?

The Toronto Police Service is subject to Ontario’s Freedom of Information and Protection of Privacy Act, which requires public institutions to make public information available and also establishes standards for safeguarding personal information. The data comes from Toronto Police Service’s open data portal, which is the same place Star reporters get other crime stats.

The third concern required more discussion. “Some editors wondered if the series would reveal that break-and-centers occur more frequently in neighborhoods that are less wealthy and less white,” Gault told me. They “were concerned that exhaustively reporting those incidents without additional context could contribute to the stigmatization of these communities.”

As it turned out, that wasn’t what the data showed. “The impression the series gives is that there tend to be somewhere between 35 and 55 break-and-enters reported in Toronto each week [and that] it’s relatively uncommon for there to be more than two break-and-enters reported in any particular neighborhood in any given week,” Gault said. They tend to be reported more frequently in neighborhoods with large populations. And they’re down in 2022, compared to the previous year.

“If we had instead discovered that a glaring disparity existed, it would have been up to the newsroom to decide whether or not to run the series — as it always is,” Gault said. “But I hope we would have found a responsible way to report it, because I share the Toronto Star’s commitment to social reform, and because I don’t know how we can hope to address inequality in our city except by reporting it when we find it.”