Your smart devices are listening to you, explained
This story is part of a group of stories called
Uncovering and explaining how our digital world is changing — and changing us.
Facebook was listening to you. Then it wasn’t. Now it’s listening again.
Google, Apple, and Amazon are listening too, but it’s not all as pernicious as it sounds.
Facebook told Bloomberg this week that it collects and stores audio from its smart speaker, Portal, when users awaken the device by saying, “Hey Portal.” Contractors may later transcribe a small portion of the ensuing dialogue to help train Portal’s algorithms so it can come up with better responses in the future. (Facebook now says it will give users the option to turn off human audio transcription.)
Still, people are concerned and confused about the possibility that their devices are spying on them and that human workers could potentially be listening to their conversations.
Why companies listen
There are valid reasons why tech companies listen to your dialogue with their smart devices. These companies review a small sample — less than 1 percent for Google and Apple — of user conversations. They say they do this because the recordings help make their products better. The workers who listen to these recordings take note of common mistakes — say an Amazon Echo hearing the word “elections” as “Alexa” — with the intent of improving the software.
The samples tech companies review are anonymized, so they don’t include personally identifiable information. But that’s not entirely reassuring because if you really want to figure out who someone is, in some cases you can just listen to what they’re saying.
“Anyone who’s dealt with voice systems will have worked in call centers, and all of them use these practices: Humans looking at errors, assessing them, and feeding corrections back into the system for machine learning to improve performance,” Bret Kinsella, founder of Voicebot.ai, a publication that covers voice-activated technology like smart speakers, told Recode. “You can’t run the system without recording, and it’s difficult to improve the system without human reviewers annotating errors.”
Really, smart assistants, which are the brains behind voice technology on smart speakers and smartphones like Google’s Assistant, Apple’s Siri, Amazon’s Alexa, and Facebook’s Portal, are always listening for their wake word, the phrase that tells them to start recording and to transmit that information to its servers so that it can figure out how best to respond. These companies then vet the wake word that devices picked up a second time, to see if you really meant to say “Hey Google.”
And as anyone with a smart devices knows, they can be set off unintentionally — when they think they hear a wake word, but really it’s just you saying something similar, so dialogue that smart assistants unintentionally pick up can be reviewed as well.
One way to avoid other people listening would be to have smart assistants answer queries on the device as opposed to transmitting the audio to the cloud, Todd Mozer, CEO of Sensory, a company that works to do just that, told Recode.
“A lot of companies will reduce privacy to keep costs down and keep the device smaller,” Mozer said. Google and Amazon have begun releasing on-device versions of software, a move that’s both faster and more secure.
The fact that tech companies often employ contractors to review this audio also exacerbates people’s concerns with having other people review snippets of the dialogue they say in their homes. These workers may not have as much incentive as a full-time employee to keep the information they hear private.
Kinsella says the best way to mitigate concerns would be to hire reviewers in-house rather than rely on contract workers.
Siri/Portal/Alexa/Assistant, are you listening to me?
There’s also the persistent fear that our phones are listening to us even when we’re not trying to talk to them. Some 43 percent of American smartphone owners think their phone is recording them without their permission. Part of the reason for the belief that our devices are listening stems from advertisers’ eerie ability to show people ads for things they were just talking about.
But there doesn’t seem to be any proof our phones and other devices are listening without our permission.
That’s not to say it’s impossible — but it’s unnecessary.
As many have pointed out, tech companies and third-party data companies already have loads of information about you. Phones and apps already collect and transmit your location, your browser and purchase history, and your contacts, among other info. A new Northeastern University and Imperial College London study found that some smart TVs send data to companies like Netflix, Facebook, and Google, even when the devices are not in use.
Through this trough of information advertisers can get a pretty good idea of what you want and when.
Are they spying? Probably not. Do they already know plenty about you? Yes.
What’s perhaps more perturbing for people is that these companies haven’t been transparent about saying what they’re doing and why.
Are they spying? Probably not. Do they already know plenty about you? Yes.
Amazon waited to disclose in plain English that Alexa relies on thousands of human beings to listen to users’ conversations until after Bloomberg reported the news. It now says Alexa training “relies in part on supervised machine learning, an industry-standard practice where humans review an extremely small sample of requests to help Alexa understand the correct interpretation of a request and provide the appropriate response in the future.”
Users of all the major assistants also now have the option to opt out of audio reviews. Most people don’t change factory settings, so having to opt out is largely meaningless. (Siri and Assistant are now opt-in services).
Does privacy even matter to people?
The jury is out on whether or not people care enough about privacy to change their tech use or demand changes from tech companies.
As OneZero writer Will Oremus demonstrated, it’s difficult to know how much privacy is worth to Americans. Our behavior is no indication. Despite news reports and a lot of hubbub on social media, companies like Facebook are still pushing their smart devices and people are still buying them.
Still, knowing that a total stranger could overhear snippets of what you say within the privacy of your own home is creepy.
Andrew Bosworth, Facebook’s head of hardware, told Bloomberg this week about Portal, “The consumer reaction the last several months to these practices, not just at Facebook but other companies, gave us insight into the fact that this was something people weren’t entirely comfortable with or weren’t sure about.”
It was an understatement.
Americans say they trust tech companies less than ever, and continual breaches of trust — like not telling users explicitly that their conversations could be reviewed by humans — are degrading this trust even further. This could have been a mundane announcement rather than something buried and obscured in terms of service
Sourse: vox.com