Among all of his mistakes, don’t forget Elon Musk is singlehandedly crushing a big chunk of Internet research for no good reason
How would you create a taxonomy for all of Elon Musk’s screwups running Twitter? Are there enough phyla, kingdoms, or classes to contain such a rapidly growing population?
Maybe you could categorize them all by their financial impact — from low (asking all Twitter engineers to print out their code for his personal inspection) to medium (launching a “verification” system that immediately exploded into market-moving chaos) to high (unbanning enough Nazis to drive away most of your biggest advertisers).
Maybe you’d group them by their degree of self-humiliation, from high (“Based on current trends, probably close to zero new cases in US too by end of April”) to extreme (yelling “I’m rich, bitch!” while being booed off a San Francisco stage) to otherworldly (accusing an employee with muscular dystrophy of faking his disability).
Or maybe you’d order them by the extent of disrespect they show Twitter’s users. Changing Twitter’s logo to the doge Shiba Inu isn’t offensive per se, but it is a persistent reminder that Musk considers the site his personal playground, not anything larger or more important to the world.
The question awaits its own Linnaeus. But before any more ecosystems of shame are discovered, let’s remember one in particular that hasn’t gotten enough attention. That would be Musk’s tossed-off decision to kill off a huge chunk of ongoing research into the internet and how we all interact on it. That includes work by everyone from academics to hobbyists, businesses to journalists. Elon Musk decided to kill it, and soon it will all be dead.
It was September 2006 — only six months into its existence — when Twitter launched the first version of its API. It was free to use and turned Twitter from a website you visited into a data source for an entire ecosystem of apps, tools, art projects, and experiments. Over time, the API’s capabilities expanded and a few paid products were added those with specific needs. But for the vast majority of cases, the Twitter API was a free window into what people were talking about in the internet’s public square.
That made it catnip for researchers. While Facebook was a much larger and more consequential social network, it was purposefully opaque, limiting outsiders’ view into its inner workings. Twitter’s generous API, meanwhile, made it the most popular choice for academics researching social media and anyone else wanting to understand online discourse. There’ve been a few hiccups along the way, of course, but overall it worked well for a decade and a half. “Twitter is the most (over-)studied social media platform precisely because it offers relatively open data access,” George Washington University’s Rebekah Tromble wrote two years ago.
So of course Elon decided to kill it. In February, desperate for anything that smells like revenue, Twitter announced that it would be eliminating free API access to tweets.1 Instead, it would offer an API level much worse than what was free before at the price of $100/month. (The old free plan for academics allowed access to 100 million tweets per month. The new $100 plan allows access to just 10,000.) To do any meaningful level of research, you’d need to move up to one of their enterprise API plans, which start at $42,000/month. (That’s right, they start there — and go up to $210,000/month.) Been working on a longitudinal research project for five years? Sorry, your data’s about to be shut off.
Because this is Elon’s Twitter, the switch to this new regime has been delayed repeatedly.2 But final pricing details were released last week, and they’re as bad as feared. On Monday, a group of researchers called the Coalition for Independent Technology Research released an open letter decrying the change, saying they “will devastate public interest research” and break more than 300 ongoing research projects that they know of. “Twitter must be held accountable to the public it impacts,” the coalition writes. “And if Twitter is to be held accountable, independent research must continue.” The new rules are set to take effect by the end of the month.
I’m sure there are a few corporations who’ll be willing to pay these outrageous sums, but a Ph.D. student isn’t going to find a spare $42,000 in their couch cushions. The research just won’t get done, and we’ll all be a little bit dumber.
This is the part of the piece where I’d like to list an idea or two for how you could try to help fix this situation. (“Write your member of Congress!”) But the reality here is that the only path to change runs through one man. A man who has spent the past year setting his reputation on fire, reply-guying his way into a laughingstock. One of the richest human beings in the history of the species is unilaterally killing off the work of thousands of researchers, all to chase a buck he won’t end up getting. We may need a new category in the taxonomy.
Here’s the Coalition for Independent Technology Research’s letter.
April 3, 2023On March 29, after weeks of delays and uncertainty, Twitter’s Developer team announced new pricing tiers for access to the Twitter API that will devastate public interest research.
Over the past decade, researchers across the world have relied on Twitter’s API to study the impact of social media on democracy, the role of social media in strengthening public health, how social media has been used to amplify marginalized voices, and much more. With free API access, researchers could systematically and reliably collect public tweets posted by public figures, gather information about network dynamics, investigate bots and other inauthentic activity, or analyze conversations around specific topics. The knowledge from this research has been shared with journalists, policymakers, and the public, enhancing understanding of issues vital to society.
Free API access also allowed researchers to build public tools like Botometer and Hoaxy that detect social bots and visualize the spread of misinformation. Thousands of users, journalists, and public servants have used these tools in their daily lives and work.
Twitter’s new system to monetize and dramatically restrict access to its API will render this research and development impossible. Unless they can pay, researchers will not be able to collect any tweets at all. The Basic tier costs $100 per month but allows researchers to collect only 10,000 tweets per month — a mere 0.3% of what could previously be collected for free in one day. The Enterprise tier, which ranges from $42,000 to $210,000 per month, is unaffordable for researchers.
Yet even these outrageously expensive Enterprise tiers provide inadequate access for systematic, large-scale research into the impact of Twitter on society. Previously, Twitter provided researchers with low-cost access to its Decahose, a real-time sample of 10% of all tweets. As of March 2023, that equated to roughly a billion tweets per month. The most expensive Enterprise tier would cut that by 80% at about 400 times the price.
In response to a questionnaire fielded by the Coalition for Independent Technology Research, public interest researchers listed over 250 projects that would be jeopardized by ending free and low-cost API access, including research into the spread of harmful content, (dis)information flows, crisis informatics, news consumption, public health, elections, and political behavior. Under the new pricing plans, studying the communications and interactions of even a small population — such as the 535 Members of the U.S. Congress or the 705 Members of the European Parliament — will be unfeasible. The new pricing plans will also end at least 76 long-term efforts, including dashboards, tools, or code packages that support other researchers, journalists, first-responders, educators, and Twitter users.
Though Twitter’s Developer team claimed that they “are looking at new ways to continue serving” academia, they provided no specifics—merely stating that “in the meantime” academic researchers could make use of the pricing plans described above.
What precisely is Twitter “looking into” for academia? How long might academics have to wait for these new options to appear? And what are Twitter’s plans for non-academic public interest researchers, including civil society organizations serving communities around the globe?
Twitter has not answered any of these questions. Indeed, to date, the company has failed to engage with the research community in any meaningful way. Twitter has had mechanisms for dialogue with our community readily available — including via the European Digital Media Observatory, shared working groups tied to the European Union’s Code of Practice on Disinformation, and the company’s own Academic Research Advisory Board. Twitter has not used any of these channels, rolling out these changes without substantive input from public interest researchers whose work will be shut down.
The Coalition for Independent Technology Research will continue to support the research community in the face of these challenges. To date, we have provided mutual aid to nearly 50 projects, focusing in particular on assisting under-resourced and junior researchers. We have supported the National German Library’s German-language Twitter archive efforts, and two groups associated with the Coalition have offered data storage support for researchers.
Our mutual aid efforts are ongoing and will persist as long as possible. Mutual aid is available to all researchers across academia, journalism, and civil society — members and non-members alike. (To request mutual aid or contribute to these efforts, please complete the mutual aid section of this form.)
Going forward, the Coalition will also help organize researchers who wish to explore alternative data-collection and data-sharing mechanisms. And we will continue our discussions with policymakers and regulators around the world.
Twitter must be held accountable to the public it impacts. And if Twitter is to be held accountable, independent research must continue.
That is why — no matter what barriers technology companies erect to public understanding of their services — the Coalition for Independent Technology Research will advance, defend, and work to sustain the right to ethically study the impacts of technology on society.
The Coalition for Independent Technology Research
Executive Board:
Rebekah Tromble
Alex Abdo
Susan Benesch
Brandi Geurkink
Dave Karpf
David Lazer
Nathalie Maréchal
Nathan Matias
James MickensMutual Aid Committee:
Megan Brown
Josephine Lukito
Kai-Cheng Yang
- After criticism, Twitter announced there would be a free API product, but it’s write-only — meaning it can post tweets but not read existing ones. That makes it useless for research purposes.
- It was supposed to take effect one week after the announcement. We’re now looking at about three months. Thanks for being unable to meet deadlines, Elon, I guess?