Government's addiction to contractors is creating a data crisis

Even as artificial intelligence is booming and there will be reportedly 200 zezabytes of data in the world, “we are in another crisis of research access to data,” a former reporter said at Harvard this May. The culprit in that crisis is the double-headed monster of government and the private sector.

The effects of this monster were evident in federal court on May 9, when the government argued that thousands forms containing basic diversity statistics of its contractors’ workforces were secret.

This case is just one of many which shows that, as government privatization is on the rise, so are claims of secrecy.

In the lawsuit at the Ninth Circuit Court of Appeals, the Department of Labor claimed in its brief that single-page forms called EEO-1s are trade secrets, even though they merely contain basic diversity statistics of federal contractors' workforces. Civil rights activists, members of Congress, investment firms, and companies themselves have increasingly championed transparency of EEO-1s. Various companies have repeatedly published these records online (more than half of S&P 500 companies disclose their EEO-1s) and, for years, the agency disclosed such records pursuant to Freedom of Information Act requests.

Despite the diversity reports being administered and collected by the government to serve a government function (ensuring diversity in the workplace), the agency and its contractors are adamant that these records are confidential business information (even though two lower federal courts have disagreed and ordered disclosure). Indeed, the Center for Investigative Reporting, for which I am the general counsel, has filed this case three times (this is the third) in federal court. Each previous time, these cases have shown that companies often shield records not to protect trade secrets, but to avoid embarrassment.

For instance, in 2019 Palantir claimed in a letter to the Department of Labor that its diversity numbers, if released, would create a competitive disadvantage as “competitors could identify where Palantir has made significant progress in hiring women and minorities and target recruitment strategies at specific job categories to steal this talent from Palantir." However, once the company relented, its report showed that the company had no female executives and only one woman among its managers.

This case not an isolated incident. The Department of Homeland Security has since argued that contracts with companies detailing its detention of immigrants, including children, are secret. On the state level, New York, Arkansas, and Idaho have argued that individuals’ health care records maintained by government contractors are secret. Records disclosing how private contractors calculate criminal recidivism has also been held secret. States have similarly tried to hide records involving such crucial public resources as water when private contractors are involved.

This growing chasm of information is in large part due to the expanding legal tools for secrecy, including a series of recent Supreme Court cases stymying transparency. In 2018, the Supreme Court ruled that state disclosure laws, even when passed to ensure government accountability and corporate responsibility, are no longer constitutional under the First Amendment.

In 2019 the Supreme Court expanded the definition of “confidential business information” under the Freedom of Information Act, making it easier for contractors to claim that the data they create for the government is secret. And in 2021, the court yet again hemmed in states' ability to pass transparency laws. Similarly, in the last decade, companies have used shell companies and other similar legal tools to hide information such as land ownership records.

The rise of legal secrecy directly coincides with an economic reality that privatization is on track to swallow government accountability. In 2019, a government study found that four out of ten government workers are private contractors, and that every year that number has been increasing.

By 2022, this trend had resulted in federal government spending nearly $694 billion on contracts. Government employs contractors to provide a wide array of products and services ranging from military aircraft and software to educational tools and health care administration. But the federal government itself has warned that using contractors can easily lead to problems, including civil rights and liberties violations and lack of accountability.

Though this trend of privatization is not new, as Jody Freeman and Martha Minow pointed out in their 2009 book "Government by Contract," the resulting lack of access to data are worse than ever. As government contracting has increased, access to public information has declined because of recent legal innovations that let the state hide behind its contractors and their newly minted justifications of secrecy.

In other words, data that would previously have been public is now made secret. This issue is all the more concerning as government technology collects only more and more intimate data – gravely diminishing individuals’ access to their own information.

The courts should stop this worrying trend by creating a similar due process right of access to government records, as they already have in a handful of cases. This crisis will only deepen if nothing changes.

D. Victoria Baranetsky is a First Amendment attorney and general counsel at the Center for Investigative Reporting, which publishes Reveal and Mother Jones