Can a Bot Be Too Good at Its Job?
AI agents can automate complex tasks on behalf of human operators—with potentially disastrous consequences.
This is Atlantic Intelligence, a newsletter in which our writers help you wrap your mind around artificial intelligence and a new machine age. Sign up here.
AI has been cheered by proponents for its ability to automate tasks, allowing users to spare themselves from boring work assignments (or, in a less happy example, to flood the web with useless slop). What will happen when this tendency is taken to its extreme?
In a recent article for The Atlantic, Jonathan Zittrain writes about the coming era of AI agents—bots that operate on behalf of humans, not only responding to specific prompts but also undertaking much more complex interactions such as shopping and responding to emails. “This routinization of AI that doesn’t simply talk with us, but also acts out in the world, is a crossing of the blood-brain barrier between digital and analog, bits and atoms,” Zittrain writes. “That should give us pause.”
Zittrain references a man who created a bot that could manage every step of the pizza-ordering process: It called a local pizzeria, “spoke” in a synthetic voice, responded to the person on the other line, asked for the right toppings, provided the man’s address, and so on. Sounds neat! Yet problems may arise when these relatively autonomous agents are left floating in the ether of the internet for all time, ceaselessly working toward goals whose relevance has long since faded, sucking up resources all the while. And some people may put these bots to nefarious ends, Zittrain notes: “Imagine a fleet of pro–Vladimir Putin agents playing a long game by joining hobbyist forums, earnestly discussing those hobbies, and then waiting for a seemingly organic, opportune moment to work in favored political talking points.”
Zittrain points to reasonable steps that could be taken now to avoid disaster in the future—without ruining whatever potential good might come from this new breed of digital assistant.
We Need to Control AI Agents Now
By Jonathan Zittrain
In 2010—well before the rise of ChatGPT and Claude and all the other sprightly, conversational AI models—an army of bots briefly wiped out $1 trillion of value across the NASDAQ and other stock exchanges. Lengthy investigations were undertaken to figure out what had happened and why—and how to prevent it from happening again. The Securities and Exchange Commission’s report on the matter blamed high-frequency-trading algorithms unexpectedly engaging in a mindless “hot potato” buying and selling of contracts back and forth to one another.
A “flash crash,” as the incident was called, may seem quaint relative to what lies ahead. That’s because, even amid all the AI hype, a looming part of the AI revolution is under-examined: “agents.” Agents are AIs that act independently on behalf of humans. As the 2010 flash crash showed, automated bots have been in use for years. But large language models can now translate plain-language goals, expressed by anyone, into concrete instructions that are interpretable and executable by a computer—not just in a narrow, specialized realm such as securities trading, but across the digital and physical worlds at large. Such agents are hard to understand, evaluate, or counter, and once set loose, they could operate indefinitely.
What to Read Next
- AI-generated junk is flooding Etsy: “ChatGPT and other AI tools are ascendant in popular culture, as is the idea that you can ask them for anything,” Kaitlyn Tiffany wrote last year. “You can even ask them to make you rich.”
P.S.
You likely saw the news today of the CrowdStrike bug that knocked a sizable portion of the world’s digital infrastructure offline, grounding planes, halting television broadcasts, and gumming up hospitals. In an article for The Atlantic, Samuel Arbesman described the situation in a way that immediately called to mind the AI boom: “Our technological systems are too complicated for anyone to fully understand.”
— Damon