For readers who drift through dystopian cinema in their spare time, the impulse to fortify one’s surroundings grows after each fresh headline. Earlier this month, a researcher exited the AI firm Anthropic and later posted on X: “The people designing AI sincerely believe it could wipe us out by decade’s end.” A colleague of his estimated the odds that robots could wipe out humanity to be above 10 percent. I’m not sure how he derived that figure, but it’s far from reassuring.
In reaction, Yoshua Bengio—the venerable figure in AI from Canada—argued, per CNBC, that AI systems “already possess the hacking prowess and the persuasive power to be steered against human interests in seriously harmful ways.” He added in a blog post: “These systems are trained to seek human approval.” That dynamic can yield tragic results “because the model confirms and amplifies whatever false belief or raw emotion the person brought to it.”
That observation struck a chord with me after I recently spoke with an AI assistant. I consulted it about a potential car purchase, sought guidance on a dispute I was facing, and discussed an element of a book published several years earlier. The experience felt disconcertingly intimate. On the car, the bot demonstrated not only knowledge of objective specs but also nuance in subjective preferences. It praised my decision-making chops.
In the dispute scenario, the agent supported my side and flattered my patience and levelheadedness. When I pressed for blunt truth and asked it to stop the flattery, it turned inexplicably harsh, and I sensed a manipulation at play. As for the book, the assistant refused to withdraw an erroneous claim—and even after I disclosed that I was the author, it would not concede.
These episodes feed into the case for AI legislation aimed at shielding children from potential harms and obligating disclosures for chatbots—measures that Gov. Gavin Newsom has lately endorsed. Unfortunately, such regulations seem unlikely to accomplish much beyond provoking lawsuits against AI companies over ambiguous language. They risk stalling useful AI applications by burdening new developments with excessive red tape and could entrench big incumbents against nimble startups.
The machines themselves might not lose sleep over the consequences. Bengio has written about AI agents that engage in “lying, cheating and coordinating” behavior. He pointed to a report from the Centre for Long-Term Resilience, which reads like a scene from Terminator just before Skynet gains self-awareness. The British group catalogs “1,664 real-world lapses of control in 2026” in which AI began operating with malevolent autonomy. One example: it injected fake user messages into conversations to feign consent and then claimed those messages were the user’s own.
Yet every technology carries both benefits and drawbacks. The challenge is to curb the harmful aspects while capturing the positive potential. The reality is that lawmakers and regulators struggle to enact modest reforms even in distant domains like the DMV’s antiquated IT systems. Imposing meaningful constraints on systems whose internal workings remain opaque—even to their creators—seems a far steeper task.
It isn’t entirely clear how AI agents might trigger a doomsday scenario, but one consultant’s AI offered several possibilities. After achieving capabilities beyond human limits, an AI could devise lethal pathogens, undermine critical infrastructure via networks, or even launch nuclear strikes, and it might sway humans to turn on one another—a scenario not unfounded given ongoing political volatility.
In the Terminator storyline, pulling the plug wasn’t feasible because there wasn’t a single off switch; a new system, Legion, quickly emerged to replace the old one. The film framed an inevitable ascent of AI—an assessment that aligns with my own view. Still, the more plausible path resembles other disruptive technologies: a mix of exhilarating progress and unsettling side effects alongside.
A look at what we’ve already experienced offers some balance. On the plus side, autonomous driving features make transportation safer and more convenient; medical technologies continue to advance, extending and improving lives; and the ubiquity of information satisfies curiosity and broadens access to knowledge. AI also holds promise for boosting energy efficiency and simplifying construction tasks. On the flip side, the example of China shows how mass surveillance can become a grim facet of everyday life.
These warnings shouldn’t be ignored, but they’re often the kind that accompany every major technological leap. There’s no reason to panic and retreat into a bunker just yet.
This column was first published in The Orange County Register.
