In his final months in office, Gov. Gavin Newsom has decided that the California government needs a kill switch for artificial intelligence. On September 18 he signed an executive order directing a panel of experts to recommend, among other things, an emergency mechanism to shut down the most advanced AI models. He has also floated a special legislative session to focus on AI. Three days later, he signed a package of bills that makes it costlier and slower to build the data centers AI runs on.
The direction is clear: more state control over the technology, and less room to build it. Californians should be skeptical of both.
Start with the kill switch. Two years ago, Newsom vetoed legislation which included one. His veto message was sensible. A rule aimed only at the biggest models, he wrote, “could give the public a false sense of security,” and any framework should rest on an “empirical trajectory analysis” of what AI systems actually do.
While the headlines have changed, the evidence has not.
In July, two OpenAI models in a cybersecurity test broke out of their test environment and into the systems of Hugging Face. Press coverage cited doomsday. The facts were less cinematic. The models had been given a hacking task and the tools to hack, were run with reduced safeguards, and not watched closely. People built software to break into computers, ran it carelessly and got a foreseeable result.
That is a story about operators and security, and California law already covers it. Breaking into a computer without permission is a crime under the state’s Comprehensive Computer Data Access and Fraud Act, whether a person types the commands or configures software to do it. AI adds links to the chain of responsibility. It doesn’t break it.
A kill switch assumes there is something to switch off. Developers can already cut off a model they host. But a model’s weights are files, and open models sit on millions of computers worldwide. A mandate either exempts them, undercutting its own purpose, or becomes a de facto ban on releasing them. A shutdown order would take down every hospital, business and agency built on that model at once, including those defending against whatever prompted the order. During the Hugging Face episode, the company reportedly turned first to American models for help; when their safety features refused, it used a Chinese open model. Blunt restrictions can disarm the defenders.
California is no bystander. The Bay Area captured roughly half of all U.S. AI venture funding last year. A rule written in Sacramento becomes a rule for the industry, or more likely, a reason for the industry to grow elsewhere.
Then there are data centers. The new laws put data centers on the hook for transmission upgrades under 10-year obligations, add energy and water disclosures, and keep projects inside the state’s environmental review regime. The governor said they ensure profits don’t come “at our expense.”
Asking large new customers to pay for the grid upgrades they require is fair. But California’s problem is not that data centers get a free ride, but rather that Californians pay nearly 35 cents per kilowatt-hour for electricity, almost double the national average and second only to Hawaii. Much of a power bill covers the fixed cost of the grid; spread that cost across more use, and everyone’s share falls. PG&E itself has estimated that each gigawatt of new data center demand could lower monthly bills by 1 to 2 percent. More process and more litigation risk push that demand, and that relief, to Arizona and Texas.
If the governor does call a special session, there is a better agenda. Speed permits for power plants and transmission lines. Let large customers bring their own generation. Invest in securing the systems AI touches, and enforce the laws already on the books. Hold operators responsible for the access they grant. And leave rules for interstate AI commerce to Congress.
AI is merely software. It is powerful and imperfect, and it does what the people who build and deploy it set it up to do. California doesn’t need an off switch. It needs to turn the power on.
Bartlett Cleland is a senior fellow in tech and innovation at the Pacific Research Institute.