Newsom Orders AI Oversight, Advances AI “Kill Switch”

SACRAMENTO – Gov. Gavin Newsom Sept. 18 issued an executive order to dramatically accelerate the implementation of new third-party oversight of safety and security risks in AI systems, as well as independent audits. 

The executive order convenes a group of experts to provide, within two months, a guide for California to reinforce and strengthen its AI safety and security laws. Proposals under consideration include requiring independent third parties to write safety plans for frontier AI companies, as well as requiring companies to develop an emergency shutoff, or “kill switch,” for frontier models.

California leads the nation in oversight of emerging frontier technologies, with laws addressing nearly every major area of AI policy: frontier-model safety, independent oversight, children and companion chatbots, privacy, deepfakes, fraud, cybersecurity, workers and government deployment. 

The executive order comes after recent alarming incidents, including the Hugging Face attack, prompting many Americans, including leaders in the AI industry, to call on the industry to pace the development of AI systems and models and push for more stringent government regulations to address security and safety risks. In response, the federal government has done nothing. No federal law requires AI companies to report dangerous incidents when they happen, and President Trump has rejected calls for new regulations that would increase security and protect the health, safety, and well-being of the American people. 

Through the order, the governor is directing the Government Operations Agency: to accelerate the implementation timelines for SB 813 and AB 1405, and, in consultation with the Governor’s office of emergency services, to convene national experts to develop recommendations for potential changes to further strengthen state law to:

• Require frontier AI companies to embed a designated independent verification organization onsite in their labs to conduct regular audits and evaluations.
• Require that the AI safety frameworks, transparency reports, and risk assessments that frontier AI companies are required to file under state law are verified pursuant to standards deemed adequate by an independent verification organization.
• Advance the creation of a “kill switch” for frontier models, with the efficacy of the switch verified on an ongoing basis by an independent verification organization.
• Update definitions of critical safety incidents to include loss-of-control incidents such as the Hugging Face attack.

Details: https://tinyurl.com/Newsom-executive-order