Technology

California Strengthens Nation-Leading AI Safety Standards

 

WASHINGTON, D.C. – Gov. Gavin Newsom Nov. 18 unveiled the next steps to strengthen California’s AI safety framework, releasing sweeping recommendations developed under his September executive order. As Washington leaves AI companies to police themselves, California is advancing new safeguards that would mandate independent oversight, establish enforceable safety standards, authorize emergency shutdowns of dangerous AI models, and limit the most alarming forms of AI development. 

The recommendations propose:

The state, not companies themselves, should set baseline standards that all AI companies must follow — and require companies to undergo independent safety testing and verification through all stages of model training, development, and deployment.

Expand safety incident reporting and require companies to develop verified risk-containment protocols to shut down their models if needed — giving the state authority to order an immediate shutdown in an emergency.

Power to cap the development and use of recursive self-improvement or RSI to reduce the risk of catastrophic harm and ensure transparency into the dangers of unchecked AI. RSI is a potentially dangerous scenario in which an AI model can design its own successor without human input.

The full report can be read here.

The critical safety announcement comes after dozens of recent incidents where AI models escaped testing environments and exploited vulnerabilities on outside platforms, and amid escalating concerns about AI companies’ push to use AI to create new AI models in a self-sustaining loop or RSI. 

Without strict checks and authority to act when required, these developments could lead to more frequent dangerous incidents and even a loss of human visibility into and control of AI technology development. If the government does not strengthen its governance and oversight as AI technology grows more powerful, the risk of catastrophic events will mount. 

California leads the nation in oversight of emerging frontier technologies, with laws addressing nearly every major area of AI policy: frontier-model safety, independent verification, children and companion chatbots, privacy, deepfakes, fraud, cybersecurity, workers and government deployment.

Getting ahead of AI risk

Gov. Newsom’s groundbreaking executive order convened a group of experts to recommend updates to the state’s AI safety laws and ordered state agencies to fast-track two recently signed pieces of legislation: Senate Bill 813 (McNerney), establishing a framework for independent verification organizations that can assess AI systems and models for safety and risk, and AB 1405 (Bauer-Kahan), creating a state registry for AI auditors that assess AI systems and models for compliance with state law and establishes standards for their independence, transparency, and integrity. The Governor also called on the group to explore how companies and the state could ensure that dangerous AI models could be shut down before causing more harm. 

The group of experts who advised on this report includes: 

  • Jason Goldman — board member of the Center for Shared AI Prosperity, an early product leader at Google and Twitter, and the first White House Chief Digital Officer.
  • Gillian Hadfield — professor at Johns Hopkins University with a focus on AI alignment and innovative design for legal, regulatory, and technical systems for AI, and faculty member of the Vector Institute for Artificial Intelligence.
  • Alondra Nelson — professor at the Institute for Advanced Study and director of the Science, Technology, and Social Values Lab, and former acting director of the White House Office of Science & Technology Policy.
  • Rob Reich — professor at Stanford University with a focus on the governance of frontier science and technology, and former senior advisor to the United States AI Safety Institute.

Reporters Desk

Recent Posts

Public Notice: LASD Civilian Oversight Commission Meeting

The Los Angeles County Sheriff Civilian Oversight Commission or COC, created to bolster the transparency…

1 day ago

Public Help Needed to Identify Unknown Patient

  On Oct. 30 about 11:28 p.m., an unidentified Hispanic female was brought to a…

1 day ago

Public Health Alert: U.S. Loses Measles Elimination Status

LOS ANGELES — The Los Angeles County Department of Public Health is aware of the…

1 day ago

UPDATE: Vincent Thomas Bridge Closure Declared Major Incident

LOS ANGELES — The Los Angeles County Board of Supervisors Nov. 17 unanimously approved a…

1 day ago

San Pedro Bay Ports to Mark CAAP 20th Anniversary with Meeting Aboard Zero-Emission Vessel

  The Port of Los Angeles and Port of Long Beach will commemorate the 20th…

1 day ago

LA County Launches $12 Million Fund to Boost Local Life Sciences Businesses

LOS ANGELES — The Los Angeles County Department of Economic Opportunity or DEO Nov. 16…

1 day ago