How SB 53 in California Could Curb Big AI

California’s SB 53 targets the biggest AI developers, with demands for safety disclosures, an incident registry, and protected whistleblower conduits. In a marketplace dominated by a few companies with the most powerful systems, the bill’s laser-like approach could be among the few effective restraints on Big AI without drowning startups in compliance.

What SB 53 Requires From Large AI Developers

The bill zeroes in on AI companies making at least $500 million in annual revenue, requiring them to release model safety reports, tell state authorities about major incidents, and establish safe channels for employees to raise concerns — even if they’ve signed NDAs. The purpose is simple: raise enough hell to identify hazards earlier and make accountability routine, not optional.

Though the precise reporting templates remain to be hashed out in rulemaking, anticipate alignment with existing frameworks you’re already using — namely NIST’s AI Risk Management Framework — and forthcoming guidance from the U.S. and U.K. AI Safety Institutes alike.

If done well, SB 53 would turn voluntary, low-key “model cards” into organized, auditable disclosures.

Why the $500 million company threshold matters

The threshold limits the reach of the law to companies with scale and leverage — think OpenAI, Google DeepMind, Anthropic (in collaboration with others), Microsoft, and Meta — rather than fledgling startups. That responds to the central criticism of last year’s broadened proposal: that it could have had a chilling effect on research and company formation.

Concentration is the point. According to Synergy Research Group, the top three cloud providers represent an overwhelming share of global cloud infrastructure spending, and those are the platforms that frontier-scale training and deployment actually and practicably occur on. But a rule intended for the very largest players penetrates down through the layers of compute and distribution that shape the market.

The California effect on AI governance and oversight

California has a history of exporting policy. The auto emissions standards established in Sacramento were a nudge, a national nudge for manufacturing. The California Consumer Privacy Act changed how companies must treat data across the United States. AI is also clustered in the state: The Stanford AI Index has highlighted California’s disproportionate share of AI investment, talent, and research output.

Given their heavy reliance on California, where most frontier labs — and their suppliers — are based, SB 53 could serve as something of a de facto baseline. Firms not even headquartered there might also rally behind it to simplify compliance across products and partner ecosystems.

Why disclosures and reports of incidents can bite

Transparency creates leverage. Routine safety reports require labs to specify how they’ll do the evaluation, what the known failure modes are, whether red-teaming was done, and any mitigations. That will provide enterprise buyers, regulators, and researchers with a common yardstick — relocating safety from marketing copy to something that can be measured.

Reporting of incidents erects a feedback loop as it is. Recent disasters — image generation producing absurd pictures, toxic jailbreaks, or models confabulating sensitive data — demonstrate how problems can arise post-deployment. Developing a formal duty to report to the government about those cases, coupled with internal postmortems, can speed up improvements and prevent any potential quiet relabeling of defects as “edge cases.” RAND and CSET have long maintained that fast incident learning is essential for frontier-scale risks.

The whistleblower channel could be the most significant piece. Previous standoffs between AI researchers and management are a sign of how NDAs and culture can create a cone of silence around safety concerns. The need to protect employees who do blow the whistle is one reason that “stings” of this sort — when discredit moves from government ranks and methods to the agents themselves — can bring out problems that never make their way into polished reports.

How this is different from last year’s doomed push

SB 53 is more timid than the vetoed SB 1047, sacrificing ambition for enforceability. It also keeps smaller startups mostly out of the heavy responsibilities and doesn’t mandate particular technical controls that might become outdated. By highlighting disclosures, events, and whistleblowers, it drives governance without tyrannizing the science.

In practice, the largest labs would formalize risk registers, beef up internal audits, and have a standardized set of safety documents attached to each major model release. Many already hint at such behaviors — like the Responsible Scaling Policy of Anthropic and industry red-teaming pledges — but SB 53 would transform aspirations into hard requirements.

Preemption risks and expected industry pushback

Anticipate claims that the bill splinters rules or reveals corporate secrets. Some of those critiques can be buried by confidential filings for sensitive details, adoption of standardized starting points for such incidents, and safe-harbor language about timely, good-faith disclosure. The largest swing factor is federal preemption: possible limits on the state AI rules have been floated in proposals in Washington. If those get on the ballot, California could end up negotiating over the edges rather than the essence.

That said, state-first momentum is real. And with the EU AI Act establishing norms for everyone and NIST articulating a risk-based approach for everyone, California’s move would not be so much an outlier as a bridge: a national-level tool that takes international principles and makes them real within the companies most capable of doing systemic harm.

How big labs are affected if SB 53 becomes law

Short term: better model documentation, clearer evaluation policies, disciplined incident response procedures, and formal whistleblower protections. Medium term: procurement teams and insurers will take that information to price in risk, favoring labs that demonstrate actual safety improvements as opposed to glossy claims.

SB 53 isn’t a silver bullet on frontier AI risks. But by focusing on the companies with the most capacity and leverage, it gives us a practical level of accountability — one that markets and researchers can get their heads around. And that’s a useful check that the industry has largely been lacking.