Former OpenAI employees William Saunders and Daniel Kokotajlo have penned a letter to California Governor Gavin Newsom, arguing that the company’s stance against a state bill that would impose stringent safety guidelines and protocols on future AI development is both disappointing yet not entirely unexpected.

“We joined OpenAI with the intention of ensuring the safety of the incredibly powerful AI systems it was developing,” Saunders and Kokotajlo wrote. “However, we resigned from OpenAI due to the loss of our trust that it would safely, honestly, and responsibly develop its AI systems,” they continued. Recommended Videos.

The duo asserted that continued development without proper safeguards “presents foreseeable risks of catastrophic harm to the public,” whether it be in the form of unprecedented cyberattacks or aiding in the creation of biological weapons.

They also promptly highlighted OpenAI CEO Sam Altman’s hypocrisy regarding regulation. They pointed to his recent congressional testimony advocating for regulation of the AI industry but noted that “when actual regulations are on the table, he opposes them.”

According to a 2023 survey by the MITRE corporation and the Harris Poll, only 39% of respondents believed that today’s AI technology is “safe and secure.”

The bill in question, SB-1047, the Safe and Secure Innovation for Frontier Artificial Models Act, would, among other things, require that a developer, before commencing the initial training of a covered model, comply with various requirements, including the ability to promptly implement a full shutdown and establish a written and separate safety and security protocol. OpenAI has experienced multiple data leaks and system intrusions in recent years.

OpenAI reportedly vehemently disagrees with the researchers’ “misrepresentation of our position on SB 1047,” as a spokesperson told Business Insider. Instead, the company contends that “a federally-driven set of AI policies, rather than a patchwork of state laws, will foster innovation and position the US to lead the development of global standards,” OpenAI’s Chief Strategy Officer Jason Kwon said in a letter to California State Senator Scott Wiener in February.

Saunders and Kokotajlo countered that OpenAI’s push for federal regulations is not in good faith. “We cannot wait for Congress to act – they have explicitly stated that they are not willing to pass meaningful AI regulation,” the pair wrote. “If they ever do, it could preempt CA legislation,” they added.

The bill has received support from an unexpected source as well: xAI CEO Elon Musk. “This is a difficult decision that will upset some people, but, all things considered, I believe California should probably pass the SB 1047 AI safety bill,” he wrote on X on Monday. “For over 20 years, I have been an advocate for AI regulation, just as we regulate any product/technology that poses a potential risk,” Musk stated. Musk, who recently announced the construction of “the most powerful AI training cluster in the world” in Memphis, Tennessee, had previously threatened to move the headquarters of his X (formerly Twitter) and SpaceX companies to Texas to avoid industry regulation in California.

Update: This post has been updated to include the comments from Elon Musk.