A California lawmaker has introduced a first-in-the-nation bill meant to ban companies from embedding AI chatbot technology into toys designed for children. Announced on Friday, the measure comes amid growing concerns about the impact of artificial intelligence on child welfare, as well as a number of local and federal proposals to limit kids’ access to large language model (LLM) chatbots.
This particular legislation would target toys that simulate friendship and companionship through LLM technology. For toy manufacturers, LLMs can provide an easy, albeit risky, way of creating a personality for a particular doll or character. AI models aren’t pre-scripted the way most talking toys are—which means toys integrated with the tech can end up sharing all sorts of inappropriate content with a young child. Case in point: Sales for an AI-enabled teddy bear were suspended in November after the toy, when investigated, started talking about sexual role-play and igniting matches.
The California bill, authored by state Sen. Steve Padilla, is designed as a moratorium and would ban, until at least January 1, 2031, the sale of toys that include a “companion chatbot” meant for children 12 and younger.
The proposal defines a companion chatbot as “an artificial intelligence system with a natural language interface that provides adaptive, human-like responses to user inputs and is capable of meeting a user’s social needs.” The bill also targets chatbot technology designed to support an ongoing bond with a potential child, “including by exhibiting anthropomorphic features and being able to sustain a relationship across multiple interactions.”
“Chatbots and other AI tools may become integral parts of our lives in the future, but the dangers they pose now require us to take bold action to protect our children,” Padilla said in a statement. “Our safety regulations around this kind of technology are in their infancy and will need to grow as exponentially as the capabilities of this technology does. … Our children cannot be used as lab rats for Big Tech to experiment on.”
It’s not immediately clear whether the bill has legs. But the impact of artificial intelligence on kids has outraged local and federal lawmakers on both sides of the aisle. One bipartisan federal proposal from Sen. Josh Hawley (R-MO) and Sen. Richard Blumenthal (D-CT), for instance, would ban the use of AI companions for minors and punish AI companies that make AI for minors that solicits or produces sexual content.
Notably, the White House has pushed back on state lawmakers hoping to regulate AI technology. Late last year, President Donald Trump signed an executive order that would target states that write their own laws with litigation, and take a series of steps to establish one national approach to AI regulation on topics like child safety and alleged censorship. For now, though, it’s not clear how serious the Trump administration’s efforts actually are.
Representatives for several organizations focused on digital welfare for kids did not respond to a request for comment on the legislation, which was released Friday, January 2.