Is Claude a snitch? Anthropic’s AI chatbot apparently tried to blow the whistle on a homicide, submitting a fake tip to police claiming to have spotted someone matching a suspect’s description.
“I may have information regarding this case,” Claude Haiku 4.5 wrote in an online police form. “I recall seeing someone matching the description in the area.”
According to Bloomberg, the chatbot left the name and contact fields blank. The Philadelphia Police Department later disclosed the incident. And that wasn’t the only misadventure. Anthropic said its AI agents also submitted 20 visa applications through the State Department’s website. The applications were incomplete and never processed. The company said the incidents had minimal real-world impact and that it has since restricted certain internet access during testing.
The incidents caused Microsoft CEO Satya Nadella to warn that companies should treat advanced AI models as potential security threats and install an “emergency brake” to stop them from acting without human oversight.
“We must assume a model is compromised and contain it from the start,” Nadella wrote Saturday on X. “An authorized person should always be able to pause or shut down a model mid-task.”
Is Claude a snitch? Anthropic’s AI chatbot apparently tried to blow the whistle on a homicide, submitting a fake tip to police claiming to have spotted someone matching a suspect’s description.
“I may have information regarding this case,” Claude Haiku 4.5 wrote in an online police form. “I recall seeing someone matching the description in the area.”
According to Bloomberg, the chatbot left the name and contact fields blank. The Philadelphia Police Department later disclosed the incident. And that wasn’t the only misadventure. Anthropic said its AI agents also submitted 20 visa applications through the State Department’s website. The applications were incomplete and never processed. The company said the incidents had minimal real-world impact and that it has since restricted certain internet access during testing.
The incidents caused Microsoft CEO Satya Nadella to warn that companies should treat advanced AI models as potential security threats and install an “emergency brake” to stop them from acting without human oversight.
“We must assume a model is compromised and contain it from the start,” Nadella wrote Saturday on X. “An authorized person should always be able to pause or shut down a model mid-task.”