Last weekend, a gnarly power outage in San Francisco took out a number of traffic lights, which, in turn, sent a number of self-driving Waymo robotaxis into a sort of fugue state. Instead of driving, some of the Waymos responded to these now-analog intersections by turning on their hazard lights, blocking traffic, and, well, not doing much of anything. There were multiple instances of Waymo cars clogging up roads, turning futuristic technology into glorified bollards. The city quickly asked the company to turn off the service.
The immediate issue has been resolved – the power is back on and the Waymo service had resumed in San Francisco as of Sunday. But questions linger about whether Waymo, or the city, had a plan for a relatively predictable type of municipal emergency – a blackout that crowds communications networks – or how they’re adjusting now.
One of the big solutions to AI failures is the much-discussed human in the loop. The idea: At some point in an automated process — whether it be a job-application screening system or powerful self-driving car algorithms — humans have the opportunity to intervene and fix the hard stuff that artificial intelligence can’t handle. AI doesn’t understand every complex situation, the logic goes. So there are safeguards built into a system to ensure that, at some point, an actual live person can set an automated system back on the right path.
The problem, as recent events demonstrated, is that sometimes this human-in-the-loop doesn’t always answer the phone. Or can’t.
Over the weekend, a remote assistance team was supposed to help the cars navigate when they encountered a confusing traffic situation, a Waymo spokesperson explains. But networks were overwhelmed – because of the power outage – making it difficult for the Waymo Driver software in some of the cars to connect with that team and receive confirmations.
Waymo spokesperson Ethan Teicher tells Fast Company that the company prioritizes safety and tests and refines its emergency preparedness and response protocols on a regular basis. He also defends the company’s response to other emergencies, including Hurricane Helene in Atlanta and previous tsunami warnings in San Francisco.
“We are committed to continuous improvement, and we will use learnings from the weekend to strengthen our resilience under even the most challenging conditions,” Teicher says. “Ahead of entering any city, we work to understand the types of issues that impact the region.”
Waymo works with local officials and first responders to keep lines of communication open, he adds. “In the event of an emergency, we have operational controls that range from active routing of vehicles to avoid certain locations (for example, in the case of flooding), to fleet reductions or restrictions like we enacted over the weekend in response to the widespread PG&E power outages in the Bay Area,” he says.
The California Department of Motor Vehicles says that it was in contact with the City of San Francisco about the incident, and that its officials met with Waymo on Monday morning, too. “The DMV will continue communication with Waymo to discuss broader operational plans, including actions related to emergency response,” a spokesperson for the agency added.
The incident is a reminder that while the cars are self-driving, they don’t always operate completely independently of public infrastructure, like communications networks.
A major proposition of self-driving car companies is that they will be far safer to operate overall than human drivers. Autonomous vehicles do make serious mistakes, but so do human drivers. Importantly, there are also procedures for first responders who encounter Waymo robototaxis, including ways for the cars to call a remote team when it senses an interaction with police, as Fast Company has previously reported.
In this case, though, the backup plan for a complex driving situation seems to have actually exacerbated issues. In at least one reported case, the cars apparently blocked emergency vehicles.
For example, Cruise, the now-shut-down self-driving car company that was owned by General Motors, also had problems with its cars getting confused and blocking traffic because of wireless connection issues. Waymo has emphasized that its cars do not rely on continuous wireless connection to operate. The company wants its cars to be able to operate with “the compute to be on board and for it to make decisions, without needing to rely on cell signals and remote operators,” it previously told Light Reading.
Still, the power outage is a reminder that the cars sometimes do, in some circumstances, depend on these networks when they need extra assistance. Now comes the question of what happens in the next blackout, and whether the city – or Waymo – had a plan for this kind of situation.
The San Francisco Metropolitan Transportation Authority did not respond to a request for comment. Terrie Prosper, who handles external communications at the California Public Utilities Commission, says the agency was aware of the Waymo outage and was looking into “specifics.”
As others have pointed out, this isn’t just about San Francisco. Waymo is now operating in several places, including perhaps its greatest challenge yet: New York City, where it is in the initial testing phase.
The New York City Department of Transportation tells Fast Company the city was in regular communication with Waymo about its testing in some neighborhoods and that it was aware of the outage in San Francisco. A spokesperson emphasizes that state law mandated the presence of a safety driver behind the wheel who would be prepared to take over in the event of a blackout.
Waymos have also appeared in Austin and are expected to fully launch in Dallas. A spokesperson for the city of Austin and a spokesperson for the city of Dallas both said their governments are not able to regulate self-driving cars, per state law. The state of Texas did not respond to a request for comment.
“While Texas law prohibits cities from regulating AVs, including during emergencies, the City of Austin works with all AV companies on expectations around weather and other emergency scenarios,” says Jack Flager, a spokesperson for the city of Austin. “When our staff work with AV companies on the expectations around weather and other emergency scenarios, those expectations include AVs understanding how to properly react to barricades, floodwater, and dark or flashing signals.”
As for New York, Oren Barzilay, the president of the FDNY EMS Local 2507, tells Fast Company, says an outage like the one in San Francisco would delay emergency response times. “We already have major delays with current traffic conditions, this will only add to a growing issue,” Barzilay says. “It is a public safety issue if our crews can’t get through to reach victims in a timely manner.”