Autonomous vehicles can be tricked into dangerous driving behavior

When a driverless car is in motion, one faulty decision by its collision-avoidance system can lead to disaster, but researchers at the University of California, Irvine have identified another possible risk: Autonomous vehicles can be tricked into an abrupt halt or other undesired driving behavior by the placement of an ordinary object on the side of the road.

“A box, bicycle or traffic cone may be all that is necessary to scare a driverless vehicle into coming to a dangerous stop in the middle of the street or on a freeway off-ramp, creating a hazard for other motorists and pedestrians,” said Qi Alfred Chen, UCI professor of computer science and co-author of a paper on the subject presented recently at the Network and Distributed System Security Symposium in San Diego.

Chen added that vehicles can’t distinguish between objects present on the road by pure accident or those left intentionally as part of a physical denial-of-service attack. “Both can cause erratic driving behavior,” said Chen.

Chen and his team focused their investigation on security vulnerabilities specific to the planning module, a part of the software code that controls autonomous driving systems. This component oversees the vehicle’s decision-making processes governing when to cruise, change lanes or slow down and stop, among other functions.

“The vehicle’s planning module is designed with an abundance of caution, logically, because you don’t want driverless vehicles rolling around, out of control,” said lead author Ziwen Wan, UCI Ph.D. student in computer science. “But our testing has found that the software can err on the side of being overly conservative, and this can lead to a car becoming a traffic obstruction, or worse.”

For this project, the researchers at UCI’s Donald Bren School of Information and Computer Sciences designed a testing tool, dubbed PlanFuzz, which can automatically detect vulnerabilities in widely used automated driving systems. As shown in video demonstrations, the team used PlanFuzz to evaluate three different behavioral planning implementations of the open-source, industry-grade autonomous driving systems Apollo and Autoware.

The researchers found that cardboard boxes and bicycles placed on the side of the road caused vehicles to permanently stop on empty thoroughfares and intersections. In another test, autonomously driven cars, perceiving a nonexistent threat, neglected to change lanes as planned.

“Autonomous vehicles have been involved in fatal collisions, causing great financial and reputation damage for companies such as Uber and Tesla, so we can understand why manufacturers and service providers want to lean toward caution,” said Chen. “But the overly conservative behaviors exhibited in many autonomous driving systems stand to impact the smooth flow of traffic and the movement of passengers and goods, which can also have a negative impact on businesses and road safety.”

Joining Chen and Wan on this project were Junjie Shen, UCI Ph.D. student in computer science; Jalen Chuang, UCI undergraduate student in computer science; Xin Xia, UCLA postdoctoral scholar in civil and environmental engineering; Joshua Garcia, UCI assistant professor of informatics; and Jiaqi Ma, UCLA associate professor of civil and environmental engineering.